Use these options to configure your replication agent
Essential information
- If you have changed your ADMIN passwords, the
target_auth.setandsource_auth.setfiles will need to be regenerated. - By default, the replicated files will be placed on the target in a directory structure that matches the directories on the source. The redirect option can be used if the path on the target machine does not match the source.
- The
unique_idkeyword is required in the multi-server environments. It is a good practice to set this keyword in any environment to set the stage for more advanced topologies. - Be sure that
log_change_detailsis commented out. This key word is only used for diagnostic testing because it can increase the size of the log and degrade performance. - To see if replication is working, use the
repadmutility with thegetstatscommand-line option.
Example configuration file
The replication agent operations are directed by a single text-based configuration file ctreplagent.cfg. An example is shown below:
; FairCom c-treeACE Server Replication Agent Configuration File - v12.0.0.89 Build(201020)
;The unique ID of the Replication Agent for multi server replications
unique_id REPLAGENT1
; Target server connection info
target_authfile ../config/replication/target_auth.set
target_server FAIRCOMS@localhost
; Source server connection info
source_authfile ../config/replication/source_auth.set
source_server FAIRCOMS@SourceIP
;syncagent yes
; Read 8 KB batches from source server
batch_size 8192
; Use a 5-second timeout when reading from source server
read_timeout_ms 5000
; Log file name
log_file_name ../config/replication/replagent1.log
; Write change details to log file
;log_change_details changes.log
; The exception_mode option determines how the replication agent handles errors
; that occur when applying operations to the target server.
;
; exception_mode transaction
; When an operation fails, abort the transaction. This option preserves
; atomicity of transactions: either all or none of the operations for the
; transaction are applied to the target c-tree Server.
;
; exception_mode operation
; When an operation fails, continue the transaction, skipping only the
; operation that failed. This option does not preserve atomicity of
; transactions: only the operations that the replication agent successfully
; applies to the target c-tree Server are committed.
exception_mode transaction
; The source server will remember the replication agent's log position
; even when the replication agent disconnects.
;remember_log_pos yes
; file filter specified in the external XML rules
file_filter <../config/replication/filter1.xml
; enables replicating create operations and other schema definition operations
replicate_data_definitions yes
; End of File
Diagnostic and monitoring options
-
debug_minlog
Enables the replication agent to log messages to ctreplagent.log when it changes its minimum log number. -
diagnostics
Enables diagnostic logging to the replication agent log file. -
exception_logging
Configures exception logging. -
function_timing
Enables the replication agent to collect function timings. -
log_change_details
Logs the change details to a file. - log_file_name
Defines the filename and location of the replication log. - log_level
- Defines what types of messages the replication agent will log. Supported values:
| Value | Type of messages logged |
| log_level off | none |
| log_level debug | debug, info, warning, and error messages |
| log_level info | info, warning, and error messages |
| log_level warning | warning and error messages |
| log_level error | error messages |
-
notify_events
Triggers the execution of an external program, named replmon, when certain events occur. -
system_event_log
Specifies for the replication agent to write its messages to the Windows event log in addition to the replication agent log file.
Transaction log read options
-
batch_size
Sets the batch size to read from the source server's transaction logs. -
convert_subst
Causes the source server to return file names of replicated files with substituted drives converted to the actual path. -
full_filenames
Determines whether the replication agent will return a full or relative file path. -
read_timeout_ms
Sets the polling timeout value to use when reading from the source server. -
remember_log_pos
Requests the source server to persist required logs in lieu of using the server-side KEEP_LOGS option. -
start_position
Specifies the starting position on the source server in the replication agent configuration file.
Parallel replication options
-
check_dependencies
Checks the dependencies between transactions it reads from the source server. -
file_group_parallel_apply
Specifies whether a file group is replicated in parallel. -
num_analyzer_threads
Specifies the number of analyzer threads to coordinate the internal sequence of operations. -
num_apply_threads
Specifies the number of threads to use for parallel replication streams. -
parallel_apply
Enables parallel replication. -
sync_log_writes
Enables immediate flushing of transaction log writes.
Server connection options
-
socket_timeout
Specifies a timeout option on the replication agent's send and receive calls. -
source_authfile
Sets the source server user connection information. -
source_nodeid
Forces the required node ID for the source server. -
source_server
Sets the source server connection information. -
source_tls_cert_file
Specifies the name of the certificate file to use when connecting to the source server using TLS. -
source_use_tls
Causes the connection to the source server to use TLS. -
target_authfile
Sets the target server user connection information. -
target_nodeid
Forces required node ID for the target server. -
target_server
Sets the target server connection information. -
target_tls_cert_file
Specifies the name of the certificate file to use when connecting to the target server using TLS. -
target_use_tls
Causes the connection to the target server to use TLS.
Replication file set definition options
-
file_filter
Specifies an XML file filter from an external file. -
resync_file_list
Specifies the name of the resync file list that contains the names of all the data files to deploy. -
unique_id
Sets the unique ID of the replication agent for multi-server replication.
Transaction apply options
-
check_update
Checks record updates for adds, deletes, and rewrites. -
delete_to_recycle_bin
Enables the replication agent to use the recycle bin when it deletes a file on the target server. -
exception_mode
Specifies how the replication agent handles errors that occur when applying operations to the target server. -
extension_library
Registers an external library to be used by the replication agent. -
lock_retry_count
Specifies how many times to attempt to lock a record. - lock_retry_sleep
Specify how much time to sleep between record lock attempts. -
redirect
Allows files to be redirected to an alternative location on the target server. -
replicate_data_definitions
Enables the replication of data definition operations. -
use_target_alignment
Specifies for the replication agent to use the alignment setting of the file on the target server.
Replication agent configuration Options
batch_size
Set the batch size to read from the source server's transaction logs. When using serial (not parallel) replication a larger batch size can improve throughput. Default is 8192. Recommended for large numbers of replicated tables 1M (1048576).
batch_size <batchBufferSizeInBytes>
Default: 8192
check_dependencies
Enable transaction dependency checking when using parallel replication with more than one apply thread.
check_dependencies <yes|no>
Default: yes
If the replication agent is configured to use parallel replication and check_dependencies yes is specified, the replication agent checks the dependencies between transactions it reads from the source server when it schedules the transactions to be applied by the replication agent apply threads.
check_update
(V11.5 and later) Configure basic conflict detection for operations that modify data records.
check_update <yes | no | log_only>
Default: yes
When check_update yes is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and fails the operation.
When check_update log_only is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and attempts to apply the operation.
The entry that is logged to the exception log in case of error 1105 includes both the information read from the source server (old and new keys and record images) and the current record image read from the target server.
convert_subst
For Windows systems only: When reading replicated file names from the source server's transaction logs, convert substituted drive letters to the actual path.
convert_subst <yes | no>
Default: no
debug_minlog
Enable diagnostic logging of the replication agent's minimum transaction log requirement.
debug_minlog <yes | no>
Default: no
When debug_minlog yes is specified, the replication agent writes a message to ctreplagent.log each time its minimum required transaction log number changes.
delete_to_recycle_bin
When replication file deletes, make the deletes recoverable.
delete_to_recycle_bin <yes | no>
Default: yes
Enable the Replication Agent to use the recycle bin when it deletes a file on the target server. Otherwise, the file is deleted from disk immediately.
diagnostics
Enables diagnostic logging to the replication agent log file.
diagnostics <diagnosticsOption>
Default: not set
Supported options:
-
diagnostics analyzer
enable logging of dependency analysis -
diagnostics dependency_graph
enable logging of dependency graph -
diagnostics file_open
enable logging of file open/close operations by apply threads -
diagnostics file_open_err
enable logging of file open errors -
diagnostics latency
enable logging of replication latency -
diagnostics logread
enable logging of log read thread -
diagnostics logship
enable logging of log ship thread -
diagnostics tran_state
enable logging of transaction state processing by log read thread
exception_logging
(Supported in V11.5 and later) Configure exception logging.
exception_logging <on | off | errors_only | target_server>
Default: errors_only
exception_logging on enables logging of every operation in a transaction where at least one operation failed.
exception_logging off turns off logging of failed operations.
exception_logging errors_only enables logging of only those operations that failed to be applied.
exception_logging target_server causes the replication agent to write exception log messages to a table on the target server instead of the server in which the replication agent is running.
The options exception_logging on, exception _logging off, and exception_logging errors_only are mutually exclusive. If more than one of these options is specified in the replication agent configuration file, the last option that is specified takes effect.
exception_mode
Configure how the Replication Agent handles errors that occur when applying operations to the target server.
exception_mode <transaction | operation>
Default: transaction
Modes:
-
transactionWhen an operation fails, abort the transaction. This option preserves atomicity of transactions: either all or none of the operations for the transaction are applied to the target FairCom DB Server. -
operationWhen an operation fails, continue the transaction, skipping only the operation that failed. This option does not preserve atomicity of transactions: only the operations that the Replication Agent successfully applies to the target FairCom DB Server are committed.
extension_library
Register an external library to be used by the Replication Agent.
extension_library <libraryName>
Default: not set
See our Replication Extension documentation on this feature.
Example:
extension_library ctrepluser.dll
file_filter
Specify an XML file filter for the Replication Agent to use to determine which files it replicates.
file_filter <xmlFilterFileName>
Default: not set
Two forms of usage are supported.
- The XML file exists on the system where the source server is running.
file_filter filter1.xml - The XML file exists on the system where the replication agent is running.
file_filter <./filter2.xml
See Also
file_group_parallel_apply
Configure the use of parallel replication within a file group.
file_group_parallel_apply <yes | no>
Default: no
When using replication file groups, file_group_parallel_apply yes enables parallel replication of transactions within a file group. Otherwise, replication of transactions within a file group is performed serially.
full_filenames
Convert file names that the replication agent reads from the source server's transaction logs to the full path.
full_filenames <yes | no>
Default: no
When the Replication Agent reads a 'file open' change entry from the source server, the entry contains a file number, and the Replication Agent makes a call to the server to get the file name. The source server returns the filename that was read from the transaction log, which is the filename that was specified by the client that opened the file. This name might specify either a full or relative path.
The configuration option full_filenames yes causes the Replication Agent to return the filename with the full path.
function_timing
(V11.5 and later) Enable the Replication Agent to collect function timings. This can be helpful to analyze replication performance.
function_timing <on | off>
Default: off
lock_retry_count
Specify how many times to attempt to lock a record. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.
lock_retry_count <count>
Default: 0 (meaning no retries)
See Also
lock_retry_sleep
Specify how much time to sleep between record lock attempts. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.
lock_retry_sleep <sleep_ms>
Default: 0 (meaning no sleep)
See Also
- Record lock error retry and diagnostics
log_change_details
Log change details to the specified text file on the system where the Replication Agent is running.
log_change_details <changeDetailLogFileName>
Default: not set
Note
This configuration should only be used during diagnostic testing due to the extensive performance impact from the numerous log entries.
log_file_name
Defines the filename and location of the replication log.
log_file_name <filename>
By default, replication outputs a log file named ctreplagent.log in its working directory. This path and filename can be configured for locating this file in a central area and to specifically identify which replication environment it is associated with.
notify_events
Trigger execution of an external program, named replmon, by the Replication Agent when certain events occur.
notify_events <yes | no>
Default: no
In V11 and later, the Replication Agent supports a configuration option, notify_events yes, that triggers execution of an external program. This external execution can be a Windows batch file, Unix shell script, or executable and is expected to be named replmon. When an event in the list of event codes below occurs, replmon is called.
Two string values are passed to replmon upon event notification: an error code and one of the numeric event codes listed below. replmon can then process this exception notification as appropriate for the environment.
- Replication Agent is starting
- Replication Agent is shutting down
- Replication Agent is entering paused state
- Replication Agent is resuming operation
- Replication Agent connected to source server
- Replication Agent connected to target server
- Replication Agent lost connection to source server
- Replication Agent lost connection to target server
- Replication Agent disconnected from source server
- Replication Agent disconnected from target server
- Replication Agent terminating due to exception
Example
In the example below, replmon.bat logs the date, time, event description, and error code to the file replevent.log:
@echo off
if "%1" == "1" (
set evdesc=Replication agent is starting.
) else if "%1" == "2" (
set evdesc=Replication agent is shutting down. Error code %2.
) else if "%1" == "3" (
set evdesc=Replication agent has been paused.
) else if "%1" == "4" (
set evdesc=Replication agent is resuming normal operation.
) else if "%1" == "5" (
set evdesc=Replication agent connected to source server.
) else if "%1" == "6" (
set evdesc=Replication agent connected to target server.
) else if "%1" == "7" (
set evdesc=Replication agent lost connection to source server. Error code %2.
) else if "%1" == "8" (
set evdesc=Replication agent lost connection to target server. Error code %2.
) else if "%1" == "9" (
set evdesc=Replication agent disconnected from source server.
) else if "%1" == "10" (
set evdesc=Replication agent disconnected from target server.
) else if "%1" == "11" (
set evdesc=Replication agent terminating due to exception. Exception code %2.
) else (
set evdesc=Replication agent event code %1. Error code %2.
)
for /f "usebackq tokens=*" %%j in (`date /t`) do set dt=%%j
for /f "usebackq tokens=*" %%j in (`time /t`) do set tm=%%j
echo %dt%%tm% %evdesc% >> replevent.log
On Windows, The Replication Agent installs an exception handler so it can notify the external process if an unhandled exception occurs.
num_analyzer_threads
Set the number of analyzer threads used with parallel replication. Analyzer threads coordinate the internal sequence of operations for correct ordering of transactions to the target server when replication is parallelized.
num_analyzer_threads <n>
Default: 1
- <n> - number of threads for the analyzer engine to use.
See Also
num_apply_threads
Set the number of threads to use when applying transactions using parallel replication.
num_apply_threads <n>
Default: 1
- <n> - number of threads
See Also
parallel_apply
Enable parallel replication.
parallel_apply <yes | no>
Default: no
See Also
read_timeout_ms
Set the polling timeout value to use when reading the source server's transaction logs.
read_timeout_ms <timeoutInMilliseconds>
Default: 5000
redirect
By default, the replication agent uses the file path from the source server's transaction logs when accessing the replicated files on the target server. The redirect option allows file names to be redirected to an alternative location on the target server. A simple string match and substitute method is used to modify the new path location.
redirect </path/on/source> </path/on/target>
Default: not set
remember_log_pos
Request the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.
For example, logs are required to be retained when the target server or network connection to the target goes offline for a period of time. When the target server is again available, replication will then commence from it's last current position through all the retained logs thereby bringing the system back to a synchronized state.
remember_log_pos <yes | no>
Default: yes
Note
Replication agent threads store their current minimum required log in the REPLSTATEDT.FCS table on the target server. This file should be removed when the replication state is reset between a pair of source and target servers. For example, after resyncing files, or starting a fresh environment.
See Also
replicate_data_definitions
Replicate data definition operations.
replicate_data_definitions <yes | no>
Default: no
The data definition operations that are replicated when this option is set to 'yes' include:
- Create a file at ISAM level
- Change field definitions (DODA)
- Create an index file
- Add a resource
- Update a resource
- Delete a resource
- Delete a file at ISAM level
- Drop an index
- Rename a file
- Add an identity field
- Delete an identity field
- Reset an identity field
- Set extended attributes for an identity field
- Create a data file at low level
- Delete a data file at low level
- Change the serial number for a file
- Update IFIL resource
- Update header
- Add change id field
- Delete change id field
- Set null key value
resync_file_list
resync_file_list <filename>
Default: not set
Specifies the name of the resync file list that contains the names of all the data files to deploy. Used when using Replication Agent Manager to deploy file groups to a Replication Agent.
socket_timeout
A loss of network connectivity can cause the Replication Agent to hang in a socket send or receive call to a source or target server. To allow the Replication Agent to detect and recover from this situation, V11 and later now supports a configuration option to set a timeout on its send and receive calls. The following option can be specified in the Replication Agent configuration file, ctreplagent.cfg:
socket_timeout <timeout_in_seconds>
Default: 5
A setting of zero disables the timeout.
source_authfile
Set the source server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:
USERID ADMIN
PASSWD ADMIN
source_authfile <sourceAuthFileName>
Default: not set
source_nodeid
Forces required node ID for the source server. This is provided as an arbitrary dotted notation such as 1.1.1.1.
Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.
source_nodeid <dotted_notation>
Default: not set
See Also
- target_nodeid
- Specify Required Node IDs for Source and Targeting
source_server
Set the source server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.
source_server <serverName>
Default: not set
source_tls_cert_file
source_tls_cert_file <certificateFileName>
Default: not set
Specifies the name of the certificate file to use when connecting to the source server using TLS. Requires source_use_tls yes to be enabled.
source_use_tls
source_use_tls <yes | no>
Default: no
source_use_tls yes causes the connection to the source server to use TLS.
start_position
start_position <startPositionOptions>
Default: not set (see note below for default behavior)
The Replication Agent supports setting its starting position on the source server in the Replication Agent configuration file, ctreplagent.cfg, with the start_position configuration option. The option accepts three possible formats:
start_position #current
start_position <log number> <log position> <minimum log>
start_position <log number> <log position> <last commit log> <last commit position>
Examples
Use the source server's current transaction log position and minimum log:
start_position #current
Start with transaction log 10 at offset 62697158, and set minimum required log to 7:
start_position 10 62697158 7
Start with transaction log 12 at offset 7782963, and set last commit position to log 12 offset 7287745:
start_position 12 7782963 12 7287745
The Replication API function, ctReplAgentSetConfiOption(), can be used to set the position. For example:
ctReplAgentSetConfigOption(ctRAOPT_start_position, "10 62697158 7", &repstt, lineno, configfile);
Default start position behavior
If start_position is not specified in the replication agent configuration file, the Replication Agent's start position is set as follows:
- If a file named
ctreplagent_<unique_id>.iniexists, the Replication Agent uses the start position that is specified in that.inifile. - Otherwise, if the Replication Agent has previously saved its start position on the target server, the Replication Agent uses that start position.
- Otherwise, the Replication Agent uses the source server's current transaction log position as its start position.
sync_log_writes
Enable immediate flushing of transaction log writes.
sync_log_writes <yes | no>
Default: yes
sync_log_writes yes causes each write to the local transaction log to be flushed to disk.
system_event_log
system_event_log <machineName>
Default: not set
This option is only supported on Windows systems
When this option is specified, the replication agent writes its messages to the Windows event log in addition to the replication agent log file.
<machineName> is the name of the machine where you want to write event log messages. Specify localhost to write to the local system's event log.
target_authfile
Set the target server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:
USERID ADMIN
PASSWD ADMIN
target_authfile <targetAuthFileName>
Default: not set
target_nodeid
Forces a required node ID for the target server. This is provided as an arbitrary dotted notation such as 1.1.1.1.
Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.
target_nodeid <dotted_notation>
Default: not set
See Also
- source_nodeid
- Specify Required Node IDs for Source and Target
target_server
Set the target server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.
target_server <targetServerName>
Default: not set
target_tls_cert_file
target_tls_cert_file <certificateFileName>
Default: not set
Specifies the name of the certificate file to use when connecting to the target server using TLS. Requires target_use_tls yes to be enabled.
target_use_tls
target_use_tls <yes | no>
Default: no
target_use_tls yes causes the connection to the target server to use TLS.
unique_id
unique_id <replagentUniqueId>
Default: REPLAGENT
Set the unique ID of the Replication Agent for multi-server replication. The unique ID is case-insensitive, and is limited to 32 bytes. This ID is also referred to as the agent ID.
use_target_alignment
use_target_alignment <yes | no>
Default: yes
When use_target_alignment yes is specified, the Replication Agent uses the alignment setting of the file on the target server when applying a record image that it has read from the source server to the target server. Otherwise, the client library's alignment is used.