FairCom DB Configuration Options

PREIMAGE_DUMP

AUTO_RESTORE_POINT (SUBSYSTEM)

FairCom Server supports a configuration option to automatically perform restore points. This option can be set in the configuration file using the SUBSYSTEM TRNLOG AUTO_RESTORE_POINT block. The supported sub-options are:

LOG_INTERVAL N, where is the number of logs between each automatic restore point. Specify 1 to write an automatic restore point to every log. Specify 0 to turn off automatic restore points. LOG_INTERVAL defaults to zero (no automatic restore points).
TIMEOUT T, where T is a time in seconds that the automatic restore point waits for active to transactions to commit. TIMEOUT defaults to 2 seconds.
FAIL_AFTER_TIMEOUT F, can be YES or NO: YES indicates that if transactions remain active after the restore point timeout period, the restore point call fails; NO indicates that if transactions remain active after the restore point period, those transactions are aborted and the restore point is logged. FAIL_AFTER_TIMEOUT defaults to NO.
CHECKPOINT C, where C is YES or NO. YES indicates that a checkpoint is to be logged with the restore point; NO indicates that no checkpoint is to be logged with the restore point. CHECKPOINT defaults to NO.

Example:

SUBSYSTEM TRNLOG AUTO_RESTORE_POINT {

; write an automatic restore point every 2 logs

LOG_INTERVAL 2

; wait for up to 3 seconds for transactions to finish

TIMEOUT 3

; fail if transactions remain active after timeout

FAIL_AFTER_TIMEOUT YES

; write a checkpoint

CHECKPOINT YES

}

Changing automatic restore point settings at runtime

Automatic restore point settings can be changed at runtime by using the ctSETCFG() API function. For example:

rc = ctSETCFG(setcfgCONFIG_OPTION, "subsystem trnlog auto_restore_point {\n\

log_interval 2\n\

timeout 3\n\

fail_after_checkpoint yes\n\

checkpoint yes\n}");

ctadmn use of ctSETCFG()

The ctadmn utility's "Change the specified configuration option" option uses this function, and it now detects when a SUBSYSTEM configuration option is specified. In that situation, it prompts for the SUBSYSTEM sub-options. Here is an example demonstrating how to use ctadmn to turn off automatic restore points at runtime:

10. Change Server Settings

10. Change the specified configuration option

Enter the configuration option and its value >> subsystem trnlog auto_restore_point {

Enter the SUBSYSTEM options you wish to change, one per line.

Finish with a line containing only a closing curly brace: }

To cancel, enter a line containing only an asterisk: *

Enter next line >> log_interval 0

Enter next line >> }

Successfully changed the configuration option.

If an error occurs when changing a SUBSYSTEM option using the ctSETCFG() function, the function returns an error code and may log a descriptive error message to CTSTATUS.FCS. Here is an example, showing a typo in which log_interval0 is specified instead of log_interval 0:

Enter the configuration option and its value >> subsystem trnlog auto_restore_point {

Enter the SUBSYSTEM options you wish to change, one per line.

Finish with a line containing only a closing curly brace: }

To cancel, enter a line containing only an asterisk: *

Enter next line >> log_interval0

Enter next line >> }

Error: Failed to change the configuration option: 749

The following error message will be logged to CTSTATUS.FCS:

Wed Jan 18 14:42:53 2017

- User# 00018 Configuration error: ctSETCFG(), line 2: The option LOG_INTERVAL0 is not supported in the TRNLOG AUTO_RESTORE_POINT subsystem block.

Monitoring automatic restore points

The system snapshot structure, ctGSMS, now includes fields for the automatic restore point settings. The system snapshot version has been updated from 19 to 20:

LONG sarplogint; /* auto restore point log interval */

LONG sarptimout; /* auto restore point tran timeout */

LONG sarplogint; /* auto restore point log interval */

LONG sarptimout; /* auto restore point tran timeout */

/* Auto restore point options bits (ctGSMS sarpoptions field): */

#define ctARP_FAIL_AFTER_TIMEOUT 0x00000001 /* fail if trans active after timeout */

#define ctARP_WRITE_CHECKPOINT 0x00000002 /* write a checkpoint */

A text snapshot now also includes the automatic restore point settings:

automatic restore point interval: 2

active transaction time limit: 3

fail if trans active after timeout: yes

write checkpoint: no

log of last automatic restore point: 81

The ctsnpr utility (included in FairCom DB PRO packages in the source directory) has been updated to support the latest version of the text snapshot output format.

AUTO_TRNLOG

AUTO_TRNLOG <filespec>

Extends the automatic transaction support to include recoverable transactions without any application change. AUTO_TRNLOG, is intended to permit transaction support for applications that do not make transaction calls. See the entry on AUTO_PREIMG for more details.

If a file is included in both AUTO_PREIMG and AUTO_TRNLOG, entries, then the AUTO_PREIMG entry will prevail.

Note: AUTO_TRNLOG accepts wildcard file specifications (see FairCom DB Standard Wildcards).

Files that already have transaction attributes are not affected by the configuration entries. An index file or c-tree Superfile host created without transaction support (and without these keywords) will not be able to be switched at open to transaction support. If such a file is included in the configuration lists for these key words, then the open will cause a CTSTATUS.FCS warning message with the name of the file.

See Also

AUTO_PREIMG

AUTO_TRNLOG_LIGHT

PREIMAGE_DUMP

AUTO_TRNLOG_LIGHT

AUTO_TRNLOG_LIGHT YES

AUTO_TRNLOG is intended to permit transaction support for applications that do not make transaction calls. In some instances, while the transaction support is desired, it may not be necessary to incur the full performance hit intrinsic with TRNLOG support. Adding AUTO_TRNLOG_LIGHT YES to a server configuration file means that the automatic transactions will be considered “light weight” which means that the transaction log entries are not required to be flushed to disk on an automatic commit. The default is NO: automatic transactions will invoke the strict transaction log management of regular transaction calls.

When AUTO_TRNLOG or AUTO_PREIMG are in effect, it may be desirable to know whether or not a legacy application is making “expected” lock calls

Default: NO

See Also

AUTO_PREIMG

AUTO_TRNLOG

COMPATIBILITY NO_COMMIT_READ_LOCK

PREIMAGE_DUMP

AUTOTRAN_ON_CREATE

In v11.5 and later, when creating a transaction-controlled ISAM file, a transaction will automatically be started if one is not open to avoid a possible deadlock caused by starting the transaction in the middle of the file create.

Default: On

CHECKPOINT_FLUSH

CHECKPOINT_FLUSH <# of checkpoints>

This keyword sets the maximum number of checkpoints to be written before a buffer (data or index) holding an image for a transaction controlled file will be flushed. The default value is 2. A value of zero causes the buffer to be flushed at least by the occurrence of the first checkpoint written after the buffer update. Reducing the value of this system parameter reduces the amount of buffering, slowing system performance, but decreases the amount of work to be performed during recovery.

Note: Enabling background transaction flush threads disables this checkpoint flush operation.

Default: 17

See Also

CHECKPOINT_INTERVAL

TRAN_DATA_FLUSH_SEC

TRAN_INDEX_FLUSH_SEC

CHECKPOINT_IDLE

CHECKPOINT_IDLE <# of seconds | -1>

Specifies the time in seconds between checkpoint checks. A checkpoint is an entry in the transaction log which lists open files, active transactions and transactions that are vulnerable due to pending buffer flushes. By default, every 300 seconds the FairCom Server checks if there has been any transaction activity, and if so, if there are any current active transactions. If there has been activity since the last checkpoint, but there is currently no active transaction, a checkpoint occurs. This strategy will not create extra checkpoints when the FairCom Server is idle, with respect to transactions, or when the FairCom Server is busy with transactions.

It is important to note that if an application routinely calls Begin() whether or not updates are imminent, this “idle” checkpoint will be inhibited because there appears to be an active transaction. The purpose of this feature is to increase the likelihood of a clean checkpoint occurring in the transaction log, thus speeding automatic recovery. Ordinarily, checkpoints occur at predetermined intervals in the transaction log. A value of negative one (-1) will disable the idle checkpoint feature.

Default: 300

CHECKPOINT_INTERVAL

CHECKPOINT_INTERVAL <interval in bytes or MB>

This keyword can speed up recovery at the expense of performance during updates. The interval between checkpoints is measured in bytes of log entries. It is ordinarily about one-third (1/3) the size of one of the active log files (L000....FCS). Reducing the interval speeds automatic recovery at the expense of performance during updates. The entry is interpreted as bytes if greater than 100 or as megabytes if less than 100. For example, CHECKPOINT_INTERVAL 2 sets an approximate 2MB interval, while CHECKPOINT_INTERVAL 150000 causes checkpoints about every 150,000 bytes of log file.

Default: 10 MB

See Also

CHECKPOINT_FLUSH

CHECKPOINT_PREVIOUS

CHECKPOINT_PREVIOUS <YES | NO >

When automatic recovery begins, FairCom DB examines S0000000.FCS and S0000001.FCS to determine the location of the last checkpoint; which is the starting point for the automatic recovery. If the checkpoint does not appear to be valid (errors 64, RLEN_ERR and 66, RCHK_ERR), it is possible to have the server attempt to start the recovery at the location of the next to last checkpoint.

Default: NO

CHECK_SYSTEM_FILEID

CHECK_SYSTEM_FILEID <YES/NO>

CHECK_SYSTEM_FILEID YES is intended to distinguish different filesystem-level copies of files and prevent transaction recovery from applying corrections to an incorrect copy of a data or index file following a crash. Operation sequences are as follows:

update A.dat
close A.dat
Filesystem level copy A.dat to B.dat
update B.dat

These sequences are vulnerable to recovery errors if CHECK_SYSTEM_FILEID NO is disabled.
Some distributed file systems or containerized environments do not provide a stable device or inode id and will result in additional CTSTATUS.FCS messages like "reassigning file ID (sysid changed)..." when opening a file, even if the file has not been copied.

Default: Yes

CHKPDFC_LOG_LIMIT

CHKPDFC_LOG_LIMIT <max logs w/o checkpoint>

Ordinarily, several checkpoints are expected within each FairCom Server transaction log file and the FairCom Server would terminate with error CHKP_ERR (529) when two consecutive log files without these checkpoints were encountered. The FairCom Server has been modified with respect to this absence of checkpoints. This keyword permits additional log files to be written without checkpoints. A short wait is introduced for each non-checkpoint log write, allowing a checkpoint to occur and resolving any possible race conditions.

CHKPDFC_LOG_LIMIT specifies how many consecutive logs may be processed without a checkpoint until the FairCom Server terminates. The default number of transaction logs is five log files and this may be lowered to as few as four log files or raised to any reasonable limit.

Once the checkpoint deficiency reaches two logs, the first write of each transaction commit to the transaction logs is slightly deferred. This improves the chances that the FairCom Server checkpoint thread is allotted a slice of time. Once the checkpoint is eventually written, this write log defer is removed.

CLEANUP_ABLIST_ON_ABORT

CLEANUP_ABLIST_ON_ABORT <YES | NO>

When key mark cleanup optimizations are enabled (NXTMARKS, UPDMARKS) and if a great many aborted transactions, checkpoint size becomes large due to many abort node list entries. Possible symptoms could include:

automatic recovery could fail with RCHK_ERR due to a checkpoint that exceeded 16 MB
slow performance

When a transaction aborts, transaction marks are now automatically cleaned in updated nodes. This behavior is on by default (YES) and can be disabled for prior compatibility.

CLEANUP_ABLIST_ON_ABORT NO

This option can be set in your server configuration file or dynamically modified at runtime.

COMMIT_DELAY

COMMIT_DELAY <milliseconds | -1>

Controls the length of time in milliseconds after a given transaction completes that the transaction manager waits before flushing the transaction to disk. By waiting, more than one transaction (that is, the first one and all others that complete before the delay period expires) may be committed at the same time reducing disk-access overhead. On average, the longer the delay, the larger the number of transactions committed.

Note: Keep this delay in mind when setting a time limit for aborting transactions.

A value of "-1" disables this support.

Note: The best practice is to consider the type of storage device your transaction logs are sitting on.

Disable this support when using Solid State Disks (SSDs).

SSD use: COMMIT_DELAY -1

Enable it If you are running on a rotating hard disk.

HDD use: COMMIT_DELAY 1

Default:

Windows 2 ms.

Unix/Linux - This value defaults to 1 ms for best performance.

See Also

COMMIT_DELAY_BASE

COMMIT_DELAY_BASE <cohort size measure>

For advanced control of the intricate commit delay timing statistics, additional controls are available for the commit delay time calculation.

The cohort size measure determines the rate at which the nominal commit delay time is adjusted as the number of cohorts increases or decreases. Increasing this value tends to reduce the amount of change in the blocking time as the number of cohorts changes.

It is recommended that these values be carefully profiled as they can impact performance in many unexpected ways.

Default: 50

See Also

COMMIT_DELAY

COMMIT_DELAY_SCALE

COMMIT_DELAY_USEC

COMMIT_DELAY_MINIMUM

COMMIT_DELAY_MINIMUM <time in microseconds>

Do not use this unless instructed to by FairCom.

In V11.0 and later, allows the minimum time allowable for commit delay to be overridden. Time specified is in microseconds. Example:

COMMIT_DELAY_MINIMUM 500

Default: 1000 microseconds or 0 depending on environment.

COMMIT_DELAY_SCALE

COMMIT_DELAY_SCALE <ratio of block to clear time>

For advanced control of the intricate commit delay timing statistics, additional controls are available for the commit delay time calculation. The ratio of block to clear time cannot be smaller than 1. As this value increases, the amount of time waiting to permit commit delay cohorts to flow past their mutual block is decreased.

It is recommended that these values be carefully profiled as they can impact performance in many unexpected ways.

Default: 2

See Also

COMMIT_DELAY

COMMIT_DELAY_BASE

COMMIT_DELAY_USEC

COMMIT_DELAY_USEC <microseconds> | -1>

Same as COMMIT_DELAY, but interpreted as microseconds. If both forms of this keyword are specified, then the last entry in the configuration file prevails.

Note: Not all systems support arbitrarily short sleep times. FairCom has found, for example on the Solaris operating system, unless using real-time capabilities of the operating system, the minimum achievable sleep time is 10 milliseconds, even if a short sleep time is requested.

See Also

COMMIT_DELAY

COMMIT_DELAY_BASE

COMMIT_DELAY_SCALE

COMMIT_LOCK_DEFER_MS

COMMIT_LOCK_DEFER_MS <defer time in milliseconds>

Provides an additional tuning mechanism for the COMMIT_READ_LOCK retry value.

The length of the defer value can be varied from zero to 100 milliseconds.

Internal tests demonstrated the effect of this change on CPU utilization was dramatic as a reader attempted to retry its read commit lock. Of course, actual performance increases will be variable, depending on any particular server environment. The trade-off with this method is introducing an unnecessary defer (i.e. if the next retry without a non-zero defer would have succeeded). In practice, this was not found to impede performance.

Commit write locks held by the transaction (i.e., locks that block read attempts during the actual commit process) are held during the entire commit. This has no direct impact upon the transaction commit, however, can cause longer delays for a read attempt when the transaction itself is comprised of a large number of write operations (e.g., committing thousands of ADDREC()s)

Default: 10

See Also

COMPATIBILITY LOG_WRITETHRU

Instructs the FairCom Server to open its transaction logs in synchronous write mode. In this mode, writes to the transaction logs go directly to disk (or disk cache), avoiding the file system cache, so the server is able to avoid the overhead of first writing to the file system cache and then flushing the file system. As of FairCom DB Version 9, this is applicable to both Windows and Unix Systems.

Note: On the Solaris operating systems, COMPATIBILITY LOG_WRITETHRU uses the O_DSYNC mode to implement synchronous log writes when available. (Direct I/O with O_SYNC is still used on those systems not supporting O_DSYNC.)

Default: OFF

Also See

COMPATIBILITY TDATA_WRITETHRU

Similar to the strategy used in transaction log flushing, the FairCom Server can avoid excessive flushing of data and index files under transaction control. Two additional keywords affect this behavior:

COMPATIBILITY TDATA_WRITETHRU and COMPATIBILITY TINDEX_WRITETHRU force transaction controlled data files and index files, respectively, to be written directly to disk (whenever c-tree determines that they must be flushed from the c-tree buffers), and the calls to flush their OS buffers are skipped. These keywords cause transaction controlled files to be written through the OS file system cache, rather than written into the file system cache and later explicitly flushed at a database checkpoint. This behavior allows I/O costs to be evenly amortized, reducing the amount of I/O that must be done at a database checkpoint. This results in a smaller variance in transaction times, and potentially greater total transaction throughput. For scenarios without heavy and continuous write activity, this alternate behavior frequently results in reduced throughput.

TDATA_WRITETHRU uses the file system cache. The file is placed into a mode that causes the write to go to file system cache and then to disk before returning; the data still resides in file system cache. Compare to UNBUFFERED_IO, which completely avoids the file system cache.

See Also

COMPATIBILITY TINDEX_WRITETHRU

See Also

COMPATIBILITY TDATA_WRITETHRU

COMPATIBILITY LOCK_EXCL_TRAN

Skipping locks on exclusively opened TRNLOG files is enabled by default. It can be disabled by specifying the option COMPATIBILITY LOCK_EXCL_TRAN in ctsrvr.cfg.

DELAYED_DURABILITY

DELAYED_DURABILITY <N>

DELAYED_DURABILITY <N> (default 0) controls whether or not to use a modified log syncing strategy.

With delayed durability enabled transaction logs are no longer sync'd to persisted storage on each commit (or other internal log buffer flush events) and instead, transaction log data is allowed to be written to the filesystem cache, and a background thread then periodically and consistently syncs transaction log contents to disk.

By allowing committed transaction entries to be written to filesystem cache and deferring the file flush can result in a very large performance gain in many cases. However, there is a trade off as a window of potential data loss vulnerability is then introduced. The period of time that transaction log contents are present in volatile filesystem cache before the flush could mean transactions already reported to the application as committed, in rare cases, might not make it to persisted storage.

With many modern storage devices there is a limited presumption that available capacitance on the system and storage device hardware that data is usually persisted even in a power outage situation, though this is not guaranteed. Thus alternate recovery strategies should be considered.

One strategy to coordinate the known state of committed database transactions with a known application state can be achieved with restore points. Restore points can be triggered by an application to create a known point in time where the application and the database are in sync. A restore point creates a special database transaction log checkpoint that can be later referenced by the application as a known good start point.

With delayed durability enabled, it is recommended to consider the use of restore points for a robust recoverable solution.

When DELAYED_DURABILITY set to 0 disables delayed durability/
When DELAYED_DURABILITY is set to a positive value, <N>, the new strategy is in use and the log sync is guaranteed to occur within <N> seconds. A setting of 1 second is recommended because it results in a good performance gain (higher values offer very little additional benefit). The following configuration options are set as shown below:

SUPPRESS_LOG_FLUSH	YES	(no idle flush of transaction files)
SUPPRESS_LOG_SYNC	YES
IDLE_TRANFLUSH	-1
COMMIT_DELAY	-1	(no commit delay)
FORCE_LOGIDX	ON	(all transaction indices use ctLOGIDX)

Note: If the configuration file has one or more of these configuration entries set inconsistently after the DELAYED_DURABILITY entry, the server logs a message to CTSTATUS.FCS and continues to start, disabling any incompatible options after processing the configuration file.

Warning

When DELAYED_DURABILITY is enabled, recently committed transactions could be lost if FairCom Server terminates abnormally. For automatic recovery to succeed after FairCom Server terminates abnormally, either of the following should be considered.

The application must write a restore point to the log (using the ctflush utility or calling ctQUIET() with mode of ctQTlog_restorepoint) such that a restore point exists prior to the time the server terminated abnormally. In this case, automatic recovery recovers to that restore point.

ctsrvr.cfg must contain the option RECOVER_TO_RESTORE_POINT NO, indicating that no restore point is needed. In this case, automatic recovery recovers to the last data that was written to the log on disk. This is the default configuration.

DELSTK_COMMIT_SEC

The DELSTK_COMMIT_SEC server configuration can be used to improve performance of committing record deletes in a fixed-length data file that uses the TDATA_WRITETHRU option. To use this option, specify the following in ctsrvr.cfg:

DELSTK_COMMIT_SEC nsec

When nsec is greater than zero, this option causes commits of fixed-length record deletes in files that use the TDATA_WRITETHRU option to write the delete stack value to the file's header at most every nsec seconds. This means that, in the event of a server crash, the delete stack value in the header can be out-of-date, not accounting for the last nsec seconds of deleted records. The only impact is that some space in the file will not be available for reuse after automatic recovery.

Default: 0 (disabled)

The maximum value that can be specified is 30.

Changing at Runtime

This option can be changed at runtime using either of these approaches:

Using the ctadmn utility:

Select menu option 10. Change Server Settings, then select 10. Change the specified configuration option, and then enter the configuration option and its value:

delstk_commit_sec 1

FIXED_LOG_SIZE

*** THIS KEYWORD IS NO LONGER SUPPORTED ***

FORCE_LOGIDX

FORCE_LOGIDX <ON | OFF | NO>

FORCE_LOGIDX allows LOGIDX support to be forced on, off, or disabled:

ON forces all indexes to use LOGIDX entries.
OFF forces all indexes not to use LOGIDX entries.
NO uses existing file modes to control LOGIDX entries.

The LOGIDX file mode is an index file mode permitting faster index automatic recovery during FairCom Server startup. Transaction controlled indexes with this file mode are recovered more quickly than with the standard transaction processing file mode TRNLOG. This feature can significantly reduce recovery times for large indexes and has not noticeably degraded the speed of index operations. LOGIDX is only applicable if the file mode also includes TRNLOG.

Note: The LOGIDX file mode is intended for index files only, and is ignored in data files. When adding the LOGIDX file mode to an existing index that is not under transaction control, be sure to rebuild the index to enable transaction control.

Default: ON

See Also

DELAYED_DURABILITY

KEEP_LOGS

KEEP_LOGS <number of inactive logs>

If greater than zero, KEEP_LOGS specifies the number of non-active transaction log files kept on disk in addition to the active log files. When a greater-than-zero KEEP_LOGS value is exceeded, the FairCom Server automatically deletes the oldest inactive log file as new log files are needed. If KEEP_LOGS is zero, inactive log files are immediately deleted by the FairCom Server. If KEEP_LOGS is -1, no inactive log files are deleted by the FairCom Server.

KEEP_LOGS permits the archiving of transaction logs. Inactive log files may be safely moved, deleted, copied or renamed. An inactive log file which is not immediately deleted by the FairCom Server is renamed from the form L*.FCS to the form L*.FCA. The last character in the extension is changed from ‘S’ to ‘A’, with the rest staying the same.

Default: 0

KEEP_RESTORE_POINTS

KEEP_RESTORE_POINTS <N>

KEEP_RESTORE_POINTS <N> allows the server to maintain information about the last N Restore Points. This is somewhat like the KEEP_LOGS keyword. The last N Restore Points are referred to as the "Active Restore Points." It is possible to set N to zero which means there will be no Active Restore Points. If there are no Active Restore Points, then automatic recovery cannot rollback to a quiet transaction state. The list of Active Restore Points is stored in each checkpoint. In the case of a Checkpoint Restore Point, the checkpoint includes itself as the last Active Restore Point.

Note: When N is greater than zero, the server automatically maintains the transactions logs necessary to ensure that a rollback to any of the Active Restore Points is possible. However, KEEP_RESTORE_POINTS does not affect the existence of the Restore Point files. These files are quite small (128 bytes), and are not deleted by the server.

See Also

RECOVER_TO_RESTORE_POINT

LEAF_NODE_READ_LOCK

LEAF_NODE_READ_LOCK YES | NO

Default: YES

When enabled, FairCom Server acquires a read lock on a leaf node if the index does not use key compression. If it finds that the node contains transaction marks, it releases the read lock and acquires a write lock on the node. This greatly improves scalability of leaf nodes reads.

This option is dynamically configurable at runtime using the ctadmn utility's option (menu option 10, and then menu option 10, Change the specified configuration option, again) to change a server configuration option value.

LOG_COMPRESSION_FACTOR

LOG_COMPRESSION_FACTOR <percent>

Default: 80

See Also

LOG_COMPRESSION_THRESHOLD

Note: Log compression is not currently supported.

LOG_COMPRESSION_THRESHOLD

LOG_COMPRESSION_THRESHOLD <bytes>

There are two parameters that control the manner in which the compression is applied to log entries.

The first is a size threshold:
If the variable region is less than this threshold, no compression is attempted.
The second is a compression factor expressed as a percent:
If the compression does not fit in a buffer of size (<factor> * input_size / 100), then no compression takes place.

Snapshot statistics include compression results.

Default: 100

See Also

LOG_COMPRESSION_FACTOR

Note: Log compression is not currently supported.

LOG_EVEN

LOG_EVEN <full_path>L

The alternative name for even numbered transaction log files. This name must be in the form of an optional directory path and the single character ‘L’ (e.g., D:\LOG0\L). The FairCom Server appends a seven-digit even number and the extension .FCS to the name given here.

If a relative path is specified for START_EVEN / START_ODD or LOG_EVEN / LOG_ODD, it is relative to the defined LOCAL_DIRECTORY path, not the startup directory of the server.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Note: The ability to give separate device and directory names for odd and even log files allows them to be directed to different physical storage devices.

Default: L

See Also

LOG_ODD (LOG_ODD, LOG_ODD)

LOG_ODD

LOG_ODD <full_path>L

The alternative name for odd numbered transaction log files. This name must be in the form of an optional directory path and the single character ‘L’ (e.g., D:\LOG1\L). The FairCom Server appends a seven-digit odd number and the extension .FCS to the name provided.

If a relative path is specified for START_EVEN / START_ODD or LOG_EVEN / LOG_ODD, it is relative to the defined LOCAL_DIRECTORY path, not the startup directory of the server.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: L

See Also

LOG_EVEN (LOG_EVEN, LOG_EVEN)

LOG_PAGE_SIZE

LOG_PAGE_SIZE <bytes>

The FairCom Server uses a transaction log buffer to manage log write requests and this is comprised of a number of page size blocks. The LOG_PAGE_SIZE configuration key word is used to change the page size, and ideally, the log page size should match the optimal size for the disk I/O subsystem. The maximum setting is 32K.

Default

The default is 8K.

See Also

PAGE_SIZE

LOG_SPACE

LOG_SPACE <Megabytes>

This is the number of megabytes of disk space allocated to storing active transaction logs, starting with a minimum of 2. The FairCom Server maintains up to 4 active log files, which consume, in the aggregate, up to LOG_SPACE megabytes of disk space. Log files are numbered consecutively starting with 1. The log file names are in the form L0000001.FCS.

For a server with existing transaction logs, If LOG_SPACE is changed and LOG_TEMPLATE is non-zero, the existing log templates will be the wrong size and should be removed. The log template (L0000000.FCT) is only created when the Log # 1 is created, so the transaction log reset procedure should be followed.

V12 Notes

The maximum allowed value for the LOG_SPACE keyword has been increased from 1GB to 4GB-1. This controls the initial size of transaction logs (of 4 logs specifically), so each log can now grow to just under 1GB. Each individual log can now grow up to 4GB if needed, although 3GB is expected to be the largest size, which will be a 1GB normal log size, plus a potential 2GB variable-length record in a transaction. Reducing the number of logs results in less log file churning with high velocity applications leading to faster consistent transaction throughput.

A change in logic allows us to avoid performing a long series of synchronous writes to the transaction log when we start the server in an empty transaction log directory.

This change significantly speeds up server startup on Linux when the file system has write barriers enabled.

Note: When write barriers are disabled on Linux, synchronous writes are much faster than with write barriers enabled, so FairCom highly recommends running with write barriers off and a battery backed power supply on the machine in production systems for best performance.

With this performance improvement, we have been able to increase LOG_SPACE to 1 GB and we have COMPATIBILITY LOG_WRITETHRU on by default.

The defaults in the Server Configuration File (ctsrvr.cfg) have been modified to optimize performance on modern systems.

LOG_SPACE has been increased from 120 MB to 1 GB

LOG_SPACE 1 GB

PAGE_SIZE has been increase from 8192 to 32768

PAGE_SIZE 32768

Note: A file created with a larger PAGE_SIZE cannot be opened by a server with a smaller PAGE_SIZE.

Default: 1 GB (prior to V12: 120 MB)

LOG_TEMPLATE

LOG_TEMPLATE <n>

Enables transaction log templates. <n> is the number of log templates for the server to maintain. A value of 0 means no use of log templates. A value of two (2) means that two blank logs (L0000002.FCT and L0000003.FCT) would be created at first server startup in addition to the template (L0000000.FCT). Log templates have been on by default since V9.

Before enabling log templates on a system that did not have them turned on, any existing transaction logs must be deleted so the server can create the log templates. To do this, shut down the server cleanly and delete Lnnnnnnn.FCS, S0000000.FCS, and S0000001.FCS. When the server is restarted after adding this keyword, initial startup may take longer due to creation of template log files (*.FCT), however, using the templates will result in better performance in high transaction volume environments.

Default: 2

See Also

LOG_TEMPLATE_COPY_SLEEP_PCT

LOG_TEMPLATE_COPY_SLEEP_PCT <percent>

When working with large log templates on high volume systems, the log template copy operation can consume excessive I/O time impacting performance. By periodically deferring the copy operation, this can smooth out I/O spikes. This option, when used with the LOG_TEMPLATE_COPY_SLEEP_TIME option specifies the percentage of data that is written to the target transaction log file after which the copy operation sleeps for the number of milliseconds specified for the LOG_TEMPLATE_COPY_SLEEP_TIME option.

Default: 15
Minimum value: 1
Maximum value: 99

See Also

LOG_TEMPLATE_COPY_SLEEP_TIME

LOG_TEMPLATE_COPY_SLEEP_TIME <milliseconds>

When working with large log templates on high volume systems, the log template copy operation can consume excessive I/O time impacting performance. By periodically deferring the copy operation, this can smooth out I/O spikes. This option causes the copying of the log template to pause for the specified number of milliseconds each time it has written the percentage of data specified by the LOG_TEMPLATE_COPY_SLEEP_PCT option to the target transaction log file.

If an error occurs using this method to copy the log template, the code logs an error message to CTSTATUS.FCS (look for "LOG_TEMPLATE_COPY: ...") and attempts to use the original log template copy method.

A suggested time is 5 ms to start with.

Default: 0 (disabled)
Minimum value: 0
Maximum value: 1000 (1 second sleep)

See Also

LONG_TRANSACTION_MS

LONG_TRANSACTION_MS <milliseconds>

A long transaction is any transaction that exceeds the elapsed transaction time specified. This will cause a message to be written to the SYSMON queue if the PERF_MONITOR configuration option is enabled.

See Also

PERF_MONITOR

MAX_DFRIDX_LOGS

MAX_DFRIDX_LOGS <max_logs>

Changes the limit FairCom Server sets for the number of active transaction logs that are kept for deferred index and Replication Agent processing.

In V12 and later, the default is 100; prior to V12, the default was 50.

Specifying a value of zero for <max_logs> removes the limit.

Tip - If you find error 96 on startup due to logs not found, it is likely be due to MAX_DFRIDX_LOGS or MAX_REPL_LOGS settings removing logs after the max is reached. Be sure to review your operational environment for appropriate settings.

See also:

MAX_REPL_LOGS (MAX_REPL_LOGS, MAX_REPL_LOGS)
Limit number of transactions logs kept for deferred index & Replication Agent

Transaction log limit for replication and deferred indexing

Note: This is a Compatibility Change for FairCom DB V11.5 and later.

The Replication Agent and deferred index processing thread for TRNLOG files read transaction logs and inform the server of their minimum transaction log requirements respectively. In early V11 releases, FairCom DB retained all required logs indicated by these clients. In some cases, this resulted in the number of transaction logs to increase without limit, filling all drive space. A typical scenario is a Replication Agent taken offline without removing its minimum transaction log requirement from the server.

FairCom DB Server now limits by default the number of active transaction logs that are kept for Replication Agent and deferred index processing to 50. This default configuration is a deliberate trade off to avoid accumulating an unlimited number of active transaction logs and potentially running out of critical drive space.

The worst case scenario with this change is your Replication Agent or internal thread that is reading transaction logs can fail with error 96 when reconnecting and searching for a log that it expects present is no longer there. In this case, it will likely be necessary to resync your files through the Replication ReSync feature, a backup and restore, or manual file copy.

Two configuration options were introduced controlling these limits independently for these two features:

MAX_REPL_LOGS <max_logs> - Sets the maximum number of logs to be held specifically for replication.
MAX_DFRIDX_LOGS <max_logs> - Sets the maximum number of logs to be held specifically for the deferred indexing thread.

Use of either of these configuration settings does not impact your FairCom DB Servers’ ability to retain any necessary logs required for Automatic Recovery.

The more comprehensive KEEP_LOGS configuration keyword always overrides both MAX_DFRIDX_LOGS and MAX_REPL_LOGS configuration settings in either case.

When a new transaction log is created and one of these limits is in effect and the required number of logs exceeds the limit, FairCom DB logs one of the following messages to CTSTATUS.FCS and sets the minimum log requirement of the deferred index thread or Replication Agent to a larger value equal to the log limit:

- User# 00039 Deferred index log requirement (current=42, required=32) exceeds MAX_DFRIDX_LOGS limit. Setting lowlog to 33.

- User# 00038 Replication reader AGENT1 log requirement (current=43, required=32) exceeds MAX_REPL_LOGS limit. Setting lowlog to 33.

These keywords can be set at runtime by using the command-line tool ctadmn, option 10 (Change the specified configuration option) and setting the new value, like this:

10. Change the specified configuration option

Enter your choice (1-10), or 'q' to return to previous menu>> 10

Enter the configuration option and its value >> MAX_REPL_LOGS 75

Successfully changed the configuration option.

They can be set programmatically by calling the ctSETCFG() function. Examples:

/* Set maximum logs for deferred index thread to 100. */

ctSETCFG(setcfgCONFIG_OPTION, "MAX_DFRIDX_LOGS", "100");

/* Set maximum logs for replication agents to 0 (no limit). */

ctSETCFG(setcfgCONFIG_OPTION, "MAX_REPL_LOGS", "0");

MAX_PREIMAGE_DATA

MAX_PREIMAGE_DATA <limit> sets the maximum size of in‑memory data that is allocated by a transaction to <limit>. After that amount of data has been allocated, subsequent allocations are stored in a preimage swap file on disk. The default value is 1 GB.

See Also

MAX_PREIMAGE_SWAP (MAX_PREIMAGE_SWAP, MAX_PREIMAGE_SWAP)

Millions of Records per Transaction

MAX_PREIMAGE_SWAP

MAX_PREIMAGE_SWAP <limit> sets the maximum size of the preimage swap file on disk. If the file reaches its maximum size and a transaction attempts to allocate more space in the file, the operation fails with error TSHD_ERR (72). The default value is zero (meaning no limit).

See Also

MAX_PREIMAGE_DATA (MAX_PREIMAGE_DATA, MAX_PREIMAGE_DATA)

Millions of Records per Transaction

MAX_REPL_LOGS

MAX_REPL_LOGS <max_logs>

Changes the limit FairCom Server sets for the number of active transaction logs that are kept for deferred index and Replication Agent processing.

In V12 and later, the default is 100; prior to V12, the default was 50.

Specifying a value of zero for <max_logs> removes the limit.

See also:

MAX_DFRIDX_LOGS (MAX_DFRIDX_LOGS, MAX_DFRIDX_LOGS) (MAX_DFRIDX_LOGS, MAX_DFRIDX_LOGS)
Limit number of transactions logs kept for deferred index & Replication Agent

MAX_TRANSACTION_JUMP

MAX_TRANSACTION_JUMP <maximum transaction increase value>

MAX_TRANSACTION_JUMP limits the maximum increase in the server's transaction number when opening a transaction-controlled file. This can prevent a shutdown due to transaction number overflow.

MAX_TRANSACTION_JUMP specifies a maximum transaction increase value. When opening a transaction-controlled file, if the file's transaction high water mark in the file header exceeds the server's current transaction number by more than the specified value, the server fails the file open with error 533 (MTRN_ERR). In this situation, use the ctclntrn utility to clean the transaction marks in the file, allowing the server to open it.

A value of 0 disables the setting.

If you don't know what to set it to, we recommend starting with a value of 1,000,000,000.

Default: 0 (disables the setting)

See also:

Transaction High-Water Marks

MAX_USER_LOG_ENTRY_BYTES

MAX_USER_LOGS <# of logs>

An optional limit for how many active transaction logs a transaction spans before it is aborted or abandoned. The default, ZERO, disables the check for long transactions.

When specified, MAX_USER_LOGS takes as its argument the maximum number of logs a transaction may span. If a transaction exceeds the limit, an attempt is made to abort the transaction. If the transaction cannot be aborted (consider the case where an abort would cause the server to fail), the transaction is abandoned. This means the client thread will lose its connection to the server, and the application may receive errors ARQS_ERR (127) or ARSP_ERR (128). There is no guarantee that a transaction will not span more logs than the specified maximum, however, the transaction will end within a reasonable number of logs.

If the transaction is aborted, then the next call by the client will return error MLAB_ERR (821) to indicate the operation was not completed and the pending transaction has been aborted. (See the end note for a special case of this error condition.) A message of the following form will be made in CTSTATUS.FCS:

Sun Dec 03 08:53:21 2006

- User# 00011 Transaction aborted at ct_mul_abandon1 for user# 9: 821

If the transaction is abandoned (that is, no explicit abort written in the log), then the client will be disconnected from the server. CTSTATUS.FCS entries such as the following reflect logs growing from a transaction that is pending, then the detection of the long transaction, then the eventual abandonment:

Sun Dec 03 09:53:42 2006

- User# 00012 The number of active log files increased to: 5

Sun Dec 03 09:53:42 2006

- User# 00012 Transaction (started in log 1) still pending.

User# 11 |GUEST||

Sun Dec 03 09:53:55 2006

- User# 00012 The number of active log files increased to: 6

Sun Dec 03 09:53:55 2006

- User# 00012 Abandoned Transaction

Sun Dec 03 09:54:10 2006

- User# 00012 The number of active log files increased to: 7

Sun Dec 03 09:54:10 2006

- User# 00012 Abandoned Transaction2

Sun Dec 03 09:54:10 2006

- User# 00012 Abandoned transaction kill request posted against user #11

|GUEST||

Sun Dec 03 09:54:10 2006

- User# 00011 ctntio: send error - O11 bytes=0 pErr=127

|GUEST||: 168

Sun Dec 03 09:54:25 2006

- User# 00012 The number of active log files decreased to: 4

The number of logs continued to grow, and then shrink, as reflected in the above excerpt because in addition to a transaction sleeping on a blocked lock, another unrelated application was continuing to add records to its files and corresponding entries in the transaction logs.

Note: In some rare situations error TRAB_COD (-823) can be returned instead of MLAB_ERR. This indicates the requested operation was completed before the abort actually took place. Usually, this is the same condition as an MLAB_ERR, as the transaction is aborted. In practice, the TRAB_COD should be rare.

Default: 0

MAX_USER_LOGS

MAX_USER_LOGS <max number of logs>

MAX_USER_LOGS controls how many logs a transaction can span before attempts are made to abort or abandon the transaction. The default, ZERO, disables the check for long transactions.

If a transaction cannot be aborted (for example, a server fault would occur) the transaction is abandoned which means that the client thread loses its connection to the server. There is no guarantee that the transaction will not span more logs than the specified maximum, but the transaction will end within a reasonable number of logs.

If the transaction is aborted, then the next call by the client will return a MLAB_ERR (821) to indicate the operation was not completed and the pending transaction has been aborted. A message of the following form will be made in CTSTATUS.FCS:

Sun Dec 03 08:53:21 2006

- User# 00011 Transaction aborted at ct_mul_abandon1 for user# 9: 821

If the transaction is abandoned (which means no explicit abort written in the log), then the client will be disconnected from the server. CTSTATUS.FCS entries such as those shown below reflect logs growing from a transaction that is pending, then the detection of the long transaction, then the eventual abandonment:

Sun Dec 03 09:53:42 2006

- User# 00012 The number of active log files increased to: 5

Sun Dec 03 09:53:42 2006

- User# 00012 Transaction (started in log 1) still pending.

User# 11 |GUEST||

Sun Dec 03 09:53:55 2006

- User# 00012 The number of active log files increased to: 6

Sun Dec 03 09:53:55 2006

- User# 00012 Abandoned Transaction

Sun Dec 03 09:54:10 2006

- User# 00012 The number of active log files increased to: 7

Sun Dec 03 09:54:10 2006

- User# 00012 Abandoned Transaction2

Sun Dec 03 09:54:10 2006

- User# 00012 Abandoned transaction kill request posted against user #11

|GUEST||

Sun Dec 03 09:54:10 2006

- User# 00011 ctntio: send error - O11 bytes=0 pErr=127

|GUEST||: 168

Sun Dec 03 09:54:25 2006

- User# 00012 The number of active log files decreased to: 4

PREIMAGE_FILE

PREIMAGE_FILE <Full_path>D

The alternative name for the file containing preimages swapped to disk. The format for this name is an optional directory path, which may include a Drive ID, followed by the single character ‘D’ (e.g., E:\SWAP\D). The FairCom Server appends a seven-digit number and the extension .FCS to the name provided here.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: D

START_EVEN

START_EVEN <full_path>S

The alternative name for even numbered start file. The start file contains the location at which the automatic recovery routines begin to scan the transaction logs. There are two start files (numbered zero and one) to reduce the risk of losing the starting point for automatic recovery. This name must be in the form of a directory path and the single character ‘S’ (e.g., C:\START\S). The FairCom Server appends a seven-digit even number and the extension .FCS to the name provided.

If a relative path is specified for START_EVEN / START_ODD or LOG_EVEN / LOG_ODD, it is relative to the defined LOCAL_DIRECTORY path, not the startup directory of the server.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: S

See Also

START_EVEN_MIRROR

START_ODD

START_ODD_MIRROR

START_ODD

START_ODD <full_path>S

The alternative name for odd numbered start file. The start file contains the location at which the automatic recovery routines begin to scan the transaction logs. There are two start files (numbered zero and one) to reduce the risk of losing the starting point for automatic recovery. This name must be in the form of a directory path and the single character ‘S’ (e.g., C:\START\S). The FairCom Server appends a seven-digit odd number and the extension .FCS to the name provided.

If a relative path is specified for START_EVEN / START_ODD or LOG_EVEN / LOG_ODD, it is relative to the defined LOCAL_DIRECTORY path, not the startup directory of the server.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: S

See Also

START_EVEN

START_EVEN_MIRROR

START_ODD_MIRROR

SUPPRESS_LOG_FLUSH

SUPPRESS_LOG_FLUSH <YES | NO>

Causes transaction begin and commit operations to skip the flushing of the log file when its argument is YES. The default is NO. Suppressing the log flush makes it impossible to perform a proper automatic recovery. However, a dynamic dump will capture the necessary log information to restore TRNLOG files to a clean, consistent state. Using this keyword without the PREIMAGE_DUMP keyword is not recommended

By turning on PREIMAGE_DUMP and using PREIMG files, your system can run much faster than with full transaction processing, and still perform on-line dynamic dumps which will permit restoring files to the time of the dump in a clean, consistent state. However, it will NOT be possible to roll forward from the restored files because transaction log entries are not maintained outside of the dump process. See also PREIMAGE_DUMP and Advanced - Faster Auto-Recovery.

Note: We do not recommend disabling this keyword, as your data integrity will suffer. Be sure you understand what you are doing if you plan to change the default setting of this keyword.

Default: NO

See Also

SUPPRESS_LOG_SYNC

Skips the sync'ing to disk from a log flush operation. This only applies to transaction begin/end operations. There are other system operations that cause log flushes, however, these are not affected. These other log flushes should also be relatively low in number.

Note: We do not recommend disabling this keyword, as your data integrity will suffer. Be sure you understand what you are doing if you plan to change the default setting of this keyword.

See Also

TRAN_HIGH_MARK

TRAN_HIGH_MARK <long integer>

Specifies a transaction number threshold value. If a file is opened with a high-water mark value greater than this threshold, a message is placed in CTSTATUS.FCS listing the file name.

For example, the following configuration entry would cause any file whose high-water mark exceeds one million to have its name listed in CTSTATUS.FCS.

TRAN_HIGH_MARK 1000000

TRAN_OVERFLOW_THRESHOLD

TRAN_OVERFLOW_THRESHOLD <transaction_number>

This keyword causes the c-tree Server to log the following warning message to CTSTATUS.FCS and to standard output (or the message monitor window on Windows systems) when the current transaction number exceeds the specified transaction number:

WARNING: The current transaction number (####) exceeds the user-defined threshold.

The message is logged every 10000 transactions once this limit has been reached. The TRAN_OVERFLOW_THRESHOLD limit can be set to any value up to 0x3ffffffffffff, which is the highest 6‑byte transaction number that FairCom DB supports.

When FairCom DB supports 6‑byte transaction numbers it does not display transaction overflow warnings until the current transaction number approaches the 6‑byte transaction number limit. But if 4‑byte transaction number files are in use, a key insert or delete will fail if the current transaction number exceeds the 4‑byte transaction number limit (however, FairCom DB will continue operating). This keyword allows the server administrator to determine when the server will issue a warning that its transaction number is approaching the limit.

See Also

Extended Transaction Number Support

TRAN_TIMEOUT

TRAN_TIMEOUT <interval>

There are occasions where it is valuable to limit the time that a FairCom Server transaction is allowed to span. Long held transactions can cause a number of application-related issues. For example, holding locks on a record, or preventing updates to be available to other users in a timely manner.

TRAN_TIMEOUT sets a time limit on a transaction: when a transaction is started, once the time period passes, if the transaction is still active it is aborted regardless of what the user is doing (the user could be idle for example).

<interval> is specified in seconds.
The minimum value for the timeout <interval> is 10 seconds. Any value between 1 and 10 is the same as 10.
If the <interval> is set to 0 (or a negative number), this feature is turned off.

TRAN_TIMEOUT is also useful to avoid increases in the number of active transaction logs which can occur due to a user starting a transaction and then remaining idle without committing the transaction, while other transaction activity occurs. TRAN_TIMEOUT aborting the transaction releases locks acquired within the transaction.

Deferred Transaction Begins

A deferred begin transaction (a transaction started using ctDEFERBEG), only starts counting the transaction time when the transaction is converted to an actual transaction (typically on the first update made in that transaction). For example, if you start isql and do some SELECTS then look at the transaction time shown for the SQL connection by ctadmn, it will show '--' indicating the transaction is not yet an actual transaction (this is because SQL threads use ctDEFERBEG transactions). Once you perform an update, ctadmn will show the transaction time counting, and if TRAN_TIMEOUT is in effect, the transaction will be aborted if it does not commit before the TRAN_TIMEOUT limit.

Transaction Timeout Statistics

The USERINFO() function returns state information for a particular connection to the FairCom Server. Included in the state information is the elapsed transaction time for that connection. When a transaction is started with the ctDEFERBEG mode, the elapsed transaction time value returned by USERINFO() is based on the time at which the TRANBEG() call was made.

For a transaction whose begin has been deferred, USERINFO() returns an elapsed transaction time of zero until the transaction begin is converted to an actual transaction begin, at which point the elapsed transaction time is calculated from that time.

Default: No timeout

TRANSACTION_FLUSH

TRANSACTION_FLUSH <# of updates>

This keyword provides control for the maximum number of updates to a buffer (data or index) before it is flushed. The buffer may well be flushed prior to this number of updates because of the LRU (Least Recently Used) scheme (used for index caches but not data caches) or because of the checkpoint limit. This system parameter affects only buffers holding images for transaction controlled files. Reducing this value reduces the amount of buffering, slowing system performance; but decreases the amount of work to be performed during recovery. A value of zero causes the buffer to be flushed upon update.

Default: 500000

UNBUFFERED_LOG_IO

UNBUFFERED_LOG_IO <YES | NO>

Enable separate unbuffered I/O for transaction logs.

OS Support

This option is supported on the Windows operating system.

In V11 and later, support for direct I/O has been enabled on Linux systems. A value of 512-bytes is used for size and alignment for direct I/O.

This feature supports both c-tree data and index files, as well as transaction logs. Configuration options are provided for both.

UNBUFFERED_IO filename (enables direct I/O for the specified file; the filename can include wildcards, such as *.dat)
UNBUFFERED_LOG_IO YES (enables direct I/O for the transaction logs).

Note: This feature requires Linux kernel version 2.6 or later, FairCom Server logs an error message to CTSTATUS.FCS if these options are used on pre-2.6 Linux kernel systems. The error messages are:

The UNBUFFERED_IO option requires Linux kernel version 2.6 or later

The UNBUFFERED_LOG_IO option requires Linux kernel version 2.6 or later

See Also

UNBUFFERED_IO

Recovery

FairCom DB can completely recover from normal server shutdowns when data is under complete transaction processing control. These options control how recovery proceeds in various scenarios.

WARNING: Use caution when specifying options skipping files as they can result in data loss if you're not absolutely certain of the operation of your application regards to data file handling.

RECOVER_DETAILS

Enables logging of detailed information about the FairCom DB automatic recovery process.

RECOVER_FILES

Allows setting separate limits on the number of files used during automatic recovery and regular operations.

RECOVER_MEMLOG

Loads one or more transaction logs into memory during automatic recovery to speed the recovery process.

RECOVER_SKIPCLEAN

Causes files that appear to have been properly closed to not be updated during recovery.

RECOVER_TO_RESTORE_POINT

Causes automatic recovery to recover to the last Restore Point.

REDIRECT

Redirects filename references in the transaction logs during automatic recovery to the specified new filename.

REDIRECT_IFIL

Specifies a file containing a list of files to be altered using the filename redirection rules specified with REDIRECT options.

SKIP_INACCESSIBLE_FILES

Forces automatic recovery to skip any file that is not accessible.

SKIP_MISSING_FILES

If a user file required during automatic recovery is missing, this keyword causes an error to be logged so the FairCom Server will successfully start up.

TRANIDX_LOPN_ERR_CONTINUE

Forces automatic recovery to continue in an unusual situation in which index reconstruction fails during automatic recovery.

RECOVER_DETAILS

RECOVER_DETAILS <YES | NO>

This keyword enables logging of detailed information about the FairCom DB automatic recovery process. The time spent for each phase of automatic recovery in addition to the number of transactions processed for each phase is provided. This keyword adds minimal overhead to FairCom Server operations.

Below is an example of messages that can be found in CTSTATUS.FCS when LOGIDX is not used during automatic recovery. The description in square brackets indicates why LOGIDX was not used:

Mon Nov 23 09:32:44 2009

- User# 00001 Index repair time: 0 seconds.

Mon Nov 23 09:32:49 2009

- User# 00001 tranrcv: Reconstructing index mark.idx [LOGIDX not in file header]

Mon Nov 23 09:32:51 2009

- User# 00001 tranrcv: Reconstructing index mark.idx M#01 [LOGIDX not in file header]

Mon Nov 23 09:32:52 2009

- User# 00001 tranrcv: Reconstructing index mark.idx M#02 [LOGIDX not in file header]

Mon Nov 23 09:32:53 2009

- User# 00001 Index composition time: 9 seconds.

Below is an example of messages found in CTSTATUS.FCS when LOGIDX is used during automatic recovery:

Mon Nov 23 10:46:26 2009

- User# 00001 Index repair time: 0 second(s) for 1 repair(s).

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Recomposing index file FAIRCOM.FCS DI:

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Processing abort node list entries.

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Recomposing index file mark.idx:

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Processing LOGIDX node entries.

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Checking index delete stack.

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Recomposing index file mark.idx M#01:

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Processing LOGIDX node entries.

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Recomposing index file mark.idx M#02:

Mon Nov 23 10:46:26 2009

- User# 00001 tranrcv: Processing LOGIDX node entries.

Mon Nov 23 10:46:26 2009

- User# 00001 Index composition time: 0 second(s).

Default: YES

RECOVER_FILES

RECOVER_FILES <number of files | NO>

RECOVER_FILES makes it possible to set separate limits on the number of files used during automatic recovery and regular operations. The reason automatic recovery may require more files than regular operations is that during recovery files opened stay open until the end of recovery. RECOVER_FILES takes as its argument the number of files to be used during recovery. If this is less than the number used during regular operation specified by the FILES keyword, the number of recovery files is set equal to the regular files and the keyword has no affect. If the number of recovery files is greater than the number of operational files, the number of files is adjusted downward at the end of automatic recovery freeing memory used by the additional control blocks, about 900 bytes per logical file.

Default: NO

RECOVER_MEMLOG

RECOVER_MEMLOG <# of logs to load | NO>

Loads one or more transaction logs into memory during automatic recovery to speed the recovery process. The argument for this keyword specifies the maximum number of memory logs loaded into memory during automatic recovery.

Default: NO

RECOVER_TO_RESTORE_POINT

RECOVER_TO_RESTORE_POINT <YES | NO>

YES causes automatic recovery to recover to the last Restore Point.

When RECOVER_TO_RESTORE_POINT is YES, then automatic recovery (after a crash) comprises two steps:

the recovery of all transactions committed before the crash; and
the rollback of transactions to the last Active Restore Point.

If DELAYED_DURABILITY is in effect at the time of the crash, then in step 1 it is not guaranteed that all transactions committed after the last Restore Point have their transaction log entries on disk (i.e., permanent storage).

Note: If DELAYED_DURABILITY is in effect and RECOVER_TO_RESTORE_POINT is NO, then automatic recovery will attempt to recover all transactions that had committed before the crash; but some transactions committed after the Restore Point and before the crash may be recovered and others lost so that the files may be in an unexpected state. There is no way to predict which transactions may have been lost.

See Also

KEEP_RESTORE_POINTS

RECOVER_SKIPCLEAN

RECOVER_SKIPCLEAN <YES | NO>

This keyword may improve recovery times under certain circumstances. Files which appear to have been properly closed are not updated during recovery.

Note: For non-server systems, set the NINT global variable ctskpclnfil to 1 to enable this feature.

REDIRECT

The Redirect feature is a useful feature allowing a file originating in one directory structure to be repositioned into another directory location during dynamic dump restore. This support has been extended to FairCom DB autorecovery.

One or more of the following configuration entries are used to specify redirection rules in the server configuration file ctsrvr.cfg:

REDIRECT <old path> <new path>

REDIRECT redirects filename references in the transaction logs during automatic recovery to the specified new filename. This option is useful when c-tree data and index files are moved to a different location (on the same system or on another system) before running automatic recovery.

To specify an empty string for one of the !REDIRECT arguments use a pair of double quotes ("").

Examples

If a file originally existed with the name and path C:\Documents and Settings\Administrator\c-tree Data\customer.dat and now exists as the file D:\Documents and Settings\Guest\customer.dat, the following option will allow automatic recovery to proceed and find the file in its new location:

REDIRECT "C:\Documents and Settings\Administrator\c-tree Data" "D:\Documents and Settings\Guest"

Here's a similar example using Unix paths, where the original file is named /users/administrator/c-tree data/customer.dat and the file now exists as /users/guest/customer.dat:

REDIRECT "/users/administrator/c-tree data" "/users/guest"

Note: Use double quotes when a filename contains spaces.

See Also

REDIRECT_IFIL

REDIRECT_IFIL <filename>

As a result of redirection during automatic recovery, if the IFIL resource of the file contained a path, this path would be incorrect after the file was redirected to the new location. To support copying c-tree files from one directory location to another (on the same system or on a different system) and accessing them in their new location, it is necessary to update any filename paths in a c-tree data file's IFIL resource.

The FairCom DB configuration option REDIRECT_IFIL <filename> provides support for automatically modifying redirected files on the server. When this option is specified, on server start up (after automatic recovery completes) the file named <filename> is opened and its list of file names is read from it. <filename> is a text file containing one c-tree data file per line. For each file specified in <filename>, FairCom DB opens the file and uses the filename redirection rules (specified with one or more of the REDIRECT options) to change the data and index file paths in the IFIL resource of the file.

Refer to the FairCom DB ctredirect standalone utility to manually modify files that may have been moved.

See Also

REDIRECT

SKIP_INACCESSIBLE_FILES

SKIP_INACCESSIBLE_FILES YES

Forces automatic recovery to skip any file that is not accessible.

See Also

SKIP_MISSING_FILES

SKIP_MISSING_FILES <YES | NO>

This keyword is available for special FairCom Server startup conditions. If a user file required by the FairCom DB Server during automatic recovery was deleted, an error 12 might be returned and the FairCom Server would not continue. By adding SKIP_MISSING_FILES to the default ctsrvr.cfg file, the error will be logged and the FairCom Server will successfully start up.

SKIP_MISSING_FILES is not recommended as a permanent setting. Deleting files under transaction processing control adversely affects database integrity.

Default: NO

See Also

TRANIDX_LOPN_ERR_CONTINUE

In unusual situations, index reconstruction during automatic recovery could fail with error LOPN_ERR (96) due to a missing transaction log (e.g., L*.FCS). Starting with FairCom DB V11.5, it is possible to use the configuration option TRANIDX_LOPN_ERR_CONTINUE to allow automatic recovery to continue by adding this keyword to ctsrvr.cfg and restarting the FairCom DB Server process.

If recovery then completes successfully, the only issues caused by the missing transaction log was an inability for the FairCom DB Server to properly update an index node on disk.

In this case, the data files under transaction logging control (e.g., with file mode ctTRNLOG) are fully intact, and only the indexes need to be rebuilt to ensure the data and index files are fully in sync. Note, this is expected to be an unusual situation and FairCom recommends using extreme caution with this keyword. FairCom does not recommend enabling this keyword by default. It should only be enabled when required to recover from a catastrophic situation.

To use this option, add TRANIDX_LOPN_ERR_CONTINUE to ctsrvr.cfg. When this option forces the automatic recovery to continue, the following message is logged to CTSTATUS.FCS:

WARNING: tranidx could not open previous log file: 96

Upon completion of automatic recovery in this situation, the following message is logged to CTSTATUS.FCS, and FairCom DB Server terminates with error 571 to indicate to the administrator that all indexes should be rebuilt:

NOTE: tranidx continued without required logs. Recovery completed but index rebuild is required.

Automatic recovery terminated with error: 571

If the automatic recovery does NOT complete successfully after adding this keyword, then the missing transaction log is preventing FairCom DB from being able to successfully recover from the catastrophic situation and a backup should be used to restore the data to a known state.

File Management

AUTO_CLNIDXX

Permits automatic CLNIDXX() during file open when a transaction high water mark, hghtrn, is found at file open to exceed the current transaction number.

AUTO_MKDIR

When creating a c-tree data or index file, causes FairCom DB to automatically create directories for the filename that do not exist.

BLOCKING_OPEN_RETRY_DEFER

When an application requests retrying a file open call that failed because the file's creation is pending, this option specifies the number of milliseconds between retry attempts.

BLOCKING_OPEN_RETRY_LIMIT

When an application requests retrying a file open call that failed because the file's creation is pending, this option specifies the maximum number of retries.

CHECK_FILENAME_VALIDITY (CHECK_FILENAME_VALIDITY, CHECK_FILENAME_VALIDITY)

Restore Driver-Relative path handling as prior to V12.

COALESCE_TRNLOG

Attempt to combine deleted space with adjacent deleted space in ctTRNLOG files that do not have a RECBYT index.

FILE_CREATE_MODE

Specifies the desired file permission mode assigned to files on Unix systems.

FILE_HANDLES

On Unix, changes the number of file handles (at O/S level) that the O/S allows to be used by the FairCom Server.

FILE_PERMISSIONS

Permits default file permissions to be assigned to one or more groups including two special groups: WORLD and OWNER.

INHERIT_FILE_PERMISSIONS

Enables or disables inheriting file permissions from world to group to owner.

KEEPOPEN_CLOSE_RETRY_LIMIT

Determines the number of times to retry the close operation before failing.

KEEPOPEN_LIST

Upon file creation or physical open, attaches the KEEPOPEN attribute to the data file (and its indexes) if the file is a data file that matches a <file spec> and the file creation/open is part of an ISAM creation or open.

MAX_DFRIDX_LOGS (MAX_DFRIDX_LOGS, MAX_DFRIDX_LOGS)

Changes the limit FairCom Server sets for the number of active transaction logs that are kept for deferred indexing.

MAX_FILES_PER_USER

Limits number of files per user when auto resizing comes into effect (e.g., a file operation uses a file number beyond the existing client file range or a new file number is assigned automatically).

MAX_KEEPOPEN_FILES

Sets a limit on the number of files that can be kept in the KEEPOPEN state (meaning the file has been closed by all connections, but the server is keeping the file open for reuse).

MAX_VIRTUAL_FILES

Specifies the maximum number of virtual files that may be opened at one time.

MEMFILE_MAX_BINS

Sets the maximum number of hash bins allowed for a memory file at run time.

MEMORY_FILE

Enables creating memory files if the file matches the specified file name.

SPLIT_NBR_OF_FILES

Sets the number of files when using the partitioning file rule.

TMPNAME_PATH

Sets the default directory for temporary files.

Compression

CMPREC_TYPE

Specifies the type of data compression type for files.

COMPRESS_FILE

Forces FairCom DB to create data files whose names match the specified name with data compression enabled.

FILESYS_COMPRESS_FILE

Enables Windows file system compression on specified data and index files.

Segmented Files

HUGE_TO_SEG_MB

Force any huge file to be created as a segmented file.

MATCHING_SEGMENT

Specifies behavior when an attempt to create a new file segment encounters an existing segment with the same name and the file ID matches the host file’s ID and other attributes.

NONMATCHING_SEGMENT

Specifies behavior when an attempt to create a new file segment encounters an existing segment with the same name and the file ID does not match the host file’s ID and other attributes.

Diagnostics

DIAGNOSTICS PTADMIN

Enables the filename and list of open instances of that file to be logged to CTSTATUS.FCS when a partition member purge fails with error 718.

AUTO_CLNIDXX

AUTO_CLNIDXX YES

Optionally permits automatic CLNIDXX() during file open when a transaction high water mark, hghtrn, is found at file open to exceed the current transaction number.

CTSTATUS.FCS receives messages concerning on the fly CLNIDXX(). Both success and failure are noted.

AUTO_MKDIR

AUTO_MKDIR <YES | NO>

When creating a c-tree data or index file, FairCom DB can automatically create directories for the filename that do not exist.

Programmers can call the CreateFileXtd8 file create function and set the ctAUTOMKDIR bit in the splval field of the xcreblk parameter to cause directories that do not exist to be created when creating that file.

Default: No

BLOCKING_OPEN_RETRY_DEFER

BLOCKING_OPEN_RETRY_DEFER <# of milliseconds between retries>

Configures the delay in milliseconds between retries when retrying a failed file open.

NOTE: Specifying a large BLOCKING_OPEN_RETRY_DEFER can slow performance, so FairCom generally recommends using the default BLOCKING_OPEN_RETRY_DEFER setting, and instead tweaking the total number of retries using BLOCKING_OPEN_RETRY_LIMIT.

BLOCKING_OPEN_RETRY_LIMIT

BLOCKING_OPEN_RETRY_LIMIT <maximum # of retries>

Configures the maximum number of retries allowed when retrying a failed file open. You can tweak this number to find your preferred balance between the time spent retrying and the number of opportunities for success.

A value of 0 means infinite retries.

Retries will be stopped if the server shuts down or an administrator terminates the connection.

Default: 10

See Also: You can use BLOCKING_OPEN_RETRY_DEFER to configure the time delay between retries, though this is not generally recommended.

CHECK_FILENAME_VALIDITY

CHECK_FILENAME_VALIDITY YES | NO

FairCom DB V12 and FairCom RTG V3 forward no longer support “Drive-Relative” Paths by default. The purpose is to ensure predictable, safe, and secure behavior. and all files must be opened using an absolute file path. For backwards compatibility, you can restore prior behavior by adding the CHECK_FILENAME_VALIDITY NO configuration option to ctsrvr.cfg.

On Microsoft Windows,® FairCom uses the concept of a "Drive-Relative Path" as an easy way to specify a relative path to where the database is installed on a drive. The path includes a drive letter without a following backslash. For example, C:mydirectory\test.dat is a drive-relative path because there is not a backslash after the drive specification (C:). On the other hand, C:\mydirectory\test.dat is not a drive-relative path; it is an absolute path. A drive-relative path is relative to whatever happens to be the current working directory of the database on that drive. A process may have a current working directory for each drive.

For example, if the working directory on C: is C:\FairCom, then the drive-relative path C:mydirectory\test.dat is equivalent to C:\FairCom\mydirectory\test.dat while the absolute path C:\mydirectory\test.dat is unaffected.

Drive-Relative Paths can cause behavior that is unpredictable, unsafe, or unsecure. Thus, the database now returns an error when a Drive-Relative Path is specified in a call to create or open a file.

Compatibility Notes: This modification breaks compatibility by disallowing certain file names that used to be allowed. The preferred solution is to use a full path. (A less preferable solution is to use the configuration option CHECK_FILENAME_VALIDITY NO in ctsrvr.cfg. This option can also be changed at runtime. In standalone mode, this behavior is controlled by a field in the CTGVAR structure, ctCheckFileNameValidity. Setting it to a non-zero value enables the check; setting it to zero disables the check.)

CMPREC_TYPE

CMPREC_TYPE < TYPE >

Specifies the type of data compression type for files. Several algorithms are supported.

CMPREC_TYPE < ZLIB | RLE | USER >

The following compression types are currently supported:

ZLIB - General purpose compression from the zlib compression library.

Note: the ZLIB compression algorithm will impact performance. A Best Practice is to consider using RLE compression. It's quick and easy to test both ZLIB and RLE, but in most cases we have found RLE to be the better combination of reduced space and minimal performance overhead.

RLE - A fast proprietary run length encoding optimized for data that is null character, space or zero (0) padded (sparse data files).
USER - A user defined compression algorithm, requiring an associated .DLL (or shared object)

With USER compression, these additional keywords must be entered in the configuration file in the order shown below, and a DLL name is required.

CMPREC_TYPE < USER >

CMPREC_VERSION < a number >= 1 >

CMPREC_DLL < name of DLL >

See Also

COMPRESS_FILE (COMPRESS_FILE, compress file configuration)

COALESCE_TRNLOG

Attempt to combine the deleted space with adjacent deleted space that already exists in ctTRNLOG files that do not have a RECBYT index (default is OFF). Combining deleted space can reduce fragmentation.

To enable this behavior, either:

Add the server keyword COALESCE_TRNLOG ON

Set the global variable ctcoaltran = YES in a standalone compile.

COMPRESS_FILE

COMPRESS_FILE <filename>

Forces FairCom DB to create data files whose names match the specified name with data compression enabled. The file name may include wildcard characters (see FairCom DB Standard Wildcards).

A file whose name matches the keyword is created as a variable-length data file even if the create option specifies that it is a fixed-length data file. Such a file is still restricted to containing records that have the defined fixed-record length.

The logic has been changed so that the COMPRESS_FILE keyword does not enable data compression for a data file that is created with resource support disabled. This prevents a rebuild from failing with error 484 (could not open sort work file).

See Also

CMPREC_TYPE in the c-tree Server Administrator’s Guide.

FILE_CREATE_MODE

FILE_CREATE_MODE <mode>

On Unix systems, FairCom DB defaults to a permission mode of 0660 (read/write access for owner and group; no access for world) for the files it creates.

When using FairCom DB, the permission mode assigned to files can be set with the server configuration keyword FILE_CREATE_MODE to specify the desired file permission mode.

Example

;Set read and write permission for owner

;and no permission for group and world.

FILE_CREATE_MODE 0600

Note: On Unix systems, the system’s umask setting also affects the permission mode assigned to a file when it is created. If the umask setting is non-zero, the specified permissions are removed from the file’s permission mode.

FILE_HANDLES

This keyword is used on Unix system to change the number of file handles (at O/S level) that the O/S allows to be used by the FairCom Server.

No Default

FILE_PERMISSIONS

FILE_PERMISSIONS groupID#pmodeA#...#pmodeZ

Permits default file permissions to be assigned to one or more groups including two special groups: WORLD and OWNER. The primary need for this capability is to enforce permission flags on files that have already been created without a permission mask (i.e., the permission mask is zero at file create). A zero permission mask is equivalent to granting everyone all rights:

OPF_ALL | GPF_ALL | WPF_ALL

Note: ALL does not include the special NOPASS flag that permits a file to be opened for reading without supplying the file password. To grant NOPASS permission, it must be included explicitly.

groupID is the name of a user group or the special groups WORLD and OWNER. The server does NOT verify that the groupIDs actually exist.
pmode entries are symbolic names for the possible permission flags: READ, WRITE, DEF, DELETE, ALL, NOPASS and NONE. NONE should not be used with any other permission flags. It indicates no permissions are granted. Granting a permission of WRITE, DEF or DELETE is equivalent to granting all of the lesser permissions, thus DELETE and ALL are equivalent. For example, #READ#WRITE is equivalent to #WRITE. The groupID and pmode entries are case insensitive.

The WORLD entry applies to file opens by a user whose group(s) do not match any of the specified groups for those files without an explicit permission mask. If there is no WORLD entry, then WORLD permissions default to ALL. The OWNER entry applies to file opens by the users that created the files without an explicit permission masks.

Consider the following entries, and assume all the files in use did not have explicit permission masks at creation. Files with explicit permission masks (except for OPF_ALL | GPF_ALL | WPF_ALL) at creation are not affected by these FILE_PERMISSIONS entries.

FILE_PERMISSIONS OWNER#DEF

FILE_PERMISSIONS inventory#WRITE

FILE_PERMISSIONS ACCOUNTING#write#nopass

FILE_PERMISSIONS WORLD#NONE

In this example, the owner of a file will have READ, WRITE and DEF permissions. The owner of the file cannot delete the file. Members of the INVENTORY group have READ and WRITE permissions. Members of the ACCOUNTING group have READ and WRITE permissions and may open a file without its password (and receive READ permission only). A user who is not the owner of a file and not a member of the ACCOUNTING or INVENTORY groups will be assigned WORLD permissions, which in this case is NONE. NONE means the file cannot be opened.

If a user belongs to multiple groups, and two or more of its groups are specified with the FILE_PERMISSIONS keyword, the permissions granted to the user will be the union of the individual group permissions. However, if the user is the OWNER of the file, it receives OWNER permissions that default to ALL.

FILESYS_COMPRESS_FILE

FILESYS_COMPRESS_FILE <filename>

The FairCom Server configuration option FILESYS_COMPRESS_FILE can be used to enable file system compression on files whose names match the specified filename, which can include wildcard characters. The compression is enabled at the time that the file is created. This option only affects data and index files, not transaction logs or CTSTATUS.FCS.

Limitations

Currently, support for file system level compression is implemented only on Windows systems, using the built-in compression feature of the NTFS file system.

https://docs.microsoft.com/en-us/windows/win32/fileio/file-compression-and-decompression

Example

FILESYS_COMPRESS_FILE custmast.dat

FILESYS_COMPRESS_FILE custordr.dat

FILESYS_COMPRESS_FILE ordritem.dat

FILESYS_COMPRESS_FILE itemmast.dat

HUGE_TO_SEG_MB

HUGE_TO_SEG_MB <segment size in MBs> [#<maximum number of segments>]

Note: Segmented Tables have been deprecated.

Force any huge file to be created as a segmented file.

On systems that do not support files greater than 2 GB or 4 GB, FairCom DB can still support huge files by creating tables as segmented files. The size of each segment stays below the OS maximum file size limit, but the aggregate size exceeds the limit.

The maximum number of segments is optional and defaults to sixteen (16). For example, to specify a segment size of 1 GB, and a maximum of 8 segments for a total file size of up to 8 GB, an entry would look like

HUGE_TO_SEG_MB 1024#8

If the file has been created with a maximum size in the XCREblk structure (see parameters for the extended create file function), then the number of segments will be computed to accommodate the maximum size.

Note: If dynamic dumps are used, then it would be appropriate to use the !EXT_SIZE script option so that the dump stream file would also be broken into automatic segments.

INHERIT_FILE_PERMISSIONS

INHERIT_FILE_PERMISSIONS YES | NO

By default, FairCom DB file permissions are inherited from world to group to owner. For example, if the SECURITY() function is called with a mode of SEC_FILEMASK and a permission mask of WPF_READ, GPF_READ and OPF_READ permissions are turned on. However, note that if GPF_NONE is specified, the permissions are not passed on to the group (and to the owner).

Note: Changing this option does not affect inheritance of permissions on files whose permissions have already been set. It only affects the inheritance of permissions when they are set at file create time or by calling the SECURITY() function with a mode of SEC_FILEMASK.

Default: YES

KEEPOPEN_CLOSE_RETRY_LIMIT

KEEPOPEN_CLOSE_RETRY_LIMIT <limit>

Determines the number of times to retry the close operation before failing.

Default: 3

The default value for KEEPOPEN_CLOSE_RETRY_LIMIT is appropriate for most situations. You may want to consider increasing it if you experience errors 165.

See Also

KEEPOPEN_LIST

KEEPOPEN_FLUSH

KEEPOPEN_FLUSH YES | NO

When a data or index file is closed, the c-tree server will flush updated cache pages and set the update flag of the file to zero under the following conditions:

The file does not use the full transaction logging (ctTRNLOG) filemode.
No other users have the file opened.
The KEEPOPEN_LIST option is set to keep the file open.
The configuration option KEEPOPEN_FLUSH YES in ctsrvr.cfg is set..

Default: NO

KEEPOPEN_LIST

KEEPOPEN_LIST <file spec>

<file spec> can be a file name or a partial name including wildcard characters. See FairCom DB Standard Wildcards. Upon file creation or physical open, (1) if the file name matches a <file spec> and (2) if the file is a data file and (3) if the data file creation/open is part of an ISAM creation or open, then the KEEPOPEN attribute is attached to the data file and its indexes.

KEEPOPEN_LIST logic has internal checks to determine if the file open or create function returned an insufficient file control block error. If so, it checks if a KEEPOPEN file is available to close, closes it, and retries the open or create operation. The FairCom Server configuration option KEEPOPEN_CLOSE_RETRY_LIMIT determines the number of times these functions retry the operation before failing. It defaults to 3. The default value for KEEPOPEN_CLOSE_RETRY_LIMIT is appropriate for most situations. You may want to consider increasing it if you experience errors 165.

To specify multiple files, repeat the KEEPOPEN_LIST keyword for each file. You can specify KEEPOPEN_LIST as many times as you need. The only limitation is only one file per entry. For example:

KEEPOPEN_LIST custmast.dat

KEEPOPEN_LIST itemmast.dat

KEEPOPEN_LIST ordritem.dat

KEEPOPEN_LIST tell*.dat

Memory files have the option of staying open after all users have closed the file. The motivation for keeping memory files open is to avoid losing the contents of the file so that subsequent users can open the file and read and/or update the contents.

When a non-memory file is physically closed, c-tree removes the data cache and/or index buffer entries associated with the file. A file is physically closed when all users that have the file open close the file. For a non-memory file, keeping the file open even after all users have closed the file permits the associated cache/buffer entries to stay in memory. Then subsequent opens have the benefit of the cache contents.

ISAM data files and their associated indexes can also be designated as KEEPOPEN files. The motivation is to keep the files in the data cache and index buffers. It also eliminates a physical open when the next user opens the file. If all users have closed a KEEPOPEN file, then ctCLSNAM() can be called to close the data file and associated indexes.

Files to be treated in this manner are specified in the server configuration file with one or more entries of the form shown above.

Note: If a file is kept open by KEEPOPEN_LIST, a call to ctFILBLK does not block access to it. Calling CloseCtFileByName will close the file and it will remain closed until unblocked by a subsequent ctFILBLK call. (The FairCom DB Developer Guide explains the ctFILBLK and CloseCtFileByName functions.)

Behavior Change for V10.3.1 and Later

For a data or index file that does not use full transaction logging (ctTRNLOG), FairCom Server now flushes updated cache pages and sets the update flag to zero for the file when the last user closes the file and the KEEPOPEN_LIST option keeps the file open.

The configuration option KEEPOPEN_FLUSH NO can be used in ctsrvr.cfg to disable this flushing behavior and revert to the behavior prior to V10.3.1, although this is not recommended.

This is a change from previous behavior: When FairCom Server's KEEPOPEN_LIST configuration option was in effect for a c‑tree data or index file that was not under transaction control, FairCom Server kept that file open after the last user closed the file. Updated data and index cache pages remained and were not written to disk before the call to close the file returned to the caller. This caused unnecessary data integrity risk should the FairCom Server abnormally terminate while the file remained open. The file is likely to fail to open with an FCRP_ERR error, 14, under such scenarios.

See Also

KEEPOPEN_CLOSE_RETRY_LIMIT

LEAF_NODE_READ_LOCK

MATCHING_SEGMENT

MATCHING_SEGMENT <RENAME | DELETE | OVERWRITE | ERROR>

The segmented file logic behaved as follows when an attempt to create a new file segment encountered an existing segment with the same name:

If the existing segment matched the host file’s file ID and other attributes, then the existing segment was simply overwritten.
If the file ID did not match, then an error was returned and no more records could be added to the file.

Overwriting a matching segment could cause a problem because the existing data is simply left in place until new records overwrite the data.

In addition to different defaults, the behavior for a matching segment (i.e., segment’s file ID matches the host file’s unique ID), is quite different than for a non-matching segment. For a matching segment, if RENAME is in effect and the rename fails, then a delete is attempted. If a delete fails, then the segment is overwritten. The error option must be explicitly requested. For a non-matching segment, if either rename or delete are in effect and they fail, then an error occurs (and overwrite is not an option).

CTSTATUS.FCS entries are made when one of these unexpected segments are encountered. When a rename occurs, the segment’s name is modified by adding the hexadecimal representation of the system’s time in seconds to the end of the file name. For example,

mydata.s04

might become

mydata.s04.4465f728

where 0x4465f728 is the system time in seconds from some arbitrary starting point.

Default: RENAME

See Also

NONMATCHING_SEGMENT

MAX_FILES_PER_USER

Client file information, on both client and server side, is automatically resized when:

A file open or create uses a file number beyond the existing client file range.
A new file number is assigned with automatic file number assignment (for example, OpenFileWithResource() with a -1 filno).

A configurable limit on files per user is enforced when this auto resizing comes into effect. An open/create which returns a FINC_ERR (604) implies that the create succeeded on the FairCom Server, but the client could not allocate memory for the local file info, and the newly created file has been closed.

Stand-alone applications support automatic resizing of file control information up to the limit imposed by ctMAXFIL, which defaults in ctoptn.h to 110. If ctMAXFIL is not defined, the limit defaults to 1024 files/FCBs.

Note: If a filno beyond the existing file range causes a resizing, the new number of files supported goes from 0 to filno + MAXMEMB + 1, with FCBs allocated for all potential file numbers in the range. Use automatic file number assignment for maximum memory efficiency.

For example, if InitISAM() requests 100 files and OpenCtFile() uses file number 1000, resizing changes the number of files supported to 1000 + MAXMEMB + 1, or 1032. All the files between 100 and 1000 are now available. By contrast, if an automatic file number assignment causes resizing, the file number range is only extended by MAXMEMB + 1. If InitISAM() requests 100 files and OpenFileWithResource(-1,...) causes resizing, the number of files supported would increase to 132.

Default: The default value of MAX_FILES_PER_USER is the same as the server's maximum supported FILES value (which is update 1 million open files). You can use the MAX_FILES_PER_USER option to set a connection's maximum number of open files to a smaller value.

See Also

FILES

MAXMEMB

In This Section

Maximum Index Members per File (MAXMEMB)

The default for the MAXMEMB (the number of index members per single index file) has been updated to 127 allowing a larger number of segments per index.

We now define ctMAXMEMB in ctopt1.h instead of in ctopt2.h, because ctport.h references ctMAXMEMB and it is included before ctopt2.h.

We also define MAXMEMB to ctMAXMEMB in ctopt1.h.

MAX_KEEPOPEN_FILES

MAX_KEEPOPEN_FILES <maximum # of KEEPOPEN files>

Sets a limit on the number of files that can be kept in the KEEPOPEN state (meaning the file has been closed by all connections, but the server is keeping the file open for reuse).

This prevents potential negative performance impacts from keeping too many files in the KEEPOPEN state.

Default: 1000

MAX_REBUILD_QUEUE

MAX_REBUILD_QUEUE <max_queue_size>

FairCom Server supports an option to use a worker thread to sort the keys read from the data file. The rebuild thread reads records from the data file and constructs key values, which it passes to the sort thread using an inter-thread queue. After all the key values have been passed to the sort thread, the sort thread sorts the key values and passes them back to the rebuild thread in a second queue.

This option is enabled by specifying MAX_REBUILD_QUEUE <max_queue_size> in ctsrvr.cfg with a positive value. The value is a number in bytes indicating the maximum size of the queue used to pass the keys to the worker thread. MAX_REBUILD_QUEUE defaults to 100 MB. A 64-bit server has a limit of 4 TB - 1 for this setting, and a 32-bit server has a limit of 4 GB - 1 for this setting.

For example, in one test comparing V10 rebuild performance with its defaults (16,000 bytes of SORT_MEMORY and no MAX_REBUILD_QUEUE support) to this optimized rebuild available in V11, rebuilding a 17 GB data file completed in an average of 1,816 seconds, and the optimized rebuild completed in an average of 623 seconds, which is 66% faster (about 1/3 the time of the original rebuild).

<maximum_queue_size> is interpreted as bytes by default, or it can be specified with a suffix of KB, MB, or GB (e.g., 10 MB).

MAX_VIRTUAL_FILES

MAX_VIRTUAL_FILES <Maximum files>

An integer argument specifying the maximum number of virtual files that may be opened at one time.

A <Maximum files> value of -1 forces all files as ctPERMANENT.

Default: 500

MEMFILE_MAX_BINS

MEMFILE_MAX_BINS <bins>

The maximum size of a memory file determines the number of hash bins that FairCom DB allocates for the memory file. However, there is a hard limit on the number of hash bins for a memory file. Previously, this limit was set at compile time to 65536. This configuration option is used to set the maximum number of hash bins allowed for a memory file at run time. This may be necessary for efficient access to very large memory files.

Default: 65537

MEMORY_FILE

The FairCom Server supports creating memory files using a server configuration keyword. This feature allows developers to create memory files using their existing application code, provided that the file is created using an Xtd8 create function such as CreateIFileXtd8(). To create a memory file using the server configuration keyword, specify one or more entries of the form:

MEMORY_FILE <file name>#<max size>

where the file name may include wildcard characters (see FairCom DB Standard Wildcards) and the maximum size is optional. If no maximum size is specified, then 4GB is used. If a file is being created and matches one of the MEMORY_FILE file name entries, then it will be created as a memory file unless it is a superfile host, superfile member, mirrored, segmented or partitioned file.

To cause all possible files to be created as memory files, add the following configuration entry:

MEMORY_FILE *

The MEMORY_FILE keyword is useful to quickly test how a file or set of files will behave as memory files.

NONMATCHING_SEGMENT

NONMATCHING_SEGMENT <RENAME | DELETE | ERROR>

The segmented file logic behaved as follows when an attempt to create a new file segment encountered an existing segment with the same name:

If the existing segment matched the host file’s file ID and other attributes, then the existing segment was simply overwritten.
If the file ID did not match, then an error was returned and no more records could be added to the file.

Overwriting a matching segment could cause a problem because the existing data is simply left in place until new records overwrite the data.

In addition to different defaults, the behavior for a matching segment (i.e., segment’s file ID matches the host file’s unique ID), is quite different than for a non matching segment. For a matching segment, if RENAME is in effect and the rename fails, then a delete is attempted. If a delete fails, then the segment is overwritten. The error option must be explicitly requested. For a non matching segment, if either rename or delete are in effect and they fail, then an error occurs (and overwrite is not an option).

mydata.s04

might become

mydata.s04.4465f728

where 0x4465f728 is the system time in seconds from some arbitrary starting point.

Default: ERROR

See Also

MATCHING_SEGMENT

RECYCLE_BIN

The recycle bin is configured with SUBSYSTEM CTFILE RECYCLE_BIN . This subsystem configuration option supports the following options:

recycle_folder_name: directory name where the recycle bin is stored and can be absolute or relative. If relative, the directory is relative to the server's LOCAL_DIRECTORY configured directory or current working directory if LOCAL_DIRECTORY is not used.
purge_frequency_minutes: time in minutes between checks for files that should be purged from the recycle bin.
max_file_age_days: maximum number of days a file is kept in the recycle bin.
max_folder_storage_size: maximum total byte size of the files in the recycle bin. If deleting a file causes the recycle bin to go over the limit, files are deleted until the size is below the limit.

Example

subsystem ctfile recycle_bin {

recycle_folder_name deleted_files

purge_frequency_minutes 60

max_file_age_days 30

max_folder_storage_size 1 gb

}

MONITORING

The statistic utility, ctstat supports a deleted_files option listing the total size and detail list of files currently in the recycle bin.

Example

ctstat -deleted_files -u ADMIN -p ADMIN -s FAIRCOMS -i 1 -h 1 -t

Thu Dec 9 15:01:48 2021

Total file size: 917500 for 4 file(s)

Deleted file of size 131071 deleted\1639081086.20211209\qatranrepl097.sup.992

Deleted file of size 131071 deleted\1639081086.20211209\qatranrepl097.sup.998

Deleted file of size 327679 deleted\1639081086.20211209\qatranrepl097.sup.1017

Deleted file of size 327679 deleted\1639081086.20211209\qatranrepl097.sup.1026

NEW SERVER FILES

Two new files are required for tracking inactive files in the recycle bin. RECBINDT.FCS and RECBINIX.FCS are transaction-controlled data and index files retaining data about the current recycle bin state. The data file contains a state record for each deleted file, and a state record that holds the total space in use by deleted files. The index enables the background thread to find files that were deleted from the oldest to newest timestamp.

REPLICATION AGENT CHANGES

The replication agent supports a configuration option to enable whether the agent uses the recycle bin any time it deletes a file on the target server. The ctreplagent.cfg option is delete_to_recycle_bin yes | no and is on by default.

SPLIT_NBR_OF_FILES

SPLIT_NBR_OF_FILES <# of files>

When using the partitioning file rule:

<file_part_ind> = ( <int_unique_key> % <nbr_of_files> ) + 1

<nbr_of_files> is configured by the SPLIT_NBR_OF_FILES configuration entry.

Note: Partitioning is currently only supported with a custom build containing the partition rule. Please contact your nearest FairCom office for availability.

TMPNAME_PATH

TMPNAME_PATH <path>

The TMPNAME_PATH location becomes the default directory for temporary files. On startup, if a TMPNAME_PATH entry is encountered in ctsrvr.cfg, the FairCom Server tests the validity of the path. If there is an error, the FairCom Server terminates. Whether or not successful, the FairCom Server enters the path name in the CTSTATUS.FCS file.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: Current server directory

DIAGNOSTICS PTADMIN

When a partition member purge fails with error 718, this enables the filename and list of open instances of that file to be logged to CTSTATUS.FCS. Below is an example. For each connection we list the task ID, user name, node name, and user file number.

Mon Dec 12 12:40:33 2011

- User# 00012 PT_ADMIN: purge failed, partition .\ctreesql.dbs\admin_pt.20111129.015307.dat is open (2 open instances):

Mon Dec 12 12:40:39 2011

- User# 00012 PT_ADMIN: Connection 16: ADMIN(SQL:CTREESQL) 64

Mon Dec 12 12:40:44 2011

- User# 00012 PT_ADMIN: Connection 17: ADMIN(SQL:CTREESQL) 45

A process stack is also created when this occurs. This option can also be enabled via SETCONFIG() and through the ctadmn utility.

Locking

AUTO_LOCK_RETRY

Enables automatic retries of data record locks when a c-tree function call fails with a data record lock error DLOK_ERR.

AUTO_LOCK_RETRY_SLEEP

Set the sleep time between retries.

BLOCKING_LOCK_TIMEOUT_SEC

Avoids excessively long blocking lock waits by returning error UTIM_OUT (827) to the caller of the lock request.

CHECKLOCK_FILE (CHECKLOCK_FILE, CHECKLOCK_FILE)

Enable or disable ctCHECKLOCK mode for specified data files.

COMPATIBILITY LKISAM_MODES

Disables error if the lokmod parameter includes any modifier bits intended for FairCom use only.

ITIM_RETRY_DEFER

Introduces a delay before internally retrying an ISAM record read operation when an ITIM_ERR (160) is encountered.

ITIM_RETRY_LIMIT

Enables retrying the operation the specified number of times before returning ITIM_ERR (160) if an index is out-of-sync with the data temporarily while another user is performing an update.

Diagnostics

DIAGNOSTICS DLOK_ERR

Enables logging of DLOK_ERR (42) lock error information to CTSTATUS.FCS.

DIAGNOSTICS LOCK_DUMP

Enable the use of the LockDump() function for non ADMIN users.

AUTO_CHECKLOCK

CHECKLOCK_FILE <YES/NO>

Available in version 13.0.3 onwards:

If YES (default), ctCHECKLOCK is enabled on all files.

If NO, ctCHECKLOCK is not enabled (which is the behavior before 13.0.3).

You can enable or disable this feature for individual files using CHECKLOCK_FILE.

Breaking Change: As of version 13.0.3, the server prevents a record from being updated if it is not locked. This has always been the intended behavior, but previous versions did not enforce it. Use AUTO_CHECKLOCK NO to revert to the previous behavior.

AUTO_LOCK_RETRY

AUTO_LOCK_RETRY <retries>

Enables automatic retries of data record locks when a c-tree function call fails with a data record lock error DLOK_ERR.

Default: 0 (retries).

AUTO_LOCK_RETRY_SLEEP

AUTO_LOCK_RETRY_SLEEP <milliseconds>

Set the sleep time between retries.

Default: 100 ms

BLOCKING_LOCK_TIMEOUT_SEC

BLOCKING_LOCK_TIMEOUT_SEC <timeoutSEC>

BLOCKING_LOCK_TIMEOUT_SEC avoids excessively long blocking lock waits. When used, this feature returns error UTIM_OUT (827) to the caller of the lock request. The function ctLOKTIMOUT() is used to set, change and clear this timeout feature.

timeoutSEC is specified in seconds, and determines the amount of time to wait before returning the UTIM_OUT error.

This configuration entry is equivalent to a member of the ADMIN group making the call ctLOKTIMOUT(-1, ctLTOallusers | ctLTOdiagnostic, timeoutSEC).

Note: The effect of the configuration entry can be turned off by a call of the form ctLOKTIMOUT(-1, ctLTOallusers, 0).

If a user calls ctLOKTIMOUT() with a datno equal to -1 in order to set a timeout value on all data files for the user, the user can selectively change or turn off the timeout by making additional calls to ctLOKTIMOUT() specifying the data file number.

Lock Statistics

Locking statistics have an inconsistency when a lock request is removed from a list of waiting lock requests. When a lock request times out with this new feature, it is removed from the wait list. For instance, if a thread is waiting for a lock and it is killed by ctadmn, the lock is removed from the waiting list, however, the lock statistics do not reflect this. In fact, the count of currently blocked locks will be off (too high) by one for each lock request removed from a wait list. A new lock statistic has been added to account explicitly for lock requests that have been removed from the wait list: “killed.” It is treated in the same manner as the deadlock category.

CHECKLOCK_FILE

Enables or disables ctCHECKLOCK mode for specified data files. This file mode can be invoked a number of ways. A file can be created with this set or it can be opened with this set. In V11.5 and later, you can use this server keyword to enable it dynamically.

Note: In V13.0.3 onwards, we enable ctCHECKLOCK on all files by default, since AUTO_CHECKLOCK is set to YES, overriding CHECKLOCK_FILE. You can change this behavior by setting AUTO_CHECKLOCK to NO.

When enabled, the server will block all updates to the file without a write lock, returning an error 57. This allows you to debug situations where two processes might attempt to update the same record in an overlapping manner. This is primarily used in non-transaction files, as transaction-controlled files always require a write lock for updates. You can use the wildcard character "*" to specify multiple files.

The following FairCom Server configuration option enables ctCHECKLOCK mode for data files whose names match filespec:

CHECKLOCK_FILE +filespec

where filespec is the name of a file, which can include wildcards, as described in FairCom DB Standard Wildcards. filespec needs to match the actual file name including extension.

This option takes effect immediately: all currently open and subsequently opened or created data files whose name match filespec have ctCHECKLOCK enabled.

The following FairCom Server configuration option is used to undo the effect of the CHECKLOCK_FILE +filespec:

CHECKLOCK_FILE -filespec

This option disables the ctCHECKLOCK mode for data files whose name match filespec (which can include wildcards) and whose ctCHECKLOCK option was turned on by the ctCHECKLOCK +filespec.

This option takes effect immediately: all currently open data files whose names match filespec have ctCHECKLOCK disabled, and subsequently opened or created data files whose names match filespec will not have ctCHECKLOCK enabled (unless another CHECKLOCK option has been specified that matches the data file name, or the file mode at file open or create time specifies the ctCHECKLOCK mode).

This feature is also available at runtime via the ctSETCFG() API call.

Default: Off

COMPATIBILITY LKISAM_MODES

LockISAM() will return BMOD_ERR if the lokmod parameter includes any modifier bits that are intended for FairCom use only.

In V11.0 and later, adding COMPATIBILITY LKISAM_MODES to the ctsrvr.cfg file will disable this.

Default: Off

Do not use this keyword unless instructed to by FairCom.

ITIM_RETRY_DEFER

ITIM_RETRY_DEFER <defer_time>

Introduces a delay before internally retrying an ISAM record read operation when an ITIM_ERR (160) is encountered. <defer_time> specifies the time in milliseconds for which the server sleeps a thread that encounters error ITIM_ERR during an ISAM record read operation before retrying the ISAM record read operation. As before, the maximum number of ITIM_ERR retries for a particular ISAM record read operation is determined by the ITIM_RETRY_LIMIT configuration option.

ITIM_RETRY_DEFER -1 disables the delay between ITIM_ERR retries.

Comments

If you frequently encounter ISAM record read operations failing with error ITIM_ERR, consider what the error code is revealing about your application’s design: if you properly use record locking, ITIM_ERR can still occur if many users are updating the same records in sequence (that is, each of a number of threads reads the record with a lock, updates the record--changing the key value--and unlocks the record). In this case, increasing the ITIM_RETRY_LIMIT and ITIM_RETRY_DEFER settings can help avoid the ITIM_ERR errors, however, this may be at the expense of application performance, by introducing additional retries and delays between retries. In such a situation, consider ways to change the application to reduce the number of users that are attempting to update the same records at the same time.

Default: -1

See Also

ITIM_RETRY_LIMIT

ITIM_RETRY_LIMIT <# of retries before returning ITIM_ERR>

In a multi-user environment, it is possible that an index may become out of sync with the data temporarily while another user is performing an update operation. Typically, error ITIM_ERR (160) is returned under these circumstances. It may be practical in some situations to retry the operation as the index is eventually updated. To accomplish this, the ITIM_ERR retry limit is configurable.

In addition, SNAPSHOT output now contains the value of the limit and the number of failed retry loops. Before this change, only the number of retries was reported by the SNAPSHOT.

Default: Typically 10 or 20

See Also

ITIM_RETRY_DEFER

DIAGNOSTICS DLOK_ERR

Enables logging of DLOK_ERR (42) lock error information to CTSTATUS.FCS.

The log entry shows the filename, data record offset, and the lock owner.

Example output:

Mon Jun 05 15:18:42 2006

- User# 00012 DLOK_ERR: file=vcusti offset=0x0000-00002899 owner=11

DIAGNOSTICS LOCK_DUMP

Enable the use of the LockDump() function for non ADMIN users. The ADMIN user account can call the LockDump() function without this keyword enabled.

Default: Disabled

I/O Control

COMPATIBILITY DIRECT_IO

Reverts to the previous behavior of using direct I/O instead of O_DSYNC synchronous writes (or O_SYNC if O_DSYNC is not defined) for transaction logs on Solaris.

COMPATIBILITY FDATASYNC

For Unix systems, enables the optional use of fdatasync() instead of fsync().

COMPATIBILITY FORCE_WRITETHRU

Forces the automatic addition of the WRITETHRU mode to all files opened without the TRNLOG file mode.

COMPATIBILITY PREV610A_FLUSH

Provides a balance between update performance and recoverability of data in the event of an abnormal FairCom Server termination affecting non-transaction files.

COMPATIBILITY WTHRU_UPDFLG

Disables the ‘update’ flag on files with the WRITETHRU file mode.

DEFAULT_CHANNELS

Changes the number of I/O channels assigned to a file with ctDUPCHANEL in its file mode at open, unless the file is in the SET_FILE_CHANNELS list.

IDLE_NONTRANFLUSH and IDLE_TRANFLUSH

Deprecated: Sets the interval FairCom DB waits before checking to see if the server is idle before flushing data and index caches during idle time. These keywords have been replaced by the following:

IO_ERROR_BLOCK_RETRY

Specifies the maximum number of failed IO_ERROR_BLOCK_SIZE-sized I/O operations that must occur before the I/O operation is considered to have failed.

IO_ERROR_BLOCK_SIZE

Causes a read or write operation that fails with Windows system error 1450 (ERROR_NO_SYSTEM_RESOURCES) to be retried in blocks of the specified size.

IO_ERROR_BLOCK_SLEEP

Specifies a time in milliseconds between IO_ERROR_BLOCK_RETRY retry attempts.

SET_FILE_CHANNELS

Permits the number of I/O channels to be explicitly set for the named file regardless of whether the file mode, at open, includes ctDUPCHANEL.

UNBUFFERED_IO

Enables unbuffered I/O for the specified file on Windows systems.

Diagnostics

DIAGNOSTICS DIRECT_IO

Enables a check on each write operation for a file for which direct I/O is requested that the properties of the write operation meet the direct I/O requirements.

DIAGNOSTICS LOWL_FILE_IO (DIAGNOSTICS LOWL_FILE_IO, DIAGNOSTICS LOWL_FILE_IO)

Logs low-level system errors into the server status file, CTSTATUS.FCS.

COMPATIBILITY DIRECT_IO

The FairCom Server for Solaris uses O_DSYNC synchronous writes for the transaction logs when the COMPATIBILITY SYNC_LOG or COMPATIBILITY LOG_WRITETHRU configuration options are specified in the server configuration file.

Furthermore, it is expected the O_DSYNC writes are more efficient than O_SYNC writes. The FairCom Server uses O_DSYNC on systems that define this file open mode and uses O_SYNC on systems that do not define the O_DSYNC file open mode.

COMPATIBILITY DIRECT_IO, is available to revert to the previous behavior of direct I/O.

When the FairCom Server starts up, it writes either the message "Transaction logs using direct I/O." or "Transaction logs using synchronous I/O." to CTSTATUS.FCS to indicate which method of synchronous writes is in use. If neither COMPATIBILITY SYNC_LOG nor COMPATIBILITY LOG_WRITETHRU is specified in ctsrvr.cfg, neither of these messages is written to CTSTATUS.FCS as in that case the server performs asynchronous writes that are flushed by a separate call.

Default: OFF

Also See

COMPATIBILITY LOG_WRITETHRU

COMPATIBILITY SYNC_LOG

COMPATIBILITY FDATASYNC

For Unix systems, enables the optional use of fdatasync() instead of fsync().

COMPATIBILITY FORCE_WRITETHRU

COMPATIBILITY FORCE_WRITETHRU enables the WRITETHRU filemode for all non-transaction files (including PREIMG files, which are transaction controlled but do not have a transaction log).

Default:

This option is off by default. Be sure to check your FairCom Server configuration file, as it might be enabled there.

Note: The performance impact of the WRITETHRU filemode depends on whether COMPATIBILITY PREV610A_FLUSH is enabled. See COMPATIBILITY PREV610A_FLUSH for details.

COMPATIBILITY PREV610A_FLUSH

COMPATIBILITY PREV610A_FLUSH suppresses the flushing of updated file system cache pages to disk when a non-transaction WRITETHRU file is updated. See COMPATIBILITY FORCE_WRITETHRU.

Default:

This option is off by default. Be sure to check your FairCom Server configuration file, as it might be enabled there.

The performance impact of WRITETHRU depends on whether COMPATIBILITY PREV610A_FLUSH is enabled:

When COMPATIBILITY PREV610A_FLUSH is NOT enabled, each update to a non-transaction WRITETHRU file is flushed to disk. Both c-tree and operating system (OS) caches are flushed. This mode, while extremely safe, can significantly slow down performance of updates to non-transaction WRITETHRU files.
When COMPATIBILITY PREV610A_FLUSH is enabled, each update to a non-transaction WRITETHRU file is flushed to the OS file system cache, but the updated pages in the OS file system cache are not flushed at the time of the update. This mode performs much faster than when each update flushes the file system cache updates to disk, but note that the contents of updated file system cache pages can be lost if the system loses power before the operating system has written the updated file system cache pages to disk. Installing a power supply on the system can reduce the likelihood of losing data in the event that a system power loss occurs by permitting time for the operating system to write updated file system cache pages to disk.

COMPATIBILITY WTHRU_UPDFLG

Disables the ‘update’ flag on files with the WRITETHRU file mode. If COMPATIBILITY WTHRU_UPDFLG is not in its configuration file and if non-transaction files are opened without WRITETHRU in the file mode, a warning is issued in CTSTATUS.FCS concerning the vulnerability to FCRP_ERR (14).

Default: Not present

DEFAULT_CHANNELS

DEFAULT_CHANNELS <nbr of I/O channels>

DEFAULT_CHANNELS changes the number of I/O channels assigned to a file with ctDUPCHANEL in its file mode at open, unless the file is in the SET_FILE_CHANNELS list. The default number of channels is not limited by the NUMCHANEL value.

Note: Multiple I/O channels are disabled for newly created files. The multiple I/O channels take affect only on an open file call. Also, depending on the default number of I/O channels, a superfile host not in the SET_FILE_CHANNELS will use no more than 2 * NUMCHANEL I/O channels.

See Also

SET_FILE_CHANNELS

IDLE_NONTRANFLUSH and IDLE_TRANFLUSH

Note: As of FairCom DB V11, these keywords have been replaced by the following keywords:
TRAN_DATA_FLUSH_SEC
NONTRAN_DATA_FLUSH_SEC
TRAN_INDEX_FLUSH_SEC
NONTRAN_INDEX_FLUSH_SEC

IDLE_TRANFLUSH <idle check interval seconds>

IDLE_NONTRANFLUSH <idle check interval seconds>

FairCom DB flushes data and index caches during idle time, launching two idle thread processes at start-up: One thread flushes transaction-file buffers and the other flushes non-transaction-file buffers. The threads wake-up periodically and check if the FairCom Server is idle to begin flushing. Subsequent activity terminates the flushes. Low priority background threads, such as the delete node thread, do not affect the idle state, however, FairCom DB clients and transaction checkpoints modify the idle status.

Filesystem Flush Performance

Starting with V8 the default behavior of the idle flush threads was changed to skip internal filesystem flush calls for better performance when performing the idle flush operation. Flushing the filesystem buffers guarantees data is secured to disk at a performance cost. To revert this behavior, add the #SAVE option after the checkpoint time which will ensure data safety in nearly all cases. Note that flushing filesystem buffers will cause measurable delays when large caches are in use, and you may notice transaction slowdowns during this period.

IDLE_TRANFLUSH <idle check interval in seconds>#SAVE

IDLE_NONTRANFLUSH <idle check interval in seconds>#SAVE

The default interval is 15 seconds. Setting the interval to zero or a negative value disables the thread.

Default: 15 seconds

See Also

IO_BLOCK_SIZE

IO_BLOCK_SIZE <size>

Splits disk read and write operations larger than the specified size into individual operations of the specified size. For example, specifying IO_BLOCK_SIZE 16 KB causes a 1 MB write request to be performed as 64 16 KB write operations.

See Also

IO_ERROR_BLOCK_RETRY

IO_ERROR_BLOCK_RETRY <retries>

Specifies the maximum number of failed IO_ERROR_BLOCK_SIZE-sized I/O operations that must occur before the I/O operation is considered to have failed. If the IO_ERROR_BLOCK_SIZE-sized I/O operations that are being attempted for a particular I/O operation fail more than <retries> times, the FairCom Server writes a READ_ERR (36) or WRITE_ERR (37) message to CTSTATUS.FCS and considers the I/O operation to have failed.

A value of -1 signifies infinite retries. The default is 0, which means that the I/O operation is tried only once in IO_ERROR_BLOCK_SIZE-sized blocks, and if any of these I/O operations fails, the entire I/O operation is considered to have failed. As another example, if IO_ERROR_BLOCK_RETRY is set to 20 and IO_ERROR_BLOCK_SIZE is set to 65536, if a 327680-byte write is retried as 5 65536-byte write operations, then the I/O operation fails if there are 20 failures to perform those 5 write operations.

See Also

IO_ERROR_BLOCK_SIZE

IO_ERROR_BLOCK_SIZE <size>

When the Windows kernel has allocated all of its paged-pool memory, it will not be able to perform many tasks and instead returns a STATUS_INSUFFICIENT_RESOURCES (0xC000009A) message. This is a restriction of 32-bit addressing (only 2GB addressable within the kernel), regardless of the amount of memory available in the system.

When the FairCom Server configuration option IO_ERROR_BLOCK_SIZE option is specified in the FairCom Server configuration file, a read or write operation that fails with Windows system error 1450 (ERROR_NO_SYSTEM_RESOURCES) is retried in blocks of the specified size. If any one of those read or write operations fails, the FairCom Server fails the read or write operation.

See Also

IO_ERROR_BLOCK_SLEEP

IO_ERROR_BLOCK_SLEEP <time>

Specifies a time in milliseconds between IO_ERROR_BLOCK_RETRY retry attempts. The default is zero, which means that retries are attempted immediately.

See Also

NONTRAN_DATA_FLUSH_SEC

NONTRAN_DATA_FLUSH_SEC <time_limit_in_seconds>

Sets the time limit in seconds that a data cache page can remain dirty before it is written to the file system cache.

Specify IMMEDIATE to cause dirty pages to be written immediately.
Specify OFF to disable time limit-based flushing.

Default: IMMEDIATE

See Also

Controls for Performance AND Safety of Non-Transaction Updates

NONTRAN_INDEX_FLUSH_SEC

NONTRAN_INDEX_FLUSH_SEC <time_limit_in_seconds>

Sets the time limit in seconds that an index buffer can remain dirty before it is written to the file system cache.

Specify IMMEDIATE to cause dirty pages to be written immediately.
Specify OFF to disable time limit-based flushing.

Default: IMMEDIATE

See Also

Controls for Performance AND Safety of Non-Transaction Updates

OPEN_FILES_ALERT_THRESHOLD

OPEN_FILES_ALERT_THRESHOLD <files>

Default: 0

FairCom Server now supports logging a diagnostic message to CTSTATUS.FCS whenever its total number of open files exceeds the specified threshold.

<files> is the number of open files at which FairCom Server should start logging messages to CTSTATUS.FCS. This option defaults to zero (no logging).

Once the limit is reached, a message is logged each time the current number of open files exceeds the previous high water mark. So for example, in the following situation:

OPEN_FILES_ALERT_THRESHOLD 100

Current open files = 99

1. Open a file: current open files = 100: message logged

2. Open a file: current open files = 101: message logged

3. Open a file: current open files = 102: message logged

4. Close a file: current open files = 101

5. Open a file: current open files = 102: no message logged because already logged 102

This option can be changed at runtime via ctadmn.

OPTIMIZE_FILE_OPEN

OPTIMIZE_FILE_OPEN YES | NO

Default: YES

Enables file open optimizations for improved performance and scalability. Currently the options can only be set in the configuration file. They cannot be changed at run time.

The following messages in CTSTATUS.FCS indicate that these optimizations are on:

File open optimization is enabled.

The following messages in CTSTATUS.FCS indicate that these optimizations are off:

File open optimization is off.

See Also

OPTIMIZE_FILE_CLOSE

OPTIMIZE_FILE_CLOSE YES | NO

Default: YES

Enables file close optimizations for improved performance and scalability. Currently the options can only be set in the configuration file. They cannot be changed at run time.

The following messages in CTSTATUS.FCS indicate that these optimizations are on:

File close optimization is enabled.

The following messages in CTSTATUS.FCS indicate that these optimizations are off:

File close optimization is off.

See Also

PENDING_FILE_OPEN_RETRY_LIMIT

PENDING_FILE_OPEN_RETRY_LIMIT <limit>

Default: 0 (wait forever) (before V12, 100000)

Sets the pending open retry limit. Normally, it is not expected a file open or create to exceed the pending file open retry limit, but this limit is provided to ensure the calling function returns an error after a limited number of retries in the unexpected case that a file remains in a pending open state for a long period of time.

It is possible to change the PENDING_FILE_OPEN_RETRY_LIMIT server setting at runtime in the usual ways (ctadmn and the ctSETCFG() API function).

See Also

SET_FILE_CHANNELS

SET_FILE_CHANNELS <file name>#<nbr of I/O channels>

Without this feature, the ctDUPCHANEL file mode bit enables a file to use NUMCHANEL simultaneous I/O channels, where NUMCHANEL is set at compile time, and is by default, set to two (2). For superfile hosts with ctDUPCHANEL, 2* NUMCHANEL I/O channels are established. The default is not to turn on ctFeatCHANNELS.

SET_FILE_CHANNELS permits the number of I/O channels to be explicitly set for the named file regardless of whether the file mode, at open, includes ctDUPCHANEL. A value of one (1) for the number of I/O channels effectively disables ctDUPCHANEL for the file. A value greater than one (1) turns on DUPCHANEL and determines the number of I/O channels used. The number of I/O channels is not limited by the compile-time NUMCHANEL value. You may have as many SET_FILE_CHANNELS entries as needed.

The <file name> can be a wildcard specification using a ‘?’ for a single character and a ‘*’ for zero or more characters. See FairCom DB Standard Wildcards.

See Also

DEFAULT_CHANNELS

SYSTEM_FILE_ID_LIST

SYSTEM_FILE_ID_LIST YES | NO

Default: YES

Enables the system file ID list, which is used to detect files being opened using different aliases. It can be turned on or off at runtime when the server is in a quiesced state.

See Also

Enables logging of adds/deletes to the system file ID list to the file SYSIDHASH.FCS in the server's LOCAL_DIRECTORY directory. Can be turned on or off at runtime.

See Also

COMPATIBILITY DISABLE_BACKGROUND_CLEANUP

TRAN_DATA_FLUSH_SEC

TRAN_DATA_FLUSH_SEC <time_limit_in_seconds>

Sets the time limit in seconds that a data cache page can remain dirty before it is written to the file system cache.

Specify IMMEDIATE to cause dirty pages to be written immediately.
Specify OFF or -1 to disable the time limit-based flushing.
In V11.4 and later, a rate limiting parameter can be specified as follows: "rate=<msec> / <number of flushes>"
For example, to defer for 1 millisecond after every two flushes of updated TRNLOG data cache pages, use this option:
TRAN_DATA_FLUSH_SEC rate=1/2

System snapshot 19 and later includes fields referencing background flush state structure, BGFLSS, which is in the system snapshot structure, ctGSMS. When a background flush thread is set to flush at a particular rate, the text snapshot shows the rate. For example:

time limit for dirty non-transaction data pages : rate=1/1

This option can be changed using the ctSETCFG() API function or the ctadmn utility.

Note: Enabling this option disables checkpoint flush operations.

Default: -1 (disabled)

See Also

CHECKPOINT_FLUSH

TRAN_INDEX_FLUSH_SEC

NONTRAN_DATA_FLUSH_SEC

NONTRAN_INDEX_FLUSH_SEC

TRAN_INDEX_FLUSH_SEC

TRAN_INDEX_FLUSH_SEC <time_limit_in_seconds>

Sets the time limit in seconds that an index buffer can remain dirty before it is written to the file system cache.

Also, attempts to do routine optimization of index nodes as they are flushed to disk by background index flush. This is intended to improve overall index lookup performance and distinct key estimate accuracy.

Specify IMMEDIATE to cause dirty pages to be written immediately.
Specify OFF or -1 to disable the time limit-based flushing.
In V11.4 and later, a rate limiting parameter can be specified as follows: "rate=<msec> / <number of flushes>"
For example, to defer for 1 millisecond after every two flushes of updated TRNLOG data cache pages, use this option:
TRAN_DATA_FLUSH_SEC rate=1/2

time limit for dirty non-transaction data pages : rate=1/1

This option can be changed using the ctSETCFG() API function or the ctadmn utility.

Note: Enabling this option disables checkpoint flush operations.

Default: -1 (disabled)

See Also

CHECKPOINT_FLUSH

TRAN_INDEX_FLUSH_SEC

NONTRAN_DATA_FLUSH_SEC

NONTRAN_INDEX_FLUSH_SEC

UNBUFFERED_IO

UNBUFFERED_IO <filename>

FairCom Server supports the use of unbuffered disk I/O operations on a per-file basis. Unbuffered I/O bypasses the file system cache and avoids double caching of data in the FairCom Server and the file system cache. The file name may include wildcard characters (see FairCom DB Standard Wildcards)

The UNBUFFERED_IO configuration option enables unbuffered I/O for the specified file. When the file is opened, the sector size of the disk on which the file resides is determined and it stores that sector size in the new file control block member.

Windows enforces the following restrictions for I/O when using unbuffered I/O:

The file offset for the I/O operation must be a multiple of the disk sector size.
The amount of data to be read or written must be a multiple of the disk sector size.
The address of the buffer used in the I/O operation must be aligned on a disk sector size boundary.

For files that use unbuffered I/O, c-tree's file I/O function checks that these requirements are met. If not, logic is in place that makes the necessary adjustment by allocating a temporary buffer that is used in the I/O operation.

When a file is configured to use unbuffered I/O, the sector size of the disk on which the file is stored is checked and if it exceeds the server's page size. If so, the following message is logged to CTSTATUS.FCS and unbuffered I/O is not used on that file:

If an existing file is being opened:

mbopen: File <filename> disk sector size (<disk_sector_size>) exceeds page size (<page_size>)

If a new file is being created:

mbcratx: File <filename> disk sector size (<disk_sector_size>) exceeds page size (<page_size>)

Note: Unbuffered I/O is not available for encrypted or segmented files and is ignored for those file types.

UNBUFFERED_IO completely avoids the file system cache. Compare to COMPATIBILITY TDATA_WRITETHRU, which uses the file system cache and then to disk before returning.

OS Support

This option is supported on the Windows operating system.

In V11 and later, support for direct I/O has been enabled on Linux systems. A value of 512-bytes is used for size and alignment for direct I/O.

This feature supports both c-tree data and index files, as well as transaction logs. Configuration options are provided for both.

UNBUFFERED_IO filename (enables direct I/O for the specified file; the filename can include wildcards, such as *.dat)
UNBUFFERED_LOG_IO YES (enables direct I/O for the transaction logs).

See Also

UNBUFFERED_LOG_IO

DIAGNOSTICS DIRECT_IO

Enables a check on each write operation for a file for which direct I/O is requested that the properties of the write operation meet the direct I/O requirements. If not, the server logs one of the following messages to the server status log:

directIOdiag: Buffer address (<address>) not properly aligned for direct I/O...

directIOdiag: Write length (<length>) not properly sized for direct I/O...

directIOdiag: File offset (<offset>) not properly aligned for direct I/O...

DIAGNOSTICS LOWL_FILE_IO

The DIAGNOSTICS LOWL_FILE_IO server keyword logs low-level system errors into the server status file, CTSTATUS.FCS. Although client applications have access to system errors through sysiocod, it is useful to see these errors logged on the server side. Activating this feature causes a high decrease in performance.

This feature is also available at run time via the ctSETCFG() API call and passing in (setcfgDIAGNOSTICS, "LOWL_FILE_IO")

From the FairCom Server administrator utility, ctadmn, choose option 10, "Change Server Settings" and then option 9, "Change a DIAGNOSTICS option". Then enter LOWL_FILE_IO to enable. Precede with a '~' character to disable.

Default: Disabled

Logging and Monitoring

CHECKPOINT_MONITOR

Determines if each occurrence of an internal FairCom Server checkpoint will cause a time stamp message to be sent to the FairCom Server console screen and to the CTSTATUS.FCS file.

CTSTATUS_MASK (CTSTATUS_MASK, CTSTATUS_MASK)

Allows certain types of entries in CTSTATUS.FCS to be suppressed.

CTSTATUS_SIZE

Controls the size of the FairCom Server status file.

DISK_FULL_ACTION

Enables FairCom DB to monitor available disk space and to shut down the database engine when the disk space falls below the specified limit.

DISK_FULL_LIMIT

Activates a disk space threshold mechanism to detect when a disk volume is getting full (the specified number of bytes must remain available on a disk volume after a file has been extended).

DISK_FULL_VOLUME

Allows volume-specific disk full checks.

FUNCTION_MONITOR

Causes the client number, function number, function name, and file name are displayed in a scrolling fashion on the FairCom Server console screen.

HEAP_DEBUG_LEVEL

Enables additional tracking and checking of internal memory suballocations - Consult with FairCom support team as this option can significantly impact performance in certain use cases.

LOCK_MONITOR

Deprecated: Monitors the number of active record locks.

MEMORY_MONITOR

Sends a message to the console whenever allocated memory exceeds the next memory threshold.

MEMORY_TRACK

Sends debug output to the console every time the net memory allocation count changes by a multiple of the threshold value.

MONITOR_MASK

Suppress the message sent to the system console if a file without a matching name does match the unique file ID when a file open is attempted.

PERF_MONITOR

Allows entries to be placed on a queue for client side processing and monitoring of server events.

REQUEST_TIME_MONITOR

Specifies a request time in seconds for monitoring function that exceed the specified time.

SNAPSHOT_FILENAME

Captures user information pertaining the to file specified to SNAPSHOT.FCS in addition to the system snapshot.

SNAPSHOT_INTERVAL

Enables automatic snapshots at specified intervals.

SNAPSHOT_LOCKWAIT_USEC

Changes the default histogram intervals (box width) for the lock waiting time histograms.

SNAPSHOT_LOGFLUSHTIME_USEC

Controls the width of the fixed transaction log flush time histogram, and the initial width of the adaptive transaction log flush time histogram.

SNAPSHOT_TRANTIME_USEC

Changes the default histogram intervals (box width) for the transaction time histogram.

SNAPSHOT_USERID

Captures user information to SNAPSHOT.FCS in addition to the system snapshot.

SYSLOG (SYSLOG, SYSLOG)

Specifies a keyword indicating contents to be stored in the System Event Log.

SYSVIEW_WHAT

Enables system status reports triggered by the specified event.

SYSVIEW_WHEN

Enables system status reports triggered by the specified event.

Diagnostics

DEADLOCK_MONITOR

Causes a time stamp message goes to the FairCom Server console screen each time the FairCom Server detects and resolves a dead lock.

DIAGNOSTICS SNAPSHOT_AUTOMATIC

Write system snapshots to the human-readable SNAPSHOT.FCS text file.

DIAGNOSTICS SNAPSHOT_IOTIME

Enables collecting disk read and write timings on a per-file basis when high-resolution timer support is activated.

DIAGNOSTICS SNAPSHOT_SHUTDOWN

Write system snapshots to the human-readable SNAPSHOT.FCS text file.

DIAGNOSTICS SNAPSHOT_WORKTIME

Enables collecting FairCom DB function call counts and timings on a per-c-tree file basis.

CHECKPOINT_MONITOR

CHECKPOINT_MONITOR <YES | NO | DETAIL>

This keyword takes one of three arguments: YES, NO, and DETAIL. If YES, each occurrence of an internal FairCom DB Server checkpoint will cause a time stamp message to be sent to the FairCom DB Server console screen and to the CTSTATUS.FCS file. The checkpoint is a snapshot of the FairCom DB Server at an instance in time and is used during automatic recovery. The checkpoint provides for a measure of the system activity. The DETAIL argument causes six intermediate milestones to be output for each checkpoint in addition to the beginning and ending checkpoint messages. These intermediate outputs aid in analyzing how the checkpoint procedure interacts with applications. If there is no system activity, no checkpoints will occur. This keyword should be used for debugging purposes only since performance may be compromised.

Default: NO

CTSTATUS_MASK

CTSTATUS_MASK <mask>

Allows certain types of entries in CTSTATUS.FCS to be suppressed.

Accepted <mask> values

DYNAMIC_DUMP_FILES	Suppress the logging of the names of the files backed up by a dynamic dump operation.
MATCH_FILE_ID	Suppresses "Matching file Ids" messages. Such messages can be output several situations: If the files share the same FairCom 12-byte file ID, but are different files in which case an open error should result. If the files are the same file, but have been opened with different names (or paths). When the file ID is reassigned because the ID was changed, for example, the file is a copy of an original file.
PRIME_CACHE_AT_STARTUP	Suppress log messages while cache priming is enabled.
WARNING_AUTO_TRNLOG	Suppresses the logging of the message: "WARNING: could not turn on AUTO_PREIMG/AUTO_TRNLOG for ... (filename)" when the AUTO_PREIMG or AUTO_TRNLOG configuration option matches the name of a c-tree index file that was not created with support for transaction control.
WARNING_FCRP_ERR	Eliminates FCRPT_ERR (14) errors in CTSTATUS.FCS.
WARNING_PAGESIZE	Disables logging PAGE_SIZE warning messages to CTSTATUS.FCS,
WARNING_PAGESIZE_SEEALL	Logs all PAGE_SIZE warnings to CTSTATUS.FCS for debugging.
WRITE_ERR	Suppresses WRITE_ERR messages. See DIAGNOSTICS WRITE_ERR_DUMP
VDP_ERROR	Suppresses logging of communications errors.

Default: Log all messages.

CTSTATUS_SIZE

CTSTATUS_SIZE <file_size | negative_file_size | 0>

CTSTATUS_SIZE controls the size of the FairCom DB Server status file. The argument to CTSTATUS_SIZE is the approximate maximum size in bytes for CTSTATUS.FCS. When this limit is reached, CTSTATUS.FCS is renamed to T0000001.FCS and a new status file is created. The T#.FCS file numbers increase each time the limit is reached, similar to the transaction log files, i.e., the next time the maximum size is reached, CTSTATUS.FCS is renamed to T0000002.FCS.

To limit the number of archived status logs, set a negative value for CTSTATUS_SIZE. Only T0000001.FCS will be kept, being replaced each time CTSTATUS.FCS is archived.

A value of 0 allows the file to expand to a size limited by the operating system and storage availability.

Default: -32000000

DBENGINE_CHECK

In V11 and later, this Replication Agent configuration keyword enables monitoring of changes to the FairCom DB "engine":

DBENGINE_CHECK = 1

Enables tracking changes to the ctsrvr.cfg, CTSTATUS.FCS, *.lib, and *.dll.

Schedules a future action for double checking the FairCom DB API Engine changes. It is scheduled with a delay of 5 seconds by default, and all the pending checks for a given c-tree installation are grouped to avoid several re-checks. The action execution is able to change an existing installation, add a new one, or delete an old one.

DEADLOCK_MONITOR

DEADLOCK_MONITOR <YES | NO>

If YES, each time the FairCom DB Server detects and resolves a dead lock situation, a time stamp message goes to the FairCom DB Server console screen.

Note: This keyword is used primarily for debugging since this feature consumes additional overhead.

Default: NO

DISK_FULL_ACTION (Original)

DISK_FULL_ACTION <action> <volume> <limit>

Enables FairCom DB to monitor available disk space and to shut down the database engine when the disk space falls below the specified limit.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

You can include more than one DISK_FULL_ACTION in a server configuration.

Scriptable actions are supported. See DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM)).

This keyword can be specified multiple times.

This keyword is independent of the DISK_FULL_LIMIT and DISK_FULL_VOLUME keywords.

Windows Examples

DISK_FULL_ACTION shutdown D:\ 500 MB

DISK_FULL_ACTION shutdown "C:\Users\Administrator\My Documents" 1 GB

Unix Examples

DISK_FULL_ACTION shutdown /users/administrator 500 MB

DISK_FULL_ACTION shutdown "/users/administrator/my documents" 1 GB

Messages

The following messages appear in CTSTATUS.FCS when this option is used:

FairCom DB logs a message for each DISK_FULL_ACTION option it finds in ctsrvr.cfg:

- User# 00001 Configuration info : DISK_FULL_ACTION threshold set for volume S:\ to 7516192768 bytes

FairCom DB logs a message to indicate that the available disk space monitoring thread has started:

- User# 00009 Thread Start: ctDISKFULLactionthrd

FairCom DB logs a message when it detects disk space below the specified threshold which causes the database engine to shut down:

- User# 00009 DISK_FULL_ACTION: Space on volume S:\ is 5338173440, which is below threshold of 7516192768. Initiating database engine shutdown.

See Also

DISK_FULL_LIMIT
DISK_FULL_VOLUME
DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))

DISK_FULL_ACTION (SUBSYSTEM)

FairCom Server supports a configuration option to call an external script when a specified DISK_FULL_LIMIT or DISK_FULL_VOLUME limit is reached. This allows server administrator to configure an external process to run when a disk-full limit is reached, for example, the process could write information to a log or notify the administrator.

To configure an external process to run when the space on the specified volume is below the specified limit, set the DISK_FULL_ACTION configuration option in ctsrvr.cfg with the run action.

The supported syntax is:

SUBSYSTEM EVENT DISK_FULL_ACTION {

volume VOLUME

limit LIMIT

run EXE [OPTIONS]

freq FREQUENCY

maxruntime MAXRUNTIME

}

The volume, limit, and run parameters are required. The freq and maxruntime are optional.

VOLUME is the name of the disk volume to check: a drive name (for example, C:\) on Windows systems, or a directory name (for example, /users/faircom/data) on Unix/Linux systems.
LIMIT is the threshold of available space that triggers the running of the script. The KB, MB, GB, TB, and PB suffixes can be used for this value (for example: "limit 100 GB" means 100 gigabytes).
EXE is the name of an executable, batch file, or Unix shell script to run, and OPTIONS are optional command-line options to pass to the external program.
FREQUENCY indicates the frequency at which this condition is checked. The default is to check the condition every 60 seconds. Minimum frequency is 1; maximum frequency is 32,000,000.
MAXRUNTIME indicates the maximum amount of time, in seconds, that the external script is permitted to run. If the external script has not terminated after the specified amount of time, the server terminates the script. The default is 60 seconds. A setting of 0 means the script will never be forced to terminate.

Note: When setting a MAXRUNTIME value, consider that no other DISK_FULL_ACTION checks will take place until the script completes, regardless of the FREQ setting. This is of particular importance if more than one DISK_FULL_ACTION is created.

In V12 and later, disk full monitoring keywords have been added to the default ctsrvr.cfg. These keywords are present in this config file, but they have been commented out for so they are there if you need them, but they are NOT enabled.

; SUBSYSTEM EVENT DISK_FULL_ACTION {

volume VOLUME

limit LIMIT

run EXE [OPTIONS]

freq FREQUENCY

maxruntime MAXRUNTIME

}

Example DISK_FULL_ACTION with external process action

The following configuration option causes the batch file diskfull.bat to be run when writing to a c-tree data or index file detects that the space on C:\ has dropped below 1 GB. This disk-full condition is checked once every 60 seconds.

SUBSYSTEM EVENT DISK_FULL_ACTION {

volume C:\

limit 1 GB

run diskfull.bat

freq 60

maxruntime 120

}

Reading parameters sent from the server to the external process

FairCom Server passes a set of ASCII key/value pairs to the external process using standard input, where the input format is key=value, one per line. The keys that FairCom Server passes to the external process are:

volume - The VOLUME setting from the DISK_FULL_ACTION configuration option.
limit - The disk-full limit in bytes that was specified for that volume.
available - The amount of disk space in bytes that is currently available on that volume.

Writing parameters from the external process to the server

The external process can optionally pass ASCII key/value pairs back to the server by writing them to standard output in the format key=value, one entry per line. FairCom Server recognizes the following keys:

action=continue - (indicating to continue FairCom Server operation; this is the default)

action=shutdown - (indicating to shut down FairCom Server)
comments=MESSAGE - A user-defined message that will be written to CTSTATUS.FCS.

Note: FairCom Server reads only the first 1024 bytes of the output data returned by the script. All data after the first 1024 bytes of output data is ignored.

If the external script's output is not in the supported format, a message is written to CTSTATUS.FCS. Example:

DISK_FULL_ACTION: Command diskfull.bat output does not match format var=value: line 3 column 8

Unlimited MAXRUNTIME script

An unlimited script runtime can be set with MAXRUNTIME 0. When configured as unlimited and a script is run due to the LIMIT and RUN options, the script will never be forced to terminate.

DISK_FULL_LIMIT

DISK_FULL_LIMIT <bytes available>

FairCom DB Servers for Unix and Windows support the DISK_FULL_LIMIT keyword, which activates a disk space threshold mechanism to detect when a disk volume is getting full. The DISK_FULL_LIMIT configuration keyword takes as its argument the number of bytes that must remain available on a disk volume after a file has been extended. If the update operation fails, a message is written in CTSTATUS.FCS naming the file involved.

FairCom DB Servers that do not support this feature ignore the DISK_FULL_LIMIT keyword.

Note: When the disk is full, the FairCom DB files cannot be extended. The FairCom DB operations that attempt to add or update data in a file and require the files to be extended will fail with specific errors. This diagnostic keyword is used to put supplemental information in the status file and alert the user.

Default: No disk full check

See Also

DISK_FULL_VOLUME
DISK_FULL_ACTION
DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))

DISK_FULL_VOLUME

DISK_FULL_VOLUME /path

Allows volume-specific disk full checks. DISK_FULL_VOLUME takes as its argument a concatenated volume name and limit. A path separator must occur between the volume name and the threshold limit, which may be zero.

In Unix this is in the form /name/<limit>. The following example places a disk full threshold of one million bytes on the volume /home:

DISK_FULL_VOLUME /home/1000000

For Windows, the form of the argument is <DRIVE>:\<limit>. The following example places a 1MB threshold on drive E:

DISK_FULL_VOLUME e:\1048576

Note: When the disk is full the FairCom DB files cannot be extended. The c-tree operations that attempt to add or update data in a file and require the files to be extended will fail with specific errors. This diagnostic keyword is used to put supplemental information in the status file and alert the user.

Default: Off

See Also

DISK_FULL_LIMIT
DISK_FULL_ACTION
DISK_FULL_ACTION Script (DISK_FULL_ACTION (SUBSYSTEM), DISK_FULL_ACTION (SUBSYSTEM))

FUNCTION_MONITOR

FUNCTION_MONITOR <YES | NO | file_name>

If YES, the client number, function number, function name, and file name are displayed in a scrolling fashion on the FairCom DB Server console screen. Alternatively, the same information, along with the return value and error codes for each function, can be routed to a file by specifying a file name. This keyword should be used primarily for debugging since this feature consumes additional overhead.

Note: Activate the function monitor dynamically under the FairCom DB Server for Windows by selecting View > Function Monitor Window.

Default: NO

Notes:

FUNCTION_MONITOR can be enabled without restarting the server using the ctadmn command-line tool as follows:

Start the ctadmn tool and connect to the server.
Select the following from the menu:

10 "Change Server Settings"

Select the following from the next menu:

1 "Configure function monitor"

Enter fmon.log at the prompt:

Enter the new function monitor value.
Allowed values are: YES or NO or filename

Function monitor information will be written to the FairCom DB process working directory in a text file named fmon.log.

When you are done, be sure to shut off function monitoring: Repeat the steps above, but disable function monitoring by entering NO in the final step.

HEAP_DEBUG_LEVEL

FairCom Database Engine heavily utilizes memory operations to optimize performance. To avoid frequent OS memory requests, it maintains a sophisticated memory suballocator. This internal subsystem allocates larger blocks of memory from the O/S and sub allocates them as needed. All memory gets and puts from this suballocator system are tagged, tracked and checked. On rare occasions, heap memory corruption can occur resulting in an eventual illegal memory access attempt, ultimately terminating FairCom DB operations. Debugging these conditions is extremely complex. FairCom DB V12 and FairCom RTG V3 enhance memory diagnostics when required.

Debugging options can now be enabled when corrupted memory is suspected triggering additional tracking and checking of internal memory suballocations. Use of these options should only be considered in consultation with your FairCom support team as these can significantly impact performance in certain use cases.

HEAP_DEBUG_LEVEL <level> with values 0-3 (Default 0):

HEAP_DEBUG_LEVEL 0 uses the default memory allocator; no checks are performed.
HEAP_DEBUG_LEVEL 1 enables the debug allocator, with some detection for memory buffer write overruns or use-after-free bugs. There is no additional memory overhead, and the CPU overhead is small.
HEAP_DEBUG_LEVEL 2 does the checks from HEAP_DEBUG_LEVEL 1 and adds a small redzone before/after each allocation, which is checked at free. This may detect some buffer write overruns, write underruns, or double frees. This option uses an additional 16 bytes of memory on each allocation.

Levels 1 and 2 generate a CTSTATUS entry and stack trace at free when a memory fault is detected.

HEAP_DEBUG_LEVEL 3 uses the system's virtual memory subsystem to immediately detect illegal memory overruns (both reads and writes) by aligning allocations at the end of a 4K page. The following types of errors may immediately generate a segmentation fault (core dump) on the invalid access: memory read/write overrun, double free, and use-after-free.

Memory write underruns or some small write overruns may instead be detected at free and generate a CTSTATUS message and stack trace.

Memory allocation failures are logged to CTSTATUS logging when HEAP_DEBUG_LEVEL 3 is enabled.

Because Level 3 debugging uses an extra 4K-8K bytes for all allocations, it is possible to restrict this to particular size(s) of allocations using HEAP_DEBUG_EXCLUSION_LIST <N>.

Be sure to see the Performance Note below.

HEAP_DEBUG_EXCLUSION_LIST <N> [,<N> ...] - When HEAP_DEBUG_LEVEL 3 (only) is specified, it is possible to have particular allocation sizes use the default (non-debug) allocator to reduce the overall memory overhead of this debugging method. By default, HEAP_DEBUG_LEVEL applies to all allocation sizes. The particular values to specify would typically be recommended by FairCom support based on analysis of prior core dumps.

HEAP_DEBUG_EXCLUSION_LIST # allocation size range (in bytes):

# Bytes

1 <=16

2 17-32

3 33-64

4 65-128

5 129-256

6 257-512

7 513-1024

8 1025-2048

9 2049-4096

10 4097-8192

Limitations: When Level 3 debugging is enabled, memory usage statistics are not tracked for those bins.

Performance Note

HEAP_DEBUG_LEVEL 3 has a negative impact on performance on any machines with 4 or more CPU cores. FairCom does not recommend the use of Level 3 on any machines with 4 or more CPU cores where performance is important.

If performance is an issue, FairCom recommends using the Level 2 version of heap debug on any location where you've seen a memory-related crash occur and where performance is important. Our testing has not revealed a noticeable performance impact with Level 2, even on a large test box with 72 cores.

Note 1:

If allocation stack traces are enabled (such as with ctstat -mt +ALL), an additional stack is dumped showing the allocation location of the buffer.

A use-after-free might cause a segmentation fault (core dump).

Note 2:

HEAP_DEBUG_LEVEL 2 has a benefit to stability: If the buffer overrun is 8 bytes or less, the heap will NOT be corrupted because only the extra redzone memory is modified. The server will stay up and only a stack trace will be generated.

Linux

On Linux, the HEAP_DEBUG_LEVEL 3 configuration option requires raising the kernel limit vm.max_map_count to be twice the number of allocations. FairCom can help you to determine the minimum value to set the Linux kernel parameter, vm.max_map_count. (This value will generally rise and fall with the amount of server activity, so include a safety factor based on how much busier it might be.) Setting vm.max_map_count too small will result in allocation failures if the number of allocations exceeds that limit, which could lead to a server crash.

This can be changed until next system reboot with the command:

sysctl -w vm.max_map_count=<N>

Example using HEAP_DEBUG_LEVEL 3 restricted to sublist #2:

Look at current memory usage at a busy server time using this command:

ctstat -ml -u admin -p ADMIN_PASSWD -s FAIRCOMS

The output will show sublist #2 as "PI2TYP." The next column shows the total allocations in bytes.

PI2TYP 98304 7400 0.01%

Divide the first column by 32 (the max size of this allocation range) to get the number of current allocations on this sublist: 98304/32=3072, and then double that: 6144.

You also need to include the MBATYP:

MBATYP 73388776- 73388776-

Divide the first column by 8192 (this would be the worst case assumption for this list), and double the result:

73388776/8192 * 2 = 17916

The sum of these values is 2x the allocation count: 17916+6144 = 24060. Apply a safety multiplier such as x10.

This is the minimum value you need to set as the Linux kernel parameter:

vm.max_map_count

This value will generally rise and fall with the amount of server activity, so include a safety factor based on how much busier it may become. Setting vm.max_map_count too small will result in allocation failures if the number of allocations exceeds that limit, which could lead to a server crash.

After setting the kernel value (Linux only) with vm.max_map_count, add the following server keywords to enable the debug suballocator on sublist #2 (PI2TYP) only:

HEAP_DEBUG_LEVEL 3

HEAP_DEBUG_EXCLUSION_LIST 1,3,4,5,6,7,8,9,10

Stack Dump Message

When a Stack Dump is generated, the following message will be logged to CTSTATUS.FCS.

"A Heap Fault was detected and stack dump created"

If you have this logic enabled, FairCom recommends routine reviews of CTSTATUS.FCS to determine if a Stack Dump has been created. If you see this message, please send the Stack Dump, and the CTSTATUS.FCS message to FairCom Support for inspection.

LOCK_MONITOR

Deprecated. See the notes below for other monitoring tools.

LOCK_MONITOR <threshold value>

Monitors the number of active record locks. This keyword, or the SetOperationState() function call, enables a lock monitor that indicates when the number of lock calls over unlocks equals a multiple of the threshold value, or if it goes below a threshold.

For example, LOCK_MONITOR 100 sends a message to the console each time the number of lock calls over unlocks equals a multiple of 100. Likewise, if the number of unbalanced lock calls falls below these thresholds, a message goes to the console. Ordinarily, when the number of locks are in balance (i.e., excess locks over unlocks equals zero) no message is routed to the console unless a message indicating an excess of locks has already been sent to the console. If you wish the message to be sent whenever the number of excess locks equals zero, enter the threshold value as a negative. For example, LOCK_MONITOR -100

Default: No lock monitor

Tools for Monitoring Locks

Several elements in the Snapshot Structure contain information about locks. For information about monitoring the state of locks, see the File Snapshot Structure in the FairCom DB Programmer's Reference Guide.

For a graphical view of locks and other system information, try using the FairCom DB Monitor and the FairCom DB Gauges available in your tools folder and in the Windows Start menu.

MEMORY_MONITOR

MEMORY_MONITOR <Bytes | NO>

Sends a message to the console whenever allocated memory exceeds the next memory threshold. The parameter specifies a size in bytes. For example, MEMORY_MONITOR 500000 sends a message every time memory consumption exceeds the next 500,000 byte range of memory. The message is also sent when memory usage decreases for each absolute memory block. This keyword should be used primarily for debugging, as there is some additional overhead for this feature.

Default: NO

MEMORY_TRACK

MEMORY_TRACK <allocation threshold value>

Sends debug output to the console every time the net memory allocation count changes by a multiple of the threshold value. The count is the number of memory allocation requests. See also DIAGNOSTICS TRACK_LOGON.

Default: 0 (indicates do not track)

MONITOR_MASK

When a file open is attempted, the FairCom DB Server checks to see if either a file with the same name has already been opened, or if a file with the same unique ID has already been opened. By default, if a file without a matching name does match the unique file ID FairCom DB sends a message to the system console indicating the names of the two files involved. Suppress this message by adding the following entry:

MONITOR_MASK MATCH_FILE_ID

Default: Enabled

MONITOR_MASK

MULTILINE_STATUS_LOG_MESSAGE YES | NO

Configure the server to write a single line message.

Thu Apr 11 17:07:04 2024 - User# 00001 Startup FairCom DB SQL Server - V13.0.0.125(Build-240406) OEM: 0x39

Thu Apr 11 17:07:04 2024 - User# 00001 For Windows 64bit Operating System

Thu Apr 11 17:07:04 2024 - User# 00001 LICENSE: License file in use: .\ctsrvr39001112.lic

Thu Apr 11 17:07:04 2024 - User# 00001 Using configuration file <../config/ctsrvr.cfg>

Thu Apr 11 17:07:04 2024 - User# 00001 DH_JVM_OPTION_STRINGS=-Xms100m;-Xmx300m;-XX:+UseG1GC

Thu Apr 11 17:07:04 2024 - User# 00001 Windows native reader/writer lock support enabled

Thu Apr 11 17:07:04 2024 - User# 00001 Process ID: 40200

Thu Apr 11 17:07:04 2024 - User# 00001 Alternative server name...

Thu Apr 11 17:07:04 2024 - User# 00001 FAIRCOMS

Thu Apr 11 17:07:04 2024 - User# 00001 Using configuration directory: ../config

Thu Apr 11 17:07:04 2024 - User# 00001 Compatibility bit maps: (1)00002000x (2)00020000x (3)00000000x (4)00000000x (5)00000000x

Thu Apr 11 17:07:04 2024 - User# 00001 Diagnostic bit maps: (1)40000000x (2)00000000x (3)00000000x

Thu Apr 11 17:07:04 2024 - User# 00001 64-bit File Address Support

Thu Apr 11 17:07:04 2024 - User# 00001 6 Byte Transaction Numbers

Thu Apr 11 17:07:04 2024 - User# 00001 Commit Read Lock Support

Thu Apr 11 17:07:04 2024 - User# 00001 NOWAIT usrsema enabled

Thu Apr 11 17:07:04 2024 - User# 00001 Smart File Sync Support

Thu Apr 11 17:07:04 2024 - User# 00001 ctIOSEMA I/O semaphore optimization is enabled

Thu Apr 11 17:07:04 2024 - User# 00001 ctIOSEMAmemfile I/O semaphore optimization is enabled

Thu Apr 11 17:07:04 2024 - User# 00001 ctFeatGNSEMAhsh index cache mutex optimization is enabled

Thu Apr 11 17:07:04 2024 - User# 00001 ctFeatDBSEMAhsh data cache mutex optimization is enabled

Thu Apr 11 17:07:04 2024 - User# 00001 Memory suballocator allocation unit: 16

Thu Apr 11 17:07:04 2024 - User# 00001 FairCom DB SQL Enterprise Edition

Historically, CTSTATUS.FCS log messages have had the timestamp on one line and the message on the next line.

Default: NO

PERF_MONITOR

PERF_MONITOR <option>

The SYSMON feature allows entries to be placed on a queue for client side processing and monitoring of server events. This option allows event entries to be specified in the ctsrvr.cfg configuration file. The following options are supported:

ALL_EVENTS

CHECKPOINT

LOG_EXTENSION

LONG_TRANSACTION

When checkpoints are monitored, an entry is written to the SYSMON_PERF client-side queue for the beginning and end of each checkpoint. Log extensions similarly generate begin and end entries. A long transaction generates one entry.

ALL_EVENTS is equivalent to listing each individual option.

When one or more event types have been enabled, then the first entry written into the SYSMON_PERF queue provides the information necessary to convert the high-resolution times into seconds and/or dates.

See Also

LONG_TRANSACTION_MS

REQUEST_TIME_MONITOR

REQUEST_TIME_MONITOR <request_time_interval>

FairCom DB supports monitoring function request times that exceed a specified time limit with the REQUEST_TIME_MONITOR <request_time_interval> keyword. <request_time_interval> is a time in seconds. A value of -1 disables the feature, meaning that it cannot be dynamically turned on using the ctSETCFG() function, as described below. A non-negative value (the minimum value is 10) specifies a request time in seconds. An internal thread checks for requests whose time has exceeded the specified request time. When the thread finds such a request, it writes a message to CTSTATUS.FCS, creates a process stack dump (on systems that support this feature), and logs diagnostic information to the file RQSTMON.FCS.

The ctSETCFG() function can be used to change the request time monitor interval value. For example:

ctSETCFG( setcfgREQUEST_TIME_MONITOR, "60" );

sets the request time interval to 60 seconds. If the caller of ctSETCFG() is not a member of the ADMIN group, ctSETCFG() returns error LADM_ERR (589, member of ADMIN group required). Attempting to specify a negative value returns PBAD_ERR (749, bad parameter value). Attempting to enable this feature when REQUEST_TIME_MONITOR -1 is specified in the configuration file returns SETO_ERR (804, cannot override configuration option).

The previously unused field scttrnavl in the system snapshot structure (ctGSMS) has been renamed sctrqtmonint and is set to the request time monitor interval value. This change resulted in the system snapshot structure version change from 7 to 8.

Default: NO

SNAPSHOT_FILENAME

SNAPSHOT_FILENAME <file name>

By default, only the system snapshot is captured with SNAPSHOT_INTERVAL. SNAPSHOT_FILENAME is used to capture user information to SNAPSHOT.FCS as well.

SNAPSHOT_FILENAME <file name>

Files added to the snapshots are said to be activated. Files may be activated whether or not the automatic snapshots are turned on in the configuration file. However, the activation has no effect until snapshots are written to the SYSLOG files.

The <file name> argument may include wildcard matching characters where “*” matches an arbitrary number of any characters and “?” matches exactly one of any character. A pattern of simply “*” matches any user or file name. See FairCom DB Standard Wildcards. For example, the following keywords activate any file ending in “.dat”, and the file journal.idx:

SNAPSHOT_FILENAME *.dat

SNAPSHOT_FILENAME journal.idx

File name case sensitivity depends on the platform. For example, Windows is case insensitive and Unix is case sensitive. The file names activated must match the file name used to first open the file. In particular, paths used in the activation list and during the call to open the file must match.

See Also

SNAPSHOT_INTERVAL

SNAPSHOT_USERID

SNAPSHOT_INTERVAL

The SNAPSHOT_INTERVAL keyword enables automatic snapshots at specified intervals:

SNAPSHOT_INTERVAL <minutes>

By default, only the system snapshot is captured. To add user- or file-specific snapshots to the data captured, use one or more of the following configuration entries:

SNAPSHOT_USERID <user ID>

SNAPSHOT_FILENAME <file name>

By default, this keyword logs the output to the c-tree Server System Event Log (SYSLOGDT.FCS) discussed at these links:

Use the DIAGNOSTICS SNAPSHOT_AUTOMATIC keyword to change the output to the SNAPSHOT.FCS human readable file.

See Also

SNAPSHOT_FILENAME

SNAPSHOT_USERID

SNAPSHOT_LOCKWAIT_USEC

SNAPSHOT_LOCKWAIT_USEC <lock wait histogram interval width in microseconds>

Histograms of waiting times for blocked lock requests are available: one for waiting times for blocked data record lock requests, and one for waiting times for blocked index lock requests (please note that the index locks are not controlled by the user). There is a small amount of overhead associated with mutex calls to collect clean statistics.

SNAPSHOT_LOCKWAIT_USEC can change the default histogram intervals (box width) for the lock waiting time histograms (default of 10,000 µsec or 0.01 seconds).

SNAPSHOT_LOGFLUSHTIME_USEC

SNAPSHOT_LOGFLUSHTIME_USEC <histogram interval width in microseconds>

The SNAPSHOT feature includes two histograms of transaction log flush times to help identify if slow transaction log writes are slowing down server performance. One is of fixed width, and the other has an adaptive width*. These histograms can be viewed in the ctstat utility.

SNAPSHOT_LOGFLUSHTIME_USEC controls the width of the fixed histogram, and the initial width of the adaptive histogram (which defaults to 10 microseconds). This may be changed at server startup by specifying SNAPSHOT_LOGFLUSHTIME_USEC in ctsrvr.cfg. It can also be changed at runtime via ctSETCFG() or the ctadmn utility (select 10. Change Server Settings and then 10. Change the specified configuration option). When the fixed width is changed or the initial adaptive width is automatically adjusted, the histogram box values are reset to zero.

*The first time the server starts, the server sets the adaptive histogram box width to the default value or the SNAPSHOT_LOGFLUSHTIME_USEC value specified in ctsrvr.cfg. As the server runs, on each checkpoint operation, it determines if a significant fraction of the log save values since the last checkpoint are at the low or high end of the histogram range. If so, the server changes the histogram box width to a value that centers the average log save time in the histogram boxes.

SNAPSHOT_TRANTIME_USEC

SNAPSHOT_TRANTIME_USEC <tran time histogram interval width in microseconds>

The SNAPSHOT feature includes a histogram of transaction times. There is a small amount of overhead associated with mutex calls to collect clean statistics.

SNAPSHOT_TRANTIME_USEC can change the default histogram intervals (box width) for the transaction time histogram (default of 50,000 µsec or 0.05 seconds).

SNAPSHOT_USERID

SNAPSHOT_USERID <user ID>

By default, only the system snapshot is captured with SNAPSHOT_INTERVAL. SNAPSHOT_USERID is used to capture user information to SNAPSHOT.FCS as well.

Users added to the snapshots are said to be activated. Users may be activated whether or not the automatic snapshots are turned on in the configuration file. However, the activation has no effect until snapshots are written to the SYSLOG files.

The <user ID> argument may include wildcard matching characters: “*” matches an arbitrary number of any characters, and “?” matches exactly one of any character. A pattern of simply “*” matches any user or file name. See FairCom DB Standard Wildcards. For example, the following keywords activate all users:

SNAPSHOT_USERID *

User IDs are not case sensitive.

See Also

SNAPSHOT_INTERVAL

SNAPSHOT_FILENAME

SYSLOG

SYSLOG <option>

FairCom DB optionally maintains a system event log, SYSLOG. This is maintained in two system files: SYSLOGDT.FCS and SYSLOGIX.FCS. These files comprise a FairCom DB data file and index pair with a record for each recordable system event. Unlike the text based CTSTATUS.FCS, SYSLOG can be encrypted such that entries cannot be added, deleted, or modified with a simple text editor, and vendors can log application specific entries.

The System Event Log contents are controlled by SYSLOG configuration keywords in ctsrvr.cfg, the ctsrvr.set settings file, or from the command line. They are entered as pairs in the form of: SYSLOG <keyword>. As many of these pairs as desired may be used at the discretion of your application vendor.

Current SYSLOG options include:

ADMIN_API	Only allow users in the ADMIN group to use the SystemLog() function to create vendor-defined entries in the log.
CTSTATUS	Log each entry to CTSTATUS.FCS in the System Event Log, except for those entries which occur before or after the system logging monitor is in operation.
DELETE_FILE	Log file deletes and restores.
DISABLE_API	Do not allow any calls to the SystemLog() function for user defined entries.
DYNAMIC_DUMP	Log the beginning and end of dynamic dumps and a result for each file dumped.
ENCRYPT	Encrypt the SYSLOG files. Requires ADVANCED_ENCRYPTION YES
LOGFAIL_PURGE	Causes an automatic purge of the oldest entries in the log if the system cannot add a record to SYSLOGDT.FCS. All the entries occurring on the oldest day are deleted unless there are only entries for the current day in which case no entries are purged. After a successful purge, an attempt is made to add the new entry that triggered the automatic purge. If this add succeeds, the system log operation continues in its usual fashion.
LOGFAIL_CTSTATUS	If there is no LOGFAIL_PURGE entry in the configuration file, or if the purge fails, the log entries will be rerouted to CTSTATUS.FCS if LOGFAIL_CTSTATUS is in the configuration file. This disables SYSLOG CTSTATUS; i.e, no more entries are made to the system log.
LOGFAIL_TERMINATE	If there is no automatic purge or it fails, and if there is no re-routing to CTSTATUS.FCS, either the system log will stop operation, or if LOGFAIL_TERMINATE is in the configuration file, the FairCom Server will shut down. Note: USE LOGFAIL_TERMINATE WITH CAUTION!
NONE	Used in a settings file to eliminate additional SYSLOG entries in a server configuration file.
RESTORE_POINT	Use RESTORE_POINT to log created restore points, recovery with restore points, and transaction rollback to restore points through utility programs. See SYSLOG Logging of Restore Point
SQL_STATEMENTS	Enables SQL statement logging to the SYSLOG audit logs.This information includes connection information, improved timing, and logging the statement before it is actually executed for a detailed audit trail of all SQL operations.
SYSLOG_EXCLUDE_SQL_USER <name>	Allows specifying a user <name> to be excluded from this logging. All SQL statements are written to the log by default when SQL query logging is enabled by SYSLOG SQL_STATEMENTS,. Multiple users can be excluded by specifying the keyword multiple times. No validation is made that the <name> specified matches an existing user name.
TRUNCATE	Over time the SYSLOGDT.FCS and SYSLOGIX.FCS files can become quite large requiring file maintenance to reduce the size. The SYSLOG() function supports purging records, optionally filtered by time and by event code. However, purging all entries one record at a time is slow, and storage device space is not released until new records are added over time. A better solution is the TRUNCATE capability available with V13. With SYSLOG TRUNCATE in your configuration file, when a complete purge is requested (no filtering by time or event) the file is truncated rather than deleting individual records. This approach is much faster and avoids limitations with space reuse. Any user reading SYSLOG files when they are truncated will encounter the FBLK_ERR error . After receiving this error, the user must close and reopen the files to proceed..
USER_INFO	Log all logons, logoffs (including SQL users as of V12), and changes to user logon profiles.

Note: Build 210901 and earlier had a 4GB limit to the SYSLOG files. It is highly recommended to limit information recorded in this file in high volume systems, and include the LOGFAIL_PURGE option to clear data as the file grows. To disable the creation of huge system log files, add SYSLOG NONHUGE to the server configuration file. Builds after 210901 no longer have a 4GB file size limit.

History

V11.8 and later support SQL_STATEMENTS for detailed statement logging.
V11.8. and later support auditing SQL logon/logoff events with USER_INFO.

SYSLOG SQL_STATEMENTS Configuration Keyword

This configuration keyword logs executed SQL statements in SYSLOG:

SYSLOG SQL_STATEMENTS

A SYSLOG SQL_STATEMENTS (SYSLOG, SYSLOG) log entry is written after statement execution so it can also include the error code (if any).

The variable part of the SYSLOG entry contains statement information in JSON format similar to SQL_DEBUG LOG_STMT.

Below is a sample showing how it is displayed by the ctalog utility:

Class = 16 (SQL)

Event = 1 (SQL statement)

Date = 09/24/2020

Time = 17:40:11

Sequence number = 37

Error code = -20005

User ID = 'admin'

Node name = 'isql'

Variable-length information:

---------------------------------------------------

{"timestamp":"Tue Sep 24 17:40:27 2020","ipaddr":"127.0.0.1","db":"CTREESQL","user":"admin","thread":29,"statement":"select * from missingtable"}

---------------------------------------------------

SYSVIEW_WHAT

SYSVIEW_WHAT <info>

This option supports the system status reports triggered by specified events such as increasing the number of log files or beginning a dynamic dump. <info> is one of the following to determine what to include in the report.

USER_COUNT

USER_LIST

FILES_COUNT

FILES_LIST

TRANS_COUNT

TRANS_LIST

MEMORY

LOGS

ALL

One or more configuration entries can be specified in ctsrvr.cfg. If the ALL parameter is used, any other entry is redundant. The LIST options (for example., USER_LIST) also report the simple COUNT statistics (for example., USER_COUNT) such that it is redundant to specify both a LIST and COUNT. The only reason to use the COUNT option is to reduce output.

The reports are added to the end of the SYSVIEW.FCS text file. Each report is time stamped and the triggering event is noted.

Example

SYSVIEW_WHEN LOG_ACTIVE_CHG

SYSVIEW_WHEN DYNDMP_BEG

SYSVIEW_WHAT ALL

See Also

SYSVIEW_WHEN

Note: This sever feature is not currently enabled.

SYSVIEW_WHEN

SYSVIEW_WHEN <event>

This option supports the system status reports triggered by specified events such as increasing the number of log files or beginning a dynamic dump. <event> is one of the following to determine when the report is generated:

CHECKPOINT_BEG

CHECKPOINT_END

SHUTDOWN

LOG_ACTIVE_CHG,

LOG_SIZE_CHG

DYNDMP_BEG

DYNDMP_END

ALL

The reports are added to the end of the SYSVIEW.FCS text file. Each report is time stamped and the triggering event is noted.

Example

SYSVIEW_WHEN LOG_ACTIVE_CHG

SYSVIEW_WHEN DYNDMP_BEG

SYSVIEW_WHAT ALL

See Also

SYSVIEW_WHAT

Note: This sever feature is not currently enabled.

DIAGNOSTICS SNAPSHOT_AUTOMATIC

Write system snapshots to the human-readable SNAPSHOT.FCS text file.

DIAGNOSTICS SNAPSHOT_AUTOMATIC

Writes any automatic snapshots to SNAPSHOT.FCS instead of to the SYSLOG files. However, only the system snapshot is written. Snapshots for activated users and/or files are ignored.

See Also:

SNAPSHOT_INTERVAL

DIAGNOSTICS SNAPSHOT_IOTIME

Enables collecting disk read and write timings on a per-file basis when high-resolution timer support is activated.

DIAGNOSTICS SNAPSHOT_SHUTDOWN

Write system snapshots to the human-readable SNAPSHOT.FCS text file.

DIAGNOSTICS SNAPSHOT_SHUTDOWN

Writes a system snapshot to SNAPSHOT.FCS at the start of the server shutdown process. However, only the system snapshot is written. Snapshots for activated users and/or files are ignored.

FairCom recommends this option in all configuration files to establish baseline statistics over time.

DIAGNOSTICS SNAPSHOT_WORKTIME

Enables collecting FairCom DB function call counts and timings on a per-c-tree file basis.

Security

FairCom DB provides a variety of keywords that can be used for security purposes.

Note: FairCom DB File and User Security are available only when using the client/server operational model.

Encryption

ADMIN_ENCRYPT

Encrypt the FAIRCOM.FCS file at the time it is created.

ADVANCED_ENCRYPTION

Enable advanced encryption for files.

ALLOW_MASTER_KEY_CHANGE

The configuration option ALLOW_MASTER_KEY_CHANGE specifies whether the master password is changeable.

CHANGE_ENCRYPTION_ON_COMPACT

Change the encryption attributes of a file using the compact function from a client.

FIPS_ENCRYPTION

When specified, FairComDB uses the OpenSSL 3.0 FIPS module for encryption routines.

KEY_EXCHANGE_PARAMS

Specifies the PEM-encoded file containing parameters for Diffe-Hellman key exchange.

LOG_ENCRYPT

Camouflages the contents of the transaction logs to deter unauthorized access.

MASTER_KEY_FILE

Specifies a file from which c-tree reads the master encryption key.

MASTER_KEY_LIB

Specify the path of a user-created library (.dll, .so, or .dylib file) that provides the advanced encryption master key at startup time.

OPENSSL_ENCRYPTION

Encrypt client passwords with OpenSSL before they are transmitted to the server.

READONLY_SERVER

Sets the server to read-only mode.

User Access

EXPIRE_IN_DAYS

Passwords to expire after this many days. Login attempts will fail with PWDEXP_ERR (1116). The ADMIN account is excluded from password expiration.

LOGON_FAIL_LIMIT

Specifies the optional limit on the number of consecutive failed logons that causes subsequent logon attempts to fail for LOGON_FAIL_TIME minutes.

LOGON_FAIL_TIME

The length of time logons are blocked after the logon limit is exceeded.

LOGON_MUST_TIME

Requires users to log on “at-least-once” within the specified time.

MINIMUM_LENGTH

Enforces a minimum length when setting a new password.

REQUIRED_CLASSES

Enforces a minimum number of character classes from lower case, upper case, numbers, and symbols when setting a new password.

STARTUP_BLOCK_LOGONS

Prevents non-ADMIN user logons when the server is started.

Tamper-Proof Settings

These keywords affect people's ability to alter system integrity by overriding settings from a command line or by altering configuration files. See also Settings File.

NULL_STRING

Defines a symbol that represents a null string so that options can be blocked in the settings file without activating them.

COMPATIBILITY NO_COMMAND_LINE

Instructs FairCom DB to ignore command-line arguments.

COMPATIBILITY NO_CONFIG_FILE

Instructs FairCom DB to ignore the standard configuration file, ctsrvr.cfg.

Restrictions

ENABLE_TRANSFER_FILE_API

Enables the file transfer function, ctTransferFile(), which is used to transfer a file to or from the server.

FILEDEF_SECURITY_LEVEL

Protects the resource APIs, ADDRES(), UPDRES(), and DELRES(), with safeguards against unauthorized modification of file definition resources such as IFIL definitions, conditional indexes, row-level filters, etc.

Security-Related Compatibility Options

COMPATIBILITY NONADMIN_FILBLK

Permits a non-ADMIN user to set a file block if the blocking user has the file opened with update permissions.

COMPATIBILITY NONADMIN_QUIET

Permits a non-ADMIN user to call ctQuiet() to quiesce the server.

COMPATIBILITY NONADMIN_TRANSFER_FILE

Permits a non-ADMIN user to call ctTransferFile() to transfer a file.

COMPATIBILITY NON_ADMIN_SHUTDOWN

Allows non-ADMIN users to shut down the Server.

COMPATIBILITY SQLIMPORT_ADMIN_PASSWORD

Instructs FairCom DB to verify the admin password passed as a parameter.

TLS

DEBUG_LOG

Enable logging to facilitate debugging of TLS connections.

SERVER_CERTIFICATE_FILE

Provide the name of the PEM-encoded certificate file that contains this FairCom server's certificate.

SERVER_ENCRYPTED_STORE_FILE

Specify an encrypted password file that is used to decrypt this FairCom server's private key file.

SERVER_PRIVATE_KEY_FILE

Indicates the name of the file containing this FairCom server's private key.

SSL_CIPHERS

Set the encryption ciphers that are allowed to be used for encrypting TLS connections to this FairCom server.

SSL_CONNECTIONS_ONLY

Specify whether clients are allowed to connect to this FairCom server using non-encrypted connections.

VERIFY_CLIENT_CERTIFICATE

Specify whether clients are required to present valid client certificates when connecting or not.

X509_AUTHENTICATION

Specify whether the client's certificate is used for authentication rather than a username/password.

Setting password requirements

To control password requirements, add SUBSYSTEM SECURITY PASSWORD to your ctsrvr.cfg configuration file.

The default values are:

SUBSYSTEM SECURITY PASSWORD {

MINIMUM_LENGTH 15

REQUIRED_CLASSES 0

EXPIRE_IN_DAYS 0

BLOCK_OLD_PASSWORD 0

PASSWORD_HASH ORIGINAL

PASSWORD_HASH_DIFFICULTY 0

}

MINIMUM_LENGTH - Enforces a minimum length when setting a new password.

Default: 15

V13.0 and earlier default: 0

Maximum: 1024

REQUIRED_CLASSES - Enforces a minimum number of character classes from lower case, upper case, numbers, and symbols when setting a new password.

Default: 0

EXPIRE_IN_DAYS - Passwords to expire after this many days. Login attempts will fail with PWDEXP_ERR (1116). The ADMIN account is excluded from password expiration.

Default: 0 (Never expires).

BLOCK_OLD_PASSWORDS - When a non-zero number (N), password change attempts that reuse any of the N previous passwords for a user will fail with error PASSWORD_BLOCKLIST_ERR(1256).

Default: 0 (No check).

PASSWORD_HASH - In V13.0.4 onwards, allows setting the hash function used when generating new entries in the password database.

Default: ORIGINAL (SHA2-512 hash scheme)

PASSWORD_HASH_DIFFICULTY - In v13.0.4 onwards, sets a scaling factor for PASSWORD_HASH values.

Default: Depends on PASSWORD_HASH value.

Advanced notes

This subsystem can be specified in an encrypted server settings file using the ctcfgset utility to prevent these settings from being changed in the configuration file or at runtime.
If the minimum length and required classes settings are changed in the configuration file or at runtime, they do not affect existing passwords because only a hash of the password is stored. The time to expiration takes effect immediately even for existing user accounts.
When changing subsystem options at runtime (for example, using ctadmn), either the entire subsystem or particular subsystem options might be blocked by having been specified in a settings file. In both cases, the attempt to change the options fails with error SETO_ERR (804), even if some of the subsystem options are not blocked. If SETO_ERR occurs when specifying multiple options at runtime, check CTSTATUS.FCS for a message indicating which options are blocked and try again with just those options that are not blocked.

ADMIN_ENCRYPT

ADMIN_ENCRYPT <YES | NO>

Encrypt the FAIRCOM.FCS file at the time it is created. Note: This keyword will not encrypt an existing file.

To enable the encryption of the user group information file, FAIRCOM.FCS, set ADMIN_ENCRYPT YES (default). The default setting uses AES encryption, adding an additional level of security for passwords and user definitions stored in the file. FairCom DB File and User Security are available only when using the client/server operational model.

To disable the encryption of this file, set ADMIN_ENCRYPT No.

Note: AES encryption is only available if ADVANCED_ENCRYPTION YES is specified in ctsrvr.cfg.

V13: ADMIN_ENCRYPT has no effect unless ADVANCED_ENCRYPTION is enabled.

Default: YES

ADVANCED_ENCRYPTION

ADVANCED_ENCRYPTION < YES | NO >

Enable advanced encryption for files.

When Advanced Encryption is enabled, FairCom DB prompts for a master password at server startup. Run the ctcpvf utility to generate an encrypted password for use when launching the Advanced Encryption enabled Server. This will generate the file ctsrvr.pvf.

Note: Developers can use the FairCom DB SDK to replace this prompt with an application-specific method of retrieving the master password.

Any time you change the advanced encryption setting, you should delete the FAIRCOM.FCS file (which contains user and group information) before restarting FairCom DB as user and group information is encrypted for protection as well. All user and group information must be recreated if the FAIRCOM.FCS file is deleted.

Client implementation of Advanced Encryption is accomplished through the use of the SetEncryption() function. Refer to the c-tree Plus Function Reference Guide for details on this function. FairCom DB File and User Security are available only when using the client/server operational model.

Default: NO

See Also

MASTER_KEY_FILE

ALLOW_MASTER_KEY_CHANGE

FairCom DB Server supports a master key file in which the advanced encryption master key can be stored. The MASTER_KEY_FILE configuration option is used to specify a local encrypted key store.

To prevent tampering with advanced encryption keys, the server configuration option ALLOW_MASTER_KEY_CHANGE must be specified to allow changing master passwords via the SECURITY API function (an administrative client-side API call). Default value: NO.

Note: The server configuration option MASTER_KEY_FILE now considers a value of NONE to indicate no master key file is in use. This option can be specified in a settings file to prevent a configuration file from using an existing master key file.

BLOCK_OLD_PASSWORDS

When a non-zero number (N), password change attempts that reuse any of the N previous passwords for a user will fail with error PASSWORD_BLOCKLIST_ERR(1256).

Default: 0 (No check)

Note: This keyword only blocks passwords that have been used since BLOCK_OLD_PASSWORDS was turned on. It cannot retrieve passwords from before it was active.

CHANGE_ENCRYPTION_ON_COMPACT

CHANGE_ENCRYPTION_ON_COMPACT <YES | NO>

Allows changing the encryption attributes of a file using the compact API function.

Default: No

If an encrypted file is compacted, the resulting file will be encrypted using the same method.

Note: Setting this keyword to YES allows a user to permanently decrypt a file without knowing the master encryption key.

In V11 and later, to change the encryption attributes of a file using the compact function from a client, add the CHANGE_ENCRYPTION_ON_COMPACT YES to ctsrvr.cfg. If this keyword is not present or set to No, the operation fails with error NSUP_ERR (454, not supported).

COMPATIBILITY NO_COMMAND_LINE

This option is used in conjunction with the tamper-proof settings file under the server. Configuration options that are in the encrypted settings file, ctsrvr.set, cannot be overridden in the ctsrvr.cfg file. This keyword instructs the FairCom DB to ignore the command line arguments.

Default: Not present

See Also

COMPATIBILITY NO_CONFIG_FILE

This option is used in conjunction with the tamper-proof settings file under the server. Configuration options that are in the encrypted ctsrvr.set settings file cannot be overridden in the ctsrvr.cfg file. This keyword instructs the FairCom DB to ignore the standard configuration file, ctsrvr.cfg.

Default: Not present

See Also

COMPATIBILITY NON_ADMIN_SHUTDOWN

Allow non-ADMIN users to shut down the Server.

Default: ADMIN group only may shutdown.

COMPATIBILITY NONADMIN_FILBLK

Permits a non-ADMIN user to set a file block if the blocking user has the file opened with update permissions.

If a non-ADMIN user attempts to set a file block without this keyword and having update permissions on the file, then error LADM_ERR (589) will be returned.

COMPATIBILITY NONADMIN_QUIET

Permits a non-ADMIN user to call ctQuiet() to quiesce the server.

If a non-ADMIN user attempts to call ctQuiet() without this keyword, then error LADM_ERR (589) will be returned.

COMPATIBILITY NONADMIN_TRANSFER_FILE

If a non-ADMIN user attempts to call ctTransferFile() without this keyword, then error LADM_ERR (589) will be returned.

See Also

ENABLE_TRANSFER_FILE_API

COMPATIBILITY SQLIMPORT_ADMIN_PASSWORD

Instructs ctSQLImportTable() server-side logic to verify the admin password passed in the clear as parameter to the function instead of using the credential of the connected user to perform the link/unlink of the table to SQL.

Support has been added for ADMIN password encryption in V11.5 and later. This compatibility keyword instructs ctSQLImportTable() and the ctsqlimp utility (and ctsqlimp in FairCom RTG) to verify the admin password (if any).This keyword affects only FairCom RTG or Server SDK compiles calling ctSQLImportTable() on the server-side reusing the current session.

This server configuration option can be changed at runtime using the ctadmn utility or the ctSETCFG() API function.

To enable this option using ctadmn:
Choose option 10, then option 10, then enter:
COMPATIBILITY SQLIMPORT_ADMIN_PASSWORD
To disable this option using ctadmn:
Choose option 10, then option 10, then enter: COMPATIBILITY ~SQLIMPORT_ADMIN_PASSWORD

History

This option was renamed from COMPATIBILITY SQLIMP_ADM_PASSWD in V11.5 and later.

DEBUG_LOG

DEBUG_LOG <log file>

(optional) For aid in debugging TLS and X.509 configuration and certificate issues, write additional messages to the specified <log file>.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

FIPS_ENCRYPTION

FIPS_ENCRYPTION < YES | NO >

When specified, FairComDB uses the OpenSSL 3.0 FIPS module for encryption routines.

ENABLE_TRANSFER_FILE_API

ENABLE_TRANSFER_FILE_API < YES | NO >

The file transfer API function ctTransferFile() can be used to transfer a file to or from the server. By default, this function is disabled for security reasons and is restricted to a member of the ADMIN group. To enable this feature, this keyword must be specified to YES in the FairCom DB configuration file, ctsrvr.cfg. Calling this function without this keyword enabled will result in a NSUP_ERR (454, feature not supported) error returned to the client.

Default: NO

See Also

COMPATIBILITY NONADMIN_TRANSFER_FILE

EXPIRE_IN_DAYS

EXPIRE_IN_DAYS - Passwords to expire after this many days. Login attempts will fail with PWDEXP_ERR (1116). The ADMIN account is excluded from password expiration.

To set this value, see Setting password requirements.

Default: 0 (Never expires).

FILEDEF_SECURITY_LEVEL

FILEDEF_SECURITY_LEVEL <level>

With row-level, system-wide, data filters, it is necessary to protect the resource APIs, ADDRES(), UPDRES(), DELRES(), with safeguards against unauthorized modification of file definition resources such as IFIL definitions, conditional indexes, row-level filters, etc.

The c-tree Server can enforce different levels of File Definition Resource (FCRES) security when users modify data file definition resources such as the IFIL and DODA resources. The actual level of protection enforced for a given resource is determined by the FILEDEF_SECURITY_LEVEL configuration keyword.

The file definition resource security keyword must be specified in the c-tree Server configuration file ctsrvr.cfg. There are three different security levels for this keyword:

FILEDEF_SECURITY_LEVEL LOW

This is the lowest security setting. There is no protection as any user may add or delete data file definition resources. This setting may be used to keep the c-tree Server data compatible with legacy applications.

FILEDEF_SECURITY_LEVEL MEDIUM

This is the default security setting. Any user may add or delete data file definition resources, but the file must be opened exclusive. This default setting may be enough to keep the c-tree Server data compatible with most legacy applications.

FILEDEF_SECURITY_LEVEL HIGH

This is the highest security setting. A user must have file definition permission before a definition resource is added or deleted. The file must be opened exclusive. This setting is appropriate for applications that require the highest level of security and may cause compatibility problems with existing legacy applications.

Default: MEDIUM.

Note: This feature is NOT currently enabled.

KEY_EXCHANGE_PARAMS

KEY_EXCHANGE_PARAMS <filename>

(optional) - This is the name of a PEM-encoded file containing parameters for Diffe-Hellman key exchange. In V13 the default is to use the predefined group "ffdhe2048". These parameters may also be included in the SERVER_CERTIFICATE_FILE, which takes precedence over any parameters specified by KEY_EXCHANGE_PARAMS.

Example

KEY_EXCHANGE_PARAMS DHparams.pem

See Also

SERVER_CERTIFICATE_FILE

LOG_ENCRYPT

LOG_ENCRYPT YES

LOG_ENCRYPT enables encryption of the FairCom Server transaction logs adding an additional level of security for user data stored in the logs. Note: This keyword will not encrypt an existing file.

With LOG_ENCRYPT YES and ADVANCED_ENCRYPTION NO (or not specified) (V13) Not allowed.
With LOG_ENCRYPT YES and ADVANCED_ENCRYPTION YES logs are encrypted with AES 256-bit encryption. (Highest strength available.)

Before changing this setting, delete existing transaction logs (L0*.FCS), log templates (L0*.FCT), and start files (S0*.FCS).

Default: NO

LOGON_FAIL_LIMIT

LOGON_FAIL_LIMIT <attempts>

The optional limit on the number of consecutive failed logons that causes subsequent logon attempts to fail for LOGON_FAIL_TIME minutes. A logon which fails during this period returns LRSM_ERR (584).

Default: 0 (no limit)

See Also

LOGON_FAIL_TIME

LOGON_MUST_TIME

LOGON_FAIL_TIME

LOGON_FAIL_TIME <minutes>

The length of time logons are blocked after the logon limit is exceeded. A value of -1 indicates that there should be a permanent “log-on block” placed on a user until the Server ADMIN intervenes.

Default: 5

See Also

LOGON_FAIL_LIMIT

LOGON_MUST_TIME

LOGON_MUST_TIME <minutes>

The LOGON_MUST_TIME keyword, when set with a non-zero value, requires users to log on “at-least-once” within the defined time (e.g: LOGON_MUST_TIME 10080 means a user must logon at least once a week). If the time expires for a specific user, their profile will be deactivated, preventing access to the FairCom DB Server.

This keyword accepts any value up to 35,791,394 minutes, which is 24855 days or approximately 68 years.

The Server Administrator, or other ADMIN group user, must reset the user’s account once the time limit has elapsed.

Default: 0 (no limit)

See Also

LOGON_FAIL_LIMIT

LOGON_FAIL_TIME

MASTER_KEY_FILE

MASTER_KEY_FILE <filename>

Specifies a file from which c-tree reads the master encryption key. On Linux 2.6 and later kernel systems, c-tree uses the keyutils support to create a user-specific key in which the master key is stored. On other Unix systems, the master key is stored in a file on disk, with permissions set so that only the user that created the file can read it (permissions are set to 400).

The file (or user key on Linux) is encrypted using AES, however, the encryption is intended to only prevent casual inspection of the data when the file's contents are viewed. The permissions on the file are the defense against an unauthorized user reading the file.

The ctcpvf utility's -s option is used to create the master key file.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

See Also

ADVANCED_ENCRYPTION

MASTER_KEY_LIB

It is now possible to implement custom solutions for retrieving the advanced encryption master key from an arbitrary library. This feature eases the way the developers can customize the master key prompt.

The new ctsrvr.cfg configuration keyword, MASTER_KEY_LIB, takes a string defining the complete library name to load, for example:

MASTER_KEY_LIB maskeylib.dll

MASTER_KEY_LIB libmaskey.so

The master key library must link with the OpenSSL libraries that are used to secure the master key exchange and implement the following functions:

int ctGetSecretVersion(void) - returns the version of the master key library SDK used to implement it.
int ctGetSecret(ctGetSecretParams_t * GetSecretParams) - returns the master key encrypted by calling ctSecureMasterKey as a member of the ctGetSecretParams_t structure.

Both functions are called by the server code in ctcryp.c. If the version does not match or ctGetSecret returns something different than 0, the master key will not be loaded and the server will be shut down.

To correctly return the encrypted master key, the following must be called to encrypt the master key before returning 0 in ctGetSecret:

int ctSecureMasterKey(ctSecureMasterKeyParams_t *SecureMasterKeyParams)

MINIMUM_LENGTH

MINIMUM_LENGTH - Enforces a minimum length when setting a new password.

To set this value, see Setting password requirements.

Default: 15

V13.0 and earlier default: 0

Maximum: 64

NULL_STRING

NULL_STRING null

This option is used in conjunction with the tamper-proof settings file under the server. Configuration options that are in the encrypted settings file, ctsrvr.set, cannot be overridden in the ctsrvr.cfg file. The NULL_STRING keyword lets you define a symbol that represents a null string so that options can be blocked in the settings file without activating them.

LOCAL_DIRECTORY null

With the above entry example in the server settings file, LOCAL_DIRECTORY is blocked from use in the standard ctsrvr.cfg file but it is not activated.

This configuration option can include an environment variable name that will be substituted with its value when the configuration file is read.

Default: Not enabled

OPENSSL_ENCRYPTION

OPENSSL_ENCRYPTION NO|YES

FairCom now supports an updated AES algorithm based on the OpenSSL standard by default. Previous algorithm implementation can be restored by specifying in ctsrvr.cfg the keyword OPENSSL_ENCRYPTION NO.

This change is expected to provide security and, with internal testing, we have seen an up to 20% performance improvements.

This modification changes file passwords encrypted on the client-side for secure transmission to use OpenSSL.

PASSWORD_HASH

Password_Hash

In V13.1 onwards, allows setting the hash function used when generating new entries in the password database.

Set scaling factor using PASSWORD_HASH_DIFFICULTY.

Clients must support the hash algorithm configured by the server. Using PBKDF2 or ARGON2 requires the BouncyCastle third-party cryptography library for both JDBC and ADO.net.*

Options:

ORIGINAL - uses SHA2-512 hash scheme. This is the only supported option in V10-V13.0.3.

PBKDF2_SHA2_512 - uses PBKDF2(SHA2-512)

PBKDF2_SHA3_512 - uses PBKDF2(SHA3-512)

ARGON2_64MB - uses ARGON2id with 64MB RAM requirement

ARGON2_2GB - uses ARGON2id with 2GB RAM requirement

Defaults to ORIGINAL.

More information:

Changes to PASSWORD_HASH will only take effect when each user's password is next added or changed.

*NOTE: To use BouncyCastle with JDBC, either the main application must run:

Security.addProvider(new BouncyCastleProvider());

Or you can add it to your java runtime configuration:

conf/security/java.security

It will be resolved at runtime if the .jar is available:

security.provider.<n>=org.bouncycastle.jce.provider.BouncyCastleProvider

PASSWORD_HASH_DIFFICULTY

Password_Hash_Difficulty

In V13.1 onwards, sets a scaling factor for PASSWORD_HASH values.

The higher the value, the higher the computational cost on the client and server during password authentication, and the more resistant the password database will be to offline brute force attacks.

Defaults:

The default value for PASSWORD_HASH_DIFFICULTY depends on the value set for PASSWORD_HASH:

PASSWORD_HASH value	PASSWORD_HASH_DIFFICULTY default
Original	(Not used)
PBKDF2_SHA2_512	210000
PBKDF2_SHA3_512	210000
ARGON2_64MB	3
ARGON2_2GB	1

More information:

Changes to PASSWORD_HASH_DIFFICULTY will only take effect when each user's password is next added or changed.

Clients must support the hash algorithm configured by the server. This may be a consideration for users with older non-native clients that may have more limited encryption support.

Warning: Increasing PASSWORD_HASH_DIFFICULTY values may lead to noticeable login delays after user passwords are changed to use the new difficulty. Ensure you change a test account password after making this change and use that account for performance testing on a variety of anticipated client hardware, as clients without hardware support for the underlying cryptographic hash function(e.g. SHA512 or SHA3-512) may be significantly slower.

READONLY_SERVER

READONLY_SERVER Yes|No

This option is available only if licensed. This configuration option sets the server in read-only mode. By default, this option is off. To enable this option, add READONLY_SERVER YES to ctsrvr.cfg.

This option can also be enabled or disabled at run-time by calling the ctSETCFG() function or using the ctadmn utility: Select menu option 10 and then menu option 10 (Change the specified configuration option) again to change a server configuration option value.

Runtime changes to this setting are persisted as documented in Dynamic Configuration Changes Now Persisted in New ctsrvr.cfg Location and Default Additions.

The Replication Manager API command ctMemphisSetDBEngineReadOnly can be used to set this keyword.

REQUIRED_CLASSES

REQUIRED_CLASSES - Enforces a minimum number of character classes from lower case, upper case, numbers, and symbols when setting a new password.

To set this value, see Setting password requirements.

Default: 0

SERVER_CERTIFICATE_FILE

SERVER_CERTIFICATE_FILE <filename>

(required) - This is the name of a PEM-encoded certificate file containing the FairCom DB server certificate. It can also optionally include the private key. The server certificate can be self-signed.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

Example

SERVER_CERTIFICATE_FILE

SERVER_ENCRYPTED_STORE_FILE

SERVER_ENCRYPTED_STORE_FILE <store filename>

(optional) - If the private key is encrypted, use the ctcpvf utility to create an encrypted store file containing the pass phrase used to decrypt the private key and specify the name of the encrypted store file with this option.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

SERVER_PRIVATE_KEY_FILE

SERVER_PRIVATE_KEY_FILE <filename>

(optional) - If the private key is in a different file than the certificate, use this option to indicate the name of the file containing the private key. If this option is not specified, the private key is expected to be found in the file whose name is specified by the SERVER_CERTIFICATE_FILE option.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

SSL_CIPHERS

(optional) - If specified this option sets the encryption ciphers that are allowed to be used for encrypting the SSL connection. Ciphers are separated by a colon, (:). An exclamation point (!) explicitly disables a cipher. @STRENGTH sorts the list in order of encryption algorithm key length. For more information, see https://www.openssl.org/

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

Default SSL_CIPHERS (as of FairCom DB V12.5)

ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256

SSL_CONNECTIONS_ONLY

SSL_CONNECTIONS_ONLY <YES/NO>

(optional) - Default: NO. If this option is specified with a value of YES, only TCP/IP connections that use SSL/TLS are permitted to connect to the FairCom DB Server. Otherwise, both unencrypted and SSL-enabled TCP/IP connections are allowed.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

STARTUP_BLOCK_LOGONS

Prevents non-ADMIN user logons when the server is started. Only users in the ADMIN group are allowed to logon.

This feature allows the server to start for ADMIN purposes before authorizing access by non-ADMIN users. This could include creating or adjusting files, adjusting security options, or any other operations that require a functioning Server but are more conveniently accomplished when users are not connected to the Server.

Default: Allow logons

Allow SIGNAL_READY and ADMIN SQL Connections

A revision changes the behavior of STARTUP_BLOCK_LOGON for FairCom DB SQL Server so that SIGNAL_READY and the SQL listener threads run immediately. For SQL connections there is a choice of two behaviors when logons are blocked at server startup:

The configuration option STARTUP_BLOCK_LOGON YES only allows ISAM connections by the ADMIN user. SQL connections are rejected with the new error code 1158 (SQL_LOGON_BLOCKED_ERR, "SQL connection is blocked from logon due to server startup logon block").
The configuration option STARTUP_BLOCK_LOGON YES:ALLOW_SQL allows both ISAM and SQL connections by the ADMIN user.

Compatibility Change: This revision changes the behavior of STARTUP_BLOCK_LOGON for FairCom DB SQL Server.

Affected Components: c-treeSQL Server

VERIFY_CLIENT_CERTIFICATE

VERIFY_CLIENT_CERTIFICATE { YES | NO }

default NO - The server requires the client to supply an X.509 certificate. Unless X509_AUTHENTICATION is enabled, a username and password must still be supplied by the client.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

X509_AUTHENTICATION

x509_AUTHENTICATION { YES | NO }

Default is NO. If the client provides an X.509 certificate at logon, use it for authentication and database authorization rather than a username/password. By default, the only trusted CA is the root CA of the SERVER_CERTIFICATE_FILE. The SERVER_CERTIFICATE_FILE must provide a complete certificate chain beginning with the certificate of the server and ending with the root CA. The client X.509 certificate must be signed by the same root CA as the server. Combine with VERIFY_CLIENT_CERTIFICATE YES to make X.509 authentication mandatory.

Must be specified within SUBSYSTEM COMM_PROTOCOL SSL (see TLS).

NOTE: X.509 authentication is not supported in combination with LDAP authentication.

If X509_AUTHENTICATION is enabled, the following keywords are used to extract a username from the subject field of a successfully authenticated client X.509 certificate.

NOTE: All name matching is case insensitive, as is the resulting user name used for login.

X509_PATH <RDN> - Mandatory. Specifies which relative distinguished name (RDN) component of the full distinguished name (DN) to parse. This should be specified using the short form name, such as CN rather than CommonName. Typical values would be X509_PATH CN or X509_PATH emailAddress
X509_PREFIX <prefix> - Optional. If specified the value found in X509_PATH must contain this prefix, which is removed when forming the username.
X509_DELIMITER <suffix> - Optional. If specified the value found in X509_PATH must contain this suffix, which is removed when forming the username.
X509_REQUIREMENT_PATH <RDN> - Optional. If specified, the DN must contain this RDN.
X509_REQUIREMENT <value> - Optional. If specified, the RDN specified by X509_REQUIREMENT_PATH must match this value.

Example

Given the following server configuration:

X509_AUTHENTICATION YES

X509_PATH CN

X509_REQUIREMENT_PATH O

X509_REQUIREMENT faircom inc

A valid certificate with the subject:

Subject: C=US, ST=Missouri, O=FairCom Inc, OU=R&D, CN=John Doe/emailAddress=john.doe@faircom.com

would resolve the user name as “John Doe”.

A valid certificate with the subject:

Subject: C=US, ST=Missouri, O=Acme Inc, OU=R&D, CN=admin/emailAddress=john.doe@gmail.com

would not resolve a user name, as “Acme Inc” does not match the x509_REQUIREMENT “FairCom Inc” and an error would be returned.

Database Backup

DYNAMIC_DUMP_DEFER

Allows an administrator to reduce the performance impact of the dynamic dump.

DYNAMIC_DUMP_DEFER_INTERVAL

Specifies the number of 64 KB blocks that are written before the DYNAMIC_DUMP_DEFER sleep is performed.

PERMIT_NONTRAN_DUMP

Improves backup performance by skipping non-transaction files and pre-image files during a dynamic dump.

PREIMAGE_DUMP

Causes all PREIMG files, even those not in a dynamic dump, to be temporarily changed to TRNLOG files and all transactions to be changed from PREIMG mode to TRNLOG mode during the dump.

VSS_WRITER

When enabled, FairCom DB loads the Volume Shadow Copy Service (VSS) writer DLL (c‑treeACEVSSWriter.dll) and initializes the VSS writer when the server starts.

Diagnostics

DIAGNOSTICS DYNDUMP_LOG

Generates dynamic dump status info, sending progress entries during each dynamic dump to CTSTATUS.FCS.

DIAGNOSTICS VSS_WRITER

Enables VSS writer to log diagnostic messages to CTSTATUS.FCS.

DYNAMIC_DUMP_DEFER

DYNAMIC_DUMP_DEFER <milliseconds>

When a dynamic dump runs, the disk read and write operations of the backup process can slow the performance of other database operations. This option allows an administrator to reduce the performance impact of the dynamic dump.

milliseconds is the time that the dynamic dump thread will sleep after each write of a 64KB block of data to the dump backup file.

An application developer can also use the c-tree ctSETCFG() API function to set the DYNAMIC_DUMP_DEFER value. For example, the following call specifies a 10-millisecond DYNAMIC_DUMP_DEFER time:

ctSETCFG( setcfgDYNAMIC_DUMP_DEFER, "10" );

The DYNAMIC_DUMP_DEFER value set by a call to ctSETCFG() takes effect immediately, so this API call can be used by administrators to adjust the speed of a running dynamic dump depending on the amount of other database activity.

Note: The maximum allowed DYNAMIC_DUMP_DEFER time is 5000 milliseconds, set at compile-time. If a value is specified that exceeds this limit, the DYNAMIC_DUMP_DEFER time is set to DYNAMIC_DUMP_DEFER_MAX.

The FairCom DB Administrator utility, ctadmn, was also updated to support the dump sleep time option to change this value at run time. The "Change Server Settings" menu is available from the main menu of the ctadmn utility.

See Also

DYNAMIC_DUMP_DEFER_INTERVAL

DYNAMIC_DUMP_DEFER_INTERVAL

DYNAMIC_DUMP_DEFER_INTERVAL <blocks>

<blocks> specifies the number of 64 KB blocks that are written before the DYNAMIC_DUMP_DEFER sleep is performed.

The DYNAMIC_DUMP_DEFER option causes the dynamic dump to pause for the specified number of milliseconds each time it writes 64 KB of data to the dynamic dump stream file. For large backups, even the smallest DYNAMIC_DUMP_DEFER value of 1 millisecond adds significant time to the dynamic dump. For example, 100 GB = 1600000 * 1 ms. = 1600 seconds of additional time.

Note: If a value greater than 5000 is specified for DYNAMIC_DUMP_DEFER_INTERVAL, the value is set to 5000. If a value less than 1 is specified, the value is set to 1.

This option can be set by the ctSETCFG() API function. A new menu option to set this value has been added to option 10 of the FairCom DB Server Administration (ctadmn) menu.

Example

DYNAMIC_DUMP_DEFER_INTERVAL 16

Causes the DYNAMIC_DUMP_DEFER sleep to occur after every 64 KB * 16 = 1 MB of data written to the dump stream file.

DYNAMIC_DUMP_SYNC_INTERVAL

To reduce the amount of cache the file system devotes to the dynamic dump file, a server configuration keyword allows specifying a periodic file sync operation on the backup generated by dynamic dump approximately every <size> bytes written.

DYNAMIC_DUMP_SYNC_INTERVAL <size> [<scale>]

<size> must be less than 4GB. Default size is 0, which results in no explicit sync calls on the backup file. The minimum <size> is 1 MB. Non-zero values less than 1MB are increased to 1MB.

This value can be changed at runtime using menu option 10 in the ctadmn command-line utility.

PERMIT_NONTRAN_DUMP

PERMIT_NONTRAN_DUMP < YES | NO >

Improves backup performance by skipping non-transaction files and pre-image files during a dynamic dump.

Default: YES

When set to NO, non-transaction files and pre-image files are skipped during a dynamic dump even if they are included in the !FILES section of the dynamic dump configuration script. The exception is if PREIMAGE_DUMP is set to YES, then pre-image files continue to be included in the dump.

PREIMAGE_DUMP

PREIMAGE_DUMP <YES | NO>

When enabled, all PREIMG files, even those not in a dynamic dump, are temporarily changed to TRNLOG files to be compatible with the upgraded transactions, and all transactions are automatically changed from PREIMG mode to TRNLOG mode during the dump. PREIMG files opened or created in the middle of the dump are also temporarily promoted from PREIMG files to TRNLOG files. All promoted files are restored to their PREIMG status at the conclusion of the dynamic dump.

The dynamic dump script file accepts a !DELAY parameter whose argument is the number of seconds to wait for a transaction to complete before aborting it in order to permit the start of a dynamic dump. When the PREIMAGE_DUMP facility is used, the !DELAY parameter is effectively ignored if a long PREIMG transaction begins prior to the dynamic dump. This means the dump will not begin until all current transactions complete. See Dynamic Dump and Advanced - Faster Auto-Recovery for additional information.

Default: NO

See Also

AUTO_PREIMG

AUTO_TRNLOG

AUTO_TRNLOG_LIGHT