FairCom — Backward Compatibility

Compatibility keywords are frequently used to provide extended "non-standard" FairCom DB behavior or backward compatibility with previous behaviors as server functionalities are changed. Typically, these keywords should only be used under the specific advice of a FairCom engineer as they may negatively impact the functionality of your application. Frequently, these keywords are used as the result of a required change in server behavior resulting in incompatibility with legacy applications. Please don't hesitate to contact your nearest FairCom office should you have any questions regarding use of any of these keywords.

WARNING: The keywords in this section should be used ONLY on the advice of your application developer. They can seriously alter the operation of the FairCom DB Server.

COMPATIBILITY 6BTRAN_NOT_DEFAULT

Six-byte transaction number support is the default with the c-tree Server. For compatibility with existing older applications, this keyword is provided to disable this default.

See Also

COMPATIBILITY EXTENDED_TRAN_ONLY

DIAGNOSTICS EXTENDED_TRAN_NO

COMPATIBILITY ABORT_ON_CLOSE

Disable “deferred close” capability. Force a transaction abort when a file altered in a transaction is closed before the transaction is committed.

Default: Defer close

COMPATIBILITY BATCH_SIGNAL

This feature causes sysiocod to be set to BTNO_COD (-844) on calls to BATSETX()/BATSET() that result in the current batch being closed. If the client application finds sysiocod set to BTNO_COD, then the application does not need to issue a BAT_CAN call before issuing a new batch request.

COMPATIBILITY BATCH_UTRFMKEY

Documentation states that keys returned with BATSET() called with BAT_RET_KEY are returned in native index order, however, uTFRMKEY() was called to put the keys back into the form of the client. While this behavior was changed to match the documentation, this configuration option reverts to this prior behavior.

COMPATIBILITY BLOCK_DDSFM_CREATE and BLOCK_DDSFM_DELETE

The FairCom DB Server can block adds and deletes of superfile members during the course of a dynamic dump using these two server configuration keywords.

COMPATIBILITY BLOCK_DDSFM_CREATE

Blocks superfile member creation during a dynamic dump. Attempts to create a superfile member return DDCR_ERR (740) with this keyword activated.

COMPATIBILITY BLOCK_DDSFM_DELETE

Blocks superfile member deletion during a dynamic dump. Attempts to remove a superfile member return DDDR_ERR (741) with this keyword activated.

Note: An application may create or delete superfile members in a superfile host waiting to be dumped while the overall dump is going on. Once the dynamic dump begins dumping the superfile host, blocked operation cannot occur until the end of the entire dump, not just the end of the dump of the superfile host itself. Therefore, the last superfile host listed in the dump script file list will have the shortest blocking period.

Default: Do not block creates and deletes

COMPATIBILITY CHAR_SCHSEG

Schema segments automatically map a c-tree field type to a key segment type (mode). Starting with V8.14, CT_CHAR fields are now treated as a signed, one-byte integer, whereas, previously, they were treated as a REGSEG segment mode. CT_CHARU remains treated as an unsigned one-byte field.

To revert this change for compatibility with versions prior to V8, use the COMPATIBILITY CHAR_SCHSEG server configuration keyword.

COMPATIBILITY CHAR_SCHSEG

COMPATIBILITY CHECKPOINT_OVERLAP

A change to the checkpoint logic now causes checkpoint requests to wait for the completion of an in-process checkpoint before beginning even flushing operations. If the checkpoint is called from the CTCHKPNT() API, and it finds an in-process checkpoint, then it simply returns with NO_ERROR and sets sysiocod to OCHK_COD (-885). If an internal checkpoint request must wait, a counter is incremented; and the total count is reported in CTSTATUS.FCS at the end of the final checkpoint (if it is nonzero). This keyword disables the new behavior.

COMPATIBILITY CHKPNT_FLUSHALL

Note: This keyword is no longer recommended as of March 2019.

Forces all checkpoints but the final checkpoint to use a single system call to flush file system buffers instead of individual calls for each file to ctsave(). The motivation is that a call to flush the buffers for a specified file may require significant time whether or not the file has many updated pages in the file system cache; hence one flush of all the buffers may be faster than repeated calls for individual files. For Unix systems this implementation automatically invokes sync() unless another definition is already in place. No other platform has a default definition.

COMPATIBILITY CHKPNT_MUTEX_REL

Forces transaction related mutexes to be released just before the checkpoint is written to disk instead of after. This should modestly reduce the time these mutexes are held by the checkpoint. It is safe because the mutexes are not released until the checkpoint has been completely composed, and its space in the log file is assigned.

Note: For FairCom internal use only.

COMPATIBILITY CLSFIL_ISAM

By default, low-level commands are prevented from closing files opened with ISAM-level opens. This keyword is used to revert to behavior in which low-level commands were allowed to close files opened with ISAM-level opens.

COMPATIBILITY CLSFIL_UNBLOCK

A FCLS_ERR (24) could occur on a file unblock that occurred within a transaction for a file whose closure had been deferred until the transaction ends. Previous behavior was that the file had to be closed prior to the unblock call. With this configuration option it is permissible for the file blocker to have the file open at the time of the unblock call. The open may be explicit or, it may be as a result of a pending close.

COMPATIBILITY COMMPORT5000

Sets the default base value to add to the SERVER_NAME ASCII sum for determining the TCP/IP port number.

For example, the sum of the ASCII characters in FAIRCOMS equals 597. Added to the default base of 5000 results in a default TCP/IP port number of 5597.

See Also

COMPATIBILITY COPY_FILE_OPEN_SHARED

Restores original shared open behavior for file copy operations. An earlier revision changed ctCopyFile() to open files to copy in exclusive mode such that a clean copy can be made. However, if a customer is able to use a potentially corrupted file copy and are prepared to handle the implications of allowing the shared open copy, this option can restore earlier behavior. When this option is in effect, ctCopyFile() opens the files in shared mode instead of exclusive mode.

The option can be set in the server configuration file and can be changed at runtime. (Remember that the way to do this at runtime is to use ctadmn's options 10, 10, and the configuration open, or the ctSETCFG() API function. To turn off a compatibility keyword you use: COMPATIBILITY ~keyword.)

COMPATIBILITY CTREE_RWLOCK

Reader/writer lock support for Windows systems was implemented, and the mutex on the memory file hash lists was changed to a reader/writer lock when this support is enabled at compile time.

When the c-tree Server starts up on a Windows system, it checks if the system supports reader/writer lock functions. If so, it uses the system's reader/writer lock functions. If not, it uses c-tree's reader/writer lock functions. This option can be used to force FairCom DB to use custom reader/writer lock support instead of Windows' reader/writer lock support even if the Windows system supports reader/writer locks, for example, older versions not supporting this synchronization technique.

The following CTSTATUS.FCS messages are logged indicating the supported reader/writer locks:

c-tree reader/writer lock support enabled (keyword specified)

c-tree reader/writer lock support enabled (no O/S support)

Windows native reader/writer lock support enabled

COMPATIBILITY DIR_BUF_RQS

During performance analysis it was noticed that get index node routine was performing retries when we expect that the node is already in a buffer. Changes were made to (a) avoid retries when the node holds the desired buffer and (b) skip the queue and directly request the holding semaphore. This option disables this new approach.

See Also

COMPATIBILITY NO_IDXBUFBLK

Note: This is intended for internal FairCom testing only.

COMPATIBILITY DISABLE_BACKGROUND_CLEANUP

Disables TRAN_INDEX_FLUSH_SEC.

See Also

TRAN_INDEX_FLUSH_SEC

COMPATIBILITY DUPL_ERR_FATAL

Rebuild behavior was changed such that in the event of a DUPX_ERR, KDUP_ERR, KSRL_ERR, or some error other than DUPJ_ERR or DUPL_ERR, one or more of the indexes will have their corrupt flags turned on. As it was, DUPJ_ERR will not cause corrupt flag to be turned on. This option forces the index corrupt flag to be set.

COMPATIBILITY ENCRYPT128

c-tree's advanced encryption uses a master encryption key to encrypt the following items using the AES cipher:

The security resource in c-tree data and index files that use advanced encryption, and
The encryption key in the transaction log file header when using advanced log encryption.

The length of the master key was increased from 128 bits to 256 bits. FairCom DB can still access files that created with the previous 128-bit master key. This keyword reverts to the original 128-bit master key usage for encryption.

Attempting to open a file created using a 256-bit master key fails with error ENCK_ERR (952) when 256-bit key support is disabled.

See Also

COMPATIBILITY LOG_ENCRYPT128

COMPATIBILITY ESTIMATE_SCALE

SQL index selection relies heavily on the precision of ESTKEY() to pick the best index. Prior behavior was for ESTKEY() to return the estimate to within 1/10 percent. New behavior increases the ESTKEY() precision with the number of keys in the file to attempt to maintain a precision of about 100 records. As this is done using a binary search on the high and low keys, it requires 2 additional index searches for each doubling of the file size. This option reverts to the prior behavior.

COMPATIBILITY EXACT_FILE_NAMES

Force different references to the same file to use the same names. For example: C:\data\temp.dat and \data\temp.dat would be considered different even if they referred to the same file.

Default: Allow different names

COMPATIBILITY EXTENDED_TRAN_ONLY

This keyword forces a R6BT_ERR (745) on an attempt to create or open a non-extended-transaction-number file. A read-only open is not a problem since the file cannot be updated.

See Also

COMPATIBILITY 6BTRAN_NOT_DEFAULT

DIAGNOSTICS EXTENDED_TRAN_NO

COMPATIBILITY FAILED_TRAN_IO

When transaction processing is in effect, a failed write operation for transaction controlled files terminates the server (or stand-alone application) with error L60 (ctcatend). This error can occur with any write operation errors of buffer and cache pages, and header write operations for transaction controlled files. If an L60 error does occur, the Mx value will be important in determining the precise location of the error.

This keyword restores behavior prior to this change.

COMPATIBILITY FILE_CREATE_CHECKPOINT

The old behavior was to generate a checkpoint if a TRNLOG (but not TRANDEP) file is about to be deleted or renamed after a file, any file, is created but with no intervening checkpoint. That is, the sequence Create FileA, Checkpoint, Create FileB, Delete FileA would cause a second checkpoint during the delete processing of FileA even though the FileA create preceding the delete occurred with an intervening checkpoint. With new default behavior a checkpoint is generated only if the file to be renamed or deleted was the one created without an intervening checkpoint. This keyword option disables this behavior.

COMPATIBILITY FILE_DESCRIPTOR_LIMIT

Running FairCom Server with insufficient file descriptors can lead to errors opening files. See FILES.

This compatibility keyword overrides the behavior described in FILES in case it is not convenient for a system administrator to set the file descriptor limit for the FairCom Server process to the required value or it is not desired to decrease the FILES or CONNECTIONS settings.

Note: Using this keyword is not recommended. It is provided for backward compatibility or short-term use until the administrator is able to increase the file descriptor limit for the FairCom Server process.

A message is logged to CTSTATUS.FCS explaining that the COMPATIBILITY FILE_DESCRIPTOR_LIMIT configuration option can be used to allow the server to start in this situation:

Tue Apr 29 12:23:44 2014

- User# 00001 ERROR: The hard limit on file descriptors available to this process (500) is lower than the database engine's file descriptor requirement (1043). Either increase the hard limit, or decrease the FILES or CONNECTIONS settings.

Tue Apr 29 12:23:44 2014

- User# 00001 Note: The configuration option COMPATIBILITY FILE_DESCRIPTOR_LIMIT can be used to allow c-tree Server to start even if the file descriptor limit is insufficient. However, this can lead to errors opening files.

See Also

FILES

COMPATIBILITY FILELIST_GROWTH

When a key search occurs on partitioned indexes that are not ordered as the partition key, all of the partition members must be opened, which can take a significant amount of time if the number of partitions is large. One operation where unnecessary time was spent was resizing (allocate new + copy + free old) the users file control block. Previously, this grew in increments of 32 (MAXMEMB + 1) files. With partition files used through SQL, thousands of files may be opened by a single query. A change was made which increased the rate at which this array grows by doubling in size up to 2 thousand files. This change reduced the total query time by 4% in a case with 1000 files. This will affect memory usage, as it will now increase in increments of up to 500 KB (for large numbers of files already opened) versus 8KB with the old approach. This option reverts to the old approach.

COMPATIBILITY KEEP_EMPTY_NODE_ON_READ delete queue when read from disk

COMPATIBILITY KEEP_EMPTY_NODE_ON_READ

Performance issues have been reported due to a recent code change that unexpectedly caused situations where empty nodes could remain in an index until someone modified that node or until the index was rebuilt. Historically, empty index nodes have been processed by the FairCom Database Engine through an "index node pruning" strategy. When the number of keys in the node is zero, the node is added to a "delete node queue" to be removed by a background thread.

The root cause of this issue involves a situation where node pruning could fail if a table lock was outstanding, leaving empty nodes in indexes and causing performance issues. If a node was in this unusual empty state (due to a failed delete node pruning operation, or a server crash), it could remain in that state until someone modified that node or the index was rebuilt. This revision adds a request to prune a node whenever an empty node is read from disk, which should alleviate this problem.

FairCom DB's delete-node thread prunes empty nodes from index files in the background. It densely packs key data for optimal index performance. To prevent index corruption, it runs during idle times and opens the index file exclusively, which prevents external processes from opening it.

To disable the "index node prune" feature, for example to allow an external process to open an index file, use the following keyword:

COMPATIBILITY NO_DELNOD_QUEUE

The above keyword prevents the delete-node thread from running, which prevents automatic index optimization, however, it allows external processes to open the index file.

Another keyword, COMPATIBILITY KEEP_EMPTY_NODE_ON_READ, prevents empty nodes read from disk being added to the delete queue. This is the V2/V11 and earlier behavior.

COMPATIBILITY LFL_WAIT

Eliminates the lflsema SHORTWAIT loop, and uses a call with the WAIT parameter.

Note: For internal FairCom use only.

COMPATIBILITY LFW_ADAPTIVE

Replaces the commit delay ct_udefer() call with a loop that permits the "prime" cohort to periodically wake up, after a much smaller defer time, and check for commit delay progress. COMMIT_DELAY_USEC determines the length of the "short" sleep when this option is enabled.

Note: For internal FairCom use only.

COMPATIBILITY LOCK_CACHE

The FairCom Database Engine attempts to lock all data and index cache pages into physical memory, using the mlock() function on Unix systems. In the event this fails, the server will fail to initialize and a message is printed to CTSTATUS.FCS:

LOCK_CACHE: mlock() error 12

This is only implemented for Unix (which includes Linux).

COMPATIBILITY LOCK_HEADER

Some header calls always acquire the header semaphore. It is possible the associated lock table entries are not needed since the semaphore provides access control. With new default behavior, lock calls skip the lock table entries, only acquiring the header semaphore. This option will restore the lock table entries.

COMPATIBILITY LOG_ENCRYPT128

c-tree's advanced encryption uses a master encryption key to encrypt the following items using the AES cipher:

The security resource in c-tree data and index files that use advanced encryption, and
The encryption key in the transaction log file header when using advanced log encryption.

The length of the master key was increased from 128 bits to 256 bits. FairCom DB can still access files that were created with the previous 128-bit master key. This keyword reverts to the original 128-bit master key usage.

Attempting to read a transaction log encrypted with a 256-bit encryption key when only 128-bit log encryption support is enabled fails and c-tree logs the following message to CTSTATUS.FCS:

Cannot read existing encrypted logs. Log is using 256-bit encryption key but server is configured for 128-bit encryption support.

See Also

COMPATIBILITY ENCRYPT128

COMPATIBILITY PUTHDR_COMMIT

A KDUP_ERR (2, duplicate key found) could be encountered while using a SRLSEG value as the key. The SRLSEG values (sernum1, sernum2) are stored in the c-tree file header. Header values are handled differently than ordinary record contents, as they are not maintained under typical transaction control; we do not want to hold a lock for the duration of a transaction for these types of values.

PUTHDR() behavior was changed such that on commit, the header value is not reset to the new value. Abort behavior remains the same. This also affects numrec1, numrec2, phyrec1, and phyrec2 header values.

For backward compatibility, the original behavior can be restored by adding this keyword to the configuration file.

COMPATIBILITY MEMORY_FILE_SKIP_FREE

When a memory file is actually closed (which is not necessarily when the last user closes the file because of the ctKEEPOPEN attribute), the individual records or nodes are returned to the memory heap.

This keyword skips the free of memory at server shutdown for the individual memory records for memory files still open and assume that the server termination will (automatically) return the memory to the system. This may quicken the shutdown in the case of a large number of memory file records and/or nodes.

COMPATIBILITY MEMORY_LIMITS

Legacy V7 backward compatibility option that enforces system or user memory limits.

COMPATIBILITY MINIMUM_INDEX_BUFFERS

Historically FairCom Server has automatically set the index cache size so that the number of index buffers is at least 3 times the maximum number of allowed connections. This ensures that sufficient index buffers are available to handle index operations (key inserts, retrievals, and deletes) if all supported connections are performing index operations at the same time.

This minimum requirement means the memory used by FairCom Server could be higher, perhaps much higher, than the amount requested in the server configuration file. For example, using the default settings of PAGE_SIZE 8192 and IDX_MEMORY 100 MB and the number of allowed connection is larger than 4267, IDX_MEMORY becomes larger than requested.

The list below shows examples for PAGE_SIZE 8192, showing the original index cache memory requirements:

CONNECTIONS	IDX_MEMORY
8	0.2 MB
16	0.4 MB
32	0.8 MB
64	1.5 MB
128	3.0 MB
256	6.0 MB
512	12.0 MB
1024	24.0 MB
2048	48.0 MB
4096	96.0 MB
8192	192.0 MB
16384	384.0 MB
32768	768.0 MB

In V11.5 and later, instead of increasing the number of index buffers to the required amount, if the IDX_MEMORY value is smaller than the required amount of index buffers, c-tree Server limits the number of concurrent threads that can be performing index operations so that concurrent requests for index buffers do not exceed the allocated number of index buffers.

Note: This is a Compatibility Change.

FairCom Server supports a configuration option to restore the old behavior in case it is needed. Specifying COMPATIBILITY MINIMUM_INDEX_BUFFERS in ctsrvr.cfg restores the old behavior. When this option is used, at startup, FairCom Server logs the following message when it increases IDX_MEMORY due to minimum index buffer requirements based on the number of supported connections:

- User# 00001 Setting IDX_MEMORY to 53805056 due to concurrent connection requirements.

COMPATIBILITY MULTI_PROCESSOR

Legacy option to ensure proper statistics on multiple-processor hardware.

COMPATIBILITY MULTIOPN_*

Se the default for enhanced locking control for files opened multiple times in the same connection. The system-level default can be controlled by using one of the following configuration keywords which sets the behavior accordingly to their names.

COMPATIBILITY MULTIOPN_DIFUSR

COMPATIBILITY MULTIOPN_SAMUSR_M

COMPATIBILITY MULTIOPN_SAMUSR_1

Starting in V10.3, FairCom DB supports opening the same file multiple times in the same connection assigning a different file number to each file or, in FairCom DB API, a different file handle. This can be useful in situations where you want to allow the same file to be opened twice by the same thread with different locking attributes applied to each thread.

Each of these sibling files is referred to as a "co-file." For example, if the file customer.dat is opened in the same connection using file numbers 5 and 10, then we say that file 5 is a co-file of file 10, and vice versa.

In this case there are considerations about how locks interact within the same connection when operating using different co-files. For example, if a write lock is acquired on a record R using file number 5 within the same connection, what is the behavior of trying to acquire a lock on R using co-file number 10?

In this example, before this enhancement, FairCom Server behaved as follows:

The lock on R issued with co-file number 10 succeed and is considered a "secondary lock", while the lock acquired first (using file number 5) is considered "primary."

The difference in the locks manifests itself during calls to unlock the record: If the primary lock is unlocked first, then the primary lock and all the corresponding locks on co-files are removed. But if a secondary lock is unlocked before the primary lock is unlocked, then only the secondary user lock is removed; and the primary lock is maintained.

Any other connection saw the record locked until the primary lock was released.

This previous behavior has been maintained and it is the system-level default behavior.

It is now possible to configure the behavior choosing among 4 different options:

NODIFUSR: The default as described above.
DIFUSR: Locks on co-files are considered as acquired from a different connection, so the lock on R issued with co-file number 10 will fail.
SAMUSR_M: Locks on record R on co-files are considered as the same lock acquired on the same file, so lock on R issued with co-file number 10 succeeds. As soon as the lock is released in one of the co-files that successfully requested the lock, the lock is released. Therefore, before acquiring the lock on R using file number 10, the lock can be released only using file number 5, but after acquiring the lock on R using file number 10, the lock can be released either by using file number 5 or 10.
SAMUSR_1: Locks on record R on co-files are considered as the same lock acquired on the same file, so lock on R issued with co-file number 10 succeeds. As soon as the lock is released in one of the co-files (whether or not the lock was requested using the co-file) the lock is released. Therefore, even before acquiring the lock on R using file number 10 the lock can be released either by using file number 5 or 10.

Recursive locks are not supported for co-files. An attempt to open a co-file when recursive locks are pending on the underlying file will fail with the error MUOP_RCR (998). An attempt to issue a lock on a co-file with the ctLK_RECR bit set in the lock mode will fail with the error MLOK_ERR (999).

Read locks behave in a manner consistent with write locks. The notable issues are:

With DIFUSR, read locks can be issued for different co-files; and unlocking one co-file's read lock does not remove the read lock from any other co-files that requested the read lock.
With DIFUSR, a read lock on a co-file cannot be promoted to a write lock if other co-files have read locks; a non-blocking write lock will fail with DLOK_ERR (42) and a blocking write lock will fail with DEAD_ERR (86).
With SAMUSR_*, read locks can be issued for different co-files, and unlocking one co-file read lock unlocks all the co-file read locks.
With SAMUSR_*, read locks can be promoted to write locks as long as no other threads have also acquired read locks.
With SAMUSR_1, a read lock on a co-file can be unlocked using another co-file's file number even if no lock has been issued using the other co-file number.

The system-level default can be controlled by using one of the following configuration keywords which sets the behavior accordingly to their names.

COMPATIBILITY MULTIOPN_DIFUSR
COMPATIBILITY MULTIOPN_SAMUSR_M
COMPATIBILITY MULTIOPN_SAMUSR_1

A connection can override the system-level default for all open instances of a file by calling:

PUTHDR(datno, mode, ctMULTIOPNhdr)

Where mode is one of the following:

ctMULTIOPNnodifusr
ctMULTIOPNdifusr
ctMULTIOPNsamusr_M
ctMULTIOPNsamusr_1

If no PUTHDR call is made, the system-level default is used for that connection's instances of the file. When a file is opened, if that connection already has the file open, the newly opened file inherits the MULTIOPN setting of the already-open file instance. An attempt to change the setting so that one instance of the file would be inconsistent with the others will fail with error MOFL_ERR. A file's MULTIOPN state can only be changed if it is the first open instance of the file and it has no pending locks.

COMPATIBILITY NLM_DEFER_THREADSWITCH

This option can improve the performance of the FairCom DB Server for Novell at the cost of decreased performance in other processes. Consult with your application developer and Novell system administrator to determine if this switch is appropriate for your system

Default: Defer

COMPATIBILITY NLM_LONG_FILE_NAMES

Instructs the FairCom DB Server to use OS/2 namespace support, provided OS/2 namespace support is enabled on the working volume. If the keyword is not used, or if the volume does not support OS/2 namespace, long file names are not supported. FairCom recommends that when using long file name support all volumes provide OS/2 namespace support to prevent an error. This keyword is only required by the NLM FairCom DB Server and is ignored in all other versions.

Default: Not supported

COMPATIBILITY NO_ADREXP_CHECK

r-tree virtual fields should be in "correct" order where correct means no forward references. Using a forward reference results in various errors, resulting in potential server crashes. Address expressions involving DODA fields can be distinguished from address expressions involving other virtual fields, and a state variable is now set in the step of the script where the address expression is executed such that during the parsing phase, an attempt to evaluate an address expression based on a DODA field will return an error instead of crashing the server. This keyword reverts this change for testing purposes only.

COMPATIBILITY NO_ATODEP

When indexes have to be recreated during a rebuild or compact call without XCREblk support, the indexes will lose their TRANDEP/RSTRDEL attributes. The rationale to change this behavior is to permit such calls to automatically assign the indexes the same TRANDEP/RSTRDEL attributes as the associated data file. A new client, even with the new default behavior turned on will not exhibit this new behavior if RBLIFIX8()/CMPIFILX8() are called since the XCREblk's passed to rebuild or compact will prevail. This option reverts this behavior.

COMPATIBILITY NO_AUTO_SKIP

To avoid requiring SKIP_MISSING_FILES when TRANDEP files are in use, a new default behavior effectively treats TRANDEP files as though SKIP_MISSING_FILES is turned on, however, for files without TRANDEP activities, recover, rollback, or roll-forward may still terminate execution if unexpected missing files are encountered.

This keyword disables this support to revert to the original behavior.

Note: It is possible that an unexpected FNOP_ERR error can still occur for a TRANDEP file, however, this change should greatly reduce the number of unexpected FNOP_ERR’s.

See Also

COMPATIBILITY NO_CLSTRAN_OPEN

COMPATIBILITY NO_BAT_FXDCMP

In V11.0 and later, batch operations were adjusted to return fixed-length compressed records as fixed-length records. Previously, they were returned as variable-length records. Add COMPATIBILITY NO_BAT_FXDCMP to the ctsrvr.cfg file if you need to restore the previous behavior.

COMPATIBILITY NO_BLOCK_KILL

Disable the ADMIN ability to kill currently connected clients.

Default: Allow kill

COMPATIBILITY NO_CHECKFIX

The ADDREC() routine checks to see if a record begins with either a delete flag (0xFF) or a resource mark (0xFEFE). If so, the ADDREC() returns FBEG_ERR (553). Behavior prior to 2002 did not make this check and this option reverts the newer behavior.

COMPATIBILITY NO_CHKMBRNAMLEN

The length of a c-tree Superfile name is restricted because the Superfile directory index has a restricted key length. A default behavior checks when the Superfile member name is restricted due to small page sizes: if the Superfile name is truncated in the Superfile member directory index, then the create of the member will fail with SNAM_ERR (418). Without this new behavior, member names that would be truncated to the same string would cause DOPN_ERR (19) and/or KOPN_ERR (18) errors.

To address the situation where names may have been truncated but no conflicts arose and would now get error SNAM_ERR at create time, COMPATIBILITY NO_CHKMBRNAMLEN is available to revert to the original behavior.

COMPATIBILITY NO_CLSTRAN_OPEN

FairCom DB is designed to permit automatic recovery and rollbacks to properly handle deleted and renamed transaction dependent files (TRANDEP) without requiring the SKIP_MISSING_FILES configuration to be enabled. Situations can occur where a TRANDEP file has been physically deleted (by a system call, for example) that will interfere with a transaction rollback during automatic recovery.

If SKIP_MISSING_FILES is active, then the rollback should succeed even with the missing file, however, should only ignore missing files for which there is a transaction controlled explanation such as a file delete or file rename.

The recovery process was modified such that a CLSTRAN log entry will attempt to open the file (if it is not already opened). Upon detecting a CLSTRAN entry triggers putting a missing file on the list of missing files. To revert to the original behavior, the server configuration keyword, COMPATIBILITY NO_CLSTRAN_OPEN is provided.

See Also

COMPATIBILITY NO_AUTO_SKIP

COMPATIBILITY NO_COMMIT_READ_LOCK

Disables commit read lock support for backward compatibility. FairCom DB V9 introduced commit read lock support as a default. Without explicit read or write locks, it is possible to have a partially updated record buffer returned in a high transaction volume environment. The returned record buffer could consist of partial old data, and partial newly updated data from a transaction commit operation from a concurrent thread. While the occurrence of this event is extraordinarily rare (on the order of one in a million record reads) it is clearly important in a high volume situation to prevent these “dirty” record reads. Commit Read Locks enable an implicit, high performance, low-level record lock ensuring consistent data record reads in high volume transaction environments.

When explicit read or write locks are enforced in the c-tree application then this feature is not required, as the transaction commits will ensure consistent data reads.

A new type of lock entry, CMT, was added to the FairCom DB Server lock table. These CMT entries are guaranteed to be very shortly held (only for the duration of the internal read operations). Under high volume server operations this new lock type may occasionally be observed as "forcei cmtlok" in a ctLOKDMP() (LockDump()) output.

Commit Read Lock Errors

An attempt to update a record without an explicit lock with the commit read logic is active results in error CMLK_ERR (768). This lock error fails the transaction commit. CULK_ERR (769) indicates an unexpected failure during the removal of a commit lock. This is an extremely rare internal FairCom DB Server error and should not be encountered in standard usage.

Performance Considerations

The commit read lock uses polling logic to permit a thread to retry a commit lock when it cannot be acquired. For example, if the updater has already acquired a write commit lock, a reader will be denied its request for a read commit lock, and the reader will retry. Particularly, for large transactions, the retries can consume a large amount of CPU time.

To avoid this high CPU utilization a more adaptive retry logic was implemented. If several retries fail in succession, the retry call is changed before each retry from a defer time of 0, to a defer time of ctredcmtdfr where ctredcmtdfr defaults to 10 milliseconds.

To provide an additional tuning mechanism for this retry value, the following configuration keyword COMMIT_LOCK_DEFER_MS <defer time in milliseconds>

The length of the defer value can be varied from zero to 100 milliseconds.

Internal tests demonstrated the effect of this change on CPU utilization was dramatic as a reader attempted to retry its read commit lock. Of course, actual performance increases will be variable, depending on any particular server environment. The trade-off with this method is introducing an unnecessary defer (i.e. if the next retry without a non-zero defer would have succeeded). In practice, this was not found to impede performance.

Commit write locks held by the transaction (i.e., locks that block read attempts during the actual commit process) are held during the entire commit. This has no direct impact upon the transaction commit, however, can cause longer delays for a read attempt when the transaction itself is comprised of a large number of write operations (e.g., committing thousands of ADDREC()s)

See Also

COMMIT_LOCK_DEFER_MS

COMPATIBILITY NO_CONFIG_PERSISTENCE

With V12 and later it is possible to store configuration options enabled or modified at run-time. These changes are persisted in an internal FAIRCOM.FCS table automatically created in existing FAIRCOM.FCS files at server startup. Stored keywords overwrite any setting in ctsrcr.cfg.

This persistence can be disabled at runtime by adding the following COMPATIBILITY NO_CONFIG_PERSISTENCE to ctsrvr.cfg:

COMPATIBILITY NO_DATAMAP_CHECK

When an index file is opened by an ISAM file open function, if the index is already open and is associated with a data file, FairCom DB checks if the index is associated with a different data file than the data file that is involved in the current file open operation. If so, the open now fails with error DMAP_ERR (957, this index file is already mapped to a different data file). This keyword disables the new approach (no check for mapping to the same data file as the currently-mapped file).

COMPATIBILITY NO_DELNOD_QUEUE

FairCom Server's delete-node thread prunes empty nodes from c-tree index files in the background. This maintains index key data densely packed for optimal performance. This activity requires directly opening the index file by the delete-node thread, which happens during idle times when it may be expected external applications can access the file. Having the file open at the time by the delete-node thread prevents external file open.

In exceptional cases, this behavior may not be desirable as external processes expect complete access to the file when it is no longer in use by the application. For example, processes may wish to immediately copy the file for other external processing.

In V11 and later, FairCom Server supports a configuration option to disable the internal delete-node thread. The option COMPATIBILITY NO_DELNOD_QUEUE disables the delete-node thread and disables the writing of entries to the delete-node queue.

Caution: This option should only be used in special situations in which you absolutely require an external process access to closed files, which is discouraged if at all possible while the server is operational.

Operating without pruning empty nodes from indexes can potentially diminish performance of certain index search operations which may non-optimally traverse many empty leaf nodes. This is especially true for applications which heavily delete records from the database.

COMPATIBILITY NO_FIXED_OFFSET_CHECK

In V11.5 and later, a feature prevents corruption and errors when an attempt is made to write to arbitrary offsets in a fixed-length file. Checks were added to ensure that a read/write/lock call on a fixed-length file is properly aligned. If an incorrect alignment is detected, the call will fail with FALG_ERR (767).

This feature can be disabled by keyword COMPATIBILITY NO_FIXED_OFFSET_CHECK.

Note: This is a Compatibility Change.

COMPATIBILITY NO_FLUSH_DIR

Reverts to the old behavior which does not force the flushing of metadata to disk immediately after creates, renames, and deletes of transaction log files and transaction-dependent files.

When a file is created, renamed, or deleted, the new name of the file is reflected in the file system entry on disk only when the containing directory's metadata is flushed to disk. If the system crashes before the metadata is flushed to disk, the data for the file might exist on disk, but there is no guarantee that the file system contains an entry for the newly created, renamed, or deleted file. In a test case we noticed that after a system power loss a transaction log containing valid log entries still had the name of the transaction log template file.

In release V11 and later, c-tree ensures that creates, renames, and deletes of transaction log files and transaction-dependent files are followed by flushing of the containing directory's metadata to disk. This change also applies to other important files such as CTSTATUS.FCS, the master key password verification file, and files created during file compact operations (even if not transaction dependent).

To revert to the old behavior, add COMPATIBILITY NO_FLUSH_DIR to ctsrvr.cfg.

COMPATIBILITY NO_IDFIELD_OVERFLOW_CONVERSION

In V10.5 and later, support for IDfield overflow detection was added. The IDfield resource is automatically updated to allow this. In V11.0 and later, this automatic conversion of the IDfield resource can be disabled by adding this keyword to the ctsrvr.cfg file.

COMPATIBILITY NO_IDFIELD_OVERFLOW_DETECTION

In V10.5 and later, support for IDfield overflow detection was added. Record operations that cause overflow or underflow return error IDVR_ERR (1002). In V11.0 and later, adding this keyword to the ctsrvr.cfg file will cause all tests for overflow/underflow to be ignored.

COMPATIBILITY NO_IDXBUFBLK

For internal FairCom use only. Do not use this keyword.

COMPATIBILITY NO_INIT_VSPACE

Under transaction control, new space was reserved at the end of a variable-length file, and the preimage space received an entry that marked the space as deleted if the transaction was aborted. When a write to this space (say as part of an ADDVREC()) was made, still under the same transaction control, preimage received another entry. However, until either a commit or an abort, the new region appeared to other users to be simply ff-filled. When another user was scanning the variable-length file in physical order, e.g., with NXTVREC(), this would cause a VFLG_ERR (158), since 0xffff is not a valid record mark. A fixed-length file does not behave this way because ff-fill appears as a deleted record to be skipped during physical order scan.

In V8.14 this behavior was changed such that not only is preimage space updated with the deleted record mark, but also a write is issued to the actual file with the same contents. This option reverts that behavior.

COMPATIBILITY NO_KEEP_OUT_TRNSEQ

ctKEEP_OUT permits TRANEND() to free locks obtained inside the transaction, but keep locks obtained outside of the transaction. One subtle point is that a lock obtained outside of the transaction on a record that is updated within the transaction will be freed. ctKEEP_OUT_ALL keeps locks even if the record is updated within the transaction if the lock was obtained outside of the transaction. However, a lock obtained inside of a prior transaction that is kept at the TRANEND() of the prior transaction is treated as if it was obtained inside of a subsequent transaction , and is released at the next TRANEND() called with ctKEEP_OUT or ctKEEP_OUT_ALL.

New default behavior allows TRANEND() to distinguish between a lock obtained within a prior transaction, and a lock obtained in its own transaction. This option reverts this behavior.

COMPATIBILITY NO_MYMARKS

In some situations, repeated calls were made to resolve key-level locks without any clean up occurring. When a key-level lock is associated with a transaction that has committed or aborted, the key-level lock is removed and the key value stays (committing an add or aborting a delete) or is removed (committing a delete or aborting an add). But if a thread is executing a long transaction, it may be repeatedly calling for cleanup without any effect when the key-level locks for a node all belong to the calling thread.

Buffer state variables were added such that this cleanup can immediately detect when all the key-level locks belong to the calling thread, and returns without checking each of the key-level locks enhancing performance. This keyword disables the new approach for testing.

Note: This is intended for internal FairCom testing only.

COMPATIBILITY NO_NXTMARKS

To improve the efficiency of key cleanup when called as part of a NXTKEY() operation, the cleanup routine was modified to only perform the clean-up necessary to satisfy the NXTKEY() operation. If NXTKEY() ends up calling GTKEY() because the last position for the calling thread is no longer valid, then the subsequent clean up would not use the special next key logic. This keyword disables this new behavior.

Note: This is intended for internal FairCom testing only.

COMPATIBILITY NO_RCVHDRUPD_NUMENT

Used for Faircom internal testing only. Do not use this keyword.

COMPATIBILITY NO_RELBUF_CHECK

A check for removing an updated buffer from the update list during a checkpoint was enabled. This new behavior can be disabled with this keyword.

COMPATIBILITY NO_REPL_DEFER_TRAN

In V11.0 and later, an optimization was added to avoid sending transactions to a Replication Agent that did not replicate the files in that transaction. Adding this keyword to the ctsrvr.cfg file will disable this optimization.

COMPATIBILITY NO_SHUTDOWN_DELAY

Forces an instant shutdown without pause for client disconnect. Not valid on NLM.

Default: Wait for client

COMPATIBILITY NO_SIGNAL_HANDLER

Developers using the Server SDK to build custom servers with their own over all control of the server can use this server configuration keyword to force the server to skip the installation of signal handlers at server startup. If this is done, then their control code will be responsible for dealing with signals. This modification only affects Unix platforms. If signal handlers are implemented for other platforms, then be sure that the compatibility test in ctsint_a.c is repeated within the appropriate platform #ifdef.

COMPATIBILITY NO_SMART_SAVE

Disables support to enable a ctsave() call to skip the file system sync if no bytes have been written since the last call to ctsave().

COMPATIBILITY NO_SPCMGT_QUEUE

By default, the FairCom DB Server reclaims the space from deleted member files of a Superfile and recovered variable-length data files. A dedicated background thread performs the space reclamation. A permanent queue stored in the Server file D0000001.FCS permits the space reclamation to be interrupted at Server shutdown, and resumed when the Server restarts. This configuration keyword disables this feature.

Default: Manage Superfile deleted space

COMPATIBILITY NO_SYS_FLUSH_ON_CLOSE

In some situations, it is possible for a file-close operation to occur without a guarantee of a data sync to disk first. For transaction-controlled files, this meant a file could be left corrupted on disk while the FairCom DB Server transaction control system would have no knowledge of this. Extra protection has been added to ensure that this situation did not occur.

This behavior will cause a system cache flush before physically closing transaction logged files or write-through files. This behavior can be disabled by adding COMPATIBILITY NO_SYS_FLUSH_ON_CLOSE to the server configuration file.

COMPATIBILITY NO_TEST_LOCAL

In some environments (e.g., WIN32 / UNIX) the tests to determine if two files with different names are really different (or just accessed with different paths or aliases, etc.) may indicate the files are different, when in fact they are the same, if one of the names is based on a network reference and the other (through aliases, device mappings or SUBST commands) appears like a local file. If this occurs, the server may attempt to open the files as two physically different files.

Because of the possibility of a performance hit, COMPATIBILITY NO_TEST_LOCAL is available to turn off the check of whether a file is local or remote.

COMPATIBILITY NO_TRAN_DISCONNECT

When the number of log files is not permitted to increase (because of FIXED_LOG_SPACE YES in the configuration), and if the need for more logs is caused by a pending transaction, the server disconnects the client associated with the transaction by default. This keyword option disables this support and if the client does not make a subsequent server request, then the pending transaction will eventually lead to the server terminating abruptly with a L56 ctcatend. The server terminates because it cannot ensure that a commit or abort will be added to the transaction logs before the log that holds the TRANBEG() will become inactive. (If the client makes a server request, it will see the transaction attribute that indicates the need to abandoned the transaction, and the ctcatend shutdown will be avoided.)

COMPATIBILITY NO_TRANDEP_SCAN

Additional transaction log pre-scanning is now done for improved handling of transaction-dependent file renames and deletes, avoiding potential LEOF_ERR or ITIM_ERR errors. This option disables this pre-scan feature.

COMPATIBILITY NO_UNIQFILE

Disables attempts to determine if files accessed with different file names (or paths) and identical FairCom DB file IDs are the same file or different files.

Default: Check file identity

COMPATIBILITY NO_UPDMARKS

Disables a transaction performance optimization.

Note: This keyword should be enabled only on recommendation from FairCom.

COMPATIBILITY NO_VAR_PEOF

A default behavior permits a variable-length record that grows to grow in place if (a) the record is at the Logical End of File (LEOF), (b) no other user is adding a record at the LEOF, and (c) the growth of the record does not exceed the current Physical End of File (PEOF). Additional behavior (default) removes the limitation that the record growth stay within the current PEOF.

When the growth would exceed the PEOF, the logic attempts to extend the physical file size and retry growing the record in place (within the newly extended PEOF). At runtime, COMPATIBILITY NO_VAR_PEOF, disables this support, and leaves the growth in place activated as long as the record fits within the current PEOF.

COMPATIBILITY NO_VARLEN_TRAN_UNUSED

Previously, there were two related aspects of variable record length space reclamation that differed for transaction-controlled files. Instances were noted where a TRANPROC file would grow unnecessarily. In a particular instance, frequent and constant FairCom DB SQL add and delete operations resulted in substantial growth of a data file.

To avoid this unexpected growth, both aspects of space reclamation are now treated the same for TRANPROC and non-TRANPROC files. COMPATIBILITY NO_VARLEN_TRAN_UNUSED reverts to the previous differentiated behavior.

COMPATIBILITY NO_VFLG_ERR

Disables new handling of NXTVREC in physical order when it encounters the "record space" for a new record that has not been committed and was written in a space that is not being reused. Restores the earlier (post V8.14) behavior for the rare circumstance in which the old behavior is desired.

A feature introduced in V8.14 affected the behavior of NXTVREC in physical order when it encountered the "record space" for a new record that had not been committed and was written in a space that was not being reused. Instead of reporting a VFLG_ERR (error 158), that new feature skips the uncommitted "record space" (unless the reading is by the transactor, which would see the uncommitted record).

Details

The change affects how variable-length records are internally marked during transaction processing of pre-image space. The behavior prior to V8.14 marked the record header in a way that was considered invalid, causing a VFLG_ERR (158) error. The newer behavior sees the record as a deleted record (actually a pending insert) and skips to the next record as in a "read committed" transaction isolation.

This change is limited to the internal handling of header marks for newly added variable-length records.

Indexed files are not affected by these changes because pending key inserts are handled differently under transaction control.

These changes do not include changes to the physical files, record structures on disk, or other transaction control.

Reverting Back to the Old Behavior

These changes can be reverted back to the original (prior to V8.14) behavior using the keyword:

COMPATIBILITY NO_INIT_VSPACE

Changes in the Latest Revision

The changes introduced in the V11 release address the issues with the earlier change as follows:

If the reader has requested acquiring locks on the records that it reads, the physical read acquires a lock on that record and respects the lock before proceeding (earlier the lock was not respected) producing one of the following outcomes:

If...	Then...
Locking the record fails with error 42 (DLOK_ERR) because another connection has the record locked and the reader requested non-blocking locking.	The physical record read function returns error 42.
The record is committed (already, or when the lock is released).	The record read proceeds as usual, and the record that would have been skipped is returned to the caller.
The record is deleted or is a “resource.”	The physical record read function continues scanning the data file, reading the next record.
The record header contains an invalid record mark.	The physical record read function returns error 158 (VFLG_ERR).

Further changes have been introduced to reduce the occurrence of the 158 (VFLG_ERR) without skipping any record by changing the record header marker management during record addition.

COMPATIBILITY NO_VFLG_ERR can be used to disable this new handling and restore the earlier (post V8.14) behavior for the rare circumstance in which the old behavior is desired

COMPATIBILITY NONE

This option is used in conjunction with the tamper-proof settings file under the server. Configuration options that are in the encrypted ctsrvr.set settings file cannot be overridden in the ctsrvr.cfg file.

The DIAGNOSTICS, COMPATIBILITY, and CONSOLE keywords do not automatically block use in a subsequent stage of configuration loading. To explicitly block any of these keywords present in a later stage, add entries in the form: <keyword> NONE where <keyword> is DIAGNOSTICS, COMPATIBILITY, or CONSOLE. For example, to turn on the abort-on-close compatibility option and prevent any subsequent stage to use the COMPATIBILITY keyword, place the following entries in ctsrvr.set:

COMPATIBILITY ABORT_ON_CLOSE

COMPATIBILITY NONE

Default: Not present

COMPATIBILITY OPEN_SHARE_RW

For c-tree data and index files that do not use the DUPCHANEL filemode, the FairCom DB for Windows now opens the files in shared read-only mode such that only the FairCom DB process can write to the files. FNAC_ERR (920), is returned when a file exists but is not accessible, say due to file system permission settings or file sharing restrictions. This configuration option can be used to restore the previous behavior of opening the files with read/write share access.

COMPATIBILITY OPEN_RANDOM_ACCESS

Restores the behavior of the obsolete FILE_FLAG_RANDOM_ACCESS option.

COMPATIBILITY PUTHDR_COMMIT

PUTHDR() called within a transaction causes a pre-image space entry that contains the old and new header member value. On an abort, the old value is restored. On commit the new value is reset. "Reset" because at the time of the call to PUTHDR(), the header member takes on the new value. But header values are not like ordinary record contents. They are not typically under transaction control. We do not want to lock the record header during a transaction and not unlock until the commit. PUTHDR() uses the more traditional transaction approach to permit special modifications to the file under transaction control. However, for the ctSERNUMhdr, ctLOGEOFhdr and ctPHYEOFhdr header members, this approach can lead to unexpected problems as these entries are always increasing and have an aspect of uniqueness about them. For ctSERNUMhdr, ctLOGEOFhdr and ctPHYEOFhdr, PUTHDR() behaves as before (creating pre-image space entries, restoring old value on abort, setting the new value at the time of the call) except that on commit the header value is not reset to the new value. The prior behavior can be restored by adding this keyword option.

COMPATIBILITY RANGE_NO_NXTKEY

Range performance can be enhanced using NXTKEY() instead of GTKEY() to skip over records that do not meet range and/or filter criteria during range operations. A modification was made to enable this behavior by default. This option disables this feature.

COMPATIBILITY REPLICATION_TRAN_LIST

Makes the checkpoint's beginning-log-position-list for active transactions included in checkpoints. This also applies to the pending SUCTRAN list.

COMPATIBILITY REVERT_TO_V6HDR

Version 9 of FairCom DB introduced extended headers for all newly created files by default. Previously, this mode was only enabled with calls from an Xtd8 specific function and defining the XCREblk structure.

The advantage of this new approach is that 6-byte transaction numbers are used by default, which avoids potential unexpected transaction number overflows, or in some cases, encountering error R6BT_ERR (745, 6BTRAN file required).

This feature can be disabled with the following keyword should this be necessary for backward compatibility:

COMPATIBILITY REVERT_TO_V6HDR

Standalone applications can disable this support by setting the cth6flg global variable to any non-zero value.

Default: OFF

COMPATIBILITY REWRITE_KEY_ERROR

The key assembly routine that processes a record image to extract key segments and compose key values may return an error condition, SDAT_ERR, if data is not available (say because of a missing or truncated field), or if some other error code if a problem arises such as a lack of Unicode support for a Unicode key. The add record routines properly return an error if a key assembly error occurs, but the rewrite routines simply treated the key error equivalent to a NUL or missing key. This is not expected to be a common occurrence since it generally takes a badly formed record image. The rewrite routines now return an error on a failed key assembly by default. This keyword reverts to the prior behavior.

COMPATIBILITY TCPIP_CHECK_DEAD_CLIENTS

COMPATIBILITY TCPIP_CHECK_DEAD_CLIENTS

With the COMPATIBILITY TCPIP_CHECK_DEAD_CLIENTS keyword in the Server configuration, the FairCom DB Server uses an older method to detect when a TCP/IP client has dropped, using the interval set by the DEAD_CLIENT_INTERVAL keyword. Every time interval, the client connection socket is rechecked to ensure that it is still a valid communications channel. If a connection is found to be invalid, the Server terminates the connection.

The FairCom DB Server normally recognizes when a client disconnects. However, the Server relies on a chain of events controlled by the operating system to recognize the disconnection. The client computer must notify the Server host computer that the connection has been dropped. For example: When a user closes an application, the socket is closed by the operating system, which sends a message to the Server host machine. However, if the network connection is temporarily interrupted or if the client machine is powered down suddenly, this message is not sent and the Server host machine can’t recognize that the client connection has dropped.

See Also:

SESSION_TIMEOUT under Miscellaneous Control
DEAD_CLIENT_INTERVAL

COMPATIBILITY SYNC_LOG

COMPATIBILITY SYNC_LOG

DEPRECATED

As of FairCom DB Version 9.0, COMPATIBILITY LOG_WRITETHRU is synonymous on both Windows and Unix platforms.

On Unix systems, instructs the FairCom DB Server to open its transaction logs in synchronous write (direct I/O on Solaris) mode. In this mode, writes to the transaction logs go directly to disk (or disk cache), avoiding the file system cache, so the server is able to avoid the overhead of first writing to the file system cache and then flushing the file system cache buffers to disk. This keyword also causes flushed writes for data and index files to use direct I/O. Using this keyword enhances performance of transaction log writes.

Default: OFF

See Also

COMPATIBILITY LOG_WRITETHRU