ISAM for C Database Technology

c-tree provides ISAM functions, a set of high-level functions for manipulating data and index files. Each function performs multiple file and index access operations.

ISAM functions assume a hierarchical relationship among the data and index files, in the sense that each data file may have many indexes, while each index relates to only one data file.

For example, an accounts payable system may define a vendor data file indexed by vendor name and number, and an invoice data file indexed by vendor number and invoice number. In this case, each data file has two indexes. Although each data file is indexed by vendor number, there is a separate vendor number index for each data file.

ISAM functions assume the key values for each index are derived from the contents of the data records of the associated data file. As described later, the Key Segment parameter specifies how to extract a key value from a data record.

ISAM Concepts

FairCom DB provides a set of high-level functions for manipulating your data and index files collectively termed ISAM functions.

Indexed

Sequential

Access

Method

These functions perform multiple file and index access operations for each function call, whereas low-level functions operate on only one individual data file or index file per call. ISAM routines are strongly recommended, especially for client/server based applications, because the quantity of functions required to send over a network with ISAM level calls is much less than with low-level functions.

ISAM routines assume that relationships among data and index files is hierarchical in the sense that each data file may have many indexes, while each index relates to only one data file.

For example, an accounts payable system may define a vendor data file indexed by vendor name and vendor number, and an invoice data file indexed by vendor number and invoice number. In this case, each of the data files has two indexes. Although each of the data files is indexed by vendor number, there is a separate vendor number index for each data file.

The ISAM routines also assume that the key values for each index can be derived from the contents of the data records of the associated data file. As described later, the Key Segment parameters specify how to extract a key value from a data record.

As an example of how the ISAM functions perform multiple actions, consider the following sequence of events. We want to open a data file and its two associated indexes, find the first key value in an index, and read the associated data record into memory. With low-level c-tree functions the steps would be (the c-tree function used is in parentheses):

• Initialize c-tree (InitCTree())
• Open the data file (OpenCtFile())
• Open the first index file (OpenCtFile())
• Open the second index file (OpenCtFile())
• Find the first key (FirstKey())
• Read the record (ReadData())

Six total steps.

The same procedure, using the ISAM functions, would be as follows:

• Initialize c-tree (InitISAM())
• Open the data file and the two indexes (OpenIFile())
• Find the first key and read the record (FirstRecord())

Half the steps of a low-level sequence! ISAM functions save you work, and make your code easy to follow and maintain.

ISAM File Relationships

ISAM parameters specify characteristics of data and index files used in your application. Parameters are stored in IFIL structures, C language based structures containing all the information needed to relate the data, indexes and key segments. These are collectively the IFIL, IIDX, and ISEG structures. These are stored as a file resource in the data file such that they persist. This means all information needed to create indexes for a data file is contained directly in that file.*

ISAM routines assume a hierarchical relationship among data and index files in the sense that each data file may have many indexes, while each index relates to only one data file. For example, an accounts payable system may define a vendor data file indexed by vendor name and number, and an invoice data file indexed by vendor number and invoice number. In this case, each of the data files has two indexes. Though each of the data files is indexed by vendor number, there is a separate vendor number index for each data file.

ISAM routines also assume that key values for each index can be derived from the contents of data records of an associated data file. As described below, the Key Segment parameter specifies how to extract a key value from a data record. See ctixmg.c for an ISAM structure example of variable-length records and their associated key values.

With only two exceptions, c-tree requires key values to be well formed up to the full length of the key (including the optional duplicate suffix). The exceptions are:

• FirstInSet() and LastInSet() for which you explicitly indicate the number of bytes of the target key which are significant, so a complete key is not necessary.
• Key segments located in varying length fields in the latter portion of a variable-length data record. In this case, c-tree automatically pads the key segment to the full segment length. Again, this is only applicable to the ISAM routines.

When using functions such as GetGTERecord(), a complete key is necessary in the sense that the bytes at the end of the target that you are not concerned about still must be set to an appropriate (padded) value. In these functions, ensure that the trailing insignificant bytes of the target are consistent with the padding used when the key value was added (typically an ASCII space or NULL (zero bytes).

When a low-level index function requires both an input key, target, and an output key, idxval, be sure that target and idxval DO NOT point to the same location.

The AddRecord() and AddVRecord() ISAM routines automatically form complete key values from the contents of the data records passed to them. The low level AddKey() function must be passed a complete key.

c-tree makes no assumptions about the individual bytes comprising a key value. In particular, a NULL byte IS NOT interpreted as the end of a key value. However, the varying length field concept supported for variable-length data records under ISAM control requires the concept of a field delimiter. The field delimiter separates fields in the variable-length portion of a variable-length data record. Therefore, a key value located in a varying-length field cannot contain the field delimiter byte.

Key Segment Modes (Key Segment Modes, /doc/ctreeplus/30863.htm) are very important for proper record retrieval. Be sure to review this section very carefully when designing your retrieval methods.

Legacy Note: Legacy ISAM support allowed for ASCII text files termed ISAM parameter files. FairCom recommends using Incremental ISAM Structures (IFILs) over the Parameter File approach for modern ISAM applications. Incremental structures provide individual file open/close/rebuild control and offer advanced features (i.e., file definitions automatically stored at file creation time, etc.), not available with the ISAM Parameter File.

Conversion utilities exist for migrating legacy parameter files to embedded IFIL resources.

Incremental ISAM Structures

Incremental ISAM structures permit your application to create (CreateIFile()), open (OpenIFile(), OpenFileWithResource()), close (CloseIFile(), CloseRFile()) and rebuild (RebuildIFile()) individual ISAM data file / index file combinations. Three structures work together to enable the incremental ISAM capability: IFIL defines the data file, IIDX defines the indexes, and ISEG defines the key segments. These structures are defined in the ctifil.h header file.

IFIL Structure

All the incremental ISAM functions require a pointer to an IFIL structure as their input parameter. The IFIL structure defines the characteristics of the data file and includes a pointer to a structure that defines the associated indexes.

The formal type definition for IFIL follows:

typedef struct ifil {

pTEXT pfilnam, /* file name (w/o ext) */

FILNO dfilno, /* data file number */

UCOUNT dreclen, /* data record length */

UCOUNT dxtdsiz, /* data file ext size */

COUNT dfilmod, /* data file mode */

COUNT dnumidx, /* number of indexes */

UCOUNT ixtdsiz, /* index file ext size */

COUNT ifilmod, /* index file mode */

pIIDX ix, /* index information */

pTEXT rfstfld, /* r-tree 1st fld name */

pTEXT rlstfld, /* r-tree last fld name */

FILNO tfilno /* temporary file number */

} IFIL;
 

Note: pTEXT is a c-tree data type that equates to TEXT *. For example, pTEXT pfilnam is the same as TEXT *pfilnam. The same holds true for all ‘p’ data types, i.e., pISEG = ISEG *.

In V12 the file number typedef was formally changed from COUNT, a two-byte value to FILNO, a four-byte value. Refer to this link for compatibility details. Four Byte File Numbering

File Names up to 4K

With V12 and prior, c-tree limited file name lengths to 255 byte. Starting with V13, this optional support allows for file name lengths up to 4K in size. This limit is controlled by a compile-time macro, MAX_NAME. Modern operating systems support file names over 255 bytes and in this revision will allow MAX_NAME to be increased, supporting file names up to 4096 bytes.

Note: When enabled, these changes break compatibility with existing c-tree binaries and dynamic dump backups.

Windows Notes:

Historically, Windows API functions have limited file names passed to its file and directory functions to MAX_PATH (260) bytes. Starting with Windows 10, version 1607, this limit has been removed from common Windows API file and directory functions. For information about Windows file naming conventions, see: Microsoft Windows 10 - Naming Files, Paths, and Namespaces.

As noted on that page, you must opt in to the new behavior. To enable this support:

1. Create the registry key HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD) and set it to 1. Then reboot the system.
2. Create a manifest file named longPath.manifest containing the following text:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">

<application xmlns="urn:schemas-microsoft-com:asm.v3">

<windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">

<ws2:longPathAware>true</ws2:longPathAware>

</windowsSettings>

</application>

</assembly>

In the Visual Studio project settings for each executable that uses Windows API functions with a file name over 260 bytes (such as ctsrvr.exe and ctrdmp.exe), specify the name of this manifest file under Properties | Manifest Tool | Input and Output | Additional Manifest Files. Note this has already been done in the makefiles created by FairCom’s mtmake make builder.

mtmake Parameter Adds Manifest for Long Paths

A new mtmake 4Kfilepaths parameter has been added to enable the extra manifest for longPathAware builds.

{"4Kfilepaths", "Add manifest to support 'longPathAware' 4K file paths.",MTOPTN_NORMAL},

IIDX Structure

Each index is described by an IIDX structure. If a data file has associated indexes, the ix member of the IFIL structure must point to an array of IIDX structures. In particular, there must be dnumidx elements (i.e., IIDX structures) in the array.

Each IIDX structure defined over a given data file receives a unique file number assigned sequentially from the data file number tfilno. For example, if tfilno is file number 5, and this data file has 3 indexes, they receive file numbers 6, 7 and 8. 6 is assigned to the IIDX structure pointed to by the IFIL element ix, 7 is assigned to the next IIDX structure, and 8 to the last IIDX structure for this particular data file. Since each IIDX structure gets its own file number, there is no difference in performance between placing all IIDX definitions for a particular data file in the same physical file or in individual physical files.

The formal type definition for IIDX follows:

typedef struct iidx {

COUNT ikeylen, /* key length */

ikeytyp, /* key type */

ikeydup, /* duplicate flag */

inulkey, /* NULL key flag */

iempchr, /* empty character */

inumseg, /* number of segments */

pISEG seg, /* segment information */

pTEXT ridxnam, /* r-tree symbolic name */

pTEXT aidxnam, /* optional index file name */

pCOUNT altseq, /* optional alternate sequence */

pUTEXT pvbyte /* optional pointer to pad byte */

} IIDX;
 

The following text provides additional information about select IIDX elements:

ikeylen - The sum of the segment lengths (defined below). If duplicates are allowed, the last 4 bytes of the key value are reserved for the 4 bytes of the associated data record offset. For example, a key length of 12 bytes allowing duplicates has 8 bytes of actual key value and 4 bytes for the tie-breaking data record offset. DropIndex() marks a transaction-controlled index for deletion by setting keylen to -keylen until the drop is committed. A GetIFile() call after the DropIndex() call, but before the Commit() or Abort() will return this interim setting.

ikeytyp - Valid key types are listed below. For additional information, consult Alternative Key Types.

See also Alternative Key Types.

Note: Key compression imposes a significant performance impact, especially when deleting records. Use this feature only when absolutely necessary to keep index space requirements to a minimum.

ikeydup - Duplicate flag. A parameter value of one indicates the index supports duplicates, add 4 bytes to the key length (see ikeylen above). A parameter value of zero indicates the index will NOT support duplicates, making all key values unique. Zero and even numbered values indicate unique indexes, odd values allow duplicates key values.

inulkey - Determines if NULL or missing key values are detected. After concatenating the key segments according to the ISEG array, the NULL key flag parameter determines whether or not to test for a missing value. If the key value matches the null key value definition according to the index's inulkey setting, as described below, it is considered NULL or missing and not entered into the index file.

iempchr - The empty character is expressed in the decimal equivalent of the character. For example, an ASCII space (0x20) is specified with a value of 32. A NULL byte (0x00) is specified with a value of zero.

inumseg - The number of segments concatenated to form a key value from the data record contents.

ridxnam - If r-tree or d-tree is used with this application, define this arbitrary index name. It must start with an alphabetic character and may include upper and lower case letters, numbers, and underscores. It may not include embedded blanks. Symbolic names are not necessary for the data files because each has a unique name.

aidxnam - (Optional) Permits Incremental ISAM files to have more than one index file and to locate an index file independently from the data file. The index file name must agree with the operating system file naming conventions. If not NULL for the first member of the IIDX array, this file name is used instead of being derived from the IFIL pfilnam parameter. This allows the index to be located in a different path or a different device. If not NULL for subsequent members of the IIDX array, this name is used to create additional host index files. All following member indexes will be located in the new index file.

ISEG Structure

Each key value is composed of one or more segments. A segment is simply a sequence of bytes from a specified offset within the data record. Each key segment is described by an ISEG structure. The seg member of each index IIDX structure must point to an array of ISEG structures. In particular, there must be inumseg elements (i.e., ISEG structures) in the array. The formal type definition for ISEG follows:

typedef struct iseg {

COUNT soffset, /* segment position (offset) */

slength, /* segment length */

segmode /* segment mode */

} ISEG;
 

Segment Position

The segment position specifies the location of the key segment within the data record. This can be specified in one of three ways, as determined by the segment mode value (see Key Segment Modes):

1. Absolute byte offset. The number of bytes from the beginning of the data record to the first byte of the key segment. The smallest offset, zero, means that the segment begins with the first byte of the record. If a segment begins with the nth byte of the record, its offset value should be n minus one.
   This option works with fixed length fields, which have known offsets.
2. Relative field number. The segment position is treated as a field number, not a byte offset. This applies only to variable-length data records. With variable-length records, all of the fields in the variable-length portion are variable-length fields terminated with a delimiter. This delimiter is typically the NULL character, unless the default is changed by SetVariableBytes(). A segment position of zero corresponds to the first variable-length field, beginning at the first byte beyond the variable-length file’s fixed-length portion. When a variable-length file is created, the record length parameter is treated as the minimum record length, which corresponds to the fixed-length portion of the record.
   This option works with variable-length string types using NULL terminators.
   The following schematic demonstrates how varying-length fields are numbered. The delimiter is shown as a NULL byte, but can be any delimiter you set:

1. Record Schema field number. When using a Record Schema, (Record Schemas), use a schema segment type and specify the segment position by the field number in the record schema. The first field in the schema is field zero.

This option works with any type of data in any part of the record, allowing key segments on non-variable-length data types in the variable-length portion of the record. All fields in the variable-length portion must be packed as opposed to aligned. No padding bytes are allowed between fields.

Though segoffset is a COUNT field, since a fixed length file or the fixed length portion of a variable-length file may exceed 32767 bytes, segment offsets exceeding 32767 bytes, but less than 65535 bytes, are supported. The internal logic deals with segment offsets that “go negative”. A minus one for a segment offset (equivalent to 65535 for a UCOUNT) still signifies the “end of segments” so that segment offsets can now range from 0 to 65534. If a value greater than 32767 is stored in a COUNT field, it appears as a negative value; but its unsigned integer value correctly corresponds to the original value. For example, storing 65534 into a COUNT results in a value of -2 when expressed as a COUNT, but -2 cast to a UCOUNT corresponds to 65534.

Segment Length

The segment length specifies the number of bytes that this key segment will take in the key itself. The sum of the segment lengths for an index file must equal the key length unless duplicate keys are allowed. In this case, the segments should sum to the key length less 4 bytes. If the sum of the segment lengths does not match the key length, CreateISAM(), OpenISAM(), CreateIFile(), or OpenIFile() return error ISLN_ERR (115).

For variable-length fields, segment modes 4 and 5, c-tree automatically pads a key segment to the full segment length using the padding byte. The padding byte defaults to an ASCII space (0x20), but can be changed using SetVariableBytes(). When using a field other than a variable-length field, make the segment length less than or equal to the actual size of the field.

Segment Mode

The segment mode determines what transformations, if any, must be made to the key segment. In addition, it determines which of the three methods of determining the segment position will be used.

Detailed information on key segment modes is presented in the Key Segment Modes section. When using a Record Schema to manage the building of keys, refer to Record Schemas for more information.

Incremental ISAM Example

The following C declarations show how the variable-length data base example shown before can be expressed in terms of initialized incremental ISAM structures. Because of the nested relationship of IFIL, IIDX, and ISEG, we define the segment information, then the index information, and then the data file information. For the sake of completeness, we added the optional r-tree symbolic names.

Note: (pTEXT) 0 is the same as (TEXT *) 0 and likewise for pCOUNT and pUTEXT.

ISEG vc_seg[] = {

{0,15,5}, /* 1st segment, 1st index */

{4, 5,2}, /* 2nd segment, 1st index */

{0, 4,0}, /* 1st segment, 2nd index */

{4, 9,2}, /* 1st segment, 3rd index */

{0, 9,5} /* 2nd segment, 3rd index */

};

IIDX vc_idx[] = {

{24, /* key length, 1st index */

12, /* key type */

1, /* duplicate flag on */

1, /* NULL key flag on */

32, /* empty char (decimal) */

2, /* number of key segments */

&vc_seg[0], /* pointer to segment array */

"name_key", /* r-tree symbolic index name */

(pTEXT) 0, /* optional index name */

(pCOUNT) 0, /* alternate collating sequence */

(pUTEXT) 0 /* optional pointer to pad byte */

},

{ 4, /* key length, 2nd index */

1, /* key type */

0, /* duplicate flag off */

0, /* NULL key flag off */

0, /* empty char ignored */

1, /* number of key segments */

&vc_seg[2], /* pointer to segment array */

"numb_key", /* index name */

(pTEXT) 0, /* optional index name */

(pCOUNT) 0, /* alternate collating sequence */

(pUTEXT) 0 /* optional pointer to pad byte */

},

{22, /* key length, 3rd index */

12, /* key type */

1, /* duplicate flag on */

1, /* NULL key flag on */

32, /* empty char (decimal) */

2, /* number of key segments */

&vc_seg[3], /* pointer to segment array */

"zipc_key", /* index name */

(pTEXT) 0, /* optional index name */

(pCOUNT) 0, /* alternate collating sequence */

(pUTEXT) 0 /* optional pointer to pad byte */

}

};

IFIL vc_dat = {

"vcust", /* root of file name */

1, /* data file number */

15, /* minimum record length */

4096, /* data file extension size */

5, /* data file mode */

3, /* number of indexes */

4096, /* index file extension size */

1, /* index file mode */

vc_idx, /* pointer to index array */

"vcust_1st", /* 1st data field */

"vcust_last" /* last data field */

/* NULL: tfilno */

};

With the above structures, the incremental ISAM functions, (CreateIFile(), OpenIFile(), CloseIFile(), RebuildIFile()), are called with a single input parameter: &vc_dat. &vc_dat is a pointer to an IFIL structure that points to the array of index structures and segment structures.

It is not necessary to put all the segment values in one array. You could use separate arrays of ISEG structures for each index. However, all indexes for a single data file must be in a single array of IIDX structures.

Improved IFIL Path Handling

Historically, the IFIL resource stored in the file includes a path component in IFIL.pfilnam and optionally IIDX.aidxnam, which could be absolute or relative. If the file is moved to a different directory, a mismatch exists between the path the application must specify to open the file and the path contained in the IFIL resource. This mismatch, particularly in the aidxnam, can cause problems, as this on-disk version of aidxnam is used by OPNRFIL() to locate these indexes.

Unless SUPPRESS_PATH_IN_IFIL NO is set in ctsrvr.cfg, the following changes take effect::

• The file becomes incompatible with older servers. This change takes effect when the PUTIFIIL() function is called, which typically occurs if a data or index definition changes. PUTIFIL() might be called automatically within operations such as file creation, an AlterTable operation, or potentially within a Rebuild or Compact operation.
• Applications may now specify IIDX.aidxnam as "+myaltindex.idx", meaning that the alternate index is expected to be in the same directory as the data file. This syntax was previously available to PRMIIDX() and TMPIIDX() only. It is now available with all calls that accept an IFIL argument (CREIFIL, OPNIFIL, RBLIFIL, CMPIFIL, RENIFIL, CLIFIL, DELIFIL)
• Applications may now specify IIDX.aidxnam as "?myaltindex.idx", meaning that the alternate index is expected to be in the same directory as the data file, and should be prefixed with the data file name as well: For pfilnam="mypath/mydata" and aidxnam="?myaltindex.idx", the name of the index in the file system will be "mypath/mydatamyaltindex.idx". ISAM-level renames of the data file will implicitly rename the alternate index.
• Beginning with V12, when a PutIFile() or one of it's extended versions is called, the path is automatically stripped off the file name (IFIL.pfilnam) and alternate index name (IIDX.aidxnam) elements of the IFIL structure.

These changes can cause compatibility issues. See these sections for information about disabling them:

• Configuration option to disable IFIL path improvements
• Programming option to disable IFIL path improvements

See also:

• Partition file directory with SUPPRESS_PATH_IN_IFIL

Configuration Option to Disable IFIL Path Improvements

New configuration option allows you to disable IFIL path improvements so files can be opened by earlier releases

FairCom DB includes enhancements to the storing of data and index file paths in the IFIL resource of a c-tree data file to avoid potential problems when files are moved to a different location. An attempt to open a data file that was created with these enhancements with a V11.5 (or earlier) server fails with error FREL_ERR (744, "file requires unavailable feature").

FairCom Server supports a configuration option that allows newly-created files to keep the IFIL resource in the old format, so that the data file can be opened by V11.5 and earlier. To use this option, add the following keyword to ctsrvr.cfg:

SUPPRESS_PATH_IN_IFIL NO

Note: This error will be seen only in environments in which a file is created on a server with the index file path enhancements and then is copied (replicated) to earlier versions of the server without this feature. As a best practice, FairCom always recommends keeping all servers in the same operating environment on the same release. If it is not possible to upgrade all servers to the same release, the keyword can be used as a last resort.

Programming Option to Disable IFIL Path Improvements

The new IFIL path logic can be deactivated programmatically using the following procedures.

Note: Programming support through the SDK only applies to standalone libraries. This technique is not supported in client-server applications.

The new IFIL path logic can be deactivated by setting the following environment variable after you have called InitISAM() and prior to creating any files:

ctSuppressPathInIFIL = NO;
 

For example:

retval = INTISAMX(6, /* index buffers */

100, /* files */

64, /* page sectors => 8192 bytes cache page size */

6, /* data file buffers */

MY_USER_PROFILE_MASK, /* UserProfile */

uid, /* user id */

upw, /* user password */

svn); /* server name */

if (retval) {

printf("\nCould not initialize c-tree Plus(R) (%d)\n",retval);

#ifdef ctThrds

ctThrdTerm();

#endif

ctrt_exit(2);

}

/* Setting this environment variable removes V11.6 and newer IFIL path logic */

ctSuppressPathInIFIL = NO;

printf(" Attempting to create.\n");

if ((retval = CREIFILX( &vcustomer, /* IFIL pointer */

" ", /* data file name ext*/

".ndx", /* indx file name ext*/

(LONG) (OPF_ALL | GPF_READ | GPF_WRITE |

WPF_READ), /* permission mask */

NULL, /* alt group id */

NULL)) /* password */

|| (retval = PUTDODA(CUSTDAT,doda,(UCOUNT) 7)))

{

printf("\nCould not create file with error %d (on file %d).\n",retval,isam_fil);

fflush(stdout);

}

else

printf("File created successfully.\n");

Incremental ISAM and Resources

If Resources are enabled when you create a file with CreateIFile(), the information in the IFIL structures is stored as a resource record in the data file. Use OpenFileWithResource() and CloseRFile() to open and close the file(s) without referring to a specific IFIL structure in memory. You can assign the file numbers via a parameter, or ask c-tree to assign appropriate file numbers. To update the IFIL structure stored in the resource record, call GetIFile() to retrieve the structure and PutIFile() to rewrite it.

Current ISAM Record

When using the ISAM functions, the concept of the current ISAM record for a data file is very important. c-tree remembers the record position of the last data record found for each ISAM data file, as well as information about the ISAM keys for that data record. This last record found is referred to as the current ISAM record.

Many of the ISAM routines use the current ISAM record. For example, DeleteRecord() deletes the current ISAM record, and NextRecord() uses the current record information to help find the next record in key sequential order. If no current ISAM record exists, as happens just after the file is opened, but is required, then the ISAM function returns error code ICUR_ERR (100).

In older versions of c-tree, V4.3 and before, you had to be very careful with record rewrites, ReWriteRecord() and ReWriteVRecord(). c-tree would use your buffer area to determine the ISAM key structure for the current record. If you changed one of the fields that was used as a part of the key, and then rewrote the record, you could damage the index files.

With c-tree you no longer have to be as careful with record rewrites. c-tree keeps its own image of the key fields. In a rewrite, c-tree determines if a key field has changed by comparing the contents of its current key image buffer with your data record buffer. If a key has been changed, the old key is deleted and the new one is added. This is automatically done for ALL indexes relating to this record.

This does not mean that you don’t have to be concerned with ISAM record rewrites. After ReWriteRecord() or ReWriteVRecord(), the current ISAM record is now the updated record. This may cause problems when your application is moving through a file in index sequential order while making updates. If the key field is updated, then the NEW current ISAM record may be substantially behind or ahead of the prior current ISAM record before the update was made. If, after an update, you wish to continue sequentially from the original record, do the following in your application:

ResetRecord(datno,SWTCURI);

Place this function call after your rewrite operation, and before any other ISAM function. This restores the current ISAM record to be the original one before the rewrite.

The same technique may be used with variable-length records. However, the record may move physically within the file if its size is extended during the rewrite. Therefore, it is possible to encounter the record again even if no key segment is changed. It may be necessary to mark scanned records to avoid this issue.

Other functions that access or modify the current ISAM record in c-tree are CurrentFileOffset() and SetRecord().

NOTE: More than one current ISAM position can be maintained in a single instance of an open file. See Multiple ISAM Contexts (Positions).

Multiple ISAM Contexts (Positions)

To permit an application to maintain more than one current ISAM position in a single instance of an open file, the following three routines are used: OpenISAMContext(), ChangeISAMContext(), and CloseISAMContext().

These functions make it possible to save multiple positions in the same file. For example, it is possible to do the following with an index defined over city name:

• Open an ISAM context. Retrieve the city “Columbia”.
• Open a second ISAM context over the same index. Retrieve the city “Denver”.
• Change to the first ISAM context. Read the next record to retrieve “Columbus”.

The following code demonstrates these concepts.

Note: FairCom DB has an internal hard limit of 65534 contexts.

Multiple ISAM Context Example

#define namebuf 32

struct { /* data record format */

TEXT name[namebuf]; /* customer name */

LONG num; /* customer number */

} parentimage;

FILNO datno, keyno;

COUNT contextID1, contextID2;

TEXT target[namebuf];

if ((contextID1=OpenISAMContext(datno,keyno,-1)) == -1)

printf("\n\tOpenISAMContext1 error - %d", isam_err);

memset(target,' ',namebuf);

cpybuf(target,"Columbia",strlen("Columbia"));

if (error =

GetRecord(keyno,TransformKey(keyno,target),&parentimage))

printf("\n\tGetRecord error - isam_err = %d",isam_err);

if ((contextID2=OpenISAMContext(datno,keyno,-1))==-1 )

printf("\n\tOpenISAMContext2 error - %d",isam_err);

memset(target,' ',namebuf);

cpybuf(target,"Denver",strlen("Denver"));

if (error =

GetRecord(keyno,TransformKey(keyno,target),&parentimage))

printf("\n\tGetRecord error - isam_err = %d",isam_err);

if (ChangeISAMContext(contextID1))

printf("\n\tError on CHGICON 1 = %d",isam_err);

if (!(error=NextRecord(keyno,&parentimage)))

printf("\n\tSuccessful NXTREC for context1-city name: %s.",

parentimage.name);

These routines permit a similar form of control as the multiple SET and BATCH routines, with the following essential differences:

• An ISAM Context is tied to one and only one data file. Each data file has a current ISAM context. A set or batch can be used with any key file for any associated data file.
• When a file is closed, its associated contexts are closed. Closing an individual file does not affect the existence of the multiple SET (or BATCH) buffers.
• CloseISAMContext() removes the context buffers for one particular ISAM context. FreeSet() and FreeBatch() remove the buffers for all the multiple sets and batches.
• Neither sets nor batches permit FairCom DB to automatically assign set or batch numbers. OpenISAMContext() allows FairCom DB to automatically assign the context ID if it is passed in as negative one (-1).
• Calling AllocateSet() or AllocateBatch() is not necessary. Calling OpenISAMContext() is necessary.

ISAM Error Handling

All ISAM functions return an error code value. Zero return indicates success otherwise, an error has occurred. Each call to an ISAM function sets the values of two global variables, and possibly a third:

• isam_err - contains the error code value.
• isam_fil - contains the data or index file number involved in the error.

Since each ISAM function can work with multiple files, check the value of isam_fil to see which file caused the error.

• sysiocod - contains the system-level error code.

The global variable sysiocod is set to the value of errno when an I/O error occurs. errno is the C Language run-time error variable automatically maintained by the C runtime library. Unlike uerr_cod, sysiocod is NOT reset by new calls to c-tree. It is only set to errno if an open, create, seek, read, write, or lock function fails. The sysiocod value can be found in the compiler documentation or compiler errno.h file. When the following error values are returned, check sysiocod for a system error.

Better Error Reporting when Exceeding the Maximum Length of VARCHAR Fields

In versions prior to FairCom DB V12 and FairCom RTG V3, when ISAM inserts a string in a VARCHAR field that exceeds the maximum size allowed by SQL, the following occurs: an error is reported, the record is truncated, and a panic is logged in CTSTATUS.FCS. This is desirable behavior, but the error code does not help identify the error and offending rows are not identified by Select * from table ctoption(badrec).

In this release, a more helpful error code of "bad record" is returned, and the offending rows are identified by Select * from table ctoption(badrec).

For example, if ISAM updates a field with 10,000 characters when the maximum number of characters in SQL is 8,000, a "bad record" occurs, a panic is logged in CTSTATUS.FCS, and Select * from table ctoption(badrec) identifies the offending rows.

ISAM Function Overview

Several ISAM functions have two versions - a standard version and an extended version. The extended version has the same name as the standard version with the suffix of Xtd. The extended version allows access to advanced functionality such as the security features of the FairCom Server.

The functions in the first section deal with Incremental ISAM structures. The functions in the second section deal with ISAM parameter files. The remaining sections work with Incremental ISAM structures and ISAM parameter files.

Incremental ISAM System Functions

These functions work with Incremental ISAM structures in the application.

CloseIFile

Closes a data file and its associated indexes, based on an incremental ISAM structure.

CloseISAM

Closes all of the data files and indexes opened with calls to OpenISAM(), CreateISAM(), OpenIFile(), or CreateIFile(). One call to the function is all that is necessary. When using the FairCom Server this call also logs you off the Server.

CloseRFile

Closes a data file and its associated indexes that have been opened with the OpenFileWithResource() function.

CreateIFile, CreateIFileXtd, and CreateIFileXtd8

Creates a data file and its associated indexes, based on an incremental ISAM structure.

GetIfile

Retrieve the IFIL structure stored in the file’s resource.

InitISAM and InitISAMXtd

Prior to calling OpenFileWithResource(), OpenIFile(), or CreateIFile(), you must initialize the c-tree system. This allocates index buffer and file handle space for the files opened as a part of the incremental ISAM system. When using the FairCom Server the extended call also logs on to the Server.

OpenIFile and OpenIFileXtd

Opens an existing data file and its associated indexes using an incremental ISAM structure.

Note: This function supports EXCLUSIVE file opens. For more information, please refer to Multi-user File Mode.

OpenFileWithResource and OpenFileWithResourceXtd

Similar to OpenIFile(), but instead of using an incremental ISAM structure declared in the program it uses similar information stored as a Resource in the data file when it was created with CreateIFile().

PermIIndex and PermIIndex8

Adds one or more new incremental ISAM indexes and automatically adds key values from the existing records. The information about the new index is added to the incremental ISAM resource.

RebuildIFile, RebuildIFileXtd, and RebuildIFileXtd8

If a data file or index becomes corrupt, use RebuildIFile() to recover them. Based on the information found in an incremental ISAM structure, RebuildIFile() goes through every record in the data file to update the file header and then recreates all the required keys.

TempIIndexXtd and TempIIndexXtd8

Similar to PermIIndex() in that one, or more, indexes will be created, with the existing records being added to the indexes. However, the incremental ISAM resource is NOT updated.

ISAM Parameter File System Functions

These functions work with the information found in the ISAM Parameter File.

CloseISAM

Closes all of the data files and indexes opened with calls to OpenISAM(), CreateISAM(), OpenIFile(), or CreateIFile(). One call to the function is all that is necessary. When using the FairCom Server this call also logs you off the Server.

CreateISAM and CreateISAMXtd

Creates all data files and indexes described in the ISAM parameter file. When using the FairCom Server this call also logs you on to the Server.

OpenISAM and OpenISAMXtd

Opens all data files and indexes described in the ISAM parameter file. The files and indexes must already exist. When using the FairCom Server this call also logs you on to the Server.

Note: This function supports EXCLUSIVE file opens. For more information, please refer to Multi-user File Mode.

Data File and Index Functions

AddRecord

Adds a fixed-length data record to a file and updates all associated indexes.

AddVRecord

Adds a variable-length data record to a file and updates all associated indexes.

CurrentFileOffset

Retrieves the byte position of the current ISAM record.

CurrentISAMKey

Supplemental function for building an ISAM key from a data record image.

DeleteRecord and DeleteVRecord

Deletes the current data record and removes the associated keys from the indexes.

GetRecord and GetVRecord

Finds the target key in an index, and reads the matching data record.

FirstRecord and FirstVRecord

Reads either the first physical record in a data file or the record that has the first key in a particular index.

GetGTERecord and GetGTEVRecord

Reads the record with a key that is greater than or equal to the target key.

GetLTERecord and GetLTEVRecord

Reads the record with a key that is less than or equal to the target key.

LockISAM

Enables or frees locks

LastRecord and LastVRecord

Reads either the last physical record in a data file or the record that has the last key in a particular index.

NextRecord and NextVRecord

Reads the next record in the data file, either in sequential physical order or key order. This is relative to the current ISAM record.

PreviousRecord and PreviousVRecord

Reads the previous record in the data file, either in sequential physical order or key order. This is relative to the current ISAM record. Note the limitations of this function in the function reference section.

ReadIsamData and ReadIsamVData

Read the data record at a given record position. Does not access the index, since the record offset within the data file is known.

ReReadRecord

Reread current ISAM record. Only reads the fixed portion of a variable-length record.

ReReadVRecord

Unlike the variable-length API functions, functions such as FirstRecord() read only the fixed-length portion of the data record. ReReadVRecord() reads the entire variable-length record after a fixed-length function. Make sure that your buffer is large enough by getting the record length with VRecordLength().

ResetRecord

Updates the current ISAM record. Use this to manipulate the c-tree internal representation of the current ISAM record, such as restoring the current ISAM record to the record position before a rewrite.

ReWriteRecord

Rewrite the current fixed-length data record.

ReWriteVRecord

Rewrite the current variable-length data record.

ReWritePartialRecord

Rewrite only a first portion of the current fixed or variable-length record.

SetRecord

Allows you to alter the current ISAM buffers and data record image location.

TransformKey

The ISAM parameters for your application specify how a key is to be built. This may include translations of segments of the key. You can use TransformKey() to build a target according to the ISAM parameters. Many ISAM functions require a properly transformed target.

VRecordLength

Returns the length of current variable-length ISAM data record.

Set Functions

A "set" is a group of records that have keys that match a certain target. These functions are used to define and access Sets. For example, an index contains the following keys:

CASE DISKCASE DISKDRIVE DISKETTE KEY KEYBOARD KEYCAP

When using functions like FirstRecord() and NextRecord() to scan through the index, looking for records, these functions work on the entire index file. FirstRecord() finds the first key in the file, CASE, and NextRecord() gives each of the following keys in turn, on through KEYCAP. If you define a set with a target of “DISK” and a target length of 4, FirstInSet() finds the key DISKCASE. NextInSet() gives each of the following keys in turn, with DISKETTE being the last key found. This allows you to establish subsets of an index, and process them as if they were a separate index.

Normally one set is worked with at a time, as established by the functions FirstInSet(), LastInSet(), or PositionSet(). However, it is also possible to define and manage multiple sets simultaneously. Call ChangeSet() with a unique set number when a new set is necessary. Each time ChangeSet() is called with a unique, previously unused, set number, AllocateSet() creates a new set of key buffers. To change between the active sets, use the unique set numbers passed at the initial ChangeSet() calls. ChangeSet in the function reference section has a good working example of this behavior.

AllocateSet

Internal c-tree function called by ChangeSet() to allocate ISAM key buffers for multiple, simultaneous sets.

ChangeSet

If you are using multiple, simultaneous sets, you must use this function to specify which set is going to be used by a set function call. If the value passed to this function has not been previously used, AllocateSet() is called automatically.

DoBatch

Performs operations on a set of records. For instance, it can delete all records in a set, read all records in a set into a buffer, or count the number of records in a set.

FirstInSet and FirstInVSet

Establishes a set based on a target key value and length. Reads the first record of the set into the buffer, making it the current ISAM record.

FreeSet

Frees memory allocated by AllocateSet(). CloseISAM() also frees the set buffers.

LastInSet and LastInVSet

Reads the last record in a set. Establishes a set, as does FirstInSet().

NextInSet and NextInVSet

Equivalent to NextRecord(), except that it works on the current set instead of the entire index. NextInSet() is limited to working on index files.

PositionSet and PositionVSet

Establishes a set, as does FirstInSet(), but the set is based on the key value for the current ISAM record, rather than on a target value passed as a parameter.

PreviousInSet and PreviousInVSet

Equivalent to PreviousRecord(), except that it works on the current set instead of the entire index. PreviousInSet() is limited to working on index files.

ISAM Examples

The example databases maintain a customer file indexed by name, number, and Zip Code. The names are maintained alphabetically by the first 14 bytes of the last name and the first 6 bytes of the first name and an automatic sequence number. The customer numbers are handled as 4-byte integers. The Zip Code index is based on the nine characters of the Zip Code and the first 9 bytes of the last name and the same automatic sequence number. This yields an alphabetical ordering of identical Zip Codes.

c-tree includes example files which make up two complete database applications using c-tree ISAM routines:

IFIL Examples

Parameter File Examples (Legacy)

A Simple Application

We use a simple file maintenance application as an example, as we did with the low-level functions. The records are fixed-length, and do not allow duplicate keys. The complete source code for this simple application can be found in isam.c. It is explained in more detail in the Sample Applications chapter. We will just outline the application here.

Outline

As with the low-level functions, the outline of our application is:

• Initialize the file system.
• Ask if the user wants to Add, Delete, or Modify a record.
• When done, close the file system.

ISAM Parameters

The characteristics of the data files and indices used in your application program can be defined in two different ways, either with ISAM structures defined in your application or with an ISAM Parameter File (the ISAM Parameter File is considered legacy).

The ISAM structures specify the layout of each data file and its associated indices in a set of structures in your application. Data and index files are created or opened with CreateIFile(), OpenIFile(), and OpenFileWithResource().

The ISAM Parameter File is a simple ASCII file created with a text editor. It contains records specifying the characteristics of all data files, indices, and keys for an application. OpenISAM() and CreateISAM() use this file to initialize the system. These files are considered legacy.

Initially, we will discuss using ISAM structures, also referred to as Incremental ISAM structures. Later we will discuss the use of ISAM Parameter Files. Although both methods are covered in detail, FairCom recommends using the ISAM Structures. This method provides individual file open, close, and rebuild control, and offers advanced features not available with the ISAM Parameter File, such as file definitions automatically stored at file creation time. This capability is recommended for use with the FairCom Drivers and will be required in some future FairCom R&D projects.

Beginning Your Application

For your application to function with the c-tree libraries, and to take advantage of various c-tree #defines, at the top of each application you must have the following include statement:

#include "ctreep.h"

Initialize c-tree and open files

The first three functions, InitISAM(), CreateIFile(), and OpenIFile(), are specific to ISAM Structures.

InitISAM

The first step is initializing the c-tree system with InitISAM(), which allocates memory for buffers and files.

CreateFile

If the data files and indexes do not already exist, use CreateIFile() to create each data file and its associated indexes.

OpenFile

If the data files and indexes already exist, use OpenIFile() to open each data file and its associated indexes.

Note: This function supports EXCLUSIVE file opens. For more information, please refer to Multi-user File Mode.

The next two functions, OpenISAM() and CreateISAM(), are specific to ISAM Parameter Files (legacy).

OpenISAM

The first step needed is to initialize the c-tree system. This is done with a call to OpenISAM(). This takes the information stored in the ISAM Parameter File, allocates memory for buffers and files, and opens all of the data files and indexes.

Note: This function supports EXCLUSIVE file opens. For more information, please refer to Multi-user File Mode.

CreateISAM

If the data files and indexes do not already exist, use CreateISAM() to create them.

The remaining functions in this section work with both Incremental Structures and ISAM Parameter Files (legacy).

Add a record

AddRecord

In the low-level function example we had to call three functions to add a record to the file system; NewData() to get the data record, WriteData() to write the record, and AddKey() to add the key to the index. With the ISAM functions, the process is much simpler. All that is needed is a call to AddRecord(). This performs all of the functions necessary to add the record and key.

Find a record

GetRecord

To find a key and read the associated record into our application we simply make a call to GetRecord(). This function finds the match to the key and reads the record into your data buffer. More sophisticated searches can be done for partial keys, or scanning up and down through the index.

Delete a record

DeleteRecord

This function will delete a record from the data file, and its key from the index. The record deleted is the current ISAM record, defined later in this chapter.

Close the system

When the application is ready to exit, it should close the data files and indexes. If not, and records or keys were added or deleted, the files will be damaged. Closing the files updates the header information to the file.

CloseISAM

This function only needs to be called once to close all open files. Every data file and index opened with an ISAM level open call, such as OpenIFile() or OpenFileWithResource(), will be closed with this one function call.