FairCom DB SQL Troubleshooting

In This Chapter
Java Configuration for FairCom DB SQL Stored Procedures, Triggers and User Defined Functions
Setting/Enabling Advanced Features in SQL Explorer
SRLSEG not Available in FairCom DB SQL When ROWID is Used
What are .fdk Files in the SQL_SYS Directory?
What is __Master.dbs?
Error While Creating SQL Database
DBLOAD Debugging Help
Stored Procedure Error: Could not initialize class sun.util.calendar.ZoneInfoFile
Stored Procedure Java Class Resolution
When do I have to specify the owner of a table?
How do I convert tables in a database to be case insensitive?
Analyzing JVM Memory usage with FairCom DB SQL Java Stored Procedures
Compiling FairCom DB SQL PHP on Linux/Unix

Java Configuration for FairCom DB SQL Stored Procedures, Triggers and User Defined Functions

FairCom DB SQL requires the Java 1.7 JDK and JRE environment for Stored Procedures, Triggers, and User Defined Functions (UDFs), JDBC, and FairCom DB API Java (FairCom DB SQL V11.0 requires 1.6 or newer). Java is readily available from the Oracle Java downloads website. FairCom DB supports Java on any platform that the Java environment is currently available, including Windows, AIX, Oracle Sun, and Linux.

Check with FairCom for the latest Java compatibility announcements and availability of the latest Java support.

Windows users, see Using Java 1.7 or Later on Windows.

The Java JDK and JRE for many platforms are available at no cost. Click here to download.

Note: 64-bit versions of FairCom DB SQL require a 64-bit version of the JVM to implement stored procedures.

 

FairCom DB SQL Configuration

Use the following FairCom Server configuration keywords to set your Java environment to take advantage of full stored procedures, triggers, and user-defined functions support.

; JDK environment settings - Be sure to set the JDK to your version.
SETENV CLASSPATH=C:\Program Files\Java\jdk1.7.0_07\jre\lib\rt.jar;C:\FairCom\VX.X.X\win32\bin\ace\sql\classes\ctreeSQLSP.jar (where VX.X.X refers to your FairCom DB version number)
SETENV JVM_LIB=C:\Program Files\Java\jdk1.7.0_07\jre\bin\server\jvm.dll
SETENV JAVA_COMPILER=C:\Program Files\Java\jdk1.7.0_07\bin\javac.exe

Setting/Enabling Advanced Features in SQL Explorer

FairCom DB SQL provides numerous options for advanced diagnostics. These can be enabled from your configuration file, built-in stored procedures, or through the FairCom DB SQL Explorer tool itself.

The FairCom DB SQL Explorer utility is an extremely useful tool in administering your SQL databases.

Enable advanced features in FairCom DB SQL Explorer:

To enable advanced features, call FairCom DB SQL Explorer from a command line adding the -adv option:

c-treeSQLExplorer.exe -adv
  • An alternative approach is to make a shortcut to the FairCom DB SQL Explorer executable. Right-click on the shortcut, select Properties, and append the -adv option to the command in the box labeled Target.

You can now open FairCom DB SQL Explorer and choose File > Debug Flags. Select one or more of the following:

Sql Opt Hm Itm
Cache Remt Phm
Ddm Display Cost Log Error
Xec (for advanced debugging) Hm Hdl SP/T Debug
SP/T Logging    

FairCom DB SQL Explorer's log (trace file)

You will find the output of FairCom DB SQL logging in the following file, found in the server's working directory:

<FairCom-Folder>\bin\ace\sql\data: file\sql_server.log

See Also

SRLSEG not Available in FairCom DB SQL When ROWID is Used

Only 1 serial segment mode (SRLSEG) is allowed per data file.

ROWID is used by default with FairCom DB API and FairCom DB SQL and uses the SRLSEG mode. SRLSEG also imposes a performance overhead as it is a special index. This mode can be disabled on a file by file basis at file creation time.

FairCom DB IDENTITY support provides a much enhanced auto-numbering option, while retaining SRLSEG for ROWID use. From SQL this is easy to include in your tables:

CREATE TABLE mytable (name CHAR(20), age INTEGER, name_id INTEGER IDENTITY(1,1));

Temporarily Disable SRLSEG or IDENTITY

What are .fdk Files in the SQL_SYS Directory?

FairCom DB SQL maintains the SQL system tables in the <dbname>.dbs\SQL_SYS\<dbname>.fdd file. This file is a standard FairCom DB superfile; a file containing all of the system tables and indexes. For example

.\ctreeSQL.dbs\SQL_SYS\ctreeSQL.fdd

There is one .dbs directory and .fdd system table per SQL database.

When FairCom DB SQL upgrades require changes to the system tables, a conversion automatically takes place upon server startup, with all databases listed in the session dictionary (ctdbdict.fsd). The process is as follows:

  1. The existing <dbname>.fdd file is copied to <dbname><timestamp>.fdk
  2. FairCom DB SQL performs the conversions on <dbname>.fdd
  3. A template database, __Master.dbs, database is created and comparisons made for validation

Each database found is converted in a separate transaction. If the conversion fails for any reason, for example, an abnormal shutdown of the server, <dbname>.fdk can be copied back to <dbname>.fdd. Check CTSTATUS.FCS for any reported conditions when updating FairCom DB SQL.

The following FairCom DB SQL versions require system table upgrades:

  • V9.5
  • V9.3
  • V8.27

Note: To prevent a database conversion, the SQL_OPTION NO_DB_CONVERSION keyword can be added to the configuration file. This should only be used upon the recommendation of FairCom support.

 

What is __Master.dbs?

__Master.dbs is a template database layout generated by <FC_PRPD_ACE> when it needs to upgrade existing databases to a new version (e.g., when upgrading from V10 to V11). It is a template form of the new database format which <FC_PRPD_ACE> uses during the upgrade.

It is automatically removed after the upgrade, although in some situations it may be left behind. If the server starts and stops correctly, it can be removed.

Error While Creating SQL Database

While creating a SQL database, it is possible to encounter the following error:

Error while creating the SQL Database: 19

When the ctreesql server is started, it will check for the file ctdbdict.fsd (the session dictionary) which has a list of all known CTDB and SQL databases. If this file doesn't exist it will be created. Next, it will check the session dictionary for the SQL_DATABASE specified in ctsrvr.cfg (ctreeSQL by default). If this database isn't known, the server attempts to create a new one.

This creation operation may fail with error 19, indicating that the database already exists on disk (but it isn't listed in ctdbdict.fsd).

To use this (or any other) existing database, you can add a reference to ctdbdict.fsd using the ctsqlcdb.exe command line utility, found in tools\cmdline\admin\client\.

Use the following command to add an existing database named ctreeSQL:

run: ctsqlcdb -add ctreeSQL FAIRCOMS

If you don't want the database that is on disk, delete the directory ctreeSQL.dbs (or SQL_DATABASE.dbs for a database called “SQL_DATABASE”), and run the following command to create a new database named ctreeSQL:

ctsqlcdb -create ctreeSQL FAIRCOMS

You can also use c-treeISAMExplorer to add or create databases by right-clicking on the root of the tree in the left-hand pane.

DBLOAD Debugging Help

By setting an environment variable, DBLOAD_DEBUG=Y the FairCom DB SQL dbload utility will output a large volume of additional internal information useful when debugging usage of the utility.

Stored Procedure Error: Could not initialize class sun.util.calendar.ZoneInfoFile

The following Java exceptions can be found when compiling stored procedures or triggers with Java 1.7 using the ctreeSQLSP.jar file compiled with Java 1.6:

java.lang.AssertionError: Default directory is not an absolute path
java.lang.NoClassDefFoundError: Could not initialize class sun.util.calendar.ZoneInfoFile

Compiling with Java 1.6 resolves the problem.

This has been corrected as of FairCom DB SQL V10.1.

Stored Procedure Java Class Resolution

Occasionally a stored procedure does not behave as expected. One possible cause can be improper resolution of the Java class. For example, a custom jar in the CLASSPATH may include a class using the same name as our stored procedures generate.

The class resolution can be traced using these procedures:

  1. Add the followig to ctsrvr.cfg
SETENV DH_JVM_OPTION_STRINGS=-verbose:class;-verbose:jni

(DH_JVM_OPTION_STRINGS may contain a variety of Java options, for example, you may be increasing the default Java heap size with -Xmx, etc. Simply add the verbose options to the string.)

  1. Start ctreesql.exe from the command line, and redirect stderr to a file:
ctreesql.exe 1>err.log
  1. Connect and call the stored procedure in question (e.g., admin.p_sp_getfddversion()).
  2. Check err.log for the results. Here is the expected load of procedure in err.log:
[Loaded admin_p_sp_getfddversion_SP from __JVM_DefineClass__]

Here is an unexpected version being loaded from a .class file sitting in the CLASSPATH:

[Loaded admin_p_sp_getfddversion_SP from file:/Q:/ctreesrc/ipv6/ctreeAPI/bin.sql/data/]

When do I have to specify the owner of a table?

The answer depends on the SQL owner of the table:

  • the user who creates the table owns the table and does not need to specify the owner's name
  • users who did not create the table must specify the owner's name

In SQL, any user can create a table within their user space (a "schema"). The user name in FairCom DB SQL acts as the schema name. If no schema is explicitly specified, the user is the assumed schema. The user who created the table does not need to specify their own name when accessing the table.

To access tables outside of your schema, you must have the appropriate privileges (through GRANT or by being an ADMIN) and you must explicitly specify the owner of the table (the schema).

Example:

The ADMIN user creates a table called "custmast" and GRANTs SELECT privileges to a user called JOHNDOE. This allows JOHNDOE to access this table by explicitly specifying "ADMIN"."custmast" to properly distinguish the correct "custmast" table. (Quotes not necessary in most cases.)

Imported Tables

With imported tables, the owner defaults to the user who authenticated with the import utility. If the user has ADMIN privileges, a different owner can be specified with the ‑o switch. See ctsqlimp Usage in the c-treeSQL Server Operations and Utilities Guide.

Setting a Default Schema

To set a default "schema" use the SET SCHEMA statement. If JOHNDOE executes the following, then "ADMIN".<tablename> will be assumed for tables without an explicit schema for the remainder of JOHNDOE's session:


SET SCHEMA ADMIN

SET SCHEMA can be called as needed within a session.

How do I convert tables in a database to be case insensitive?

Sometimes you have data and index files that were created with the case-sensitive option and you want to make them case-insensitive. The SQL_OPTION DB_CASE_INSENSITIVE option enables case-insensitive comparisons, sorting, and identifiers within a database. This option is a database-level attribute that is set when a database is created, so it only affects databases that are created after it is enabled.

Notice that this option does not actually convert your existing tables to be case-insensitive. Instead it is used in a process of creating new, case-insensitive tables and importing the existing data into them.

This option affects the key segment definition. An index created in a database that has been created as case-insensitive uses the USCHSEG segment mode instead of SCHSEG.

How To

If you have data and index files that were created with the case-sensitive option and you want to use them in a case-insensitive database, follow these steps:

  1. Add the SQL_OPTION DB_CASE_INSENSITIVE keyword to ctsrvr.cfg.
  2. Use the ctsqlcdb utility to create the new database.
  3. Use PUTIFIL() to change your segment modes to case-insensitive segment modes. You will need to write some code to do this (it is not available in a utility).
  4. Rebuild the indexes.
  5. Use the SQL import utility (ctsqlimp) to import the tables into the new database.

The new database will be case-insensitive.

Analyzing JVM Memory usage with FairCom DB SQL Java Stored Procedures

Find the committed and maximum sizes of the JVM heap in your ctreesql process.

  1. Run jconsole (found in the JDK's bin directory).
  2. In the New Connection dialog box, choose the Local Process entry with the process ID of the ctreesql process and click Connect.
  3. Select the Memory tab and view the Heap Memory Usage chart. What are the values of the Committed and Max heap sizes that are shown below the graph? In the example below it is "Committed: 59,072 bytes", which is 59 MB, and "Max: 699,072 kbytes" which is about 699 MB. (Note that even if you click "Perform GC" and the used amount of the heap is reduced, the committed value, which is the actual memory in use by the JVM and not available for other purposes, does not change.)

If you find that the JVM is using a significant amount of memory, FairCom recommends setting smaller initial and maximum JVM heap sizes, with the goal of making more memory available for FairCom DB SQL use. This is accomplished using the following option in ctsrvr.cfg and restarting FairCom DB SQL:

SETENV DH_JVM_OPTION_STRINGS=-Xms100m;-Xmx100m

This example sets initial and maximum heap sizes to 100 MB each. After setting this option and restarting FairCom DB SQL, run jconsole again and you should expect to see Committed and Max values are both 100 MB.

Advanced JVM Configuration

The JVM is loaded into FairCom DB SQL and becomes part of the process when the appropriate configuration options point to a valid Java Runtime Engine (JRE). Stored procedures, triggers, and user-defined functions are executed within this JVM. The JVM environment can be configured and tuned with numerous options. The deployment architect should be familiar with these options from the available Oracle Java documentation.

http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html

The SETENV DH_JVM_OPTION_STRINGS configuration keyword can be used to specify exact parameters for the JVM to use at runtime.

Note: Java stored procedures and triggers developers may take advantage of many different Java features and have wide latitude in creating stored procedure logic. Due to this, it is difficult to make general recommendations, however, information and links are provided below to demonstrate the vast variety of options available.

 

Different options may be available on different platforms as Java is a highly cross-platform environment.

 

Heap Memory

One of the most important considerations is memory usage of the JVM. This can result in unexpected memory usage of the FairCom DB SQL process. As the JVM gradually allocates additional heap memory, the process space can grow quite unexpectedly. While Java takes advantage of advanced automatic garbage collection of unused memory references, the triggers for this are many, and in some cases, require careful tuning for the best balance of performance and memory use. Two options of immediate use are the minimum and maximum size of the memory heap.

The JVM defaults to certain proportions of available memory, depending on the OS and if the process is 32-bit or 64-bit.

The initialheap size of the JVM is the following:

  • Larger of 1/64th of the machine's physical memory or some reasonable minimum. Before J2SE 5.0, the default initial heap size was a reasonable minimum, which varies by platform.
  • If the nursery size (the part of the heap memory reserved for allocation of new objects) is set with -Xns, the default initial heap size will be scaled up to at least twice the nursery size.

You can specify the initial (and minimum) JVM heap size with the -Xms option.

SETENV DH_JVM_OPTION_STRINGS=-Xms:4m

Note: The initial Java heap cannot be set to a value smaller than 8 MB, which is the minimum Java heap size. If you attempt to set it to a smaller value, JVM defaults to 8 MB.

 

The -Xms value cannot exceed the value set for -Xmx (the maximum Java heap size).

The maximum heap size of the JVM is the following:

  • Smaller of 1/4th of the physical memory or 1GB. Before J2SE 5.0, the default maximum heap size was 64MB. You can specify the maximum JVM heap size with the -Xmx command-line option.
SETENV DH_JVM_OPTION_STRINGS=-Xmx:16m

http://docs.oracle.com/javase/7/docs/technotes/guides/vm/gc-ergonomics.html
 

Alternative Garbage Collectors

Alternative garbage collectors (GC) are available within the JVM. For particular environments, it may be advantageous to choose an alternate GC for throughput or latency reasons.

  • -XX:+UseParallelOldGC (default)
  • -XX:+UseConcMarkSweepGC
  • -XX:+UseG1GC (New in Java 6 and 7)

Many numerous options are available to tune each of the Java garbage collectors for performance, throughput and low-latency. Check the current Java documentation for your chosen JVM for complete details.

GC Monitoring

It is frequently necessary to examine actual garbage collection statistics. Java garbage collection has many debugging options available for this task. Here is a minimum set to consider:

-Xloggc:<filename>
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintTenuringDistribution
-XX:+PrintGCApplicationConcurrentTime 
-XX:+PrintGCApplicationStoppedTime

Monitoring Tools

The JVisualVM utility included with your Java installation is recommended for advanced monitoring of the Java GC process. Load the Visual GC plugin from the "Tools->Plugins" menu.

  • See Debugging Java Stored Procedures

Compiling FairCom DB SQL PHP on Linux/Unix

FairCom DB SQL PHP is provided as a ready to go driver for currently available versions of PHP. However, PHP regularly advances, and new versions out pace current support quickly. In that case, you usually have to rebuild the PHP driver to match the specific version of PHP in your environment.

Usually, you can simply execute our provided build script from the /src directory of your FairCom DB SQL PHP driver installation:

> phpize
> ./configure --make-ctsql
> make
> make install

However, you may find that the configure scripts don't always match the current autotools version of your Linux flavor. In that case, you need to regenerate the ./configure script. Consider the following sequence. Actual steps may differ slightly depending on your autotools version and Linux flavor. Check with the autotools documentation for details.

> phpize
> libtoolize
> aclocal
> autoreconf -i
> ./configure --make-ctsql
> make
> make install (optional)

The driver will be in the newly created /modules folder. You are free to copy this to an appropriate library location in your environment.

Note: You may need elevated privileges to install this into /lib or other system locations.

 

IoT and FairCom Edge Help

In This Chapter
Core Files on the Raspberry Pi
Backing up the Raspberry Pi

Core Files on the Raspberry Pi

For Raspbian on Raspberry Pi, core dumps are disabled by default. This was done to save disk space on limited size SD cards, but it can be annoying if you are developing a program and need to run down a difficult crash.

To verify that core dumps are disabled you can use the ulimit command. ulimit-c will display the current core dump file size limit. Typically, it will be 0, meaning no core files at all.

You can enable core dumps in a particular terminal window using the same ulimit command. ulimit -c unlimited will enable core dumps for programs run within that terminal window until it is closed or until the Pi is restarted.

Enabling Core Dumps for the Entire Machine

If you want to enable core dumps for the entire machine you will have to edit the /etc/security/limits.conf file. Look for a line similar to the following:

#*              soft    core            0

Change that line to the following:

*               soft    core            unlimited

You will have to edit the file as a superuser. One way to do this is via nano:

sudo nano /etc/security/limits.conf

Use the arrows to move the cursor, make your changes, then hit Control+o return to save your changes and Control+x to exit.

Note: If you turn on core files for the entire machine, keep in mind any program that crashes for any reason will generate a core file and you may have to go find them and delete them if they start stacking up.

 

Backing up the Raspberry Pi

It can take quite a while to get a Raspberry Pi set up and configured just the way you like. It would be nice to be able to easily back up the SD card to avoid having to do all the setup again for a second Pi or in the event that you have to reformat the card and start over.

If your Pi is up-to-date, it will have an SD Card Copier utility you can use for to make a quick and easy backup. This utility is located in Raspberry Menu > Accessories.

Connect a second storage device using a USB SD card reader or a USB thumb drive. Be sure this device is at least as large as the SD card you are copying.

Run the utility and specify the following:

  • Set the Copy From Device to the device you booted from, which will have a name similar to /dev/mmcblk0.
  • Set the Copy To Device to the second storage device.

Click Start and the utility will duplicate the card you are booted from to the card or drive you connected.

COBOL Troubleshooting

In This Chapter
COBOL Compilers Supported by FairCom RTG COBOL
Troubleshooting Data Conversion Errors
Error: Requested def blk is empty
FairCom RTG/ISAM SQL Access Errors

COBOL Compilers Supported by FairCom RTG COBOL

Tested and Supported

  • Veryant isCOBOL
  • ACUCOBOL-GT 6
  • ACUCOBOL-GT 7
  • ACUCOBOL-GT 8
  • ACUCOBOL-GT 9
  • Micro Focus Visual COBOL
  • Micro Focus Net Express
  • Micro Focus Server Express 5
  • Micro Focus COBOL 4
  • Micro Focus Server Express 4
  • Cobol-IT
  • Fujitsu NetCOBOL for .NET
  • Fujitsu NetCOBOL for Windows
  • Fujitsu NetCOBOL for Linux
  • P3/COBOL

Not Tested: Likely Supported

  • Fujitsu PowerCOBOL (through EXTFH interface)

Not Tested

  • Fujitsu COBOL
  • CA Realia COBOL

Not Supported

  • OpenCOBOL

Troubleshooting Data Conversion Errors

The fact that the XDD initially does not contain any error handling information immediately exposes data conversion errors to a SQL user. This provides the way to begin troubleshooting data conversion errors and to identify the proper settings to specify in your XDD file.

Visually Check Data with FairCom DB SQL Explorer

FairCom DB SQL Explorer and FairCom DB Explorer include a button to simplify identifying "bad" records.

To check for bad records in FairCom DB SQL Explorer or FairCom DB Explorer:

  1. Select a sqlized table, such as custmast in the image above.
  2. Click the Table Records tab.
  3. A button labeled Check Bad Records appears at the right of the row of buttons (the image above shows the Java version of FairCom DB Explorer where the buttons are at the top of the tab; the .NET version, called FairCom DB SQL Explorer, shows this row of buttons at the bottom of the tab).
  4. Click the button to execute a SQL query to find records that did not sqlize properly.

Check using a SQL Query

You can use the procedures in this section to identify data-conversion errors and use that information to fine-tune your XDD files.

If a query fails, it is possible the failure is due to a problem with SQL data conversion. Troubleshooting this type of error is quite easy with the following steps:

  1. Identify the table (in case of a complex query) on which the conversion error occurs by running the following SQL statement on each table involved in the query:
SELECT * FROM <table>

If none of the queries fail, the original query failure is not due to a conversion problem.

  1. Run the following SQL statement to select only the records that do not properly convert:
SELECT * FROM <table> ctoption(badrec)

ctoption(badrec) is a FairCom DB extension to SQL indicating the query should return only records having conversion errors and expose values that do not properly convert as NULL.

  1. Look for NULL values returned from the query in step 2. These are the fields that do not properly convert. The remaining record values should be sufficient to identify the record that requires investigation.

CTSQLCBK.FCS Log

The ctoption(badrec) command generates a log file, ctsqlcbk.fcs, in the FairCom DB SQL Server directory that can be used to determine the exact conversion error and the data causing it. This file lists all the fields that caused a conversion error exposed in SQL along with the value of the data that could not be converted. Note that the log contains information for the fields that result in propagating the conversion error to SQL. It does not log conversion errors that result in a SQL value because they were already handled successfully following the settings in the XDD.

Each log entry is made of three lines:

  1. The first line is similar to the following:

Convert error XXXX on field [YYYY]

Where XXXX is the error code indicating the cause of the conversion error, YYYY is the field on which the conversion error occurred.

  1. The second line contains a message which gives internal information for FairCom technicians to identify where the error occurs in the code, as well as a message explaining the problem.
  2. The third line is a hexadecimal dump of the field content.

For example:

Convert error 4028 on field[FIELD1]
	{ctdbStringFormatToDateTime} on 0000000000 failed
	00000000
Convert error 4028 on field[FIELD2]
	{ctdbStringFormatToDate} on 00000000 failed
	3030303030303030
Convert error 4118 on field[FIELD3]
	[NormalizeCobolStr] ascii value is not a digit
	2020202020202020
Convert error 4118 on field[FIELD4]
	[NormalizeCobolStr] ascii value is not a digit
	2020202020202020

Error: Requested def blk is empty

The following message was seen when attempting to connect to FairCom DB SQL after a COBOL table had been imported with the ctutil-sqlize option:

Error : -17438 "ExecuteReader - CT - Requested def blk is empty"

Along with that, the following message is logged in CTSTATUS.FCS

Mon Oct  1 13:59:43 2012 
 - User# 00001 LoadSQLSDK: Failed to load SQL callback library libctsqlcbk.so: dynamic linker : ./ctreesql : could not open libctsqlcbk.so 
Mon Oct  1 13:59:43 2012 
 - User# 00001 : PANIC - TPEUTIL LoadSQLSDK callback library load failed PID 20028

A COBOL resource in the file is closely related to the referenced callback routine needed by SQL for data type conversions.

Configuring the LD_LIBRARY_PATH environment variable to correctly pick up the path to libctsqlcbk.so for the server process owner should correct this error.

FairCom RTG/ISAM SQL Access Errors

The FairCom RTG/ISAM solution is enabled through a server-side callback module, which implements FairCom DB API callback routines. Errors that occur within these routines generate a standard FairCom DB API error code that is context sensitive to this implementation. Here is a list of possible return codes from this module, and their meaning in FairCom RTG/ISAM XDD handling.

Symbolic Error Code Description
CTDBRET_CALLBACK_1 4109 Could not find schema in XDD
CTDBRET_CALLBACK_2 4110 Record length does not match extended data definitions
CTDBRET_CALLBACK_3 4111 Invalid or corrupted XDD resource
CTDBRET_CALLBACK_4 4112 Syntax error parsing extended data definitions
CTDBRET_CALLBACK_5 4113 Internal error
CTDBRET_CALLBACK_6 4114 Could not find field in XDD
CTDBRET_CALLBACK_7 4115 Could not find filter in XDD
CTDBRET_CALLBACK_8 4116 Too many schemas in XDD
CTDBRET_CALLBACK_9 4117 Error setting filter condition on schema
CTDBRET_CALLBACK_10 4118 Field value conversion error
CTDBRET_CALLBACK_11 4119 Unsupported CLOB/BLOB definition in XDD
CTDBRET_CALLBACK_12 4120 Index not supported on field type
CTDBRET_CALLBACK_13 4121 Missing or bad dbtype information in XDD
CTDBRET_CALLBACK_14 4122 Invalid filter key settings in XDD
CTDBRET_CALLBACK_15 4123 "Regular" field found after virtual fields in the XDD <schema> (in earlier releases: Invalid field default settings in XML definitions)
CTDBRET_CALLBACK_16 4124 Not used
CTDBRET_CALLBACK_17 4125 Key definition does not match XDD information
CTDBRET_CALLBACK_18 4126 Missing or bad value in XDD
CTDBRET_CALLBACK_19 4127 Internal error (unexpected field type)
CTDBRET_CALLBACK_20   Not used

Reference Material

In This Chapter
mtmake Command Line
Typical Unix Errors from errno.h Header File
FairCom DB Server Files
FairCom DB Utilities
NULL Handling
ISAM Parameter Files (Legacy)
Prototyping
2xx Internal Error Codes
749X Internal Error Codes
Google Chrome Tips for Developers

mtmake Command Line

FairCom DB mtmake Make Utility

The mtmake build utility can be used to configure your FairCom DB libraries. Note this is optional for client/server applications since the mtclient.lib (libmtclient.a) is included in the FairCom DB package, and it is mandatory for Standalone customers. You will find this utility in the drivers\build\build_sdk folder for FairCom DB and later (/pro directory for V11 and earlier) and in the build_sdk\build folder for the FairCom DB Standalone SDK package.

Note: Starting with V12 the "mtmake" subsystem is in a .zip file (a gzipped .tar ball on Linux/Unix). Go to buildsdk\build and expand the source*.zip to obtain access to the mtmake.exe make subsystem.

 

Note: If you have previously constructed automated build processes you may need to change some keystroke options passed to mtmake as they may have changed.

 

Advanced Options

Many users may not know that mtmake has numerous advanced options to tailor the FairCom DB libraries for specific needs or exacting size footprints (consider embedded devices). Execute mtmake with a question mark ("?") for a listing of these advanced options.

> mtmake ?

A display of options and their descriptions is shown below. Contact FairCom if you have any questions for how to tailor the FairCom DB libraries for your unique application needs.

Required Libraries for compiling applications

For examples of how to compile your client application, consult the ctree.mak file created when you execute mtmake.exe.

The following libraries are required when building C/C++ applications.

Linux:

OpenSSL 1.0.2t

SSL support needs to be installed on most Linux distros.

The correct version of OpenSSL is included in our package and our make file points to it.

To include the proper encryption system libraries, you will likely need to add the following link switches:

-lssl -lcrypto

You can review /drivers/c.isam/tutorials/cmdline/Makefile for a working example of building C and C++ applications.

Libz Compression support. needs to be installed on CentOS.
libncurses5 Text based interface is included on most Linux distros.
gcc, make Compiling tutorials are included on most Linux distros.
glibc Included in most Linux distros. If your glibc is before 2.17 you will also have to include the rt library (-lrt) for “clock_gettime”.

Windows:

MS Visual Studio Runtime
  • user32.lib
  • gdi32.lib
  • winspool.lib
  • comdlg32.lib
  • advapi32.lib
  • shell32.lib
  • kernel32.lib
  • oldnames.lib
  • crypt32.lib

Typical Unix Errors from errno.h Header File

sysiocod errors are system specific, and are reported directly from the operating system. For WIndows you can use the built-in helpmsg system to obtain the meaning of these errors:

>net helpmsg <msg_no>

From Unix, you need to reference the errno.h header file. Here is a typical file (from AIX) provided as a handy reference.

#define EPERM    1       /* Operation not permitted              */
#define ENOENT   2       /* No such file or directory            */
#define ESRCH    3       /* No such process                      */
#define EINTR    4       /* interrupted system call              */
#define EIO      5       /* I/O error                            */
#define ENXIO    6       /* No such device or address            */
#define E2BIG    7       /* Arg list too long                    */
#define ENOEXEC  8       /* Exec format error                    */
#define EBADF    9       /* Bad file descriptor                  */
#define ECHILD   10      /* No child processes                   */
#define EAGAIN   11      /* Resource temporarily unavailable     */
#define ENOMEM   12      /* Not enough space                     */
#define EACCES   13      /* Permission denied                    */
#define EFAULT   14      /* Bad address                          */
#define ENOTBLK  15      /* Block device required                */
#define EBUSY    16      /* Resource busy                        */
#define EEXIST   17      /* File exists                          */
#define EXDEV    18      /* Improper link                        */
#define ENODEV   19      /* No such device                       */
#define ENOTDIR  20      /* Not a directory                      */
#define EISDIR   21      /* Is a directory                       */
#define EINVAL   22      /* Invalid argument                     */
#define ENFILE   23      /* Too many open files in system        */
#define EMFILE   24      /* Too many open files                  */
#define ENOTTY   25      /* Inappropriate I/O control operation  */
#define ETXTBSY  26      /* Text file busy                       */
#define EFBIG    27      /* File too large                       */
#define ENOSPC   28      /* No space left on device              */
#define ESPIPE   29      /* Invalid seek                         */
#define EROFS    30      /* Read only file system                */
#define EMLINK   31      /* Too many links                       */
#define EPIPE    32      /* Broken pipe                          */
#define EDOM     33      /* Domain error within math function    */
#define ERANGE   34      /* Result too large                     */
#define ENOMSG   35      /* No message of desired type           */
#define EIDRM    36      /* Identifier removed                   */
#define ECHRNG   37      /* Channel number out of range          */
#define EL2NSYNC 38      /* Level 2 not synchronized             */
#define EL3HLT   39      /* Level 3 halted                       */
#define EL3RST   40      /* Level 3 reset                        */
#define ELNRNG   41      /* Link number out of range             */
#define EUNATCH  42      /* Protocol driver not attached         */
#define ENOCSI   43      /* No CSI structure available           */
#define EL2HLT   44      /* Level 2 halted                       */
#define EDEADLK  45      /* Resource deadlock avoided            */

#define ENOTREADY       46      /* Device not ready             */
#define EWRPROTECT      47      /* Write-protected media        */
#define EFORMAT         48      /* Unformatted media            */

#define ENOLCK          49      /* No locks available           */

#define ENOCONNECT      50      /* no connection                */
#define ESTALE          52      /* no filesystem                */
#define EDIST           53      /* old, currently unused AIX errno*/

/* non-blocking and interrupt i/o */
/*
 * AIX returns EAGAIN where 4.3BSD used EWOULDBLOCK;
 * but, the standards insist on unique errno values for each errno.
 * A unique value is reserved for users that want to code case
 * statements for systems that return either EAGAIN or EWOULDBLOCK.
 */
#if _XOPEN_SOURCE_EXTENDED==1
#define EWOULDBLOCK     EAGAIN   /* Operation would block       */
#else /* _XOPEN_SOURCE_EXTENDED  */
#define EWOULDBLOCK     54
#endif /* _XOPEN_SOURCE_EXTENDED */

#define EINPROGRESS     55       /* Operation now in progress */
#define EALREADY        56       /* Operation already in progress */

/* ipc/network software */

        /* argument errors */
#define ENOTSOCK        57      /* Socket operation on non-socket */
#define EDESTADDRREQ    58      /* Destination address required */
#define EDESTADDREQ     EDESTADDRREQ /* Destination address required */
#define EMSGSIZE        59      /* Message too long */
#define EPROTOTYPE      60      /* Protocol wrong type for socket */
#define ENOPROTOOPT     61      /* Protocol not available */
#define EPROTONOSUPPORT 62      /* Protocol not supported */
#define ESOCKTNOSUPPORT 63      /* Socket type not supported */
#define EOPNOTSUPP      64      /* Operation not supported on socket */
#define EPFNOSUPPORT    65      /* Protocol family not supported */
#define EAFNOSUPPORT    66      /* Address family not supported by protocol family */
#define EADDRINUSE      67      /* Address already in use */
#define EADDRNOTAVAIL   68      /* Can't assign requested address */

        /* operational errors */
#define ENETDOWN        69      /* Network is down */
#define ENETUNREACH     70      /* Network is unreachable */
#define ENETRESET       71      /* Network dropped connection on reset */
#define ECONNABORTED    72      /* Software caused connection abort */
#define ECONNRESET      73      /* Connection reset by peer */
#define ENOBUFS         74      /* No buffer space available */
#define EISCONN         75      /* Socket is already connected */
#define ENOTCONN        76      /* Socket is not connected */
#define ESHUTDOWN       77      /* Can't send after socket shutdown */

#define ETIMEDOUT       78      /* Connection timed out */
#define ECONNREFUSED    79      /* Connection refused */

#define EHOSTDOWN       80      /* Host is down */
#define EHOSTUNREACH    81      /* No route to host */

/* ERESTART is used to determine if the system call is restartable */
#define ERESTART        82      /* restart the system call */

/* quotas and limits */
#define EPROCLIM        83      /* Too many processes */
#define EUSERS          84      /* Too many users */
#define ELOOP           85      /* Too many levels of symbolic links      */
#define ENAMETOOLONG    86      /* File name too long                     */

/*
 * AIX returns EEXIST where 4.3BSD used ENOTEMPTY;
 * but, the standards insist on unique errno values for each errno.
 * A unique value is reserved for users that want to code case
 * statements for systems that return either EEXIST or ENOTEMPTY.
 */
#if defined(_ALL_SOURCE) && !defined(_LINUX_SOURCE_COMPAT)
#define ENOTEMPTY       EEXIST  /* Directory not empty */
#else   /* not _ALL_SOURCE */
#define ENOTEMPTY       87
#endif  /* _ALL_SOURCE */

/* disk quotas */
#define EDQUOT          88      /* Disc quota exceeded */

#define ECORRUPT        89      /* Invalid file system control data */

/* errnors 90-92 reserved for future use compatible with AIX PS/2 */

/* network file system */
#define EREMOTE         93      /* Item is not local to host */
#define ESOCKTNOSUPPORT 94      /* Socket type not supported */
#define EOPNOTSUPP      95      /* Operation not supported on transport endpoint */
#define EPFNOSUPPORT    96      /* Protocol family not supported */
#define EAFNOSUPPORT    97      /* Address family not supported by protocol */
#define EADDRINUSE      98      /* Address already in use */
#define EADDRNOTAVAIL   99      /* Cannot assign requested address */
#define ENETDOWN        100     /* Network is down */
#define ENETUNREACH     101     /* Network is unreachable */
#define ENETRESET       102     /* Network dropped connection because of reset */
#define ECONNABORTED    103     /* Software caused connection abort */
#define ECONNRESET      104     /* Connection reset by peer */
#define ENOBUFS         105     /* No buffer space available */
#define EISCONN         106     /* Transport endpoint is already connected */
#define ENOTCONN        107     /* Transport endpoint is not connected */
#define ESHUTDOWN       108     /* Cannot send after transport endpoint shutdown */
#define ENOSYS          109     /* Function not implemented  POSIX */

/* disk device driver */
#define EMEDIA          110     /* media surface error */
#define ESOFT           111     /* I/O completed, but needs relocation */

/* security */
#define ENOATTR         112     /* no attribute found */
#define ESAD            113     /* security authentication denied */
#define ENOTRUST        114     /* not a trusted program */

/* BSD 4.3 RENO */
#define ETOOMANYREFS    115     /* Too many references: can't splice */

#define EILSEQ          116     /* Invalid wide character */
#define ECANCELED       117     /* asynchronous i/o cancelled */

/* SVR4 STREAMS */
#define ENOSR           118     /* temp out of streams resources */
#define ETIME           119     /* I_STR ioctl timed out */
#define EBADMSG         120     /* wrong message type at stream head */
#define EPROTO          121     /* STREAMS protocol error */
#define ENODATA         122     /* no message ready at stream head */
#define ENOSTR          123     /* fd is not a stream */

#define ECLONEME        ERESTART /* this is the way we clone a stream ... */

#define ENOTSUP         124     /* POSIX threads unsupported value */

#define EMULTIHOP       125     /* multihop is not allowed */
#define ENOLINK         126     /* the link has been severed */
#define EOVERFLOW       127     /* value too large to be stored in data type */

#endif /* _ANSI_C_SOURCE */

#endif /* _H_ERRNO  */

FairCom DB Server Files

We want you to know how easy and simple your FairCom Server is to operate and manage. No registry entries or other system configurations are involved in the installation and maintenance of your FairCom Server. Only a small collection of server maintained "housekeeping" and transaction log files are used and are located in the server binary directory unless otherwise directed.

We briefly describe these files here so you can be confident in your knowledge of the FairCom Server installation.

License Object

  • ctsrvr-xxxxxxxx.lic - Your License Authorization File assigned with your serial number and licensed settings. Must be present for server to start. This file replaces prior activation steps.

Status Logs

  • CTSTATUS.FCS - FairCom Server Status Log File. Text based.
  • SYSLOGDT.FCS and SYSLOGIX.FCS - SYSLOG facility security logging data file. These are FairCom DB data and index files maintainable only by the FairCom Server.

User and Group Information

  • FAIRCOM.FCS - Maintains FairCom DB User and Group information in a standard FairCom DB data file format. Passwords are encrypted to prevent casual inspection. For a higher level of security, it is recommended to use the ADMIN_ENCRYPT configuration keyword to fully encrypt this file.

Transaction Logs

(located in <faircom>\tranlogs)

  • Lxxxxxx.FCS - Transaction Log Files ever increasing in number as a rotating set. A minimum of four are kept at any given time.
  • S0000000.FCS - Transaction Log Start File generated during checkpoints.
  • S0000001.FCS - Backup Transaction Log Start File generated during checkpoints.
  • Lxxxxxx.FCT - Transaction Log File template for enhanced performance in creating new transaction log files.

Housekeeping Files

  • D0000000.FCS - Preimage swap file
  • D0000001.FCS - Holds queue entries for the space reclamation of deleted superfile members and recovered variable length data files
  • I0000001.FCS - Index of the file IDs currently open with the FairCom Server
  • I0000002.FCS - FairCom DB Server Transaction Processing Lock File
  • DNODEQUE.FCS - Permanent delete node queue file for enhanced performance of the delete node thread
  • sql_server.log - FairCom DB SQL log file used to log SQL specific messages
  • ctdbdict.fsd - c-treeDB Database Session File
  • ctreeSQL.dbs/SQL_SYS/ctreeSQL.fdd - FairCom DB SQL dictionary file. (FairCom DB SQL System Tables)
  • FREOPENX.FCS - Backup stream files for use when file handles become scarce

FairCom DB Utilities

A large collection of FairCom DB utilities are available for monitoring and managing your FairCom Server. Here is a quick list of included utilities immediately useful for the most common administrative tasks.

Browser-Based Graphical Utilities

Browse and manage your c-tree Server environment from nearly anywhere on the network directly from your browser! V12 makes available a suite of browser‑based utilities featuring full-functioned visual user interfaces for monitoring operations and exploring your FairCom DB databases.

Periodic updating of these tools is also now easier than ever keeping your entire organization up-to-date. All components are centrally located on the server via a web service. Simply replace components when provided and all users will immediately enjoy the latest utility updates and features. No more desktop application version and installation hassles.

Details on these tools are available in FairCom Graphical Tools.

Command-Line Utilities

  • ctadmn - The Administrator utility is a menu-driven utility for performing many common administrative tasks.
    • Add and manage users and groups
    • List current logged on users
    • Stop the server
    • Pause the server for maintenance
    • View active files and locks
    • Set file passwords
  • ctstop - The Stop utility halts server operation.
  • ctpass - The Password utility is used to change the password for a FairCom Server user account.
  • sa_admin - The Security Administrator utility is used to add and change user and group security information. Easily create scripts with this utility.
  • ctinfo - The information utility provides useful information about FairCom DB files.
  • ctstat - The Statistics utility has many options for monitoring the FairCom Server. Over 200 metrics are available for monitoring.
  • ctdump - The Dynamic Dump utility submits scripts for instructing the FairCom Server to perform live "hot" backups.

You can learn more about these tools in Command-Line Utilities.

NULL Handling

NULL fields are used to represent "missing information and inapplicable information." The use of NULL allows a table to distinguish between an unknown value and a value of 0 or FALSE.

To properly handle NULL, the table must contain the $NULFLD$ field, a hidden field generated by FairCom DB API at creation time. Tables created with the FairCom DB API Interface (used under FairCom DB SQL) have a hidden field, $NULFLD$, which is used to determine if each user-created field in the record buffer has a NULL value. FairCom DB SQL requires this capability to implement constraints.

FairCom DB API and FairCom DB SQL will access tables without the $NULFLD$ field, but the table’s fields will always return a non-NULL status.

The ctdbClearRecord() function handles NULL fields when $NULFLD$ support is available.

ISAM Parameter Files (Legacy)

The ISAM parameters specify the characteristics of the data files and indexes used in an application program. The parameters are stored in a regular ASCII text file called the ISAM parameter file. If your application uses the high-level ISAM routines, you must construct an ISAM parameter file or use the incremental ISAM structures previously defined. Any text editor can be used to create or modify ISAM parameter files, provided special control characters are not embedded in the parameter file.

This section deals with the ISAM Parameter File. When using ISAM parameter files, all files in the parameter file are opened and closed together. With Incremental ISAM structures, the programmer has individual file open/close control and automatic file definition storage; making ISAM structures the recommended choice. Some applications work better with Incremental ISAM Structures. FairCom highly recommends using Incremental ISAM structures.

ISAM Parameter File Organization

The ISAM parameter file is composed of five types of records:

  • Initialization Record
  • Data File Description Records
  • Index File Description Records
  • Optional Index Member Records
  • Key Segment Description Records

Each parameter file begins with one Initialization Record which specifies, among other things, the number of data files. Then for each data file there are a group of records beginning with a Data File Description record. Each Data File Description record specifies the number of indexes it uses. For each of these indexes there is a group of records beginning with the Index File Description record or Index Member record. Each Index File Description record and/or optional Index Member record is followed by one or more Key Segment Description records.

The overall organization of the records is shown in the following schematic and reflects the hierarchical relationship among the data files and their corresponding indexes:

Parameter File Organization

Initialization Record

  Data File Description Record
    First Index File Description Record
      First Key Segment Description Record
      :::
      Last Key Segment Record

    Optional Index Member Records
      First Key Segment Description Record
      :::
      Last Key Segment Record
    :::

    Last Index File or Index Member Record
      First Key Segment Description Record
      :::
      Last Key Segment Record

  REPEAT FOR EACH DATA FILE

Parameter File Contents

Each of the fields comprising the parameter file records is separated from neighboring fields by one or more “white space” characters (blanks, tabs, or new lines). In the following subsections, if a parameter corresponds to a formal parameter in a FairCom DB function call, its name is included after the parameter’s brief description (e.g., bufs in the first position of the initialization record).

In This Section
Initialization Record
Data File Description Record
Index File Description Record
Optional Index Member Record
Key Segment Description Record

Initialization Record

The Initialization Record is made up of four numeric fields:

  • Number of index file buffers (bufs)
  • Number of indexes
  • Number of node sectors (sect)
  • Number of data files

The first and third parameters correspond to the first and third parameters of the InitCTree() function, bufs and sect, described earlier. For each index specified in the second parameter, there is either an Index Description record or an Index Member record. For each data file specified in the fourth parameter there is a Data File Description record. The Number of Node Sectors times 128 is the size, in bytes, of the B-Tree nodes. The number of data files specified by the fourth parameter of the Initialization Record must match exactly the number of Data File Description Records. However, the number of index files given by the second parameter can be equal to, OR GREATER than, the actual number of indexes. If greater, FairCom DB reserves extra file control blocks that your application may use with its own calls to OpenCtFile(), OpenIFile(), or for more flexible numbering of files in the parameter file.

Data File Description Record

Each Data File Description record is made up of six required fields and two additional fields if r-tree is to be used:

  • Data file number (datno)
  • Data file name (filnam)
  • Record length (datlen)
  • File size extension parameter (xtdsiz)
  • File mode (filmod; see Data Management in the FairCom DB Professional Developers Guide)
  • Number of associated index files

If RTREE is defined in your library, which is the default in ctopt2.h unless NO_RTREE is defined, each Data File Description record must have two additional fields:

  • Symbolic name (without any spaces) of the first field in the data record
  • Symbolic name of the last field in the data record

The first five fields correspond to the parameters of CreateDataFile(). The data file name must agree with the operating system’s file naming conventions. Usually it can include a disk drive designation and/or a directory path name. The record length parameter is used only when the file is created. However, it should be properly maintained since the rebuild utility verifies that the parameter file value matches the value in the data file header record. filmod specifies:

  • the multi-user access to the file
  • whether the file is opened permanently or virtually
  • whether the data file will have fixed- or variable-length records
  • the level of transaction processing

The sixth parameter specifies how many indexes will be maintained for the data file. Each such index will either be a separate FairCom DB file or a member of an index file. See “CreateIndexFile” and “CreateIndexMember” in the FairCom DB Professional Developers Guide. If RTREE is defined in ctoptn.h, then the seventh and eighth parameters must be present. If the parameter file is to be used in conjunction with r-tree, these parameters must match actual entries in the data object definition array, DODA, otherwise r-tree will report an error. The DODA is described in Record Schemas in the FairCom DB Professional Developers Guide. If the parameter file is not actually for use with r-tree, then any arbitrary symbolic names may be used as long as there are no embedded blanks.

Index File Description Record

Each Index File record has eleven required fields and one additional field if r-tree is to be used:

  • Index file number (keyno)
  • Index file name (filnam)
  • Key length (keylen)
  • Key type (keytyp; see Data Management in the FairCom DB Professional Developers Guide)
  • Duplicate flag (keydup)
  • Number of additional index file members (nomemb)
  • File size extension parameter (xtdsiz)
  • File mode (filmod; see FairCom DB Concepts in the FairCom DB Professional Developers Guide)
  • NULL key flag
  • Empty character
  • Number of key segments

If RTREE is defined in ctoptn.h, then one additional parameter is required:

  • Symbolic index name

The first eight fields correspond to the parameters of CreateIndexFile(). The index file name must agree with the operating system’s file naming conventions. Usually, the index name may contain a disk drive designation and/or a directory path name. The third through seventh parameters are used at the time the index file is created. However, they should be maintained carefully since the rebuild utility verifies that the ISAM parameter file values agree with the corresponding values stored in the index file header record. Remember, if duplicates are allowed, the last 4 bytes of the key value are reserved for the 4 bytes of the associated data record position. For example, a key length of 12 bytes combined with duplicate values results in 8 bytes for the actual key value and 4 bytes for the tie-breaking data record position. Skipping the ninth and tenth parameters for now (see the term “Null Key Values” below), the eleventh parameter specifies the number of segments concatenated to form a key value from the data record contents.

If RTREE is defined in ctoptn.h, the twelfth parameter is required. It is a symbolic name for the index file. It should start with an alphabetic character and may include upper and lower case letters, numbers and under scores. It may not include embedded blanks. Unlike the field names in the Data Description record, these do not, and must not, match entries in the r-tree DODA. Symbolic names for the data files are not necessary because each data file has a unique name.

Null Key Values

The ninth parameter of the Index File Description record, NULL key flag, determines if NULL or missing key values will be detected. After the key segments have been concatenated according to the Key Segment Description records, the NULL key flag parameter determines whether or not to test for a missing value. If the NULL key flag parameter is zero, no check for a NULL or missing value is made. If the NULL key flag is set to one, then the resulting key value is compared with the empty character (tenth parameter). If the key value is composed entirely of empty characters, then the key value is considered NULL or missing and is not entered into the index file.

The empty character is expressed in the decimal equivalent of the character. For example, an ASCII space (0x20) is specified with a value of 32. A NULL byte (0x00) is specified with a value of zero.

Optional Index Member Record

It is not necessary for each index associated with a data file to be in a separate file. Indexes can be combined in the same physical file. If the nomemb parameter of an Index File Description Record is greater than zero, the following nomemb Index File Description Records must be replaced by Index Member Records. For example, if a data file has three indexes, and the second index has an additional member, nomemb equals one, use two Index Description Records and one Index Member Record.

The Index Member Record is composed of seven required fields and one additional field to be used with r‑tree:

  • Key number (keyno)
  • Key length (keylen)
  • Key type (keytyp)
  • Duplicate flag (keydup)
  • NULL key flag
  • Empty character
  • Number of key segmens

If RTREE is defined in ctoptn.h, each Index Member record needs one additional parameter:

  • Symbolic index name

These fields have the same interpretation as the corresponding fields in the Index Description Record. Index Member records do not include explicit member numbers (membno). The sequence of Index Member records following an Index File Description record determines the member number of each Index Member, (the first Index Member record is member number one, and so on). While a key number (keyno) is specified for each member, there is no freedom in the keyno value. The keyno specified in each Index Description Record must equal the keyno value of the host index file plus the member number, or error IMKY_ERR (108) will be raised in CreateISAM() and OpenISAM().

Key Segment Description Record

Each key value is composed of one or more segments. A segment is simply a sequence of bytes from specified offset within the data record. Each Key Segment Description record is comprised of three fields:

  • Segment position
  • Segment length
  • Segment mode

Fixed-Length Parameter File Examples

The contents of ctexam.p will be similar to those shown below.

ISAM Parameter File for Fixed Record Length

  18 3 4 1
    3     cust.dat   128  4096  1  3
      0    custnam.idx  24 0 0  1  4096  1 1  32  3
          6  14 2
         30   6 2
        121   4 3
      1                  4 0 0             0   0  1
          2   4 1
      2    custzip.idx  22 0 0  0  4096  1 1  32  3
        112   9 2
          6   9 2
        121   4 3

This parameter file is interpreted as follows. There will be eighteen index file buffers, up to three index files, 512-byte B-Tree nodes, and exactly one data file.

The first index is assigned key number 0, and is in the file named custnam.idx which will be incremented in 4,096-byte chunks. The keys are unique, 24 bytes long and scanned left-to-right. The keys have three segments: 14 bytes, 6 bytes and 4 bytes long. The first two segments are converted to upper case, and are made up of the last name and first, respectively. The third segment is based on the automatic sequence number maintained by FairCom DB for each data file. The sequence number ensures that keys, which are otherwise the same, will be stored in order of entry in the data base.

Note: FairCom DB automatically places the sequence number in the record at byte offset 121 as specified by the third segment’s offset value.

 

The second index is assigned key number 1, and is the first (and only) additional member of custnam.idx. The keys are unique, 4-byte values. Keys are not checked for missing values (i.e., all data records will generate a key value). The key is comprised of one segment treated as an unsigned integer.

 

The last index is assigned key number 2, is named custzip.idx and has 22-byte unique keys. The key segments are made up of the nine character Zip Code, the first 9 bytes of the last name, and the same automatic sequence number used in the name index.

Note: If more than one index uses the automatic sequence number, all such indexes must use the same sequence number. Otherwise, the results will be unpredictable. The segment length of the sequence number must be 4.

 

Note: Each data file, index file, and index file member is assigned a unique number. This is the number by which the data file and indexes are referenced in the FairCom DB functions. The additional index members are numbered consecutively based on the host index number.

 

If RTREE is enabled in ctoptn.h, the preceding parameter file might look as follows:

ISAM Parameter File with RTREE Enabled

18 3 4 1
  3   cust.dat   128  4096  1  3 delflag cust_last
    0  custnam.idx  24 0 0  1  4096  1 1  32  3 name_key
        6  14 2
       30   6 2
      121   4 3
    1                4 0 0             0   0  1 numb_key
        2   4 1
    2  custzip.idx  22 0 0  0  4096  1 1  32  3 zipc_key
      112   9 2
        6   9 2
      121   4 3

Variable-Length Parameter File Example

The example parameter file for the variable-length example takes advantage of the key compression and the variable-length fields supported by FairCom DB. ctvxam.p is the associated parameter file. The contents of ctvxam.p will be similar to those shown below (assuming RTREE is not enabled).

ISAM Parameter File For Variable Record Length

18 3 4 1
  3        vcust.dat     15  4096  5  3
    0    vcust.idx  24  4 1  2  4096  1 0  32  2
      0  15 5
      4   5 2
    1                  4  0 0             0   0  1
      0   4 1
    2                 22 12 1             1  32  2
      4   9 2
      0   9 5

The important points to note in this example are:

  • The last name index (keyno = 0) has a key type of 4 (leading character compression will take place). Padding compression is not used since the second segment of the key is based on the Zip Code and will usually be the full length.
  • The first segment of the last name index has a mode value of 5. The segment is found in the zeroth varying length field, right after the fixed-length portion of each data record, not at absolute byte position zero. It will be padded, if necessary, to 15 bytes, and converted to upper case.
  • The Zip Code index (keyno = 2) has a key type of 12 (both leading character and padding compression will take place). Since the second segment of this index is based on the last name, many of the keys should have padding.

The main differences between ctvxam.p and ctexam.p are:

  • The data file uses variable-length records, (file mode of 5 specifies virtually opened, variable-length records, and file sharing in multi-user systems), with a minimum record length of 15 bytes;
  • All the indexes have been combined into one physical index file;
  • The key segment offsets are modified since all the fixed-length fields have been moved to the beginning of the record structure.

These examples use a non-zero file size extension parameter. A zero file size extension parameter means the file grows one record at a time instead of growing in large chunks. The advantage of growing in chunks is that every time the file is extended, FairCom DB forces the update of directory entries associated with the file.

Multiple Data File Parameter Setup

The ISAM parameter file can accommodate more than one data file. The following data show how a parameter file would look for two data files, each with two index files. Indentation is ignored by the programs, but is helpful in reading the material.

24 4 4 2
  0 VENDOR.DAT  384  8192 0  2
    2  VENDOR.IDX  14  0  1  1 8192 0 1  32  1
       0 10 0
    3               6  0  0           0   0  1
      12  6 0
  1 INVOICE.DAT 192  8192 0  2
    4  INVOICE.IDX 16  0  1  1 8192 0 0   0  2
      32  6 0
       0  6 0
    5              10  0  1           1  32  1
       0  6 0

The application using this parameter file refers to the vendor data file, VENDOR.DAT, as file number 0, and the invoice data file, INVOICE.DAT, as file number 1. Each data file has two indexes. In both cases, the indexes are stored in one physical file. Except for key number 3, which does not accept duplicates, all the segment lengths sum to 4 bytes less than the actual key lengths to accommodate the suffixes.

Parameter Files in Client Server Models

Client applications using parameter files to define file definitions must either specify a proper path, from the FairCom Server perspective, to the location of the parameter file or the parameter file must be located in the FairCom Server directory.

Parameter files not containing the optional r-tree parameters must be modified. The parameter file data description record contains positions for r-tree’s symbolic first and last field name and the index description record contains a position for the symbolic index name. Since the FairCom Server has RTREE defined, these values must be specified. If r-tree will not be used with the parameter file, dummy values can be used as placeholders. For example, the sample parameter file ctmark.p uses the letter ‘d’ as a provisional placeholder for the three r-tree symbolic values.

Refer to ISAM Parameter File System Functions in the FairCom DB Professional Developers Guide.

Prototyping

If your compiler supports function prototyping, we strongly suggest that you use this feature. A function declaration, or function prototype, establishes the name and return type of a function, as well as the types and number of arguments to the function. This information enables the compiler to check the types of the actual arguments when the function is used. If you accidentally use too few parameters when you invoke a prototype function, your compiler should give you a warning.

With some compilers it is very important that you use prototyping to avoid serious memory overwrites, for instance, with the Large memory model with the Microsoft C compiler, and functions that use TEXT pointers. If you do not include a function prototype in your code, the compiler will have to determine if it should use a 16-bit or 32-bit address for the function arguments. If it chooses the wrong one, the function may copy the information to the wrong location in memory. By including the function prototype in your code, you inform the compiler of the type of parameter that will be used, and the correct address will be passed.

The header file ctdecl.h contains declarations for the commonly used FairCom DB functions. There are two sets of declarations: one with prototypes and one without. By default, PROTOTYPE is defined in ctoptn.h. This will include the prototype definitions. If your compiler does not support prototyping, remove the PROTOTYPE define from ctoptn.h.

2xx Internal Error Codes

The internal error codes in the 2xx range should not occur during proper operation of FairCom DB. Their main function is to help find programming bugs, especially if you modify FairCom DB code. Experience shows that almost all occurrences of an internal error are caused by memory overwrites from the application code.

200 Neither LOW_HIGH nor HIGH_LOW defined in ctoptn.h.
201 Both LOW_HIGH and HIGH_LOW defined in ctoptn.h.
202 NOTFORCE, FPUTONLY, nor FPUTFGET defined in ctoptn.h.
203 More than one of NOTFORCE, FPUTONLY, and FPUTFGET defined in ctoptn.h.
206 Update flag inconsistency between index file header and buffer status information.
207 Corrupt node found in nodser() function.
208 Updated (but not written) node found in FPUTFGET disk I/O mode.
210 Undefined key type found in compar.
211 Negative number of key values indicated in node.
212 No leaf node found during FirstKey() operation.
213 No leaf node found during LastKey() operation.
214 Corrupt tree found in fndkey().
215 No leaf node found in fndkey().
216 Attempt to save node with negative number of key values.
217 Attempt to transfer key values between buffers for different index files.
218 Corrupt tree found in .
219 No leaf node found in .
220 Corrupt tree found in .
221 No leaf node found in .
222 Undefined file access flag found: filacs member of file control structure must be set ‘y’ (active), ‘v’ (active, temporarily closed), or ‘n’ (inactive).
225 VARLDATA not defined in ctoptn.h. Add VARLDATA option and recompile ctaddk.c.
226 Undefined key segment translation mode.
227 While extending the size of a fixed-length data file, delete flags could not be written into new (but unused) records at the end of the file.
228 Expected deleted record mark not found while collapsing consecutive deleted areas.
230 B-Tree node has conflicting member number value during write operation.
231 B-tree node has conflicting member number value during read operation.
232 Attempt to expand non-existent compressed key value.
233 Illegal value for key compression bytes.
235 Illegal comparison value during node insert.
236 Illegal key value shift detected with compressed keys.
237 Attempt to get an index node at byte offset zero.
238 Illegal key value shift detected with compressed keys.
239 Variable key expansion request with NO_VARLK.
240 Exceeded maximum b-tree levels. Increase number or size of node sectors, or increase MAXLEV. In standalone models, may be a symptom of adding index entries sequentially, creating a badly balanced tree. Compact the index or shift to a client/server model.

749X Internal Error Codes

Most 749X errors are the result of memory corruption on the host machine. Although these errors are extremely rare, they can be serious. In most cases, restarting FairCom DB will clear a transient memory error. If these errors repeat, please contact your application vendor.

Value Cause
7490 A request for memory beyond reasonable limits.
7491 Attempting to return memory carrying an unreasonable large length value in its control header.
7492 Encountered an invalid "availability" counter during a sub-allocator get operation.
7493 An invalid "availability" counter while returning a block of memory.
7494 An invalid header pointer for an allocation block.
7495

A list connected with FairCom DB’s memory sub-allocator has become invalid or corrupted.

For single-user, check your parameters passed to our functions. For a server, this is usually a bug.

This error has been seen in V11.2.2. due to bad Unicode data. V11.2.3 has been patched to catch bad data and log it instead of passing it into the code, causing a crash.

7496 Failure in memory swapping operation.
7497 Attempting to return memory carrying a zero length in its control header.
7498 An invalid semaphore owner while getting memory.
7499 An invalid semaphore owner while returning memory.

Google Chrome Tips for Developers

Chrome appears to have fixed the problems documented below, but if you are having trouble connecting to an application server that you are running locally on your computer, the solutions below may be helpful.

Tip: If the domain name localhost is not working, use the IP Address 127.0.0.1

Tip: If you have problems with getting Windows to connect to the proper domain (including localhost), this document from Microsoft explains how Windows works with DNS: Disable DNS client-side caching on DNS clients.

Configuring Chrome for Localhost Connections

The following information is not useful for users who only browse the web. It is for those running and testing local application servers, such as the FairCom Database.

It is common for developers, QA engineers, and testers to test new applications on their laptop by running a local web application server without a valid certificate. It is too time consuming and expensive to generate local certificates.

The new security settings in the Google Chrome Web Browser prevent users from using HTTP (where a certificate is not required), and it prevents the HTTPS protocol from connecting to application servers that do not have a valid certificate. Because Google combined these new behaviors, users cannot open locally-run applications in Google Chrome.

The sections below show how to configure Google Chrome for running local applications.

Use 127.0.0.1 Instead of Localhost

Tip: If the localhost domain name is not working, use the IP Address 127.0.0.1

To test FairCom Database Web Tools, use http://127.0.0.1:8080/ or https://127.0.0.1:8443/ instead of http://localhost:8080/ or https://localhost:8443/.

It is a best practice to use 127.0.0.1 instead of localhost to ensure you connect to your local computer. This is because localhost does not work reliably in some languages, such as Python. Also your computer’s hosts file and/or DNS settings may be configured to redirect the localhost domain name to another computer besides your local computer.

If you use 127.0.0.1, you don’t have to worry about the behavior of localhost and the hosts file.

If you want to verify how your hosts file modifies the behavior of localhost, you can open it in any editor to see what IP address it maps to localhost.

In Windows, the hosts file is located in C:\Windows\System32\drivers\etc. By default, it uses DNS to map localhost. Your DNS server typically maps localhost to 127.0.0.1, but it can map it anywhere and this can cause problems.

In Linux, the hosts file is located in the /etc folder. By default, Linux maps localhost to 127.0.0.1

HTTP

How to enable HTTP on Google Chrome Web Browser

This section shows you how to connect to connect to FairCom’s database browser-based tools over HTTP using: http://127.0.0.1:8080/ when the database is running on your local computer.

As of 2020, Google Chrome Web Browser automatically redirects all HTTP URLs to HTTPS. This is a great security feature called HSTS, but it causes trouble for developers, QA engineers, and testers who are testing the user interface of application servers running on their local computers.

You can stop Google Chrome from automatically redirecting HTTP URLs to HTTPS for localhost by doing the following:

Put the following URL in the Chrome Browser:

chrome://net-internals/#hsts

In the textbox under the section, [Delete domain security policies],

type localhost and push the [Delete] button

This only changes the setting for the localhost domain. It does not compromise security for other domains.

HTTPS

How to enable HTTPS on Google Chrome Web Browser when the local app server does not have a valid certificate

This section shows you how to connect to connect to FairCom’s database browser-based tools over HTTPS using: https://127.0.0.1:8443/ when the database is running on your local computer.

As of 2020, Google Chrome Web Browser prevents users from going to a website that has an invalid certificate. This is a great security feature, but it causes trouble for developers, QA engineers, and testers who are testing web applications served from their local computers.

You can configure Google Chrome to revert to its original behavior of displaying a warning page when you go to a website that has an invalid certificate. On the warning page, you can choose the Advanced Options and go to the site, or you can choose not to go to the site.

To configure Chrome to use this approach, you can do the following.

Put the following URL in the Chrome Browser: chrome://flags/#allow-insecure-localhost

In the section, [Allow invalid certificates for resources loaded from localhost.]

Select [Enabled] from the dropdown

When you go to a site that has an invalid certificate, Google Chrome will display “Not secure” next to the URL, as shown below. If you hover over the “Not Secure” icon, it notifies you that the certificate is invalid. This is expected because the FairCom Database cannot ship with a certificate that works with your local computer.

It is safe to connect locally to a FairCom database without a valid certificate. In a production environment, our default certificate must be replaced with a valid certificate if applications and users are allowed to connect from other computers.