configureTopic

JSON MQ "configureTopic" action configures an MQTT topic in FairCom MQ and FairCom Edge

The "configureTopic" action configures an existing MQTT topic or creates a new one. Configuring a topic is similar to altering a topic except the FairCom server automatically creates a topic when you specify a non-existent topic name.

Note If a topic already exists, you must use "alterIntegrationTable" to change the retention settings: "retentionPolicy", "retentionPeriod", and "retentionUnit".

  • You may send a "configureTopic" message repeatedly to update as few or many properties as desired. When you omit a property, its current value remains unchanged. Each time you send this message is sent, it modifies the topic’s settings.
  • If the topic has already been published to FairCom's servers, "configureTopic" configures it.
  • If the topic has not yet been published to FairCom's servers, "configureTopic" creates and configures it.
  • Creating a topic includes creating an integration table to store the topic’s messages. By default, tables are created in the ctreeSQL database, however, you can reconfigure this to any database you like by changing the configuration or the API session defaults.
  • Each time this message is sent, it modifies the topic’s settings.
  • The MQTT Publication service and the MQTT Subscription service are also preconfigured.

 

Request examples

Minimal

{
  "apiVersion": "1.0",
  "action": "configureTopic",
  "params": {
    "topic": "test/topic1"
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

Maximal

Note This example (which shows all of the available parameters) requires additional support in order to work. MQTT messages that clients publish to this topic ("acmefactory/line1/station1/acidbath/telemetry") will be automatically stored in the table's "source_payload" field when they arrive. 

The broker will then publish an MQTT message to all subscribers of that topic. The contents of that message will come from the table field specified by the "outputPayloadField" parameter. If that parameter had specified the "source_payload" field, subscribing clients would receive the same message that arrived and was stored in the table. However, in this example, the "outputPayloadField" parameter is specifying the "aValidFieldName" field of the table. That means this topic requires a transform that reads the MQTT message payload from the "source_payload" field, performs some type of transformation on that data, and then stores the results in the "aValidFieldName" field (which will then be copied into the payload of the MQTT messages which the broker sends to all of this topic's subscribers). 

{
  "api": "mq",
  "apiVersion": "1.0",
  "requestId": "2",
  "action": "configureTopic",
  "params": {
    "topic": "acmefactory/line1/station1/acidbath/telemetry",
    "ownerName": "admin",
    "databaseName": "faircom",
    "tableName": "acmefactory_line1_station1_acidbath_telemetry",
    "rebuildTable": "false",
    "threads": 1,
    "mqttPayloadType": "binary",
    "retentionPolicy": "autoPurge",
    "retentionPeriod": 30,
    "retentionUnit": "day",
    "maxInflightMessages": 20,
    "maxDeliveryRatePerSecond": 0,
    "retrySeconds": 5,
    "downgradeQoS": false,
    "newSubscriberDeliveryMode": "newMessages",
    "newSubscriberMaxDeliveredMessages": 3,
    "outputPayloadField": "aValidFieldName",
    "operationIntent": "upsert",
    "metadata": {},
    "forwardToTopics": [],
    "subscribeToExternalBroker": "MqBroker1",
    "forwardToExternalBrokers": [],
    "validateMqttPayload": "true",
    "label": {
      "group": "group name",
      "name": "label name"
    }
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

 

Response examples

Minimal

{
  "result": {},
  "errorCode": 0,
  "errorMessage": "",
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

Maximal

{
  "result": {},
  "requestId": "2",
  "errorCode": 0,
  "errorMessage": "",
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
 
 

 

Properties

Request properties ("params")

Property Description Default Type Limits (inclusive)

databaseName

The "databaseName" property specifies the database that contains an object, such as a table or code package. If it is set to null or is omitted, it defaults to the default database of the JSON Action session, see "createSession" and the "defaultDatabaseName" property. 

You specify this property when you want to use a different database instead of the default. 

This property is useful because objects, such as tables and code packages, can have the same name in multiple databases. This feature allows you to create multiple environments in the same server and reuse the same JSON actions in each environment. For example, you can create "dev", "test", "stage", and "prod" databases on the same server and use the "defaultDatabaseName" or "databaseName" properties to specify the desired environment.

It is an error to set "databaseName" to the empty string "".

If no default database is specified during "createSession", the server sets the "defaultDatabaseName" to the "defaultDatabaseName" value specified in the services.json file.

 Defaults to the session's "defaultDatabaseName" property string 1 to 64 bytes

downgradeQoS

The "downgradeQoS" property controls how FairCom's servers manage the Quality of Service (QoS) of subscriptions to this topic. 

  • When false, FairCom's servers honor the QoS requested by each subscriber regardless of the QoS of publishers.
  • When true, FairCom's servers downgrade the QoS of subscribers to match the QoS of the publisher.
 Optional with default of false Boolean

true

false

forwardToExternalBrokers

 The "forwardToExternalBrokers" property specifies one or more brokers to forward the message to. All messages sent to this topic will be forwarded to each of these external brokers using the specified topic name. For more details, see “forwardToExternalBrokers”.

Optional with default of []

 array of objects

Zero or more objects including some or all of the following properties:

"brokerConnectionName"

"forwardQos"

"topic"

forwardToTopics

The "forwardToTopics" property causes FairCom's servers to take all messages sent to this topic and automatically forward them to each of the topics specified in the array. These messages are forwarded within the same instance. 

  • If a topic does not already exist, FairCom's servers create it as if an external client sent a message to the topic.
  • Each time the "ConfigureTopic" message is sent, FairCom's servers will completely replace its existing list for "forwardToTopics" with the new list.
  • FairCom's servers will not forward a topic to itself. Thus, if the "forwardToTopics" array contains the topic from "ConfigureTopic", FairCom's servers ignore it.
    • Be careful not to create infinite message loops by forwarding topics to each other. For example, topic A forwards to topic B and topic B forwards to topic A. FairCom's servers currently do not detect or shut down infinite message loops.
  • This feature is designed to ease the pain of topic migration, which occurs when clients begin publishing to different topics before subscribers can adjust.

Optional with default of []

 array of strings One or more strings

label

The "label" property assigns a label to an object. The "label" object contains "group" and "name" properties that uniquely identify a label. If you omit the "group" and "name" properties, they default to the empty string, "", which is a valid group and name for a label.

A label is a tag that you can add to some objects, such as an MQTT topic or session. You can use list actions such as "listTopics" and "listMqSessions" to find objects with a specific label.

The "label" property is mutually exclusive from the "labelId" property.

 Optional with default of null object 
"label": {
  "group": "group name",
  "name": "label name"
}

label

.group

The "group" property groups labels into a lookup list or tag set. It is a namespace for a set of labels that identifies their purpose. You can use the "listLabelGroups" action to return the list groups. You can use the "listLabels" action to return labels in one or more groups. 

The "group" and "name" properties work together to uniquely identify each label. They are its natural key. A group assigned to the empty string "" is the default group.

The group name may contain up to 64 bytes of UTF-8 encoded characters.

When you assign a group name to a label, the server automatically checks if the group name exists in the list of groups that the "listLabelGroups" action returns. If the group name does not exist, the server adds the group name to the list. When you rename a group assigned to a label, the server automatically adds a new group name to the list and removes the previous group name if no other label uses it.

Tip If your application creates many groups, you can use a delimiter character, such as the forward slash / in your group names to create a group hierarchy. Include the delimiter after each part of the hierarchy and at the end of the group name. In the "listLabels" action, you can use the "partialGroupFilter" filter to return subsets of groups in the hierarchy. For example, you if have groups named "product/size/", "product/weight/", "product/ranking/", "person/gender/", and "person/ranking/", you can set the "partialGroupFilter" to "product/" to return the product groups.

 Optional with default of "" string 1 to 64 bytes

label

.name

The "name" property is the name of a label or field. 

The "group" and "name" properties combined uniquely identify each label. The "createLabel" and "alterLabel" actions return an error if the "group" and "name" properties do not create a unique label name.

The "id" property also uniquely identifies each label, so you can rename a label's group and name without breaking "id" references to the label.

 

 Required - No default value string 1 to 64 bytes

maxDeliveryRatePerSecond

The "maxDeliveryRatePerSecond" property sets the throttle rate for current and new subscribers of the topic. If omitted, it defaults to 0, which means no throttling is applied.

  • It must be a value between 0 and 2,147,483,647, but a more realistic range is between 0 and 1,000. It is useful to keep a client from being overwhelmed with too many messages sent too quickly.
  • You can set it to a number greater than 0 to throttle the delivery rate. This allows you to avoid saturating the network and other resources.
    • For example, you can set it to 100 to throttle the delivery to 100 messages per second. This is a maximum setting because the actual message delivery rate is bounded by the network, local resources, the hardware running FairCom's servers, and a client’s ability to process messages quickly.
 Optional with default of 0

int32

0 to 2147483647

maxInflightMessages

The "maxInflightMessages" property protects subscribers from receiving messages faster than they can handle.

  • The default value of 0 means there is no maximum limit.
  • This property defines the maximum number of unacknowledged messages that FairCom's servers will send to each subscriber of this topic.
    • For example, setting this property to 10 allows FairCom's servers to send up to 10 publish packets to a subscriber before the servers wait for the subscriber to acknowledge receipt of at least one publish packet.
 Optional with default of 0

int32

0 to 2147483647

metadata

 The "metadata" property contains user-defined properties that add keywords and tags about the code package. The server indexes this field with a full-text index so you can search for any word or phrase to find code packages. Optional with default of {} object 0 or more key/value pairs

mqttPayloadType

The "mqttPayloadType" property specifies the variant type format of the "source_payload" field. For more details, see mqttPayloadType. 

 Optional with default of "binary" string enum

"json"

"xml"

"binary"

"jpeg"

"siemensUdt"

"utf8"

"variantObject"

newSubscriberDeliveryMode

The "newSubscriberDeliveryMode" property defines how new subscribers receive messages. 

  • It may be set to one of two values:
    • "newMessages"
    • "storedMessages"
  • When set to "newMessages" new subscribers only receive new messages.
  • When set to "storedMessages" new subscribers will receive some or all previously sent messages. The "newSubscriberDeliveryMode" property defines the maximum number of messages.

Optional with default of "newMessages"

 string

"newMessages"

"storedMessages"

newSubscriberMaxDeliveredMessages

The "newSubscriberMaxDeliveredMessages" property is the maximum number of previously stored historical messages a client receives automatically when subscribing to a topic.

  • A value of -1 or "all" defaults to delivering all currently stored messages.
  • A non-negative number limits the delivery of currently stored messages to that number, which may be a maximum of 2,147,483,647 messages.
  • This property only applies when the "newSubscriberDeliveryMode" property is set to "storedMessages".
  • The broker can only send messages stored in the topic’s integration table. The table's retention policy automatically removes old messages.
  • Each time a client unsubscribes from a topic and then subscribes to the topic, the broker sends historical messages as specified by this property. A client can use this behavior to reprocess messages, such as refresh a local data cache or recalculate aggregates.

Note A topic’s "retentionPolicy", "retentionUnit", and "retentionPeriod" properties define how long the broker retains messages in a topic’s integration table.

 Optional with default of -1 integer or "all" -1 to 2147483647 or "all"

operationIntent

The "operationIntent" property clarifies the intent of the method as an "insert", "update", or "upsert". 

 

The "operationIntent" property values and their descriptions are as follows:
  • "upsert"
    • This is the default value. It treats this method as an "upsert". If the connection already exists, it updates it. If the connection does not exist, it inserts it. It does not throw an error.
  • "insert"
    • It treats this method as an insert. If the connection already exists, FairCom's servers log an error stating, "Cannot insert service connection xxx because it already exists."
  • "update"
    • It treats this method as an update. If the connection does not already exist, FairCom's servers log an error stating, "Cannot update service connection xxx because it does not exist."
Optional with default of "upsert" string

"upsert"

"insert"

"update"

outputPayloadField

The "outputPayloadField" property specifies the field that the MQTT Subscription service should use as the payload when it sends the payload to subscribers.

  • "outputPayloadField" must be one of the following values: 
    • "source_payload"
    • A user-defined field

This makes it possible for the output from any transform to be used as the payload delivered to subscribers.

Optional with default of "source_payload"

 string

"source_payload"

A user-defined field

ownerName

The "ownerName" property specifies the account that owns an object, such as a table or code package. See "createSession" and the "defaultOwnerName" property for more details. 

You specify this property when you want to use a different account instead of the default. Your session's account must have the appropriate privileges to access the code package. 

This property is useful because objects, such as tables and code packages, can have the same name in the same database as long as different accounts own each object. This feature allows you to create duplicate objects for different users on the same server and reuse the same JSON actions on those objects. For example, an administrator can copy objects from a production environment to her account so she can troubleshoot an issue using the same JSON actions, JavaScript, and SQL code.

It is an error to set "ownerName" to the empty string "".

If no default owner is specified during "createSession", the server sets the "defaultOwnerName" to the "defaultOwnerName" value specified in the services.json file.

 Optional with default of the session's "defaultOwnerName" property string 1 to 64 bytes

rebuildTable

The "rebuildTable" property causes the server to stop accepting read, insert, and update requests and rebuilds the table when true. Rebuilding a table creates new table files and writes existing records to the new files.

Warning Rebuilding a table interrupts data collection until the rebuilt operation finishes. The rebuild time is proportional to the number of table records.

Tip You may want to rebuild the table in the following cases: 

  • To apply a transform to previously inserted or updated records. 
  • To change the table's retention policy, period, or unit. 
  • To repair a data file corrupted by a storage failure.
 Optional with default of false Boolean

true

false

retentionPeriod

The "retentionPeriod" property specifies how many units of data to retain. It refers to the unit of time specified by the "retentionUnit" property. For more details, see "retentionPeriod".

Optional with default of 30

 integer 1 to 100

retentionPolicy

The "retentionPolicy" property controls how messages are persisted. 

If not specified, the default found in the services.json file is used. Initially, it is "autoPurge".

 

retentionPolicy values:
  • "autoPurge"
    • This is the default. It is automatically applied when a new topic is created. It is preferred because it allows FairCom's servers to automatically remove messages that are older than the retention time. This helps ensure message data does not consume all storage space. It also minimizes storage costs and speeds up data access. The server partitions a table into multiple files so it can efficiently delete expired files.
  • "neverPurge"
    • This stores messages on disk and never removes them. This is useful when you need the entire history of the message stream. If message velocity is high, this can consume all storage space and cause an outage. The server creates a non-partitioned table, which is slightly faster than a partitioned table because it stores all records in one file.

Optional with default of "autoPurge"

 string

"autoPurge"

"neverPurge"

retentionUnit

The "retentionUnit" property specifies a unit of time that the server will use to purge expired messages. For example, if you want a week's worth of messages to be purged once a week, set "retentionUnit" to "week". This property is optional.

If not specified, the default found in the services.json file is used. Initially, it is "week". 

  • This property is used in concert with "retentionPeriod" to determine retention time.
  • "retentionUnit" values:
    • "minute"
    • "hour"
    • "day"
    • "week"
    • "month"
    • "year"
    • "forever"

Note 

  • For best performance, set the "retentionUnit" to a value that keeps "retentionPeriod" between 5 and 30. 
  • When you set "retentionUnit" property to "forever" the server will not purge messages. This setting is the same as setting "retentionPolicy" to "neverPurge".
  • The "retentionUnit" and "retentionPeriod" properties are used only when the "retentionPolicy" is set to "autoPurge".

Optional with default of "day"

string

"minute"

"hour"

"day"

"week"

"month"

"year"

retrySeconds

The "retrySeconds" property specifies the number of seconds that FairCom's MQTT broker will wait for the next expected packet from a client during a QoS 1 or QoS 2 PUBLISH packet exchange/PUBLISH handshake with an MQTT client. It applies when a client is publishing to the broker and when the broker is publishing to a client. This property does not apply to QoS 0 PUBLISH operations because QoS 0 PUBLISH operations are not answered. "retrySeconds" is optional.

  • If the "retrySeconds" time expires, FairCom's broker will disconnect the MQTT client with MQTT error code 7. MQTT 5 clients will also receive a Reason Code. An error message will be logged to <faircom>/data/CTSTATUS.FCS explaining why the disconnect happened. For example: 
    • MQTT connection for client ID [myMqttClient7] has been disconnected due to inactivity timeout. retrySeconds is set to [25] seconds for topic [my/test/topic12].: 14226
    • When the MQTT client reconnects, FairCom's MQTT broker will attempt to resume the PUBLISH handshake by resending the unanswered packet to the client, hoping that the client will reply with the next step of the handshake.
  • The minimum value is 1 and the maximum value is 65535.
  • If "retrySeconds" is omitted or its range is exceeded, FairCom's broker uses the default value.
  • Note that this parameter is not related to MQTT Keep Alive or when PINGREQ packets should be sent by the client.

Optional with default of 20

 integer

1 to 65535

subscribeToExternalBroker

The "subscribeToExternalBroker" property causes FairCom's servers to connect to the external MQTT broker and subscribe to this topic on that broker when it is set to a valid broker connection. Thus, these servers become a client to another MQTT broker. Each message sent to this topic on the external broker becomes an incoming message in FairCom 's servers – as if it were sent directly to them. The topic works the same as any other topic.

  • Any client can publish messages to this topic.
  • A valid Broker Connection must be created before you set "subscribeToExternalBroker" to a value (see "configureBrokerConnection").
  • If this broker connection name does not already exist, it does not set up an external subscription and it logs an error stating, “Cannot subscribe to the external broker because the Broker Connection name xxx has not yet been defined by ConfigureBrokerConnection.”
Optional with default of "" string

Any previously defined name of a broker connection (see "configureBrokerConnection")

tableName

The "tableName" property contains the unique, user-defined name of a table.

See table name in System specifications for the table naming requirements and limitations.

 

"params": {
  "tableName": "ctreeTable"
}
 Required - No default value string 1 to 64 bytes

threads

The "threads" property specifies the number of threads assigned to a topic to write published messages to disk. 

  • The default of 1 is enough to handle typical workloads.
  • For heavy workloads, you may increase the number of threads to 2 or 3.
 Optional with default of 1 integer

1

2

3

topic

The "topic" property specifies the unique, user-defined name for the topic.

  • The "topic" name is typically less than 150 characters.
  • The "topic" name will be used in all subsequent references to the topic, such as when re-configuring or deleting the topic.
 Optional with default of "" string 1 to 65,500 bytes

validateMqttPayload

The "validateMqttPayload" property specifies whether the broker will validate the payload of each MQTT message.

  • When true, the MQTT broker validates the payload of each MQTT message to verify that it matches the type specified by the "mqttPayloadType" property.
    • If a payload fails validation, the message is delivered without change to all subscribers. The broker also sets the error field to true and adds an error object to the log field (see the error object example). Users, applications, and JavaScript transform steps can read the values of the error and log fields to know when incoming content is invalid.
  • When false, the MQTT broker does not validate the payload. This increases message throughput because validation increases the time it takes to process each message. Without validation, the broker cannot update the error and log fields to indicate when content is invalid. Lastly, invalid content can cause problems for subscribers and JavaScript transforms that expect valid content.

 

Example error object describing an invalid MQTT payload

[
  {
    "errorCode": 12046,
    "errorMessage": "Payload is not a valid JSON.",
    "errorSource": "MQTT payload format"
  }
]
 Optional with default of the "defaultValidateMqttPayload" value in the services.json file, which is true by default. Boolean

true

false

 

 

 

Was this article helpful?

Give feedback about this article

Related Articles