FIX log ingestion

The following options for FIX log data ingestion are available.

Geneos FIX Analyser to Obcerv publishing Copied

You can seamlessly send all FIX messages captured by the FIX Analyser 2 plugin in Geneos directly to Obcerv. When you enable the publishToObcerv setting in the FIX Analyser 2 plugin, all FIX messages will be published to Obcerv for consumption by the FIX Monitor app. See publishToObcerv.

Kubernetes log scraping Copied

If your FIX engine application is running in Kubernetes, you can configure Obcerv to stream the log data from your FIX engine application pod(s) into Obcerv.

OpenTelemetry (OTel) Copied

Follow the OpenTelemetry instructions to configure the OTel client on your FIX application. You do not have to modify the source code for you FIX application.

Set the following configuration parameters:

Fluentd Collector Copied

The Fluentd Forward Protocol can be used to capture log data from your FIX engine(s) and send to a ITRS Fluentd Collection Agent plugin.

The Collection Agent then streams the logs data to the ingestion service in Obcerv.

Follow the two configuration examples below to guide you with the setup process.

Fluentd config Copied

The Fluentd configuration example streams log data from a single log file fix.log using the <source> directive, adds a set of dimensions and attributes using the <filter> directive, and then sends the log data to an ITRS collection agent using the <match> directive. This is a basic setup, and more complex configurations can be used. See the Fluentd Config Reference for more details.

# Source directive to set FIX log file location, position file (pos_file) 
# and tag log events for future matching
  @type tail
    @type none
  path ../fix-equities/fix.log
  pos_file ./fix-sell-logs.pos
  tag fix-equities-log

# Filter directive for adding dimensions and attributes
<filter fix-equities-log>
  @type record_transformer
    service.namespace fix-data equities "#{hostname}" fix-equities-log
    level info

# Match directive to stream logs to a Fluentd collection agent plugin
<match fix-equities>
  @type forward
  send_timeout 1s  
    name collection-agent
    host localhost
    port 24224

ITRS Fluentd Collection Agent config Copied

plugin-directory: ./plugins

  enabled: true

# Plugin configuration
  - type: plugin
    name: fluentd
    class-name: FluentdForwardCollector
    # listen port
    port: 24224
    # message event key combination
    message-key: message
    # dimesion mapping fluentd -> obcerv
      service.namespace: service.namespace
  store-directory: ./run
    reporter: obcerv 
    reporter: obcerv 
    reporter: obcerv 
    reporter: obcerv
    reporter: obcerv

# Ingestion service URL, username and password
  # Send logs to stdout
  - type: logging
    name: stdout
  # Send logs to Obcerv ingestion service  
  - type: plugin
    name: obcerv
    class-name: GrpcInternalIngestionServiceReporter
    port: 443
    username: ingestion-api
    password: ingestion
    use-compression: true
    call-deadline: 3000
["Obcerv"] ["Obcerv > FIX Monitor"] ["User Guide"]

Was this topic helpful?