--- title: "MDM Sampler and Feed Adapter API" description: "" url: "https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/mdm-sampler-api/index.html" --- ## Sampler Lua API The pages in this section document the Lua API calls available for use in a Lua sampler script. For more detailed discussions on using each module, see the [MDM Writing Samplers and Feed Adapters](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/mdm-writing-samplers/index.html) guide in this document. The Lua API is split into separate modules which are described in the pages listed below. Each module must be loaded using the Lua `require` function. - [geneos.helpers](#geneoshelpers) - [geneos.latency.files](#geneoslatencyfiles) - [geneos.latency](#geneoslatency) - [geneos.marketdata](#geneosmarketdata) - [geneos.sampler.commands](#geneossamplercommands) - [geneos.sampler](#geneossampler) ### geneos.helpers The helpers module defines several general-use functions.
Functions Description
gettimeofday() Gets the current time.
formatTime() Formats a time-of-day with a fractional seconds component.
parseTimeInterval() Parses a time interval from a string in ISO8601 syntax.
zip() Combine two sequences (arrays) into a single key-value map.
createtable() Creates a pre-sized table.
examine() Outputs an object to io.output (which is usually stdout).
#### Functions `geneos.helpers.``gettimeofday`() Gets the current time. This function is similar to [os.time](http://www.lua.org/manual/5.1/manual.html#pdf-os.time), but returns the time to microsecond accuracy using the C function 'gettimeofday'. The output is a single number representing the time as seconds since the UNIX Epoch, with its fractional component indicating the microseconds. This number may be split using the functions in the math module. *Returns:* The current time at the point the call was made. `geneos.helpers.``formatTime`([*timefmt*, ]*time*) Formats a time-of-day with a fractional seconds component. This function splits the time using the [math.modf](http://www.lua.org/manual/5.1/manual.html#pdf-math.modf) function, and formats this as a local time value (not UTC) using [os.date](http://www.lua.org/manual/5.1/manual.html#pdf-os.date). **Note** This function does not perform any type checking on its arguments for performance reasons. *Parameters:* - `timefmt`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `time`: (`number`) *Returns:* ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) Formatted time in "HH:MM:SS.sss" format (or as specified by timefmt) The fractional part will be rounded to the specified number of decimal places. `geneos.helpers.``parseTimeInterval`(*interval*) Parses a time interval from a string in ISO8601 syntax. This function interprets time intervals in the format `[PT][nH][nM][nS]`, for example `PT27H15M37,5S` (27 hours 15 minutes 37.5 seconds), `1.5H`, `90M` or `5400S` (three different ways of saying 90 minutes). The input string must contain at least one of 'H', 'M' or 'S'. Intervals expressed as a number of years, months or days are not supported. *Parameters:* - `interval`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - (optional `number`) The number of seconds in the interval, nil on failure - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure `geneos.helpers.``zip`(*keys*, *values*) Combine two sequences (arrays) into a single key-value map. *Parameters:* - `keys`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `values`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) A map whose keys are the values from the first sequence, and whose values are the corresponding values from the second sequence. *Usage:* ``` local gh = require "geneos.helpers" local keys = { "A", "B", "C" } local vals = { 10, 20, 30 } local map = gh.zip(keys, vals) -- map == { A=10, B=20, C=30 } ``` `geneos.helpers.``createtable`(*narr*, *nrec*) Creates a pre-sized table. This is a direct wrapper for the lua\_createtable function in the Lua C API. *Parameters:* - `narr`: (`number`) - `nrec`: (`number`) *Returns:* A new empty table pre-allocated with space for `narr` array elements and `nrec` non-array elements. `geneos.helpers.``examine`(*obj*) Outputs an object to [io.output](http://www.lua.org/manual/5.1/manual.html#pdf-io.output) (which is usually stdout). The purpose of this function is to display the content of an object for debugging. This function will only cope with simple types; functions, threads or userdata types are not described in any detail. Cycles of tables are handled, by omitting the contents of the table when repetitions are encountered. For a more complete debugging helper which handles more conditions, please see *Parameters:* `obj`: ### geneos.latency.files The files module provides classes implementing tick loggers. A tick logger manages a log file and provides a history writer function (see `geneos.latency.historyWriter()`) suitable for passing to the `geneos.latency.Context:addWriter()` method. Two ready-made tick logging classes, [`TickHistory`](#Class4 "geneos.latency.files.TickHistory") and [`MatchedTicks`](#Class5 "geneos.latency.files.MatchedTicks") inherit from a common base class, [`TickLogger`](#Class6 "geneos.latency.files.TickLogger"). TickLogger provides methods to open a file with a name that includes date and time fields. Additional custom logger classes can be created based on TickLogger; this requires some understanding of object-oriented programming in Lua.
Functions Description
newTickHistoryFile() Constructs a new tick history file logger.
newMatchedTicksFile() Constructs a new matched tick file writer.
newTickLogger() Used only when creating a custom logger.
Class TickLogger Description
TickLogger.filename Template to use for the name of the logging file.
TickLogger.timefmt Template for tick timestamps.
TickLogger.file File handle.
TickLogger:new() Constructor method inherited by derived classes.
TickLogger:init() Placeholder method, called when a tick logger is constructed.
TickLogger:open() Opens a file, expanding any time format codes in the name.
TickLogger:getActualFilename() Returns the name of the current log file.
TickLogger:setFileRoll() Enables timed rollover on the logging file.
TickLogger:checkFileRoll() Tests whether file should be rolled over.
TickLogger:sampleEnd() Flushes the logging file.
TickLogger:close() Closes the logging file.
Class TickHistory Description
TickHistory:getWriter() Provides a tick history writer function.
Class MatchedTicks Description
MatchedTicks:getWriter() Provides a tick history writer function.
#### Functions `geneos.latency.files.``newTickHistoryFile`([*filename*[, *timefmt*[, *fieldNames*]]]) Constructs a new tick history file logger. A [`TickHistory`](#Class4 "geneos.latency.files.TickHistory") logger logs full details of each tick received, in a format that can be read by the TestFile adapter. This type of logger cn be used independently of the latency algorithm. However, if the set of fields subscribed to will change after a feed is started, the `fieldNames` parameter should be used and should be set to cover all the field names that will need to be logged. This is because, for performance reasons, the sequence of field names used for each feed is fixed after the first tick. *Parameters:* - `filename`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `timefmt`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `fieldNames`: (optional [table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`TickHistory`](#Class4 "geneos.latency.files.TickHistory")): The newly constructed tick history logger. `geneos.latency.files.``newMatchedTicksFile`(*configNames*[, *filename*[, *microsec*]]) Constructs a new matched tick file writer. A [`MatchedTicks`](#Class6 "geneos.latency.files.MatchedTicks") logger writes a file in CSV (comma separated values) format, showing the matches computed by the tick matching algorithm. To correctly write the CSV file, this class needs information about the names of the feeds and of the fields used for matching. Since this information is supplied when configuring a latency context, the Context class provides a convenience method that can be used when calling this method (see the usage example below.) *Parameters:* - `configNames`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `filename`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `microsec`: (optional `boolean`) *Returns:* - ([`MatchedTicks`](#Class6 "geneos.latency.files.MatchedTicks")) The newly constructed matched ticks logger. *Usage:* ``` local lt = require "geneos.latency" local lf = require "geneos.latency.files" local ctx = lt.newContext{} --[[ calls to ctx:setBase() and ctx:addfeed() go here ]] local configNames = ctx:getConfiguredNames() local matchedTicksFile = lf.newMatchedTicksFile(configNames, 'matches.csv') ctx:addWriter(matchedTicksFile:getWriter()):start( ``` `geneos.latency.files.``newTickLogger`(*properties*) Used only when creating a custom logger. This method creates a prototype for a custom logger. To construct a ready-to-use tick logger, use [`newTickHistoryFile()`](#geneos.latency.files.newTickHistoryFile "geneos.latency.files.newTickHistoryFile") or [`newMatchedTicksFile()`](#geneos.latency.files.newMatchedTicksFile "geneos.latency.files.newMatchedTicksFile"). The object returned by this method is intended to act as a prototype for a customised logger class. Instances of such a class are created by invoking the `new()` method of the prototype. These instances inherit the methods of TickLogger and any methods replaced in or added to the prototype. An instance of a TickLogger subclass should open a file as soon as it is created (or after it has checked that all required fields are present). However, creating the prototype in order to customise it should not cause a file to be opened. To resolve this conflict of requirements, the TickLogger constructor calls the `init()` method. [`TickLogger:init()`](#TickLogger:init() "geneos.latency.files.TickLogger:init") does nothing, but subclasses should override it to call [`TickLogger:open()`](#TickLogger:open() "geneos.latency.files.TickLogger:open"), after performing any required checks. To use the logger with a latency context, a method to create a writer function is also needed. *Parameters:* - `properties`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* ([`TickLogger`](#Class6 "geneos.latency.files.TickLogger")) The newly constructed prototype logger. #### Class TickLogger *class*`geneos.latency.files.``TickLogger` Base class for tick loggers. This class provides methods for opening a file with a name that includes a formatted date, and for rolling over logging to a new file at the end of a specified interval. This class is not normally used directly; it is subclassed by [`TickHistory`](#Class4 "geneos.latency.files.TickHistory") and [`MatchedTicks`](#Class5 "geneos.latency.files.MatchedTicks"). `TickLogger.``filename` Template to use for the name of the logging file. Can include format codes as understood by `geneos.helpers.formatTime()`, which will be interpolated into the file name when the file is opened (or when it is rolled over.) Defaults to `history.%Y%m%d`. `TickLogger.``timefmt` Template for tick timestamps. Defaults to `%H:%M:%S.%3f`, that is, millisecond precision. If overridden, this should be set to a format acceptable to `[geneos.helpers.formatTime()](#geneos.helpers.formatTime)`. `TickLogger.``file` The [`TickLogger:open()`](#TickLogger:open() "geneos.latency.files.TickLogger:open") method sets this to the file handle returned by [io.open](http://www.lua.org/manual/5.1/manual.html#pdf-io.open). `TickLogger:``new`(*properties*) Constructor method inherited by derived classes. Derived classes should not override this method. *Parameters:* - `properties`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`TickLogger`](#Class6 "geneos.latency.files.TickLogger")) The newly constructed prototype logger. `TickLogger:``init`() Placeholder method, called when a tick logger is constructed. This base class implementation does nothing. Derived classes should override it to check or set fields representing configurable propertties and then call [`TickLogger:open()`](#TickLogger:open() "geneos.latency.files.TickLogger:open"). `TickLogger:``open`() Opens a file, expanding any time format codes in the name. The file is opened for append, so that if the sampler is restarted, an existing file will not be lost. `TickLogger:``getActualFilename`() Returns the name of the current log file. If the file is in the course of being rolled over, returns the name of the last file used. Also returns any error message encountered if the file could not be opened. This function may be used to populate a headline in an 'admin' dataview showing the name of the file. *Returns:* - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) The file name, including the expansion of any time format codes; nil if the file was not opened. - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) The error message returned if the file could not be opened; nil if the file was opened successfully. `TickLogger:``setFileRoll`([*rollInterval*[, *firstRollTime*[, *rollCallback*]]]) Enables timed rollover on the logging file. After this method is called, each time the specified roll interval expires, the file will be closed and reopened. If [`filename`](#TickLogger.file "geneos.latency.files.TickLogger.filename") contains time format codes understood by `[geneos.helpers.formatTime()](#geneos.helpers.formatTime)`, these will be reinterpreted, so that a new file can be opened. Optionally, a callback function will be invoked each time the file is rolled over; this function could, for example, run a script to archive the file that has just been closed. *Parameters:* - `rollInterval`: (`number`) - `firstRollTime`: (`number`) - `rollCallback`: (`function`) *Returns:* - ([`TickLogger`](#Class6 "geneos.latency.files.TickLogger")) The tick logger (i.e. `self`) *Usage:* ``` local lf = require "geneos.latency.files" local gh = require "geneos.helpers" -- Tick history with hourly rollover, starting 5 minutes from now local oneHour = gh.parseTimeInterval("1H") local fiveMinutesFromNow = gh.gettimeofday() + gh.parseTimeInterval("5M") local tickHist = lf.newTickHistoryFile("ticks.%y%m%d%H%M"):setFileRoll(oneHour, fiveMinutesFromNow) -- Matched ticks CSV with daily rollover, starting at midnight local matchedTickCsv = lf.newMatchedTicksFile("matches.%y%m%d.csv"):setFileRoll() ``` `TickLogger:``checkFileRoll`(*sampleTime*) Tests whether file should be rolled over. Derived classes should call this method before logging each sample, passing the sample time. If a roll time has been set and has elapsed, this method will do the following, in this order: - Calculate the next roll time by adding the roll interval to the roll time that has just expired. - Close the existing logging file by calling [`TickLogger:close()`](#geneos.latency.files.TickLogger:close "geneos.latency.files.TickLogger:close"). - Invoke the roll callback function, if any. - Open the new logging file by calling [`TickLogger:open()`](#geneos.latency.files.TickLogger:open "geneos.latency.files.TickLogger:open"). *Parameters:* - `sampleTime`: (`number`) *Returns:* - (`boolean`) True if the file was rolled over. `TickLogger:``sampleEnd`() Flushes the logging file. `TickLogger:``close`() Closes the logging file. #### Class TickHistory *class*`geneos.latency.files.``TickHistory` This class Logs full details of each tick received, in a format that can be read by the TestFile adapter. Ticks from all subscriptions are logged in order of their timestamps, with information showing how they were matched. Note that this file can become very large quite quickly; the [`TickLogger:setFileRoll()`](#TickLogger:setFileRoll "geneos.latency.files.TickLogger:setFileRoll") method can be used to mitigate this. See also these methods inherited from from TickLogger: [`TickLogger:setFileRoll()`](#TickLogger:setFileRoll "geneos.latency.files.TickLogger:setFileRoll"), [`TickLogger:getActualFilename()`](#TickLogger:getActualFilename() "geneos.latency.files.TickLogger:getActualFilename"). Instances of this class are constructed by the [`geneos.latency.files.newTickHistoryFile()`](#geneos.latency.files.newTickHistoryFile "geneos.latency.files.newTickHistoryFile") function. `TickHistory:``getWriter`() Provides a tick history writer function. This method returns a function which can be passed to `geneos.latency.Context:addWriter()`. The writer function calls `[TickLogger:checkFileRoll](#TickLogger:checkFileRoll)` at the start of each sample. *Returns:* - (`geneos.latency.historyWriter()`) A history writer function *Usage:* ``` local lf = require "geneos.latency.files" local gh = require "geneos.helpers" -- Create a tick and enable daily rollover local tickHist = lf.newTickHistoryFile("ticks.%y%m%d"):setFileRoll() --[[ code to create ctx as a latency context omitted here ]] ctx:addWriter(tickHist:getWriter()) ``` #### Class MatchedTicks *class*`geneos.latency.files.``MatchedTicks` This class writes a file in CSV (comma separated values) format, showing the matches computed by the tick matching algorithm. Each line of the file displays a matched tick from the baseline feed against matches found from the other configured feeds. Baseline ticks with no matches are not written to the file. This class overrides overrides [`TickLogger.timefmt`](#TickLogger.timefmt "geneos.latency.files.TickLogger.timefmt") to `%H:%M:%S.%6f` if microsecond precision is specified when an instance is created. See also these methods inherited from from TickLogger: [`TickLogger:setFileRoll()`](#TickLogger:setFileRoll "geneos.latency.files.TickLogger:setFileRoll"), [`TickLogger:getActualFilename()`](#TickLogger:getActualFilename() "geneos.latency.files.TickLogger:getActualFilename") Instances of this class are constructed by the [`geneos.latency.files.newMatchedTicksFile()`](#geneos.latency.files.newMatchedTicksFile "geneos.latency.files.newMatchedTicksFile") function. `MatchedTicks:``getWriter`() Provides a tick history writer function. This method returns a function which can be passed to `geneos.latency.Context:addWriter()`. The writer function calls `[TickLogger:checkFileRoll](#TickLogger:checkFileRoll)` at the start of each sample. *Returns:* - (`geneos.latency.historyWriter()`) A history writer function ### geneos.latency The latency module leverages the market data module to compute the relative latency between two or more feeds.
Functions Description
newContext() Constructs a latency context.
getTickComparator() Constructs a tick comparison function.
historyWriter() Specification of a tick history writer function.
TickPreprocessor() Specification for tick pre-processing functions used by the Context class.
LcsComparator() Specification for comparison functions used by the LCS algorithm.
LcsMatchListener() Specification for callback functions used by the LCS algorithm.
Class Context Description
Context:setBase() Specifies name and configuration of the baseline feed.
Context:addFeed() Specifies name and configuration of a monitored (not baseline) feed.
Context:addWriter() Adds a tick history writer.
Context:getConfiguredNames() Provides a table that can be used to configure a tick history writer.
Context:start() Establishes subscriptions for the feeds that have been specified by calling setBase and addFeed.
Context:sample() Process one sample.
Context:getMetrics() Get metrics for given feed or subscription.
Context:getLastTick() Get last non-duplicate tick received on given subscription.
Context:getFeedStatus() Get status of given feed.
Context:writeHistory() Invokes history writer(s) to log tick history for previous sample.
Class Lcs Description
Lcs:new() Constructs an Lcs matrix.
Lcs:populate() Populates the LCS matrix from a list of ticks.
Lcs:populateFromArray() Populates the LCS matrix from an array of values.
Lcs:compute() Computes the LCS for a populated matrix.
Class Metrics Description
Metrics:new() Constructor.
Metrics:onSampleBegin() Called at start of a sample to reset the statistics.
Metrics:onMatch() Called to update the statistics for each tick match detected by the LCS algorithm.
Metrics:aggregate() Called to update feed metrics with a set of computed subscription metrics.
Metrics:onSampleEnd() Called at the end of processing for each sample.
Metrics:getAverage() Calculates the mean of match latency for the sample.
Metrics:getStdDev() Calculates standard deviation of match latency for the sample.
#### Functions `geneos.latency.``newContext`(*options*) Constructs a latency context. Optional settings for the context can be specified by setting any of the following fields in the `options` parameter. **tolerances**, **timeDiffMax**, **timeDiffMin**: used when constructing a tick comparison function. See [`getTickComparator()`](#geneos.latency.getTickComparator "geneos.latency.getTickComparator") below. If no value is provided for these fields in `options`, a default of two seconds is used for *timeDiffMax* and the other fields are left as nil. *Parameters:* - `options`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`Context`](#Class3 "geneos.latency.Context")) The new latency context `geneos.latency.``getTickComparator`(*tolerances*[, *timeDiffMax*[, *timeDiffMin*]]) Constructs a tick comparison function. This factory function is used by [`Context:setBase()`](#geneos.latency.context.setbase "geneos.latency.Context:setBase") and [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") to construct tick preprocessing and matching functions if the corresponding parameters are omitted in the function call. It can be called directly to create a customised matching function. *Parameters:* - `tolerances`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `timeDiffMax`: (optional `number`) - `timeDiffMin`: (optional `number`) *Returns:* - a function that meets the requirements both of a tick matching function [`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator") and of a tick pre-processing function [`TickPreprocessor()`](#geneos.latency.TickPreprocessor "geneos.latency.TickPreprocessor") *Usage:* ``` local lat = require "geneos.latency" -- Create a matcher with following customisations: -- Allow Mid field to match with tolerance of .005 -- Require exact match (and allow string matching) for Venue field -- Do not accept matches with latency > 1.5 sec or less than zero local tolerances = { Mid = 0.005, Venue = 0 } local match\_fn = lat.getTickComparator(tolerances, 1.5, 0.0) ``` `geneos.latency.``historyWriter`(*event*, *data*) Specification of a tick history writer function. You can define one or more functions that match this specification and add them to a latency context. If any history writer is present, all ticks received in a sample by the latency context will be: - Sorted into sequence in order of arrival time. - A field named `seq` will be added to each tick giving its position in the sequence of all ticks received since the context was constructed. At the end of each sample, each history writer will be called for the ticks that arrived in the **previous** sample. History writing is deferred in this way because a the `remark` field of a tick may be updated if a match is found for that tick against data in the next sample. *Parameters:* - `event`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `data`: (optional `number`, `Tick`) *Usage:* ``` local gh = require "geneos.helpers" local lat = require "geneos.latency" -- Define a function to output the time of each sample and the arrival time -- and sequence number of each tick local function printLogger(event, data) if event == 'sampleTime' then io.write(gh.formatTime("== Sample, %H:%M:%S ==", data)) elseif event == 'tick' then io.write(string.format("%s %s %d %s %s", data.feed, data.inst, data.seq, gh.formatTime("%H:%M:%S.%3f", data.timeFirst), data.remark or '-')) end end -- Register the function (ctx is a latency context created elsewhere) ctx:addWriter(printLogger) ``` `geneos.latency.``TickPreprocessor`(*prev*, *current*) Specification for tick pre-processing functions used by the [`Context`](#Class3 "geneos.latency.Context") class. The [`Context`](#Class3 "geneos.latency.Context") class uses a function of this type to detect and remove duplicate ticks from each feed, including the baseline feed. An alternative tick pre-processing function can be passed to [`Context:setBase()`](#geneos.latency.context.setbase "geneos.latency.Context:setBase") and [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") to inspect, amend or discard ticks before they are passed to the LCS algorithm. Each time the function is called, it is passed two tick values: one is the previous accepted tick in the subscription (that is, the stream of ticks for a given instrument from a given feed), the other is the current tick. The 'previous' tick will have `nil` values for feed and instrument if no ticks have yet been accepted in the subscription. *Parameters:* - `prev`: (`geneos.marketdata.Tick`) - `current`: (`geneos.marketdata.Tick`) *Returns:* - (`boolean`) `nil` or `false` if the current tick is to be accepted, `true` (or any non-nil value other than `false`) if it is to be discarded. `geneos.latency.``LcsComparator`(*base*, *feed*) Specification for comparison functions used by the LCS algorithm. A comparison function (comparator) is used by the LCS algorithm (as implemented by the [`Lcs`](#Class_Lcs "geneos.latency.Lcs") class) to compare values from different sequences against each other for equality. When used in the Latency API, the function will be called to compare each tick from the baseline feed with each tick from a comparison feed, to identify possible matches. An alternative comparison function can be passed to [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") to change either the criteria for deciding whether ticks match or the weighting assigned to a match. *Parameters:* - `base`: - `feed`: *Returns:* - (optional `number`) `nil` or `false` for no match, or if matched a weighting value. The weighting is used by the LCS algorithm to select between multiple match candidates, with larger values indicating a lower ranking meaning the match is less likely to be chosen. - An optional value to associate with the match. If the match is selected by the LCS algorithm, this value will be passed to the specified [`LcsMatchListener()`](#geneos.latency.LcsMatchListener "geneos.latency.LcsMatchListener") (see [`Lcs:compute()`](#Lcs:compute "geneos.latency.Lcs:compute")). `geneos.latency.``LcsMatchListener`(*value*, *index*, *base*, *feed*) Specification for callback functions used by the LCS algorithm. A match listener function is used receive results from the LCS algorithm (as implemented by the [`Lcs`](#geneos.latency.Lcs "geneos.latency.Lcs") class). The listener will be called-back repeatedly during execution of the algorithm (during [`Lcs:compute()`](#Lcs:compute "geneos.latency.Lcs:compute")). Each call will identify a single pair of ticks matched between the "base" and "feed" sequences. *Parameters:* - `value`: - `index`: - `base`: - `feed`: #### Class Context *class*`geneos.latency.``Context` The geneos.latency.Context class performs relative feed latency calculations. Instances of this class are constructed by the [`geneos.latency.newContext()`](#geneos.latency.newContext "geneos.latency.newContext") function. `Context:``setBase`(*name*, *baseConfig*[, *preprocCmp*]) Specifies name and configuration of the baseline feed. This method should be called before [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") *Parameters:* - `name`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `baseConfig`: (`geneos.marketdata.FeedConfig`) - `preprocCmp`: (optional [`TickPreprocessor()`](#geneos.latency.TickPreprocessor "geneos.latency.TickPreprocessor")) *Returns:* - ([`Context`](#Class3 "geneos.latency.Context")) The context object (i.e. `self`) `Context:``addFeed`(*name*, *feedConfig*[, *preprocCmp*[, *matchCmp*[, *metrics*]]]) Specifies name and configuration of a monitored (not baseline) feed. **Note** The configuration is pruned so that those instruments and fields that are not present in the configuration of the baseline feed are removed from the configuration of the monitored feed. *Parameters:* - `name`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `feedConfig`: (`geneos.marketdata.FeedConfig`) - `preprocCmp`: (optional [`TickPreprocessor()`](#geneos.latency.TickPreprocessor "geneos.latency.TickPreprocessor")) - `matchCmp`: (optional [`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator")) - `metrics`: (optional [`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) *Returns:* - ([`Context`](#Class3 "geneos.latency.Context")) The context object (i.e. `self`) `Context:``addWriter`(*writer*) Adds a tick history writer. This method allows a function to be supplied which will be called once per sample for all the input ticks in the previous sample. See also [`Context:writeHistory()`](#Context:writeHistory() "geneos.latency.Context:writeHistory") If used, this method should be called before calling [`Context:start()`](#geneos.latency.context.start "geneos.latency.Context:start") *Parameters:* - `writer`: ([`historyWriter()`](#geneos.latency.historyWriter "geneos.latency.historyWriter")) *Returns:* - ([`Context`](#Class3 "geneos.latency.Context")) The context object (i.e. `self`) `Context:``getConfiguredNames`() Provides a table that can be used to configure a tick history writer. This function is useful when configuring a matchedTicks history writer. See `[geneos.latency.files.newMatchedTicksFile()](#geneos.latency.files.newMatchedTicksFile)` for a usage example. *Returns:* - ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) A table with the following fields: `fieldNames`: the field names configured for the base feed, as a sorted sequence; `baseName`: the name of the base feed; `feedNames`: the names of the other feeds, as a sequence. `Context:``start`() Establishes subscriptions for the feeds that have been specified by calling setBase and addFeed. This method should not be called until after calling [`Context:setBase()`](#geneos.latency.context.setbase "geneos.latency.Context:setBase") and [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") (and [`Context:addWriter()`](#Context:addWriter "geneos.latency.Context:addWriter"), if required) *Returns:* - ([`Context`](#Class3 "geneos.latency.Context")) The context object (i.e. `self`) `Context:``sample`([*sampleTime*]) Process one sample. Collects data from all subscriptions, runs LCS algorithm and invokes history writer(s) to log the previous sample *Parameters:* - `sampleTime`: (optional `number`) `Context:``getMetrics`(*feedId*[, *inst*]) Get metrics for given feed or subscription. *Parameters:* - `feedId`: - `inst`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* (optional [`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) The requested metrics object, or nil if *feedId* or *inst* is not valid. `Context:``getLastTick`(*feedId*, *inst*) Get last non-duplicate tick received on given subscription. *Parameters:* `feedId`: `inst`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* (optional `[geneos.marketdata.Tick](#geneos.marketdata.Tick)`) The requested tick, if available. if the subscription does not exist, the return value is nil. If no ticks have been received, the return value is a tick with times set to zero and and empty table of field values. `Context:``getFeedStatus`(*feedId*) Get status of given feed. *Parameters:* - `feedId`: *Returns:* (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) The status of the specified feed, or nil if `feedId` is not valid. `Context:``writeHistory`() Invokes history writer(s) to log tick history for previous sample. **Note** history writing is delayed by one sample so that matches can be annotated across a sample boundary. This method is called by [`Context:sample()`](#geneos.latency.context.sample "geneos.latency.Context:sample"); it can also be called directly if required, to log the last sample if the sampler is being stopped. This can be achieved by defining `geneos.sampler.finalise()`, as in the example below. *Usage:* ``` local gs = require "geneos.sampler" --[[ ctx is a latency context with a history writer ]] gs.finalise = function() ctx:writeHistory() end ``` #### Class Lcs *class*`geneos.latency.``Lcs` An implementation of the Longest Common Subsequence (LCS) algorithm. This class can be used to run the LCS algorithm on any pair of sequences of comparable items. In the Latency API, it is used to compare a series of ticks from the baseline feed against a series of ticks from the comparison (or monitored) feed. The default implementation uses a comparison function which matches applying constraints to time and field values. Output from the algorithm is a series consisting of pairs of matched ticks (a baseline and feed tick) which are then used to compute latency statistics. To use this class, callers should: 1. Create a new instance with [`Lcs:new()`](#Lcs:new() "geneos.latency.Lcs:new"), 2. Fill the matrix using [`Lcs:populate()`](#Lcs:populate "geneos.latency.Lcs:populate") or [`Lcs:populateFromArray()`](#Lcs:populateFromArray "geneos.latency.Lcs:populateFromArray") passing an [`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator") function. 3. Run the LCS algorithm with [`Lcs:compute()`](#Lcs:compute "geneos.latency.Lcs:compute") and process the matches via an [`LcsMatchListener()`](#geneos.latency.LcsMatchListener "geneos.latency.LcsMatchListener") function. *Usage:* ``` local md = require "geneos.marketdata" local lat = require "geneos.latency" local comparator -- A comparator function, e.g. lat.getTickComparator -- Get ticks for processing local ticks1 = feed\_1:getTicks("myInstrument") local ticks2 = feed\_2:getTicks("myInstrument") -- Run LCS algorithm and place results into metrics object local metrics = lat.Metrics:new() metrics:onSampleBegin(md.countTicks(ticks2)) lat.Lcs:new():populate(ticks1, ticks2, comparator):compute(metrics) metrics:onSampleEnd() ``` `Lcs:``new`([*proto*]) Constructs an Lcs matrix. The functionality of the new matrix may be altered by providing a table argument `proto`, populated with new (or overriding) function and field definitions. *Parameters:* - `proto`: (optional [table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`Lcs`](#geneos.latency.Lcs "geneos.latency.Lcs")) A new LCS matrix. `Lcs:``populate`(*baseTicks*, *feedTicks*, *cmp*[, *baseLength*[, *feedLength*]]) Populates the LCS matrix from a list of ticks. This method populates the LCS matrix with a linked-list series of ticks, such as those returned from a call to `[geneos.marketdata.Feed:getTicks()](#geneos.marketdata.Feed.getTicks)`. *Parameters:* - `baseTicks`: (`geneos.marketdata.Tick`) - `feedTicks`: (`geneos.marketdata.Tick`) - `cmp`: ([`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator")) - `baseLength`: (optional `number`) - `feedLength`: (optional `number`) *Returns:* - ([`Lcs`](#geneos.latency.Lcs "geneos.latency.Lcs")) This LCS matrix object (i.e. `self`). `Lcs:``populateFromArray`(*baseSeq*, *feedSeq*, *cmp*[, *baseLength*[, *feedLength*]]) Populates the LCS matrix from an array of values. This method populates the LCS matrix with values from two source arrays. *Parameters:* - `baseSeq`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `feedSeq`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `cmp`: ([`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator")) - `baseLength`: (optional `number`) - `feedLength`: (optional `number`) *Returns:* - ([`Lcs`](#geneos.latency.Lcs "geneos.latency.Lcs")) This LCS matrix object (i.e. `self`). `Lcs:``compute`(*listener*) Computes the LCS for a populated matrix. This method should not be called until the LCS matrix has been populated. When called it will compute the longest common subsequence from the input sequences, and call the specified `listener` function for each match in the output sequence. The output sequence is selected by minimising the total weight of the sequence using the weighting values returned by the [`LcsComparator()`](#geneos.latency.LcsComparator "geneos.latency.LcsComparator") function, while maximising the number of matches that make up the sequence. The `listener` function should follow the [`LcsMatchListener()`](#geneos.latency.LcsMatchListener "geneos.latency.LcsMatchListener") function specification. This function will be called once for each match found, and matches will be returned in reverse order (matches found later in the input sequences will be returned first). Alternatively, the `listener` may be a table with an `onMatch` field that is a function following the [`LcsMatchListener()`](#geneos.latency.LcsMatchListener "geneos.latency.LcsMatchListener") function specification. *Parameters:* - `listener`: ([`LcsMatchListener()`](#geneos.latency.LcsMatchListener "geneos.latency.LcsMatchListener"), [table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - lastMatchedBase Index of base item selected for last match (0 if there were no matches) - lastMatchedFeed Index of feed item selected for last match (0 if there were no matches) ### Class Metrics *class*`geneos.latency.``Metrics` The Metrics class stores and computes latency statistics. Instances of the Metrics class are constructed by [`geneos.latency.Context`](#Class3 "geneos.latency.Context") for each feed, and for each instrument subscription. Each sample, metrics data is gathered at the instrument subscription level as the LCS algorithm is run. Data for instruments of a feed are then rolled up to the feed level at the end of processing. A Metrics class instance is a Lua table containing the methods described below, and the following fields. Note that because latencies are calculated for monitored (not baseline) feeds, metrics objects for the baseline feed have only the `numTicks` statistics field. - `feed` The name of the feed for which this object collects metrics. - `inst` The name of the instrument, or `nil` for feed-level metrics. - `numTicks` The number of ticks in the current sample. - `matches` The number of matched ticks found in the current sample. - `latMax` The maximum latency value for the current sample. - `latMin` The minimum latency value for the current sample. - `latSum` The sum of latency values for the current sample. - `latSumSq` The sum of of the squares of the latency values for the current sample. You can use the [`Metrics:new()`](#Metrics:new "geneos.latency.Metrics:new") method to create a specialisation of this class and pass it to [`geneos.latency.Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") to be used as a prototype for metrics objects of subscriptions of that feed. *Usage:* ``` -- Statistics calculation follows the template method (design) pattern. -- The algorithm runs according to the following pseudocode: for f in feeds() do f.metrics:onSampleBegin(0) for i in f.instruments() do i.metrics:onSampleBegin(tickCount) -- series of i.metrics:onMatch calls as LCS algorithm is executed i.metrics:onSampleEnd() f.metrics:aggregate(i.metrics) -- roll up stats end f.metrics:onSampleEnd() end ``` `Metrics:``new`([*proto*]) Constructor. Use this method to create a new instance, or derive a new class from the Metrics object. A derived class instance may be passed as an argument to [`geneos.latency.Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed"), where it will be used a prototype for creating new metrics objects. *Parameters:* - `proto`: (optional [table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) An object which can be passed to [`Context:addFeed()`](#geneos.latency.context.addFeed "geneos.latency.Context:addFeed") to be used as the metrics object for the feed. The object will also and as a prototype to create metrics objects for instrument subscriptions. *Usage:* ``` local lat = require "geneos.latency" -- Derive a metrics object with an onMatch function that outputs match information local fmt = "Saw match on %s.%s at %f" local displayMatchMetrics = lat.Metrics:new { onMatch = function(self, latency, nm, bt, ft) lat.Metrics.onMatch(self, latency, nm, bt, ft) -- must call inherited method io.write(fmt:format(ft.feed, ft.inst, ft.timeFirst)) end } -- Use the specialised metrics object. -- ctx is an instance of geneos.latency.Context created and initialised elsewhere. ctx:addFeed(feedName, feedConfig, nil, nil, displayMatchMetrics) ``` `Metrics:``onSampleBegin`(*numTicks*) Called at start of a sample to reset the statistics. This method will: - Save the number of ticks in the `numTicks` field - Set the `matches`, `latSum` and `latSumSq` fields to 0. - Set the `latMin` and `latMax` fields to `nil`. You should override this method if you add any fields to the class that need to be initialised each sample. If you override this method, you must ensure the inherited method is called from within your overriding method. *Parameters:* - `numTicks`: (`number`) *Returns:* - ([`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) This metrics object (i.e. `self`). `Metrics:``onMatch`(*latency*, *nm*, *bt*, *ft*) Called to update the statistics for each tick match detected by the LCS algorithm. This method updates the contained statistics to include the new match. In addition it will add/update the `remark` field of the base- and feed-tick to register the matches (see `[geneos.marketdata.Tick.remark](#geneos.marketdata.Tick.remark)`). You can override this method to do additional processing on each match. Note that the LCS matching algorithm reports matches in reverse chronological order (i.e. matches for more recent ticks are reported first). If you override this method, you must ensure the inherited method is called from within your overriding method. **Note** this method does not validate its input arguments for performance reasons. *Parameters:* - `latency`: (`number`) - `nm`: (`number`) - `bt`: (`[geneos.marketdata.Tick](#geneos.marketdata.Tick)`) - `ft`: (`[geneos.marketdata.Tick](#geneos.marketdata.Tick)`) `Metrics:``aggregate`(*other*) Called to update feed metrics with a set of computed subscription metrics. This method updates the statistics fields of the object, namely: - `numTicks` - `matches` - `latMax` - `latMin` - `latSum` - `latSumSq` **Note** `latMax` and `latMin` values will be `nil` unless the metrics object has a non-zero match count. Override this method if you add any fields that need to be aggregated at the feed level. If you override this method, you must ensure the inherited method is called from within your overriding method. *Parameters:* - `other`: ([`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) *Returns:* - ([`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) This metrics object (i.e. `self`). `Metrics:``onSampleEnd`() Called at the end of processing for each sample. This method does nothing; it is provided as a hook to be overridden if required. This method is called for subscription-level metrics before aggregation into the feed. It is called for feed-level metrics after all subscriptions have been aggregated. *Returns:* - ([`Metrics`](#geneos.latency.Metrics "geneos.latency.Metrics")) This metrics object (i.e. `self`). `Metrics:``getAverage`() Calculates the mean of match latency for the sample. *Returns:* - (optional `number`) Total latency divided by number of matches, or `nil` if there were no matches. `Metrics:``getStdDev`() Calculates standard deviation of match latency for the sample. *Returns:* - (optional `number`) Root mean squared deviation of observed latency values from the mean, or `nil` if there were no matches. ### geneos.marketdata The market data module provides access to market data feeds. The majority of the functionality of this module is provided by the Feed class.
Functions Description
addFeed() Configures a connection to a data feed.
countTicks() Counts the number of ticks in a tick list, as returned by Feed:getTicks().
ticksToArray() Converts a list of ticks (as returned by Feed:getTicks()) to an array.
Class FeedConfig Description
FeedConfig.feed Settings that control how the feed adapter library is loaded.
FeedConfig._type_ Feed adapter library settings.
FeedConfig.instruments Instruments to be subscribed to when feed is started.
FeedConfig.fields Default field name mapping.
Class Feed Description
Feed:start() Starts the feed.
Feed:stop() Stops the feed.
Feed:getStatus() Gets the status of the feed.
Feed:subscribe() Subscribes to an instrument.
Feed:unsubscribe() Unsubscribes from an instrument.
Feed:getTicks() Collect ticks for an instrument.
Feed:getStats() Obtain tick statistics for an instrument.
Feed:__tostring() Gets a string representation of the feed object.
Class Tick Description
Tick.feed The name of the feed, as supplied to geneos.marketdata.addFeed().
Tick.inst The display name of the instrument.
Tick.next The next tick in the list.
Tick.timeFirst The time the tick (or first conflated tick) arrived.
Tick.timeLast The time the tick (or last conflated tick) arrived.
Tick.field The market data fields of the tick.
Tick.seq The arrival sequence number of the tick.
Tick.image Flag that indicates this tick is an image.
Tick.remark Annotation by latency API.
#### Functions `geneos.marketdata.``addFeed`(*name*, *feedConfig*) Configures a connection to a data feed. *Parameters:* - `name`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `feedConfig`: ([`FeedConfig`](#geneos.marketdata.FeedConfig "geneos.marketdata.FeedConfig")) *Returns:* - (optional [`Feed`](#geneos.marketdata.Feed "geneos.marketdata.Feed")) A feed object representing the new connection, or nil on failure. - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure. *Usage:* ``` local md = require "geneos.marketdata" local config = { feed = { type = "example", ["library.filename"] = "flm-feed-example", }, example = { publishingPeriod = "2000", } instruments = { ["GBP/USD"] = "some.code.for.GBP", ["USD/EUR"] = "the.code.for.EUR", }, fields = { Trade = "TRDPRC\_1", Bid = "BID", Ask = "ASK", } } local exampleFeed = assert(md.addFeed("Example", config)) ``` `geneos.marketdata.``countTicks`(*first*[, *last*]) Counts the number of ticks in a tick list, as returned by [`Feed:getTicks()`](#geneos.marketdata.Feed.getTicks "geneos.marketdata.Feed:getTicks"). This function counts the number of ticks in the inclusive range [first,last]. If last is `nil`, then all ticks in the list will be counted. *Parameters:* - `first`: ([`Tick`](#geneos.marketdata.Tick "geneos.marketdata.Tick")) - `last`: (optional [`Tick`](#geneos.marketdata.Tick "geneos.marketdata.Tick")) *Returns:* - The count of ticks in the list. `geneos.marketdata.``ticksToArray`(*first*[, *last*[, *count*]]) Converts a list of ticks (as returned by [`Feed:getTicks()`](#geneos.marketdata.Feed.getTicks "geneos.marketdata.Feed:getTicks")) to an array. This function converts the ticks in the inclusive range [first,last] from a list to an array. The resulting output has the same order, so the tick at index 1 will be `first`. If `last` is `nil` then all ticks in the list will be converted. If count is not `nil`, it will be used to optimise the allocation of the array **Note** the `next` field in each tick will remain untouched. This may be of particular importance if you reorder the ticks in the array. *Parameters:* - `first`: ([`Tick`](#geneos.marketdata.Tick "geneos.marketdata.Tick")) - `last`: (optional [`Tick`](#geneos.marketdata.Tick "geneos.marketdata.Tick")) - `count`: (optional `number`) *Returns:* - ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) An array containing the ticks. - (`number`) The number of ticks in the array. #### Class FeedConfig *class*`geneos.marketdata.``FeedConfig` The feed configuration 'class' is a Lua table containing the configuration for a data feed connection. A table conforming to this type is a required parameter for the [`geneos.marketdata.addFeed()`](#geneos.marketdata.addFeed "geneos.marketdata.addFeed") function. `FeedConfig.``feed` Settings that control how the feed adapter library is loaded. The following fields can appear in this nested table: - **type** (Required.) The feed type. If this is "custom", the feed adapter library file name is interpreted relative to the Netprobe working directory; otherwise it is interpreted relative to the `flm` subdirectory of the Netprobe directory. - **library.filename** (Required.) The file name of the feed adapter library. - **library.skipVersionCheck** (Optional.) Set to "true" if the version of the feed adapter library does not need to match the version of the Netprobe. - **library.debug.load** (Optional.) Set to "true" to debug the process of loading the feed adapter library - **verbose** (Optional.) Interpreted by the feed adapter library. In general, setting it to **True** enables verbose output from the feed adapter. It is a boolean and writes to the Netprobe logs. **Note** Where the name of a field includes dots, it can be specified either using Lua's syntax for field names that are not valid identifiers `["key.subkey"]="value"` or using a nested table `key={subkey="value"}`. *Usage:* ``` -- The following examples show a configuration for an RFA feed, -- and are equivalent due to the default values for the parameters. -- verbose syntax, with redundant optional fields local config = { feed = { type = "rfa", -- The feed type. See note above. ["library.filename"] = "flm-feed-rfa.so", }, -- Required: File name of the feed adapter library. ["library.skipVersionCheck"] = "false" -- Do the version check (this is the default) ["library.debug.load"] = "false" -- Don't debug loading the library (also default) verbose = "false", -- No verbose output from adapter (also default) }, rfa = {} -- see FeedConfig.type } -- More concise syntax (but still with one redundant optional field) local config = { feed = { type = "rfa", library = { filename = "flm-feed-rfa.so", skipVersionCheck = "false" } }, rfa = {} -- see FeedConfig.type } ``` The default value of `verbose` is **False** when not included in the feed configuration. This is also set to false, unless specific logs are expected from the feed. Setting the `verbose` to **True** provides more detailed logs in the Netprobe log which is possibly more useful in development, testing, or debugging. For example, after subscription to a certain instrument, Netprobe includes the logs for every tick per changes on the subscribed instrument. Here is an example of logs when `verbose` is set to **True**: ``` DEBUG: MARKET-DATA-MONITOR:bbDirect:INFO Sending subscription request to instrument 'GBPJPY Curncy' on service '//blp/mktdata' (0x7fc5040035b0) DEBUG: MARKET-DATA-MONITOR:bbDirect:INFO Received event SubscriptionStarted 3 on '0x7fc5040035b0': SubscriptionStarted = { exceptions[] = { } } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { LAST_PRICE = 148.793000 BID = 148.785000 ASK = 148.800000 HIGH = 148.965000 LOW = 148.749000 } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { BID = 148.785000 } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { LAST_PRICE = 148.793000 } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { ASK = 148.800000 } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { LAST_PRICE = 148.793000 } DEBUG: MARKET-DATA-MONITOR:bbDirect:DEBUG Received event MarketDataEvents 8 on '0x7fc5040035b0': MarketDataEvents = { BID = 148.785000 } ``` Here is an example of logs when `verbose` is set to **False**: ``` DEBUG: MARKET-DATA-MONITOR:bbDirect:INFO Sending subscription request to instrument 'GBPJPY Curncy' on service '//blp/mktdata' (0x7f3764000bd0) DEBUG: MARKET-DATA-MONITOR:bbDirect:INFO Received event SubscriptionStarted 3 on '0x7f3764000bd0': SubscriptionStarted = { exceptions[] = { } } 26SEP2018_10:36:45.991 24706:139876365346560 ERROR blpapi_platformtransporttcp.cpp:553 blpapi.session.transporttcp.{1}.<127.0.0.1:13523> Connection failed ``` `FeedConfig.``_type_` Feed adapter library settings. The name of this nested table must match the value of the `feed.type` parameter. If the feed type is `rfa` for example, the table must be named **rfa** (see example 1 below). The contents of this table depends on the feed parameters accepted by the feed adapter library. For non-"custom" feed types, parameters passed to the feed adapter will be prefixed with the feed type. If the feed type is "custom", the key names will be passed verbatim (i.e. without the "custom" prefix). *Usage:* ``` -- Example 1: rfa feed configuration local config = { feed = { type = "rfa", library = { filename = "flm-feed-rfa.so" } }, rfa = { configFile = "RFA\_DLL.cfg", session = "Session2" } } -- The feed adapter receives the following parameters: -- feed.type = "rfa" -- feed.library.filename = "flm-feed-rfa.so" -- rfa.configFile = "RFA\_DLL.cfg" -- rfa.session = "Session2" -- Example 2: "custom" feed configuration local config = { feed = { type = "custom", ["library.filename"] = "feed\_x.dll", }, custom = { feed\_x = { config = "local.ini" }, volume = "11" } } -- The feed adapter receives the following parameters: -- feed.type = "custom" -- feed.library.filename = "feed\_x.dll" -- feed\_x.config = "local.ini" -- volume = "11" ``` `FeedConfig.``instruments` Instruments to be subscribed to when feed is started. This nested table is optional. If it is present, it must be a mapping from the instrument names used for display and debugging purposes to the feed vendor API codes used by the feed adapter to identify the instruments. If this nested table is present, the fields nested table is required and will be used to specify the fields to subscribe to for each instrument. *Usage:* ``` local config = { feed = { }, rfa = { }, -- see FeedConfig.feed, FeedConfig.\_type\_ instruments = { ABC = "DIRECT\_FEED.ABC.N", DEF = "DIRECT\_FEED.DEF.N", }, fields = { } -- see FeedConfig.fields } ``` `FeedConfig.``fields` Default field name mapping. This nested table is optional, unless the instruments table is present, in which case it is required. If it is present, it must be a mapping from the field names used for display and debugging purposes to the feed vendor API codes used by the feed adapter to identify the fields. This table is used to specify the fields to subscribe to for each instrument, unless a different field mapping is specified in a call to [`Feed:subscribe()`](#geneos.marketdata.Feed:subcribe() "geneos.marketdata.Feed:subscribe") *Usage:* ``` local config = { feed = { }, rfa = { }, -- see FeedConfig.feed, FeedConfig.\_type\_ instruments = { }, -- see FeedConfig.instruments fields = { Trade = "TRDPRC\_1", Bid = "BID", Ask = "ASK" } } ``` #### Class Feed *class*`geneos.marketdata.``Feed` The feed class is a Lua object representing a feed connection. Feed objects are returned by the [`geneos.marketdata.addFeed()`](#geneos.marketdata.addFeed "geneos.marketdata.addFeed") function. `Feed:``start`() Starts the feed. This method configures the feed adapter and sets up the subscriptions configured when the feed was created. `Feed:``stop`() Stops the feed. This method cancels instrument subscriptions and removes the feed adapter connection. `Feed:``getStatus`() Gets the status of the feed. *Returns:* - ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) Status reported by feed adapter or 'Not started' if the connection has not been started. `Feed:``subscribe`(*instName*, *instCode*[, *fields*[, *maxHeldTicks*]]) Subscribes to an instrument. Use this method to make additional subscriptions after starting the feed. **Note** that [`Feed:start()`](#geneos.marketdata.Feed:start "geneos.marketdata.Feed:start") will subscribe to all instruments configured in the call to [`addFeed()`](#geneos.marketdata.addFeed "geneos.marketdata.addFeed"). *Parameters:* - `instName`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `instCode`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `fields`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) - `maxHeldTicks`: (`number`) *Returns:* - (optional [table](http://www.lua.org/manual/5.1/manual.html#5.5)) `true` if the call succeeded or `nil` if there was an error.\ - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure. One of the following: - 'Required argument(s) missing in call to subscribe' - 'Subscription to '*instCode* (*instName*)' failed' `Feed:``unsubscribe`(*instName*) Unsubscribes from an instrument. Use this method to cancel subscriptions without stopping the feed. **Note** Thr [`Feed:stop()`](#geneos.marketdata.Feed:stop "geneos.marketdata.Feed:stop") will cancel all instrument subscriptions. *Parameters:* - `instName`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - (optional `boolean`) `true` if the call succeeded or `nil` if there was an error. - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure: - 'no subscription found for '*instName*' in call to unsubscribe' `Feed:``getTicks`(*instName*) Collect ticks for an instrument. Each time this method is called, it returns a linked list containing all the ticks received since the last call for the specified instrument. *Parameters:* - `instName`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - (optional [`Tick`](#geneos.marketdata.Tick "geneos.marketdata.Tick")) The head of a linked list of ticks if the call succeeded or `nil` if there are no ticks or if an error occurred. - (Tick or string) The last tick received for the instrument, or nil if there are no ticks, or the message 'no subscription found for '*instName*' in call to getTicks' `Feed:``getStats`(*instName*) Obtain tick statistics for an instrument. Each time this method is called, it returns a new table with 4 fields: - **totalTicks** - The total number of ticks received for this instrument since the subscription was made. - **totalConflations** - The total number of ticks which were conflated (and so not seen) since the subscription was made. - **currentTicks** - Number of ticks received since the last 'getStats' call. - **currentConflations** - Number of ticks conflated since the last 'getStats' call. *Parameters:* - `instName`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - (optional [table](http://www.lua.org/manual/5.1/manual.html#5.5)) The stats table if the call succeeded, or `nil` if an error occurred. - (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) The message "no subscription found for '*instName*' in call to getStats" on error. `Feed:``__tostring`() Gets a string representation of the feed object. The current implementation of this method simply returns the name of the feed. *Returns:* - ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) The name of the feed *Usage:* ``` local feed = md.addFeed('myFeed', myconfig) local feedName = tostring(feed) -- assert(feedName == 'myFeed') ``` #### Class Tick *class*`geneos.marketdata.``Tick` The tick class is a Lua table representing a feed tick. Tick objects are returned by the [`getTicks()`](#geneos.marketdata.Feed:getTicks "geneos.marketdata.Feed:getTicks") method of feed objects. Ticks are generally members of linked lists representing part of the stream of ticks received for a single instrument on a feed. Because a tick is represented by a Lua table, Lua functions (whether in your script or in another API module) can join or split these lists as required by setting or clearing the [next](http://www.lua.org/manual/5.1/manual.html#pdf-next) field of the relevant ticks. Lua's garbage collector will dispose of ticks when and only when they are no longer part of a list whose head is still referenced by some variable that is still in scope. Normally a Tick object represents a single tick from the feed adapter and the [`timeFirst`](#geneos.marketdata.Tick.timeFirst "geneos.marketdata.Tick.timeFirst") and [`timeLast`](#geneos.marketdata.Tick.timeLast "geneos.marketdata.Tick.timeLast") fields will be the same. However, if a number of ticks have been conflated (for instance, if the buffer between the feed adapter and the sampler has been overrun), then the [`timeFirst`](#geneos.marketdata.Tick.timeFirst "geneos.marketdata.Tick.timeFirst") field will contain the arrival time of the first conflated tick and the [`timeLast`](#geneos.marketdata.Tick.timeLast "geneos.marketdata.Tick.timeLast") field will contain the arrival time of the last conflated tick. `Tick.``feed` The name of the feed, as supplied to [`geneos.marketdata.addFeed()`](#geneos.marketdata.addFeed "geneos.marketdata.addFeed"). `Tick.``inst` The display name of the instrument. `Tick.``next` The next tick in the list. If there is no next tick, this key will be absent. `Tick.``timeFirst` The time the tick (or first conflated tick) arrived. The value of this field is a Lua number representing a number of seconds since the Unix epoch (Midnight UTC on 1st January 1970) That is, it has the same interpretation as a C language time\_t, but it also has a fractional part. `Tick.``timeLast` The time the tick (or last conflated tick) arrived. `Tick.``field` The market data fields of the tick. This is a table mapping the display name of each of the fields configured for the instrument to its value, as reported by this tick or copied from the previous tick. Field values may be numbers or strings, depending on the implementation and configuration of the feed adapter. `Tick.``seq` The arrival sequence number of the tick. This field is added to each tick by the Latency API if a tick history writer is configured. `Tick.``image` Flag that indicates this tick is an image. The boolean flag will be present and set to true if the tick was created as an image from the underling data source. If the flag is not present then the tick represents and update to an exising image. `Tick.``remark` Annotation by latency API. The Latency API adds up to one of the following remarks to certain ticks as it processes each sample: - `Duplicate` indicates that the tick was excluded by the pre-processing function defined for the feed; - `MatchforN` indicates that this feed tick matches the baseline tick with sequence number `N`; - `MatchedbyN` indicates that this baseline tick was matched by the feed tick with sequence number `N` (and may also have been matched by ticks from other feeds); - `MatchwithlatencyL` indicates that this feed tick matches the baseline tick that arrived `L` seconds previously; - `MatchedwithlatencyL` indicates that this baseline tick was matched by the feed tick that arrived `L` seconds later (and may also have been matched by ticks from other feeds). ### geneos.sampler.commands The commands module provides helper functions to construct a command definition. These functions are intended to provide a fluent interface for commands, but are optional. Advanced users may simply create a table of the correct format should they wish.
Functions Description
newDefinition() Creates a new command definition.
Class Definition Description
Definition:setMenuPath() Sets the menu path of the command.
Definition:setDescription() Sets the command description.
Definition:addStringArgument() Adds a string-valued argument to the command.
Definition:addIntegerArgument() Adds an integer-valued argument to the command.
Definition:addNumericArgument() Adds a numeric-valued (floating-point) argument to the command.
Definition:addBooleanArgument() Adds a Boolean-valued argument to the command.
Definition:addChoiceArgument() Adds a choice of string-valued arguments to the command.
Definition:addRowTarget() Adds a target to the command, for a row (or rows) of the specified view.
Definition:addColumnTarget() Adds a target to the command, for a column of the specified view.
Definition:addHeadlineTarget() Adds a target to the command, for a headline of the specified view.
#### Functions `geneos.sampler.commands.``newDefinition`() Creates a new command definition. *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The new command definition. #### Class Definition *class*`geneos.sampler.commands.``Definition` A command definition specifies the properties and arguments of a command. `Definition:``setMenuPath`(*path*) Sets the menu path of the command. The menu path defines the hierarchy where the command is displayed, when performing a right-click context-menu action in a Geneos GUI. *Parameters:* - `path`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``setDescription`(*description*) Sets the command description. The description will appear at the top of the arguments dialog (if the command requires arguments). *Parameters:* - `description`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addStringArgument`([*description*[, *default*]]) Adds a string-valued argument to the command. String arguments will be rendered as a text-box in a Geneos GUI. *Parameters:* - `description`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `default`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addIntegerArgument`([*description*[, *default*]]) Adds an integer-valued argument to the command. Integer arguments will be rendered as number spinner in a Geneos GUI. *Parameters:* - `description`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `default`: (optional `integer`) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addNumericArgument`([*description*[, *default*]]) Adds a numeric-valued (floating-point) argument to the command. Floating-point arguments will be rendered as a formatted text-box in a Geneos GUI. *Parameters:* - `description`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `default`: (optional `number`) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addBooleanArgument`([*description*[, *default*]]) Adds a Boolean-valued argument to the command. Boolean arguments will be rendered as a check-box in a Geneos GUI. *Parameters:* - `description`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `default`: (optional `number`) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addChoiceArgument`(*description*, *options*) Adds a choice of string-valued arguments to the command. Choice arguments will be rendered as a drop down list in a Geneos GUI. *Parameters:* - `description`: (`nil`, [string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `options`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addRowTarget`(*view*[, *pattern*]) Adds a target to the command, for a row (or rows) of the specified view. *Parameters:* - `view`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4), `View`) - `pattern`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addColumnTarget`(*view*, *column*) Adds a target to the command, for a column of the specified view. *Parameters:* - `view`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4), `View`) - `column`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). `Definition:``addHeadlineTarget`(*view*, *headline*) Adds a target to the command, for a headline of the specified view. *Parameters:* - `view`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4), `View`) - `headline`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) Returns: - ([`Definition`](#geneos.sampler.commands.Definition "geneos.sampler.commands.Definition")) The definition object (i.e. `self`). ### geneos.sampler The sampler module controls basic integration with the Geneos framework. This module provides basic information about the sampler, and functions to interact with the Geneos framework. Current functionality allows publishing of dataviews and command interactions.
Functions Description
doSample() User-supplied function to perform sampling.
finalise() User-supplied function called when sampler is stopped.
logMessage() Writes a message to the Netprobe log.
createView() Creates a new dataview published to the Geneos gateway.
publishCommand() Publishes a command to Gateway for this sampler.
Tables Description
logLevel A table of constants for use with the logMessage() function.
params A table of parameters defined in the sampler configuration.
Fields Description
name The sampler name, as configured in the gateway setup file.
type The sampler type.
entityName The name of the managed entity the sampler is attached to.
sampleInterval The sample interval for the sampler.
Class View Description
View.name The dataview name.
View.headline The headline field is a table mapping headline names to values.
View.columns The columns field is a Lua array of column names.
View.row The row field contains a table of row data.
View:publish() Publishes the dataview to gateway.
View:refresh() Refreshes the contents of the view object.
#### Functions `geneos.sampler.``doSample`() User-supplied function to perform sampling. This function must be defined by the sampler script. It is called at the start of each sampling interval and whenever a user invokes the "Sample Now" command. In standalone test mode, if this function returns a boolean false value (literally `false`, not `nil`), the sampler (and Netprobe) will stop. When running under the control of a Gateway, the return value is ignored. *Returns:* - (optional `boolean`) See above. `geneos.sampler.``finalise`() User-supplied function called when sampler is stopped. This function may optionally be defined by the sampler script. It is called just before the sampler is stopped, whether because it is being removed from the configuration or because the Netprobe itself is stopping. *Usage:* ``` local gs = require "geneos.sampler" gs.finalise = function() gs.logMessage("INFO", "Stopping") end ``` `geneos.sampler.``logMessage`(*level*, *...*) Writes a message to the Netprobe log. Specify a logging level followed by a series of arguments to be concatenated. Concatenation will occur directly, without any added separator characters. *Parameters:* - `level`: - `...`: *Usage:* ``` -- The following are equivalent: local gs = require "geneos.sampler" gs.logMessage(40, "Hello ", "World!") gs.logMessage(gs.logLevel.INFO, "Hello ", "World!") gs.logMessage("INFO", "Hello ", "World!") ``` `geneos.sampler.``createView`(*name*, *columns*) Creates a new dataview published to the Geneos gateway. This function creates a new dataview for the sampler which can then be populated and published. The newly created view will contain no table rows, and a single headline variable `samplingStatus` with the (string) value `OK`. View names must be unique within a given sampler. Both view and column names should use only alphanumeric characters, underscore, dash or space (A-Za-z0-9\_- ). *Parameters:* - `name`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `columns`: ([table](http://www.lua.org/manual/5.1/manual.html#5.5)) *Returns:* 1. (optional [`View`](#geneos.sampler.View "geneos.sampler.View")) A view object representing the new view, or nil on failure. 2. (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure. One of the following: - "view name must be a non-empty string" - "view '$name' already exists" - "view defines empty column name at index $ind" - "view defines duplicate column name' $col'" *Usage:* ``` local gs = require 'geneos.sampler' local view = assert(gs.createView("newView", { "rowName", "dataColumn1", "dataColumn2" })) ``` `geneos.sampler.``publishCommand`(*name*, *execute*[, *cmdDef*]) Publishes a command to Gateway for this sampler. This function publishes a command associated with this sampler to the Gateway. From Gateway, the command will then become available to clients such as Active Console 2 or Open Access. The precise rendering of the command and its input fields for arguments will vary depending upon the client. When publishing, users should specify a short descriptive `name` for their command. This name must be non-empty string and should ideally contain only the characters a-z, A-Z, 0-9, \_ (underscore). - (dash) and space. The short name must be unique for each command, and will produce an error if not. Short names identify the different commands within a particular sampler. For use in the rest of Geneos a "full command name" is constructed, by prefixing the short name with the entity and sampler names (and also the sampler type if specified). The full command name will be returned by the function if the command is successfully published. Permissions or executions (e.g. from a rule or action) of commands can be applied on the Gateway as normal, using the full command name. When a command is executed the `execute` function will be called. This function is passed two arguments, a `target` table and an `args` array. The target identifies which data item (e.g. a dataview cell) the user clicked on to issue the command. Args is an array containing the command arguments supplied by the user (if the command specified any arguments). The `cmdDef` argument takes a command definition which specifies additional properties for the command, including arguments and targets. This definition can be provided either directly as a table, or via the fluent API available in the `geneos.sampler.commands` module. Arguments allow the command to prompt for users for input when the command is executed. Targets specify which data items the command will be available for (i.e. which items the command will show up for on a right-click). If no command definition is provided, the command will take no arguments and be available by right-clicking on any dataview published by the sampler. For a more complete discussion on these topics, please see the user guide section. *Parameters:* - `name`: ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) - `execute`: (`function`) - `cmdDef`: (optional `geneos.sampler.commands.Definition`) *Returns:* 1. (`nil`, [string](http://www.lua.org/manual/5.1/manual.html#5.4)) On success returns the full command name as a string. Returns `nil` on error. 2. ([string](http://www.lua.org/manual/5.1/manual.html#5.4)) On success returns `nil`. On error returns a message describing the problem. *Usage:* ``` local sampler = require 'geneos.sampler' local cmd = require 'geneos.sampler.commands' assert(sampler.publishCommand( 'aNewCommand', function(target, args) if target.headline then print('command executed on ' .. target.view .. ':' .. target.headline) elseif target.row and target.column then print('command executed on ' .. target.view .. ':' .. target.row .. '.' .. target.column) elseif target.view then print('command executed on ' .. target.view) end print('Message was ' .. args[1]) end, cmd.newDefinition() :setDescription('A new command') :addStringArgument('Message', 'A default message value') :addRowTarget('viewName', 'rowPattern\*') :addColumnTarget('otherView', 'aColumnName') :addHeadlineTarget('otherView', 'headlineName') )) -- or equivalently assert(sampler.publishCommand( 'another command', function(target, args) --[[ function body here ]] end, { description = 'A new command', args = { { type='string', description='Message', default='A default message value' } }, targets = { { type='row', view='myView', name='rowPattern\*' }, { type='column', view='otherView', name='aColumnName' }, { type='headline', view='otherView', name='headlineName' } } } )) ``` #### Tables `geneos.sampler.``logLevel` A table of constants for use with the `logMessage()` function. The name of the constant corresponds with the severity description output by the Netprobe. *Fields:* - `FATAL`: (`num`) - `ERROR`: (`num`) - `WARN`: (`num`) - `INFO`: (`num`) - `DEBUG`: (`num`) `geneos.sampler.``params` A table of parameters defined in the sampler configuration. The parameters are specified as key-value pairs, and are stored in the table as strings. ### Fields `geneos.sampler.``name` The sampler name, as configured in the gateway setup file. `geneos.sampler.``type` The sampler type. The name of the type the sampler was referenced from. This will be an empty string if the sampler was not referenced from a type. `geneos.sampler.``entityName` The name of the managed entity the sampler is attached to. `geneos.sampler.``sampleInterval` The sample interval for the sampler. This field may have a `nil` value if the sampler is configured to sample manually or at specific times. #### Class View *class*`geneos.sampler.``View` The view class is a Lua object representing a dataview. Views are returned by the [`geneos.sampler.createView()`](#geneos.sampler.createView "geneos.sampler.createView") function. `View.``name` The dataview name. A copy of the dataview name as specified in the call to [`geneos.sampler.createView()`](#geneos.sampler.createView "geneos.sampler.createView") which created this view. This field is provided for informational purposes; changing the value will not rename the view. `View.``headline` The headline field is a table mapping headline names to values. A new headline variable can be: - Created by adding a new entry to the table. - Updated by changing the value. - Deleted by setting the value to `nil`. **Note** the `samplingStatus` headline cannot be deleted, even if the table entry is set to `nil` (the value will just be cleared). If the headline table is not present in a call to [`View:publish()`](#geneos.sampler.View:publish "geneos.sampler.View:publish") then all headlines will be removed, aside from the `samplingStatus` headline whose value will be cleared. *Usage:* ``` local view = sampler.createView(name, cols) view.headline.newHeadline = "newValue" view.headline.update = 123 view.headline.removeMe = nil -- or equivalently view.headline = { newHeadline = "newValue", update = 123 } ``` `View.``columns` The columns field is a Lua array of column names. The first entry in the array is the row name column. Subsequent entries are value columns. This array is provided for information only - the columns of a dataview are fixed by the [`geneos.sampler.createView()`](#geneos.sampler.createView "geneos.sampler.createView") call. *Usage:* ``` view.columns = { "rowName", "column1", "column2" } ``` `View.``row` The row field contains a table of row data. A new table row can be: - Created by adding a new entry to the table. - Updated by changing the value. - Deleted by setting the value to `nil`. The value associated with a row must be a table, and can either be defined as an array or associative-map. If using an array, the values will be assigned to columns in the order they are defined. If using a mapping, the values are assigned to columns by name. If the row table is not present on calling ${view.publish}, all rows in the dataview will be removed. *Usage:* ``` local view = sampler.createView("newView", { "rowName", "column1", "column2" }) -- Array usage view.row.newRow = { 11, 22 } -- Equivalent mapping usage view.row.newRow = { ["column2"] = 22, ["column1"] = 11 } ``` `View:``publish`() Publishes the dataview to gateway. This function takes a single argument (the view) containing the data to be published. The contents of the [`View.row`](#geneos.sampler.View.row "geneos.sampler.View.row") and [`View.headline`](#geneos.sampler.View.headline "geneos.sampler.View.headline") tables will replace the current view contents and be transmitted onwards to the gateway. When called as a class method this function takes no additional arguments (i.e. using colon syntax `view:publish()` to publish). If necessary you may instead provide a different table containing view data by using the dot syntax `view.publish(dataTable)` to publish. *Returns:* 1. (`boolean`) true on success, false on failure. 2. (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure. One of the following: - "view '$name' no longer exists" - "expected table for first argument, got $type" *Usage:* ``` local gs = require 'geneos.sampler' local v = gs.createView('newView', { 'rowName', 'col_1', 'col_2' }) v.headline.total = 47 v.row.example = { 'col\_1 value', 'col\_2 value' } v:publish() ``` `View:``refresh`([*mode*]) Refreshes the contents of the view object. This function takes a single argument (the view) which will be populated with the currently published dataview state. When refreshing a view, the current contents of the [`View.name`](#geneos.sampler.View.name "geneos.sampler.View.name"), [`View.columns`](#geneos.sampler.View.columns "geneos.sampler.View.columns"), [`View.headline`](#geneos.sampler.View.headline "geneos.sampler.View.headline") and [`View.row`](#geneos.sampler.View.row "geneos.sampler.View.row") fields are replaced with new objects representing the current state. Any unpublished changes will therefore be lost. The optional mode argument defines the format of the row contents. When called as a class method this function takes no additional arguments (i.e. using colon syntax `view:refresh()` to refresh). It can be convenient to instead pass a different table to receive the data, by using the dot syntax `view.refresh(dataTable)` to refresh. *Parameters:* - `mode`: (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) *Returns:* 1. (`boolean`) true on success, false on failure. 2. (optional [string](http://www.lua.org/manual/5.1/manual.html#5.4)) An error message on failure. One of the following: - "view '$name' no longer exists" - "expected table for first argument, got $type" *Usage:* ``` local gs = require 'geneos.sampler' local v = gs.createView('newView', { 'RowName', 'Column' }) -- -- ... v.headline, v.row = nil, nil v:refresh() assert(v.headline ~= nil) assert(v.row ~= nil) ``` ## Feed adapter API reference This section contains reference documentation for feed adapter library APIs in several different programming languages. For a more structured description of creating feed adapter using these APIs, please see the [Write a feed adapter](https://docs.itrsgroup.com/docs/geneos/7.11.0/collection/mdm-writing-samplers/index.html#write-a-feed-adapter) section. ### C template This template contains a reference implementation of the [get\_error\_text()](#size_t_get_error_text) method, and implementations of the other API functions. ``` /* template.c - Basic template for a feed adapter library */ #include #include #include #include #include "api/FeedAdapterCApi.h" #define MAX_ERROR_LEN 8192 /* 8kB */ const char* get_version() { return "theVersion"; } static char last_error[MAX_ERROR_LEN]; void set_error_text(const char* format, ...) { /* set error messages with printf-style formatting */ va_list vl; va_start(vl, format); vsnprintf(last_error, MAX_ERROR_LEN - 1, format, vl); last_error[MAX_ERROR_LEN - 1] = '\0'; va_end(vl); } size_t get_error_text(size_t size, char* msg_buf) { const size_t req_len = strlen(last_error) + 1; if (size >= req_len) { /* We have enough space for the full message, including NUL terminator. */ memcpy(msg_buf, last_error, req_len); return 0; } else if (size > 0) { /* Not enough space, copy as much as we can. */ memcpy(msg_buf, last_error, size - 1); msg_buf[size - 1] = '\0'; } return req_len; } /* Global function pointers passed from Netprobe when library is initialised. */ struct Functions global_feedapi; int initialise_library(struct Functions funcs) { global_feedapi = funcs; last_error[0] = '\0'; /* TODO any other required initialisation. */ return 1; } void finalise_library() { /* TODO any other required finalisation. */ } int create_feed(feed_id_t id, size_t num_params, const char** keys, const char** values) { set_error_text("TODO implement create_feed"); return 0; } int destroy_feed(feed_id_t feed) { set_error_text("TODO implement destroy_feed"); return 0; } size_t get_feed_status(feed_id_t feed, size_t size, char* msg_buf) { if (size >= 5) { strcpy(msg_buf, "TODO"); return 0; } return 5; } int subscribe_instrument(feed_id_t feed, inst_sub_t subscription, const char* instrument, size_t num_fields, const char** field_names) { set_error_text("TODO implement subscribe_instrument"); return 0; } int unsubscribe_instrument(feed_id_t feed, inst_sub_t subscription, const char* instrument) { set_error_text("TODO implement unsubscribe_instrument"); return 0; } ``` ### Feed adapter C API #### Typedefs ``` feed\_id\_t typedef FeedID feed\_id\_t inst\_sub\_t typedef instSubscription inst\_sub\_t ``` #### Enums `MdmLogLevel` Logging levels for use with the log\_message() callback method. - Mdm\_Fatal = 10 - Mdm\_Error = 20 - Mdm\_Warning = 30 - Mdm\_Info = 40 - Mdm\_Debug = 50 #### Functions **const char\* get\_version()** Gets the version of the feed adapter shared-object. For Geneos feeds, this should match the Netprobe version the feed will be used with. e.g. “GA3.0.6-130315”. For custom feeds, the version is displayed in the Netprobe log for information purposes. *Returns:* The feed version. **int initialise\_library(struct [Functions](#FeedAdapterC_Function) funcs)** This function is called immediately after successfully loading the shared-object. Users should perform any required global setup in this method. *Parameters:* - funcs: Callback functions for data output. *Returns:* Non-zero (true) if initialisation was successful, zero (false) otherwise. **void finalise\_library()** This function is called immediately before the shared object is unloaded. The library should use this to perform all necessary cleanup. int create\_feed([feed_id_t](#feed_id_t) feed, size_t num_params, const char \*\*keys, const char \*\*values) Creates a new feed. The feed identifier is a unique id which will only refer to the given feed for its lifetime. Parameters are passed as name-value pairs. For an index 0 <= ix < num\_params, ‘values[ix]’ is the value for the corresponding key ‘keys[ix]’. *Parameters:* - feed: The identifier for this feed. - num\_params: The number of parameters passed (i.e. the size of ‘keys’ and ‘values’). - keys: A list of parameter names. - values: A list of parameter values. A value at offset N corresponds to the key at the same offset N. *Returns:* Non-zero (true) if the feed was successfully created, or zero (false) otherwise. **int destroy\_feed([feed_id_t](#feed_id_t) feed)** Destroys a previously created feed. On destruction, the feed should stop publishing any subscribed instruments. Following destruction, the feed identifier becomes available again and may eventually be reused in a new ‘create\_feed’ call. *Parameters:* - feed: The identifier of the feed to be destroyed, as returned by a prior call of ‘create\_feed’. *Returns:* Non-zero (true) if the feed was successfully destroyed, zero (false) otherwise. **size\_t get\_feed\_status([feed_id_t](#feed_id_t) feed, size_t size, char \*msg_buf)** Obtains a short description of the feed status, which should be suitable for display in a dataview. A correctly operating feed is recommended to return a status of “OK”. *Parameters:* - feed: The feed to describe. - size: Size of the message buffer passed to this method. - msg\_buf: Buffer to populate with the message. *Returns:* Returns 0 if the call was successful. Otherwise returns the minimum buffer size to receive the full null-terminated text. **int subscribe\_instrument([feed_id_t](#feed_id_t) feed, inst_sub_t subscription, const char \*instrument, size_t num_fields, const char \*\*field_names)** Subscribes to an instrument for a particular feed. The ‘inst\_sub\_t’ subscription record is an opaque pointer that must be provided for every publish callback. *Parameters:* - feed: Feed identifier as used in a prior call of ‘create\_feed’. - subscription: The (unique) subscription record for this feed/instrument. - instrument: Name of the instrument to subscribe to. - num\_fields: Length of the field names array. - field\_names: The field names requested for this instrument. Note that the order of fields in a ‘publish\_image’ or ‘publish\_tick’ call should match the order of field names specified in this call. *Returns:* Non-zero (true) if the subscription was successful, zero (false) otherwise. **int unsubscribe\_instrument(feed_id_t feed, inst_sub_t subscription, const char \*instrument)** Unsubscribes from an instrument for a particular feed. Upon completion of this call, the ‘inst\_sub\_t’ subscription record will become invalid, and no further calls to publish data should be made for the subscription. *Parameters:* - feed: Feed identifier, as returned by a prior call of ‘create\_feed’. - subscription: The (unique) subscriber for the feed/instrument. - instrument: Name of the instrument being removed. *Returns:* Non-zero (true) if the instrument was successfully unsubscribed, zero (false) otherwise. **size\_t get\_error\_text(size_t size, char \*msg_buf)** Obtains a description of the latest feed-API error to occur. This method will be called after every failed API call, for example if ‘create\_feed’ returns failure. The description of the failure will then be logged as appropriate. If no error has occurred an empty string should be returned. *Parameters:* - size: Size of the message buffer passed to this method. - msg\_buf: Buffer to populate with the message. *Returns:* Returns 0 if the call was successful. Otherwise returns the minimum buffer size to receive the full null-terminated error text. #### Classes **FeedId** A feed identifier. *Public Members:* *int **id*** The feed identifier number. **InstSubscription** An instrument subscription record. *Public Members:* *void \***data*** The associated MDM data for this subscription. **TickField** Structure representing a single field value from a tick. *Public Members:* *int **updated*** Was this field updated for this tick? Non-zero (true) if updated, zero (false) if unchanged. *enum TickField::DataType* **data\_type** The data type for this field. *double **dbl\_value*** Numeric double-precision floating-point value. *const char **\*str\_value*** String value. *Public Types:* **DataType** The data type for this field. - DT\_DOUBLE - DT\_STRING #### Functions Structure of callback functions provided by MDM to the feed adapter shared-object. Public Members: *void**(\*log_message)**(enum MdmLogLevel level, const char \*module, const char \*message)* Logs a message to the Netprobe log. *Parameters*: - level: The logging level. This code is translated and appears as descriptive text in the log output. - module: Name of the module writing the log message. - message: Message to log. *void**(\*publish_image)**(feed_id_t feed, inst_sub_t subscription, int utc_secs, int usecs, size_t num_fields, const struct TickField \*fields)* Publishes the image for an instrument. An image (or snapshot) is an update which contains values for all (requested) fields of an instrument. An image must be published before publishing any ticks (updates) for the same instrument. When publishing an image, all provided fields are examined and their value extracted. *Parameters*: - feed: Index of the feed, as provided in a previous ‘create\_feed’ call. - sub: The instrument subscription, as provided in a previous ‘subscribe\_instrument’ call. - utc\_secs: Timestamp of the image as number of seconds from Unix epoch. - usecs: Microseconds offset from ‘utc\_sec’ value. - num\_fields: Number of fields provided in ‘fields’ array. - fields: Field values from the image **void*(\*publish_tick)*(feed\_id\_t feed, inst\_sub\_t subscription, int utc\_secs, int usecs, size\_t num\_fields, const struct TickField \*fields** Publishes a tick (or update) for an instrument. When publishing a tick, only fields marked as “updated” will have their value extracted. An image must previously have been published for the same instrument. Otherwise, the tick updates will be ignored. ### Feed adapter C++ API #### Main API *type* geneos::mdm::feedapi::StringMap typedef std::map< std::string, std::string > StringMap *type* geneos::mdm::feedapi::FieldNameList typedef std::vector< std::string > FieldNameList *class* geneos::mdm::feedapi::FeedFactory The [FeedFactory](#class_geneos::mdm::feedapi::FeedFactor) is used by the [FeedManager](#class_geneos::mdm::feedapi::FeedManager) to create concrete instances of the Feed abstract class. **Public Functions:** `virtual ~FeedFactory()` Destructor. Feed\* createFeed (const StringMap& *params*, Functions *funcs*, feed_id_t *id*) const = 0 Creates a new [Feed](#class_geneos::mdm::feedapi::Feed) instance in response to a create\_feed API call. *Parameters:* - params: Feed-specific parameters. - funcs: API callback functions. - id: The Netprobe generated identifier for this feed instance. *Returns:* A new dynamically allocated feed instance. The caller will take ownership of memory for the object. ##### **class geneos::mdm::feedapi::FeedManager** The FeedManager class manages feed instances hosted with the feed adapter shared library. These feeds are created (and destroyed) in response to C API calls made by the Netprobe. Other calls (instrument subscriptions and status requests) are redirected to the appropriate feed instance as specified by the feed identifier. Public Functions: *void **setErrorMessage**(const std::string& msg)* Sets the text that the next call to ‘get\_error\_text’ will return. *Parameters*: msg: The error message to set. *bool **initialise\_callback\_funcs**(Functions funcs)* Initialises the FeedManager with the C API callback functions. This method should be called once immediately after the library is loaded, before ‘create\_feed’ is called. *Parameters*: funcs: C API callback functions. *Returns*: True on success, otherwise false. *bool **create\_feed**(feed_id_t feed, size_t num_params, const char\*\* keys, const char\*\* values, const FeedFactory& ff)* Creates a new Feed instance. *Parameters*: - feed: Identifier of the new feed. - num\_params: Count of parameters passed. - keys: Parameter keys (names), ‘num\_params’ in length. - values: Parameter values, ‘num\_params’ in length. - ff: Factory object to create Feed instances. *Returns*: True on success, false on error. *bool **destroy\_feed**(feed_id_t feed)* Stops and destroys an existing Feed instance. *Parameters*: feed: Identifies the feed to destroy. Returns: True on success, false on error. *size\_t **get\_feed\_status**(feed_id_t feed, size_t size, char\* msg_buf)* Obtains a short description of the feed status, which should be suitable for display in a dataview. *Parameters*: - feed: The feed to describe. - size: Size of the message buffer passed to this method. - msg\_buf: Buffer to populate with the message. *Returns*: Returns 0 if the call was successful. Otherwise returns the minimum buffer size required to receive the full null-terminated text. *bool **subscribe\_instrument(**feed_id_t feed, inst_sub_t sub, const char\* instrument, size_t num_fields, const char\*\* field_names)* Subscribes a feed to a new instrument. *Parameters*: - feed: The feed to subscribe on. - sub: Instrument subscription record for this request. - instrument: Name of the instrument (stock code or symbol) to subscribe to. - num\_fields: Count of entries in field\_names. - field\_names: List of field names for this instrument, ‘num\_fields’ in length. *Returns*: True on success, false on error. *bool u**nsubscribe\_instrument(**feed_id_t feed, inst_sub_t sub, const char\* instrument)* Unsubscribes an instrument for a feed. *Parameters*: - feed: The feed to unsubscribe for. - sub: Record of the subscription to remove. - instrument: Name of the instrument being removed. *Returns*: True on success, false on error. *size\_t **get\_error\_text**(size_t size, char\* msg_buf) const* Returns text describing the last error from an API call. *Parameters*: - size: Size of buffer msg\_buf passed in. - msg\_buf: Storage for message to be written to. *Returns*: Returns 0 if the call was successful. Otherwise returns the minimum buffer size to receive the full null-terminated text. *void **log\_message**(MdmLogLevel level, const char\* module, const char\* message)* Writes the given message to the Netprobe via the C API logging function callback. *Parameters*: - level: Geneos logging level code. See enum LOG\_LEVEL in FeedAdapterCApi.h for valid input. - module: The module the log message is coming from. - message: Message to log. ##### class geneos::mdm::feedapi::Feed The feed class provides basic functionality expected from feeds, upon which a full implementation can be created. Each feed instance corresponds to a unique feed id, as provided by the Netprobe during a call to the Feed Adapter C API ‘create\_feed’ call. Implementors should extend this class and implement the pure-virtual methods with their own specific functionality. Please see the ExampleFeed class for an example implementation. **Public Functions**: ***Feed**(const StringMap& params, Functions callbacks, feed_id_t id)* Constructor. *Parameters*: - params: The configuration parameters for this feed definition. - callbacks: Callback functions used to publish output from this feed. - id: Unique feed identifier for this feed. *virtual **~Feed**()* Destructor. *bool **start**()* Instructs the feed to start and begin providing data. *Returns*: True if the feed started successfully, false on error. *bool **stop**()* Instructs the feed to stop and cease providing data. *Returns*: True if the feed stopped successfully, false on error. *bool **subscribe**(inst_sub_t subscription, const std::string& instrument, const FieldNameList& fieldList, std::string& errorMsg)* Instructs the feed to subscribe to the named instrument. *Parameters*: - subscription: A unique record for this subscription request, required for publishing. - instrument: Symbolic name (or code) of the instrument to subscribe to. - fieldList: Set of fields names of interest for this subscription. - errorMsg: This variable will be populated with a message on error. *Returns*: True on success, false on error. *bool **unsubscribe**(inst_sub_t subscription, const std::string& instrument, std::string& errorMsg)* Instructs the feed to unsubscribe from the named instrument. *Parameters*: - subscription: Record of the subscription to remove. - instrument: Symbolic name (or code) of the instrument. - errorMsg: This variable will be populated with a message on error. *Returns*: True if the instrument was successfullly unsubscribed, false otherwise. *bool **hasParameter**(const std::string& name) const* Checks whether a configuration parameter has been defined. *Parameters*: name: Name of the parameter to search for. *Returns*: True if parameter has been defined, false otherwise. *std::string **getParameter**(const std::string& name) const* Gets the value of a configuration parameter. *Parameters*: name: Name of the parameter to obtain. *Returns*: The parameter value if defined, or an empty string otherwise. *StringMap **getParameters**() const* Gets a mapping containing all configuration parameter name-value pairs. *Returns*: The configuration parameters for the feed. *std::string **getName**() const* Gets the name of this feed, as specified by the ‘feed.name’ parameter. *Returns*: The feed name. *bool **getVerbose**() const* Gets whether this feed should produce verbose output (typically used to help debugging). This flag is controlled by the ‘feed.verbose’ parameter. *Returns*: True for verbose output, false otherwise. *bool **getRawLogging**() const* Gets whether this feed should log raw message data (for debugging purposes). This flag is controlled by the `feed.rawlog` parameter. *Returns*: True to log raw message data, false otherwise. *bool **getAllowDuplicateTicks**() const* Gets whether this feed should allow publishing of duplicate tick data (ticks where no field values have changed). This flag is controlled by the ‘feed.allowDuplicateTicks’ parameter (which is false by default). *Returns*: True if allowing duplicates, false otherwise. *void **setAllowDuplicateTicks**(bool allow)* Sets whether the feed should allow publishing of duplicate tick data. Calls to this method will override the value read from the ‘feed.allowDuplicateTicks’ parameter value. *Parameters*: allow: True to allow duplicates, false to prevent publishing. *std::string **getStatus**() const* Obtains a short status description for the feed. *Returns*: The feed status. **Protected Functions:** *void **setStatus**(const std::string& status)* Sets the status message of the feed. *Parameters*: status: New status to set. *void **log**(MdmLogLevel level, const std::string& message)* Outputs a message to the Netprobe log file. *Parameters*: - level: Severity level for this message. - message: Message to be logged. *void **logRaw**(const char\* data, size_t dataLength)* Outputs a message to the raw log file, if raw logging is enabled. This method is intended for recording raw message data for debugging purposes. *Parameters*: - data: Buffer containing the message data to be logged. - dataLength: Length of the data buffer. *void **publishImage**(InstrumentInfo& info)* Publish an image for the specified Instrument. *Parameters*: info: The instrument to publish, populated with the image information. *void **publishUpdate**(InstrumentInfo& info)* Publish an update (tick) for the specified Instrument. *Parameters*: info: The instrument to publish, populated with the update information. **Private Functions:** *bool **doValidateConfig**() = 0* Validate the configuration parameters for this feed. *Returns*: True if the configuration is valid, false otherwise. *bool **doStart()** = 0* Connect to a data source and begin publishing. This method is expected to start a feed and then return. If the feed handler API you are interfacing with is not asynchronous, you may need to start a new thread. *Returns*: True on successful startup, false otherwise. *bool **doStop**() = 0* Stop publishing and disconnect from the data source. Any threads created by doStart() should be stopped here. *Returns*: True on successful shutdown, false otherwise. *bool **doSubscribe**(InstrumentInfo& info, std::string& error) = 0* Subscribe to the instrument as defined by the ‘info’ argument. *Parameters*: - info: Subscription information for the instrument. - error: Populate this variable with an error message, if the call fails. *Returns*: True on successful subscription, false otherwise. *bool **doUnsubscribe**(InstrumentInfo& info, std::string& error) = 0* Unsubscribe to the instrument defined by the ‘info’ argument. *Parameters*: - info: Subscription information for the instrument to remove. - error: Populate this variable with an error message, if the call fails. *Returns*: True on successful unsubscription, false otherwise. ##### **class geneos::mdm::feedapi::InstrumentInfo** The instrument information class stores field values for a specific instrument subscription. This data is stored in a format to allow publishing back to the Netprobe, and the class provides several helper methods to allow values to be easily populated by a feed. **Public Functions**: ***InstrumentInfo** (const std::string& instrument, const inst_sub_t sub, const FieldNameList& fieldList)* Constructor. *Parameters*: - instrument: Name of the subscribed instrument. - sub: Unique identifer for this instrument subscription. - fieldList: List of fields names of interested for this instrument. **~InstrumentInfo**() Destructor. *bool **operator==(**const InstrumentInfo& other) const* Equality operator. Checks whether the instrument information (but not the field values) are the same. If so, then the info object is referring to the same instrument. *Parameters*: other: The object to compare against. *Returns*: True if the info objects are referring to the same instrument, false otherwise. *bool **operator!=(**const InstrumentInfo& other) const* Inequality operator. Checks whether the instrument information (ignoring field values) are different. *Parameters*: other: The object to compare against. Returns: True if the info objects refer to different instruments, false otherwise. *const std::string& **getInstrument**() const* Gets the name of the subscribed instrument. *Returns*: The instrument name. *const inst\_sub\_t& **getSubscriptionRecord**() const* Gets the unique (Netprobe-generated) subscription record for this instrument. *Returns*: The subscription record. *const FieldNameList& **getFieldNames**() const* Gets the ordered list of field names the Netprobe is interested in for this instrument. *Returns*: A list of field names. *bool **isFieldOfInterest**(const std::string& fieldName) const* Gets whether the specified field name is of interest for this instrument. *Parameters*: fieldName: Name of the field to test. *Returns*: True if the field is of interest, false otherwise. *void **newUpdate**(const timeval\* tv=NULL)* Prepares the info object to receive a new update. All fields are set to an unchanged (not updated) state. The timestamp of the change is set to the specified time (the current system time by default). This method should be called when a market data update has been received, before any field values are updated. *Parameters*: tv: Timestamp of the update. If NULL, the current system time will be used as the timestamp. *void **setFieldValue**(size_t index, double value)* Sets the value for the field identified by index to the specified numeric value. Does nothing if index >= getFieldNames().size() *Parameters*: - index: Index of the field to set within the list returned by getFieldNames. - value: The field value to set. *void **setFieldValue**(const std::string& field, double value)* Sets the value for the named field to the specified numeric value. *Parameters*: - field: Name of the field to set. - value: The field value to set. *void **setFieldValue**(size_t index, const std::string& value)* Sets the value for the field identified by index to the specified numeric value. Does nothing if index >= getFieldNames().size() *Parameters*: - index: Index of the field to set within the list returned by getFieldNames. - value: The field value to set. *void **setFieldValue**(const std::string& field, const std::string& value)* Sets the value of the named field to the specified string value. *Parameters*: - field: Name of the field to set. - value: The field value to set. *void **setPartialFieldValue**(const std::string& field, size_t offset, const std::string& partValue)* Updates part of a string-valued named field with new string data, overwriting any previous data. The update range will be from [offset, offset + data.size) in the original field value string. Data outside of the update range will not be altered. If the field value is not currently a string, the update will be ignored. *Parameters*: - field: Name of the field to update. - offset: The 0-based index into the original string data where the update begins. A value of 0 will overwrite from the first character onwards. - partValue: The replacement data to write into the string. The size (or length) of the data indicates the extent of the original value to overwrite. If offset + size is longer than the original value length, the value will be extended to fix the new data. *void **getRecordedTime**(int& utc_secs, int& usecs) const* Gets the timestamp of the update for this instrument. *Parameters*: - utc\_secs: Populated with the timestamp in UTC seconds (seconds since Unix Epoch). - usecs: Populated with the microsecond component of the timestamp. *size\_t **getFieldCount**() const* Gets the number of field values stored for this subscription. *Returns*: The number of fields. *const TickField\* **getFieldValues**() const* Gets the field values for this subscription, as a contiguous array of TickField structures. *Returns*: Pointer to the instrument field value array. *bool **hasFieldChanges**() const* Gets whether the instrument contains any field value changes. *Returns*: True for changes, false for none. ##### class geneos::mdm::feedapi::TimeRecorder The TimeRecorder class provides a common way to obtain the current system timestamp across both Unix and Windows platforms. This class mostly acts as a wrapper for the “gettimeofday” system call. **Public Functions**: ***TimeRecorder**()* Constructor. Obtains the current time on creation. *void **recordTimeNow**()* Sets the recoded time to be the current system time. *void **setRecordedTime**(const timeval& tv)* Sets the recorded time to the specified value. *Parameters*: tv: The time value to set. *const timeval& **getRecordedTime**() const* Obtains the recorded timestamp stored by this instance. *Returns*: The recorded timestamp. *void **getRecordedTime**(int& utc_secs, int& usecs) const* Gets the recorded timestamp stored by this instance, as *Parameters*: - utc\_secs: Populated with the number of UTC seconds (i.e. the POSIX/Unix timestamp) - usecs: Populated with the number of microseconds after utc\_secs ##### Helper classes ##### **class geneos::mdm::feedapi::Mutex** The Mutex class implements a mutual exclusion lock. The inner Lock class is an RAII class to allow for automatic unlocking of the mutex when the lock goes out of scope. **Note** This mutex is non-reentrant. **Public Functions**: **Mutex***()* Constructor. ***~Mutex**()* Destructor. *void **lock**()* Locks the mutex. *void **unlock**()* Unlocks the mutex. *class **Lock*** RAII class for lock acquisition. **Public Functions**: ***Lock**(Mutex& mtx)* Constructor. Locks the mutex on construction. Usage example: ``` // ... unprotected code { Mutex::Lock lock(theMutex); // ... mutex protected code } // 'theMutex' unlocked at end of enclosing scope ``` *Parameters*: mtx: The mutex to lock. ***~Lock**()* Destructor. Unlocks the mutex. *Parameters*: mtx: The mutex to lock. ***~Lock**()* Destructor. Unlocks the mutex. ##### **class geneos::mdm::feedapi::Runnable** The Runnable interface should be implemented by classes which will be executed in a thread. **Public Functions**: *virtual **~Runnable**()* Destructor. *void **run**() = 0* Invoked from a thread following a call to Thread::start(). *void **stop**() = 0* Invoked when Thread::stop() is called. **class geneos::mdm::feedapi::Thread** The Thread class implements a simple C++ wrapping over the underlying OS thread details. Only basic start and stop functionality is provided. More advanced control such as detaching, cancelling or obtaining status information of the thread are omitted. **Public Functions**: ***Thread**(Runnable& runnable, const std::string& name)* Constructor. Creates a thread which will execute the provided runnable. *Parameters*: - runnable: The runnable object to execute. This object must remain valid for the lifetime of the thread. - name: Optional thread name. ***~Thread**()* Destructor. Will terminate (and wait for) the thread if it was not already stopped. *const std::string& **getName**() const* Gets the thread name. *Returns*: Name of the thread. *int **start**()* Starts execution of the runnable in a thread. As the Thread class cannot be reused, start() should only be called once. If called multiple times, the additional calls will return EAGAIN. *Returns*: 0 on success, or a system error code on failure. *void **stop**()* Signals the runnable to stop executing, and waits for the thread to terminate. ##### Example feed classes **class Example::ExampleFeedFactory** The ExampleFeedFactory creates ExampleFeed instances on behalf of the FeedManager. **Public Functions**: *geneos::mdm::feedapi::Feed\* **createFeed**(const geneos::mdm::feedapi::StringMap& params, Functions funcs, feed_id_t id) const* Creates a new Feed instance in response to a create\_feed API call. *Parameters*: - params: Feed-specific parameters. - funcs: API callback functions. - id: The Netprobe-generated identifier for this feed instance. *Returns*: A new dynamically allocated feed instance. The caller will take ownership of memory for the object. *class Example::**ExampleFeed*** The ExampleFeed class is a concrete implementation of the Feed abstract class. This class provides a simple demonstration of using the Feed Adapter C++ API. Feeds configured for the resulting adapter shared-object will print their parameters to the Netprobe log, and publish an incrementing value for each field of every requested instrument. This feed takes a single parameter; new instrument values will be published every “example.publishingPeriod” milliseconds. **Public Functions**: ***ExampleFeed**(const geneos::mdm::feedapi::StringMap& params, Functions funcs, feed_id_t id)* Constructor. The arguments are required to construct the Feed base class. *Parameters*: - params: Configuration parameters for this Feed. - funcs: Feed Adapter C API callback functions for publishing. - id: Netprobe-generated feed identifier. *virtual **~ExampleFeed**()* Destructor. **Private Functions**: *bool **doValidateConfig**()* Validate the configuration parameters for this feed. *Returns*: True if the configuration is valid, false otherwise. *bool **doStart**()* Connect to a data source and begin publishing. This method is expected to start a feed and then return. If the feed handler API you are interfacing with is not asynchronous, you may need to start a new thread. *Returns*: True on successful startup, false otherwise. *bool **doStop**()* Stop publishing and disconnect from the data source. Any threads created by doStart() should be stopped here. *Returns*: True on successful shutdown, false otherwise. *bool **doSubscribe**(geneos::mdm::feedapi::InstrumentInfo& info, std::string& error)* Subscribe to the instrument as defined by the ‘info’ argument. *Parameters*: - info: Subscription information for the instrument. - error: Populate this variable with an error message, if the call fails. *Returns*: True on successful subscription, false otherwise. *bool **doUnsubscribe**(geneos::mdm::feedapi::InstrumentInfo& info, std::string& error)* Unsubscribe to the instrument defined by the ‘info’ argument. *Parameters*: - info: Subscription information for the instrument to remove. - error: Populate this variable with an error message, if the call fails. *Returns*: True on successful unsubscription, false otherwise. *class Example::**ExamplePublisher*** The ExamplePublisher class implements threaded instrument publishing for the ExampleFeed class. The publisher is a Runnable that will be executed in a thread. Every publishing period, all currently subscribed instruments will be updated with new field values which are then published to the Netprobe. Values for fields will be incrementing numeric values except for the special field names ‘DateTime’ and ‘Partial’ which contain string values. **Public Functions**: ***ExamplePublisher**(ExampleFeed& feed, size_t period)* Constructor. *Parameters*: - feed: The feed to publish to. - period: The publishing period in milliseconds. *virtual **~ExamplePublisher**()* Destructor. *void **addInstrument**(geneos::mdm::feedapi::InstrumentInfo& info)* Adds an instrument to the local instrument map. *Parameters*: info: The instrument info to add. *void **removeInstrument**(geneos::mdm::feedapi::InstrumentInfo& info)* Removes an instrument from the local instrument map. *Parameters*: info: The instrument info to remove. *void **run**()* Invoked from a thread following a call to Thread::start(). *void **stop**()* Invoked when Thread::stop() is called. ### Feed adapter lua input API The feed module is loaded automatically by the ITRS Lua feed adapter shared-object. This module allows the Lua input script to receive Netprobe requests (in the form of events) and publish back instrument tick data. #### Functions *feed.**gettimeofday**([mode])* Obtains the current time of day to microsecond resolution. This function is a Lua-callable equivalent of the gettimeofday UNIX system call. Users may call this to obtain a time to microsecond resolution, as a replacement for the os.time function. The optional mode argument specifies how the time should be returned: - "r": As multiple return values, the first as a number of seconds and the second the microseconds. - "n": As a single floating-point number, with the fractional part for microseconds. *Parameters*: mode: (optional string) The mode to return output in. The default is mode “n”, as a single number. *Returns*: - (number) The current time in UTC seconds (seconds since the UNIX Epoch). This number may have a floating-point fractional component, depending upon the mode. - (optional number) The microsecond component of the current time in “r” mode, or nil otherwise. *Usage*: ``` -- Multiple return values, stored into a table local time = {} time.sec, time.usec = feed.getTimeOfDay("r") -- A single return valuelocal time = feed.getTimeOfDay() -- or time = feed.getTimeOfDay("n") ``` *feed.**sleep**(interval)* Suspends the caller for the specified time interval. *Parameters*: interval: (number) Specifies a wait time in seconds. Negative values will not cause suspension. Floating-point values may be used to specify fractional wait times, so a value of 0.5 means to sleep for half a second. *feed.**getEvent**([waitDuration])* Checks for events from the Netprobe, waiting up to the specified duration for an event to occur. This function checks for events from the Netprobe, and returns the first event if any have occurred since the last check. Otherwise, the call will wait up to the specified duration for a new event. If an event occurs while waiting, the function will return the event immediately. *Parameters*: waitDuration: (number) Optionally specifies a (non-negative) wait time in seconds. Floating-point values may be used to specify fractional wait times, so a value of 0.5 means a half second wait duration. If the argument is not specified or invalid, the function will not wait and instead perform a check and return immediately. *Returns*: (nil, terminate, subscribe, unsubscribe) If no events are available the value nil will be returned. If an event is present, the oldest event object will be returned. Events are represented by tables, and contain a type field specifying the event type by name. *Usage*: ``` local evt = feed.getEvent(timeout) while evt do processEvent(evt) -- handle the event evt = feed.getEvent() -- check for any new events (no timeout, so returns immediately) end ``` *feed.**logMessage**(level, ...)* Writes a message to the Netprobe log. Specify a logging level followed by a series of arguments to be concatenated. Concatenation will occur directly, without any added separator characters. *Parameters*: - level: The logging severity level for the message. This should be one of the levels defined in the logLevel table. - ...: All arguments following the level will be treated as strings and concatenated before being written to the Netprobe log. *Usage*: ``` -- The following are equivalent: feed.logMessage(40, "Hello ", "World!") feed.logMessage(feed.logLevel.INFO, "Hello ", "World!") feed.logMessage("INFO", "Hello ", "World!") ``` *feed.**publish**(tick)* Publishes a tick to the Netprobe. The publish function publishes an instrument update (or tick) to the Netprobe for processing. The tick must specify a valid instrument name, and at least one (requested) field value. If the time field is omitted, the tick will be published using the current time obtained from gettimeofday(). *Parameters*: tick: (tick) The tick to publish. *Usage*: ``` local tick = { instrument = "inst name", time = 1379409456.763708, -- alternatively: time = { sec=1379409456, usec=763708 } fields = { FieldName1 = 200.4, FieldName2 = "str" } } feed.publish(tick) ``` *feed.**setStatus**(status)* Sets the feed status. This function takes a single argument, which is the status message to set for the feed. The message should be a short status summary suitable for display in a dataview. This function is the Lua equivalent of the Feed::setStatus C++ API method. *Parameters*: status: (string) The status message to set. #### Tables *feed.feed.**params*** Feed parameters. The parameters table is populated by the Lua shared-object from Netprobe configuration. Keys and values in the table are strings, for example "value.max" and "1000". *feed.**logLevel*** A table of constants for use with the logMessage() function. The name of the constant corresponds with the severity description output by the Netprobe. *Fields*: - FATAL: (num) 10 - ERROR: (num) 20 - WARN: (num) 30 - INFO: (num) 40 - DEBUG: (num) 50 *feed.**tick*** A tick (market data update) to publish to Netprobe. *Fields*: - time: (nil, number, table) The time the tick was received. This field value may either be omitted (i.e. nil) and a time will be generated, a number (as returned by gettimeofday()), or a table containing sec and usec fields. - fields: (table) A table of tick field names and field values for this update. Values should be of type string or number. - instrument: (string) The name or stock-code of the instrument being published. *Usage*: ``` local tick = { instrument = "MY.CODE.N", time = 1379409456.763708, fields = { ASK=128.75, BID=128.50, COMMENT="A string value" } ``` *feed.**terminate*** A termination event is received from the Netprobe when the feed should terminate. *Fields*: type: (string) The event type, containing the text “terminate”. *Usage*: ``` local terminateEvent = { type = "terminate" } ``` *feed.**subscribe*** A subscription event is received from the Netprobe to add an instrument subcription. *Fields*: - type: (string) The event type, containing the text “subscribe”. - instrument: (string) The name or stock-code of the instrument to subscribe to. - fields: (table) A table enumerating the fields we are interested in for this instrument by name. The field names are specified as keys in the table. *Usage*: ``` local subscribeEvent = { type = "subscribe", instrument = "MY.CODE.N", fields = { ASK="", BID="", TRDPRC_1="" } } ``` *feed.**unsubscribe*** An unsubscription event is received from the Netprobe to remove an instrument subscription. *Fields*: - type: (string) The event type, containing the text “unsubscribe”. - instrument: (string) The name or stock-code of the instrument to unsubscribe from. *Usage*: ``` local unsubscribeEvent = { type = "unsubscribe", instrument = "MY.CODE.N" } ```
Feed function Description
gettimeofday() Obtains the current time of day to microsecond resolution.
sleep() Suspends the caller for the specified time interval.
getEvent() Checks for events from the Netprobe, waiting up to the specified duration for an event to occur.
logMessage() Writes a message to the Netprobe log.
publish() Publishes a tick to the Netprobe.
setStatus() Sets the feed status.
Table Description
params Feed parameters.
loglevel A table of constants for use with the logMessage() function.
tick A tick (market data update) to publish to Netprobe.
terminate A termination event is received from the Netprobe when the feed should terminate.
subscribe A subscription event is received from the Netprobe to add an instrument subcription.
unsubscribe An unsubscription event is received from the Netprobe to remove an instrument subscription.