Kafka publishing message formats
Overview
This topic outlines the formats of the messages sent from Gateway Hub to the downstream Kafka instance.
Messages are sent in a JSON format and contain normalised metric and event data.
Message payloads - metrics
Type |
Payload |
Comments |
---|---|---|
Headline schema | { "version":"1.0", "schema":{ "schemaId":1042, "type":"headline", "properties":[ { "name":"samplingStatus", "type":"string" }, { "name":"execDuration", "type":"float32", "unit":"seconds" }, { "name":"numZones", "type":"int32", "enriched":true } ] }, "operation":"schema" } |
This is a combined schema. Both raw and enriched headlines (if present) are included. Normalisation: In this example, |
Table schema |
{ "version":"1.0", "schema":{ "schemaId":1043, "type":"table", "rowNameProperty":"name", "plugin":"TOOLKIT", "properties":[ { "name":"input", "type":"dateTime" }, { "name":"utc", "type":"dateTime", "enriched":true }, { "name":"epoch", "type":"float64", "unit":"seconds", "enriched":true }, { "name":"utcAgain", "type":"dateTime", "enriched":true } ] }, "operation":"schema" } |
This is a combined schema. Both raw and enriched columns (if present) are included. Normalisation: In this example, |
Table metric |
{ "version":"1.0", "data":{ "sampleTime":"2018-11-08T13:19:27.06Z", "netprobeTime":"2018-11-08T13:19:27.06Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756", "row":"July" }, "schemaId":1043, "type":"table", "row":{ "input":"2018-07-26T00:34:46.257Z" } }, "operation":"create" } |
This is a raw data message, only raw columns appear. Normalisation: timestamps include fractions of a second. |
Table metric |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T13:19:27.078Z", "sampleTime":"2018-11-08T13:19:27.078Z", "netprobeTime":"2018-11-08T13:19:27.06Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756", "row":"July" }, "schemaId":1043, "type":"table", "row":{ "utc":"2018-07-26T00:34:46Z", "epoch":1532565286, "utcAgain":"2018-07-26T00:34:46Z" } }, "operation":"update" } |
This is an enriched data message, only enriched columns appear. Normalisation: timestamps include fractions of a second. |
Headline metric |
{ "version":"1.0", "data":{ "sampleTime":"2018-11-08T13:19:27.06Z", "netprobeTime":"2018-11-08T13:19:27.06Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756" }, "schemaId":1042, "type":"headline", "row":{ "samplingStatus":"OK", "execDuration":0.098934 } }, "operation":"create" } |
Normalisation: timestamps include fractions of a second. |
Message payloads - events
Type |
Example Payload |
---|---|
Severity |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T13:43:05.323Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756" }, "type":"severity", "event":{ "severity":"UNDEFINED", "active":true, "snoozed":false, "snoozedParents":0, "userAssigned":false } }, "operation":"update" } |
Snooze (simple example) |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T13:43:05.000Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756", "column":"samplingStatus" }, "type":"snooze", "event":{ "snoozed":true, "snoozedBy":"admin", "comment":"" }, "operation":"update" } |
Snooze (complex example) |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T13:53:19.000Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756", "row":"July", "column":"utc" }, "type":"snooze", "event":{ "snoozed":true, "snoozedBy":"admin", "comment":"Using every bell and whistle available", "untilSeverity":{ "value":"WARNING", "when":"not equals" }, "untilTime":{ "value":"2018-11-09T13:53:19.000Z" }, "untilValue":{ "value":"2018-07-26T00:34:46Z", "when":"not equals" } } }, "operation":"update" } |
Unsnooze |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T14:58:47.433Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756", "row":"July", "column":"utc" }, "type":"snooze", "event":{ "snoozed":false, "unsnoozedBy":"admin" } }, "operation":"update" } |
User assignment (simple) |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T15:00:05.000Z", "directoryId":17, "target":{ "gateway":"ExampleGateway", "probe":"vp", "managedEntity":"GatewayInfo" }, "type":"userassignment", "event":{ "userAssigned":true, "assignedTo":"admin", "assignedBy":"admin", "comment":"" } }, "operation":"update" } |
User assignment (complex) |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T15:04:18.000Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756" }, "type":"userassignment", "event":{ "userAssigned":true, "assignedTo":"otherUser", "assignedBy":"admin", "comment":"More options", "untilSeverity":{ "value":"OK" }, "untilTime":{ "value":"2018-11-09T15:04:18.000Z" } } }, "operation":"update" } |
User assignment (unassign) |
{ "version":"1.0", "data":{ "timestamp":"2018-11-08T15:21:43.105Z", "directoryId":42, "target":{ "gateway":"ExampleGateway", "probe":"theProbe", "managedEntity":"TimeFormat", "type":"", "sampler":"AA-4756", "dataview":"AA-4756" }, "type":"userassignment", "event":{ "userAssigned":false, "unassignedBy":"admin" } }, "operation":"update" } |