Geneos ["Geneos"]

Fluentd Forward

Overview

Fluentd is an open-source application used for unifying log collection and aggregation. Fluentd supports the Fluentd Forward protocol for inbound and outbound routing of events between other Fluentd instances.

The Fluentd Forward plugin provides a collector that supports the inbound Fluentd Forward protocol messages and allows Geneos to receive Fluentd unified logs for visualisation, analysis, and storage.

Compatibility with Fluentd Forward protocol specification

The protocol specifies many options and only some are currently supported. The following section provides an overview of what is currently supported and not supported:

Protocol Supported Not supported
Heartbeat Message

UDP heartbeats on the same port as the TCP listener

TCP heartbeats
Network Transport Protocol

Plain TCP

TLS/SSL

  
Connection Phases

Transport — the server goes directly to this phase and ignores handshakes from the client.

Handshake — including associated message exchanges, such as authentication.
Event Modes

All MessagePack modes, such as:

  • Message mode

  • Forward mode

  • PackedForward mode

  • CompressedPackedForward mode

  • JSON payloads

    		•
    Timestamp

    EventTime format

    Integer format

    		  
    Acks

    Acks via the chunk option

    		  
         

    Prerequisites

    Geneos environment

    The latest version of the Fluentd Forward plugin requires Gateway and Netprobe version 6.1.x or higher. The same version must be used for the GSE schema.

    The Fluentd Forward plugin binaries are packaged with Netprobe 6.1.x and are stored in the collection_agent folder.

    Alternatively, you can download separate binaries for the Fluentd plugin from the ITRS Downloads.

    Configure Geneos to deploy the Fluentd plugin

    The Fluentd plugin supports Collection Agent publishing into Geneos using dynamic Managed Entities. To set up the Fluentd plugin in Geneos, follow these steps:

    1. Set up your plugin either by:

      • Setting up your collector in the Gateway Setup Editor by adding the following configuration in Managed Entities > Collectors. For more information, see Collectors in Dynamic Entities.

      • Adding the following configuration in collection-agent.yml file on your local machine where the binaries are stored. 

        
                                    
      
                                
      2. 
                                
    2. 
                                    
      Configure your custom mappings in the Dynamic entities > Mappings setting in the Gateway Setup Editor. For more information, see Mapping and mapping group in Dynamic Entities. Below is an example of a custom mapping for Fluentd.
      
                                    
      
                                        
                                    
      
                                
      3. 
                                
    3. 
                                    
      Configure your other Dynamic Entities in the Gateway. See Create  Dynamic Entities in Collection Agent setup for a more detailed procedure.
      
                                
      4. 
                            
    
                            
    Alternatively, you can use Self-announcing Netprobes to enable dynamic entities. See Self-announcing Netprobe in Collection Agent setup.
    
                            
    Note: To check if there are any errors in the mapping, you can set up the Dynamic Entities Health, or look at the Collection Agent log file in Collection Agent setup.
    
                            
    
                                
    Example dataview
    
                                
    Below is a sample dataview using the custom mappings above.
    
                                
    
                                    
                                
    
                                
    To define how items in the Geneos tree structure are generated from the labels of a datapoint, see Geneos items in Dynamic Entities.