Setup Observability

Observability stacks require sinks for different types of logs - Application logs from each pod and access logs, and Verbose logs from Traffic Manager. Once the sinks are configured, configure ingestion ports/paths for Fluent Bit side-car from each pod to send logs.

Check the reference implementation on GitHub repository for AWS OpenSearch. Refer to Send Logs to Elasticsearch Service using Fluent Bit on the Fluent Bit website for detailed instructions on sending logs to the Elasticsearch Service.

Configuring Fluent Bit Sidecar

Log forwarding

The default output for Fluent Bit in Local Edition is the console. To make useful decisions with logs, you must configure Fluent Bit to send logs to the observability stack.

There are two Fluent Bit config maps: one specific to the traffic manager and another common to other pods. You must replace these config maps with your own to configure Fluent Bit to send logs to external destinations. A community implementation is available in the open-source repository. You must set up and configure log storage destinations before bringing up the cluster. You can use secure communications between Fluent Bit and the external system. The TLS configuration secrets required for the log flow are specific to each Fluent Bit output.