Connect Fluentd to Bronto

Configure the Fluentd open-source data collector to tail application log files and forward structured events to Bronto over HTTP with batching and retries.

Fluentd forwards logs to Bronto using its built-in http output plugin. This page covers Fluentd configuration only.

For installation instructions, see the Fluentd installation guide. For the full Fluentd configuration reference, see the Fluentd documentation.

Endpoint and authentication

Use the ingestion endpoint for your Bronto region:

Region	Endpoint
EU	https://ingestion.eu.bronto.io:443
US	https://ingestion.us.bronto.io:443

Every request requires these headers:

Header	Required	Description
x-bronto-api-key	Required	Your Bronto API key.
x-bronto-dataset	Required	The dataset to route logs to.
x-bronto-collection	Required	The collection to group datasets under.
x-bronto-tags	Optional	Key-value tag pairs (e.g. env=prod,team=platform). See Partitions.

Minimal configuration

Tail a JSON log file and forward to Bronto.

/etc/fluent/fluentd.conf

<source>
  @type tail
  path /path/to/your/logs
  tag app.logs
  refresh_interval 5s
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/fluentd.pos
</source>

<match app.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post

  <buffer>
    @type file
    path /var/log/td-agent/buffer/http
    flush_interval 10s
    chunk_limit_size 5MB
    overflow_action block
  </buffer>

  <format>
    @type json
  </format>

  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "<YOUR_DATASET_NAME>", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>

For the full HTTP output configuration reference, see the Fluentd HTTP output documentation.

Parsing unstructured logs

Rather than building Fluentd parsers for unstructured text, ship raw log lines to Bronto and use the Bronto Custom Parser to extract structured fields server-side. The Custom Parser uses LLMs to generate parsers automatically and ships with built-in support for Apache, IIS, HAProxy, Syslog, key-value, and custom formats — no regex maintenance required.

Common patterns

Adding metadata to every log

Use the record_transformer filter to inject metadata into every record. Values can be expressions (Ruby) or environment variables.

<filter **>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
    branch "#{ENV['APP_GIT_BRANCH']}"
  </record>
</filter>

Routing multiple log sources to separate datasets

Tag each <source> distinctly, then use a separate <match> block for each tag with the appropriate x-bronto-dataset header.

<source>
  @type tail
  path /var/log/api.log
  tag api.logs
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/api.pos
</source>

<source>
  @type tail
  path /var/log/worker.log
  tag worker.logs
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/worker.pos
</source>

<match api.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post
  <format>
    @type json
  </format>
  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "api-service", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>

<match worker.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post
  <format>
    @type json
  </format>
  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "worker-service", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>

Verify log collection

Once you have applied your configuration and restarted Fluentd, you can expect to see your log data being ingested to Bronto and accessible via the Search page.

Further reading

• Fluentd installation
• Fluentd configuration reference
• http output plugin
• Buffering
• Bronto Custom Parser — extract structured fields from unstructured logs