Log Architecture & Structural Overview

The HAProxy Standard HTTP Access Log Parser is an essential telemetry stream within the Web Servers & Reverse Proxies ecosystem. Parse HAProxy HTTP logs with frontend, backend, active server, connection queue timings, and termination state.

This schema defines a structure of 24 extracted attributes, including 15 numeric metrics and 9 string dimensions. In production observability architectures, these tokens provide high-cardinality indexing keys for telemetry pipelines before shipping to storage backends such as ClickHouse, Elasticsearch, Amazon S3, or Datadog.

Raw Telemetry Ingestion Profile

A typical raw event line for haproxy-http-log averages 144 bytes across 24 tokens. Modern collectors such as Fluent Bit and Vector require zero-backtracking regular expressions to avoid CPU spikes during traffic surges.

Extracted Field Schema & Data Types

The transpiled Grok pattern extracts the following schema fields from each raw event line. Data collectors cast these values according to the typed mappings below.

Field Name	Inferred Type	Description & Collector Semantics
client_ip	string	Client IP.
client_port	integer	Client source port.
timestamp	string	Log timestamp.
frontend_name	string	Frontend proxy configuration name.
backend_name	string	Backend pool configuration name.
server_name	string	Active server responding to request.
tq	integer	Time to receive client request headers (ms).
tw	integer	Time waiting in queues (ms).
tc	integer	Time to establish connection with server (ms).
tr	integer	Time to receive server response headers (ms).
tt	integer	Total request lifetime (ms).
status	integer	HTTP status code.
bytes	integer	Bytes transferred.
term_state	string	Session termination state flags.
actconn	integer	Total concurrent process connections.
feconn	integer	Total frontend connections.
beconn	integer	Total backend connections.
srvconn	integer	Active connections on this server.
retries	integer	Connection retry attempts count.
srv_queue	integer	Server queue length.
backend_queue	integer	Backend queue length.
method	string	HTTP method.
uri	string	Requested URI.
http_version	string	HTTP version.

Common Regex Traps & Production Edge Cases

Engineers frequently encounter ingestion failures or pipeline drops due to subtle variations in real-world event logs. Watch out for these verified pitfalls:

1The 5-timer slash string (tq/tw/tc/tr/tt) requires careful slash escaping in regex.

2term_state flags (e.g. sH, cD, cR) explain whether client or server timed out first.

Production Collector Setup & Configurations

Pre-configured parser definitions ready to be dropped into your infrastructure repository.

Fluent Bit (parsers.conf)

Format: regex

# ==============================================================================
# Fluent Bit Parser Configuration (parsers.conf)
# ==============================================================================
[PARSER]
    Name        logpipeline_parser
    Format      regex
    Regex       ^(?<client_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))):(?<client_port>(?:[+-]?(?:[0-9]+))) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<frontend_name>\S+) (?<backend_name>\S+)/(?<server_name>\S+) (?<tq>(?:[+-]?(?:[0-9]+)))/(?<tw>(?:[+-]?(?:[0-9]+)))/(?<tc>(?:[+-]?(?:[0-9]+)))/(?<tr>(?:[+-]?(?:[0-9]+)))/(?<tt>(?:[+-]?(?:[0-9]+))) (?<status>(?:[+-]?(?:[0-9]+))) (?<bytes>(?:[+-]?(?:[0-9]+))) - - (?<term_state>\S+) (?<actconn>(?:[+-]?(?:[0-9]+)))/(?<feconn>(?:[+-]?(?:[0-9]+)))/(?<beconn>(?:[+-]?(?:[0-9]+)))/(?<srvconn>(?:[+-]?(?:[0-9]+)))/(?<retries>(?:[+-]?(?:[0-9]+))) (?<srv_queue>(?:[+-]?(?:[0-9]+)))/(?<backend_queue>(?:[+-]?(?:[0-9]+))) "(?<method>\b\w+\b) (?<uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))"$
    Time_Key    timestamp
    Time_Format %Y-%m-%dT%H:%M:%S%z
    Types       client_port:integer tq:integer tw:integer tc:integer tr:integer tt:integer status:integer bytes:integer actconn:integer feconn:integer beconn:integer srvconn:integer retries:integer srv_queue:integer backend_queue:integer

# ==============================================================================
# Fluent Bit Pipeline Filter (fluent-bit.conf)
# ==============================================================================
[FILTER]
    Name         parser
    Match        *
    Key_Name     log
    Parser       logpipeline_parser
    Reserve_Data On

Vector.dev (Remap VRL)

parse_regex!

# ==============================================================================
# Vector.dev Remap Language (VRL) Transform
# Use inside a 'remap' transform in vector.yaml
# ==============================================================================
.parsed, err = parse_regex(.message, r'^(?<client_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))):(?<client_port>(?:[+-]?(?:[0-9]+))) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<frontend_name>\S+) (?<backend_name>\S+)/(?<server_name>\S+) (?<tq>(?:[+-]?(?:[0-9]+)))/(?<tw>(?:[+-]?(?:[0-9]+)))/(?<tc>(?:[+-]?(?:[0-9]+)))/(?<tr>(?:[+-]?(?:[0-9]+)))/(?<tt>(?:[+-]?(?:[0-9]+))) (?<status>(?:[+-]?(?:[0-9]+))) (?<bytes>(?:[+-]?(?:[0-9]+))) - - (?<term_state>\S+) (?<actconn>(?:[+-]?(?:[0-9]+)))/(?<feconn>(?:[+-]?(?:[0-9]+)))/(?<beconn>(?:[+-]?(?:[0-9]+)))/(?<srvconn>(?:[+-]?(?:[0-9]+)))/(?<retries>(?:[+-]?(?:[0-9]+))) (?<srv_queue>(?:[+-]?(?:[0-9]+)))/(?<backend_queue>(?:[+-]?(?:[0-9]+))) "(?<method>\b\w+\b) (?<uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))"$')

if err == null {
    . = merge(., .parsed)
    del(.parsed)

    # Type coercions
    .client_port = to_int!(.client_port)
    .tq = to_int!(.tq)
    .tw = to_int!(.tw)
    .tc = to_int!(.tc)
    .tr = to_int!(.tr)
    .tt = to_int!(.tt)
    .status = to_int!(.status)
    .bytes = to_int!(.bytes)
    .actconn = to_int!(.actconn)
    .feconn = to_int!(.feconn)
    .beconn = to_int!(.beconn)
    .srvconn = to_int!(.srvconn)
    .retries = to_int!(.retries)
    .srv_queue = to_int!(.srv_queue)
    .backend_queue = to_int!(.backend_queue)

} else {
    log("LogPipeline parsing warning: " + err, level: "warn")
}

# ==============================================================================
# vector.yaml Pipeline Component
# ==============================================================================
transforms:
  parse_logs:
    type: remap
    inputs: ["source_logs"]
    source: |
      .parsed, err = parse_regex(.message, r'^(?<client_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))):(?<client_port>(?:[+-]?(?:[0-9]+))) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<frontend_name>\S+) (?<backend_name>\S+)/(?<server_name>\S+) (?<tq>(?:[+-]?(?:[0-9]+)))/(?<tw>(?:[+-]?(?:[0-9]+)))/(?<tc>(?:[+-]?(?:[0-9]+)))/(?<tr>(?:[+-]?(?:[0-9]+)))/(?<tt>(?:[+-]?(?:[0-9]+))) (?<status>(?:[+-]?(?:[0-9]+))) (?<bytes>(?:[+-]?(?:[0-9]+))) - - (?<term_state>\S+) (?<actconn>(?:[+-]?(?:[0-9]+)))/(?<feconn>(?:[+-]?(?:[0-9]+)))/(?<beconn>(?:[+-]?(?:[0-9]+)))/(?<srvconn>(?:[+-]?(?:[0-9]+)))/(?<retries>(?:[+-]?(?:[0-9]+))) (?<srv_queue>(?:[+-]?(?:[0-9]+)))/(?<backend_queue>(?:[+-]?(?:[0-9]+))) "(?<method>\b\w+\b) (?<uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))"$')
      if err == null {
        . = merge(., .parsed)
        del(.parsed)
      }

Datadog Log Pipeline Grok Parser

match_rules

# ==============================================================================
# Datadog Log Processing Pipeline Grok Parser
# Navigate to: Logs -> Configuration -> Pipelines -> Add Processor -> Grok Parser
# ==============================================================================

# Match Rule:
rule %{IP:client_ip}:%{INT:client_port} \[%{HTTPDATE:timestamp}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:tq}/%{INT:tw}/%{INT:tc}/%{INT:tr}/%{INT:tt} %{INT:status} %{INT:bytes} - - %{NOTSPACE:term_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{INT:retries} %{INT:srv_queue}/%{INT:backend_queue} "%{WORD:method} %{NOTSPACE:uri} HTTP/%{NUMBER:http_version}"

# Complete Datadog Pipeline Processor JSON:
{
  "type": "grok-parser",
  "name": "LogPipeline Grok Parser",
  "is_enabled": true,
  "source": "message",
  "samples": [],
  "grok": {
    "match_rules": "rule %{IP:client_ip}:%{INT:client_port} \\[%{HTTPDATE:timestamp}\\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:tq}/%{INT:tw}/%{INT:tc}/%{INT:tr}/%{INT:tt} %{INT:status} %{INT:bytes} - - %{NOTSPACE:term_state} %{INT:actconn}/%{INT:feconn}/%{INT:beconn}/%{INT:srvconn}/%{INT:retries} %{INT:srv_queue}/%{INT:backend_queue} \"%{WORD:method} %{NOTSPACE:uri} HTTP/%{NUMBER:http_version}\"",
    "support_rules": ""
  }
}

# Target Fields Created:
# client_ip (string), client_port (integer), timestamp (string), frontend_name (string), backend_name (string), server_name (string), tq (integer), tw (integer), tc (integer), tr (integer), tt (integer), status (integer), bytes (integer), term_state (string), actconn (integer), feconn (integer), beconn (integer), srvconn (integer), retries (integer), srv_queue (integer), backend_queue (integer), method (string), uri (string), http_version (string)

OpenTelemetry Collector (transform processor)

regex_parser

# ==============================================================================
# OpenTelemetry Collector Configuration (otel-collector-config.yaml)
# Option 1: Filelog Receiver with regex_parser Operator
# ==============================================================================
receivers:
  filelog:
    include: [ /var/log/**/*.log ]
    start_at: beginning
    operators:
      - type: regex_parser
        id: logpipeline_regex_parser
        regex: '^(?<client_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))):(?<client_port>(?:[+-]?(?:[0-9]+))) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<frontend_name>\S+) (?<backend_name>\S+)/(?<server_name>\S+) (?<tq>(?:[+-]?(?:[0-9]+)))/(?<tw>(?:[+-]?(?:[0-9]+)))/(?<tc>(?:[+-]?(?:[0-9]+)))/(?<tr>(?:[+-]?(?:[0-9]+)))/(?<tt>(?:[+-]?(?:[0-9]+))) (?<status>(?:[+-]?(?:[0-9]+))) (?<bytes>(?:[+-]?(?:[0-9]+))) - - (?<term_state>\S+) (?<actconn>(?:[+-]?(?:[0-9]+)))/(?<feconn>(?:[+-]?(?:[0-9]+)))/(?<beconn>(?:[+-]?(?:[0-9]+)))/(?<srvconn>(?:[+-]?(?:[0-9]+)))/(?<retries>(?:[+-]?(?:[0-9]+))) (?<srv_queue>(?:[+-]?(?:[0-9]+)))/(?<backend_queue>(?:[+-]?(?:[0-9]+))) "(?<method>\b\w+\b) (?<uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))"$'
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%dT%H:%M:%S%z'

# ==============================================================================
# Option 2: Transform Processor (OTel Transformation Language - OTTL)
# ==============================================================================
processors:
  transform:
    error_mode: ignore
    log_statements:
      - context: log
        statements:
          - merge_maps(attributes, extract_patterns(body, "^(?<client_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))):(?<client_port>(?:[+-]?(?:[0-9]+))) \\[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\\b)/(?:\\b[0-9]{4}\\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\\] (?<frontend_name>\\S+) (?<backend_name>\\S+)/(?<server_name>\\S+) (?<tq>(?:[+-]?(?:[0-9]+)))/(?<tw>(?:[+-]?(?:[0-9]+)))/(?<tc>(?:[+-]?(?:[0-9]+)))/(?<tr>(?:[+-]?(?:[0-9]+)))/(?<tt>(?:[+-]?(?:[0-9]+))) (?<status>(?:[+-]?(?:[0-9]+))) (?<bytes>(?:[+-]?(?:[0-9]+))) - - (?<term_state>\\S+) (?<actconn>(?:[+-]?(?:[0-9]+)))/(?<feconn>(?:[+-]?(?:[0-9]+)))/(?<beconn>(?:[+-]?(?:[0-9]+)))/(?<srvconn>(?:[+-]?(?:[0-9]+)))/(?<retries>(?:[+-]?(?:[0-9]+))) (?<srv_queue>(?:[+-]?(?:[0-9]+)))/(?<backend_queue>(?:[+-]?(?:[0-9]+))) \"(?<method>\\b\\w+\\b) (?<uri>\\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\\.[0-9]+)?|\\.[0-9]+)))))\"$"), "insert")

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [transform]
      exporters: [otlp]

Logstash Filter Configuration

filter.grok

# ==============================================================================
# Logstash Pipeline Configuration (/etc/logstash/conf.d/logpipeline.conf)
# ==============================================================================
filter {
  grok {
    match => { "message" => "%{IP:client_ip}:%{INT:client_port:integer} \[%{HTTPDATE:timestamp}\] %{NOTSPACE:frontend_name} %{NOTSPACE:backend_name}/%{NOTSPACE:server_name} %{INT:tq:integer}/%{INT:tw:integer}/%{INT:tc:integer}/%{INT:tr:integer}/%{INT:tt:integer} %{INT:status:integer} %{INT:bytes:integer} - - %{NOTSPACE:term_state} %{INT:actconn:integer}/%{INT:feconn:integer}/%{INT:beconn:integer}/%{INT:srvconn:integer}/%{INT:retries:integer} %{INT:srv_queue:integer}/%{INT:backend_queue:integer} \"%{WORD:method} %{NOTSPACE:uri} HTTP/%{NUMBER:http_version}\"" }
    tag_on_failure => [ "_grokparsefailure" ]
  }

  date {
    match => [ "timestamp", "ISO8601", "dd/MMM/yyyy:HH:mm:ss Z" ]
    target => "@timestamp"
    remove_field => [ "timestamp" ]
  }
}