Log Architecture & Structural Overview

The Amazon S3 Server Access Log Parser is an essential telemetry stream within the AWS Cloud Infrastructure ecosystem. Parse Amazon S3 bucket server access logs to audit bucket operations, data transfers, and anonymous requests.

This schema defines a structure of 17 extracted attributes, including 5 numeric metrics and 12 string dimensions. In production observability architectures, these tokens provide high-cardinality indexing keys for telemetry pipelines before shipping to storage backends such as ClickHouse, Elasticsearch, Amazon S3, or Datadog.

Raw Telemetry Ingestion Profile

A typical raw event line for aws-s3-access-logs averages 294 bytes across 17 tokens. Modern collectors such as Fluent Bit and Vector require zero-backtracking regular expressions to avoid CPU spikes during traffic surges.

Extracted Field Schema & Data Types

The transpiled Grok pattern extracts the following schema fields from each raw event line. Data collectors cast these values according to the typed mappings below.

Field Name	Inferred Type	Description & Collector Semantics
bucket_owner	string	Canonical user ID of the source bucket owner.
bucket	string	The name of the target S3 bucket.
timestamp	string	Date and time request was received in HTTPDATE format.
remote_ip	string	Internet Protocol address of the caller.
requester	string	IAM user ARN or canonical ID of requester, or '-' for unauthenticated.
request_id	string	String generated by S3 to uniquely identify request.
operation	string	REST operation performed (e.g. REST.GET.OBJECT).
key	string	Object key target or '-'.
method	string	HTTP method.
request_uri	string	Request-URI portion of HTTP request.
http_version	string	HTTP protocol version.
status	integer	Numeric HTTP response status code.
error_code	string	S3 error code (e.g. AccessDenied, NoSuchKey) or '-'.
bytes_sent	integer	Number of bytes sent in response.
object_size	integer	Total size of object in bytes.
total_time	integer	Total milliseconds request took from first to last byte.
turn_around_time	integer	Milliseconds S3 spent processing request.

Common Regex Traps & Production Edge Cases

Engineers frequently encounter ingestion failures or pipeline drops due to subtle variations in real-world event logs. Watch out for these verified pitfalls:

1Key names containing spaces or special characters appear URL-encoded.

2Anonymous requests show requester as a hyphen (-).

3S3 log files can be delivered multiple times for the same time window; ensure your collector uses request_id for deduplication.

Production Collector Setup & Configurations

Pre-configured parser definitions ready to be dropped into your infrastructure repository.

Fluent Bit (parsers.conf)

Format: regex

# ==============================================================================
# Fluent Bit Parser Configuration (parsers.conf)
# ==============================================================================
[PARSER]
    Name        logpipeline_parser
    Format      regex
    Regex       ^(?<bucket_owner>\S+) (?<bucket>\S+) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<remote_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))) (?<requester>\S+) (?<request_id>\S+) (?<operation>\S+) (?<key>\S+) "(?<method>\b\w+\b) (?<request_uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))" (?<status>(?:[+-]?(?:[0-9]+))) (?:(?<error_code>\S+)|-) (?<bytes_sent>(?:[+-]?(?:[0-9]+))) (?<object_size>(?:[+-]?(?:[0-9]+))) (?<total_time>(?:[+-]?(?:[0-9]+))) (?:(?<turn_around_time>(?:[+-]?(?:[0-9]+)))|-)$
    Time_Key    timestamp
    Time_Format %Y-%m-%dT%H:%M:%S%z
    Types       status:integer bytes_sent:integer object_size:integer total_time:integer turn_around_time:integer

# ==============================================================================
# Fluent Bit Pipeline Filter (fluent-bit.conf)
# ==============================================================================
[FILTER]
    Name         parser
    Match        *
    Key_Name     log
    Parser       logpipeline_parser
    Reserve_Data On

Vector.dev (Remap VRL)

parse_regex!

# ==============================================================================
# Vector.dev Remap Language (VRL) Transform
# Use inside a 'remap' transform in vector.yaml
# ==============================================================================
.parsed, err = parse_regex(.message, r'^(?<bucket_owner>\S+) (?<bucket>\S+) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<remote_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))) (?<requester>\S+) (?<request_id>\S+) (?<operation>\S+) (?<key>\S+) "(?<method>\b\w+\b) (?<request_uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))" (?<status>(?:[+-]?(?:[0-9]+))) (?:(?<error_code>\S+)|-) (?<bytes_sent>(?:[+-]?(?:[0-9]+))) (?<object_size>(?:[+-]?(?:[0-9]+))) (?<total_time>(?:[+-]?(?:[0-9]+))) (?:(?<turn_around_time>(?:[+-]?(?:[0-9]+)))|-)$')

if err == null {
    . = merge(., .parsed)
    del(.parsed)

    # Type coercions
    .status = to_int!(.status)
    .bytes_sent = to_int!(.bytes_sent)
    .object_size = to_int!(.object_size)
    .total_time = to_int!(.total_time)
    .turn_around_time = to_int!(.turn_around_time)

} else {
    log("LogPipeline parsing warning: " + err, level: "warn")
}

# ==============================================================================
# vector.yaml Pipeline Component
# ==============================================================================
transforms:
  parse_logs:
    type: remap
    inputs: ["source_logs"]
    source: |
      .parsed, err = parse_regex(.message, r'^(?<bucket_owner>\S+) (?<bucket>\S+) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<remote_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))) (?<requester>\S+) (?<request_id>\S+) (?<operation>\S+) (?<key>\S+) "(?<method>\b\w+\b) (?<request_uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))" (?<status>(?:[+-]?(?:[0-9]+))) (?:(?<error_code>\S+)|-) (?<bytes_sent>(?:[+-]?(?:[0-9]+))) (?<object_size>(?:[+-]?(?:[0-9]+))) (?<total_time>(?:[+-]?(?:[0-9]+))) (?:(?<turn_around_time>(?:[+-]?(?:[0-9]+)))|-)$')
      if err == null {
        . = merge(., .parsed)
        del(.parsed)
      }

Datadog Log Pipeline Grok Parser

match_rules

# ==============================================================================
# Datadog Log Processing Pipeline Grok Parser
# Navigate to: Logs -> Configuration -> Pipelines -> Add Processor -> Grok Parser
# ==============================================================================

# Match Rule:
rule %{NOTSPACE:bucket_owner} %{NOTSPACE:bucket} \[%{HTTPDATE:timestamp}\] %{IP:remote_ip} %{NOTSPACE:requester} %{NOTSPACE:request_id} %{NOTSPACE:operation} %{NOTSPACE:key} "%{WORD:method} %{NOTSPACE:request_uri} HTTP/%{NUMBER:http_version}" %{INT:status} (?:%{NOTSPACE:error_code}|-) %{INT:bytes_sent} %{INT:object_size} %{INT:total_time} (?:%{INT:turn_around_time}|-)

# Complete Datadog Pipeline Processor JSON:
{
  "type": "grok-parser",
  "name": "LogPipeline Grok Parser",
  "is_enabled": true,
  "source": "message",
  "samples": [],
  "grok": {
    "match_rules": "rule %{NOTSPACE:bucket_owner} %{NOTSPACE:bucket} \\[%{HTTPDATE:timestamp}\\] %{IP:remote_ip} %{NOTSPACE:requester} %{NOTSPACE:request_id} %{NOTSPACE:operation} %{NOTSPACE:key} \"%{WORD:method} %{NOTSPACE:request_uri} HTTP/%{NUMBER:http_version}\" %{INT:status} (?:%{NOTSPACE:error_code}|-) %{INT:bytes_sent} %{INT:object_size} %{INT:total_time} (?:%{INT:turn_around_time}|-)",
    "support_rules": ""
  }
}

# Target Fields Created:
# bucket_owner (string), bucket (string), timestamp (string), remote_ip (string), requester (string), request_id (string), operation (string), key (string), method (string), request_uri (string), http_version (string), status (integer), error_code (string), bytes_sent (integer), object_size (integer), total_time (integer), turn_around_time (integer)

OpenTelemetry Collector (transform processor)

regex_parser

# ==============================================================================
# OpenTelemetry Collector Configuration (otel-collector-config.yaml)
# Option 1: Filelog Receiver with regex_parser Operator
# ==============================================================================
receivers:
  filelog:
    include: [ /var/log/**/*.log ]
    start_at: beginning
    operators:
      - type: regex_parser
        id: logpipeline_regex_parser
        regex: '^(?<bucket_owner>\S+) (?<bucket>\S+) \[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\b)/(?:\b[0-9]{4}\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\] (?<remote_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))) (?<requester>\S+) (?<request_id>\S+) (?<operation>\S+) (?<key>\S+) "(?<method>\b\w+\b) (?<request_uri>\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\.[0-9]+)?|\.[0-9]+)))))" (?<status>(?:[+-]?(?:[0-9]+))) (?:(?<error_code>\S+)|-) (?<bytes_sent>(?:[+-]?(?:[0-9]+))) (?<object_size>(?:[+-]?(?:[0-9]+))) (?<total_time>(?:[+-]?(?:[0-9]+))) (?:(?<turn_around_time>(?:[+-]?(?:[0-9]+)))|-)$'
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%dT%H:%M:%S%z'

# ==============================================================================
# Option 2: Transform Processor (OTel Transformation Language - OTTL)
# ==============================================================================
processors:
  transform:
    error_mode: ignore
    log_statements:
      - context: log
        statements:
          - merge_maps(attributes, extract_patterns(body, "^(?<bucket_owner>\\S+) (?<bucket>\\S+) \\[(?<timestamp>(?:(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9]))/(?:\\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|Nov(?:ember)?|Dec(?:ember)?)\\b)/(?:\\b[0-9]{4}\\b):(?:(?:(?:(?:2[0123]|[01]?[0-9])):(?:(?:[0-5][0-9]))(?::(?:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?))))) (?:(?:Z|[+-](?:(?:2[0123]|[01]?[0-9]))(?::?(?:(?:[0-5][0-9]))))))\\] (?<remote_ip>(?:(?:(?:(?:[0-9A-Fa-f]{1,4}:){7}[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,7}:|(?:[0-9A-Fa-f]{1,4}:){1,6}:[0-9A-Fa-f]{1,4}|(?:[0-9A-Fa-f]{1,4}:){1,5}(?::[0-9A-Fa-f]{1,4}){1,2}|(?:[0-9A-Fa-f]{1,4}:){1,4}(?::[0-9A-Fa-f]{1,4}){1,3}|(?:[0-9A-Fa-f]{1,4}:){1,3}(?::[0-9A-Fa-f]{1,4}){1,4}|(?:[0-9A-Fa-f]{1,4}:){1,2}(?::[0-9A-Fa-f]{1,4}){1,5}|[0-9A-Fa-f]{1,4}:(?:(?::[0-9A-Fa-f]{1,4}){1,6})|:(?:(?::[0-9A-Fa-f]{1,4}){1,7}|:)|fe80:(?::[0-9A-Fa-f]{0,4}){0,4}%[0-9a-zA-Z]+|::(?:ffff(?::0{1,4})?:)?(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])|(?:[0-9A-Fa-f]{1,4}:){1,4}:(?:(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])\\.){3}(?:25[0-5]|(?:2[0-4]|1?[0-9])?[0-9])))|(?:(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))) (?<requester>\\S+) (?<request_id>\\S+) (?<operation>\\S+) (?<key>\\S+) \"(?<method>\\b\\w+\\b) (?<request_uri>\\S+) HTTP/(?<http_version>(?:(?:(?:[+-]?(?:[0-9]+(?:\\.[0-9]+)?|\\.[0-9]+)))))\" (?<status>(?:[+-]?(?:[0-9]+))) (?:(?<error_code>\\S+)|-) (?<bytes_sent>(?:[+-]?(?:[0-9]+))) (?<object_size>(?:[+-]?(?:[0-9]+))) (?<total_time>(?:[+-]?(?:[0-9]+))) (?:(?<turn_around_time>(?:[+-]?(?:[0-9]+)))|-)$"), "insert")

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [transform]
      exporters: [otlp]

Logstash Filter Configuration

filter.grok

# ==============================================================================
# Logstash Pipeline Configuration (/etc/logstash/conf.d/logpipeline.conf)
# ==============================================================================
filter {
  grok {
    match => { "message" => "%{NOTSPACE:bucket_owner} %{NOTSPACE:bucket} \[%{HTTPDATE:timestamp}\] %{IP:remote_ip} %{NOTSPACE:requester} %{NOTSPACE:request_id} %{NOTSPACE:operation} %{NOTSPACE:key} \"%{WORD:method} %{NOTSPACE:request_uri} HTTP/%{NUMBER:http_version}\" %{INT:status:integer} (?:%{NOTSPACE:error_code}|-) %{INT:bytes_sent:integer} %{INT:object_size:integer} %{INT:total_time:integer} (?:%{INT:turn_around_time:integer}|-)" }
    tag_on_failure => [ "_grokparsefailure" ]
  }

  date {
    match => [ "timestamp", "ISO8601", "dd/MMM/yyyy:HH:mm:ss Z" ]
    target => "@timestamp"
    remove_field => [ "timestamp" ]
  }
}