Stream AWS Access Logs into OpenTelemetry (OTLP)

NLB access logs capture Layer 4 TCP and TLS connections, including TLS handshake duration, ALPN negotiated protocols, cipher suites, incoming TLS alert codes, and byte counts transmitted across connections.

tls 2.0 2026-09-06T12:34:56Z net/my-nlb/1234567890abcdef sub-12345678 198.51.100.25:52134 10.0.2.100:443 15.201 2.304 4096 8192 - arn:aws:acm:us-east-1:123456789012:certificate/uuid 0 ECDHE-RSA-AES128-GCM-SHA256 TLSv1.3 - db.example.com h2 - 2026-09-06T12:34:40Z

Processes both legacy standard W3C tab-delimited gzipped logs and modern AWS CloudFront Parquet log files. Automatically identifies distribution domains, edge location pop codes (IAD79-C1), cache outcomes (Hit, Miss, RefreshHit), and time-to-first-byte measurements.

2026-09-06	12:34:56	IAD79-C1	3842	203.0.113.42	GET	d111111abcdef8.cloudfront.net	/static/app.js	200	https://example.com/	Mozilla/5.0	-	-	Hit	64abc123-request-id	example.com	https	1240	0.012	-	TLSv1.3	TLS_AES_128_GCM_SHA256	Hit	HTTP/2.0	-	-	51240	0.004	Hit	application/javascript	3842	-	-

AWS WAF logs contain structured JSON security events detailing terminating rules, actions (ALLOW, BLOCK, CAPTCHA, COUNT), rule group evaluations, JA3 / JA4 client TLS fingerprints, and country ISO codes. Blocks are promoted to WARN severity in OTel.

{
  "severityNumber": 13,
  "severityText": "WARN",
  "body": { "stringValue": "POST /login BLOCK" },
  "attributes": [
    { "key": "aws.waf.action", "value": { "stringValue": "BLOCK" } },
    { "key": "aws.waf.terminating_rule_id", "value": { "stringValue": "AWS-AWSManagedRulesCommonRuleSet" } },
    { "key": "tls.client.ja4", "value": { "stringValue": "t13d1516h2_8daaf6152771_b4b60e34b9d0" } },
    { "key": "client.geo.country_iso_code", "value": { "stringValue": "US" } },
    { "key": "client.address", "value": { "stringValue": "198.51.100.77" } },
    { "key": "http.request.method", "value": { "stringValue": "POST" } },
    { "key": "url.path", "value": { "stringValue": "/login" } }
  ]
}

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Production Deployment of otel-aws-log-processor

Parameters:
  OtlpEndpoint:
    Type: String
    Default: "http://otel-collector:4318/v1/logs"
    Description: HTTP endpoint for OTLP log ingestion

Resources:
  LogDeadLetterQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: otel-log-processor-dlq
      MessageRetentionPeriod: 1209600

  LogIngestionQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: otel-log-processor-queue
      VisibilityTimeout: 360
      RedrivePolicy:
        deadLetterTargetArn: !GetAtt LogDeadLetterQueue.Arn
        maxReceiveCount: 5

  OtelLogProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: otel-aws-log-processor
      CodeUri: ./lambda.zip
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures: [arm64]
      MemorySize: 512
      Timeout: 300
      Environment:
        Variables:
          OTLP_HTTP_LOGS_ENDPOINT: !Ref OtlpEndpoint
          MAX_BATCH_SIZE: 500
          MAX_RETRIES: 3
          MAX_CONCURRENT: 10
      Policies:
        - S3ReadPolicy:
            BucketName: "*"
        - SQSPollerPolicy:
            QueueName: !GetAtt LogIngestionQueue.QueueName
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt LogIngestionQueue.Arn
            BatchSize: 10
            FunctionResponseTypes:
              - ReportBatchItemFailures

#!/usr/bin/env bash
set -euo pipefail

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REGION=${AWS_REGION:-us-east-1}
ROLE_NAME="otel-log-processor-lambda-role"
QUEUE_NAME="otel-log-processor-queue"
FN_NAME="otel-aws-log-processor"

echo "1. Building Lambda zip package..."
make lambda-package

echo "2. Creating SQS queue..."
QUEUE_URL=$(aws sqs create-queue --queue-name "$QUEUE_NAME" \
  --attributes VisibilityTimeout=360 --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" \
  --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

echo "3. Creating Lambda Execution Role..."
cat <<EOF > /tmp/lambda-trust.json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "lambda.amazonaws.com"},
    "Action": "sts:AssumeRole"
  }]
}
EOF

ROLE_ARN=$(aws iam create-role --role-name "$ROLE_NAME" \
  --assume-role-policy-document file:///tmp/lambda-trust.json \
  --query 'Role.Arn' --output text 2>/dev/null || \
  aws iam get-role --role-name "$ROLE_NAME" --query 'Role.Arn' --output text)

echo "4. Deploying Lambda Function..."
aws lambda create-function \
  --function-name "$FN_NAME" \
  --runtime provided.al2023 \
  --handler bootstrap \
  --architectures arm64 \
  --zip-file fileb://lambda.zip \
  --role "$ROLE_ARN" \
  --timeout 300 \
  --memory-size 512 \
  --environment "Variables={OTLP_HTTP_LOGS_ENDPOINT=https://ingest.signoz.cloud:443/v1/logs,MAX_BATCH_SIZE=500}"

echo "5. Connecting SQS Event Source Mapping..."
aws lambda create-event-source-mapping \
  --function-name "$FN_NAME" \
  --event-source-arn "$QUEUE_ARN" \
  --batch-size 10 \
  --function-response-types ReportBatchItemFailures

echo "✓ Deployment complete!"

# Build Docker image locally
make docker-build

# Or build multi-arch (AMD64 + ARM64)
make docker-build-multiarch

# Push image to Amazon ECR
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
docker tag otel-aws-log-processor:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/otel-aws-log-processor:latest
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/otel-aws-log-processor:latest

# Deploy Lambda using Container Image package type
aws lambda create-function \
  --function-name otel-aws-log-processor \
  --package-type Image \
  --code ImageUri=123456789012.dkr.ecr.us-east-1.amazonaws.com/otel-aws-log-processor:latest \
  --role arn:aws:iam::123456789012:role/otel-log-processor-lambda-role \
  --timeout 300 \
  --memory-size 512