-
Couldn't load subscription status.
- Fork 19
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Describe the bug
When AWS VPC Flow logs are partitioned by every hour, rather than the default 24 hours, it adds an 'hour' folder
If you enable hourly partitions, the files are delivered to the following location.
bucket-and-optional-prefix/AWSLogs/account_id/vpcflowlogs/region/year/month/day/hour/
and this causes the regex to fail, and result in {"errorMessage":"type of S3 event could not be determined for object ...
To Reproduce
Steps to reproduce the behavior:
- Deploy
lambda-promtail,- Create S3 bucket,
- Create VPC flow log, with hourly partitioning:
resource "aws_flow_log" "example" {
log_destination = aws_s3_bucket.example.arn
log_destination_type = "s3"
traffic_type = "ALL"
vpc_id = aws_vpc.example.id
destination_options {
file_format = "parquet"
per_hour_partition = true
}
}- Observe
lambda-promtaillogs
Expected behavior
Either:
- update docs to highlight unsupported hourly partition
- update regex
Environment:
- Infrastructure: AWS Lambda
- Deployment tool: Terraform
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working