We ingest high-volume GCP Cloud DNS query logs into Google SecOps through Cloud Logging direct ingestion. We want to reduce ingestion volume without randomly sampling security events.
Is there a supported native way in Cloud Logging, Log Router or Google SecOps to aggregate repeated events, for example:
key = vmProjectId + queryName + responseCode + queryType window = 5 minutes output = one event with count, firstSeen and lastSeen
We would keep NXDOMAIN, SERVFAIL, first-seen domains, unknown domains and security-sensitive destinations at 100%.
If this is not supported natively, what architecture does Google recommend: Pub/Sub/Dataflow, Bindplane, or another collector? Can the aggregated records still be parsed as GCP DNS events in Google SecOps?
