Splunk
Grafana tells me if a box is sick. Splunk tells me what the firewall, the IDS, and the guests actually saw. Dedicated Splunk Enterprise VM on the cluster. Internal only. Not a search bar on a shared guest.
The problem with stuffing it elsewhere
The IDS sensor is a sensor. Grafana is scrape metrics. Neither is a SIEM. Putting firewall syslog and CloudTrail on the same box that already runs Suricata, or into a Grafana panel, mixes jobs. This project is the dedicated indexer.
Architecture
Traffic flow: Edge firewall / UniFi / guests / IDS / CloudTrail → syslog → Splunk indexes → internal dashboards
What it covers
firewall. Blocks, allows, and port noise from the actual edge, not a lab copy.network.os and app.ids. The sensor stays a sensor.aws. Same box as the firewall logs, different index.Dedicated VM, not an LXC
Index disk stays on the SIEM guest. Splunk is not stuffed into the IDS sensor or into a container that already has another job.
Index per source
One bucket would make every hunt a host-guessing game. Firewall, UniFi, guests, IDS, and CloudTrail each land in their own index.
Internal only
Wildcard TLS and the reverse proxy. No public Splunk URL. Searches with real hosts stay off this page.
Ansible knows the host. Terraform does not provision it yet. Recovery path is create the VM, install the package, tune ingest. Trial/dev license. Not a 24/7 SOC contract.
Roadmap
Related reading: Splunk got a VM, not a container.
Stack