I recently sat down with the developers of the Illumio VEN (Virtual Enforcement Node) to learn how to export its traffic logs to a data cruncher like Splunk, Pentaho, or Sumologic.
It works like this:
- The Policy Compute Engine (PCE) gathers the VEN traffic logs via the collector service.
- The PCE can then forward a copy of those logs to another fluentd aggregator via the fluentd_source_service process.
Wait, what’s Fluentd?
Fluentd is a cool open-source data collector that collects logs from all over your data center and structures them in an easy-to-organize JSON format. It’s coded in C and Ruby so it’s super lightweight and has a very small footprint.
Okay, cool. But Jimmy Ray, why would I want to do this?
- Managers love reports and it’s close to performance eval time
- Compliance auditing, data achieving or even traffic flow analysis
- Because your name is Jason and you love JSON formatted objects
Sweet. I’m ready to try!
Great. Basically you’re going to need to do two things:
- First, get your fluentd server all config’d up to accept VEN traffic events and where to store them.
- Now simply point the PCE’s fluentd_source_service via the environment yaml file to the IP address to the fluentd aggregator server. For example:
external_fluentd_aggregator_servers: 172.16.1.11:24224
The logs are stored in JSON format with the following structure:
tab tab
For example:
2015-18-01T18:13:12Z accepted.12345678-2222… hostname: Dude defined…
Now you can import these into the data cruncher of your choice and generate amazing reports. Pat yourself on the back. You're the hero once again!
—Jimmy Ray
{{cta('ca8808ed-81f2-479d-8ab3-b274c5fe9a3c')}}