Exporting VEN Traffic Logs

I recently sat down with the developers of the Illumio VEN (Virtual Enforcement Node) to learn how to export its traffic logs to a data cruncher like Splunk, Pentaho, or Sumologic.

It works like this:

1. The Policy Compute Engine (PCE) gathers the VEN traffic logs via the collector service.
2. The PCE can then forward a copy of those logs to another fluentd aggregator via the fluentd_source_service process. 

Wait, what’s Fluentd?

Fluentd is a cool open-source data collector that collects logs from all over your data center and structures them in an easy-to-organize JSON format. It’s coded in C and Ruby so it’s super lightweight and has a very small footprint.

Okay, cool. But Jimmy Ray, why would I want to do this?

1. Managers love reports and it’s close to performance eval time
2. Compliance auditing, data achieving or even traffic flow analysis
3. Because your name is Jason and you love JSON formatted objects

 Sweet. I’m ready to try!

Great. Basically you’re going to need to do two things:

1. First, get your fluentd server all config’d up to accept VEN traffic events and where to store them.
2. Now simply point the PCE’s fluentd_source_service via the environment yaml file to the IP address to the fluentd aggregator server. For example:

external_fluentd_aggregator_servers: 172.16.1.11:24224

The logs are stored in JSON format with the following structure:

 tab  tab 

 For example:

2015-18-01T18:13:12Z accepted.12345678-2222… hostname: Dude defined…

 Now you can import these into the data cruncher of your choice and generate amazing reports. Pat yourself on the back. You're the hero once again!

—Jimmy Ray

 {{cta('ca8808ed-81f2-479d-8ab3-b274c5fe9a3c')}}