It’s been a while since my last post! I’ve been spending some time in my homelab with the latest vRealize Suite products. One of those fantastic tools is vRealize Log Insight. I won’t get into the details of the product, but one of the use cases I wanted to tackle for my homelab was ingesting any and all syslog messages I could find.

A key piece of my homelab is a PFSense VM that I use for routing and firewall separation between my home network and the components nested snuggly in my HP Z800 Workstation. PFSense supports sending syslog messages for one, many or all of the services it hosts so it was a simply matter of configuring a syslog server in PFSense and watching the logs roll in to vRealize Log Insight.

However, something I noticed was that vRealize Log Insight doesn’t have a content pack for PFSense. Not a problem, as vRealize Log Insight makes it incredibly easy to extract fields from the messages and turn them into powerful structured data. So, that’s what I did. At least, for the firewall messages that PFSense was sending. All up it took about 30 minutes to create the handful of extracted fields, and another few minutes to create some widgets for a dashboard.

Screenshot of messages received from PFSense

Extracted fields I created based on the log messages. Fields starting with ‘pf’ are the new fields.

Shows the regex used to identify the source port in a firewall message.

Basic dashboard with even more basic widgets, using the extracted fields above.

Netgate provide great docs on PFSense. I was able to use them to identify the fields within the log message by reading the Raw Filter Log Format page.

Caveats

As I’m only running IPv4 in my network, I’ve only accounted for that traffic. My body (and homelab) isn’t ready for IPv6 yet so I’ve left it for the time being. Additionally, the only real traffic has been TCP. I haven’t tested many of the others.

Contributions / Requests

If you have any requests for additional fields or widgets let me know in the comments. I’d be more than happy to give it a crack in the homelab. If there’s something you’d like to contribute, post up your field and regex in the comments and I’ll include it in the content pack (with attribution of course).

Download

It’s nothing fancy, and definitely a humble v0.1 but if you are using PFSense in your homelab or home network (or even at work!) and you’d like some more visibility, here’s a basic building block to get you started.

Download currently unavailable.

Changelog

v0.1 - 13th May 2020

Created the following fields:
- pf_fw_action
- pf_fw_dest_ip
- pf_fw_dest_port
- pf_fw_direction
- pf_fw_interface
- pf_fw_ipv4_protocol
- pf_fw_ipversion
- pf_fw_ruleid
- pf_fw_src_ip
- pf_fw_src_port
- pf_service
Included basic dashboard example.