Full Packet Capture Options

T

Track Drew

Limp Gawd
Joined
Dec 6, 2007
Messages
511
Looking to run something as a backup to the commercial full packet capture solution we use.

I've used Niksun NetDetector, Solera DeepSee, and RSA NetWitness / Security Analytics. All have their pros and cons. They also aren't cheap and are far from perfect.

Goal is to capture all packets from our internet taps. We do some filtering on our commercial appliance to try to maximum space and do have redundancy already - so not really a secondary system, more of a tertiary.

Most simple is literally tcpdump running with a rotating output file attached to several TBs of storage. Have a cron job that then deletes the older pcaps as storage fills up. Can use tcpslice/mergecap/tcpdump to then filter these as needed.

More complicated would be something like Moloch. Although OpenFPC looks like a bit of a lighter weight alternative.

I'm also personally interested as I've been itching to get something with this capability at home.

Anyone have any experience doing this? Any advice?
 
I really hope that you are doing this for a business network and not as an ISP. The privacy implications are huge.
 
You can do this with a web filter..............

You can do this with snort...................

You can do this with a million other things.......

You will need a device that can decrypt HTTPS if you want it, and if you choose it, you will need to add a key file on the device you want to gather data from so the traffic is view able.
 
klank said:
I really hope that you are doing this for a business network and not as an ISP.
Click to expand...

It's a business. Systems and network are for work only - no expectation of privacy.

Tee said:
...web filter... snort... million other things... decrypt HTTPS...
Click to expand...

Our web proxy/filter will only produce logs and limits inspection to HTTP/HTTPS. It's managed by a different department, but I also doubt replacing it is within the scope.

I really hadn't considered using Snort's Packet Logger Mode. My experience has been that it's really best used as an IDS that will record only the payload of packets that match signatures - at least that's how I use it now. I don't want to tax my boxes too much more, and setting up new instances just for this functionality seems like overkill.

We already MITM SSL - so now I need to record it somewhere. (somewhere else, it's already being recorded by 2 different commercial FPC solutions)

The "million other things" is what I'm interested in - and real life experience with these.
 
This really comes down to what you want to spend. Driven by that would be how much traffic is being monitored and desired retention.

Doing things like a rotating tcpdump is possible sure, but interacting with that traffic is very cumbersome once say, you get an ids alert and have to find surrounding traffic.
 
I would really try Snort and run a tap so it doesn't impact the network. This will do the job as long as Snort knows to use your MITM SSL.

Plus you cannot really beat the price!
 
My budget is $0. I'll probably buy some hardware once I have the software details figured out. My current solution is a 6 figure expense - I can't justify spending more on top of that.

Ideally I'll never have to interact with this traffic. I have a solution that already works great. Just want some additional redundancy. If I did have to use it I would likely just export large chunks of data to somewhere I can work with it.

Tee said:
I would really try Snort
Click to expand...

Tee - you haven't communicated any reason or capability that makes Snort the solution for this task, it seems like it isn't.

Just for fun, I went ahead and spun up Snort on one of our old Endace boxes connected to our tap infrastructure to try out packet logging. Traffic from hosts is filtered into individual files once I define my home network. The format is non-pcap ascii, so it's worthless. Switching to binary mode puts everything in a single file - this is the same as tcpdump. So the Snort solution really seems to be for ad-hoc debugging.

I still can't believe no one is doing this at home...
 
What about using security onion. I've used that at home before and it captured packets just fine. It has a bunch of different tools including snort and snorby.
 
Do you do MITM SSL on all your SSL traffic or just parts of it? Our PA firewalls have that capability but I've never configured it.
 
We MITM parts, but they're pushing for all of it - our legal team says we're good to go. I honestly would consider not recording it - I've almost never needed to use the capability.

Security Onion is a linux distro - so a preconfigured collection of tools. Snort is an IDS and Snorby is a Snort front-end... Is there a specific tool you think would accomplish this task?
 
Track Drew said:
My budget is $0. I'll probably buy some hardware once I have the software details figured out. My current solution is a 6 figure expense - I can't justify spending more on top of that.

Ideally I'll never have to interact with this traffic. I have a solution that already works great. Just want some additional redundancy. If I did have to use it I would likely just export large chunks of data to somewhere I can work with it.



Tee - you haven't communicated any reason or capability that makes Snort the solution for this task, it seems like it isn't.

Just for fun, I went ahead and spun up Snort on one of our old Endace boxes connected to our tap infrastructure to try out packet logging. Traffic from hosts is filtered into individual files once I define my home network. The format is non-pcap ascii, so it's worthless. Switching to binary mode puts everything in a single file - this is the same as tcpdump. So the Snort solution really seems to be for ad-hoc debugging.

I still can't believe no one is doing this at home...
Click to expand...

No, I simply answered your questions about tools you could use.

Snort can be put in to packet logger mode http://manual.snort.org/node5.html

Snort is on most of the equipment you already use to collect data, those things just have a pretty pre-configured setup with a shiny interface.

You can go download a free version of SPLUNK and you can data mine all of your Snort logs with ease. You can search interfaces, IP's, traffic types, traffic to from, whatever......

Not to mention you can take all the data coming in to Snort and put it in to separate files based on specific criteria such as interface, traffic type, protocol, etc.. You just need to know how to do it.

This is my final answer.
 
You must log in or register to reply here.
Back
Top