Router OS to log bandwidth usage by IP?

munkle

[H]F Junkie
Joined
Jan 16, 2005
Messages
11,799
I have a dell poweredge 830 (3.0 dual core pentium, 2gb ram, 2x160gb hdd in raid 1, dual port intel gigabit card). I'm trying to find a router OS that logs bandwidth by ips on the lan. I tried untangle and clearos neither of which have a feature (or one that I can find) that logs the bandwidth by ip. Apparently pfsense has a package you can install but I have never been able to get pfsense to work on this pc, it always errors in one way or another (on the 2.0rc3 I can't access the web page, it says it is missing a file). Is there something else I should look at or am I missing something in the ones I have tried?
 
Set up the needed monitor session in your environment and set up something link Ntop on a linux install.
 
Try PFsense 1.2.3. That's the most recent stable version. 2.0 is still in RC status, and it my experience still has a little bit to go. It's close, but not there yet.

You'll be looking specifically at the "BandwidthD" package.
 
Try PFsense 1.2.3. That's the most recent stable version. 2.0 is still in RC status, and it my experience still has a little bit to go. It's close, but not there yet.

You'll be looking specifically at the "BandwidthD" package.

This.
 
"Per User" summary in the reports of Untangle.

Also agree, BandwidthD is a great add-on for PFSense..try a different version.
Also Smoothwall has some basic per user graphs.
 
"Per User" summary in the reports of Untangle.

Also agree, BandwidthD is a great add-on for PFSense..try a different version.
Also Smoothwall has some basic per user graphs.

When I go to the user reports there is nothing listed is there something special I need to do?
 
That what the other one was! I couldn't remember off the top of my head what the other package was. As I recall wasn't Darkstat a little primitive in the reporting? I gotta look at my PFSense box again. I set it up a few years ago, and haven't really touched it in months.
 
When I go to the user reports there is nothing listed is there something special I need to do?

I'll have to get to a box at a clients where I have the web filters loaded....ours at the office I don't have the web filters enabled. However....we have the Bandwidth Control module running...which is a "pay for" feature....and it does give total bandwidth consumed per user and/or per IP. And even breaks that bandwidth down into various ports.
 
I'll echo the pfsense with BandwidthD and even Darkstat.

Also as someone mentioned stick with v1.2.3. If you want 2.0 then find a STABLE release (when I say stable I mean one where everything actually works, not necessarily what pfsense calls "stable") and stick with it. Do not do any of the inplace updates. I'll be rebuilding my 2.0 box again this week after I was tempted on the forum to update to a recent build. After doing so snort and country block no longer works. My hosted forums have been getting spammed now every day because of it. Not the first time this has happened. Daily updates are known to break packages. I should have known better after the first time.

off topic
Country Block = gold!
/off topic
 
I'll try pfsense 1.2.3 again but last time I tried it I couldn't get xbox live to have open nat. I tried everything (forwarding the proper ports, turning on upnp, other solutions suggested by google searching and forum inquiries), even put the xbox on its own nic and made it a dmz still didn't work. Took forever to find games.

After researching I found there were a few others in my position that had the same problem that no matter what you did xbox live always said moderate nat. Someone told me that it has to do with the way pfsense routes traffic if you have 2 xboxs on your network. (I have no idea if this is true, but we have 2, but only one has xbox live subscription). I switched to clearOS and didn't have any problems with xbox live.
 
I still haven't found a decent bandwidth monitor. Bandwidthd makes no sense to me. If a monitor can do day-to-day breakdowns, it doesn't do it by client. If a monitor does it by client, it can't do any kind of time breakdowns. I am pulling my hair out about this. I need to be able to tell a specific person how much bandwidth they used on any day, to tell them they've been using too much, otherwise I really have no proof. And they all seem to not clear out and start a new set of data by the month. I am about to go back to my Asus routher with DD-WRT on it, as it was very easy to upload data from the router and disseminate it into a day-to-day breakdown for each client. There was just no graphs.
 
Take a look at ntop too.

Personally I have had no issues breaking down my BandwidthD data for a daily, hourly, weekly, or monthly set per user, but that's just me.

bandwidthd.jpg


I get my daily breakdown of my top 20 consumers, I can click on each IP which then also gives me the DNS name of the PC, and then gives me an hourly breakdown of their data usage by protocol. For free BandwidthD and Ntop are hard to beat without a whole bunch of setup and administration time to pull reports.
 
I found that BandwithD wasn't very accurate.

How many users are we talking how are your switches setup are they managed?
 
I found that BandwithD wasn't very accurate.

How many users are we talking how are your switches setup are they managed?

BandwidthD is running on my pfSense 2.0 Release box. I have about 25 devices on this network at any given time, my switches are a Dell Powerconnect 5324 and a 2816. I have found that while it may not be accurate to the KB, its good enough to figure out where my hogs are. If I want real enterprise class network monitoring I go to orion from solarwinds.
 
I don't like the way ANY of these monitors report. The daily number is not the amount used that day, it seems to be an average, which is retarded. You should be able to choose a day, and see how much bandwidth was used, per client, on that day. The same should apply to the month. NTOP was way too cluster-fucked for me, and still didn't do much of anything. I don't even want to see a breakdown of each protocol. Smoothwall looked somewhat interesting with ipacsum, but that's a command line deal, and that is too much effort, to have to enter the date and time range, then the client, among other options. No thanks.
 
Even SNMP is not 100% on the nose with how much data is used its just snap shots. You would need maybe a Sonicwall , Firebox or some other type of gateway and they are just too expensive really.

You can't even get up to the second data from the CLI in cisco routers.

I have seen peopel talk about vnstat but I can't say I have used it.
 
I don't like the way ANY of these monitors report. The daily number is not the amount used that day, it seems to be an average, which is retarded. You should be able to choose a day, and see how much bandwidth was used, per client, on that day. The same should apply to the month. NTOP was way too cluster-fucked for me, and still didn't do much of anything. I don't even want to see a breakdown of each protocol. Smoothwall looked somewhat interesting with ipacsum, but that's a command line deal, and that is too much effort, to have to enter the date and time range, then the client, among other options. No thanks.

Yes BandwidthD does average, however if you actually look at the specific client's stats graph you can get an accurate breakdown of what they are doing that day at what time. NTOP does the same thing (though does require a lot of extra setup).

I too used to have DD-WRT dump my daily usage into a MySQL DB which made nice pretty reports, but to be honest it was a pain to setup, and it didn't really offer much over BandwidthD or any of the others (maybe being easier to read?). If I want real time traffic graphs and don't want to pay for it I can use solar winds free network monitor. SNMP based, just point it at my switches, and select the ports I want it to report.
 
Even SNMP is not 100% on the nose with how much data is used its just snap shots. You would need maybe a Sonicwall , Firebox or some other type of gateway and they are just too expensive really.

You can't even get up to the second data from the CLI in cisco routers.

I have seen peopel talk about vnstat but I can't say I have used it.

Eh I personally don't find that my Sonicwalls running 5.8.1.x are that much more accurate then my bandwidthd + pfsense box. However the graphs are a lot prettier and do break things down a lot better.
 
Yes BandwidthD does average, however if you actually look at the specific client's stats graph you can get an accurate breakdown of what they are doing that day at what time. NTOP does the same thing (though does require a lot of extra setup).

I too used to have DD-WRT dump my daily usage into a MySQL DB which made nice pretty reports, but to be honest it was a pain to setup, and it didn't really offer much over BandwidthD or any of the others (maybe being easier to read?). If I want real time traffic graphs and don't want to pay for it I can use solar winds free network monitor. SNMP based, just point it at my switches, and select the ports I want it to report.

Yeah, it shows a graphical representation of what someone did. I need actual numbers. I had DD-WRT set up to upload away the information, but as you said, that's a pain in the ass. I just don't see why there hasn't been more people needing the same thing that I need.
 
Yeah, it shows a graphical representation of what someone did. I need actual numbers. I had DD-WRT set up to upload away the information, but as you said, that's a pain in the ass. I just don't see why there hasn't been more people needing the same thing that I need.

They exist, they just aren't free. For instance if you were to look at something like Solar Winds Orion NPM, ManageEngine's NetFlow, Websense, etc you are looking at starting around $3,000 a year for 100 endpoints. You could look at doing something like a cymphonics box that would show you what you want and do content filtering / bandwidth management, but again they aren't cheap (Start at $2k and jump very quickly to the $20k range).
 
That's stupid, for something that can be so simply integrated into something like bandwidthd. Again, I do not care about protocols, so that makes the problem even less complex. Renting software is for chumps, even if it is a business renting it.
 
We use Webroot proxy for all our offices all over the world and it blocks user who hit 1GB in 24 hours. It gives us a very good way to monitor people who are in the middle of Africa or the frozen roads of Alaska. Its not cheap though!

As for the comment above me... :rolleyes:
 
As for the comment above me... :rolleyes:

Do you have a problem with what I said? I'd pay a small amount for something to do what I want. Enterprise stuff is a rip-off. It does not cost that much, they just jack up prices because companies will pay whatever the cost is. And people wonder why inflation is running out of control. Why would I pay a good portion of my yearly earnings towards something I wouldn't even own? I think everyone is getting the impression that this is for some kind of business.
 
If you want that much accuracy, you need to take flow logs (NetFlow). There are 3 ways you can do that on pfSense. If you have a pretty simple design, you can use Ntop or pfflowd in the package manager or if you have a more complex (multi-LAN or WAN) install softflowd (via pkg-add) and record per interface.

From there you need to send the data to another server that actually breaks it down for you. There are many tools but what I mostly use is FlowViewer/FlowGrapher/FlowTracker

If you want more options, see the "Open Source -> NetFlow" section on this page: http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
 
What accuracy? I don't see how bandwidthd isn't accurate enough. It takes count of how much data is used every 2.5 minutes. It perfectly counts the data, the front-end for it is just stupid. I don't need crazy second-by-second breakdowns with crazy protocols. I also don't want to build another box just for something that can logically be accomplished on the router. If bandwidthd didn't aggregate to the log each time, it would make things easier to read. If it just modified the values for the current day, and only aggregated daily, that would be ideal, and save space.

NTOP sucks. When I could even find how much a client used, it still didn't break it down into a window. It's bloated compared to what I need. I don't even want to have to enter all the info like that flowviewer thing.

DD-WRT graphed the bandwidth almost ideally. In a nice graph with daily bars. You could go to any month, and hover over the days, and see how much was sent and received. It just didn't break it down per client. I also disliked how it didn't total the up and down traffic. If I could get something like WRTBWMON for pfSense, I would just go back to sending my data to my hosted server.
 
What accuracy? I don't see how bandwidthd isn't accurate enough. It takes count of how much data is used every 2.5 minutes. It perfectly counts the data, the front-end for it is just stupid. I don't need crazy second-by-second breakdowns with crazy protocols. I also don't want to build another box just for something that can logically be accomplished on the router. If bandwidthd didn't aggregate to the log each time, it would make things easier to read. If it just modified the values for the current day, and only aggregated daily, that would be ideal, and save space.
.

I feel like if you found something that you half like, that would be easy to fix, you may want to take it to the developers of that open source software. They (and their forum, or pfSense's forum) can probably help you a lot better then we can.

EDIT: They may also be able to tell you why that software doesn't exist (for instance copyright or patent held by one of the big boys that doesn't allow anyone else to graphically display results they way you want them to be).
 
Last edited:
Do you have a problem with what I said? I'd pay a small amount for something to do what I want. Enterprise stuff is a rip-off. It does not cost that much, they just jack up prices because companies will pay whatever the cost is. And people wonder why inflation is running out of control. Why would I pay a good portion of my yearly earnings towards something I wouldn't even own? I think everyone is getting the impression that this is for some kind of business.

Well if you had said it wasn't for business we wouldn't have got that impression. People here are trying to help you so there isn't really any need for a bad attiude. Good luck with your search.
 
I'm with C7. ntop is excellent for this purpose. If you want to play with it, it's on the BackTrack LiveCD. I feed it a mirrored port off my primary switch, and I get all the data I need toward this end.
 
Back
Top