Looking for solution to minimize bandwidth

Discussion in 'Networking & Security' started by noremacyug, Sep 11, 2018.

  1. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    So, I live in an area that doesn't have dsl or cable services. My current service is ATT Fixed Wireless which is quite good minus the data cap of 340Gb/month with a $10/50Gb overage fee. I'm wanting a solution that will limit bandwidth for my network. For instance if anyone starts watching YouTube they'll be limited to 480p or 720p. Stripping ads away would be welcomed as well. Just want to stretch my data as far as I can, what are my options?
     
  2. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    Bandwidth limiting can be done via QoS; most consumer routers are pretty good at this, and perhaps better than 'enterprise' equipment (FOSS or other), due to development for the target market.

    I'd start with a consumer ASUS or Netgear router and look for reviews to find one that fits your criteria. If you don't find one, I can suggest some commercial (or so) solutions.

    For blocking ads, the easiest solution is pi-hole. You can run it in a VM or a Raspberry Pi easily, just note that it needs to be on 24/7 to be effective, and it does prevent the downloading of ads outright. Do note that there are some functions blocked by pihole that may be undesirable, like ad links in google searches.
     
  3. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    I suppose I can buy another edgerouter if that's my best bet. I was hoping for some sort of lightweight proxy server that would run on a Pi or something.
     
  4. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    Not sure if an ER is the best case- it's a good case, but the per-application limiting is the hard part. A Pi wouldn't be an improvement for that purpose, though fitting for pihole which is just DNS.

    And budget is also a consideration. The venerable ER-X goes for ~US$50 usually/on sale, but it won't be doing anything other than firewalling, routing, and perhaps basic QoS.

    This is why I suggested a consumer product; these can be faster for your specific use case, to which they are tailored, while also being easier to set up. The pihole can certainly be easily added if still needed.
     
  5. tedych

    tedych [H]Lite

    Messages:
    89
    Joined:
    Jan 18, 2013
    DNS based ad blocking is not a viable alternative to other more effective solutions like ad-blocking add-ons on users' browsers. I'd use that.
    I'd also use a router capable of the other things. Something like Mikrotik (or even maybe your ER) can count the traffic and do something when the cap is to be reached soon (via scripting). You can limit bandwidth to certain users in the network who are not too gentle on their use of the connection.
    But even a single Win10 machine could exhaust many gigabytes if allowed to auto-update itself which can be a pain. WIn10 has something called metered connection etc but..
     
  6. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    Would the EdgeRouterLite DPI allow site specific throttling? For instance have it throttle YouTube, but not Facebook.

    IdiotInCharge what solution do you recommend? In general I'm not a fan of box store routers, hence my movement in the past to Ubiquiti gear. However currently I'm just using the supplied ISP router and a couple TPLink pieces of gear to expand wifi.
     
  7. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    I'll double check W10 settings. But honestly I suppose I don't mind it if it updates. We ran through about 1400Gb last month. A lot of that was via usenet but then a lot of streaming as well. Just trying to avoid another $265 internet bill.
     
  8. Mr. Baz

    Mr. Baz 2[H]4U

    Messages:
    2,796
    Joined:
    Aug 17, 2001
    What? DNSBL with good lists is WAY more effective than browser-based ad blocking. Bonus feature -- you don't have to do a darn thing on ANY endpoint devices. I would not recommend a MicroTik anything to my worse enemy...well OK yeah to my enemy, but not my friends.


    I think you answered your own question right there -- stop using usenet. Setting bandwidth limits will also help. The level of fidelity you want in that will be hugely based on how much money you want to spend on a new firewall/router.
     
    IdiotInCharge likes this.
  9. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004

    Yeah, I've put a halt to usenet for now but I'd still like to limit YouTube traffic and directvnow. Money wise, cheap as possible.
     
  10. tedych

    tedych [H]Lite

    Messages:
    89
    Joined:
    Jan 18, 2013
    I wouldn't speak so easily.
    Of course central ad blocking is better than doing it individually on users machines. But this requires L7 filtering and analyzing which is Very taxing on resources especially if you have a fast connection (which is not the case here though).
    DNS is not an option (for me) except some edge cases or malicious sites.
    Most of sites I visit or will visit have ads (or "taxing" content like flash clips or anything within the domain) from different sources, including own domain in subfolders etc. Ad-blocking is far from only blocking a domain or two or thousand. I will definitely not block the domain I use daily but will block from 10 to 80% of ads it contains.
    Ad-blocking that happens right at the core, in the browser, that already works with URLs, is way more versatile and flexible.

    As to mikrotik, this is moot. No idea what they've done to you but their gear is working perfectly, especially if we compare to most other "home" crap. For all people I know who have one.
     
    Last edited: Sep 12, 2018
  11. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    You're missing something.

    Yes, use a browser-based ad blocker!

    However, also use a DNS filter like pihole. Especially when bandwidth limitations are in place. Every ad that pihole 'blocks' does not cross the edge in the first place; the request for the ad is simply dropped at the DNS level. It never leaves the network. Browser ad blockers may be used for what a DNS ad blocker misses, but they require the ad to be downloaded, and that requires bandwidth.

    The OP wants to limit bandwidth usage, and a DNS-based network ad blocker is a simple way to do just that.
     
  12. tedych

    tedych [H]Lite

    Messages:
    89
    Joined:
    Jan 18, 2013
    No.
    ublock origin for example, does block the request to the resource. When a page requested some uri via for example <img>, and it matches a rule, it is not requested.
    You can block entire domains this way and whole subdomains or wildcards etc.
    Central blocking of dns in addition to browser adblockers is always better but not especially necessary if all users in the lan are trustworthy enough to at least not mess with browser addons settings. But the point was that this is very taxing the router or the edge device depending on what is in place.
     
  13. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    It runs on a Raspberry Pi. It's not taxing.
     
  14. tedych

    tedych [H]Lite

    Messages:
    89
    Joined:
    Jan 18, 2013
    Oh, I saw now how it works. All machines in the network should point their DNS setting to this machine (Pi in this case) where pihole is installed.
    Crap.
    To delegate such important function to a pi device....
    I meant if we perform L7 filtering on a router device which... by default is meant to be the reliable device in the network. And L7 filtering is more versatile than simple DNS blocking.
    Everything this pihole can do, can be done on users machines with reliable ad blockers. If he can put aside a pi device with pihole installed and try to rely on its DNS for all LAN's requests, then Ok. I wouldn't rely on a Pi device for critical functions of my network.
     
  15. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    I'd recommend throwing it into a server of some sort, even an 'always on' laptop works. Individual resources and needs apply honestly, but it's quite effective while being low-weight.
     
  16. tedych

    tedych [H]Lite

    Messages:
    89
    Joined:
    Jan 18, 2013
    If he has a server to delegate as a router/blocker/dns etc. it's absolutely Ok, if he can rely on it 24/7.
    There are many approaches to OP's task. For someone a Pi/laptop/server_machine performing most LAN-central functions like routing (pfSense?!)/blocking/DNS/NTP etc. can be viable. For another one a compact device like mikrotik or edgerouter, or even very small pfSense mini computer is all they need plus more responsibility delegated to users machines.
     
    IdiotInCharge likes this.
  17. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    I do have an always on server. What's the best option now? Any software that can run on it to shape traffic and block ads?
     
  18. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    Traffic shaping is much harder than blocking ads, in terms of hardware- the software you can get for free, as mentioned above (pfsense and other router distributions).

    You can get cheap QOTOM minicomputers, fanless even, that can handle pfsense which can do the QoS (traffic shaping) as well as the ad blocking and many other things.

    Recommend tossing one into a VM to take a look!
     
  19. noremacyug

    noremacyug [H]ard|Gawd

    Messages:
    1,564
    Joined:
    Apr 15, 2004
    The edge router lite did traffic shaping beautifully (port based). Eventually I moved from that to their smart queue qos as it worked well. But the smart queue would still allow maximum bandwidth if available to a YouTube stream. However ERL traffic shaping wouldn't help me with YouTube streams without limiting all other traffic on port 80/443. I suppose I need to read up on their DPI to see if it will allow more granular control.

    I'll look into a cheap pc to see about pfsense or smoothwall. I thought I saw something about openwrt on a Pi, any experience with that? I also have a old netgear router (wndr3800 I think) that I used to run tomato on somewhere. Perhaps I just need to put some effort into finding it.
     
  20. IdiotInCharge

    IdiotInCharge Not the Idiot YOU are Looking for

    Messages:
    6,928
    Joined:
    Jun 13, 2003
    I wouldn't use a Pi for anything other than pihole for DNS in the network stack- and at that point, I don't use a Pi as I already have excess hardware. DNS is just not an intensive thing. DHCP is similar in terms of resource usage, but it's usually easier to keep that on the router, and any type of filtering/firewalling/routing/queuing you'll want on more robust hardware, i.e., something that has at least two real gigabit ethernet interfaces, which the Pi does not have one of (it's a hardwired USB adapter).

    With the Edgerouters, you're simply not going to get the needed granularity, at least not easily- and probably not adaptably over time. That's why I mentioned a consumer router, as those are developed with consumer usage and consumer apps as a target. Many times they're simply more appropriate, supposing you find one that fits your needs and is well built. From what I've seen, ASUS is shipping a WRT spin on theirs now too.

    As for cheap PCs, just digging up the QOTOM stuff on Amazon will get you an idea. I have one, but it's for tinkering as much as usage, and it's just me; I wouldn't be messing with it if I had to keep others fed with internet access too.
     
  21. Private_Ops

    Private_Ops [H]ard|Gawd

    Messages:
    1,806
    Joined:
    Jun 4, 2007
    I have my pfsense box setup to use pfblockerNG. Works great.