Ubiquity Edge Router ERPoe‑5 Any Good?

Zarathustra[H]

Zarathustra[H]

Extremely [H]
Joined
Oct 29, 2000
Messages
39,015
Hey,

Has anyone used Ubiquiti's edge routers in general, or specifically their 5 port POE model?

For a long time I have used pfSense on my ESXi box as my firewall and router (with a passed through WAN Ethernet adapter, to minimize security implications).

While I really like pfSense, lately, this has really become an annoyance. Every time I need to take the server down for maintenance, my internet goes down as well, making it more difficult to download patches, look up guides, go on forums for questions, etc. when I am in the middle of my projects.

I have considered building a dedicated pfSense box using spare hardware I have kicking around, but I feel like even with the low power stuff I have, this would be a significant step up in power use, which I am trying to avoid.

I'd also hate to move back down to consumer hardware.

Because of this, and because of my appreciation of enterprise networking hardware, I have started thinking about getting a dedicated edge router. Because I already use - and am very happy with - Ubiquiti Unifi products for my WiFi, and they tend to be reasonably priced, they were the first ones I looked at.

If I was going to go with a Unifi router, I figured I might as well go with one that has PoE so I can power my current Unifi AP from it, declutter a little bit by getting rid of the PoE adapter, and hopefully save a little power, by having them powered from the same source. If I was going to go with PoE also wanted to have at least one PoE port open for future expansion, which is why I started looking at the ERPoe-5, which is fairly reasonable at ~$150

Does anyone have any experience with these? It seems like they might integrate nicely with the Unifi controller software, and I've been very impressed with the only other products of theirs I've used thus far, so if it is any indication of how their edge routers perform, I'm sure I'd be happy with them as well?

Any comments regarding these?

Are there any other reasonably priced dedicated routers I should be looking at? (they don't have to have PoE)

I'd appreciate any thoughts on this subject matter. One thing I've found to be kind of tricky with pfSense is to get traffic shaping set up the way I like it. The queues and firewall rules get a little complex and confusing. If there is any product that simplifies this process, it would be a big plus.

--Matt
 
I've been using it for about two years. Great little piece of hardware, if you're not against CLI.
The webUI wizards have gotten better though.

It can't do Unifi controller natively. (Edit this might have changed, seeing mixed things on there forum)

* If you want that you need to get the Unifi Gateway. But seeing as it's just VyOS underneath there have been people that have experimentally hacked in the Unifi controller. *

Version 1.8 of the firmware was released about a month a go, much better wizards with more features.
 
Ruffy said:
I've been using it for about two years. Great little piece of hardware, if you're not against CLI.
The webUI wizards have gotten better though.

It can't do Unifi controller natively. (Edit this might have changed, seeing mixed things on there forum)

* If you want that you need to get the Unifi Gateway. But seeing as it's just VyOS underneath there have been people that have experimentally hacked in the Unifi controller. *

Version 1.8 of the firmware was released about a month a go, much better wizards with more features.
Click to expand...


Ahh,

I am typically very comfortable with CLI, but I have no familiarity with VyOS at all.

Yeah, I just made this discovery as well, I assumed that EdgeOS would somehow feed information to the latest version of the Unifi Controller so that it could be monitored all in one place. I did not realize that there was a separate USG device. Since the USG is just based on a repackaged EdgeRouter lite, it would be nice if there were a USG5-Poe unit.

Hmm. if I'm not going to get Unifi integration with the Edge Router line, maybe I should consider something else.

I've also heard good things about Mikrotik's routers, especially in the ease of traffic shaping.

They seem to be lower power use, and they have a $179 model that has 10x Gigabit ports in two switch groups AND poe...
 
Last edited:
Thanks for that info. I went ahead and ordered one last night, based in large part on your favorable experience with the Ubnt unit, and your thoughts on MikroTik. I'm looking forward to configuring it.

A couple of questions:

1.) I understand only the last three ports are switched, but if you want to use 4 internal LAN ports and one WAN port you can bridge eth1, with the switched ones (eth2, eth3 and eth4). Do you know if this results in reduced performance between eth1 and the switched ports? Bridges don't have the best reputation when it comes to performance.

2.) There appears to be a section in the Router/DHCP configuration where you can enter the address/ip of your Unifi controller. Do you know what this accomplishes? Do you get some level (albeit more limited than the USG device) of cooperation between the Unifi controller and EdgeOS?

Thanks again,
Matt
 
Running 3 Edgerouters 2x POE and 1 lite, and very happy for them :)

2.) The unfi controller ip is only for pushing out this information to clients eg. AP's and other Unifi equipment.
 
diizzy said:
Have a look at anything based on Mediatek MT7621, great SoC and good speed.

Gmarket - [Netis] WF2881／GIGA／Wi-Fi／Internet／Wired And Wireless／... (eBay in Korea)
Alibaba Manufacturer Directory - Suppliers, Manufacturers, Exporters & Importers (also has SATA)

All running OpenWRT :)
Click to expand...

No thanks!

Zarathustra[H] said:
Thanks for that info. I went ahead and ordered one last night, based in large part on your favorable experience with the Ubnt unit, and your thoughts on MikroTik. I'm looking forward to configuring it.

A couple of questions:

1.) I understand only the last three ports are switched, but if you want to use 4 internal LAN ports and one WAN port you can bridge eth1, with the switched ones (eth2, eth3 and eth4). Do you know if this results in reduced performance between eth1 and the switched ports? Bridges don't have the best reputation when it comes to performance.

2.) There appears to be a section in the Router/DHCP configuration where you can enter the address/ip of your Unifi controller. Do you know what this accomplishes? Do you get some level (albeit more limited than the USG device) of cooperation between the Unifi controller and EdgeOS?

Thanks again,
Matt
Click to expand...

rma is correct. This is only to forward the controller IP to the equipment that uses this info. Hopefully, in the near future, we will be able to manage the EdgeOS devices from the Unifi controller. I don't use EdgeOS devices primarily, but I have a few clients that would benefit from this.
 
Huge improvements on the Ubiquity backend. I have a Edgerouter POE and I've on firmware 1.6 . I didn't upgrade due to the fq_codel implementation isn't as good in the 1.8 firmware as 1.6 + fq_codel wizard. But Firmware 1.9 appears to be focusing on polish and bugfixs, so that's probably the next one I'lll update to.

Edgerouter firmwares before 1.5 are crap.
 
Mackintire said:
Huge improvements on the Ubiquity backend. I have a Edgerouter POE and I've on firmware 1.6 . I didn't upgrade due to the fq_codel implementation isn't as good in the 1.8 firmware as 1.6 + fq_codel wizard. But Firmware 1.9 appears to be focusing on polish and bugfixs, so that's probably the next one I'lll update to.

Edgerouter firmwares before 1.5 are crap.
Click to expand...

I just installed 1.8 when I took it out of the box.

I like it thus far, but it does have one shortcoming that bugs me quite a lot. QoS is not offloaded in hardware, and the MIPS CPU in the router is only powerful enough to handle QoS up to about 60Mbit/s (half duplex), so 60mbit down, 60mbit up or 30/30...

This is kind of a huge disappointment. My FiOS benches 160/160 which means I can't use it for QoS at all :(
 
The CPU is the same old Cavium 500Mhz dual core as on the EdgeRouter Lite 3 which isn't going to win any speed records if it doesn't use hardware acceleration which I've mentioned before and ppl have been very fast to tell me that it's the greatest thing since sliced bread and I'm just being negative/UBNT hater...

The MT7621 (ER-X and other boxes) is faster, pretty sure it does at least 140-150 using CoDeL or at least that's what users report on OpenWRT at least on similar hardware. If you have that kind of connection you don't really need any QoS at all however....

I have a MT7621 board here that I could give it a go on if you want to, I think I can give the old ERL (3-port) boxes a spin too but I have no idea of OpenWRT works on the 5 port boxes if you're interested.
 
diizzy said:
I have a MT7621 board here that I could give it a go on if you want to, I think I can give the old ERL (3-port) boxes a spin too but I have no idea of OpenWRT works on the 5 port boxes if you're interested.
Click to expand...

Hmm. I hadn't even considered putting a third party OS on it. I'd imagine controlling the static PoE ports could be problematic.

I tend to do a lot with my connection, so having QoS would be nice, so I don't have to micromanage and throttle everything by hand.

I wonder if it is possible to offload the QoS but it just isn't implemented in the software, or if the hardware is incapable of this all together.
 
160/160 and you want to QoS?, what are you running?.

for torrents just set a 100/100 limit in the client, and everything would be fine.
One of my ERpoe are mounted on a 400/400 Fiber connection, and we don't do any QoS, other then on a torrent client limited to 200/200, been running for 2 years now, before that we restartet the old router every night to prevent it for jamming up.
 
rma said:
160/160 and you want to QoS?, what are you running?.

for torrents just set a 100/100 limit in the client, and everything would be fine.
One of my ERpoe are mounted on a 400/400 Fiber connection, and we don't do any QoS, other then on a torrent client limited to 200/200, been running for 2 years now, before that we restartet the old router every night to prevent it for jamming up.
Click to expand...

Well, that's part of the problem. It's a shared connection, and I don't necessarily control what is running all the time. Sometimes it is fine, other times people complain that their streams (Netflix or whatever) are problematic. Could be a torrent surge, or could be a 20mb/s Steam download. I try to encourage good civil behavior on the network, but it would be easier to just set and forget QoS.
 
  • Like
Reactions: rma
like this
Zarathustra[H] said:
I just installed 1.8 when I took it out of the box.

I like it thus far, but it does have one shortcoming that bugs me quite a lot. QoS is not offloaded in hardware, and the MIPS CPU in the router is only powerful enough to handle QoS up to about 60Mbit/s (half duplex), so 60mbit down, 60mbit up or 30/30...

This is kind of a huge disappointment. My FiOS benches 160/160 which means I can't use it for QoS at all :(
Click to expand...

On the edgerouter lite CPU it maxes out on a single policy fq_codel shaper at 80Mbps = that's up/down combined or 70/10 or 0/80. Most user's that have pipes of >50 Mbit are just applying fq_codel to their upload only. The problem is the CPU binaries used for acceleration are closed source. There's a possibility that BQL will be implemented but that will just raise the ceiling, not actually give you offload.

The REAL Edgerouter can run fq_codel at around 150Mbps Ubiquiti EdgeRouter Router - 1 Gbps - Gigabit Ethernet


The Edgerouter pro tops out around 220Mbps for fq_codel traffic flows Ubiquiti EdgeRouter Pro Router - 1 Gbps - Gigabit Ethernet
 
With what you are trying to do.... Using Ubiquiti equipment, I'd suggest the following:

Because you can not QOS 160Mbps on an edgerouter list or edgerouter POE with fq_codel... but you may be able to do something using a limiter that would work as RMA suggested.

My gut reaction is to suggest you buy an Edgerouter Pro:

Implement fq_codel on upload only to your WAN. fq_codel is a QOS type designed to alleviate buffer bloat issues that create additional latency and is quite CPU intensive.

Use the basic traffic shaper on the other clients as shown here: EdgeMAX - Set traffic policies for upload, download and VoIP
 
That's not really a fix, you most likely need at least an ARM platform (Marvell/QCA) or even better (x86) to use fq_codel.
 
There are many users whom want a refresh for the edgerouter line, if that occurred I m guessing that the edgerouter lite v2 would be equal to the performance of the current Edgerouter, with a more route-able ports.
 
That's not going to happen if you want the UBNT branding and the price to stay the same.
 
It'll still sell but sane ppl will look at other hardware/products. The 11ac products haven't recieved that much of praise and from what I heard more ppl are looking at Mikrotiks APs as they seem have performance better overall but I haven't used them myself.
 
So, I've decided to return the Ubiquiti router and eat the restocking fee.

I'm going to build a bare metal pfSense box instead using - as much as possible - parts I have kicking around, adding a few I need to order.

I'm going to go overkill, to make sure I have enough horsepower for all the QoS i want to do, and to make sure it can have other uses if I ever stop using it as a router in the future.

This is what I'm thinking:
  • Supermicro Socket 1150 server board (with dual Intel 210 gigabit LAN)
  • 35W TDP dual Core i5-4570T with HT disabled for power savings. Base clock 2.9, turbo 3.6 (for pfSense fast cores are better than more cores)
  • Whatever DDR3 SO-DIMMS I can find in my parts bin (will probably wind up being 2x2GB of Laptop RAM left over from an 8GB upgrade a while back, can't remember)
  • Sandisk USB drive as only disk to run off of.
  • Mini-Box M350 chassis and 60W pico-PSU

I'm hoping to get under 15W at the wall with this setup. I might undervolt and underclock a bit if needed.

Now my only problem is that my RMA expires before I can get all the parts here :p I'll have to figure out a temporary router :p

I probably went WAY overkill on the CPU, but I found a good deal on it on eBay, and it's still a 35W part.
 
I did some more research, an ARM platform would have handled this just fine been much cheaper but oh well... ;-)
Kinda waste going for a server/workstation board when you're using non ECC-memory and CPU.
Please consider a small HDD or something rather than a USB stick, it's much more reliable.
 
diizzy said:
I did some more research, an ARM platform would have handled this just fine been much cheaper but oh well... ;-)
Click to expand...

Routing is a great application for ARM, but I am not aware of any good routing distributions for that. Last I checked pfSense, smoothwall, Sophos, all the big ones are x86/AMD64 only. I don't really want to use DD-WRT which really comes across as flaky and hackish when you go through their webpages, wikis, etc. It is great to get more functionality and performance out of a consumer router you already have, like back in the day on my first Linksys Router, but I wouldn't purpose build a system for it.

Besides, this way, if I later decide to use it for something else, I can, and not be stuck with an ARM based paperweight.

diizzy said:
Kinda waste going for a server/workstation board when you're using non ECC-memory and CPU.
Click to expand...

I was originally going to go with one of the Haswell Celerons or Pentiums that support ECC, but surprisingly enough, after pulling the trigger on the Supermicro X10SLV-Q, I found that the motherboard doesn't actually support ECC. (Oops). Truth is, routing doesn't really need ECC. In the unlikely event of a flipped bit in the short period of time the packets pass through the RAM of the router, the error checking in TCP will discover and request a repeat of the packet. Then one might say, what about UDP packets which don't have any error correction? Well, it's more likely something else is between my network and the source is the point of failure in that case, than the nanoseconds worth of time that packets touches the router RAM. ECC is mostly a waste on routers in non-mission critical settings.

The main reason for the Supermicro board was the dual Intel gigabit ports. There arent too many consumer boards that have this, especially in Mini-ITX. I'm not about to run my router on non-intel NIC's.

Once I had a motherboard coming my way that didn't support ECC, I did some poking on eBay and found the i5-4570T for the same price as the 35W Celerons and Pentiums were going for, so I pulled the trigger.

diizzy said:
Please consider a small HDD or something rather than a USB stick, it's much more reliable.
Click to expand...

Nah, USB is plenty stable enough. You'll find that booting servers off of USB is actually a very common practice even in production systems. This is the official recommended approach - for instance - for FreeNAS - and most of our members over in the ESXi threads are booting their hypervisors off of USB sticks. For something as lightweight as pfSense it's actually a pretty good match, and on a modern good USB3 stick, like the Sandisk one I plan on using you get fantastic read and write speeds too.

Besides, this way I can completely disable the onboard SATA and - hopefully - save some power.

If I absolutely need to I can always grab one of my old 8GB Super Talent SATA SSD's from my spare parts bin, but I'd rather not, as it probably would use more power, and probably not perform as well.

pfSense only uses less than a gig of space installed, so write cycles arent a concern either, as the $8 USB stick I bought is 16GB, so it will be plenty overprovisioned.
 
Last edited:
  • Like
Reactions: rma
like this
OpenWRT works great and is also used in commercial products.

No, USB isn't common practice. iSCSI is however ;-)
 
diizzy said:
OpenWRT works great and is also used in commercial products.

No, USB isn't common practice. iSCSI is however ;-)
Click to expand...

FreeNAS recommends booting off of USB in their official documentation.

Many people boot their ESXi servers off of USB sticks, and use iSCSI as a datastore.

Which for the record makes no sense to me, but not because of the USB part.

iSCSI is just about the worst way to have remote storage. I'd use NFS over it 10 out of 10 times.

iSCSI tends to force async writes, which leaves you open to data corruption. It also requires you to use disk images, so you constantly have to estimate how much space you'll need in advance, or create sparse images, which grow based on disk activity and are messy. It's also very inconvenient because you can only access the data from one place at a time.

Honestly, I don't understand why anyone uses iSCSI. It is terrible. NFS is a better protocol for networked storage in every way shape and form, IMHO.
 
Last edited:
1. They do not...
1. Introduction — FreeNAS User Guide 9.3 Table of Contents <-- Nothing in favour of USB as far as I can tell
Hardware recommendations (read this first) | FreeNAS Community <-- This is not official but it's sane

2. That doesn't make a good solution.... I'm also pretty sure that it's not really recommended

3. Yes, You Can Virtualize FreeNAS - FreeNAS - Open Source Storage Operating System <-- Sounds like virtualization itself is somewhat discouraged which I can see why.

Yes, you can run ESXi Embedded but most seems to have found out that it's not all the reliable.
 
diizzy said:
1. They do not...
1. Introduction — FreeNAS User Guide 9.3 Table of Contents <-- Nothing in favour of USB as far as I can tell
Hardware recommendations (read this first) | FreeNAS Community <-- This is not official but it's sane
Click to expand...

Interesting. Looks like the recommendations have changed in the more recent documentation. In the old official guide the recommendation was to boot it off of USB so you could use all your free SATA/SAS ports for storage rather than wasting a port on a OS drive.

diizzy said:
3. Yes, You Can Virtualize FreeNAS - FreeNAS - Open Source Storage Operating System <-- Sounds like virtualization itself is somewhat discouraged which I can see why.
Click to expand...

I don't recall us talking about this, but yes, the FreeNAS team has alwasy been extremely conservative when it comes to virtualization of any kind, despite the fact that the community ahs been doing it reliably since the original launch of ESXi 5. You usually only get into trouble if you try doing stupid things, but virtualizing it on a known stable passed through SAS controller is a pretty much flawless approach.

diizzy said:
Yes, you can run ESXi Embedded but most seems to have found out that it's not all the reliable.
Click to expand...

Interesting. Could be that my information on this subject is somewhat outdated then, as I usually only read up on the latest and greatest when I have a need to change something, in advance of a project or something like that. Last time I did a major overhaul of the server setup was in late 2014.

I'm still going to start by booting pfSense from USB and see how it goes. If I see instability I may enable the SATA inferface in bios and boot from one of my old Supertalent 8GB drives instead.
 
I fail to see why you'd run ESXi now that you have bhyve but oh well.. It's no my data...
 
diizzy said:
I fail to see why you'd run ESXi now that you have bhyve but oh well.. It's no my data...
Click to expand...

I'm not that familiar with bhyve, but the whole purpose of this effort was to split my router off from my VM Server so that I can take the server down, maintain it, without the internet going down. I was also motivated from the security concerns involved with a consolidated server as a edge router device. I had passed through a dual port adapter in order to try to isolate it as much as possible, but still it is usually considered not a very good idea to have your NAS, etc. etc. stored on your edge device.

With that in mind, it was supposed to be either a enterprise-type hardware router or a dedicated appliance build, relatively cheap and low power. I've definitely blown through my budget target, and possibly blown through my power target as well, but this should work.


All that being said, the plan IS to get rid of ESXi. I am so tired of their shit. I'm planning on migrating the entire server over to a mix of KVM/LXC containers in the near future. I may use Proxmox, or I may spin my own based on an Ubuntu or Debian server install. Haven't decided yet.
 
Unless you run a lot of services you can run it all on one box, do jails if you're worried about stuff leaking. Also, have a look at bhyve and possibly vimage on FreeBSD.
 
Zarathustra[H] said:
FreeNAS recommends booting off of USB in their official documentation.

Many people boot their ESXi servers off of USB sticks, and use iSCSI as a datastore.

Which for the record makes no sense to me, but not because of the USB part.

iSCSI is just about the worst way to have remote storage. I'd use NFS over it 10 out of 10 times.

iSCSI tends to force async writes, which leaves you open to data corruption. It also requires you to use disk images, so you constantly have to estimate how much space you'll need in advance, or create sparse images, which grow based on disk activity and are messy. It's also very inconvenient because you can only access the data from one place at a time.

Honestly, I don't understand why anyone uses iSCSI. It is terrible. NFS is a better protocol for networked storage in every way shape and form, IMHO.
Click to expand...


If you are talking, on the cheap, yeah I agree. But if you are talking about enterprise...I don't agree at all.

Some services and configurations don't like remote storage. Block level duplication isn't going to happen on NFS either. iSCSI has it's place, but alot of people talk it up as if it's a end all be all, but that's not really the case. You have to be smart and use the right tool for the job. There are plenty of times when NFS is the right choice as well.

We use lots of iSCSI devices here, but in my case we're also talking about 800TBs of Data shared over 600 VMs.



But lets stay back on topic...

Anyone seen or played with the new Ubiquiti UAPIW? It was meant for hotel retrofits, but we're using it to add Wifi to a single remote room at our facility. Works very well. It's supporting 1 guy, sometimes as many as 3 in receiving room at the opposite end of the building. Where it was difficult to wire for add infrastructure. Took a $1000 problem had solved for for $100 and 20 minutes of time.
 
  • Like
Reactions: rma
like this
Zarathustra[H] said:
So, I've decided to return the Ubiquiti router and eat the restocking fee.

I'm going to build a bare metal pfSense box instead using - as much as possible - parts I have kicking around, adding a few I need to order.

I'm going to go overkill, to make sure I have enough horsepower for all the QoS i want to do, and to make sure it can have other uses if I ever stop using it as a router in the future.

This is what I'm thinking:
  • Supermicro Socket 1150 server board (with dual Intel 210 gigabit LAN)
  • 35W TDP dual Core i5-4570T with HT disabled for power savings. Base clock 2.9, turbo 3.6 (for pfSense fast cores are better than more cores)
  • Whatever DDR3 SO-DIMMS I can find in my parts bin (will probably wind up being 2x2GB of Laptop RAM left over from an 8GB upgrade a while back, can't remember)
  • Sandisk USB drive as only disk to run off of.
  • Mini-Box M350 chassis and 60W pico-PSU

I'm hoping to get under 15W at the wall with this setup. I might undervolt and underclock a bit if needed.

Now my only problem is that my RMA expires before I can get all the parts here :p I'll have to figure out a temporary router :p

I probably went WAY overkill on the CPU, but I found a good deal on it on eBay, and it's still a 35W part.
Click to expand...

Well, I got this little mini system built yesterday.

As expected the CPU was totally overkill. After figuring out how to enable power saving on the CPU (it's in the web based GUI menus in pfSense) it spends most of it's time at normal loads lowering it's clock to between 900 and 1500mhz, which is great for power use. Power use is between 12W and 13.5W as measured at the wall with a Kill-A-Watt device. Really not bad for an x86 device with full (non-atom) desktop cores.

Without enabling the power saving settings it sits at base clocks (2.9ghz) and about 20-22W, with the little low profile rosewill cooler at min fan speed resulting in a 38C to 42C temp. The case is cooler to the touch than the Unifi router was :p

I have never seen the unit draw more than 22W at the wall, but I have only loaded it up with speedtest.net, and my connection maxes out at 160/160Mbit. The fact that it at normal use sits between 900-1500Mhz, and I've never seen it turbo boost above base clock (2.9ghz) its reported 3.6Ghz max turbo results in a huge safety margin where it can go under periods of heavy load :p

I fully expect both power use and clock speeds to go up once I enable full QoS, which is my next little project once I get done with my massive VMWare ESXi to Proxmox/KVM/LXC migration I am doing in parallel to my router upgrade.

While I would have preferred a system with ECC, I'm pretty happy with this build.
 
You must log in or register to reply here.
Back
Top