FASTEST Server Networking $55K 400GbE Switch Time! A Look at the FS N9510-64D A 64x

erek

erek

[H]F Junkie
Joined
Dec 19, 2005
Messages
10,875
This is so baller! Love all the hardware! Really big Broadcom Chip!!!

"We built a setup to get two servers attached to a $55,000 switch to see just how fast and cutting edge having a 64-port 400GbE switch is. If you are a networking enthusiast, this is the video for you. #networking Today we are going to take a look at the FS N9510-64D. This is a 64-port 400GbE switch powered by the Broadcom Tomahawk 4 switch chip. In this piece, we are going to look at networking hardware that is 4-8x faster than what many are using these days. Let us get to it, and some of the challenges we faced in the process."

1685137791245.png



Source: https://www.servethehome.com/a-look...tch-broadcom-nvidia-intel-amd-pny-supermicro/
 
erek said:
why not?
Click to expand...

Because their products are junk. I've had problems out of their crap since the early 2,000's. They won't behave with some switches, SAN devices etc.

Don't get me wrong. Broadjunk is fast when it works, but in my experience their products are far from reliable. There is a reason why it's the baseline for a lot of servers with other controllers being an upgrade option.
 
Dan_D said:
Because their products are junk. I've had problems out of their crap since the early 2,000's. They won't behave with some switches, SAN devices etc.

Don't get me wrong. Broadjunk is fast when it works, but in my experience their products are far from reliable. There is a reason why it's the baseline for a lot of servers with other controllers being an upgrade option.
Click to expand...
it's a Broadcom Tomahawk 4 chip

https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56990-series
 
Dan_D said:
Because their products are junk. I've had problems out of their crap since the early 2,000's. They won't behave with some switches, SAN devices etc.

Don't get me wrong. Broadjunk is fast when it works, but in my experience their products are far from reliable. There is a reason why it's the baseline for a lot of servers with other controllers being an upgrade option.
Click to expand...
Er, I know you like to shit on Broadcom, but, open most datacenter and carrier switches and you'll find Broadcom switch chips in them. Obviously different than their NICs, which you seem to be conflating here.
 
Dan_D said:
Yes I am.
Click to expand...
I've done that before. Broadcom mostly deserves the punches for NICs, though in the past... 3-4? ish years, they've kind of turned that ship around (at least in the limited exp I have with them lately, they've been just as plug and play as Intel. Still vastly prefer Intel though).
 
  • Like
Reactions: erek
like this
tangoseal said:
400gbe is not fast at all

No where near. That's like flying to Mars with a molasses covered f15 jet.

Now this is fast networking:

https://www.techexplorist.com/world-record-internet-speed-319-tb-s-over-3001-km/40257/
Click to expand...
This doesn’t seem to be an industry standard connection, “wavelength division multiplexing technology. The technology combines different amplifier technologies to achieve a transmission demonstration with a date rate of 319 terabits per second”
 
erek said:
This doesn’t seem to be an industry standard connection, “wavelength division multiplexing technology. The technology combines different amplifier technologies to achieve a transmission demonstration with a date rate of 319 terabits per second”
Click to expand...
Well, WDM is in every sort of transport technology really, and for most of the higher speed stuff It's DWDM, which depending on spacing can be 80/96/160 channels (wavelengths) in the C Band space running over a single fiber (throw in other bands to get more wavelengths on it if needed I suppose) , since they're (Ciena) up to 1.6 TB/wavelength or (Nokia , Infinera) 1.2 there's a lot squeezed in there. Heck, PON even uses WDM to technically work .

Though 400GB isn't the fastest in the traditional networking space (Lets say with Grey optics) , Since there are 800G QSFP-DD (and OSFP) transceivers and switches out there (with 1.6 /3.2 in the works for next steps) .

400G QSFP-DD optics really aren't the neat ones by themselves, the Coherent Pluggables (QSFP-DD-ZR+) are the neat ones, especially as the RON (Routed optical networking) stuff is out there, kinda killing the need for ROADMs.

And yea, pretty much like every higher speed switch uses Broadcom chips in it (There are of course things like Cisco's silicon one, some other ones too out there I suppose) , which is why it REALLLY Sucked for the past 2-3 years as they were capacity restrained, so to bump up production you'd have OEM's (Cisco of course) pay to move up the Queue to get Chips for their products, as everyone was competing over the same stuff.
 
Balkroth said:
Well, WDM is in every sort of transport technology really, and for most of the higher speed stuff It's DWDM, which depending on spacing can be 80/96/160 channels (wavelengths) in the C Band space running over a single fiber (throw in other bands to get more wavelengths on it if needed I suppose) , since they're (Ciena) up to 1.6 TB/wavelength or (Nokia , Infinera) 1.2 there's a lot squeezed in there. Heck, PON even uses WDM to technically work .

Though 400GB isn't the fastest in the traditional networking space (Lets say with Grey optics) , Since there are 800G QSFP-DD (and OSFP) transceivers and switches out there (with 1.6 /3.2 in the works for next steps) .

400G QSFP-DD optics really aren't the neat ones by themselves, the Coherent Pluggables (QSFP-DD-ZR+) are the neat ones, especially as the RON (Routed optical networking) stuff is out there, kinda killing the need for ROADMs.

And yea, pretty much like every higher speed switch uses Broadcom chips in it (There are of course things like Cisco's silicon one, some other ones too out there I suppose) , which is why it REALLLY Sucked for the past 2-3 years as they were capacity restrained, so to bump up production you'd have OEM's (Cisco of course) pay to move up the Queue to get Chips for their products, as everyone was competing over the same stuff.
Click to expand...
What about Pulse Width Modulations for high bandwidth transmissions? Seems to be used in telecommunications
 
erek said:
What about Pulse Width Modulations for high bandwidth transmissions? Seems to be used in telecommunications
Click to expand...
I can't really say I've really ever heard of it for transmission of data on anything really currently being used , pretty much everything uses QAM as the modulation technique , with QPSK thrown in where needed.
 
Last edited:
erek said:
What about Pulse Width Modulations for high bandwidth transmissions? Seems to be used in telecommunications
Click to expand...
Problematic with repeaters switching lag limiting speeds, only really viable for single point to point without repeaters at high speeds.
 
  • Like
Reactions: erek
like this
tangoseal said:
400gbe is not fast at all

No where near. That's like flying to Mars with a molasses covered f15 jet.

Now this is fast networking:

https://www.techexplorist.com/world-record-internet-speed-319-tb-s-over-3001-km/40257/
Click to expand...

That's not the same kind of deal. Wave division is magic where you stuff a bunch of individual signals onto a single fiber, but nothing processes all the logical signals at the same time; this record has 552 channels multiplexed on the fiber (well it says 4-core fiber, so about 145 channels per), which probably are a mix or 400gbps and 800gbps signals. A 3000km cable is going to need periodic amplifers (about ever 100km) and those are super magic too. You use that kind of multiplexing for long distance cabling, not within a datacenter; within a datacenter it's more reasonable to just run many fibers than to multiplex.

A 64-port 400gbe switch is a lot of switching on 4u, though. Not too many sites are going to need one of these today.
 
erek said:
Even with amplifiers?
Click to expand...
Yes,
Any step where it is converted adds switching delays or more properly clock skew which limits the timing differences in a pulse width modulation systems.
The skew is also different for the rise and fall of the signal, and can vary based on pulse width and temperature of the electronics.
There are much better modulation schemes available such as QAM.
 
You must log in or register to reply here.
Back
Top