Network pics thread

[RichX]

I was thinking 3750X for access switches was pretty extreme also.

Everyone is different & has different reasons, but if you've got the coin to burn, why not i suppose.

Conversely on some mid-size sites I typically use 3560X for access and 3750X for core/L3, generally in a collapsed core design model.

If i had 3750x's in all my network cabinets I would be happy man, but would probably be asked why the network costs several million more than it should.

Here is on of the 3750 cores I was commissioning a few months ago.

 

[thrash408]

I'm not saying what i support is the whole world. But i do a LOT with schools and I realize that 99% of the districts do not need that type of hardware for L2. I'm not ignoring what the 3750x's can do better then the 2960's, but i'm saying it's overkill for the majority of districts. Again, don't get your panties in a wad over my opinion but i still stand by it. Schools with erate of 90% or more normally abuse it and vendors abuse it and go overboard.

 

[Vito_Corleone]

So only 1% of the schools in the US are going to use ISE with advanced features? Only 1% are going to use more than four switches in a stack? Only 1% want redundant power at the edge? What you're saying is completely false.

 

[obrith]

So only 1% of the schools in the US are going to use ISE with advanced features? Only 1% are going to use more than four switches in a stack? Only 1% want redundant power at the edge? What you're saying is completely false.

Remind me to never hire you if I'm on a budget.

 

[thrash408]

I'm not saying only 1% use these things, I'm saying they don't have to have these things to work efficiently. Sure stacking more then 4 is nice, but if it wasn't for e-rate none of them would front the money out of pocket for this stuff just for those features.

Catalyst 3750X 48 Port Data IP Base Layer 3 Switch
Model: WS-C3750X-48T-S
Price: $6,210.00

Catalyst 2960S 48 GigE 4 x SFP LAN Base
Model: WS-C2960-48TS-L
Price: $2,897.10


If you have never dealt with e-rate, or schools for that matter i guess you just won't understand. But in education there is a tight budget and the network is always put on the back burner.

 

[The Spyder]

Folks, this is a Network Pictures thread. Everyone is entitled to their opinions, but please keep on topic. Now back to more pictures!

 

[cyr0n_k0r]

So only 1% of the schools in the US are going to use ISE with advanced features? Only 1% are going to use more than four switches in a stack? Only 1% want redundant power at the edge? What you're saying is completely false.

Vito, I understand where you're coming from, but unless you've worked with E-Rate before everything you're saying isn't feasible for schools (at least the ones that rely on e-rate funding)

For instance, buying a switch that is almost 3x the price of a comparable 2960S switch will not pass a USAC Auditor General "cost effectiveness" review. Secondly, USAC does not allow schools to use E-rate monies for redundant equipment. That means that they will not pay for failover/redundant supervisors, power supplies, etc.

They also won't pay for other things like storage. Which means when you buy a server you most "cost allocate" the hard drives in the servers out of the reimbursement amount or else you will get an audit hit and could be subject to fines up to and including giving back the entire reimbursement amount.

There are A LOT of rules and red tape surrounding the e-rate program and because of that schools must make decisions on equipment based around the constraints of e-rate.

 

[Vito_Corleone]

My company does TONS of E-Rate. Obviously the rules are interpreted differently if ToX (and our customers) can run 3750X (or 3560X) instead of 2960S. My point is there are a lot of reasons to take the 2960 off the table and blanketly saying otherwise is ignorant. ISE is covered under E-Rate last I heard, so that alone basically negates all these arguments.

Also, assuming that I have no visibility into E-Rate deployments, simply because I disagree, is silly.

 

[TeeJayHoward]

Updated diagram with the new hosts included, and WiFi excluded:

What it looks like, physically:

 

[PigLover]

Enough debate about E-Rate and Government waste. This is a pictures thread...

My toys, sitting in a corner of my garage:

Bottom to top:
APC SmartUPS 1500 RT
Norco 4220, 20x 2TB Hitachi, X8SIA-F, X3450, 32GB, 3xM1015, Intel X520-DA2 10Gbe
Dell C6100 4-node server, 4 nodes each with 2xL5638, 48GB, 10Gbe. Running as 4-node Proxmox cluster
4U IstarUSA case with 11x 3.5" hotswap, 2x 2.5" hotswap. Currently no electronics.
Norco RPC-270 case. Currently empty, acting as more of a shelf than anything else. Was ESXi host.
Avocent 1U 17" slide-out KVM (with 16 port Trendnet KVM switch on the rear)
Juniper EX-2500 24x 10Gbe SFP+ switch
Cisco SG500x-48. 48x Gbe + 4x 10Gbe SFP+
24 port patch panel to house cable in cabinet above
(cabinet also houses cable modem, Mikrotik 450G router and HD Home Run + network tuner.

 

[waz0wski]

Enough debate about E-Rate and Government waste. This is a pictures thread...

My toys, sitting in a corner of my garage:

Bottom to top:
APC SmartUPS 1500 RT
Norco 4220, 20x 2TB Hitachi, X8SIA-F, X3450, 32GB, 3xM1015, Intel X520-DA2 10Gbe
Dell C6100 4-node server, 4 nodes each with 2xL5638, 48GB, 10Gbe. Running as 4-node Proxmox cluster
4U IstarUSA case with 11x 3.5" hotswap, 2x 2.5" hotswap. Currently no electronics.
Norco RPC-270 case. Currently empty, acting as more of a shelf than anything else. Was ESXi host.
Avocent 1U 17" slide-out KVM (with 16 port Trendnet KVM switch on the rear)
Juniper EX-2500 24x 10Gbe SFP+ switch
Cisco SG500x-48. 48x Gbe + 4x 10Gbe SFP+
24 port patch panel to house cable in cabinet above
(cabinet also houses cable modem, Mikrotik 450G router and HD Home Run + network tuner.

wow ... That's alot of nice gear for home!
How did you end up with a C6100 with 10Gb in your garage?!

 

[PigLover]

wow ... That's alot of nice gear for home!
How did you end up with a C6100 with 10Gb in your garage?!

The C6100s are showing up on eBay cheap right now. Lots of inventory coming off lease and flooding the resale market, driving the prices to the floor. You can find 4-sled "barebones" (stripped of CPU, memory and disk trays) for as low as $225. You can find them ready to plug in an go at $899 with 4 sleds each with L5520 and 24GB (total 8x L5520 & 96GB, no drives).

Used server pulls of X55xx and X56xx CPUs are showing up at good prices too, as are 4 & 8 GB Registered ECC DIMMs. Its a really good time to buy right now if you are interested in one-generation-old server gear (OK, nearly two generations old with the IB-2011s showing up in samples/ES).

Mellanox EN SFP+ 10Gbe cards are on the market at ~$70/each. Not the best since they don't do full stack offloads, but really fine cards for home use where the major workflow is hitting an NFS-based SAN. I started with these, but the better Intel 82599-based mezz cards showed up at a special price so I "upgraded".

The whole C6100, equipped with better CPUs (L5638), lots of memory and 4x 10Gbe cards cost less than $2k to equip.

Juniper 10Gbe switch is a long story...

 

[s0lid]

Heh, my C6100 box should be shipped here next week, atleast one node... another one was DOA as it was shipped cpu sink unmounted...sigh...
Just to clarify shipping route is: TX->OR(where the parts are atm)->Finland

So new ESXi HW arriving, each node with 2x L5520 and 48GB of ram.

@PigLover did you run any benches on those Mellanox cards? I would love to snag 3 off ebay for SAN/NAS <-> Node1/2 communication

So that this doesn't go 100% chitchatting, some pictures related to networking:
Ever wondered what 2kms of fiber looks like in 15m pieces ?

 

[PigLover]

... @PigLover did you run any benches on those Mellanox cards? ...

Just some simple stuff.

Netperf from debian client with Mellanox EN card to Solaris server with Intel X520: 9891.16 Mbits/sec (pretty darn close to 10G)
Reversed: 9877.13 Mbits/sec (negligible difference)

Bonnie++ from Debian (Proxmox) client to NFS mounted disk on same Solaris server:

Version 1.03d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
proxmox-node 96576M 68845 98 551492 34 153044 15 68486 94 424345 19 213.3 0

Hard to read with forum/HTML formatting. Come out to about 551.5MBytes/sec write, 424.3MBytes/sec read. This was without any tuning (array on ZFS server hits over 1GByte/sec when benched directly on server, so test is not disk speed limited yet - the gap is mostly NFS overheads and lack of tuning).

 

[Brak710]

Not nearly as good as PigLover's setup... but... Working on it!

Here is what my rack looks like right now as I'm moving stuff over to the Force10 48 port switch while I have free time here and there:

This is my first fiber connection's usage as according to the router's LCD:

Here is the total throughput for ~8 hours on the router:

And even with Snort running on two fiber lines, OpenVPN tunnels coming in, web traffic, game servers, tor relays.... The router is basically idle even at 150k states:

Album with more network pr0n: http://imgur.com/a/u7Trn

I'll see if I can post pics of our Nexus 2K/5K/7K deployment, and dual VBLOCK 700 at my work's temporary "datacenter". It's to hold us over while we build a second one to deal with growing out of our 90k Sqft one and be our own hot-failover site, but... No guaranteeing for obvious security concerns. Although, right now the Cisco boxes are stacked sky-high and it's a sight to see - there are pallets of like 600 Cisco APs for redoing our wireless, too.

I wish I could take some home.

 

[green91]

s50?

 

[D-EJ915]

I've also got a C6100...unless my friend coughs up some CPUs I may resell mine though...

 

[Angus]

What router is that? Looks nice...

Not nearly as good as PigLover's setup... but... Working on it!

Here is what my rack looks like right now as I'm moving stuff over to the Force10 48 port switch while I have free time here and there:

This is my first fiber connection's usage as according to the router's LCD:

Here is the total throughput for ~8 hours on the router:

And even with Snort running on two fiber lines, OpenVPN tunnels coming in, web traffic, game servers, tor relays.... The router is basically idle even at 150k states:

Album with more network pr0n: http://imgur.com/a/u7Trn

I'll see if I can post pics of our Nexus 2K/5K/7K deployment, and dual VBLOCK 700 at my work's temporary "datacenter". It's to hold us over while we build a second one to deal with growing out of our 90k Sqft one and be our own hot-failover site, but... No guaranteeing for obvious security concerns. Although, right now the Cisco boxes are stacked sky-high and it's a sight to see - there are pallets of like 600 Cisco APs for redoing our wireless, too.

I wish I could take some home.

 

[Brak710]

s50?

Yep, wish it could do L3, but... For $200, can't complain.

What router is that? Looks nice...

Custom built, PM for details if you want... Haven't launched our website yet. :/

 

[green91]

unixsurplus?

I work with a lot of Force10 at work. LOVE them. Its my favorite CLI.

Haven't used the S50, but have worked with several S25, S55, and S4810

 

[Jay_2]

your images don't work, probably due to post count

 

[crisu]

Grabbed pair of Palo Alto PA-2020's and upgraded from 4.0.3 to 4.1.11. The 3750 switches are for a FTTH project and thus not really my switches. Although I could possibly borrow the 3750G for a while to test GNS3 QinQ lab setup.

Some equipment for my CCIE R&S lab, only 4 switches (3560 PoE-48) are relevant for this. The 3560G in the middle will have some better use. The 2511-RJ will need an AUI transceiver and rollover cables.

Not shown:
- Juniper SSG5 home firewall
- TP-Link 8-port GigE switch (fanless )
- Watchguard x7 something, not in use.

 

[Adam]

I know you are all jealous of this awesome rack... hey its a simple home setup. Just moved so ran cables to each room (9 to my office alone)... its simple but I like it, keeps things neat in the home, no switches upstairs now:

Also those that know me, may wonder (what no wire manager)... the damn wire managers I use are on BO for a while, and I was moving soon

Also forgot to order blank inserts (do'h)

 

[BigBadAl]

Radio Scanners eh!?

 

[Adam]

Police scanners, yes old ones, tossed them this week, someone had sent me them. I'm a former EMT and used to freelance for the papers shooting accidents/fires... always have a scanner nearby And lots of old radio equipment

 

[Red Squirrel]

Nice, good idea on using the pipe clamps for the cabling, I never even thought of that.

 

[Blue Fox]

Not technically networking, but I don't care! Something has to power all those servers and here's what is responsible for some of mine. Figured I could take a photo while I'm resetting a breaker I tripped.

 

[Red Squirrel]

Fun stuff. I always find the power equipment in large buildings rather fascinating. We have a couple pieces of switchgear equipment like that at work too. Two ATSes, hydro in and Generator in. I like the fact that it's bigger that a fridge and is essentially an on/off switch. Though it has a bit more intelligence to it like a meter and stuff. Our building draws around 400 amps on each phase. 208 volts I think... 1500 amps on the 48vdc side. It's fun to watch bubbles inside the batteries.

 

[Blue Fox]

Certainly is interesting, but I don't screw with it as I would be fried in an instant. Building is rather large too. The big breakers are 4000A at 480V, or about 2MW each (I think about 20 total, but I could be way off). I had only tripped a lowly 30A one. The batteries are in separate rooms and the generators are outside for obvious reasons.

 

[FLECOM]

Certainly is interesting, but I don't screw with it as I would be fried in an instant.

death is final... some of those breakers carry serious current, improper use can cause an arc flash

ex

https://www.youtube.com/watch?v=W6Im7PLduwc

 

[MikeTrike]

Just about done cleaning up this cabinet.



Here's the before shot I posted a while back.

 

[Blue Fox]

Added more to my rack at work:

 

[agrikk]

Blue Fox,

I love those Supermicro cases. Their gear has always impressed me as being high qualirt and very reliable..

For content:

I'm in the middle of setting up a new home lab that should look something like this:

It'll be a 3-node ESXi 5.1 cluster and a 2-node Microsoft SQL Server 2012 cluster with maybe some Hyper-V tossed in for kicks.

The sixth pizza box will either be a vSphere server to manage the whole shebang (if I don't decide to cluster the vSener server for HA reasons) or an exchange server...?

I'm taking advantage of this NetApp FAS2050 for a while before I sell it off, since I don't need the IOPS for a home lab and its loud as hell and draws too much power.

Pics coming as I get it cabled.

 

[Krazypoloc]

Recently updated my server and moved to vSphere 5.1. Its nice having 8 NICs for a home lab. Planning on getting a house this year, then I will rebuild this server into a rackmounted chassis and build out a SAN instead of using local storage.

 

[Vashypooh]

Recently updated my server and moved to vSphere 5.1. Its nice having 8 NICs for a home lab. Planning on getting a house this year, then I will rebuild this server into a rackmounted chassis and build out a SAN instead of using local storage.

Yea, I am working towards getting a house this year, looking at building my own.

The guy started getting confused when I said I want cat 6 everywhere, I want zero phone lines ran, just cat 6, and have it all goto the utility room, unterminated and I can finish, just run the wires for me since they refuse to let me do anything due to liability if I hurt myself etc etc etc.

I cant wait to see their look when I ask for a ethernet cable in the ceiling for my AP's.

Guy was also a little confused when I said I will likely want my own L5 30amp circuit depending on how much they charge.

Room will be big enough for 4 post rack, and probably a folding table for working down there on it.

If I am going to be building a home, might as well get the IT requirements for my home out the way and matching what I want for the future as I start towards a smart home.

 

[agrikk]

If I am going to be building a home, might as well get the IT requirements for my home out the way and matching what I want for the future as I start towards a smart home.

I'm jealous.

Remember: When in doubt, more outlets!

 

[MinPins]

...or just buy a 1000' box and run the cables yourself so that you know they are done correctly.

 

[Vashypooh]

...or just buy a 1000' box and run the cables yourself so that you know they are done correctly.

That's the joy of building a home.

I will be able to see it throughout the whole process to make sure it is.

I don't want to have to cut the holes in the walls for boxes, etc. I much rather have people who do it for a living.

 

[FLECOM]

...or just buy a 1000' box and run the cables yourself so that you know they are done correctly.

1000' box does not do as much as you think it does... I used 2 boxes doing my 2 bedroom condo

 

[siliconnerd]

Yea, I am working towards getting a house this year, looking at building my own.

The guy started getting confused when I said I want cat 6 everywhere, I want zero phone lines ran, just cat 6, and have it all goto the utility room, unterminated and I can finish, just run the wires for me since they refuse to let me do anything due to liability if I hurt myself etc etc etc.

I cant wait to see their look when I ask for a ethernet cable in the ceiling for my AP's.

Guy was also a little confused when I said I will likely want my own L5 30amp circuit depending on how much they charge.

Room will be big enough for 4 post rack, and probably a folding table for working down there on it.

If I am going to be building a home, might as well get the IT requirements for my home out the way and matching what I want for the future as I start towards a smart home.

Plan for the future. Have all the Cat6 wiring ran in conduit. That way you can pull fiber or whatever takes over in the future. Also, I would do Cat6a, not Cat6.