Had to replace a downed switch on Christmas Eve

Shadowspawn

[H]ard|Gawd
Joined
Sep 17, 2002
Messages
1,870
One of the Cisco 3750's in our Enterprise stack died last night and I am the only one on call this week, requiring me to drive back in and replace it. I hate that damned stack - seriously needs to be upgraded. They are the old 1.5u models with little ram.

Anybody else have a holiday emergency?
 
Last edited:

Jay_2

2[H]4U
Joined
Mar 20, 2006
Messages
3,583
No, I'm on call from 12am until 7am tomorrow then in the office from 7am until 3pm. Same all week :-(
 

Ruoh

Supreme [H]ardness
Joined
Sep 16, 2009
Messages
5,858
I checked all the logs before I left work on Monday. Blew some air through the rack on a couple of the switches for good measure... All was good.
 

Jay_2

2[H]4U
Joined
Mar 20, 2006
Messages
3,583
I checked all the logs before I left work on Monday. Blew some air through the rack on a couple of the switches for good measure... All was good.

We have 4 Datacenters, it would take me about a week to do that :D

We actually lost a blade today, noticed the alert, HA sorted that out for me so I can wait until tomorrow morning.
 

Ruoh

Supreme [H]ardness
Joined
Sep 16, 2009
Messages
5,858
We have 4 Datacenters, it would take me about a week to do that :D

We actually lost a blade today, noticed the alert, HA sorted that out for me so I can wait until tomorrow morning.

Sometimes being small is better. ;)
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
Thankfully nothing eventful was going on this year.

last year there was storms throughout parts of Quebec and we had tons of COs/cabinets/cell sites on generator power and NOC was running on 2 guys even off regular hours just to keep up with coordinating techs throughout the region. only dropped a few small sites but overall managed to keep mostly everything operational through the massive outages and ice storms. Was pretty crazy. For us it was not that bad but for the CO techs they pretty much had to work throughout Christmas. Sucked for them, though I'm sure it was good money.
 

jeffmoss26

2[H]4U
Joined
Aug 1, 2002
Messages
2,267
Internet was down for some period of time, got a text at 6 this morning and a couple VPN users emailed me. I rebooted the firewall when I got in and everything seems okay. We were closed Tuesday and Wednesday.
 
Joined
Sep 14, 2005
Messages
751
What firewalls do you run? We've been using Juniper at about 100 locations and have yet to see a single firewall related issue in 10 years, most of these are the cheaper SSG5 units too
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
20,289
wow, co-incident, our ISP had a distribution switch that took out 32 clients go down x-mas eve about 10pm -6 time., and god knows why no fail over.. they had to manually change the switch and reconfigure apparently.. i assume because no one does config backups....

What firewalls do you run? We've been using Juniper at about 100 locations and have yet to see a single firewall related issue in 10 years, most of these are the cheaper SSG5 units too


Any suggestions for a Juniper FW for a network that has about an avg of 45k states at any given time....since it seems most of their cap out at 50k sessions, or am i confusing states and sessions...
 

calvinj

[H]ard|Gawd
Joined
Mar 2, 2009
Messages
1,738
Both Air Handlers in our server room needed recharged. Ended up getting 98+ in there for a while. Only thing saving us was being below zero temps. We opened a few windows on the floor and the big fans in the door did the rest. Was able to get it down to 94-95 while the AC guys trying to get it all figured out
 

Shadowspawn

[H]ard|Gawd
Joined
Sep 17, 2002
Messages
1,870
If it was a redundant stack why couldn't it wait? lol

Was this question aimed at me? </me checks original post> Nope no mention of redundancy.

Actually, it is a redundant stack, but not fully redundant. Most of the VMs running on that stack of switches can switch to another stack in the other building, but there are a couple that cannot for whatever reason.

No matter, it was a quick swap, no programming needed. Just swap the physical switch. It was just annoying to be in my sweats and have to throw on some clothes and drive 30 minutes each way.
 

jeffmoss26

2[H]4U
Joined
Aug 1, 2002
Messages
2,267
What firewalls do you run? We've been using Juniper at about 100 locations and have yet to see a single firewall related issue in 10 years, most of these are the cheaper SSG5 units too

Our core firewall is a WatchGuard Firebox X1250e, we have smaller units at 2 branch offices.
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
20,289
we had an x500e, while it worked i found it very cumbersome on the interface and configs, mind you this was...8 years ago..

How are they now, in relation to Juniper and Cisco for FW / Routers, since Cisco does the support fee's like WatchGuard does...
 

jeffmoss26

2[H]4U
Joined
Aug 1, 2002
Messages
2,267
WatchGuard is the only firewall I've had much working experience with...it can be a bit painful at times. Certain configuration settings are not where you might logically think they would be.
I have not had to call for support but I am sure my boss has over the years.
 

Jay_2

2[H]4U
Joined
Mar 20, 2006
Messages
3,583
I have used watchguard... really poor IMO bad interface and software. It was about 3 years ago now though.
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
20,289
Ya i hated our watchguard box for configuration, it is like the GUI and interface applications were purely designed by developers, not users.
 
Top