Scalable management of non-default SNMP for 10-100k devices?

Howdy!

I am on an adventure to disable SNMPv1 (unless required by vendor) and configure SNMPv2c/3 (only highest and most secure possible) on 100-300 clients or 20k-100k devices that are SNMP capable. This includes ensuring devices do not use default SNMP strings and credentials.

Does anyone have any recommendations on a way to reconfigure SNMP on devices that are already SNMP-enabled with default strings? Reconfiguring SNMP on 20k-100k devices from firewalls, printers, CCTV systems and cameras, IoT, and beyond by hand is practically a secured full-time job for several years alone -- massive technical debt! Surely there are other organizations out there who have figured this out.

This is in preparation for a SNMP-based network monitoring system like Auvik

Cerulean said:

Howdy!

I am on an adventure to disable SNMPv1 (unless required by vendor) and configure SNMPv2c/3 (only highest and most secure possible) on 100-300 clients or 20k-100k devices that are SNMP capable. This includes ensuring devices do not use default SNMP strings and credentials.

Does anyone have any recommendations on a way to reconfigure SNMP on devices that are already SNMP-enabled with default strings? Reconfiguring SNMP on 20k-100k devices from firewalls, printers, CCTV systems and cameras, IoT, and beyond by hand is practically a secured full-time job for several years alone -- massive technical debt! Surely there are other organizations out there who have figured this out.

This is in preparation for a SNMP-based network monitoring system like Auvik

Click to expand...

Something like lansweeper might help you out. From my experience, you might need to build out a package and deploy it once you had everything added/scanned in

Also, out of curiosity. Did you figure out how many can be done in 8 hours, then determine how many years of job security that gives you?

Well depending on what type of stuff is in this environment maybe there are tools to do this from those manufacturers

HP Printers have jet admin server, cisco you can use the enhanced device interface, some firewalls have central config servers, etc etc.

Is this all hodgepodge gear? With this many devices I'm thinking this is an MSSP environment?

MSP environment with hundreds of locations, probably at least a thousand different subnets, some with hodgepodge of gear (not our preferred solutions), some that use our preferred solutions (but we don't support every brand of printers and MFC, CCTV systems, and myriad of other SNMP devices yet we're responsible for security-wellbeing of the client's network so we have to touch these devices). Still need to figure out how we're going to handle new devices (like printers), as it isn't like the clients need to inform us of new devices.

FNtastic said:

Also, out of curiosity. Did you figure out how many can be done in 8 hours, then determine how many years of job security that gives you?

Click to expand...

The last number I heard from a colleague that has been investing time in wading through the swamp is... 1 hour per device #EpicRageMemeFaceHere#

Surely other MSPs have figured this out (as most network monitoring tools out there, like Auvik and Logic Monitor, depend on SNMP as opposed to agent-based like Dell Quest PacketTrap) ... or maybe most of them don't care about security/SNMP/have access to tools like Auvik ?_?

Have you looked at spiceworks and solarwinds? I haven't used either in ages, they are probably two of the most widely used and probably more affordable options.

Also most devices that I see don't have SNMP write enabled out of the box, it's usually just the public community and it's read only, so not sure how you're going to use SNMP to change anything on those.

And to your co worker's 1 hour per device...hahahahahahaha

Our guys just script it using the the CLI interface of the device family. Usually this is done is stages.

1. Gather IP addresses
2. Script login
3. Confirm access
4. Resolve any access issues

5. Script changes


We do this fairly often on walk in takeover deals. Please note I am speaking to network infrastructure here, switches, routers and firewalls. We don't do printers and such but, if it has a cli then it should be easily scripted. The biggest issue is always step 3 because no matter what the client says there are always devices with access/password issues.

As for an hour per device ... fire this person with extreme prejudice. They're just stupid.

1 hour per device? Fire that person ASAP lol.

You can get someone in the Philippines to do this for $5/h if you want honestly.

But in all honesty, why do you want to set SNMP on Every device? Just set it on the critical ones you need to monitor, throw them in Logic monitor or whatever and call it a day.