I definitely see that point, and agree that the typical janitor probably doesn't need to go in there. However the flip side of these conversations usually end up with if no one can get in there, who actually puts out a fire or let's the fire department in when there is smoke rolling out from under the door? It's difficult to lock up an entire room, but certainly you need to put sensitive material inside of something else that can be locked. So in this case simply tossing the HDDs into a cabinet would have prevented them from wandering off. I've seen where people start going down the road where you restrict access into areas, and pretty soon every area has it's own sets of keys and it's a complete mess to actually get anything accomplished.
There's this process by which a fire department can break down a door. Or you know you can include a button similar to whats in elevators for them to bypass security but one that would trigger an alarm if its used. These problems are solvable with a little planning.