Yup, if its a shitty job with bad managers and its killing you on the inside, go for it but just keep in mind every individual bridge you burn could haunt you 2 years down the road. Now that I'm in management I have cost more than a few people potential jobs with us because of the way they burned bridges in the past with previous companies or how they treated me/others before. I will NEVER fault an employee for quitting for a better opportunity but I do expect some basic respect while doing it. I actually just had an employee put in a 2 week notice then asked if its possible to take him off the schedule sooner as he would love some extra time off and I was 100% able to do that for him. Not only did he leave us on super good terms he has a standing job offer if it doesn't work out for him and a written reference to use for future jobs because he was nothing but respectful to us the entire time and that's all I ask for.Even if you hate the place and never plan on going back it's helpful to leave a good impression. It's a small world and chances are you'll cross paths with someone there in a future opportunity, or at least someone they know.
"Having an in" is the easiest way to get a job. Interviewing and the whole hiring process is a crapshoot. It's really hard to tell if someone is good or bad from an application and even an in-person interview. When people someone on the hiring side knows a candidate is good from past experience it pretty much guarantees they get the job over the other candidates where they would be taking a risk with all the uncertainty. And the opposite is also true.
Like you said in some industries the world is way smaller than you expect and its not worth the risk most times to destroy those connections.