Loop failure - help me figure out what went wrong

[DanNeely]

My ~18-20 month loop failed earlier today with a hole forming in the Tygon tubing. It happened while I was sitting next to the machine but in a break from gaming, so I was able to here the noise from water spurting from the break in the tubing and shut it down before the pump ran dry or water got into the PSU -so I think I successfully averted any major disaster.

The break occurred where the tubing was most tightly curved, but on closer inspection I saw occasional dark streaks - that I assume are cracks - running parallel to the tubing even in strait//no tension sections of the tubing so it appears that something was degrading the tubing everywhere.

The other thing that was immediately apparent when I pulled the case out far enough to get a good look inside (vs just opening enough to top up evaporative losses in the loop) was that despite not putting any dye in my loop the main tubing had acquired a distinct green stain. I'm guessing this means that something has been corroding and that I might need to replace my CPU block and radiator before getting my system back online. (I'm not particularly concerned about the GPU block, it's a GTX 980 and due to be replaced.) The residual water in my resevoir itself doesn't appear to be colored; so whatever stained the tubing doesn't appear to be present in any significant concentration.

A few things I want to take a closer look at once I have things apart are that the kill coil in the res looks like a very dull gray and not shiney like silver should be, and there're marks of some sort on the inside of the res, opposite where the intake is.

My loop consists of:
XSPC Razor GTX 980 Full Coverage VGA Block
Alphacool NexXxoS XT45 Full Copper Quad 140mm Radiator
EK Supremacy EVO Universal CPU Liquid Cooling Block
Swiftech MCP35X 12 VDC Pump
Swiftech MCP35X Optional Tank Reservoir Attachment - Rev. 2b
MSI Z97 MPOWER Max AC LGA 1150 Intel Z97 Mosfet waterblock (Wasn't a factor in buying the mobo, but using it was easier than rigging a fan on them to make up for the lack of breeze from a CPU cooler.)
A bunch of XSPC 3/8 5/8 compression fittings
A silver kill coil
1 random barb (used to weight down my foam reservoir filter to keep turbulence from flipping it on its side).
Tygon tubing from MC Master
Distilled water

Full loop (minus radiator located below):

Spoiler

Closeup of resevoir

Spoiler

 

[Bageland2000]

Have you considered you may have just got a batch of shitty tubing? I've been running the same XSPC tubing for 6 years and it looks brand new. Maybe everything is related strictly to the tubing quality.

Also, cheers on not having to worrying about a 980 because it old! That's impressive

 

[DanNeely]

Have you considered you may have just got a batch of shitty tubing? I've been running the same XSPC tubing for 6 years and it looks brand new. Maybe everything is related strictly to the tubing quality.

I suppose bad tubing is possible. This is the 2nd half of a batch of tubing I used in a previous loop. The tubing in it failed after 5(?) years, with the inner 2/3ds having broken down into crumbly bits; at the time I assumed it was the antifreeze/dye mix I'd used as an anti-microbial. I think I had some tight bends in that loop, but probably not any quite as tight as the one that went this time. Just bad tubing doesn't explain where the green stains came from though.

Also, cheers on not having to worrying about a 980 because it old! That's impressive

I didn't want to pay the crazy prices when they first came out this summer; by the time they'd finally gone non-stupid and water blocks for custom cards were available I was getting ready for the next league in Path of Exile and didn't want to deal with the downtime from breaking the loop then. Also POE's engine is/was mostly CPU (a series of major patches over the last few months have been steadily fixing this) bound and wouldn't benefit much from a faster card, so part of me wanted to see if nVidia'd launch a 1080 Ti for Christmas while I waited too.

 

[thesmokingman]

that despite not putting any dye in my loop the main tubing had acquired a distinct green stain

That is algae, algae blooms are green. I bought a used 980ti block recently, popped it open and it was stinky as hell and green with weird green mold growing. Yea, green is bad. Right now you need to open everything up and clean it all out, seriously.

Then once cleaned, swap to a different brand of tubing, Primochill Advanced, EK, XSPC, etc. When putting it all back together run some coolant or other type of mix to prevent corrosion and biological growth. Also, you don't need the anti-kink stuff it's ugly and prevents you from seeing the hose condition. Get colored hose, to keep light out unless you are going with colored liquid then go clear.

 

[Nobu]

Just FYI, automotive antifreeze isn't antimicrobial, at least from what Linus heard and experienced. (linus explains lack of antimicrobial at 14:20 mark) The kill coil should have done the job, but it may need to be cleaned or replaced occasionally.

 

[DanNeely]

Just FYI, automotive antifreeze isn't antimicrobial, at least from what Linus heard and experienced. (linus explains lack of antimicrobial at 14:20 mark) The kill coil should have done the job, but it may need to be cleaned or replaced occasionally.

Cleaning the pump and CPU block shouldn't be hard since I can just remove the tops of both; but how do I clean the radiator out? The only thing that occurs to me is running hot tap water through it for an extended time, and then run something else to remove the assorted minerals/etc that were deposited in the process. If the kill coil's a crock (they were being hyped as idiotproof when I bought mine); I'll probably just go back to PT Nuke (copper sulfate) as an anti-microbial.

 

[DanNeely]

Then once cleaned, swap to a different brand of tubing, Primochill Advanced, EK, XSPC, etc. When putting it all back together run some coolant or other type of mix to prevent corrosion and biological growth. Also, you don't need the anti-kink stuff it's ugly and prevents you from seeing the hose condition. Get colored hose, to keep light out unless you are going with colored liquid then go clear.

My case is windowless, so the only light getting on the tubing is from the assorted idiot lights on my mobo/etc. Needing to be opaque to keep sunlight out shouldn't be an issue, and if it is wouldn't clear tubing and dye defeat the purpose?

I'm leery about putting anti freeze in as anti-corrosion; last time around everyone was convinced ti was responsible for my tubing breaking down; and without any aluminum in my loop I shouldn't need an anti-corrosion additive anyway.

As far as the anti-kink goes, I have had tight curves like the ones around my CPU block area kink badly in the past. I ran it everywhere for consistency, but since it's all hidden behind a solid door the appearance is irrelevant to me.

 

[Nobu]

Cleaning the pump and CPU block shouldn't be hard since I can just remove the tops of both; but how do I clean the radiator out? The only thing that occurs to me is running hot tap water through it for an extended time, and then run something else to remove the assorted minerals/etc that were deposited in the process. If the kill coil's a crock (they were being hyped as idiotproof when I bought mine); I'll probably just go back to PT Nuke (copper sulfate) as an anti-microbial.

Not sure what's the recommended way to clean a contaminated radiator--probably running hot water would be fine if it's not too bad, may want to run it separate from your loop with a pump and filter and some pt nuke to kill/remove whatever might be in there. Like I said, the kill coil should have done the trick, but who knows what may have happened between the time it was put in there and now--almost two years is a long time. I use Liquid Utopia, didn't have any growth for about a year, then started seeing stuff in the res (after being unplugged/stagnant for about a month). 6 months is probably a good timeframe for fresh coolant.

I'm leery about putting anti freeze in as anti-corrosion; last time around everyone was convinced ti was responsible for my tubing breaking down; and without any aluminum in my loop I shouldn't need an anti-corrosion additive anyway.

Sounds plausible, but I'm not familiar with the reactive properties of af. It really isn't necessary, imo.

 

[thesmokingman]

Just FYI, automotive antifreeze isn't antimicrobial, at least from what Linus heard and experienced. (linus explains lack of antimicrobial at 14:20 mark) The kill coil should have done the job, but it may need to be cleaned or replaced occasionally.

Who said to use auto antifreeze? There are many pc coolants with anti-biologicals and corrosion inhibitors.

My case is windowless, so the only light getting on the tubing is from the assorted idiot lights on my mobo/etc. Needing to be opaque to keep sunlight out shouldn't be an issue, and if it is wouldn't clear tubing and dye defeat the purpose?

I'm leery about putting anti freeze in as anti-corrosion; last time around everyone was convinced ti was responsible for my tubing breaking down; and without any aluminum in my loop I shouldn't need an anti-corrosion additive anyway.

As far as the anti-kink goes, I have had tight curves like the ones around my CPU block area kink badly in the past. I ran it everywhere for consistency, but since it's all hidden behind a solid door the appearance is irrelevant to me.

I guess you know what is what then?

 

[DanNeely]

Who said to use auto antifreeze? There are many pc coolants with anti-biologicals and corrosion inhibitors.

Such as? The last time I was building a loop all I found were "non conductive" coolants that were universally reviled as overpriced snake oil that cooled worse than water, dye only products (no interest to me), anti-microbials (eg PT nuke), and corrosion inhibitors that were just anti-freeze rebottled into small containers and marked up 5-10x over what auto-zone charged for the big jug.

 

[DanNeely]

I've started taking the loop apart.

Initial observations:

1) The water smells perfectly normal.

2) The green discoloration and breakdown of the tubing goes about halfway through the plastic. The outer half, while not visibly degraded is brittle enough that I had several more cracks open up around the pressure fittings due to flexing while i was working on the other end. The breakdown of the plastic itself resembles what happened with the other half of the tubing after a significantly longer period of use.

Spoiler

https://i.imgur.com/auPqFXf.png

3) The only visible indication of a potential problem in the closed CPU block is that the metal on the outer part of the baseplate looks more gray than coppery (my camera exaggerated the effect, in real life it looks more like a patina on top of the copper).

Spoiler

https://i.imgur.com/P9sZmau.png

4) I was mistaken earlier in thinking the radiator was going to be the biggest headache to clean. It's going to be the chipset cooler because the fasteners don't look like they've got a screw head I can use to detach it to make it easier to work on, and because I need to clean WTF is in it out before I can temporarily convert my system back to being air cooled. Other than running a short loop consisting of just it and my pump with heavily PT Nuked water I'm not sure what to try here, I need to dig around in my parts bin to see if I have any spare tubing to connect that up though. (I don't have any more tygon, but I might have some decade old 3/8" ID 1/2" OD masterclear from the kit that was my original water cooling setup.)

 

[NeedleArtist]

I can almost guarantee that your rad (s) are harbouring TONS of crud. The most effective (and relatively simple) method of thoroughly cleaning them is using Mayhems Blitz. you can check the kit out at mayhems.co.uk . Last time I checked Performance PCS, DazMode and ModMyMods all carry the kit. It costs about £20 directly from Mayhems. I've been using the stuff for years and have never found another solution that cleans so well. In fact, I clean all newly obtained rads with it before using for the first time (albeit only Part 1). It does a great job clearing out any corrosion, flux from the manufacturing processes, etc.If you decide to try this, get the complete twi-stage kit and not just part 1. The kit comes with PH testing strips but I find the electronic PH tester to be more accurate and easy to use. Mayhems carries that as well (don't know if the aforementioned resellers do as well) ar a reasonable price. Check out some reviews for the Mayhems Blitz kit if you're skeptical .
Best over luck with the loop.

Ps. Have you considered using rigid acrylic or PETG tubing instead of the soft stuff? No worries of plasticizers breaking down/leaching into the water, and you can get nice tight bends with no fear of kinking .

 

[DanNeely]

I'll take a look at your cleaner suggestion later tonight.

I wasn't even aware that rigid tubing was an option at all. My immediate - no Googling because I'm about to step out the door for something else - reaction is to wonder how I'd be expected to route it. If I can't bend the tubing I assume that means I'd need to use a large number of angled couplings to go around bends meaning I'd have a lot more joints to worry about leaking because they weren't tight enough. I can't see managing to make all the connections work without at least some flexible couplings, how do they work and what failure modes would I need to worry about with them. A related concern would be that the points where GPU blocks have their barbs is subject to moving between different cards; flexible tubing would give me a better chance of being able to reuse everything when connecting the new card.

 

[Nobu]

Who said to use auto antifreeze? There are many pc coolants with anti-biologicals and corrosion inhibitors.

Nobody did. He had used it in the past (and may have been currently), so I mentioned it, "for [his] information."

 

[thesmokingman]

Nobody did. He had used it in the past (and may have been currently), so I mentioned it, "for [his] information."

He obviously didn't use any inhibitor/coolant judging by his cpu block with its exposed copper. And the kill coil really prevented the green? On the surface it looks like the coil picked up more than its fair share of plating. The pics are scary!

 

[DanNeely]

This loop just had a kill coil which clearly obviously failed. Next time I'm probably going back to PT Nuke since i have it on hand and it has worked for me in the past. Not sure what I'm going to do with the coil, it's too small an amount to be worth effort needed to recycle it but on general principles I don't want to trash it either.

 

[DanNeely]

... or something failed anyway. Between the lack of odor and a few barbs that look like this one with very copper oxide looking scale, I'm honestly wondering if the green might have only been from corrosion.

Spoiler

 

[Nobu]

He obviously didn't use any inhibitor/coolant judging by his cpu block with its exposed copper. And the kill coil really prevented the green? On the surface it looks like the coil picked up more than its fair share of plating. The pics are scary!

Clearly, though he posted that after I made my post. Not sure about picking up plating, but the coil is definitely done, and that fitting looks like it was left on the statue of liberty for a month...

 

[EvilWays]

4) I was mistaken earlier in thinking the radiator was going to be the biggest headache to clean. It's going to be the chipset cooler because the fasteners don't look like they've got a screw head I can use to detach it to make it easier to work on, and because I need to clean WTF is in it out before I can temporarily convert my system back to being air cooled.

It just needs a 2.5mm allen key for those screws, per the manual.

I wasn't even aware that rigid tubing was an option at all. My immediate - no Googling because I'm about to step out the door for something else - reaction is to wonder how I'd be expected to route it. If I can't bend the tubing I assume that means I'd need to use a large number of angled couplings to go around bends meaning I'd have a lot more joints to worry about leaking because they weren't tight enough. I can't see managing to make all the connections work without at least some flexible couplings, how do they work and what failure modes would I need to worry about with them. A related concern would be that the points where GPU blocks have their barbs is subject to moving between different cards; flexible tubing would give me a better chance of being able to reuse everything when connecting the new card.

You can bend acrylic and PETG tubing with a heat gun. A tubing kit would be recommended as they include the tools required to work with rigid tubing (saw with cutting block, benders for various angles, silicon insert that goes in to the tube when heating to prevent kinking when bending, etc.). The kit I purchased even had a means of measuring out distances between runs in both vertical and horizontal directions). You'd have to change your fittings from barbed to compression wherever rigid tubing would go. There's a bunch of videos on YouTube on working with acrylic/PETG tubing.

 

[DanNeely]

I opened the GPU block up all the way, definite corrosion inside. Doesn't appear to've gunked up anywhere that would matter if I was planning to reuse it. Hopefully I'll've lucked out with CPU block too when I have time to open it in another day or or two.

Spoiler

 

[Ihaveworms]

Are you sure the mosfet heatsink isn't aluminum? I have seen some boards with these mosfet heatsinks that go with aluminum due to cost.

 

[thesmokingman]

Are you sure the mosfet heatsink isn't aluminum? I have seen some boards with these mosfet heatsinks that go with aluminum due to cost.

It's not a true waterblock, it's literally a heat pipe but instead of cooling fluid, they put barbs on each end of the heat pipe, smh.

 

[Ihaveworms]

It's not a true waterblock, it's literally a heat pipe but instead of cooling fluid, they put barbs on each end of the heat pipe, smh.

The water still is flowing through something. I am just wondering if what it is flowing through is made of aluminum.

 

[thesmokingman]

The water still is flowing through something. I am just wondering if what it is flowing through is made of aluminum.

They don't make heat pipes out of alu. The corrosion is typical ek/silver plating flaw left unchecked for way too long. Alu would have destroyed everything clearly.

 

[DanNeely]

The water still is flowing through something. I am just wondering if what it is flowing through is made of aluminum.

I can't tell from visual inspection. The barb itself is nickel plated what I can see with a flashlight pointed down it is a mix of the dulled iridescence you sometimes see on metal that was overheated (quality welding/whatever job there) and a hint of dull orangish brown that could be copper under a think film of discoloration.

 

[Nobu]

Searching around, can't find anything saying for sure that it's aluminum, people that had it say it appears to be nickel plated copper. Other motherboards of the time with similar cooling solutions had been known to use aluminum, though.