2kW+ PSU options?

Joined
Oct 23, 2018
Messages
669
Looking at upgrading from quad Turing GPUs to quad 3090s and having a little trouble finding a PSU which can handle the load. I'd really like to avoid dual supplies if possible.

The EVGA Supernova 2000 looks nice, but 2kW doesn't leave a lot of overhead for anything beyond the GPUs. Did the 2200 ever make it to North America?
 

Tsumi

[H]F Junkie
Joined
Mar 18, 2010
Messages
13,480
I think a large part of it is due to household 120v limitations. The typical household has 15 or 20 amp circuit breakers, which means a circuit would top out at 2400 watts. Factoring efficiency overhead, a 2000 watt 90% efficient PSU close at 18.5 amps. Factor in monitor and other accessories, and you are likely pushing over 20 amps on 120 volts.

In addition to the above restriction, the consumer market for a 2+ kw PSU is extremely small, which is likely why they never really made it over to the US.
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
No matter if you find a 2 kW+ PSU, or have to use two PSUs; You're going to have to install a second power circuit to your computer area. Either a second 120v circuit, or a dedicated 240v circuit to be pulling that kind of power.
 
Joined
Oct 23, 2018
Messages
669
No matter if you find a 2 kW+ PSU, or have to use two PSUs; You're going to have to install a second power circuit to your computer area. Either a second 120v circuit, or a dedicated 240v circuit to be pulling that kind of power.

That's the plan. I already have two dedicated 20A 120V circuits for the computer. I was thinking I would convert one to 240V if I could find a suitable power supply.

I read somewhere today that the mythical Supernova 2200 didn't accept 60hz input, so that has eliminated it from contention even if I could find one.
 

Ready4Dis

2[H]4U
Joined
Nov 4, 2015
Messages
2,466
I would prefer to change over to 240v at that point, as the wire size for 20a is the same whether you're using 120v or 240v. I'm pretty sure at this point electrical code in most areas probably requires a 4-wire connector, but there are plenty of 3-prong 240v outlets available (may need to check in your area). If you can get by with a 3-prong and you don't have anything else on your circuit, you could just get a 240v breaker and plug in your existing wiring on one end, and new outlet on the other end and not even have to run a new wire and be able to pull twice the watts through it. Even if it's only able to support 15-amps, that's still 3,600watts, if it's 20-amp, you can get up to 4,800watts. Dual PSU's can be a real hassle and I would avoid if you can. There isn't a great way to isolate the power supplies completely unless you use a PCIE extender that you can supply power directly into (like mining rigs do, but those are mostly 1x slots). Otherwise you have to be careful that the PSU is made for it, or you have some sort of power aggregation that can handle 2 PSU's that may not be supplying the exact same voltages (I have never researched these, so my terminology is probably off). You can find a lot of mining PSU's, but they are typically non-standard cases, so you may have to adapt them if you were to go this route. A side benefit of going to 240v is that most PSU's are more efficient at 240v than they are at 120v as well.

Bleh, after looking around, these are a pita to find, lol. I can only find non-standard ones (or at least for a normal PC) which would be a pain to mount into a normal case. Well, sorry I couldn't be much help honestly, after doing a little bit of looking around, I'm not sure what the best course of action is, so take what I said above with a grain of salt. It sounds good in theory, but if you can't find a PSU to fit, it won't help. Although, even if you have to go dual PSU, 240vac is still more efficient and can still use a single drop to power both PSU's.
 
Joined
Oct 23, 2018
Messages
669
Ended up buying the EVGA Supernova 2000 G+ from Newegg. Even though the second bullet on the product listing says "80 PLUS Gold certified, with 90% (115VAC) / 92% (220VAC~240VAC) efficiency or higher under typical loads" AND the seller threw a NEMA 5-15 cord into the shipping box, this PSU is 220V only. That's ok since I was planning to convert the circuit anyway, but this means I can't even do a test boot until after I've had the electrician over. What a PITA.
 

Tsumi

[H]F Junkie
Joined
Mar 18, 2010
Messages
13,480
That doesn't sound right. Modern APFC PSUs should be capable of 100-240 VAC.
 

Zarathustra[H]

Fully [H]
Joined
Oct 29, 2000
Messages
32,077
Wow. That's a lot of juice.

Have you already been able to get your hands on the 3090's (if so, that's impressive) or is this a future plan type of build?
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
Ended up buying the EVGA Supernova 2000 G+ from Newegg. Even though the second bullet on the product listing says "80 PLUS Gold certified, with 90% (115VAC) / 92% (220VAC~240VAC) efficiency or higher under typical loads" AND the seller threw a NEMA 5-15 cord into the shipping box, this PSU is 220V only. That's ok since I was planning to convert the circuit anyway, but this means I can't even do a test boot until after I've had the electrician over. What a PITA.

Live dangerously! Lop off a plug and shove wires into the dryer socket, you don't need no hippie approved connector to get in the way of you and your man box.

Else you can go on down to the home despot and get a dryer plug pigtail and rig up a cable to power your shiny new PSU.
 
Joined
Oct 23, 2018
Messages
669
Wow. That's a lot of juice.

Have you already been able to get your hands on the 3090's (if so, that's impressive) or is this a future plan type of build?
lol I wish. The way this "launch" is going so far, I won't be able to get them until late 2021. So, the new build will still be running the Titans until Nvidia finally gets some stock of the unicorns. The new PSU is planning ahead for when I'm able to get the 3090s. If they're 350W each, then that's 1400W alone. Since these are long duration loads (usually about a day, but not uncommonly 10+ days), I really should try to stay under 80% loading. This would be 87.5% just from the GPUs if I stuck with the AX1600i. The 2kW EVGA gives me 200W of additional load beyond the GPUs while staying under 80%.

Live dangerously! Lop off a plug and shove wires into the dryer socket, you don't need no hippie approved connector to get in the way of you and your man box.

Else you can go on down to the home despot and get a dryer plug pigtail and rig up a cable to power your shiny new PSU.
While I'm drawn to the excitement of energy and chaos, I'm much more comfortable with the boredom of structure. So, I had a licensed guy over who swapped the existing 20A circuit over to 220 and put a code-compliant outlet on it. It's like a 110V 20A outlet, except the opposite tine is turned 90deg. It was kind of tempting to go for a full 40A dryer outlet just to say I did, but finding cables for that would be a real PITA with no real benefit since I'm never going to be running a single machine over 4kW anyway.
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
6,226
That doesn't sound right. Modern APFC PSUs should be capable of 100-240 VAC.
Probably has to do with efficiency/current capacity: 220 can easily do 2kw on a 15a circuit (220v x 15a = 3300va), whereas it would have to derate if it dropped too low (110v x 15a = 1650va, meaning you lose nearly 18% of the capacity before factoring efficiency losses). A 20a circuit could barely handle 2kw before factoring in losses, but their PSU might not meet the efficiency spec they claim at that voltage.
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
While I'm drawn to the excitement of energy and chaos, I'm much more comfortable with the boredom of structure. So, I had a licensed guy over who swapped the existing 20A circuit over to 220 and put a code-compliant outlet on it. It's like a 110V 20A outlet, except the opposite tine is turned 90deg. It was kind of tempting to go for a full 40A dryer outlet just to say I did, but finding cables for that would be a real PITA with no real benefit since I'm never going to be running a single machine over 4kW anyway.

I think that's either a 5-20 or 6-20 NEMA connector. Not really a fan of those because I've seen people who know just enough to be dangerous make "adapters" or cut the ends of 120v plugs off and put the 5/6-20 connector on and end up with lots of excitement when they plugged the thing in. But it's in your own home so you know it won't be a problem.

The only thing I'd suggest is feeling the connector when you have a high load on it to see if it gets uncomfortably warm.
 

drescherjm

[H]F Junkie
Joined
Nov 19, 2008
Messages
14,920
At work I like the round locking style L5-30P (125V 30A) we have on a few UPSs.

They have the L6 series for 250V.
 
Joined
Oct 23, 2018
Messages
669
I think that's either a 5-20 or 6-20 NEMA connector. Not really a fan of those because I've seen people who know just enough to be dangerous make "adapters" or cut the ends of 120v plugs off and put the 5/6-20 connector on and end up with lots of excitement when they plugged the thing in. But it's in your own home so you know it won't be a problem.

The only thing I'd suggest is feeling the connector when you have a high load on it to see if it gets uncomfortably warm.
I went with a certified premade 6-20 -> C-19 cord. Turns out that McMaster of all places carries them.

I also went to power on the build for the first time last night and I think something inside of the PSU exploded (probably a giant cap, maybe several of them). It was about as loud as a small caliber pistol. Almost needed the 220 feed to restart my heart.

Needless to say, I'm not a happy camper right now.
 

drescherjm

[H]F Junkie
Joined
Nov 19, 2008
Messages
14,920
It was about as loud as a small caliber pistol.

I have had a similar experience with a 15K USD medical imaging monitor however it was not mine and it was fixed under warranty. The engineers even sent me a picture of the part that caused the short and their plan to redesign the part and the shipping container.
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
I went with a certified premade 6-20 -> C-19 cord. Turns out that McMaster of all places carries them.

I also went to power on the build for the first time last night and I think something inside of the PSU exploded (probably a giant cap, maybe several of them). It was about as loud as a small caliber pistol. Almost needed the 220 feed to restart my heart.

Needless to say, I'm not a happy camper right now.

If one of the line capacitors went off, it would have deformed the PSU housing. I'd guess that the bridge rectifier and/or the switching mosfets failed. As to why they failed, it could be several different possibilities. The big three that come to mind are:

1) Wiring fault with the 240v plug. I'd check across the two tangs with a multimeter to see what the actual voltage is.
2) Defective/incorrect PSU. Maybe EVGA incorrectly labelled a 115v PSU as a 240V unit? PSU could have also been defective.
3) Something got inside the PSU housing and shorted parts of the PSU out, so when power was applied, it went bang.

It'd be interesting to open up the supply and see what failed, but I'm sure you want to return it to EVGA for a refund or replacement. The problem is though, you don't know if it would happen again. I would get a cheap universal ATX PSU, or even a non-PFC unit in 220/240v mode and plug it into your dedicated 240v outlet to see if it behaves or not. You'll have to make an adapter though.
 
Joined
Oct 23, 2018
Messages
669
I took a look from the outside and can't see anything in there that looks exploded and it doesn't smell like burning, so maybe the bang wasn't from the PSU. It was loud enough that it was difficult to identify the exact location. Since the breaker didn't trip and nothing caught on fire, I have to imagine that the circuit is fine and the bang was from the PSU. I have a couple of spare autoswitching Corsair PSUs sitting on the shelf. I might try again with the cheaper one of those and see what happens.

Good news is that the motherboard and CPU appear to be fine. I gave it a successful test boot into BIOS tonight using one of the Corsair units connected to a different circuit.

The EVGA unit was bought from a seller on NewEgg based in China, so I'm sure the RMA process will be loads of fun.
 

atp1916

[H]ard|DCoTM x1
Joined
Jun 18, 2004
Messages
4,809
Didn't even know EVGA had 2/2.2kw supplies available.

NICE!
 

toast0

[H]ard|Gawd
Joined
Jan 26, 2010
Messages
1,219
I would prefer to change over to 240v at that point, as the wire size for 20a is the same whether you're using 120v or 240v. I'm pretty sure at this point electrical code in most areas probably requires a 4-wire connector, but there are plenty of 3-prong 240v outlets available (may need to check in your area). If you can get by with a 3-prong and you don't have anything else on your circuit, you could just get a 240v breaker and plug in your existing wiring on one end, and new outlet on the other end and not even have to run a new wire and be able to pull twice the watts through it.
I've been doing some electrical work lately, and haven't seen anything about requiring 4-wires for 240v; my barn appears to be to code (there's inspection certificates up anyway), and most of the 240v are L6 with just a three wire feed (hot,hot, ground), although a couple are L14 with hot,hot,neutral, ground.
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
I've been doing some electrical work lately, and haven't seen anything about requiring 4-wires for 240v; my barn appears to be to code (there's inspection certificates up anyway), and most of the 240v are L6 with just a three wire feed (hot,hot, ground), although a couple are L14 with hot,hot,neutral, ground.

A lot of older houses have a third option, hot-hot-neutral, with an optional ground bonded to the neutral so it can be both or just neutral.
 

Ready4Dis

2[H]4U
Joined
Nov 4, 2015
Messages
2,466
I've been doing some electrical work lately, and haven't seen anything about requiring 4-wires for 240v; my barn appears to be to code (there's inspection certificates up anyway), and most of the 240v are L6 with just a three wire feed (hot,hot, ground), although a couple are L14 with hot,hot,neutral, ground.
Like I said, check local rules, I'm not an electrician, but I know you can run 240 over 3 wires, not sure if the latest rules still allow it or if 4 is now required. If it's an older installation, it only has to adhere to the code at the time of inspection, it doesn't mean it's up to current code standards, just that it was when installed ;). I don't know for sure what the rules are for specific applications so I just say as precaution to check your local codes.
 
Joined
Oct 23, 2018
Messages
669
The EVGA unit was bought from a seller on NewEgg based in China, so I'm sure the RMA process will be loads of fun.
Small update here. I finally got off my glutes and pinged EVGA to see if they'd honor the warranty on this. I'm happy to say that they have issued an RMA without any pushback at all. This doesn't guarantee a replacement, but I've gotta give them props for at least trying. Most other companies wouldn't touch a product which is on a different continent than the market it was destined for.
 

Krazy925

Supreme [H]ardness
Joined
Sep 29, 2012
Messages
5,762
Small update here. I finally got off my glutes and pinged EVGA to see if they'd honor the warranty on this. I'm happy to say that they have issued an RMA without any pushback at all. This doesn't guarantee a replacement, but I've gotta give them props for at least trying. Most other companies wouldn't touch a product which is on a different continent than the market it was destined for.
Further solidifying why EVGA is top notch.

Manufacturer's I've worked for would tell you to kick rocks.
 
Joined
Oct 23, 2018
Messages
669
One more update, probably the final one:
After some delay (100% on my end), I sent EVGA the kaboomed PSU. WIthin two days of receiving it, they gave me a call to let me know my options. They couldn't replace it with the original Chinese-market unit, but they did offer their very best North American unit in its place. Within about an hour of that call, I had already received the tracking for the replacement as well as the notice that it had been automatically registered to my EVGA account.

So, big props to EVGA for this. Massively exceeded my expectations.
 

Format _C:

2[H]4U
Joined
Jun 12, 2001
Messages
3,489
I think a large part of it is due to household 120v limitations. The typical household has 15 or 20 amp circuit breakers, which means a circuit would top out at 2400 watts. Factoring efficiency overhead, a 2000 watt 90% efficient PSU close at 18.5 amps. Factor in monitor and other accessories, and you are likely pushing over 20 amps on 120 volts.

In addition to the above restriction, the consumer market for a 2+ kw PSU is extremely small, which is likely why they never really made it over to the US.

This is the #1 reason I can see also NEC (possibly CEC also) limits a circuit to 80% of the breakers rating for a continuous load. I see this is like having you vehicle at the red line all of the time yes it can do that for passing or merging but not all of the time.

No matter if you find a 2 kW+ PSU, or have to use two PSUs; You're going to have to install a second power circuit to your computer area. Either a second 120v circuit, or a dedicated 240v circuit to be pulling that kind of power.

That's the plan. I already have two dedicated 20A 120V circuits for the computer. I was thinking I would convert one to 240V if I could find a suitable power supply.

I read somewhere today that the mythical Supernova 2200 didn't accept 60hz input, so that has eliminated it from contention even if I could find one.
4 wires is needed if the load also needs 120VAC like a dryer (240 VAC for the heater and 120VAC for the drum motor and controls)
You need 2 hots neutral and the ground wire.
If you do this I would go with a small sub panel run the needed 4 wires (a 60 or 100A panel is a good size) and have seperate breakers for the PC loads.
I've been doing some electrical work lately, and haven't seen anything about requiring 4-wires for 240v; my barn appears to be to code (there's inspection certificates up anyway), and most of the 240v are L6 with just a three wire feed (hot,hot, ground), although a couple are L14 with hot,hot,neutral, ground.

For a straight 240VAC appliance no neutral is needed (like a AC motor or heater with a 240VAC blower or my last wall AC had a NEMA 10-30P it was a giant 28,000BTU with electric heat strips). 4 wires are only needed if it has 120VAC parts but it is good to pull one if doing it now as it is easier to run it now then digging it up and doing it over.
 

GiGaBiTe

[H]ard|Gawd
Joined
Apr 26, 2013
Messages
1,712
4 wires is needed if the load also needs 120VAC like a dryer (240 VAC for the heater and 120VAC for the drum motor and controls)
You need 2 hots neutral and the ground wire.

You don't need a separate ground wire. In older houses like mine, the 240v outlet is just 3 prongs, two hots and a neutral which is bonded to ground at the breaker panel.
 

Format _C:

2[H]4U
Joined
Jun 12, 2001
Messages
3,489
You don't need a separate ground wire. In older houses like mine, the 240v outlet is just 3 prongs, two hots and a neutral which is bonded to ground at the breaker panel.
I know this was done a lot in the older days for stoves and electric dryers those had the NEMA 10-50P or 10-30P but this changed in 1996 a ground is now needed and a NEMA 14-50P or 14-30P. In my apartment the range cable has 4 wires but uses a 10-50R the green and white are on the same lug and the green is one guage smaller then the rest but this was built in the late 1970's so a lot of stuff that was common back then is no longer done or allowed now.
 

Format _C:

2[H]4U
Joined
Jun 12, 2001
Messages
3,489
I had a dedicated 240v circuit run for this, with a NEMA 6-20 socket on the wall.

This is good! I had another house I used to live at had a wall AC that used the NEMA 6-15P but the wiring was 12AWG and the outlet was the 20A 6-20R type
Also for anyone wondering what a NEMA is that is the chart of the common ones

What is needed is 12AWG wire if he goes to 20A.
12AWG can be used for 120VCA or 240VAC for a circuit no larger then 20A
 

Format _C:

2[H]4U
Joined
Jun 12, 2001
Messages
3,489
If he has a 15A breaker chances are he has 14AWG wire.
Yes this is true not common for a 12AWG circuit with a 15A breaker unless the whole run was not 12AWG just because you see 12AWG on the circuit does not mean it was completely done with 12AWG mostly the case with home owner "fixes" and add ins.
 

Andrew_Carr

2[H]4U
Joined
Feb 26, 2005
Messages
2,065
Just a thought, but now that you have a 240v circuit why not try to mimic a mining or compute setup? I'd go for 2 x PSUs either way, but maybe get a PDU and buy a server PSU and a breakout board and some PCIe and that should power 2-3 GPUs while the ATX PSU could power the CPU (threadripper?) and the remaining GPU and all the accessories. They're small so you might be able to fit them into whatever case you've got. I've heard of people having trouble getting multi-PSU setups working but the only minor issue I've had is that one of my rigs if you powered off the main ATX PSU via the PSU switch the server PSU would sometimes still keep the GPUs running, but that's easily solved by it off via the PDU or what have you or just using the power button.

Someone's apparently power tested this before and came out to about 1700+ watts and I'm not sure if your CPU setup would use even more power, so I'd be wary about a single PSU.
https://www.pugetsystems.com/labs/articles/Quad-GeForce-RTX-3090-in-a-desktop---Does-it-work-1935/

Attached is a picture of what I'm talking about. You can get HP PSUs like these up to 1200W on 240v fairly cheaply. There's higher rated ones but then it gets more complicated (except for maybe PSUs built for antminers that go above 3kW but basically only have PCIe connectors).

Some good reading in the comments section here as well: https://timdettmers.com/2020/09/07/which-gpu-for-deep-learning/#3-slot_Design_and_Power_Issues
 

Attachments

  • 1200 server psu.png
    1200 server psu.png
    479.6 KB · Views: 0
Last edited:
Joined
Oct 23, 2018
Messages
669
Those PSUs are very loud, otherwise they'd be an easy option.

I'm currently running the machine on a Corsair AX1600i. It gets the job done with the GPUs underclocked, but it isn't quite enough power for when the GPUs are running full bore at stock clocks. The case is a Corsair 1000D, which was designed to have a miniITX system in it in addition to the main ATX machine. As a result, it has a mount for an additional SFX PSU. Fortunately, I have a Corsair SF750 sitting on the shelf. So, I'll just run one GPU off of the little PSU and the rest of the machine + GPUs off of the big one. Both will be connected to the 240V circuit from before.

The incoming EVGA unit will end up going into a gaming rig that I haven't fully specced out yet.
 

DanNeely

2[H]4U
Joined
Aug 26, 2005
Messages
3,965
I went with a certified premade 6-20 -> C-19 cord. Turns out that McMaster of all places carries them.

McMaster is a supply house that carries any and everything in the way of parts you need to keep a business running. Lots of industrial/commercial equipment needs bigger than consumer grade power cables; so of course they carry them.

OTOH so does Amazon. The Orange Box carries them as well, although my local location doesn't have them in stock (they're happy to send one from the nearest warehouse for free in store pickup though). To my mild surprise, the Blue Box doesn't appear to carry them although thier search will helpfully point you to things that won't work.
 
Last edited:
Joined
Oct 23, 2018
Messages
669
Have you decided how you're going to cool this beast?

People are going to want photos.

🔥 🚒
It's running on temporary plumbing right now, so it's too much of a mess to snap photos. But for an idea of the cooling, it's on three loops at the moment. The GPUs are set up as 2 per loop with each loop having a 480x60mm rad up front with regular retail ML120s in push-pull. The CPU is on its own loop with a 360x60mm rad up top with the high RPM AIO version of ML120s in push-pull. There are two ML140s for rear exhaust, and a few more ML120s on the top tray that I haven't really decided what to do with. The GPU blocks are the latest Aquacomputer ones with the XCS active backplates. Pumps are all Aquacomputer D5 NEXT, reservoirs are all Aquacomputer Ultitube glass tubes, and everything is controlled by a combination of a bunch of Aquacomputer gear incl an Aquaero 6 LT and a Farbwek 360.

Everything is plumbed with QDCs for modularity and quick component swaps + replumbing. If ugly routing is ok, the GPU loops can actually be reconfigured into a single loop without even turning off the computer. The QDCs also allow me to fill the radiators when they're not in the computer. On a build this size (80-100lbs is my guess on weight), that's a big deal because you can't just shake the case around to work out bubbles. The CPU rad, fans, and res are one module and can be pulled out of the machine as one chunk just by disconnecting two QDCs, two cables (fans + RGB), and then sliding the top tray out of the machine.

All of the fans are RGB, as well as the pump rings, CPU block (Aquacomputer Cuplex Kryos NEXT Vision), and there are RGB rings on the the reservoirs. Don't puke yet. The RGB is programmed so that everything but the OLED screens are completely off during normal operation. At the first coolant temp threshold, the fans on that loop will illuminate solid red. At the second threshold, the fans on that loop will flash red and an email gets sent out to alert me. The CPU block RGB follows the same rules as the rest of the CPU loop. The GPU blocks don't have their RGB connected because there isn't space between block and mobo for the connector. I think it would look pretty cool if I could get those connected and then program them to go from blue to red depending on component temp or current TDP or some similar thermal rule.

During non-compute usage, the system runs completely fanless. In fact, the fans don't even turn on for renders under 2 minutes.

That's probably more info than you were looking for. :) I may post a build thread once the plumbing isn't temporary.
 
Last edited:

DanNeely

2[H]4U
Joined
Aug 26, 2005
Messages
3,965
Everything is plumbed with QDCs for modularity and quick component swaps + replumbing. If ugly routing is ok, the GPU loops can actually be reconfigured into a single loop without even turning off the computer.

I'm not sure this would be a good idea. Most internal PC power connectors aren't designed for hot swapping, so you'd run a risk unplugging the water pumps on the loops being modified; and unless you've got T connectors with unused ends breaking the loop to change connections around would leave the pump running but unable to move water which isn't good for it either.
 
Top