CAD system for client - Intel 5520 chipset + 2x Quadro FX5800's = issues

sabregen

sabregen

Fully [H]
Joined
Jun 6, 2005
Messages
19,501
I have spec'd the following system (and have it next to me) for a client:

Supermicro CSE743-645B chassis (4U tower)
Supermicro X8DA3 motherboard (Intel Tylersburg 5520 chipset)
2x Intel X5560 CPUs (Nahalem QC Xeons w/HT 2.8GHz)
2x Intel SSC Coolers (up to 125W Thermal capacity)
Silverstone ST-1200 1200w fully modular PSU
4x Seagate Cheetah 15k.5 15k RPM 146GB SAS 3.5"
2x PNY Nvidia Quadro FX5800 cards (not intended to run SLi) w/4GB RAM each
2x HP 30" LCD screens (one on each card)
Vista Ultimate x64 SP1

The problem that I am having is that after installing the NVidia Quadro FX5800 WHQL drivers, the system will power on just fine, and then begin its boot process into Windows. I am pretty sure that the NVidia Quadro FX5800 drivers are the cause, but the issue is that once the system initiates the drivers to bring up the desktop, then entire system just shuts down.

Prior to the driver installation, the system would boot just fine. However, in the device manager, the second of the Quadro FX5800's is listed as "Unable to start the device."

Anyone know what the hell is going on? The system is to be delivered in less than 5 business days.

I should mention that the intended use with for AutoCAD 2010 products to leverage the CUDA capabilities of the Quadro FX cards to render in 3D, and that the cards are not currently, nor intended to be SLI'd.
 
Last edited:
First off........whoa im jealous.

Ok im assuming that you have a clean install.

But what it sounds like to me is that you have a bad GPU maybe, just because that before driver installation, it was listed as unable to start device....which even without nvidia drivers windows should be able to initialize and use the device.

Try just using one at a time, in each slot.....if it doesn't do the reboot with one card and does with the other or only does in a particular slot then you know you have a hardware problem....either video card or mobo(hope not). If it still does it then there is a driver problem or compatibility issue.

Also just curious did you install the OS with both cards installed or just 1?
Only reason I ask is because on my old dualie the OS install had weird issues if I had installed Vista with more than one graphics card installed. Dont know if something similar could be the case.

I am curious though....why two fx5800's? You do know that CUDA cannot scale across multiple cards, so unless he is running more than one instance he wont really be fully utilizing them both.

This mobo is too new to know of any compatibility issues just yet, so that kinda stinks...my google-fu didnt come up with anything except this thread.

Which drivers are you using? Release # and Graphics driver, AutoCad perfermance driver, 3dmax performance driver, quadro performance driver, or the quadro certified partner driver?
After you determine what the underlying issue is hopefully we can find a fix.
 
Last edited:
ya nice system,

to start off nitro had some good suggestions, try 1 card at a time in each of the pcie slots, try different nvidia drivers, last time i configured CAD system i think there were like 5-6 drivers to choose from.

Was the vista dvd used clean retail sp1 dvd or was it in house made with slipstreamed patches, drivers etc? I found that when I used my slipstreamed vista dvd at home the system installed fine but after installing newest official nvidia drivers the video card would stop working and I had to go through safe mode to clean them up and reinstall.
 
axan said:
Was the vista dvd used clean retail sp1 dvd or was it in house made with slipstreamed patches, drivers etc? I found that when I used my slipstreamed vista dvd at home the system installed fine but after installing newest official nvidia drivers the video card would stop working and I had to go through safe mode to clean them up and reinstall.
Click to expand...

Yea thats a good point, I didnt even consider that.
When you upgrade Nvidia drivers, you must uninstall and reboot, then Install the new ones or you can run into problems.
 
Guys, thanks for jumping in, first off. I should have noted that the actual error code is as follows:

"This device cannot start (Code 10)."

The initial install was done clean from an OEM Vista Ultimate SP1 x64 DVD. The drivers for the LSI 1068e were pulled down from the supermicro website, and installed into the system at the disk configuration menu. The drives are in a RAID-10 array. The initial installation was performed with both cards installed. After the desktop came up (install finished) I installed the Intel Chipset drivers, and rebooted. I then installed the Audio Drivers, and rebooted. Then the LSI MegaRAID manager, and rebooted. Then installed NVidia Quadro drivers. Driver version information is here:

Quadro Release 182 WHQL
Version: 182.65
Release Date: May 4, 2009
Operating System: Windows Vista 64-bit
Language: U.S. English
File Size: 63.5 MB
# WHQL Certified driver.
# Improved performance on workstation applications.
# Added support for the following NVIDIA products –

* NVIDIA Quadro FX 380
* NVIDIA Quadro FX 580
* NVIDIA Quadro FX 1800
* NVIDIA Quadro FX 3800
* NVIDIA Quadro NVS 295

# Added support for DDC-Ci/MCCS over the DisplayPort AUX channel.
# Added 10-bits per color (10bpc) DisplayPort support for full-screen applications on supported products.
# Integrated AutoCAD and 3ds Max Performance Drivers in the graphics driver with auto-detection, providing increased performance. Support for three latest versions of AutoCAD and 3ds Max. Standalone versions or version updates can still be downloaded from the NVIDIA driver download page.
# Workstation application compatibility fixes. Please read the release notes for more information on product support, feature limitations, driver fixes and known compatibility issues.
# If you would like to be notified of upcoming drivers for Windows, please subscribe to the

Since it said that it had integrated and automated the AutoCAD driver package, I figured that this would be good. Additionally, I am aware of the single instance limitations for CUDA access. The customer plans to run the following programs:

AutoCAD 2010 MEP
Autodesk Revit 2010 MEP

There's another package as well, but I forget the name. The idea is that he will have 2x 30" screens, and each screen on a separate quadro, and each program set to use a specific screen, thereby enabling CUDA acceleration for each application, independent of each other, running simultaneously.

FWIW, I have removed one of the Quadros from the system, and re-installed the driver into the system, and Windows booted just fine, and set the resolution properly on the screen, and Device Manager lists it correctly as a Quadro FX5800.

I know what you mean...this would be one badass toy to have...but I would need more storage (and i KNOW both of you would, as well). I will try taking the currently installed card and moving it to the second PCI-E x16 slot when I get home this afternoon, and then also try it with the second card, in both slots. I am currently in Los Alamos dealing with two other problem Supermicro's that are only seeing one physical CPU (both of them), despite that there's two physically installed...and there were 13 others just like them, without issue.


Update: The performance driver listed on the Nvidia Website is the same release date, file version, driver version, and file size as the regular graphics driver. The Partner certified driver does not exist for Vista x64, and the AutoCAD Performance driver is automatically installed by the main driver package, and lays dormant until and AutoCAD instance is run (for the first time) on the system, and then the user is prompted to install it at that time.
 
Last edited:
So far, I have tested the following:

CARD#1 in PCI-E x16 Slot#1 (worked)
CARD#1 in PCI-E x16 Slot#2 (Vista re-installed display driver after getting to the desktop, prompted for a reboot...which was done...and then properly recognized the card in Slot#2)

CARD#2 in PCI-E x16 Slot#1 (worked)
CARD#2 in PCI-E x16 Slot#2 (worked)

CARD#1 in Slot#1, CARD#2 in Slot#2 = entire machine shuts down before I get to desktop
CARD#2 in Slot#1, CARD#1 in Slot#2 = entire machine shuts down before I get to desktop

Going to try with both cards in, confirming driver loads along the way
 
You have a BIOS mismatch on the cards. I'm not sure there's any way to fix that at all on the FX5k's. It's a common issue on older cards, since they're special order and anything "in-stock" has likely been on the shelf for months.

Also, they may have finally implemented SLI lockout. (Cards refusing to operate when more than one is in the system, if they aren't SLI cards.) This is why I refuse to sell Quadros; PNY just screws you seven ways from Sunday with crap like this.
 
that card has been out, so it should have mature drivers
Try calling SM they might know something, we dont about mobo compatibility.
 
The cards are SLi capable, but SLi bridge is not installed, and they are not intended to be run that way. If I am understanding SLi lockout correctly, the system may refuse to see the second card, or go to desktop on cards that are NOT SLi capable, when more than one is installed in the system, correct?
 
sabregen said:
The cards are SLi capable, but SLi bridge is not installed, and they are not intended to be run that way. If I am understanding SLi lockout correctly, the system may refuse to see the second card, or go to desktop on cards that are NOT SLi capable, when more than one is installed in the system, correct?
Click to expand...

The way it was explained to me on several occasions was that the plan for SLI lockout was a lot nastier, and almost exactly like what you are seeing.
On cards installed into systems without an SLI approved chipset, one or both cards would be turned off - as in powered down mid-operation, disassociated, they were not specific enough - at driver load. If not by card BIOS during boot/load. In other words, they would basically crash your system hard as soon as they could.

What it sounds like is occurring in your system right now is that at driver initialization, one or both of the cards is simply bombing the whole system. I can tell you for an absolute fact that it is NOT a board power issue; the X8DA3 can handle dual high draw PCIe's with ease. I'll let you guess what that leaves.
Try it with the SLI bridge connected, let's see what happens. Also make sure that PEG 1x is set to YES if it's still there in the BIOS.
 
There's no PEG setting in BIOS, but the only DVI output that will drive the monitor is card#1, DVI#1
 
Correct. At boot, and up until the point that the driver loads, the only DVI that will give monitor ouput is CARD#1 DVI#1.

I am on the phone with Supermicro as I type this. so far, they have had me uninstall the NVidia drivers, and confirm uninstall (Standard VGA Graphics adapter listed in Device Manager). I have just physicall installed the second card, to see if I get a code 10, again.
 
Okay, so let me see if I understand correctly:

Card1, Slot1 does DVI output on only DVI1.
Card2, Slot1 does not work at all.
Card1 plus Card2 does not work at all.

Is that correct?
 
Card1, Slot1 does DVI output on only DVI1. - Correct. DVI#2 does not display output during boot, or POST. It does, however, display output after you enable DVI#2 in the Display Properties (by telling it monitor#2 is connected). Without drivers installed, DVI#2 cannot be configured to display output.

Card2, Slot1 does not work at all. - If installed by itself, it exhibits the same behavior as CARD#1, displaying only on DVI#1. Without drivers installed, but both cards installed, the card is Code 10, "Cannot start this device," and is therefore not accessible by Windows, and cannot be configured.

Card1 plus Card2 does not work at all. - Without drivers installed, I can get into Windows. Card#2 is listed as Code 10 "Cannot start this device." With drivers, I get all the way to the Windows progress bar (before desktop) and then the entire system shuts down like I had hit the power button.

Supermicro has requested a Device Manager screenshot, and said that they were going to engage the BIOS engineer team, and send me a PCI bus scanning tool, and have requested the output from the tool. I am awaiting their e-mail.
 
Did they say hardware or software tool? Probably they'll be sending you PCIScope; excellent software tool.

I'm confused again. You answered the question very badly. You're saying Card2 doesn't work at all, then you're saying it does work. Which is it? Does it work by itself in Slot2 then?
Here, fill out this monstrosity for me. :)

Code: 
      S1 S2 S1 S2 - slot
Card1
Card2
Mark the combinations that output on DVI1 on one or both cards. nVidia is stupid, and can not turn on DVI2 until driver, so I'm only interested in DVI1 output.
 
I am confused by your monstrosity, :p why are Slot 1 and Slot 2 listed twice, if we're only concerned with DVI#1 on each card?
 
Card#1 and Card #2 both successfully output on DVI#1, if each is installed by themselves in either Slot#1 or Slot#2. These are the only configs that produce DVI#1 video output. Did that make sense? Did I un-FUBAR it?
 
Yes, that makes MUCH more sense.
So EITHER card is capable of single operation in EITHER slot, then?
But BOTH cards, only Card1, Slot1 outputs video. And Card2, Slot1 does not work at all?
 
The tool is PCIEX.exe Version 4.8 by Intel Corporation (c)2003. It can see both cards, and lists one in PCI Slot 5, and one in PCI Slot 6
 
AreEss said:
Yes, that makes MUCH more sense.
So EITHER card is capable of single operation in EITHER slot, then?
But BOTH cards, only Card1, Slot1 outputs video. And Card2, Slot1 does not work at all?
Click to expand...

Either card, either slot = good, output on DVI#1 only

Both cards = Card#1, Slot#1, DVI#1 is the only one that will output video. I can swap Card#1 and Card#2 in this configuration, and get Card#2, Slot#1, DVI#1 to output video.

Once the driver is installed, having both cards in the systems causes a hard power off at driver init, while looking at the Windows Load Screen bouncing progress bar.

Event Viewer is useless, and nbtlog.txt contains nothing.
 
weird update:

Booting into Windows Safe Mode says that the device driver for the Quadro FX5800 was installed successfully (must have pulled it down from Microsoft), and both video cards are listed correctly...


without a Code 10 error

Driver date is March of 2009, and is listed as Nvidia provided. Doing a reboot now to see if it will come up into Windows normally.
 
sabregen said:
Either card, either slot = good, output on DVI#1 only

Both cards = Card#1, Slot#1, DVI#1 is the only one that will output video. I can swap Card#1 and Card#2 in this configuration, and get Card#2, Slot#1, DVI#1 to output video.

Once the driver is installed, having both cards in the systems causes a hard power off at driver init, while looking at the Windows Load Screen bouncing progress bar.

Event Viewer is useless, and nbtlog.txt contains nothing.
Click to expand...

Okay, that tells us something very, very important. That tells us that it is likely not an abnormal shutdown occurring. Instead, it's likely a normal halt. (Boy, aren't you thrilled now?)
Here's what we need to do. I need you to connect the SLI bridge, and see if you can get to the login screen with the SLI bridge connected. DVI output may be on a different port. If that doesn't work, I'm going to want some data that should already be in the output from PCIEX.exe, and we'll see what Supermicro says. It could still be a BIOS issue.
 
no dice. load screen comes up, then goes away. I see a mouse cursor, and the Vista spinning circle...then power off, same as before
 
trying safe mode with SLi brige installed, first...I'll report back on what (if anything different) it says. Then, I'll try normal Windows boot with SLi bridge installed.

Regarding the output of the tool..you can have it, if you think you can make heads or tails of it...it's a 768k .TXT file
 
safe mode with SLi bridge connected showed nothing different than before, both cards were correctly listed, and neither had an error.

regular boot into windows got past the load screen, produced a cursor with the spinning circle, then a brief flash of the vista logo that you get right before the desktop, and then a power off.
 
Yeah, the full text file would be awesome. Is there somewhere you can post it, or would you prefer to email it to me?
 
There's nowhere that I can post it, really. I can e-mail it to you, PM me.

FWIW - I just reloaded Vista from scratch, with just 1 card installed. Installed went as expected. Driver installation went as expected. Installed the second card, and got to the desktop...just about jumped for joy...then Windows started installing the device driver for the newly added card. As soon as the driver finished installing, the machine went down.

This is most definitely a driver lockout issue. I didn't hear anything more back from Supermicro. I may have to engage our higher level PNY contacts tomorrow to find out a resolution.
 
attaching a display to the second card, dvi#1 did not help. I read on another forum that linux drivers will do basically the same action, but in linux it's caused by the additional card not having any active displays attached, which causes something to go awry with xserve. Back to the drawing board.
 
Well after a request from one of your buddies in out #[H]ard|OCP IRC channel I have a few suggestions for you.
1st: be sure you are running the latest bios for your motherboard.
2nd: update both FX cards bios to the latest for that card manufacturer.
- Check what Bios your cards have: http://www.techpowerup.com/downloads/1379/TechPowerUp_GPU-Z_v0.3.4.html
- Look for newer Bios files: http://www.mvktech.net/component/option,com_remository/Itemid,26/func,select/id,58/
- Bios utility: http://www.mvktech.net/component/op...temid,26/func,select/id,135/orderby,2/page,6/
- Flash utility: http://www.mvktech.net/component/option,com_remository/Itemid,26/func,fileinfo/id,2877/
- Make bootable flash floppy: http://k7jo.de/tools/drdos/drdflash.exe
 
Yeah, no, do not do what Razor is suggesting. Unless you definitely want it to not work ever again, and PNY to refuse support and warranty.
This is not toy PC hardware. This is real hardware.

Email address PM'd.
 
Razor_FX_II said:
Well after a request from one of your buddies in out #[H]ard|OCP IRC channel I have a few suggestions for you.
1st: be sure you are running the latest bios for your motherboard.
2nd: update both FX cards bios to the latest for that card manufacturer.
- Check what Bios your cards have: http://www.techpowerup.com/downloads/1379/TechPowerUp_GPU-Z_v0.3.4.html
- Look for newer Bios files: http://www.mvktech.net/component/option,com_remository/Itemid,26/func,select/id,58/
- Bios utility: http://www.mvktech.net/component/op...temid,26/func,select/id,135/orderby,2/page,6/
- Flash utility: http://www.mvktech.net/component/option,com_remository/Itemid,26/func,fileinfo/id,2877/
- Make bootable flash floppy: http://k7jo.de/tools/drdos/drdflash.exe
Click to expand...

Motherboard BIOS is only @ revision 1.0a so far, and that's what's already on there. I have engaged Supermicro, and have a call into PNY, and will make another tomorrow.

I know exactly what you're suggesting, and I do appreciate it. However, I cannot risk having anything on this system go any more awry than it already is. I appreciate your suggestion, and were it just my personal desktop card, I would certainly be willing (and have, many times) downloaded modded Nvidia card BIOS's or used NiBiToR to mod the BIOS myself. These cards are $3k a piece, at our cost...I just can't risk it, especially due to the delivery time frame.
 
Okay, I reviewed the PCI register output. I don't have the data to interpret the full register dumps on the cards, but I don't see anything out of place. No address space conflicts, no interrupt conflicts, PERR and SERR low.

BAH!
 
This is looking more and more like a device driver behavior. After reinstalling windows last night with a single card configuration, and loading the display driver last, windows came up (expected). Powered off, added second card, Windows came up, until it started installing the driver for the second card. Once the driver for the second card was loaded, the entire machine shut down.
 
That is just illogical....I realize that is what is happening, but why would it be setup to do something like that?
I mean wouldnt PNY want you to buy more than one?

Let us know what they tell you.
 
Ya no shit. I am banging away at my PNY contacts list now. Supermicro has updated and said that they have sent the PCI device scan over to their BIOS engineer to help narrow the scope of what's going on.
 
Can anyone give me any information on where to get ahold of some earlier driver releases, either from PNY or Nvidia? I can't seem to find any way to access earlier drivers

ninja: nevermind...found 7x previous.
 
Last edited:
I finally called in on the desktop XLR8 card side of tech support, and this guy's is being nice enough to open the ticket for me, after telling him that the two previous calls to the Quadro support line have gone unreturned. I have downloaded the 7x previous Nvidia listed WHQL drivers to test on the system, when I get home, this afternoon. In the meantime, I've got Supermicro and PNY agreeing to assist from their end. How frustrating.
 
sabregen said:
I finally called in on the desktop XLR8 card side of tech support, and this guy's is being nice enough to open the ticket for me, after telling him that the two previous calls to the Quadro support line have gone unreturned. I have downloaded the 7x previous Nvidia listed WHQL drivers to test on the system, when I get home, this afternoon. In the meantime, I've got Supermicro and PNY agreeing to assist from their end. How frustrating.
Click to expand...

You'll need to call desktop support, then request a team lead in Quadro. That's the only way you can get anyone at all in Quadro; they still have less than a dozen people, if not less than a half dozen. PNY won't help Supermicro at all; they never do. It's empty promises. This is part of why I refuse to sell or support Quadro; I cannot get any support from PNY, even for obviously failed hardware.
You don't have any additional cards in the system, correct?
 
You must log in or register to reply here.
Back
Top