Xeon + consumer chipset = ECC support?

[mux]

I'm currently investigating a kind of gray area of Intel specifications that is ECC support. In the past, ECC support was only a chipset thing. Buy a certain chipset, get the relevant checkbox features. Great.

Right now, as memory support has become a CPU thing but sockets are interchangeable, it's physically possible to put for instance a Xeon-1225L into any H61 chipset motherboard. My question is: do all memory support features just depend on the processor, or does the motherboard and/or chipset still matter? Specifically, do I get ECC support if I try this or not? Even more specifically: I'm aiming at putting a Xeon-1225L into an Intel DH61AG or one of the other upcoming 19V-accepting mini-ITX motherboards. As server chipsets don't come in mITX form factor, this would be a totally awesome combination.

And if not, what exactly is the problem? Are there physical wires missing on the motherboard? is it just a software thing, an SPD thing maybe? Is it an 'ECC on/off'-control line on the DIMMs? What I'm trying to get at here is: I'm a guy who is willing to mod the heck out of motherboards and any other PCB for that matter. Can I 'fix' ECC support like this on any motherboard? The answer I'm hoping not to get is: it's a BIOS thing.

 

[elph]

This would be totally awesome. Not sure what the answer is, but I'll definitely be keeping an eye on this thread.

 

[German Muscle]

Back then Memory controllers were on the motherboards, now the memory controllers are on the CPU die.

 

[paperwastage]

Right now, as memory support has become a CPU thing but sockets are interchangeable, it's physically possible to put for instance a Xeon-1225L into any H61 chipset motherboard. My question is: do all memory support features just depend on the processor, or does the motherboard and/or chipset still matter? Specifically, do I get ECC support if I try this or not? Even more specifically: I'm aiming at putting a Xeon-1225L into an Intel DH61AG or one of the other upcoming 19V-accepting mini-ITX motherboards. As server chipsets don't come in mITX form factor, this would be a totally awesome combination.

any reason for a xeon 1225L when a 2500 can do?(i guess besides the IGP)

And if not, what exactly is the problem? Are there physical wires missing on the motherboard? is it just a software thing, an SPD thing maybe? Is it an 'ECC on/off'-control line on the DIMMs? What I'm trying to get at here is: I'm a guy who is willing to mod the heck out of motherboards and any other PCB for that matter. Can I 'fix' ECC support like this on any motherboard? The answer I'm hoping not to get is: it's a BIOS thing.

knowing Intel, it won't be as easy as that... maybe someone else can answer that

 

[pxc]

ECC memory uses a few additional signals between the DIMM slots and memory controller. On newer Intel CPUs, the memory controller is built into the CPU. The chipset (actually is a south bridge on desktop platforms and is called a PCH) does not directly connect to memory like in Core 2 and earlier Intel processors.

The physical difference in boards that support ECC or not is the addition of the check bit traces between the memory controller and DIMM slots. Cheap desktop boards almost certainly lack those traces and any other support to use ECC memory. Look at the qualified memory list on boards to see which support ECC memory.

 

[mux]

That actually sounds great, because it means that 1) I can measure if those lines are connected and 2) I can hotwire those pins to to the DIMM slots. The only problem being that Intel hasn't published the processor pinout for LGA1155 AFAIK. Does anybody know actual pinouts for ECC DIMM and LGA1155?

No consumer boards 'support' ECC memory because none of the boards actually accept Xeon processors. And that answers paperwastage's question too: a 2500 doesn't support ECC (at least not officially, until proven otherwise). I'd be happy to try it out if I had the financial means to mess up a board and processor or two, though.

 

[Blue Fox]

That actually sounds great, because it means that 1) I can measure if those lines are connected and 2) I can hotwire those pins to to the DIMM slots. The only problem being that Intel hasn't published the processor pinout for LGA1155 AFAIK. Does anybody know actual pinouts for ECC DIMM and LGA1155?

No consumer boards 'support' ECC memory because none of the boards actually accept Xeon processors. And that answers paperwastage's question too: a 2500 doesn't support ECC (at least not officially, until proven otherwise). I'd be happy to try it out if I had the financial means to mess up a board and processor or two, though.

Incorrect. Pretty much any board will accept Xeon CPUs. Many consumer boards also support ECC. This was fairly common on X58 for example (Asus P6T6 comes to mind). ECC support is mostly dependent upon the CPU at this point. The current Core i3 and Pentium lines support it for example.

 

[mux]

Okay, however, prior to that post I have gone through the entire MSI and ASUS product catalog and none of them list Xeon CPUs as supported. Obviously, that doesn't mean they absolutely don't work, but it at least shows anecdotally that these boards haven't been designed with ECC support in mind.

Which leads me to the following: indeed, from memory documentation I gather that ECC memory needs exactly 8 additional lines from CPU to memory bank (i.e. 8+1 bit ECC with a bus width of 64 bits). Without any hardware to back this up, i'd say it's possible that a substantial amount of boards, if not all, don't have these lines installed.

If I look through the Intel datasheets on the subject, I read the following:

The type of memory supported by the processor is dependent on the PCH SKU in
the target platform
— Desktop PCH platforms support non-ECC un-buffered DIMMs only
— All In One platforms (AIO) support SO-DIMMs

So your claim that current Core i3 and Pentium lines support ECC is either not true or something on consumer platforms disables ECC functionality.

Coincidentally, the documentation also lists the pinout that I earlier thought was not published. Silly me. I'm looking through it right now.

 

[Sycraft]

I would rather doubt it. First off, Xeons seem to be going to different sockets. In the SB series they use LGA 1567 instead of 1155, so I'm going to guess they won't fit. Also I'm not sure that you will find ECC memory that works in normal sockets. These days the server/ECC stuff is all FBDIMMS, which are designed not to work in normal sockets since they aren't directly compatible. They have additional electronics on board to allow for better electrical stability and thus more RAM.

It would theoretically be possible if you got a board that supported the chip you wanted, and ECC RAM that fit in the sockets. Since the memory controller is on the chip with SB stuff, there isn't anything else that should be needed.

However it doesn't look to me like that is very feasible.

 

[mux]

I would rather doubt it. First off, Xeons seem to be going to different sockets. In the SB series they use LGA 1567 instead of 1155

Well no, they use the exact same LGA1155. Pinout is identical as well. Check it yourself: pentium/i3/i5/i7 vs Xeon LGA1155 (pages 90 and onwards). The only difference in the datasheet are the CFG pins, which are not actually different on the xeon, they just printed the characters in white on a white background. So they're pin-for-pin the same sockets.

, so I'm going to guess they won't fit. Also I'm not sure that you will find ECC memory that works in normal sockets. These days the server/ECC stuff is all FBDIMMS, which are designed not to work in normal sockets since they aren't directly compatible. They have additional electronics on board to allow for better electrical stability and thus more RAM.

The only difference between buffered and unbuffered RAM is the AMB onboard, which needs additional circuitry. However, you can easily find normal DDR3 sticks with ECC support which aren't fully buffered (just registered). No problems there.

It would theoretically be possible if you got a board that supported the chip you wanted, and ECC RAM that fit in the sockets. Since the memory controller is on the chip with SB stuff, there isn't anything else that should be needed.

However it doesn't look to me like that is very feasible.

In this thread I'm trying to compile the know-how on exactly WHY this is or isn't possible. There's enough people that have told me either way but without certainty. On multiple occasions they have referred me to Hardforum. Already this thread has yielded more concrete knowledge than anywhere else, so I'm confident we will find out the answer.

 

[Blue Fox]

Okay, however, prior to that post I have gone through the entire MSI and ASUS product catalog and none of them list Xeon CPUs as supported. Obviously, that doesn't mean they absolutely don't work, but it at least shows anecdotally that these boards haven't been designed with ECC support in mind.

Which leads me to the following: indeed, from memory documentation I gather that ECC memory needs exactly 8 additional lines from CPU to memory bank (i.e. 8+1 bit ECC with a bus width of 64 bits). Without any hardware to back this up, i'd say it's possible that a substantial amount of boards, if not all, don't have these lines installed.

If I look through the Intel datasheets on the subject, I read the following:



So your claim that current Core i3 and Pentium lines support ECC is either not true or something on consumer platforms disables ECC functionality.

Coincidentally, the documentation also lists the pinout that I earlier thought was not published. Silly me. I'm looking through it right now.

They may not list them, but I can assure you they'll work. They are no different than the desktop components except for the fact that they support registered ECC RAM (certain desktop counterparts support unbuffered ECC) and the lack of any IGP (some Xeon CPUs do have it though).

In modern sockets, this is dependent not only on the CPU but also the PCH. To prove a point, here's a 1155 motherboard that not only support Xeon CPUs, but also lists Core i3 and Pentium lines as well (with ECC support): http://www.supermicro.com/products/motherboard/Xeon/C202_C204/X9SCA-F.cfm

I would rather doubt it. First off, Xeons seem to be going to different sockets. In the SB series they use LGA 1567 instead of 1155, so I'm going to guess they won't fit. Also I'm not sure that you will find ECC memory that works in normal sockets. These days the server/ECC stuff is all FBDIMMS, which are designed not to work in normal sockets since they aren't directly compatible. They have additional electronics on board to allow for better electrical stability and thus more RAM.

It would theoretically be possible if you got a board that supported the chip you wanted, and ECC RAM that fit in the sockets. Since the memory controller is on the chip with SB stuff, there isn't anything else that should be needed.

However it doesn't look to me like that is very feasible.

Wrong again. There are Xeon chips available in 775, 1366, 1156, and 1155. Which you rather that I use for the sake of argument (see above for an example)? FBDIMMs were exclusively used on socket 771. No current ECC memory is fully buffered and therefore not an FBDIMM. The last part is also not completely true. While registered memory does take a strain off the memory controller, all modern server motherboard (Intel and AMD) will permit you to completely fill each slot with non-ECC memory (which also therefore will not be registered).

 

[mux]

I have to amend my earlier statement that the sockets are exactly the same: the ECC pins on the desktop LGA1155 have 'RSVD' (reserved) pins in the positions where the ECC pins are on the Xeon LGA1155.

 

[iconiK]

Incorrect. Pretty much any board will accept Xeon CPUs. Many consumer boards also support ECC. This was fairly common on X58 for example (Asus P6T6 comes to mind). ECC support is mostly dependent upon the CPU at this point. The current Core i3 and Pentium lines support it for example.

No non-Xeon CPU supports ECC memory, apart from some embedded mobile Core i. Even if you stick a Xeon, the BIOS has to support it, if they had the traced routed. X58 boards DO have them and DO support ECC with a Xeon... well as long as the board is decent. AM3/AM3+ also supports ECC, as all AMD CPUs have ECC enabled, it's just a matter of whether the BIOS does, and on most mobos it does, even if it isn't stated in the specs, the hardware compatibility list typically lists an ECC DIMM like CT25672BA1339.

First off, Xeons seem to be going to different sockets. In the SB series they use LGA 1567 instead of 1155, so I'm going to guess they won't fit.

LGA 1567 is Nehalem-EX and Westemere-EX, these are 8 and 10-core respectively, with quad channel memory, and up to 4-way/8-way respectively. SB-E isn't out yet, that's LGA 2011. Normal SB CPUs are E3-1200 and they use LGA 1155.

Also I'm not sure that you will find ECC memory that works in normal sockets.

DDR3 uses the same DIMM, be it ECC or not, registered or not.

These days the server/ECC stuff is all FBDIMMS, which are designed not to work in normal sockets since they aren't directly compatible. They have additional electronics on board to allow for better electrical stability and thus more RAM.

Fully Buffered DIMMs died with DDR2. DDR3 is either unbuffered, unbuffered with ECC or registered with ECC (1156, 1366 and 1567 Xeons only; 1155 does NOT support registered ECC, so no cheap 8GB DIMMs).

It would theoretically be possible if you got a board that supported the chip you wanted, and ECC RAM that fit in the sockets. Since the memory controller is on the chip with SB stuff, there isn't anything else that should be needed.

BIOS support, also ECC traces on the motherboard.

However it doesn't look to me like that is very feasible.

ASUS P8B WS + Xeon E3-1235 = best deal ever. Cheaper than i7 2600, makes up for extra mobo cost. ECC memory ain't that much more either.

 

[mux]

What exactly does the BIOS do that enables ECC support, if that is the only missing link?

 

[iconiK]

What exactly does the BIOS do that enables ECC support, if that is the only missing link?

To enable parity checking and correction of course. In addition, it advertises that to the OS.

 

[E4g1e]

Okay, however, prior to that post I have gone through the entire MSI and ASUS product catalog and none of them list Xeon CPUs as supported. Obviously, that doesn't mean they absolutely don't work, but it at least shows anecdotally that these boards haven't been designed with ECC support in mind.

Actually, those ASUS P67 and C206 motherboards with a "WS" in the model name (e.g. P8P67 WS Revolution and P8B WS) support those E3 series Xeons.

 

[mux]

To enable parity checking and correction of course. In addition, it advertises that to the OS.

This is - pardon me - stating the obvious. Of course 'in some nebulous way' it enables ECC. I want to know that nebulous way! Does it do this with a simple hardware switch, does it require commands to be sent to the CPU, if so, what commands and via what communication channel?

In other words: can I do anything about it? (well, of course I can, but is it easy enough?)

I'm not trying to think inside any box here. I'm willing to take up my soldering iron and embedded hard/software skills to make this happen. I just lack the specific knowledge and you guys have been awfully helpful so far in giving me pointers.

@E4g1e: You are totally right, but that still doesn't yield a great deal of choice, especially in mITX. And that is my end goal. You are very right in pointing out that P67 motherboard though, I have obviously not been thorough in my research.

 

[Blue Fox]

No non-Xeon CPU supports ECC memory, apart from some embedded mobile Core i. Even if you stick a Xeon, the BIOS has to support it, if they had the traced routed. X58 boards DO have them and DO support ECC with a Xeon... well as long as the board is decent. AM3/AM3+ also supports ECC, as all AMD CPUs have ECC enabled, it's just a matter of whether the BIOS does, and on most mobos it does, even if it isn't stated in the specs, the hardware compatibility list typically lists an ECC DIMM like CT25672BA1339.

Once again, that is incorrect. The Core i3 and Pentium lines in both 1156 and 1155 support unbuffered ECC with the proper PCH. Look around the storage subforum and you'll find plenty of people running systems like that. Even the motherboard I linked shows that it's supported. As for the Xeon support, you are wrong about that too. You don't need extra traces on the board to support it. Motherboard that do not support ECC RAM will still support Xeon CPUs.

 

[Dan_D]

This is - pardon me - stating the obvious. Of course 'in some nebulous way' it enables ECC. I want to know that nebulous way! Does it do this with a simple hardware switch, does it require commands to be sent to the CPU, if so, what commands and via what communication channel?

In other words: can I do anything about it? (well, of course I can, but is it easy enough?)

I'm not trying to think inside any box here. I'm willing to take up my soldering iron and embedded hard/software skills to make this happen. I just lack the specific knowledge and you guys have been awfully helpful so far in giving me pointers.

@E4g1e: You are totally right, but that still doesn't yield a great deal of choice, especially in mITX. And that is my end goal. You are very right in pointing out that P67 motherboard though, I have obviously not been thorough in my research.

Technical stuff aside, just why the hell are you so hell bent on having ECC support? In reality it isn't going to do anything for you. It isn't as if ECC capable systems are impervious to crashes.

And according to ASUS' website, only their "WS" series boards at present support the Xeon or ECC memory. Yes it is true that all X58 boards work with many Xeon processors, however this support is unofficial. I have heard of issues using Xeons on some X58 boards like the EVGA X58 3X SLI Classified Edition with regard to early BIOS ROMs lacking the necessary microcode support for those CPUs. As to whether or not you can actually use ECC memory with a board that doesn't officially support it, I'm going to say, "probably not." At least not without the BIOS options to do so. Without those, there is nothing you can do without the knowledge necessary to write / modify a BIOS and flash your board with it. There is a potential need for some trial and error which could get expensive. And while the Rampage III Formula and Rampage III Black Edition do not officially support ECC in their specifications and in their manuals, (or Xeon's for that matter), I believe it will work on those boards, because I believe they have the settings in BIOS to toggle that on or off.

 

[iconiK]

I want to know that nebulous way! Does it do this with a simple hardware switch, does it require commands to be sent to the CPU, if so, what commands and via what communication channel?

Well... that's information under NDA. Why do you think coreboot doesn't support all motherboards? Because the BIOS code and all instructions necessary to initialize the CPU, RAM and other components are classified.

In other words: can I do anything about it? (well, of course I can, but is it easy enough?)

As easy as writing a new BIOS, or disassembling and modifying the existing one.

Once again, that is incorrect. The Core i3 and Pentium lines in both 1156 and 1155 support unbuffered ECC with the proper PCH.

Intel does not list support for ECC on any CPUs. Yes, you can stick ECC memory in any motherboard, but it WILL NOT use the error-correcting code, thus you have paid for a feature you can't use. Get your facts straight.

Motherboard that do not support ECC RAM will still support Xeon CPUs.

You need extra traces to support ECC, not the CPU.

 

[Blue Fox]

Intel does not list support for ECC on any CPUs. Yes, you can stick ECC memory in any motherboard, but it WILL NOT use the error-correcting code, thus you have paid for a feature you can't use. Get your facts straight.

Is this not clear enough? http://www.supermicro.com/products/motherboard/Xeon3000/3400/X8SIL.cfm?IPMI=N

Dual Core processors of Ci3 and Pentium: support ECC UDIMM only

 

[iconiK]

Is this not clear enough? http://www.supermicro.com/products/motherboard/Xeon3000/3400/X8SIL.cfm?IPMI=N

Umm... you can stick ECC unbuffered and it'll run, but ECC won't work. If you stick registered it won't even post. The MOTHERBOARD supports ECC unbuffered and registered, but the CPU HAS TO support it as well, that's where the memory controller is.

EDIT: On further research, X8SIL-F suports ONLY ECC RAM, NON-ECC won't post. That's why they say to stick ECC unbuffered memory and not registered or non-ECC. I believe it's a limitation of the Xeon 3400 chipset or Supermicro's BIOS.

 

[mux]

Technical stuff aside, just why the hell are you so hell bent on having ECC support? In reality it isn't going to do anything for you. It isn't as if ECC capable systems are impervious to crashes.

I want all my future systems to have ECC support if I can do anything about it. Correctable memory glitches are a major if not the main hardware cause of system and application crashes. As per this white paper, high-utilization DRAM has in the order of thousands or tens of thousands of correctable errors a year. Even low-utilization desktop systems will see hundreds of correctable errors per year, of course most in unused memory. This is at least an order of magnitude more than the theoretical error density in hard drives and two orders more than what occurs in flash drives through cosmic rays, which AFAIK is the dominant bit failure mechanism. Of course it's not going to help me against bad programming and all the other million causes, but this seems to be one of the most underrated problems in modern-day computers.

Besides, even if that weren't true (and i'm always open to falsification) I still want to know the how and why. I'm a power electronic engineer and embedded electronics guy by trade and fortunately, VRMs and DC-DC converters are 'readable' by just looking at them, I don't need any external help for that. This problem, however, is a lot less clear to me and I'm just curious as to how it works. I'm surprised that the CPU doesn't just 'use' the ECC memory when it's available but needs some sort of command to start using it. It seems strange, because both the SPD (via the SCL/SDA lines) and memory lanes are directly accessible to the CPU. You would really think that the CPU itself does the SPD reading and enables/disables ECC when it's available, that is the 'engineering' solution, the easiest solution if you will. And usually, that solution is exactly what people use.

Edit: some more reading material (the references list a few nice papers).

 

[iconiK]

I'm just curious as to how it works. I'm surprised that the CPU doesn't just 'use' the ECC memory when it's available but needs some sort of command to start using it.

It's probably nothing more than market segmentation. Intel wants people to pay a premium for their Xeon chipsets and processors. For example, AM2+/AM3/AM3+ has supported ECC DDR3 with Phenoms since... forever, as long as the BIOS supports. Gigabyte has a few boards with official support, ASUS has MOST boards, even though it's not stated in the specs, there are ECC unbuffered memory modules in the HCL and ECC does work.

You should look at the coreboot source on some of the server motherboards. H8DMR-i2 is fully supported by coreboot v4, just stick one or two old Opteron 2000 series (eBay?) in there, like this Opteron 2376 HE.

 

[Dan_D]

I want all my future systems to have ECC support if I can do anything about it. Correctable memory glitches are a major if not the main hardware cause of system and application crashes. As per this white paper, high-utilization DRAM has in the order of thousands or tens of thousands of correctable errors a year. Even low-utilization desktop systems will see hundreds of correctable errors per year, of course most in unused memory. This is at least an order of magnitude more than the theoretical error density in hard drives and two orders more than what occurs in flash drives through cosmic rays, which AFAIK is the dominant bit failure mechanism. Of course it's not going to help me against bad programming and all the other million causes, but this seems to be one of the most underrated problems in modern-day computers.

Besides, even if that weren't true (and i'm always open to falsification) I still want to know the how and why. I'm a power electronic engineer and embedded electronics guy by trade and fortunately, VRMs and DC-DC converters are 'readable' by just looking at them, I don't need any external help for that. This problem, however, is a lot less clear to me and I'm just curious as to how it works. I'm surprised that the CPU doesn't just 'use' the ECC memory when it's available but needs some sort of command to start using it. It seems strange, because both the SPD (via the SCL/SDA lines) and memory lanes are directly accessible to the CPU. You would really think that the CPU itself does the SPD reading and enables/disables ECC when it's available, that is the 'engineering' solution, the easiest solution if you will. And usually, that solution is exactly what people use.

Edit: some more reading material (the references list a few nice papers).

I get it. Someone started doing a little research, read a white paper, and now is an expert in the field of computer hardware / software, and now believes ECC is essential to their computing needs. While that white paper has some truth to it, the "problem" is over stated in the desktop environment. The reality is that system crashes in well built machines are fairly rare. And instances in which a crash might have been prevented are hard to determine. In some cases a crash occurs and ECC wouldn't have changed the outcome of the crash.I can't tell you the last crash I had that wasn't related to some kind of hardware failure or bizarre software compatibility issue in which ECC wouldn't have made any difference. I've got machines with just as much uptime as my server with FB-DIMMs and are just as stable. My dual Opteron 254 system had registered ECC memory, and still crashed on occasion with early NVIDIA drivers back in the Windows XP Professional x64 Edition days. Other crashes were related to software on that OS as well. I also work in IT and i've seen many, many crashes on desktops and workstations with ECC memory. ECC isn't a guarantee of problem free computing. It's not a magic bullet for system stability. Bad software is still bad, bad drivers are still bad, OSes have issues, and sometimes hardware just fails. I've seen more than my fair share of ECC modules in data centers and workstations. I've probably had more machines with ECC memory in them than most people probably ever will. Again, it's not as big a deal to have it as you are making it out to be. For some uses, yes, I'd agree with using it and I'd even recommend it. But for Joe Blow playing Call of Duty or Counter Strike and ripping DVDs, hell no. There is little to no point in getting it.

I have to wonder what the hell has sparked all these ECC threads as of late? Why do some of the people posting in them think they need or want ECC in a gaming box, or normal desktop machine? Honestly who gives a shit if you get a 1,000 errors in a year which do not result in data corruption or a system crash, or even an application crash?

 

[tlc4tvm]

Yes, you can stick ECC memory in any motherboard, but it WILL NOT use the error-correcting code, thus you have paid for a feature you can't use. Get your facts straight.You need extra traces to support ECC, not the CPU.

Your absolutely correct, get your own facts straight before you critique so harshly,

I have 2 sticks of ddr2 ecc ram which don't work in my am2+ board, nor in my Dell XPS 774 socket ddr2 board. So just because its ram and fits doesn't mean it will work.

It's more likely the chipset bios doesn't recoginse the ECC feature of the CPU in the bios, so the CPU simply thinks ECC is turned off. Note; You can turn off ecc in some server boards so this makes sence.

 

[Dan_D]

It's been my experience that placing ECC RAM into motherboards that do not support it will result in a no POST condition.

 

[mux]

@Dan_D: I didn't want to imply anywhere that I'm an expert. As you said, the only evidence I have is a few white papers. I also said earlier that I would be just as happy if somebody falsified my claims.

You are just as anecdotal in your arguments as basically everybody I spoke to on the subject. Admit it - you have no statistically significant data to back up your claims. I have no use for more of that. There are a multitude of white papers and independent research papers that indicate that DRAM errors are a significant reliability issue, and they have the numbers to back it up. Sure, I totally agree with you that there are tons of other reliability issues. Really, I would immediately believe you if you said physical damage (assembly tolerances, damage during transit, installation error, etc.), software errors and user error outweigh correctable memory errors by an order of magnitude. There have been no studies (that I know of) that compare DRAM BER to other system failure statistics. But show me the numbers! I want evidence!

Let's sidestep for a moment to make some things more clear. This is not just another computer, I am assembling the knowledge right now for what you could call a personal though no less scientific project in the same vein as my 8.5W (idle) Core i3 system. You said earlier that I seemed hell-bent on ECC support, but that's not completely what I'm trying to convey. I ask the things I ask to methodically get to an actual answer, not just some generic application information. I want to know what there is to know on the subject, and if that's not possible: I want to know what questions cannot be answered. Most other people just ask questions on the application of a technology, i.e. what thermal compound to use. Go with arctic silver. Question answered. I want to know why exactly you say arctic silver, what compound in there makes it the best and where I can find the numbers to back it up. Well, in this case I do. I'm not always as anal as I am in this thread.

As a reference, the system I am trying to build (and that I am investigating right now) will be a combination of three concepts of which two I have built earlier: (1) a completely passively cooled mini-ITX or smaller PC (already built a 45W max. mITX PC that can run passively in a Mini-Box M350), (2) a computer that uses 5W DC in idle (very confident that I can make this happen with the knowledge I gathered from the 8.5W build) and (3) a few extra features that I want for video editing (Quick Sync is one of them). I'm trying to build a 'have your cake and eat it too'-computer. ECC functionality would be a bonus. However, it's one of the first things to research, because it changes what components I've got to look for.

I hope this gives you enough background information as to why I'm asking what I'm asking.

 

[IntelEnthusiast]

Let me see if I can help a little on this. With the c204 and c206 chipset they do have support for ECC memory. The Intel® board that uses the c206 chipset which is the Intel S1200BT again does support ECC memory and it does support the Intel Core™ i3-2100, i3-2100T, i3-2105 and i3-2120. Based on the information on page 17 of the technical product specification that can be found here http://download.intel.com/support/motherboards/server/s1200bt/G13326-003_S1200BT_TPS_v1_0.pdf. The Intel Core i3 listed above do support ECC memory when on a board that does support them and the ECC memory. So if you want to pay for a WS or Server board that does support ECC memory and use a low power processor on it you are welcome to use the Intel Core i3s listed above.

 

[Dan_D]

@Dan_D: I didn't want to imply anywhere that I'm an expert. As you said, the only evidence I have is a few white papers. I also said earlier that I would be just as happy if somebody falsified my claims.

You are just as anecdotal in your arguments as basically everybody I spoke to on the subject. Admit it - you have no statistically significant data to back up your claims. I have no use for more of that. There are a multitude of white papers and independent research papers that indicate that DRAM errors are a significant reliability issue, and they have the numbers to back it up. Sure, I totally agree with you that there are tons of other reliability issues. Really, I would immediately believe you if you said physical damage (assembly tolerances, damage during transit, installation error, etc.), software errors and user error outweigh correctable memory errors by an order of magnitude. There have been no studies (that I know of) that compare DRAM BER to other system failure statistics. But show me the numbers! I want evidence!

Let's sidestep for a moment to make some things more clear. This is not just another computer, I am assembling the knowledge right now for what you could call a personal though no less scientific project in the same vein as my 8.5W (idle) Core i3 system. You said earlier that I seemed hell-bent on ECC support, but that's not completely what I'm trying to convey. I ask the things I ask to methodically get to an actual answer, not just some generic application information. I want to know what there is to know on the subject, and if that's not possible: I want to know what questions cannot be answered. Most other people just ask questions on the application of a technology, i.e. what thermal compound to use. Go with arctic silver. Question answered. I want to know why exactly you say arctic silver, what compound in there makes it the best and where I can find the numbers to back it up. Well, in this case I do. I'm not always as anal as I am in this thread.

As a reference, the system I am trying to build (and that I am investigating right now) will be a combination of three concepts of which two I have built earlier: (1) a completely passively cooled mini-ITX or smaller PC (already built a 45W max. mITX PC that can run passively in a Mini-Box M350), (2) a computer that uses 5W DC in idle (very confident that I can make this happen with the knowledge I gathered from the 8.5W build) and (3) a few extra features that I want for video editing (Quick Sync is one of them). I'm trying to build a 'have your cake and eat it too'-computer. ECC functionality would be a bonus. However, it's one of the first things to research, because it changes what components I've got to look for.

I hope this gives you enough background information as to why I'm asking what I'm asking.

I get what you are saying. I love knowing the "why" as well. But at some point you have to either go to school to learn to engineer these specific pieces of hardware, or you learn to accept certain things. There are tons of erratum that occur with each piece of hardware in your computer. If I had just read the white paper (or one like it) on the subject of ECC memory for the first time, I'd probably be a little scared on some level and think, "Oh my god, I have to have ECC memory in all my machines now." I've read plenty of information on the subject, but I've tempered that knowledge with experience. Yes, I know about ECC and DRAM errors. I just don't care about that beyond scenarios where I need that extra reliability. The reality of it is, you probably won't notice a difference in practice between one configuration with ECC and the same configuration without it. If you go and read the errata for the Athlon 64 X2 or the Phenom, you'd probably shit yourself. It's rough. One could easily make a conclusion that those CPUs were pieces of shit and weren't fit to use. (The errata really did read like that.) But at the time the Athlon 64 X2 was new, it was faster than anything Intel had to offer, and a machine built with the right hardware was just as stable in practice. There is a fine line between what is technically true and reality. At the end of the day the user experience is all that matters.

You can build for peace of mind but the reality is, I don't think you'd notice any difference using the same hardware on a day to day basis whether ECC memory was in the box or not. You made it sound like a rabid obsession when you were talking about getting out a soldering iron, running wire, building trace paths, etc. to make it work. If you need ECC memory support that badly, buy a motherboard and processor that support the feature, along with the right RAM and be done with it. I just don't think you have many if any options for small form factor machines, but if ECC is that important, you'll buy what you really need.

 

[iconiK]

Based on the information on page 17 of the technical product specification that can be found here http://download.intel.com/support/motherboards/server/s1200bt/G13326-003_S1200BT_TPS_v1_0.pdf. The Intel Core i3 listed above do support ECC memory when on a board that does support them and the ECC memory.

Once again, the board supports ECC, but the memory controller does NOT. So if you stick ECC in, it'll run, but the ECC function will be disabled. All server boards list Core i3 because Intel explicitly does not support i5 and i7 on Xeon chipsets, only i3 and Xeon (to make it easy for OEMs to make cheap upgradeable first-server systems).

 

[mux]

But at some point you have to either go to school to learn to engineer these specific pieces of hardware, or you learn to accept certain things.

What I was trying to say with the soldering iron and all, is that I am actually at a level where I can do such a thing. I'm willing to go the extra mile to do something 'new' that might be worth it, or even just for kicks.

There are tons of erratum that occur with each piece of hardware in your computer. If I had just read the white paper (or one like it) on the subject of ECC memory for the first time, I'd probably be a little scared on some level and think, "Oh my god, I have to have ECC memory in all my machines now."

If I sounded that way, which I might have in retrospect, that was unintentional. I don't feel like I need ECC, I would just like to have it. Wait, that still sounds wrong.

I just don't think you have many if any options for small form factor machines, but if ECC is that important, you'll buy what you really need.

When I started this thread, I was really under the impression that it shouldn't take hardware and BIOS modifications, so that it would be a real though inconspicuously unlisted option. With the above knowledge, especially IntelEnthusiast's post (which I still need to verify), it seems inevitable that there is indeed something that needs to happen in BIOS to enable ECC. That kind of rules out ECC for my machine, because I'm more hell-bent, if you will, on mITX than on ECC. Accessing/probing the CFG-lines and reverse engineering that stuff can't really be done with my analyzer scope, nor any other scope I have access to. If it would just have stayed at connecting the ECCA[0-8] and ECCB[0-8] lines I would have certainly given it a shot.

However, that doesn't mean this is dead in the water, I still really want to know exactly what must happen to enable ECC. Intel is pretty open (more open than AMD) on datasheets and application notes and I'm sure someone or some website knows more about this. I can only deduct from the chipset/processor combinations that advertise ECC support that there is more than just hardware that determines ECC support, I still don't know exactly why this is.


EDIT: I spoke too soon, iconiK has the answer. This still doesn't rule out my possibilities. I should refresh before replying

 

[iconiK]

EDIT: I spoke too soon, iconiK has the answer. This still doesn't rule out my possibilities. I should refresh before replying

Pretty much what I've been saying this whole page... repeated several times. Xeon board can take ECC, but you absolutely need Xeon CPU to use it. Short summary.

Here's something for you: http://www.portwell.com/products/detail.asp?CUSTCHAR1=WADE-8011

No I don't know where to get it from except directly from them. Yes, it will cost significant coin if you can get it in a small number AT ALL. µATX is the cheapest you can go with ECC.

 

[mux]

That's an excellent find. Maybe they can get me a sample board or something, who knows.

 

[Xennex81]

Hi Mux, it is more than 3 years ago, but...

Such a shame that you let your depression take hold of you, and not go through with your clear desire!!

The funny thing is....

My intuition is really strong (when I'm not the subject of any witch hunt directed at my person, as it has been as of late). Here was my idea:

I want to build a development server for my software needs. This server will also turn into a host for a certain network environment, that is going to turn into something of a social network / communication thing.

So, stability is very important to me. EXTREMELY.

Based completely on my intuition, I chose these components: I still have to order them::

- Gigabyte mITX board GA-H61N-D2V
- Intel i3 - 3220T 3rd Gen Core 2.8 35W.
- Crucial 32 GB ECC DDR3 1333 Mhz.

Now obviously, or perhaps not, but... this board will never by default accept 32 GB (instead of 16) ECC memory.But I know, thanks to this old thread, believe that the solution lies in a few lanes of wire to be drawn along the backside of the board.

First of all I'll need to get ECC working. I believe that you were entirely correct in this thread that it will take no more than a few physical wires, and no BIOS mod.

But the board will also not accept 32 GB, because this chipset will only allow 32 GB as 4x 8, and I want 2x 16 [..correction, the CPU, whatever]. That's another story, for another day, but I'm going ahead with this anyway as far as my mental stability will allow me to.

I have no experience (almost) in electrical engineering, but I have an astute (keen?) mind and once I get the info, soldering won't be hard to learn.

And besides, perhaps someone here (you? ) might be able to help me even.

My intuition says the following for this board and memory:

- it will only require wires between CPU socket and PCH.
- nothing required between memory and CPU (for the ECC mod, it will require this for the 32 GB mod).
- the "RSVD" pins won't actually matter for me.
- it will just requires those (eight?) wires for me.

Now for the 32 GB mod.....

My intuition might be able to say:

- it's not that the chipset won't support it. The CPU supports 32 GB over 2 channels, but H61 is said to only support 1 DIMM per channel. To be more precise, 1 double sided DIMM per channel. Apparently, this means two double sided DIMMs of max 8 GB, or four single sided DIMMS of max 4 GB. Now, obviously (!!?) the H61 is just a rebrand of the H67. Intel not going to produce almost identical chips using a different manufacturing process, right? Intel not going to spend money on that. Intel just going to break some bridges on the chip to disable functionality. (I only ever once closed some bridge(s) on a CPU to get something working again, that was an AMD Sempron, I believe).

Of all the Sandy Bridge chipsets, ONLY the H61 doesn't support 16 GB per channel. We don't believe that, do we? Now this board also doesn't support RAID, but that's perhaps easiest to fix with an addon card in that single PCI slot. What is more, H67 (and P67) were released one month prior to H67 (and B65). Then the only difference between H61 and H67 is RAID and the number of SATA/USB ports and the like. [[(Also, I seriously want to stay away from USB 3.0, it is a failed and flawed idea and technology, in my perception and truth)]].

So what I would assume and expect is that these chips are exactly the same, but they get a laser to cut some bridges. To change the behaviour. And then brand them differently.

[[ I will get 4 3.5" HDDs in a RAID 0 (stripe) in the end. A single PCI SATA RAID controller can handle 4 disks. The case I will use allows for 2 expansion slots. It shouldn't be hard to find a SATA to eSATA bracket for backup purposes. FOUND and under €10. Then my external ports will just run from the mainboard. ]]

If you check the block diagrams for H61 and H67 there is hardly a difference:

So what then of the memory? If the H67 supports 4 sides per channel, (2 doublesided dimms) and the H61 only 2 sides per channel (1 doublesided dimm), then that must mean that:

• there are some wires missing
• there are some bridges burned
• you just need to double the wires
• you need to reconfigure the bridges using e.g. silver paint (hope it is that easy...)

The extra wires then go from the memory to the CPU, and only on the PCH is a mod necessary. That is what I think.

Other than that, the BIOS of the motherboard will just be a minor branch of their main BIOS that they also use for other chipsets. They won't even be bothered to disable some invisible functionality. It might even just pull a register or port to check whether to enable something; that would be the most maintenance-friendly way to do that.

 

[wirk]

So, stability is very important to me. EXTREMELY.

You wanna have a system with the big E... word and even with the ECC memory, and make it messing with soldering wires and getting over standard RAM configuration . Make it decent man, buy X99 mobo, cheap Xeon v3 processor and ECC memory for it.

 

[Xennex81]

You wanna have a system with the big E... word and even with the ECC memory, and make it messing with soldering wires and getting over standard RAM configuration . Make it decent man, buy X99 mobo, cheap Xeon v3 processor and ECC memory for it.

Where is the fun in that? That way I would not learn anything. And besides, I have good reasons for doing it this way. It has to be mini-ITX. There is no room in my house for a bigger case, in that room.

It is a bedroom too. It has to be quiet as well. Memory and harddisk are more important than CPU speed, for my purposes.

Besides... achieving something that is difficult, can only make you more skilled and thereby more empowered in the future in case you need that expertise for other things. I also want to do Sega Saturn programming, I want to work on Squeezebox firmware, I want to do a lot of things still. You shouldn't immediately assume that "getting the system running stably" is my only aim or requirement or desire. I also want to develop electronical engineering skills, at least for now the physical aspects as well as initial embedded device knowledge (PCB manipulation, ...) are important. Also, I do not want USB3 (at least, on-board) I want a serial port and a parallel port.....

Socket 2011-3 boards are also not cheap. It is pretty clear then that a typical server board is not really my thing....

The funny thing is that the CPU doesn't support ECC supposedly but it has all the ECC-RSVD pins listed ;-).

Getting all this knowledge about how memory works is a lot of fun and will prove to be extremely invaluable. Another E- word ;-).

 

[Xennex81]

Thus far I found out not very concrete, final things, but....

Those Intel CPUs support at most 4 ranks per channel. All modules have 8 banks per rank, so that is not an issue. Typically you would have 2 x 8 GB modules for each channel. That would imply 2 ranks per module, and 16 chips per module (8 on each side). Which comes down to 8 chips per rank. But those 16 GB ECC modules have 36 chips per module. The CPU supports up to "4Gb" chips which then normally comes down to 4 x 16 / 8 = 8 GB. It (the motherboard) would need additional lanes to the sockets to get that addressing (chip-select) up. Not because the CPU doesn't support it, but because it is normally spread over 2 sockets instead of one, I guess. A wikipedia text suggests that the CPU doesn't really care about PCBs (sockets/modules), it only cares about ranks and banks and chips.

The channels are completely independent, so that is not an issue. The issue is how to get 16 GB addressable over one socket instead of two. The supported 8 GB configuration involves 4 Gb chips of 8 bits (which implies 8 chips per rank, to get to 64 bits) which then implies 512M blocks per chip, and 16 chips on the module (dual sided). What I want to achieve at this point, is to get 16 GB modules of 4Gb x 8, but with either 4 ranks on one module (4R x8) .. well actually I think that is the only option, if the CPU is hardcoded to only accept that configuration. It is the most straightforward way to get the CPU to treat 1 module as if they were two.

There don't seem to be any 4R x8 modules though, they are all (the only one, Kingston) 2R x4. So that is 1G blocks 4 of bits, but according to the specs, these CPUs don't support any x4. Well, I will see later.

 

[Xennex81]

Funny thing is, I am still doing this thing. Not much advanced but I have obtained at least a 4Rx8 set of 2x 16GB RAM modules. Currently writing about it on the Debian forums.

It was a rebate too. Something that was original about 400 and now about 200 euro? Gave me enough spare money to buy some normal 2x8GB chips as well ;-).

 

[GrandAdmiralThrawn]

Hey,

I just registered here for this thread! I am trying to do something similar with a Xeon X5690 on an ASUS P6T Deluxe (X58 Tylersburg). I do know already that my BIOS is likely missing the code necessary to "initialize" ECC. On top of that, the traces on the board may be missing too.

However, that's not why I wanted to post.

While researching the matter I found a solution for AMD platforms on Linux. Even if processors and chipsets might not pose a problem on the AMD side, the BIOS can. Amazingly, there seems to be a way to force-enable ECC at runtime!

To do this, one needs to use Linux, and load the AMD EDAC (error detection and correction) kernel driver with a specific override option set. So let's set it, and then unload/load the kernel module:

echo "options amd64_edac_mod ecc_enable_override=1" >> /etc/modprobe.d/99local
modprobe -r amd64_edac_mod && modprobe amd64_edac_mod

You can then check your syslogs to see whether it actually worked. More information [here]!

Now maybe - just maybe - a similar mechanism exists, that can turn on ECC for Xeons? I inspected the module i7core_edac with modinfo -p i7core_edac, but unfortunately there are no such options for it it seems. Despite its name, the modules does handle ECC for Xeons too, at least Nehalems and Westmeres.

I looked through the C source code of i7core_edac.c [here], but it's a bit over my head. I can't find a clear function to enable ECC by writing to some MSR or whatever. But IF the code is there, it could be ported over to a small tool + kernelmode driver for Windows too... I also studied amd64_edac.c, but I can't really understand it either.

I thought I should share this, as it might help someone to make more progress on the matter.