Big Box

M

mikeblas

[H]ard|DCer of the Month - May 2006
Joined
Jun 26, 2004
Messages
12,777
Does anyone have access to a 4-way or 8-way Opteron machine with a lot of memory (6 gigs minimum on the 4 way; 10 gigs minimum on the 8-way)? I'd like to run a few benchmarks for a paper I'm writing ...
 
I've got a free quad Intel, 16Gb of RAM at work if need anything run on it.
 
I've a Dual-dual Opteron with 8 gigs of memory in it.
Its only got CentOS Linux64 as an operating system and I dont know much Linux.
So if the benchmarks are idiot proof under Linux, I'll run then.

Luck ........ :D
 
Are you registered at 2cpu.com? They have a larger multi-cpu userbase; you might have more luck over there.

The only 4-way Opterons I have access to are running Solaris and under production load, sorry.
 
Let me know what you need, I can see what I can do.
 
I'll bite if I don't have to do much. I thoroughly enjoy your posts. :) Not sure what I have lying around but off the top of my head, I have a DL585 4x 885 and 16GB mem, I would have to check. At the very least I have one that has 4x 880 and 8GB mem.

PM if you are interested (since you have a few offers already) and I'll send you my work email addy.
 
I'm happy to run benchmarks, but I don't do that Windoze thing ;)
 
It would involve just running a program and reporting its output to me. Thing is, it'll run for a couple of hours, total.

I'm writing a paper about NUMA, and want to compare the same algorithm on UMA and NUMA machines, plus NUMA machines where the access is non-local and where the access is carefully localized.
 
mikeblas said:
It would involve just running a program and reporting its output to me. Thing is, it'll run for a couple of hours, total.

I'm writing a paper about NUMA, and want to compare the same algorithm on UMA and NUMA machines, plus NUMA machines where the access is non-local and where the access is carefully localized.
Click to expand...

Well it's not quite the specs you want, but I have a dual opteron 248 w/ 4gb of RAM running XP x64 I could run the benchmarks on. I can run it UMA or NUMA.
 
Thanks, but I've got the low end machines covered. I'll use my dual Opteron 254 and AMD 4400+ machines, plus the QX6700 and one of my dual-proc machines at home.
 
mikeblas said:
It would involve just running a program and reporting its output to me. Thing is, it'll run for a couple of hours, total.

I'm writing a paper about NUMA, and want to compare the same algorithm on UMA and NUMA machines, plus NUMA machines where the access is non-local and where the access is carefully localized.
Click to expand...

You won't be able to reserach the effect of NUMA without carefully taking care of what exactly the OS kernel is doing with NUMA. That is extremely nontrivial and different OS kernels or kernel policies will give drastically different results for the same benchmark.
 
Of course. But since it's the Windows Kernel, and I know which version of Windows is running on the machine hosting the benchmark, I'm all set.
 
mikeblas said:
Of course. But since it's the Windows Kernel, and I know which version of Windows is running on the machine hosting the benchmark, I'm all set.
Click to expand...

Uh.

It is very unlikely that all Windows kernels implement the same policies in all regards that could influence NUMA performance, or that different Windows versions tune the Windows kernel the same way by default after installation.
 
Why do you think it's so unlikely, and what do you think differs? What work have you done that's demonstrated such differences?
 
mikeblas said:
Why do you think it's so unlikely, and what do you think differs? What work have you done that's demonstrated such differences?
Click to expand...

Reading about "page migration" in the Linux kernel would be a good start for you. It's a prime example about how nasty things get if you try to keep memory in whatever memory bank is associated with your process.

The problem of finding the right tradeoff between restricting the scheduler's freedom to move processes from one CPU to another on one hand and trying to keep processes in the CPU/memorybank combo whereever "their" memory pages live is obvious.

Naive people always think that restricting the scheduler is "obviously" right, but that breaks down for anybody who understand basic OS principles, in particular the problem of where the kernel is gonna live.

Throw in multithreaded applications and mixes of applications with different cache and memory demands and you have a mess at your hands, and you know the reason why people invest so much time researching solutions. Or why the Linux people seriously think that copy memory around to follow the processes can possibly be a good idea.
 
uOpt said:
Reading about "page migration" in the Linux kernel would be a good start for you. It's a prime example about how nasty things get if you try to keep memory in whatever memory bank is associated with your process.
Click to expand...
Do you believe that the Windows kernel implements page migration for NUMA support?

It's your assertion, then, that the Windows Kernel will self-patch in order to implement page migration?

I'm confused, becasue you've told me that I'm making a naieve assumption and that I need to read about page migration in response to my question about what you think is different between each class of Windows kernels and what you believe is altered at runtime. I don't think Windows XP or Windows Vista implements page migration. I think the only server SKU that might implement page migration is Windows 2003 Server Enterprise Edition, since it's the only version that supports memory hot-plugging, and would need page migration to implement that feature anyway.
 
I bet Windows doesn't go page migration right now.

But that doesn't change anything about the fact that there are a couple parameters, in particular with scheduling, but also in memory placement, where changing them around can have a huge impact on behavior in artificial benchmarks and some workloads. So it's still a bad idea to ignore the question which Windows you use and blindly assume they all behave the same.
 
I don't think my assumption is blind, though I might be missing something. I'm not ignoring which version of Windows is hosting the benchmark; it's part of the data I expect to analyse with the rest of the results.

Why do you think affinitizing threads to single processors and locking memory to the node local to the processor is not enough? What are the specific parameters that you're thinking of?
 
mikeblas said:
Why do you think affinitizing threads to single processors and locking memory to the node local to the processor is not enough? What are the specific parameters that you're thinking of?
Click to expand...

That "locking" never actually works in the real world, except for artificial benchmarking and maybe for a gaming machine that isn't supposed to do anything but run the game (but even if you do that, it would be much better to just kill all that Windoze background junk that disturbs the game).

That locking of CPUs and allocating only it that CPU's back is impractical nonsense for any kind of workstation load and for almost all server loads, unless they are true one-instance one-application servers with their own write cache control.

The scheduler needs the freedom to pick any CPU it likes, and the memory allocator needs the freedom to allocate more than 1/2 of the system memory to the processes on one CPU. And that doesn't even start to evaluate the problem of where to place pages from the filesystem buffer cache.

OS makers spend millions of dollars on finding a good tradeoff and they actually move away from hard associations.
 
uOpt said:
That "locking" never actually works in the real world,
Click to expand...
I think "never" is a bit of an overstatement. SQL Server, for example, is very successful in sitting on memory allocations after it has done what it can to assure they're localized to a processor (or node).

uOpt said:
That locking of CPUs and allocating only it that CPU's back is impractical nonsense for any kind of workstation load and for almost all server loads,
Click to expand...
Again, I think this is a bit hyperbolic. Certainly, SQL Server does its own caching and lots of its own scheduling. But under Windows, the paradigm can be applied successfully to other applications as well.

uOpt said:
OS makers spend millions of dollars on finding a good tradeoff and they actually move away from hard associations.
Click to expand...
If you read up on it, I think you'll find that Windows hasn't (yet). Slava Oks' blog might be a good place for you to start.
 
Mikeblas,

When I reinstall my system in 2-3 days, I'll have 8-10G of ram (depends if the x4 2G modules co-operate with the x8 512M modules) and a blank disk. I suppose I could grab a trial of windows server 64-bit, although I don't know if there are numa limitations in it.

Drop me a pm if you need a hand.
 
Thanks, RobStar. My test program isn't quite ready, so the timing is a bit off... I'll post back here when I've had a chance to work on it.
 
You must log in or register to reply here.
Back
Top