One thousand dollar webserver challenge!

S

servernewb

n00b
Joined
May 17, 2007
Messages
24
Hello Hard OCP duders. This is my first post in these here forums. I am a long time lurker, and seems like I always end up here when I need hardware assistance.

I propose a friendly challenge. I have One thousand dollars, and I wish to build an apache webserver that can handle 4TB of monthly internet traffic. The server will handle several users. The users will be free to do much of anything, including using PHP forums, wiki and photo gallery stuff, icecast media streaming, ruby on rails apps, and file serving.

I propose this challenge for two reasons. I have a similar dilemma in real life, except I must build several of these puppies. I also have some prototypes in mind, but I find myself in need of some professional critique and perhaps insights towards the project.

Here are the guidelines:

Must be a 1u
CPU must be 64bit
Must have Raid 1 or 10 (mirrored) disk storage. Storage capacity is not the biggest concern.

Given the requirements, and the monetary bind. I wonder where you folks would make sacrifices. Please leave back any feedback and insight regarding this amazing challenge!
 
4TB of "apache" traffic isn't specific enough. If it's straight HTML, any old box will do.
If you're serving complex PHP apps, that's another story.
Are we talking large files, video streaming, or just a ton of hits to get your 4TB figure?
 
What kind of internet connection do you have anyways? What kind of firewall do you have? Please don't tell me you're using some software firewall like norton on your webserver..
 
Vertigo Acid said:
4TB of "apache" traffic isn't specific enough. If it's straight HTML, any old box will do.
If you're serving complex PHP apps, that's another story.
Are we talking large files, video streaming, or just a ton of hits to get your 4TB figure?
Click to expand...
Exactly. I wouldn't be surprised if the majority of the bandwidth is due to pictures or what not. But who knows since the OP didn't specify much.
 
Vertigo Acid said:
4TB of "apache" traffic isn't specific enough. If it's straight HTML, any old box will do.
If you're serving complex PHP apps, that's another story.
Are we talking large files, video streaming, or just a ton of hits to get your 4TB figure?
Click to expand...

It wont be straight HTML, several PHP apps like forums and photo galleries. Perhaps even some sound/video streaming software. Large files too!
 
if it needs to be 1u, its unlikely you'll be able to build something.

buy a sun v120 for solaris, or a dell poweredge for linux.
 
You are rite. I did not clarify if the hw should be new or not. I would rather the hardware not be second hand.
 
I was thinking the same thing. also, why restrict it to 1U? dell has some good deals on their poweredge 2U servers, maybe look into those.
 
You can do RAID 1 in a 1U, but I have never seen any 1U servers which have more than 2 drives (which is necessary for RAID 0+1, 1+0, 5, and those other higher RAID levels.

I'd say save yourself the trouble and get a poweredge from Dell (however, I am incredibly biased since I deal with Poweredges at work and I really like them) but with your $1000 constraint, that could prove difficult. The base configuration that has a 64-bit CPU, 512 MB RAM, RAID 1 7.2K 80 GB SATA drives is about $1050, unless you have a discount or some other way to get a discount through Dell. I have not attempted to spec out building from scratch, but given the price for most server-grade parts (ECC or FB-DIMM RAM, server mobos, hard drives, RAID controller) you could end up with something more expensive than an OEM prebuilt.

What are your usage patterns like? You say you will probably have video streaming and dynamic webpages and such, but will the server have many connections simultaneously? How many users at the same time?

compslckr said:
I was thinking the same thing. also, why restrict it to 1U? dell has some good deals on their poweredge 2U servers, maybe look into those.
Click to expand...

Rackspace might cost more if he uses 2U, I assume, since he's collocating.
 
imzjustplayin said:
What kind of internet connection do you have anyways?.
Click to expand...

Maybe I slipped a decimal point, but 4TB/month = 1.5MB/second continuous - peak might be 4-6MB/sec or more. Something tells me that kind of transaction volume's gonna stress out a $1k machine, unless he gets one a minute for 90MB at a time.

On the web sites I run (two - one a wiki) a typical page is about 2KB with a big page being about 15KB (the very biggest, with graphics, is about 150KB)- that would be 100 transaction/sec on the 15KB pages to meet that demand (average, more on peak). That's an appreciable fraction of the volume of a national carrier's airline reservation system I used to work on (not a US carrier).
 
bashPenguin said:
You can do RAID 1 in a 1U, but I have never seen any 1U servers which have more than 2 drives (which is necessary for RAID 0+1, 1+0, 5, and those other higher RAID levels.

I'd say save yourself the trouble and get a poweredge from Dell (however, I am incredibly biased since I deal with Poweredges at work and I really like them) but with your $1000 constraint, that could prove difficult. The base configuration that has a 64-bit CPU, 512 MB RAM, RAID 1 7.2K 80 GB SATA drives is about $1050, unless you have a discount or some other way to get a discount through Dell. I have not attempted to spec out building from scratch, but given the price for most server-grade parts (ECC or FB-DIMM RAM, server mobos, hard drives, RAID controller) you could end up with something more expensive than an OEM prebuilt.

What are your usage patterns like? You say you will probably have video streaming and dynamic webpages and such, but will the server have many connections simultaneously? How many users at the same time?



Rackspace might cost more if he uses 2U, I assume, since he's collocating.
Click to expand...

There are some 1u cases that fit more drives. I have built them before. But you are right, 10 is unlikely. I just put it there in case someone found it more economical to use more drives at smaller capacities.

I know that buying from a manufacturer would cost more, so I did not even research buying from anyone (even dell, who do have some cheap deals!)

I dont have specific stats in mind, just trying to get some good ideas from you folks. I understand that having a more specific scenario in mind would help in determining the proper system. I will think about this and update the post. I am thinking a normal apache setup with several users, doing separate stuff. Some would have forums (LAMP), some might have icecast going. Dont know for sure.

Also, have to use 1u, because space costs big bucks!
 
Michael Daly said:
Maybe I slipped a decimal point, but 4TB/month = 1.5MB/second continuous - peak might be 4-6MB/sec or more. Something tells me that kind of transaction volume's gonna stress out a $1k machine, unless he gets one a minute for 90MB at a time.

On the web sites I run (two - one a wiki) a typical page is about 2KB with a big page being about 15KB (the very biggest, with graphics, is about 150KB)- that would be 100 transaction/sec on the 15KB pages to meet that demand (average, more on peak). That's an appreciable fraction of the volume of a national carrier's airline reservation system I used to work on (not a US carrier).
Click to expand...

Whats your wiki rig look like?
 
Michael Daly said:
Maybe I slipped a decimal point, but 4TB/month = 1.5MB/second continuous - peak might be 4-6MB/sec or more. Something tells me that kind of transaction volume's gonna stress out a $1k machine, unless he gets one a minute for 90MB at a time.
Click to expand...

4TB = ~13.2mbit, again depending on whether this is 95th percentile or GB/xfer billed. I do some colo/dedicated servers, and we have a customer right now that is doing 1tb/month of PHP web proxy on a 2.8ghz P4, 2gb of RAM, 80gb 7200 RPM SATA and the machine doesn't really break a sweat.
The focus of the build is really all dependent on what sort of power you need. I/O may not be a big deal for some things, but, for a forum, the database server portion is going to be really important, and MySQL will eat all the disk speed you give it.

So, this is what I would suggest:

1. LGA775 server-class motherboard with a quality on-board SATA RAID controller and dedicated NIC ($200)
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182120


2. Grab a pair of 74gb Raptors. Hopefully that's enough disk space ($300)
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136033


3. 2x or 4x 1gb sticks of DDR2 667, your choice ($90 or $180)
http://www.newegg.com/Product/Product.aspx?Item=N82E16820134332

4. E6420 ($190)
http://www.newegg.com/Product/Product.aspx?Item=N82E16819115016

If you go for the full 4gb, that leaves you $130-(shipping for above items) for a case
 
Hey, I think some people really underestimate hardware. Unless you're doing super complex, headless compiling type of stuff, 15Mbps sustained (about 4.5tb bandwidth) will do fine on a dual core machine with 2gb of ram and a strong harddrive setup. Raid is a waste, its much better to have an offsite backup solution (rsync is best imo) and leave internal backup/mirroring as an afterthought if budget allows.

I run a fairly large hosting company and we have a customer who has a PHP driven site that, at one point was using in excess of 20Mbps sustained on a 100Mbps port and his Celeron 2.4ghz/2gb ram server stood up respectably.
 
taylorwilsdon said:
Raid is a waste, its much better to have an offsite backup solution (rsync is best imo) and leave internal backup/mirroring as an afterthought if budget allows.
Click to expand...

RAID and backup are neither mutually exclusive nor replacements for eachother. Having RAID 1, 5, or any combination with redundancy, is about avoiding downtime. Backups still need to be performed, regardless of RAID.

If you go single drive instead of RAID in the above setup I propose, what would you spend the extra $150 on?
 
Vertigo Acid said:
4TB = ~13.2mbit, again depending on whether this is 95th percentile or GB/xfer billed. I do some colo/dedicated servers, and we have a customer right now that is doing 1tb/month of PHP web proxy on a 2.8ghz P4, 2gb of RAM, 80gb 7200 RPM SATA and the machine doesn't really break a sweat.
The focus of the build is really all dependent on what sort of power you need. I/O may not be a big deal for some things, but, for a forum, the database server portion is going to be really important, and MySQL will eat all the disk speed you give it.

So, this is what I would suggest:

1. LGA775 server-class motherboard with a quality on-board SATA RAID controller and dedicated NIC ($200)
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182120


2. Grab a pair of 74gb Raptors. Hopefully that's enough disk space ($300)
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136033


3. 2x or 4x 1gb sticks of DDR2 667, your choice ($90 or $180)
http://www.newegg.com/Product/Product.aspx?Item=N82E16820134332

4. E6420 ($190)
http://www.newegg.com/Product/Product.aspx?Item=N82E16819115016

If you go for the full 4gb, that leaves you $130-(shipping for above items) for a case
Click to expand...

I like where you are headed. I had been looking long and hard at AMD, but it seems like intel is outperforming them p/p. Is this why you picked them instead of AMD? I pondered perhaps using dual 939 AMD and the older tyan motherboards...

The 74GB raptors should be enough, but I was thinking about the 150GB versions, and leaving the memory to 2GB. The leftover cash should be enough to cover the 1u, pw and CPU cooling.
 
taylorwilsdon said:
Hey, I think some people really underestimate hardware. Unless you're doing super complex, headless compiling type of stuff, 15Mbps sustained (about 4.5tb bandwidth) will do fine on a dual core machine with 2gb of ram and a strong harddrive setup. Raid is a waste, its much better to have an offsite backup solution (rsync is best imo) and leave internal backup/mirroring as an afterthought if budget allows.

I run a fairly large hosting company and we have a customer who has a PHP driven site that, at one point was using in excess of 20Mbps sustained on a 100Mbps port and his Celeron 2.4ghz/2gb ram server stood up respectably.
Click to expand...

You should buy that celly a pint or pitcher of fat tire. Was that machine doing any SQL queries?

What is a "strong" hard drive setup? SCSI? Raptors?

Finally, I would be using raid to avoid downtime.
 
SCSI is not reasonable given your budget. Since Raptors are the fastest SATA drives on the market, you're not going to get better than them
 
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182067
Add $100 to your board price

Since you wanted new components, we can't grab any e-bay deals.
Any of the following drives will outperform a raptor:
Seagate 10K.7
Fujitsu MAT/MAW (the MAW is RoHS compliant version)
Maxtor Atlas 10K V
http://www.storagereview.com/php/be...&devID_1=264&devID_2=278&devID_3=275&devCnt=4


$159 for 10K.7 http://www.computergiants.com/items...2&zmam=6444850&zmas=1&zmac=18&zmap=ST373207LW

$165 for Atlas V
http://www.computergiants.com/items...ff=2&zmam=6444850&zmas=1&zmac=18&zmap=8J073J0

$120 for MAT
http://www.apricorn.com/product_detail.php?type=family&id=36

(there are some cheaper prices on pricewatch for refurb/recert drives, but those are to be avoided like the plague)

So, you end up spending $100 more on the mobo (even if you bought an add-in SCSI RAID card rather than buy a mobo w/ onboard, it still wouldn't be much cheaper, and definitely not faster), drives are similarly priced. Granted, under heavy I/O loads, those SCSI drives will wipe up the floor with the Raptor, but, we still don't know how important I/O is for your applications ;)
 
Vertigo Acid said:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813182067
Add $100 to your board price

Since you wanted new components, we can't grab any e-bay deals.
Any of the following drives will outperform a raptor:
Seagate 10K.7
Fujitsu MAT/MAW (the MAW is RoHS compliant version)
Maxtor Atlas 10K V
http://www.storagereview.com/php/be...&devID_1=264&devID_2=278&devID_3=275&devCnt=4


$159 for 10K.7 http://www.computergiants.com/items...2&zmam=6444850&zmas=1&zmac=18&zmap=ST373207LW

$165 for Atlas V
http://www.computergiants.com/items...ff=2&zmam=6444850&zmas=1&zmac=18&zmap=8J073J0

$120 for MAT
http://www.apricorn.com/product_detail.php?type=family&id=36

(there are some cheaper prices on pricewatch for refurb/recert drives, but those are to be avoided like the plague)

So, you end up spending $100 more on the mobo (even if you bought an add-in SCSI RAID card rather than buy a mobo w/ onboard, it still wouldn't be much cheaper, and definitely not faster), drives are similarly priced. Granted, under heavy I/O loads, those SCSI drives will wipe up the floor with the Raptor, but, we still don't know how important I/O is for your applications ;)
Click to expand...

What would you grab from ebay? SCSI disks?

It does look like any mobo with built in SCSI is just about 100 bucks more, which is the same price you would pay for a standalone SCSI controller. The raptors are probably the best bet to stay in budget.
 
seriously. if you buy a poweredge 1850 from ebay it will be more reliable then a 'new' homebuild.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
if you don't want a dell, get a dl360 :rolleyes:

most of the gear on ebay is very lightly used, often dr/standby kit. others have pointed out that scsi is higher mtfb than sata, by getting a dl360/1750/whatever allows you to pick up a spare powersupply, caddy etc so that if there is a problem you can quickly sort things out.

$1000 is probably over the odds for a 1750
 
sovs said:
if you don't want a dell, get a dl360 :rolleyes:

most of the gear on ebay is very lightly used, often dr/standby kit. others have pointed out that scsi is higher mtfb than sata, by getting a dl360/1750/whatever allows you to pick up a spare powersupply, caddy etc so that if there is a problem you can quickly sort things out.

$1000 is probably over the odds for a 1750
Click to expand...

I am not concerned over name brands, just performance within the budget. I fail to see how either of those ebay suggestions could outperform what Vertigo Acid proposed. Maybe I am missing something.
 
It used to make me sad when people made specific hardware recommendations without knowing anything about the application. Now, it just cracks me up.
 
mikeblas said:
It used to make me sad when people made specific hardware recommendations without knowing anything about the application. Now, it just cracks me up.
Click to expand...
only a bot would write something like that :eek:
 
mikeblas said:
It used to make me sad when people made specific hardware recommendations without knowing anything about the application. Now, it just cracks me up.
Click to expand...

:rolleyes:
We do know something about the application.

PHP forums, wiki and photo gallery stuff, icecast media streaming, ruby on rails apps, and file serving.
Click to expand...

It's a general purpose multi-user webserver. Sure, we don't know exact specifics about forum-this and wiki-that, but, neither does the OP. Since we're not really tuning for anything in specific (ie, it's not a server dedicated to just serving a single site that runs XYZ), what is wrong with the suggestions given thus far?
The 1850 linked is a fine deal. It's slower memory-bandwidth and disk subsystem wise, but has 4 logical processors vs 2, which may be an advantage depending how CPU intensive the web apps are vs disk I/O. Threading isn't a concern, as apache already has that taken care of in spades.

Sovs: The linked raptors are 1.2 Million hours MTBF. The 10K.7 1.4, MAT 1.2, Atlas V 1.4.
As for servers on e-bay being "lightly used" or "standby", I can tell you from personal experience that the majority of stuff out there, especially at anything one would consider a deal, are often beat to hell. It's not a big deal if rack-mount hardware is scratched up, but, expect get something different than pictured, with a small part or two missing here or there, if you buy a rack-mount server from a big seller on e-bay. Smaller, individual sales are much more likely to be in good condition, and for the seller to give a shit when you complain.
 
Vertigo Acid said:
It's a general purpose multi-user webserver. Sure, we don't know exact specifics about forum-this and wiki-that, but, neither does the OP. Since we're not really tuning for anything in specific (ie, it's not a server dedicated to just serving a single site that runs XYZ), what is wrong with the suggestions given thus far?
Click to expand...
What's wrong with them? They're blind guesses, not suggestions--that's what's wrong.

One very fundamental question is figuring out where money would be best spent. Are the apps CPU-bound, or I/O-bound? That is, would money be better spent on faster disk or faster processor? If they're I/O bound, would they benefit from faster STR, or faster seek? Are the I/O patterns more random, and should the money be spent on a subsystem with less random-access latency?

What's wrong with the guesses people have taken here is that they might lead to wasting money. If servernewb has money to spend on buying two or three systems to get it right, then fine. If he just wants to get a system up and running, then measure later, he's all set.

But if he's prudent and wise, he'll meausre twice and buy once.
 
I'd say the accuracy of any bandwidth or resource utilization forecasts would make the whole task of profiling a machine totally pointless.
 
mikeblas said:
What's wrong with them? They're blind guesses, not suggestions--that's what's wrong.

One very fundamental question is figuring out where money would be best spent. Are the apps CPU-bound, or I/O-bound? That is, would money be better spent on faster disk or faster processor? If they're I/O bound, would they benefit from faster STR, or faster seek? Are the I/O patterns more random, and should the money be spent on a subsystem with less random-access latency?

What's wrong with the guesses people have taken here is that they might lead to wasting money. If servernewb has money to spend on buying two or three systems to get it right, then fine. If he just wants to get a system up and running, then measure later, he's all set.

But if he's prudent and wise, he'll meausre twice and buy once.
Click to expand...

If this were an "upgrade my server with X specs to perform better at Y application", I would totally be with you. But profiling a server before you have it is a catch-22, and since the OP said this is going to be for user sites (shared hosting server?), it's not really going to be possible.

Let's examine our options:
1. CPU-bound app
2. STR bound
3. Seek-bound
4. RAM hog!

If #1 is the case, to significantly outperform the E6240, we have the following options:
a. E6700 ($319)
b. Q6600 ($545)
c. Go dual. $100 more for the lowest-end dual 771 board, 2x $245 5110 (~$600), and seriously more expensive FBDIMMs ($85/gig).

OP wants RAID 1, so, we still need a pair of drives. ~$80 for a pair of 80gb SATA, giving us $220 for the above upgrades. No Quad, No Dual. So the best we can do is the 6700, 2x533mhz faster.

#2
Since OP want RAID w/ redundancy, RAID-5 or 10/01 are the options for increasing transfer rate. Not only do we need another drive or 2 ($150-$300), but we need a 1u case that has more than 2 spots, which will fetch a higher price, more than the $130 allocated for case + PSU. Where are you going to pull $200 from in the build?

#3
If seek performance is the issue, we go 15K SCSI. $100 for either a mobo w/ onboard or a quality controller. Maybe $50 if soft-RAID is adequate? $200+ for each drive. Where does the $200 come from?

#4
Mobo speced supports 2gb DIMMs. Buy cheaper drives and throw 8gb in.

Building the right machine for the job is important, sure. But this is a general purpose webserver, so I have no qualms with suggesting a middle of the road build.
 
Vertigo Acid said:
If this were an "upgrade my server with X specs to perform better at Y application", I would totally be with you. But profiling a server before you have it is a catch-22,
Click to expand...
You've got it all wrong. We don't need to profile the server; we need to have a profile of the applications.

Vertigo Acid said:
and since the OP said this is going to be for user sites (shared hosting server?), it's not really going to be possible.

Let's examine our options:
1. CPU-bound app
2. STR bound
3. Seek-bound
4. RAM hog!
Click to expand...
So you do understand that there are wildly different performance aspects we could be interested in. Don't you? Of these four, which are we aiming at?
 
mikeblas said:
You've got it all wrong. We don't need to profile the server; we need to have a profile of the applications.

So you do understand that there are wildly different performance aspects we could be interested in. Don't you? Of these four, which are we aiming at?
Click to expand...

We don't know the specific apps, and neither does the OP!
We just know in general that they're "web apps"
Since this server is going to be multi-tasking, consider that one app may benefit more from faster drives, another from CPU. But budget is not unlimited. That's why I'm proposing a swiss-army knife build. It's not the best in any category, but, it will get the job done.
 
You must log in or register to reply here.
Back
Top