Disk performance in the cloud (Amazon/AWS)

G

Gagnon

Weaksauce
Joined
Oct 28, 2009
Messages
93
If we have a disk heavy application using SQL, can we get respectable disk performance on reads and writes in the cloud or are we much better off (like order of magnitudes better) rolling our own SSD RAID?

We're capturing a lot of tick data (small size, many writes, frequent writes) for various financial instruments of which we build candlestick bars and perform decision logic off of these bars.
 
Amazon/AWS provides their IOPs as well as offering a higher IOPs instance.

However, last I checked neither were near SSD performance.

DigitalOcean or Ramnode may work for you though.

Are you using MySQL? If so there are forks that perform even better...
 
the amount you will spend to get IOPs from amazon is going to be obscene.

you would be better off building something yourself. as an example a 2U server with 24 2.5" drive bays and some enterprise write heavy SATA SSDs connected via 3 x LSI 9207-8I cards could scream.

I would recommend a ZFS based OS and this intel server because you can get that server without a backplane allowing you to run a dedicated port to each of the 24 SSDs.

build 2 of those and use ZFS replication to keep them in sync. Could do the whole project for under 50K probably.
 
madrebel said:
the amount you will spend to get IOPs from amazon is going to be obscene.

you would be better off building something yourself. as an example a 2U server with 24 2.5" drive bays and some enterprise write heavy SATA SSDs connected via 3 x LSI 9207-8I cards could scream.

I would recommend a ZFS based OS and this intel server because you can get that server without a backplane allowing you to run a dedicated port to each of the 24 SSDs.

build 2 of those and use ZFS replication to keep them in sync. Could do the whole project for under 50K probably.
Click to expand...


$50k is a bit out of our price range, but interesting post nonetheless.

I priced out the AWS High I/O reserved instance using light utilization (trying to figure out what the differences are in utilization levels) which uses SSD and it came out to $12k per year, I realize there are other fees on top of that but for running it 24x5 I thought the price was reasonable.
 
Yes, but your comparing a normal machine with 2 ssd's in it, cost? somewhere in the 4-8k range, depending on how cpu/memory crazy you want to get.

for 50k, we are looking at a min of 24 ssd's.
 
How many IOPs will you require?

If you just want higher performance than RAID10 15K RPM SAS drives then even 1 SSD could do that...

You don't need a 5k server or a 50k server with SSD.

You need to share a bit more info about your load or if you just want to bump current performance a bit by going to SSD.
 
My disk performance is pretty awful on EC2..

Depending on the size of your application, you may want to consider doing your heaving IO on ramdisks if possible (if it's not too large) and then writing out the result to platters once it's ready to be archived..
 
You must log in or register to reply here.
Back
Top