Advice - What do I need to do/build to keep my data safe?

C

Chellexelle

Weaksauce
Joined
Sep 8, 2014
Messages
108
This is a question for the experts. I have slowly been growing my collection of server equipment and I want to put together a plan of what I need to consolidate and protect my data. So far I have one file server that I have been using for my music and I have acquired 3 server racks that had 2 1U servers and misc equipment in them. I don't know what the 1U servers do.

I know I need a redundant backup power and a safe place to put my servers. I was thinking about building a climate controlled fireproof, EMP resistant server room out of a 10 or 20 foot shipping container encased in concrete.

Do you have any advice on where I can get servers and equipment especially UPS's cheap? I have been trying my state governments auction but most of the stuff is outdated and servers are usually missing the drive trays.

Building my one server cost me over $4000 and I can't afford to spend that much on my other servers which is why I need to know where the thrifty server enthusiasts go for the best deals.

I need to know where I can get rack mountable UPS's cheap and I need to know what kinds of networking equipment I need for a fast home network.

Basically, I want to have what you have.
 
That must be quite the music collection... :eek:

I'm no expert on servers, but you'll probably wanna share at least the broad strokes of your setup if you want actual UPS recommendations. Hard to estimate your needs otherwise...
 
You should use ZFS if you are really concerned about data integrity. And use a backup offsite.
 
My current server just sits on a shelf and is connected to my PC through my router. I am already using ZFS for my server.
 
zfs on BSD of course, where it should be....

and if it is that important, something offline, and offsite, or at least in a fire /flood proof safe and multiple copies.
 
To suggest a good ups we would need to know what you have to know minimum requirements to run the server as far as power consumption. Spending 4k on a system. Really wondering where did you put the money. Cpu? Ram? Hdd?
 
I have one 4U server and I want to build 5 more. One for videos. One for my porn. One for misc files and 2 backup servers using the Seagate 8TB archive drives.

I need a UPS for these servers, and would like to know how\where I can buy used servers and equipment in good condition that is not too old.

I would like to put together a rack like those in the storage showoff thread.
 
Chellexelle said:
I have one 4U server and I want to build 5 more. One for videos. One for my porn. One for misc files and 2 backup servers using the Seagate 8TB archive drives.

I need a UPS for these servers, and would like to know how\where I can buy used servers and equipment in good condition that is not too old.

I would like to put together a rack like those in the storage showoff thread.
Click to expand...

an honest guy :D

If you want get a MB and CPU cheap with some parts check out the deals thread. There is an IBM tower server for $285. You can use that for parts.

Ebay has lots of older servers that are plenty fast enough for mere backups. May cost a little more electricity but some decent ones can be found like westermere and such. first gen i7s.

I still think the IBM server is a good start for getting core components.

What some people do is those chaisy chain builds. They build a single massive server and just connect dumby boxes with HDDs to the main server and make virtual drives.

That would allow you to cut out some RAM/MB/CPU costs and save energy too. That could save a couple grand in initial costs.


Your best bet is to just make a massive array at first and then break it down when current gen becomes old gen and you have some extra cash to make separate servers.

On a side note holy shit...how much data do you need to store? -_-
 
Last edited:
Well I can tell you right now storing all your eggs in one basket is a recipe for disaster. All it takes is one electrical fire or short and its all gone.

At the very least you will need a halon fire system to go with that since it's relatively inaccessible.

Ideally you would have several sites like that to store data on identical machines. At least 3-4 sites should be sufficient.

If you are looking for longterm storage of archives. Best bet is to vacuum seal drives and put them in trash cans full of water underground.
 
staticlag said:
Well I can tell you right now storing all your eggs in one basket is a recipe for disaster. All it takes is one electrical fire or short and its all gone.

At the very least you will need a halon fire system to go with that since it's relatively inaccessible.

Ideally you would have several sites like that to store data on identical machines. At least 3-4 sites should be sufficient.

If you are looking for longterm storage of archives. Best bet is to vacuum seal drives and put them in trash cans full of water underground.
Click to expand...

the biggest and immediate threat is not properly storing it in a digital sense so that's priority one. Making it "nuke" proof is his least concern.

If he has that much data best bet is getting a single solid server with redundant power and then worry about some Faraday case. Again the biggest threat is bad data storage set up and that has to be built first. It is more likely that causing problems then someone blowing a nuke up in the sky. It'll be a little longer til Iran has that ability...a few years? :D:rolleyes:

He has enough time to get a solid system before needing the cage.
 
SomeGuy133 said:
an honest guy :D

What some people do is those chaisy chain builds. They build a single massive server and just connect dumby boxes with HDDs to the main server and make virtual drives.

That would allow you to cut out some RAM/MB/CPU costs and save energy too. That could save a couple grand in initial costs.
Click to expand...

HOW HAVE I NOT HEARD ABOUT THIS! Please tell me more. I thought I was going to have to build complete independent servers.

SomeGuy133 said:
On a side note holy shit...how much data do you need to store? -_-
Click to expand...

About 75TB currently while accounting for an additional 25TB/year of new data.

staticlag said:
Well I can tell you right now storing all your eggs in one basket is a recipe for disaster. All it takes is one electrical fire or short and its all gone.

At the very least you will need a halon fire system to go with that since it's relatively inaccessible.

Ideally you would have several sites like that to store data on identical machines. At least 3-4 sites should be sufficient.

If you are looking for longterm storage of archives. Best bet is to vacuum seal drives and put them in trash cans full of water underground.
Click to expand...

That was already the idea. I am going to build a climate controlled fireproof, EMP resistant server room out of a 10 or 20 foot shipping container encased in concrete and although I didn't mention it in my first post, I was planning on doing a halon fire suppression system.
 
Earthquake proof? flood? meteor strike?

It is rather expensive to attempt to protect against all things at one location, it is hundreds of times cheaper to build in two or three locations.
 
You can buy a used 40' shipping container for $2,000. I know someone who does concrete. I don't think it will be as expensive as you think.
 
I mean like what happens if your psu fries it's regulator and takes 40 drives with it? It does happen. You need completely separate systems, raid is not a backup.

If you want high capacity, there are backblaze pods. The most expensive item in any server are the hard drives. 40x$150 = $6000 your looking at easily $10-20k in drives alone. Consider they will fail every 5-10 years and it gets expensive. This stuff isn't buy once and forget. Even psu's and ups's need replacing every few years, specifically they need new capacitors as the liquid dries up inside them
 
Chellexelle said:
You can buy a used 40' shipping container for $2,000. I know someone who does concrete. I don't think it will be as expensive as you think.
Click to expand...

I don't think the panels in standard shipping containers will support concrete. The frame is strong, but the panels, not so much. There was an episode of Mythbusters where they tried to bury such a container by covering it with dirt. Not all that much dirt, IIRC. And the roof of the container failed.
 
JoeComp said:
I don't think the panels in standard shipping containers will support concrete. The frame is strong, but the panels, not so much. There was an episode of Mythbusters where they tried to bury such a container by covering it with dirt. Not all that much dirt, IIRC. And the roof of the container failed.
Click to expand...

dirt doesn't support itself. Concrete can...notice can. Depends on how you use it. If the structure if engineered correctly it would be fine but i bet internal braces would be a safe bet just like any house has pillars.

I am building my first NAS still so i am not the person to consult but it has to do with just using some dummy motherboard to power an exander card in a case.

so you have full blown NAS lets say a Norco 4224 with MB/CPU ect. You have a RAID/HBA with external ports or an adapter that makes internals into externals. Take that external cable to another Norco 4224 case which has a dumby motherboard or something with another expander card. One cable from main server goes to second case connecting to the expander so the two systems are linked. The second norco case HDDs connect to that expander which was connecting to the original server.

That is one way of doing it from what i have gathered but again...i am totally fresh to this stuff. Looking at how long I have been on this forum shows i have only started learning this stuff over the last 5 months :) So please talk to one of those guys with the crazy 250TB setups :)

As i said and another person said worry about getting first system build and then get an off site storage set up then worry about the apocalypse box. Getting a working system is priority one. Personally I would focus on the faraday/fire proofing and not bother with the crush proof part. That is the least likely and if you got a secondary site i wouldn't worry about it. (the crush proof).
 
Chellexelle said:
HOW HAVE I NOT HEARD ABOUT THIS! Please tell me more. I thought I was going to have to build complete independent servers.
Click to expand...
You want something called DAS chassis. Or JBOD chassis. Like this one 45 disk chassis:
http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm
There are 90 disk chassis models too, if you look at the Supermicro website.

You need a SAS HBA card (like a dumb RAID card with no RAID functionality) in your server, typically an LSI2008 (or the new LSI3008) based card. Then you connect your SAS card to the disk chassis with a cable called SFF8088 for the LSI2008 card. When you boot your disk chassis, the server believes that all the 45 disks are plugged right into the server motherboard. You can even install OSes and boot from the disk chassis.

The LSI2008 card (typically an IBM 1015 card can be found on Ebay for $100 which is excellent for ZFS usage) has two cable connections called SFF8088. So you plug both cables to the same disk chassis and get 4.8 GB/sec. Or you connect one cable to one disk chassis, and the other cable to the other disk chassis and then you get 2.4GB/sec each. If you need to accesses more disk chassis, you need to insert several SAS HBA cards. Each SAS card accesses up to two disk chassis.

The disk chassis typically contains an SAS expander when it contains more than 24 disks. If you dont have an SAS expander, the disk chassis can only contain 8 disks or so. Not sure on these numbers, but the point is that large disk chassis must have an SAS expander. So if you build one yourself, you need to add an SAS expander.
 
brutalizer said:
You want something called DAS chassis. Or JBOD chassis. Like this one 45 disk chassis:
http://www.supermicro.com/products/chassis/4U/847/SC847E26-RJBOD1.cfm
There are 90 disk chassis models too, if you look at the Supermicro website.

You need a SAS HBA card (like a dumb RAID card with no RAID functionality) in your server, typically an LSI2008 (or the new LSI3008) based card. Then you connect your SAS card to the disk chassis with a cable called SFF8088 for the LSI2008 card. When you boot your disk chassis, the server believes that all the 45 disks are plugged right into the server motherboard. You can even install OSes and boot from the disk chassis.

The LSI2008 card (typically an IBM 1015 card can be found on Ebay for $100 which is excellent for ZFS usage) has two cable connections called SFF8088. So you plug both cables to the same disk chassis and get 4.8 GB/sec. Or you connect one cable to one disk chassis, and the other cable to the other disk chassis and then you get 2.4GB/sec each. If you need to accesses more disk chassis, you need to insert several SAS HBA cards. Each SAS card accesses up to two disk chassis.

The disk chassis typically contains an SAS expander when it contains more than 24 disks. If you dont have an SAS expander, the disk chassis can only contain 8 disks or so. Not sure on these numbers, but the point is that large disk chassis must have an SAS expander. So if you build one yourself, you need to add an SAS expander.
Click to expand...

if you want to take the time the HP SAS exander is well liked for the 32 connection vs the intels 24 but you have to upgrade the BIOS, which i personally find too much work but you might be able to do it since you'll need several. I only need one expander personally.

Also in the main machine you might rather go with a 16 port card and not an 8 port card so a 9211 16i might be best for doing such a large build.

also those DAS boxes are a better way to go then my recommendation but it may cost more for all i know but it is done right.
 
Chellexelle said:
I am going to build a climate controlled fireproof, EMP resistant server room out of a 10 or 20 foot shipping container encased in concrete and although I didn't mention it in my first post, I was planning on doing a halon fire suppression system.
Click to expand...

Top tip:
Make sure your data lines in and out are fibre. If you want to go ultraparanoid, you do not want lightning strike to go in your container via the cheap cat6 cable...
 
rsq said:
Top tip:
Make sure your data lines in and out are fibre. If you want to go ultraparanoid, you do not want lightning strike to go in your container via the cheap cat6 cable...
Click to expand...

Lightning has enough power to travel through the AIR. It certainly has enough power to travel along the materials that make up a fiber cable.

The way to protect against lightning is to give it a low resistance path to ground. In other words, a good lightning rod with a thick, highly conductive cable connected to a good earth ground.
 
JoeComp, fibre in the ground is safe because the resistivity of the eart is lower then that of the fibre.

https://en.wikipedia.org/wiki/Elect...stivity_and_conductivity_of_various_materials

The cladding in fiber cable certainly has a higher resistivity then air and much higher then copper.

If you are employing your shipping container as a faraday cage, you also do not want a copper line going in because it is like a big antenna picking up radiated emissions.

Unfortunately it is unavoidable for the power lines. But you can protect this.

The proper way would be to ground the whole container and feed in the cables from below.Cables in 4 inch HDPE conduit, with a separating transformer and a varistor bank on the primary of said transformer. Some sparkgaps would also be nice. A low pass filter may also be a good idea against the radiated high frequencies.

I suspect it will become cost prohibitive at this point :) Its certainly more expensive then some offsite backup and replication. But still... Cool idea.
 
rsq said:
JoeComp, fibre in the ground is safe because the resistivity of the eart is lower then that of the fibre.

https://en.wikipedia.org/wiki/Elect...stivity_and_conductivity_of_various_materials

The cladding in fiber cable certainly has a higher resistivity then air and much higher then copper.
Click to expand...

If the cable is completely buried, it does not matter what the cable is made of, since the cable does not provide a path between air and earth ground for the lightning to travel along. So your comment about fiber in the ground is irrelevant. If lightning is an issue at all, then it must mean that the container and cables are NOT completely buried.

Fiber cable does NOT have a higher resistivity than air. You seem to be forgetting that a fiber cable is not just silica. By volume, it is mostly various polymers (plastic or rubber cover, kevlar or other aramid fiber reinforcement).

Again, rather than worrying about using fiber cables versus CAT6 or whatever, the real way to protect against lightning is to have a good lightning rod, connected to a good earth ground with thick, high conductance cable.
 
This is all really good advice people. Thanks. my current strategy is storing most of my valuable data on a ZFS file server (as I have already stated through out this post) while storing the rest on external USB drives with RAID 1 in a fireproof file cabinet.

Now I just need to know what hardware I need and how to get it inexpensively as possible so I can build stacks like in the storage showoff thread. I have three server racks I want to fill.
 
JoeComp said:
Again, rather than worrying about using fiber cables versus CAT6 or whatever, the real way to protect against lightning is to have a good lightning rod, connected to a good earth ground with thick, high conductance cable.
Click to expand...
I had a colleague at a large finance company, and she had a lightning strike somewhere in her neighbor hood sometime in 1995 in a rich western european country. And all the electronics in the hood got thrashed. She lost a TV and a stereo. Maybe the lightning struck a power station, I dont know. But the point is, I dont believe anything can protect against lightning. Sometimes I see power surge protections up to 4000 joule. Well, lightning has millions of joule. If lightning strikes badly, all electronics in the village will go out.

Therefore, I am paranoid against this. So my solution is to have an SSD as an OS disk, and another large HDD for storing data. Whenever that HDD is full, I connect a DAS holding a large raid with ZFS and I transfer data during a couple of hours. Then I immediately disconnect all the cables to the DAS chassis. And if lightning strikes and my PC is toast, it does not matter because my backup is my large ZFS raid in the JBOD DAS. I only have working data on my HDD.
 
You must log in or register to reply here.
Back
Top