Why isn't having many small hard drives for backup more popular?

Discussion in 'SSDs & Data Storage' started by carrierPigeon, May 20, 2018.

  1. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    I am talking here about home users or very small businesses & HDDS under 500GBs.

    The benefit that I see in having many small hard drives for backup is 2 fold:
    1- you protect against malicious code by having many of the drives powered off at any given time.
    2- you protect against hard drive failure because you have your risk more spread out.

    So, why aren't small drives more popular?

    Is there something about larger drives that I am missing? For example, maybe a lot of the ones that were manufactured 5 years ago are in the 2nd half of their life, and there is no way to justify manufacturing more of them. Are there efficient ways to protect against malicious code on a large drive, such as having part of a drive on a file system that your current operating system can't write to?

    I understand that small drives can take up a lot of space, and also are fairly heavy. But, this shouldn't be too much of a concern for these small time users.
     
  2. Luke M

    Luke M Limp Gawd

    Messages:
    350
    Joined:
    Apr 20, 2016
    Small USB flash drives are very popular. If you want some format diversity, there's also BD-R. There's no need for tiny hard drives.
     
  3. If you're talking about rotating backup devices?
    Yeah, a lot of businesses do this.
    The main problem with "smaller hard drives", even for small businesses? The data load can grow QUITE quickly.
     
  4. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,901
    Joined:
    Jul 24, 2008
    From a business standpoint it is entirely convenience, employee time sitting there watching and switching the drives (if they are recent 500GB drives it could be as little as 5 minutes per drive) and external backup storage costs. I can send out a single 12TB native (Which depending on your datasets could actually yield 18TB(Forget 30)) LTO8 or HDD with the same capacity of 24 500GB drives. If your motivation is to spread out offline backups across multiple physical media, that is what tape backup is for. It has bit error rates that hard drives can't touch (Average consumer HDD have 1 URE in 10^14, Enterprise in 10^15 and LTO 7+ 1 in 10^19. As for why they don't make smaller drives anymore, for the general public there is no reason. With bit density where it is, the cost on lower end drives to add a single extra head and coat the other side of the platter is negligible. Why wouldn't they double the size to 1TB or if they want to sell 500GB drives take a 1TB with errors on one side of the platter and lock out that side and sell a flawed 1TB drive as a working 500GB drive.
     
    drescherjm likes this.
  5. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    Tape drives- good point. I'd imagine that those disks are smaller than 3.5 inch drives & I didn't know that data accuracy is better on those (but I think I do remember reading something like that). I was reading a few days ago about I think it was a 350 tb tape (obviously that's very expensive but it's amazing the capacity of the high end stuff). Another unintended consequence of tape drives is that when you are in the process of backing up data, malware wouldn't be able to encrypt much of your data as it just wouldn't have time.

    Blue ray drives have become pretty cheap I have noticed. I guess the issue that I have with them is creating a disc image -- even the 100GB ones are too small for a lot of systems. Maybe you just need to have your working drive partitioned so that you can create a disc image of the part that has everything installed (or install an extra hard drive and put the files on that).

    Flash drives have the benefit of being multi-purpose (you can carry them around, compact, long term storage of important specific files, etc).
     
  6. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    For data archival, I have an LTO-3 tape drive. It was inexpensive and pretty easy to setup for what I needed without spending a ton. 400gb tapes that are approx the size of an old 10 pack of 3.5" floppies. This is economical for a few terabytes of archived data give or take.
     
  7. Ranulfo

    Ranulfo [H]ard|Gawd

    Messages:
    1,382
    Joined:
    Feb 9, 2006
    Whats a good place to shop for LTO drives?
     
  8. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    I have a Dell Powervault 110T which I got off Ebay for about $100 with a cable. There is tons of them on there for that price range. All this stuff is too old for businesses, so it's fairly inexpensive.. I use https://www.z-dbackup.com/ You need the paid version of the base program and the paid version of the Z-dat dump program so that you can write across multiple tapes. It's the cheapest software you'll find and it works well for home backup.
     
    Ranulfo and GoldenTiger like this.
  9. kdh

    kdh Gawd

    Messages:
    700
    Joined:
    Mar 16, 2005
    @OP, Your logic is completely correct. But keep in mind, what works for one site, may not work for another. Every situation you walk into is different based on the requirements. If you have physical foot print space, then you have more flexibility. Some sites, you barely have a closet much less a rack you have to make work.
     
  10. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    Do you hook SCSI up to USB (is that the cable that you are referring to)?

    I think it's not economical until you get to really large sizes.
    For example, 500GB HDDs are worth about $25
    Your drive is $100 + $8 per 400GB
    Also, there is the cost of the software that you mentioned.

    But, it saves space and is more reliable.
     
  11. WhoBeDaPlaya

    WhoBeDaPlaya 2[H]4U

    Messages:
    2,485
    Joined:
    Dec 16, 2002
    I've been using USB3 portable HDDs as backups for ages.
    Great $/GB, no random access penalty unlike tape.

    Just make sure to keep it in a safe area, and spin it up every once in awhile.
     
  12. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    The cable is the SCSI cable. I also had to add an old Adeptec PCIE SCSI controller, and a terminator but that's cheap stuff.. It's probably not the best solution, but I make it work
    I have hard drives backed up with most of the data also.
     
  13. Nenu

    Nenu [H]ardened

    Messages:
    18,601
    Joined:
    Apr 28, 2007
    I rotate large drives for backups.
    No need for small (although it would help to define "small").

    This party has been swinging long time baby.
     
  14. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    Let me get this straight as I am a noob.
    Correct me if I am wrong about any of this
    You have this on a desktop operating system (windows 7,10, or a linux distro). By desktop operating system, I mean not a server or enterprise operating system. You installed the scsi card into your pci-e slot, which creates an input plug in the back of your computer, which you then plug the external scsi drive into.
    What's a terminator? I'm not sure what words to search to read about that.
    How are you powering that scsi drive?
    In my computer, I am already using the PCIe x16 3.0 slot for my video card.
    According to the specs that I am reading, there are still come PCIe 2.0 and a PCIe half length mini.
     
  15. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    Another drawback of hard drives is geomagnetic storms and EMPs. In that department, this guy says optical media > flash drives > HDDs. He makes no mention of tape drives.
    http://briantomasik.com/backing-data-geomagnetic-storms-emps/

    This thread is starting to remind me of the 24" CRT thread, with the common theme being that in some ways x years later still nothing trumps the old fashioned way. Only in this case, the old fashioned way might be paper. The article I linked to said something like some paper has lasted over 1000 years.
     
    Last edited: May 22, 2018
  16. Nenu

    Nenu [H]ardened

    Messages:
    18,601
    Joined:
    Apr 28, 2007
    Not that common round here, got any stats?
     
  17. westrock2000

    westrock2000 [H]ardForum Junkie

    Messages:
    8,939
    Joined:
    Jun 3, 2005
    Something to store them in
    Something to power them all with
    Something to plug them all in to
    Waiting for them to all spin up

    I had 10 drives for 20TB. It made a lot of heat and took a while to spin up when not accessed for a while. Plus I had drives all over the place, trying to keep track of them. Replaced them with 3 8TB drives and now use the smaller drives to backup what’s on the big array.
     
  18. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    Correct. It's a Windows 7 workstation ( Dell Precision T3500 ), so I have many PCIE slots, but the card is only X1, and most computers have a PCIE X1 slot. The card I use is a common cheap card. Adaptec 29320LPE

    The terminator is used with SCSI to terminate the chain of devices, without it, it does not work simply put. Looks like the end of a SCSI cable with an LED usually.. The drive being external, just powers with a standard power supply cable.
     
    carrierPigeon likes this.
  19. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    There is not a lot of information about your approach online. I was looking into one tape drive that I found and the manufacturer didn't have Windows 10 drivers (not to say that it wouldn't work).

    Does there have to be special support/hardware on your motherboard for this to work? Like SAS support, for example. Does it depend on what specific LTO 3 drive I go with?

    Should you be firm on your decision of what operating system you are going to be using? Can I share the drive on my network (from a desktop computer, not a server) and write to it from other computers?
     
  20. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    I think it's tough to have "derived" probabilities of various things happening. Unless of course it's a probability problem that's very straightforward (like rolling dice). Probably a lot of probabilities that people have in their heads are way off. I'd imagine that there are some studies out there but I haven't read up on things.
     
  21. TeleFragger

    TeleFragger Gawd

    Messages:
    787
    Joined:
    Nov 10, 2005
    Well I am a home user with family photos, videos, personal files, etc...
    I run a Windows Server 2016 with Essentials role installed
    then I have
    5x2tb wd black drives

    I use Stablebit DrivePool and love it... I have family pics/videos set to 5 copies and other things set to 1 or 2 copies. I get that this wont be the best for performance but I can lose any drive and have copies on others. Even if the system goes down, there is a hidden folder that you navigate too and pull your files off of it.

    for me has worked and is the best $20 I have spent. They even have a DrivePool for cloud spaces like one drive, amazon, etc.. gather in all of your storages and "Pool" them...

    I also was using an HDHomerun Prime card and DVR'ing live tv that this storage pool keeps the videos...then a separate plex server hits it as remote storage... and I do not see any performance issues at all... but I dropped cable tv and no longer can dvr... hah... well could use my HDHomerun Quatro OTA card... but DirectTV Now has changed how we watch tv..
     
  22. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    I don't think Windows 10 natively supports these tape drives. I use Windows 7 and the drivers for the controller card and the drive were picked up automatically.. So I would say Win 7 is best

    The controller card should work with any PCI-E slot in any motherboard. The Powervault 110T that I use, is actually an IBM drive branded for Dell, and it will come up as an IBM in the device manager.. And you can't share the drives over network, they don't work like that. They write the data through the special software.. Additionally, unless you have Gigabit networking, the network is not fast enough to supply the drive with data and will 'shine' the tape which is caused by stop/ start conditions..

    If you are considering an LTO-3 drive, look at Dell or IBM drives as they have the most compatibility. The external Dell Powervault 110T's ( IBM hardware ) are cheap and work good as any. It's not hard to get it working
     
  23. Nenu

    Nenu [H]ardened

    Messages:
    18,601
    Joined:
    Apr 28, 2007
    Probably ;) not something most people need worry about.
     
  24. Dan_D

    Dan_D [H]ardOCP Motherboard Editor

    Messages:
    53,215
    Joined:
    Feb 9, 2002
    There are a lot of reasons why this model isn't used by anyone.

    This is what is referred to as an "offline" backup. This is actually the only real way to protect your data from malicious code when its not in use. All other methods pose some level of risk. Even offline backups are still at risk so long as they are located in the same building / location as the source of this data.

    I see what you are saying here, but this is false. Increasing the amount of mechanical and flash based devices increases the chances that one or more drives can suffer a failure. I own 20 or so physical drives that I have in use at home. I see one or two drive failures a year. Back when I had a single machine with one or two drives, I would go two or three years in between drive failures. Perhaps longer. By the same token, if you work in a large company with a massive IT infrastructure hard drive failures can be a weekly or even a daily occurrence. The more devices you have, the more rolls of the dice and thus, a higher chance of failure. The risk is only mitigated in that you ensure that you won't lose all of that data, but you increase the chances of losing some of it. Different strategies can offset this, but it depends on how much data we are talking about and the frequency it gets backed up and how many copies of that data which exist at any given time.

    Small drives aren't popular because people don't want to buy a bunch of drives and integrate them into their solutions. Not only can the cost per GB be higher this way, but the power consumption and physical drive prices may exceed a larger one. This of course varies somewhat by the size and price points of the drives we would be comparing in this model. It's also a pain in the ass to keep purchasing drives and as you said, having more drives means taking up more space. In a sense this model is still in use today, but mostly in the form of tape backup and not hard disks.

    The drive size is actually irrelevant when it comes to protecting data. Any data that's online and accessible is subject to the same vulnerabilities. All the ways you would guard a large drive apply to smaller drives. The only way to make the small drive model work is to have them used as removable media. Frankly, there are cheaper and easier ways to accomplish the same thing without buying a ton of hard drives and handling them all the time. Moving and handling drive adds risk to the equation as well. Drop a drive and you could damage it beyond any type of cost effective recovery.

    Smaller businesses are adopting cloud type strategies where they essentially use an "infrastructure as a service" type provider. Basically they rent virtual infrastructure and backups are part of the service contract. This is cheaper than incurring the costs associated with having your own infrastructure and backup strategies. Individuals who use cloud storage are basically doing the same thing. You also push certain security concerns off on someone else. This doesn't eliminate those concerns, but you end up leaving it in the hands of professionals instead of incurring all the risks yourself.
     
  25. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    Thanks for all of the great info. If there is a good chance that Windows 10 is not going to work for a lot of these drives, then I prefer to go with Linux (but not a server version). However, my Windows machine will still be my main machine for a good while, at least. So, I would probably do something clunky like first create a disk image, or move individual files/folders to the Linux machine and then write them to tape on the Linux machine. At least I am a little bit familiar with some of the Linux command lines. This could come in handy if I use the LTFS file system as Windows and OSx will take 10's of minutes to do stuff vs a much faster command line. A score for the command line. How often do you see that :) I was reading about it here
    https://forums.creativecow.net/docs/forums/post.php?forumid=330&postid=348&univpostid=348&pview=t

    In the discussion that I linked to, one of the posters says that for LTFS, HP drives have some sort of native support for Unix based operating systems. So, maybe that's the direction I should go brand wise. I am not sure, though, if I will do the LTFS or use software in the long run (meaning that HP might not be the right choice). I imagine that software will be much more attractive long term (once you spend the money and familiarize yourself with it). Also, their discussion is on LTO5, which might be different than LTO3. I am even open to the idea of using an LTO2.

    One thing I do wonder with these drives is what year the discs will get manufactured until. The technology is already something like 12 years old so you have to wonder a bit.
     
    Last edited: May 24, 2018
  26. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    You can always do very seldom backups (like once every 5 or 10 years) of key files, if say you are on a seldom trip to some relatives that have the needed hardware. But yea, probably doesn't play into almost any small time users' decision
     
  27. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    My OP was not very clearly written. In hindsight, my real question was why don't families/ tiny businesses use many small drives where a small drive is defined as something big enough to hold 20% - 75% of your data. This way you are more protected against hard drive failures and malicious code.

    But, you are right. It's more cumbersome and expensive. I think the point about moving hard drives around a lot is a good one (contributes to wear and tear).
     
  28. matt167

    matt167 Gawd

    Messages:
    707
    Joined:
    Dec 18, 2016
    LTO tapes are all backwards compatible, so just upgrade the hardware if the media becomes obsolete.. LTFS came in LTO-4 and later and that's part of the reason that 3 is pretty cheap
     
    carrierPigeon likes this.
  29. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,901
    Joined:
    Jul 24, 2008
    This is true to a very limited point. Here is the relevant text from Wikipedia so I don’t have to paraphrase:
    • Up to and including LTO-7, an Ultrium drive can read data from a cartridge in its own generation and the two prior generations. LTO-8 drives can read LTO-7 and LTO-8 tape, but not LTO-6 tape
    • An Ultrium drive can write data to a cartridge in its own generation and to a cartridge from the one prior generation in the prior generation's format. Additionally, some LTO-8 drives may write previously unused LTO-7 tapes with an increased, uncompressed capacity of 9 TB
    • An Ultrium drive cannot make any use of a cartridge from a more recent generation.
     
    drescherjm likes this.
  30. carrierPigeon

    carrierPigeon Limp Gawd

    Messages:
    137
    Joined:
    Sep 22, 2017
    When you have a drive that's a lot bigger than the data that you are trying to back up (let's say 10x the size), is there a good way to keep most of the drive offline so that it's protected against malware (mainly ransomware)? The 2 approaches that I thought of is to
    (1) have a small drive that you use to move data back to the backup drive. In other words, if you have 200GB of data and a 2TB drive, then have a 250GB drive as the go between. The reason for the intermediate drive is because maybe malware doesn't as much live in peripheral files but instead more in core files. One thing that I don't like about this option is that it wouldn't work with "versioning" backup (Windows file history, OS X Time Machine, etc).
    (2) have a lot of the drive formatted in a format the your current OS can't write to. This way, that part of the drive is protected. Periodically, you would use a different operating system to copy the "working part" to the other file system. I guess it might still be tough to do versioning.
     
  31. tedych

    tedych Limp Gawd

    Messages:
    362
    Joined:
    Jan 18, 2013
    Yeah but all this is almost meaningless if you just use remote backup.
    For "one-machine" I ususally use the simplest "defense" by making all unneeded disks Offline in disk management. This is usually disks of other multi-boot OSes on the same machine. For partitions I also don't assign drive letters. I don't know but I guess most malware depends on normal access through drive letters to partitions and not by their lower end ID. At least Offline disks are relatively safe. Of course I wouldn't count on this for critical data but... Partitions I need online for backup are not under some defense, they are used to backup data locally against simple HDD/SSD failures. I haven't been hit by any malware or virus since 2004.

    On the topic, no disk, larger or small, new or old should be trusted today for single backup. But if we have multi-tiered backup (on 2+ disks and places) then the size or age of the disks is irrelevant. 200-500GB drives are by most chances old ones (4-8 years at least) so they are not so reliable anymore anyhow. For some low-volume backups they are Ok. I have a 320GB one I use with a USB3 adapter for external backups once a month. My most critical data and projects fit in under 100GB anyway.
     
  32. westrock2000

    westrock2000 [H]ardForum Junkie

    Messages:
    8,939
    Joined:
    Jun 3, 2005
    Make the drive read only when you don't want anything to write to it? Might not be good enough for Stuxnet level viruses, but I would think it would be ok for normal ransomware or other accidental things. Needs elevated privileges to do. So shouldn't be something that can just happen by visiting a website. You could even write a batch file or Powershell script that sets it back to write, runs robocopy to back up, and then set back to read only. Run as a scheduled task. Just an idea.

    https://winaero.com/blog/disk-partition-readonly-windows-10/
     
  33. HammerSandwich

    HammerSandwich [H]ard|Gawd

    Messages:
    1,104
    Joined:
    Nov 18, 2004
    You can use Disk Management to offline the drive, but physically disconnecting power (e.g., via USB dock) would be better.