Large Repositories on Windows.

Discussion in 'SSDs & Data Storage' started by bigdogchris, Sep 11, 2019 at 4:22 PM.

  1. bigdogchris

    bigdogchris [H]ard as it Gets

    Messages:
    17,885
    Joined:
    Feb 19, 2008
    In a generic sense, what are your thoughts on scaling out in large storage repositories in Windows?

    One option may be to scale a single volume across multiple RAID sets.

    Another option is to create multiple volumes and scale up via software extents.

    What's your preference on Windows?
     
  2. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,937
    Joined:
    Jul 24, 2008
    What is the workload for the repository in question? How large an array or installation are you thinking about? What increase in storage do you see adding in the next 60 months to meet needs? What is your current infrastructure? How many clients do you expect to be hitting the box(es)? What are your uptime needs (5 9's? for example)? What is your backup methodology and what is your backup window? What are the primary applications that will be accessing the data? What is your budget for both the initial capital expenditure and the ongoing expense budget?
     
    GoldenTiger and Wade88 like this.
  3. x509

    x509 [H]ard|Gawd

    Messages:
    1,768
    Joined:
    Sep 20, 2009
    OK, I'm uninformed here. What is a "storage extent" Is this something in Windows, or just Linux?
     
  4. bigdogchris

    bigdogchris [H]ard as it Gets

    Messages:
    17,885
    Joined:
    Feb 19, 2008
    What is the workload for the repository in question? Backups
    How large an array or installation are you thinking about? 100-150TB
    What increase in storage do you see adding in the next 60 months to meet needs? 50%
    What is your current infrastructure? RAID60
    What are your uptime needs (5 9's? for example)? 98%
    What is your budget for both the initial capital expenditure and the ongoing expense budget? Budget irrelevant already purchased but have money to add on.

    I'm leaning towards extents.
     
  5. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,937
    Joined:
    Jul 24, 2008
    Well, since you state you have already purchased hardware and thus "budget is irrelevant" than please provide a list of what hardware you have to work with, and we'll give you the best suggestion based upon what you have. How many clients will be hitting the box concurrently? Are they image or file based backups? 1G, 10G, 40G or 100G (Per client). What is your backup window (in hours?) What app(s) are you utilizing for backup? You mentioned Windows for the host OS on this box, is it mandatory for your backup host software? Can you VM a ZFS host and a Windows host on this box to support the robustness of ZFS as your storage OS for the array even if you are using Windows as a conduit? You are asking very general questions where the needs usually have very specific answers.
     
    GoldenTiger likes this.
  6. Meeho

    Meeho [H]ardness Supreme

    Messages:
    4,469
    Joined:
    Aug 16, 2010
    Must it be Windows? I would never consider Windows for such a task and would go ZFS.
     
  7. ND40oz

    ND40oz [H]ardForum Junkie

    Messages:
    11,421
    Joined:
    Jul 31, 2005
    If you're using Windows 2019 and have controllers that will actually pass the disks through, I'd ditch the RAID setup(s) and use Storage Spaces with ReFS and enable dedupe. If you have to use the RAID arrays, just have a few of them and they can each be their own repository. What backup software and underlying hardware?