How to maintain the data integrity on windows?

zhup

n00b
Joined
Apr 15, 2015
Messages
18
Hello,

I use the Stablebit DrivePool and Scanner on the Windows Server 2012 R2 Essentials – this is my NAS system.
There is only lack of protection against the silent file corruption.

Are there any automated solution (on Windows) to this issue?
Thanks.
 
Hello,

I use the Stablebit DrivePool and Scanner on the Windows Server 2012 R2 Essentials – this is my NAS system.
There is only lack of protection against the silent file corruption.

Are there any automated solution (on Windows) to this issue?
Thanks.

If you need protection against any sort of errors, you need
- checksums on metadata and data (otherwise you cannot detect any errors)
- copy on write to avoid corrupted filesystems

Microsoft adopted these features from ZFS with its new ReFS filesystem
 
Unfortunately Stablebit DrivePool does not support the ReFS filesystem.

I do not want to go in Ms Storage Spaces and I do not know if Drive Bender or PoolHD are good enough sollution?

Are there any other possibilities?
 
What is your current disk setup? Just few disk pooled together? You can try snapraid and it works fine with DrivePool but I think you'll need to get a parity drive (you should get one anyway if you don't have any form of "raid" setup at the moment).
 
SnapRAID keeps checksums of all the files and will tell you if a file has become corrupt and allow you to rebuild an uncorrupted copy from parity.
 
What is your current disk setup? Just few disk pooled together? You can try snapraid and it works fine with DrivePool but I think you'll need to get a parity drive (you should get one anyway if you don't have any form of "raid" setup at the moment).
I have 2x3TB = pooled (drivepool).
Will be the additional 3TB hdd enough for parity or I should install 4TB? Only one hdd for parity is secure?

SnapRAID keeps checksums of all the files and will tell you if a file has become corrupt and allow you to rebuild an uncorrupted copy from parity.
How it will be work? (2 x hdd with content + 1 x hdd with parity).
What happends if I remove one hdd with content? Will all files be available/accessible directly from another computer?
 
What happends if I remove one hdd with content? Will all files be available/accessible directly from another computer?

With SnapRAID yes. The files are stored as normal files on the native filesystem. You can pull any or all drives out and put them in any other machine with the ability to read that native filesystem and read the disk. You do not need SnapRAID on the other machine for that.

Only one hdd for parity is secure?
Although you can I would not use less than 2 parity drives.

Will be the additional 3TB hdd enough for parity or I should install 4TB?

The parity drive needs to be a little bigger (because of fragmentation) than the amount of space that is used on the most used drive.
 
Thank you for your help.

I still have some questions. How SnapRaid works together with DrivePool?
The DrivePool locate the files on two drives simultaneously and the SnapRaid makes the parity for eventual repair of files?
What should be installed first?
 
In the end it does not matter
as any backup solution like Snapraid must trust unverified data from initial save.
If something happens prior a backup run, you are only able to restore garbage.

NTFS + backup is ok but not a perfect solution.
If you need to trust data, you cannot do that at stage 2 or three but only on initial save with checksums. Backups on untrusted data is untrusted, does not matter what security you have at that stage.
 
If you have only one desktop that is your only machine, but you have put all your hard drives in it, can snapraid be installed on this machine if it has win 7 already?
 
If you have only one desktop that is your only machine, but you have put all your hard drives in it, can snapraid be installed on this machine if it has win 7 already?

Yes. you do not even have to change the format of the disks or copy your data (assuming your data is stored on several independent drives). Your parity drives need to be on a filesystem that can support a file that is nearly as large as the disk ( meaning no fat32 - I am not sure what the exfat limitations are since I do not use that).
 
In the end it does not matter
as any backup solution like Snapraid must trust unverified data from initial save.
If something happens prior a backup run, you are only able to restore garbage.

NTFS + backup is ok but not a perfect solution.
If you need to trust data, you cannot do that at stage 2 or three but only on initial save with checksums. Backups on untrusted data is untrusted, does not matter what security you have at that stage.

So, what do you suggest?
I have Dell T20 server (Stablebit DrivePool and Scanner on the Windows Server 2012 R2 Essentials) as NAS for my photos.
All the photos from this machine are also backuped to the cloud.
 
Your initial question was:
How to maintain data integrity on Windows.

The correct answer is: Storage Spaces and ReFS
as this is the Microsoft solution for this problem.

All other solutions on a Microsoft system cannot achieve a similar level
of data integrity but you can use NTFS and tools around what can give
you some flexibility and performance that is currently a problem with ReFS.
 
Your initial question was:
How to maintain data integrity on Windows.

The correct answer is: Storage Spaces and ReFS
as this is the Microsoft solution for this problem.
...
What happends if I remove one hdd with content from the storage spaces - two-way-mirror? Will all files be available/accessible directly from another computer on WIndows 8?
 
stay away from storage spaces. theres absolutely no reason to introduce unnecessary risk of striping on home media. not to mention unnecessary abstraction layer of storage space pool, which doesn't allow you to remove single disk and read in another machine.

Drivepool + SnapRAID is superb. _Gea is talking about ideals and enterprise data and his answer will always be ZFS. but that's not necessary for a couple HDDs of movies and music. I don't give a shit about a bit flip in a 40GB movie, assuming one ever even happened. the whole issue is overblown in the context of home media storage.
 
What happends if I remove one hdd with content from the storage spaces - two-way-mirror? Will all files be available/accessible directly from another computer on WIndows 8?

As far as I know, there is no ReFS support in Win 8.0
I have not tried but it should be in 8.1


@DPI
The answer for data integrity problems is not ZFS but data checksums and Copy On Write
ZFS is the most widely used and developped free solution with these new features.
They are in btrfs and ReFS as well and if you have enough money, this is a main feature in WAFL (NetApp).

The question of the filesystem features are also independent from the raid question.
No one hinders you to use a new gen filesystem without realtime raid but with a backup solution like Snapraid.

You can build storage without these features but not when you talk about data integrity especially in the multi-terabyte area.
 
Super refreshing to hear _Gea who created and maintains the awesome tool Nappit and yet is not a fanboi/zealot.
 
migrate from ntfs to refs and enable user data hash summing

https://technet.microsoft.com/en-us/library/hh831724.aspx

https://technet.microsoft.com/en-us/library/jj218351.aspx

just make sure you don't enable integrity streams over running vms as it's not supported (guess because of the performance impact)

if you're lucky to have proper storage spaces with redundancy refs would even heal itself (with some limitations and issues)

https://social.technet.microsoft.co...healing-and-vhdx-volumes?forum=winserverfiles

Hello,

I use the Stablebit DrivePool and Scanner on the Windows Server 2012 R2 Essentials – this is my NAS system.
There is only lack of protection against the silent file corruption.

Are there any automated solution (on Windows) to this issue?
Thanks.
 
Back
Top