Windows Storage Server Single Instance Storage software dedup

[Thuleman]

Couldn't decide where to post this and figured it's most appropriate here since it is about image based VM backups.

I recently sat through a Data Domain webex on their hardware dedup appliance were some case studies showed what I would call extreme backup "compression" rates based on de-duplication of image based VM backups. In one case the ratio was 150:1, though the marketing material says at least 20:1 and the general tenor was about 30-40:1 for most VMs.

I never was a fan of dedup appliances just because for the capital cost of one of those I could just buy more all purpose storage and have space for backups and general purpose use.

However, I also just now saw Windows Storage Server 2008 which now includes a software de-dup like feature called Single Instance Storage. It's basically hard links in the file system pointing to one instance of the file even if it exists multiple times on the file system.

I am inclined to like that solution because it's vendor agnostic (if you don't count MS as the vendor in this case). If a <insert vendor here> hardware appliance goes belly up you need to deal with their support, support contracts, etc., the same is true if you buy an appliance that runs Windows Storage Server (like the Dell NX3100). However, if you spin up your own Windows Storage Server, you won't have that issue. Apparently Windows Storage Server is only available to OEMs though it may be possible to get a copy through a VAR.

Ok, on to the actual question!
Does anyone have experience with Windows Storage Server and SIS? I am tempted to download it from TechNet to have a look myself and see what's what, but if someone else already has a "first look" then I'd take your word for it one way or another.

 

[SJConsultant]

Ok, on to the actual question!
Does anyone have experience with Windows Storage Server and SIS? I am tempted to download it from TechNet to have a look myself and see what's what, but if someone else already has a "first look" then I'd take your word for it one way or another.

We are in the process of rebuilding our storage infrastructure and using WSS2008 as the backend storage.

Just an FYI Microsoft's SIS is file based, meaning it looks at the entire file where other vendors are "block based" meaning they compare blocks of data inside the file.

Once you understand those differences, you'll understand why some of those appliances are so expensive.

Microsoft's SIS is "ok" meaning that the files have to be identical in order to get any benefit from SIS. They can be named differently, but the contents must be identical to get SIS'd. We have a need to have multiple copies of things like ISOs and other utilites in multiple locations so it will be a benefit to us to have the data SIS'd.

If you don't have a lot of duplicate files, then SIS isn't going to do anything for you.

If you have multiple files with identical data *in* the files, then block based dedupe would be more beneficial for saving storage.

We've been experimenting with Nexentastor which uses a ZFS file system with inline block level deduplication to see how much more storage we can save.

Unfortunately for us, Nexentastor hasn't been stable on various hardware thus we are going to shelve it and revisit it at a later time.


The only other feature in WSS2008 that may prove to be useful is the ISCSI Target. We'll be experimenting with that once we get the rest of our data migrated to it.

 

[Thuleman]

That's a good observation of block based vs file based. I knew that but for some reason it didn't stand out when I wrote the original post. File based de-dup won't actually help me when all I am looking at is storing image based VM backups. Thanks for clearing that up.

 

[NetJunkie]

What SJ said. Data Domain is a great product..we sell a lot of it. VERY good dedupe ratios as it is sub-file level dedupe. Windows SIS would be OK for user file directories but you'd still get worse dedupe over something else, like the dedupe on an EMC Celerra array as it does sub-file dedupe and compression. So if you send me a Word file and I change something and save it, it'll still dedupe all the stuff that is the same between them. Makes a big difference.

 

[nitrobass24]

So if you are using SIS, and lets say i have 50 ISO images and i want them in three different places. SIS just places a "link" and just maintains one copy of the file.

So i have my original on say Z:\ISOs and then i copy my file 2 other places. What happens when i delete Z:\ISOs? Will my other two copies still work?

 

[SJConsultant]

So if you are using SIS, and lets say i have 50 ISO images and i want them in three different places. SIS just places a "link" and just maintains one copy of the file.

Correct, the deduped ISO image would be placed in a SIS folder which is hidden from normla view and "links" are put in the original location.

So i have my original on say Z:\ISOs and then i copy my file 2 other places. What happens when i delete Z:\ISOs? Will my other two copies still work?

If you delete a file from one location, then only the link is deleted not the SIS'd file and your other links continue to work.

 

[geiger]

If you have a Windows box that is 2003 R2 or later you could add the File Storage Resource Management role and run the duplicate files report on a volume. That will give you an idea of the space saving from SIS.

The DD boxes are great for backup to disk configurations. It allows you to keep a much longer retention period on data that may be large, but only have a small amount of change. A backup image may have slight block changes from backup to backup. I would not use a DD box as a fileserver though.

The SIS option would be good for an active fileserver especially something like a user profile silo or departmental share.

 

[Thuleman]

WSS is on MSDN AA (departmental account, not sure whether a student subscription would have it), but it's called Windows Storage Server 2008 Embedded Basic, which makes me wonder just what exactly embedded means and whether that's going to be an issue if attempting to just install on a whitebox. Guess there's one way to find out ...

 

[nitrobass24]

I just installed it on a hyper-V VM and it didnt have any issues.

The only thing is it never asked me to setup the administrator account, and it already has a password. Not sure what it is?

 

[SJConsultant]

I just installed it on a hyper-V VM and it didnt have any issues.

The only thing is it never asked me to setup the administrator account, and it already has a password. Not sure what it is?

The default password on all installs is "wSS2008!"

 

[Thuleman]

Decided to bump this rather than make a new thread.

So I read elsewhere that there will not be a service pack for WSS and that WSS features are being rolled into the normal Windows Server lineup though SIS for example will only be available to OEM customers of Windows Server and will not be available in retail versions.

Anyone able to shed some light onto that? Is it generally speaking a bad idea to install WSS on a production machine?

 

[nitrobass24]

I wish WSS08 had block level dedup. or there was some block level dedup software for windows that wasnt so F$$$$n expensive.

@Thuleman
Just signed up for the beta on connect and it does appear that WSS is now going to be an addon to Server2008 opposed to a different OS.

Have not had time to install it and see what is there/not there.
BTW its an open beta so anyone can sign up.

 

[Thuleman]

They have WSS on MSDNAA as well where I got it from though I'd rather not actually use it if MS decides to discontinue it. It's probably safest to just install Standard R2 and wait for the SIS addon rather than be stuck with a discontinued OS.

I have my eyes set on Data Domain's dedup appliance, just need to find the cash for it really quick and I'll be in business with that.