FreeNAS and Thin Provisioning

[sjw7]

Hi All

I am running ESXi 4.1 update 1 and trying to get FreeNAS 7.2 setup primarily as a UPnP server (would like to go to version 8 but it doesnt have UPnP)

I want to create a few disks to attach to it and have them thinly provisioned. Unfortunately when i come to format them with UFS in FreeNas it fills everything up with zeros so they end up being fully provisioned.

Has anyone managed to setup FreeNAS with thin provisioned disks?

Many thanks

Simon

 

[Red Squirrel]

I would not do thin provisioned storage. We did it at work and it was just a disaster. We had thin provisioned VMs on top of thin provisioned storage. It's impossible to know how much space you actually have left and you could burst any minute but have no way of really knowing.

Thick provision the storage, and only thin provision the VMs. Thick provisioning the VMs is even better. But if you don't have all the space in the world and need to think provision somewhere, do it at the VM level.

 

[defuseme2k]

With NFS the NAS device decides on the provisioning, so you need to take a look at its options. Remember with NFS/NAS you're using the underlying device's file system and such... not VMFS.

 

[defuseme2k]

I would not do thin provisioned storage. We did it at work and it was just a disaster. We had thin provisioned VMs on top of thin provisioned storage. It's impossible to know how much space you actually have left and you could burst any minute but have no way of really knowing.

Thick provision the storage, and only thin provision the VMs. Thick provisioning the VMs is even better. But if you don't have all the space in the world and need to think provision somewhere, do it at the VM level.

Sounds like you didn't really manage it properly, no offense. We're doing it with many thousands of VMs, so I know it can be done . We're doing thin on thin . The storage views in vCenter usually have very good information on tracking space utilization from the VMware perspective. You must use both the storage tools and vCenter to get a good guage of it. You can see overcommitment values, etc, so I disagree that you don't really know how much space you have left. If you have a thin provisioned server, but have so much visible free space in there that someone can drop a big ass file in there and blow it up... well, size your servers better or leave the ones that could potentially grow a lot as thick disks.

Ultimately if you have trouble, I wouldn't thin provision on the storage side of the house. In my personal experience you're better off just thin provisioned in VMware land. Because, if you roll out a thick VM.. it allocates that space up front and you'd have to do a zero reclaim on the storage side to get the space back.

By default, thick vs thin has almost NO performance difference, with the caveat of eagerzeroedthick disks. Since the default thick disks are lazyzeroed, it zeros on write much as thin provisioned disks do. Thin disks may have to allocate more space ("to grow") and make a VMFS metadata update -- in that case is the only time you might see some performance hit at all. I always use 8mb block formatted datastores to help with that, although its somewhat of a religious debate if that even matters. So, its so neglible it doesn't matter. Do a google search for performance and thin provisioned disks. VMware has a white paper showing the numbers.

 

[sjw7]

Thanks for the replies. A bit more info from me

The server i am using is a HP Microserver with four 750GB drives that are being used as VMFS stores. All disks added to it are thinly provisioned to save space. I dont need any real performance out of it as its going to be running a very light workload as a backup target and for streaming videos to my PS3. Asside from that it will be used for trying out new software and OS as an evaluation environment.

When creating the disks for the FreeNAS VM i can thinly provision them but as soon as i format them from within FreeNAS it fills them up with zeros and i end up with empty but fully provisioned disks which isnt what i want.

At some point i was going to replace the 750GB disks with 2TB ones and migrate the VMDK files over. This is the reason for not using RDMs also I was going to setup one virtual disk per share on the FreeNAS box and set them to be around 1.5TB in size some of which will only have a few tens of gb in so i cant have them end up being fully provisioned.

Cheers

Simon

 

[defuseme2k]

I understand this a bit better now. Ouch, yeah, that stinks if you can't do a format with freenas that doesn't write zeros. Sucks. The only thing I can think of as a quick around solution is to either setup vCenter using the 60 day trial which is unrestricted. That would let you use svMotion and thin it back out, or you could use VMware Converter to move it to the new datastore with a different name, etc temporarily.

 

[Red Squirrel]

Sounds like you didn't really manage it properly, no offense. We're doing it with many thousands of VMs, so I know it can be done . We're doing thin on thin . The storage views in vCenter usually have very good information on tracking space utilization from the VMware perspective. You must use both the storage tools and vCenter to get a good guage of it. You can see overcommitment values, etc, so I disagree that you don't really know how much space you have left. If you have a thin provisioned server, but have so much visible free space in there that someone can drop a big ass file in there and blow it up... well, size your servers better or leave the ones that could potentially grow a lot as thick disks.

Ultimately if you have trouble, I wouldn't thin provision on the storage side of the house. In my personal experience you're better off just thin provisioned in VMware land. Because, if you roll out a thick VM.. it allocates that space up front and you'd have to do a zero reclaim on the storage side to get the space back.

By default, thick vs thin has almost NO performance difference, with the caveat of eagerzeroedthick disks. Since the default thick disks are lazyzeroed, it zeros on write much as thin provisioned disks do. Thin disks may have to allocate more space ("to grow") and make a VMFS metadata update -- in that case is the only time you might see some performance hit at all. I always use 8mb block formatted datastores to help with that, although its somewhat of a religious debate if that even matters. So, its so neglible it doesn't matter. Do a google search for performance and thin provisioned disks. VMware has a white paper showing the numbers.

Oh, it's doable, it's just a pain in the ass. I rather use up and allocate the space so I know exactly how much space I'm using and how much I have left, and not artificial numbers where I need to start doing my own tracking. We were against it from the start but someone else with more power did it. To make matters worse we had an undersized SAN. so we were using 3TB out of 4TB but in reality we had like 10TB worth of data (from the OS perspective). It was about to burst any moment and we had no way of knowing when.

 

[defuseme2k]

Oh, it's doable, it's just a pain in the ass. I rather use up and allocate the space so I know exactly how much space I'm using and how much I have left, and not artificial numbers where I need to start doing my own tracking. We were against it from the start but someone else with more power did it. To make matters worse we had an undersized SAN. so we were using 3TB out of 4TB but in reality we had like 10TB worth of data (from the OS perspective). It was about to burst any moment and we had no way of knowing when.

Once it becomes politically charged it isn't a fair discussion any longer. Playing around with data in that sense is lunacy. Being against it probably had no political traction, but if they didn't listen and mitigate concerns that was fairly stupid I'll agree there. I'm all for thin provisioning, but the servers have to be right sized, well monitored, and must be performing OK to even attempt it.