Best option for adding more sata ports

warhol76

n00b
Joined
Jan 1, 2013
Messages
63
Is there a limit to how often it can be run? Can I set it to run hourly or every 3-4 hours, or would that be detrimental to performance?
That would depend on how much new information there was for it to write. For example, my initial 'sync' took about 2.5 hours. This was for about 3.5TB. If you data changes very little, the 'sync' will only take a couple of minutes. If there is no change, it will take a seconds.

It does use some CPU. I have an overclocked 6600k. It uses about 8% of CPU (I assume that's only one core, but I haven't checked.) But, that is only while the 'sync' is running.

So, after your initial 'sync' you could theoretically run it hourly without issue. On a server, I wouldn't think this would cause any issues.
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
Like what is SMB/Samba? No idea what that even means lol.
This is what Microsoft calls their network protocol for sharing files. When you tell Windows to 'share' a directory or file in File Explorer, it does so using SMB, and Linux systems access this through 'Samba', which is the Linux implementation of the protocol.
Hyper-V VM – is that different from a "regular" VM? I just built my first computer with the help of my brother-in-law, so that's the level of skill I'm at now ha ha
Hyper-V is Microsoft's VM implementation. It's built-in to Windows, and can be used with Windows 10 Pro. Setting up a VM is something like using Partpicker to select the parts you want to use; you tell Hyper-V how much RAM, how many threads, how much hard disk space etc. to give a particular VM, and then you feed it an ISO to load the operating system from and install it exactly as you would if you were using a USB stick.
The whole point of running a raid array or any other mirroring/duplication is for data integrity, so if Windows Storage Spaces cannot be relied upon for data integrity then it's something I should probably avoid.
ZFS is about the only solution that will actually ensure data integrity that is also actually accessible and affordable. FreeNAS is the most approachable operating system that implements ZFS and exposes it through a user interface.
The SAS HBA I got is in fact PCIe 3.0 (LSI 9207-4i4e). I actually initially ordered a 2.0 card per my earlier posts, but then decided on a 3.0 card per comments from UhClem and Nobu a few posts up. I still don't fully understand the concept of PCIe lanes, but UhClem had indicated this 9207 card could run on 4 lanes with the same bandwidth of the 9211 on 8 lanes. I do know that each version of the PCIe specification has doubled in speed, so the math makes sense. But what I don't know is if that means I'm capped at using only 1 of the 2 SAS ports on the card when running on only 4 lanes or if it's load-dependent. My 2060 KO is taking 16 lanes, and the only other PCIe card I have is a Rosewill USB 3.0 x1 card. My 3rd x16 slot is empty. If my understanding of my board's specs is correct, I should be able to keep this HBA in the bottom x16 slot to force it on 4 lanes. If I moved it to the middle x16 slot, it would/could run on 8 lanes which would limit my GPU to 8 lanes. This is all pretty new to me, so I'm not sure I really get the lane / bandwidth allocation aspect of PCIe yet.
I think you've got that right. Main thing with PCIe and using fewer lanes is whether the card in use will behave in such a scenario. The answer to that is 'highly likely', but understand that it's a question nonetheless. From a bandwidth perspective, PCIe 3.0 x4 would be plenty. Spinners max out a <300MB/s per under the best case scenario today. You'd need to use SSDs to actually push PCIe bandwidth limits and you'd probably hit other limits before that matters.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
for real time protection of data (its not a backup)

you have ZFS (like freeNAS that is soon to be TrueNAS) but more ideal for a dedicated system not for main pc as its for nas use

if your wanting to stay with windows and only one PC but want real time redundancy then a SAS Raid card (with battery if you want write back enabled) instead of a SAS HBA card you can get them quite cheap, and that will allow you to use RAID6 and the card will do all the hard work,, use LSI megaRAID software to manage it, so you can setup times when you want it to do read patrol and consistency check, common suggestion is 1 week for read patrol and once a month for consistency check) if you do go with raid card make sure you buy 2 of them (just in case rare case one card fails)

(GPU be fine at 8x speed it really does not need 16x)
I've seen it mentioned several places that Raid isn't a backup, but based on everything I've read I think that's pretty misleading. The arguments in favor of that statement usually mention issues unique to parity arrays and striping, rebuilding issues due to workload placed on other drives in the array, fire, flooding, theft, etc. But fundamentally any raid array that's not pure striping like raid 0 is in theory providing some amount of security to your data. If it didn't, there'd be no point in using it. If performance was the only factor, everyone would use raid 0. If downtime of a server was the only concern, raid 1 seems like the ideal solution since the only downtime would be the time required for the system to switch over to the mirror. The fact that parity exists to rebuild the data on a failed drive makes it a backup solution by definition, just perhaps not a complete or ideal or fullproof solution.

And when it comes to software solutions like drivepool mirroring, you quite literally are creating a complete independent backup of data that could be used instantly, on any similar system without any configuration, in the event of drive failure. Sure, the physical drive may still be stored in the same physical location as the original. So it too would likely also get lost with fire, flood, or theft, but that only means it's just not a complete backup solution either.

I get that using Raid alone wouldn't constitute an acceptable level of security for a business environment, but consumers usually can't afford and really don't need 100% complete data security. That's a very expensive proposition, so consumers need to weigh the risks and benefits of each option available. Of course being a consumer myself, I'm doing just that. I'm only looking for a Raid solution for my Plex server. If I were to lose all that data, it's not going to be the end of the world. I won't lose any money (aside from having to replace hardware), I won't lose my job, my wife won't divorce me...i don't think lol. It'll just be a royal pain in the ass to rebuild myself, that's all.


Sorry to turn this into kind of a rant. It just seems like sometimes people (probably those who work in the industry) lose sight of the big picture. I want the best balance of security, flexibility, and cost. That's highly subjective of course, but I can tell you that options like cloud storage or physical storage at 2+ independent locations is not on the table in my book... Far too expensive. Even mirroring isn't ideal for me due to cost, but when considering traditional raid 6 vs. mirroring, it's tough not to just pick mirroring due to the benefits gained by using software like drivepool or drive bender. I'm not really concerned with performance based on my use case and the fact that I bought 7200rpm drives. Being very new to all this, the simpler solutions are more attractive to me: simpler setup, simpler management, simpler expansion, and simpler recovery.

So with that said, i think I've ruled out a dedicated raid card for a few different reasons, unless you have a really compelling argument that I haven't heard before. I'm still currently looking into drivepool+snapraid, freenas VM, and unraid VM. VMs are completely foreign to me, so I'm still trying to gage just how much work it's going to be for a newbie like me to get one setup and running. I'm also primarily leaning towards the unraid VM option because of the flexibility with expansion. I'm not getting a rosey feeling about Windows raid solutions, so I think I'd rather not risk it with that option.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
That would depend on how much new information there was for it to write. For example, my initial 'sync' took about 2.5 hours. This was for about 3.5TB. If you data changes very little, the 'sync' will only take a couple of minutes. If there is no change, it will take a seconds.

It does use some CPU. I have an overclocked 6600k. It uses about 8% of CPU (I assume that's only one core, but I haven't checked.) But, that is only while the 'sync' is running.

So, after your initial 'sync' you could theoretically run it hourly without issue. On a server, I wouldn't think this would cause any issues.
That's not that bad. I currently have about 5.5TB data on my WD Blue that needs to go over to the Gold array whenever I decide how to build the array. Once that's all copied over, it's not going to be all that much work for snapraid writing. I don't put new data on all that often. I was just asking about hourly more for knowledge about performance. I don't think I actually need it to sync quite that often.
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
I've seen it mentioned several places that Raid isn't a backup, but based on everything I've read I think that's pretty misleading.
RAID is an availability solution. Meaning that it can keep you working when a drive fails. It's not a backup because that's all it protects against: drive failures. By itself, RAID doesn't protect against a myriad of other failures, nor user error, nor natural disasters. If your data is that important to you, you'll have RAID for storage pooling / availability, an offline local backup, and an offsite backup, at least.

Now, if you're worried about cost: use RAIDZ1 or RAIDZ2. One or two drive parity. And if you only have 5.5TB of data... what's stopping you from picking up the cheapest external USB drive and just syncing to that occasionally? Toss it in a fire-proof document safe etc.?
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
This is what Microsoft calls their network protocol for sharing files. When you tell Windows to 'share' a directory or file in File Explorer, it does so using SMB, and Linux systems access this through 'Samba', which is the Linux implementation of the protocol.

Hyper-V is Microsoft's VM implementation. It's built-in to Windows, and can be used with Windows 10 Pro. Setting up a VM is something like using Partpicker to select the parts you want to use; you tell Hyper-V how much RAM, how many threads, how much hard disk space etc. to give a particular VM, and then you feed it an ISO to load the operating system from and install it exactly as you would if you were using a USB stick.

ZFS is about the only solution that will actually ensure data integrity that is also actually accessible and affordable. FreeNAS is the most approachable operating system that implements ZFS and exposes it through a user interface.

I think you've got that right. Main thing with PCIe and using fewer lanes is whether the card in use will behave in such a scenario. The answer to that is 'highly likely', but understand that it's a question nonetheless. From a bandwidth perspective, PCIe 3.0 x4 would be plenty. Spinners max out a <300MB/s per under the best case scenario today. You'd need to use SSDs to actually push PCIe bandwidth limits and you'd probably hit other limits before that matters.
Thanks for clarifying some of those things. Kind of a challenge when the original explanation involves terms you are also unfamiliar with. It's like translating Chinese into German. I don't speak Chinese, and i don't speak German. German and English are at least derived from the same parent language, so I get something out of it but I'm still mostly lost lol.

So for VM options, why would one choose third party solutions like Virtualbox or VMware over Hyper-V in Windows? And with VM setups, are you allocating resources similar to partitioning a drive... Such that those resources are inaccessible to Windows once allocated to the VM? Or is it more like setting a resource usage maximum for the VM, an upper limit?

You mentioned ZFS is the best for data integrity, and I've heard that before, but i still don't understand specifically why it's the best. Fundamentally, it sounds pretty similar to other options, similar to traditional raid. It build data across all disks, like traditional raid, right? It uses parity, but in the form of a dedicated parity disk not striped parity, which I would assume is actually less secure. It places the same limitations on expansion and recovery as traditional raid. So how is it superior? What am I missing?
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
So for VM options, why would one choose third party solutions like Virtualbox or VMware over Hyper-V in Windows? And with VM setups, are you allocating resources similar to partitioning a drive... Such that those resources are inaccessible to Windows once allocated to the VM? Or is it more like setting a resource usage maximum for the VM, an upper limit?
I can't say that there's a strong reason to use any one over the other; main reason to use Hyper-V is that it's native to Windows, which is your host OS.
You mentioned ZFS is the best for data integrity, and I've heard that before, but i still don't understand specifically why it's the best. Fundamentally, it sounds pretty similar to other options, similar to traditional raid. It build data across all disks, like traditional raid, right? It uses parity, but in the form of a dedicated parity disk not striped parity, which I would assume is actually less secure. It places the same limitations on expansion and recovery as traditional raid. So how is it superior? What am I missing?
ZFS is the whole storage stack. It's doing the low-level stuff on the drive like the RAID controller would do, but it's also managing checksums and volumes and backups (via snapshots) and so on as well. It's basically three or four solutions in one, depending on what you're comparing it to. Also, parity is 'striped' across the array, same as RAID; we just say 'one disk' for single-parity because it's one disk worth of data, same as RAID. So RAIDZ1 is one parity stripe (one disks worth), and RAIDZ2 is two, RAIDZ3 is three, which is the max that ZFS supports.

Also, one big advantage with ZFS is that with the built-in checksumming, ZFS can put your data back together 'correctly' if there's a corruption issue. Most consumer-accessible RAID options do not do this and thus 'guess' as to what's corrupt and what isn't.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
RAID is an availability solution. Meaning that it can keep you working when a drive fails. It's not a backup because that's all it protects against: drive failures. By itself, RAID doesn't protect against a myriad of other failures, nor user error, nor natural disasters. If your data is that important to you, you'll have RAID for storage pooling / availability, an offline local backup, and an offsite backup, at least.

Now, if you're worried about cost: use RAIDZ1 or RAIDZ2. One or two drive parity. And if you only have 5.5TB of data... what's stopping you from picking up the cheapest external USB drive and just syncing to that occasionally? Toss it in a fire-proof document safe etc.?
Yeah that kind of just reinforces what I'm saying. I get that it's not a complete backup, but it does provide backup. Maybe the issue is semantics. I'm not talking about the industry definition of "backup", which without even looking up I'm sure involves some of the qualities previously mentioned like off-site storage. I'm just talking layman's "backup", in that a damaged drive can be rebuilt with no data loss, ergo a safety net or "backup" of the data exists in theory. Not saying things can't go wrong with the raid array and you won't lose everything. I understand that part.

As far as my backup solution, they don't even make external drives large enough to backup these golds... Once they're filled up more. The 5.5tb is only my current amount of data on my WD Blue 6tb drive... It's nearly at capacity. I still have a ton of movies to rip, but I can't until I get more storage. Any kind of external array for a backup would be wise, yes, but also very expensive. I think I would sooner use drive mirroring on my current drives than create an entirely separate backup solution. I just can't afford that much expense. It isn't worth it to me even if i had that money. Not for Plex data.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
I can't say that there's a strong reason to use any one over the other; main reason to use Hyper-V is that it's native to Windows, which is your host OS.

ZFS is the whole storage stack. It's doing the low-level stuff on the drive like the RAID controller would do, but it's also managing checksums and volumes and backups (via snapshots) and so on as well. It's basically three or four solutions in one, depending on what you're comparing it to. Also, parity is 'striped' across the array, same as RAID; we just say 'one disk' for single-parity because it's one disk worth of data, same as RAID. So RAIDZ1 is one parity stripe (one disks worth), and RAIDZ2 is two, RAIDZ3 is three, which is the max that ZFS supports.

Also, one big advantage with ZFS is that with the built-in checksumming, ZFS can put your data back together 'correctly' if there's a corruption issue. Most consumer-accessible RAID options do not do this and thus 'guess' as to what's corrupt and what isn't.
Ok, so is it that ZFS is statistically better at preventing failures and/or rebuilding after a failure? Or is it just better in theory based on the features offered? Like in the real world use case, does ZFS have lower failure rates and more accurate/successful data restoration ability?

ZFS backups are built into the array itself or are you talking about ZFS's ability to interface with the backup hardware?

What differentiates ZFS from a traditional hardware raid solution? It sounds the same to me now, except nomenclature and the z3 option. And when you say other options just kind of guess on the data to restore a drive with, are you including unraid and snapraid in that?
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
Ok, so is it that ZFS is statistically better at preventing failures and/or rebuilding after a failure? Or is it just better in theory based on the features offered? Like in the real world use case, does ZFS have lower failure rates and more accurate/successful data restoration ability?
Answering your question definitively would require a doctoral thesis; as it stands, ZFS is a 'feature-complete' solution, and the most feature-complete solution that's available to consumers (and, well, free).
ZFS backups are built into the array itself or are you talking about ZFS's ability to interface with the backup hardware?
ZFS has a built-in ability to send and receive snapshots, which can be entire backups as well as incremental backups. You can also set up datasets instead of exposing the pools directly and then back those up independently.

What differentiates ZFS from a traditional hardware raid solution? It sounds the same to me now, except nomenclature and the z3 option. And when you say other options just kind of guess on the data to restore a drive with, are you including unraid and snapraid in that?
Checksumming, datasets, basically the whole volume management thing. Unraid is more flexible, but does so by running significantly slower; I can't speak to their solution being better at recovery, I just find their proprietary approach with a key on a USB stick to be somewhat objectionable if the system is anything more than a media server.

As far as my backup solution, they don't even make external drives large enough to backup these golds... Once they're filled up more. The 5.5tb is only my current amount of data on my WD Blue 6tb drive... It's nearly at capacity. I still have a ton of movies to rip, but I can't until I get more storage. Any kind of external array for a backup would be wise, yes, but also very expensive. I think I would sooner use drive mirroring on my current drives than create an entirely separate backup solution. I just can't afford that much expense. It isn't worth it to me even if i had that money. Not for Plex data.
Assuming you keep the optical media, for the Plex data, that is a backup. Just one that requires more work to recover from. What I'm thinking of is a cheap 8 / 10 / 12 terabyte external USB drive that you can backup your important stuff and whatever else onto, then throw in a safe.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Answering your question definitively would require a doctoral thesis; as it stands, ZFS is a 'feature-complete' solution, and the most feature-complete solution that's available to consumers (and, well, free).
That's not very compelling from my standpoint though, because I still do not understand why it's better than the others and what "better" even means. To someone well versed in linux and server design/management, ZFS might be the best thing on earth. But I might not be using 99% of the features it offers, as that would require years of experience in linux, server management, and probably a million other areas. I don't even know your professional qualifications, so for all I know you could either be a joe-shmoe consumer like me or you could be a high level network engineer for Amazon AWS or Oracle. If the latter, it would be understandable that you would want something as versatile and feature-rich as ZFS. It might cause me more headache and grief just trying to get it setup than is even worth in the end. I'm trying to figure out why it would be the best choice for me, given the limited use for which I need it for and given my lack of knowledge in many areas required for it's configuration.

The free component is a plus, no doubt, but it's not super high on my list considering unraid is just a one-time fee.

ZFS has a built-in ability to send and receive snapshots, which can be entire backups as well as incremental backups. You can also set up datasets instead of exposing the pools directly and then back those up independently.
I really don't even know what a snapshot is, and I definitely do not know what dataset means in this context. I imagine snapshot is something similar in concept to parity? Like a screenshot of the contents of the storage device? Do snapshots require building like restoring a drive with parity? Or is it complete data? Either way, doesn't that require additional storage to host the snapshot files?

Checksumming, datasets, basically the whole volume management thing. Unraid is more flexible, but does so by running significantly slower; I can't speak to their solution being better at recovery, I just find their proprietary approach with a key on a USB stick to be somewhat objectionable if the system is anything more than a media server.
I do understand the basic concept of a checksum; I just never knew it was really that important. Or rather I didn't know it was ever really needed...seemed like something for the paranoid. Though my only experience with checksums was with downloading files off places like Github. Never bothered to use the checksums for checking the file in the end. I get why it might be more critical when we're talking about data storage, especially multi-terabyte drives, given the statistics on failure rates of drives.

Regarding unraid, my understanding is that it will run at normal drive speed. Is this not the case? I don't need performance beyond the specs of a single drive, and I could surely even deal with slightly lower performance. But if unraid is going to dog my drives to the point of causing issues with Plex, then obviously there's no point in using it. I've seen enough unraid plex server tutorials on YouTube to believe that isn't the case though.

I do not really like the idea of using a USB stick to host the OS either, but every tutorial I've seen for FreeNAS says to do just that. I actually watched one yesterday saying FreeNAS was designed to run on a USB stick. So it seems that FreeNAS and Unraid are the same in that respect. I do have a bunch of extra Verbatim USB-A and USB-C (2.0 & 3.0 of each) drives laying around that my wife got for free from work, so I certainly have the means to do it. But it does seem risky.

Assuming you keep the optical media, for the Plex data, that is a backup. Just one that requires more work to recover from. What I'm thinking of is a cheap 8 / 10 / 12 terabyte external USB drive that you can backup your important stuff and whatever else onto, then throw in a safe.
My BD drive technically could serve as a backup, but quad layer discs top out at 100GB and are not cheap! Not to mention, if I want true permanent backups with no data creep (or whatever its called), I'd need to use M-discs, which are also super $$$. When I built this machine, I bought a 100GB M-disc on amazon thinking it was a 4 or 5 pack. It was 1 disc and it cost $20. So to backup my full 36TB storage onto 100GB quad layer M discs would require 369 discs at a total of $7,380...not gonna happen lol. I just don't see the point of the external drive idea. A "cheap" 12TB external drive still isn't cheap, and I'd need 3 of them to fully backup my 36TB. If I were to go that route, the only real benefit of doing that over shucking the drives and creating a mirror is the fire/flood/theft factor. And if my house burns down, my Plex data is the last of my worries, you know?
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
That's not very compelling from my standpoint though, because I still do not understand why it's better than the others and what "better" even means. To someone well versed in linux and server design/management, ZFS might be the best thing on earth. But I might not be using 99% of the features it offers, as that would require years of experience in linux, server management, and probably a million other areas. I don't even know your professional qualifications, so for all I know you could either be a joe-shmoe consumer like me or you could be a high level network engineer for Amazon AWS or Oracle. If the latter, it would be understandable that you would want something as versatile and feature-rich as ZFS. It might cause me more headache and grief just trying to get it setup than is even worth in the end. I'm trying to figure out why it would be the best choice for me, given the limited use for which I need it for and given my lack of knowledge in many areas required for it's configuration.

The free component is a plus, no doubt, but it's not super high on my list considering unraid is just a one-time fee.
'Better' in that it solves all of the problems of storage availability and integrity at once without a significant hit to performance, and that it's portable because it solves those problems on the drives themselves. That means that you can take the drives, boot them up with any OS that has ZFS of the same feature level or higher, and your data is right where you left it.

RAID controllers don't do that. Storage Spaces does, but it's less robust (less well developed), and obviously needs a Windows box to access. I'd still probably take Storage Spaces over a hardware RAID solution though, personally. Hardware RAID ties you to a physical controller and the hope that it's doing everything you need it to, and that you can find another controller to use to recover your data if the current one dies.

I really don't even know what a snapshot is, and I definitely do not know what dataset means in this context. I imagine snapshot is something similar in concept to parity? Like a screenshot of the contents of the storage device? Do snapshots require building like restoring a drive with parity? Or is it complete data? Either way, doesn't that require additional storage to host the snapshot files?
It's a backup slice of the data in a pool or dataset. Main thing is that snapshots can be pointed anywhere, can be encrypted, and can be done as a whole or as increments, by ZFS itself.
I do understand the basic concept of a checksum; I just never knew it was really that important. Or rather I didn't know it was ever really needed...seemed like something for the paranoid. Though my only experience with checksums was with downloading files off places like Github. Never bothered to use the checksums for checking the file in the end. I get why it might be more critical when we're talking about data storage, especially multi-terabyte drives, given the statistics on failure rates of drives.
It's important to guard against bitrot, which is the corruption of data at rest. A basic checksum, done by say 7zip, will tell you whether or not the data you have is the data that was originally stored; ZFS has a process called 'scrubbing' that can be run periodically to check data against checksums and repair data that may have been corrupted using the redundancy within the array.
Regarding unraid, my understanding is that it will run at normal drive speed. Is this not the case? I don't need performance beyond the specs of a single drive, and I could surely even deal with slightly lower performance. But if unraid is going to dog my drives to the point of causing issues with Plex, then obviously there's no point in using it. I've seen enough unraid plex server tutorials on YouTube to believe that isn't the case though.
Unraid will at best run as fast as the slowest drive. You're using nice drives, so that's not likely to be an issue; still, because Unraid is a clever combination of separate technologies, it is also somewhat proprietary. As a solution in and of itself I remain impressed with Unraid for its purpose, but I also don't recommend it by default because of its limitations.
I do not really like the idea of using a USB stick to host the OS either, but every tutorial I've seen for FreeNAS says to do just that. I actually watched one yesterday saying FreeNAS was designed to run on a USB stick. So it seems that FreeNAS and Unraid are the same in that respect. I do have a bunch of extra Verbatim USB-A and USB-C (2.0 & 3.0 of each) drives laying around that my wife got for free from work, so I certainly have the means to do it. But it does seem risky.
I'll say that you can certainly do that, but that relying on USB for the OS is... a bad idea. Just too many unknowns, and too much work to replace something that is inherently unreliable. Thing is, with Unraid you don't get a choice; with FreeNAS, no problem installing it to an SSD (or pair of SSDs in a mirror). Of course if you virtualize FreeNAS, no worries period on that front.
My BD drive technically could serve as a backup, but quad layer discs top out at 100GB and are not cheap! Not to mention, if I want true permanent backups with no data creep (or whatever its called), I'd need to use M-discs, which are also super $$$. When I built this machine, I bought a 100GB M-disc on amazon thinking it was a 4 or 5 pack. It was 1 disc and it cost $20. So to backup my full 36TB storage onto 100GB quad layer M discs would require 369 discs at a total of $7,380...not gonna happen lol. I just don't see the point of the external drive idea. A "cheap" 12TB external drive still isn't cheap, and I'd need 3 of them to fully backup my 36TB. If I were to go that route, the only real benefit of doing that over shucking the drives and creating a mirror is the fire/flood/theft factor. And if my house burns down, my Plex data is the last of my worries, you know?
So, I wasn't clear, I do apologize for that.

What I mean is, all of the stuff that you're putting on drives is coming from discs; you have the actual movies on physical media, so do you really need to back that up?

What I'm recommending is a smaller drive, and a smaller dataset on the array, to put the important stuff on. Toss that drive into a safe.


Also, I figured I'd share the reference: https://openzfs.org/wiki/System_Administration

OpenZFS is also ZFS on Linux; they merged the project, where ZFS on Linux is the lead for development, and the BSD branch (that is, FreeBSD and thus FreeNAS) pulls from there.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
'Better' in that it solves all of the problems of storage availability and integrity at once without a significant hit to performance, and that it's portable because it solves those problems on the drives themselves. That means that you can take the drives, boot them up with any OS that has ZFS of the same feature level or higher, and your data is right where you left it.

RAID controllers don't do that. Storage Spaces does, but it's less robust (less well developed), and obviously needs a Windows box to access. I'd still probably take Storage Spaces over a hardware RAID solution though, personally. Hardware RAID ties you to a physical controller and the hope that it's doing everything you need it to, and that you can find another controller to use to recover your data if the current one dies.


It's a backup slice of the data in a pool or dataset. Main thing is that snapshots can be pointed anywhere, can be encrypted, and can be done as a whole or as increments, by ZFS itself.

It's important to guard against bitrot, which is the corruption of data at rest. A basic checksum, done by say 7zip, will tell you whether or not the data you have is the data that was originally stored; ZFS has a process called 'scrubbing' that can be run periodically to check data against checksums and repair data that may have been corrupted using the redundancy within the array.

Unraid will at best run as fast as the slowest drive. You're using nice drives, so that's not likely to be an issue; still, because Unraid is a clever combination of separate technologies, it is also somewhat proprietary. As a solution in and of itself I remain impressed with Unraid for its purpose, but I also don't recommend it by default because of its limitations.

I'll say that you can certainly do that, but that relying on USB for the OS is... a bad idea. Just too many unknowns, and too much work to replace something that is inherently unreliable. Thing is, with Unraid you don't get a choice; with FreeNAS, no problem installing it to an SSD (or pair of SSDs in a mirror). Of course if you virtualize FreeNAS, no worries period on that front.

So, I wasn't clear, I do apologize for that.

What I mean is, all of the stuff that you're putting on drives is coming from discs; you have the actual movies on physical media, so do you really need to back that up?

What I'm recommending is a smaller drive, and a smaller dataset on the array, to put the important stuff on. Toss that drive into a safe.


Also, I figured I'd share the reference: https://openzfs.org/wiki/System_Administration

OpenZFS is also ZFS on Linux; they merged the project, where ZFS on Linux is the lead for development, and the BSD branch (that is, FreeBSD and thus FreeNAS) pulls from there.
That was a big help. Thank you.

My biggest beef with FreeNAS is the inability to expand the array. Is there no possible way to add on more drives incrementally? I know you can replace existing drives with larger drives, but you need to replace at least 2 of them to see any increase in storage capacity. So at that point on a small Z1 array you might as well just create a whole new Z1 array and have tons of extra capacity. Not the end of the world I suppose, but it could start to get excessive on hardware demands (new case, new PSU, etc.).

I still don't really get the idea of a snapshot. I mean, I understand what you said mostly, but I don't understand how a "backup slice" of data doesn't require storage for itself. Even compressed data still takes up a lot of space.

What you said about the OS drive for VM is the kind of thing I would have never known unless you told me. There's so much I still do not understand about VMs, which concerns me much more than setting up FreeNAS itself. And seeing as how I use this server as my workstation for my job, screwing things up in Windows is not an option I can entertain. I don't suppose you know of a good comprehensive resource for beginners on setting up VMs? Google and YouTube are garbage anymore. No matter if it's PC-related or something related to home construction, cars, etc., I always find myself sifting through hours of crap before finally finding anything of substance. Its become so frustrating in the last couple years.

Now I get what you're saying about the discs. So yes, I would obviously still have the backups of many many movies and TV shows on the physical discs. But there's much more involved in creating those files. My first test rip was Deadpool in 4k using MakeMKV. If you've ever used MakeMKV, it doesn't technically bypass DRM, it merely gets you access to the files on the disc. From there you need to locate the 1 file that contains the actual movie, copy it out, and then populate the metadata on your own. Its a bit of work sometimes. A newer movie like Deadpool isn't so hard to find metadata for, and I have a license for a program that searches for the metadata on a few different servers then auto-writes it to the MKV. Some of my older movies or "cheaper" movies ala Amazon sketch merchants are harder to locate the data for. Then there's matching that metadata to Plex metadata, which isn't always the most seamless process. So while I technically have backups for these discs, it's still quite a process to get them from physical disc to Plex entry.

In addition, there are a few movies that we have exclusively in digital format. I also just got married in November, and I have all of our wedding movies on Plex, including all the raw files. Those were all digital downloads, but I have no less than 3 copies of those files, including on Google Drive lol.
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
My biggest beef with FreeNAS is the inability to expand the array. Is there no possible way to add on more drives incrementally? I know you can replace existing drives with larger drives, but you need to replace at least 2 of them to see any increase in storage capacity. So at that point on a small Z1 array you might as well just create a whole new Z1 array and have tons of extra capacity. Not the end of the world I suppose, but it could start to get excessive on hardware demands (new case, new PSU, etc.).
This is the case with every 'RAID' solution. Either you don't redistribute the data, so you add arrays (pools in ZFS-speak) or you just deal with lower performance envelope with adding a disk like Unraid does, or you take the time to redistribute the data (I'm not aware of an automated process). The manual process is to move the data around but to do that you obviously need more space than data.

Now I get what you're saying about the discs. So yes, I would obviously still have the backups of many many movies and TV shows on the physical discs. But there's much more involved in creating those files. My first test rip was Deadpool in 4k using MakeMKV. If you've ever used MakeMKV, it doesn't technically bypass DRM, it merely gets you access to the files on the disc. From there you need to locate the 1 file that contains the actual movie, copy it out, and then populate the metadata on your own. Its a bit of work sometimes. A newer movie like Deadpool isn't so hard to find metadata for, and I have a license for a program that searches for the metadata on a few different servers then auto-writes it to the MKV. Some of my older movies or "cheaper" movies ala Amazon sketch merchants are harder to locate the data for. Then there's matching that metadata to Plex metadata, which isn't always the most seamless process. So while I technically have backups for these discs, it's still quite a process to get them from physical disc to Plex entry.
It's mostly that if the array fails (or worse), that data is recoverable. Thus backing it up would be a function of availability; at worst, you'd just need to get another copy. It's the unrecoverable, say personal or family stuff, that I'd want backed up.
I still don't really get the idea of a snapshot. I mean, I understand what you said mostly, but I don't understand how a "backup slice" of data doesn't require storage for itself. Even compressed data still takes up a lot of space.
A snapshot is just a backup, whether that be a full backup or an incremental backup. Main thing is that the ZFS version is built in, can be sent to another location, and can be encrypted. All of that being part of the filesystem itself is an advantage.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
This is the case with every 'RAID' solution. Either you don't redistribute the data, so you add arrays (pools in ZFS-speak) or you just deal with lower performance envelope with adding a disk like Unraid does, or you take the time to redistribute the data (I'm not aware of an automated process). The manual process is to move the data around but to do that you obviously need more space than data.


It's mostly that if the array fails (or worse), that data is recoverable. Thus backing it up would be a function of availability; at worst, you'd just need to get another copy. It's the unrecoverable, say personal or family stuff, that I'd want backed up.

A snapshot is just a backup, whether that be a full backup or an incremental backup. Main thing is that the ZFS version is built in, can be sent to another location, and can be encrypted. All of that being part of the filesystem itself is an advantage.
Yeah, that's why I've been leaning towards Unraid. It doesn't seem like the performance hit would be enough to affect me at all, so then it just comes down to data integrity.

On another note, I was reading a little over on the FreeNAS forum, and I found this post that seems to state that running FreeNAS in a VM is a horrible idea because shit hits the fan the second something goes wrong with the pool. If the VM part of running either FreeNAS or Unraid is going to be a potential nightmare, there's no way I'm going to bother with VMs at all.
https://www.ixsystems.com/community...nas-in-production-as-a-virtual-machine.12484/

If the above is true of VMs, then that only really leaves me with Drivepool+Snapraid as my solution...or perhaps just using Windows raid 5 if you think that's actually reliable enough. I don't know anything about windows raid other than what Ive read about recently which is largely negative, especially when it concerns upgrading to the latest build of Windows. No matter the solution I end up choosing, it should work on this new HBA card too. While I can technically put these WD Golds on the board now, I may not be able to later if I expand.

So snapshot features are really only a software integration feature? It does nothing to eliminate or reduce the additional physical storage requirements needed for the backup? Like Windows restore point – you can backup to any given point, but you need to partition the drive to store said restore points?
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
On another note, I was reading a little over on the FreeNAS forum, and I found this post that seems to state that running FreeNAS in a VM is a horrible idea because shit hits the fan the second something goes wrong with the pool. If the VM part of running either FreeNAS or Unraid is going to be a potential nightmare, there's no way I'm going to bother with VMs at all.
https://www.ixsystems.com/community...nas-in-production-as-a-virtual-machine.12484/
Something goes wrong with the pool -- which means a drive failing -- is going to be a problem regardless of where the OS is hosted.

Now, the follow-on link for smaller setups (like yours, sort of), is here: https://www.ixsystems.com/community...ative-for-those-seeking-virtualization.26095/
That's in the first post, so I assume you read it.

Now, disregard: these guys are assuming that their audience is going to make 'virtual' drives on an already running RAID array and share those to FreeNAS. Essentially, they're interested more in the capability of FreeNAS as a file-server than they are of the ability of ZFS to keep data safe.

This is not what I'm suggesting you do. I'm suggesting that you run the FreeNAS VM wherever / however, but pass the storage drives to the VM and let it manage them directly. Yes, FreeNAS would be virtualized, but ZFS will have direct access to the drives and will behave no differently than if FreeNAS were running directly on the hardware in question.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Something goes wrong with the pool -- which means a drive failing -- is going to be a problem regardless of where the OS is hosted.

Now, the follow-on link for smaller setups (like yours, sort of), is here: https://www.ixsystems.com/community...ative-for-those-seeking-virtualization.26095/
That's in the first post, so I assume you read it.

Now, disregard: these guys are assuming that their audience is going to make 'virtual' drives on an already running RAID array and share those to FreeNAS. Essentially, they're interested more in the capability of FreeNAS as a file-server than they are of the ability of ZFS to keep data safe.

This is not what I'm suggesting you do. I'm suggesting that you run the FreeNAS VM wherever / however, but pass the storage drives to the VM and let it manage them directly. Yes, FreeNAS would be virtualized, but ZFS will have direct access to the drives and will behave no differently than if FreeNAS were running directly on the hardware in question.
I did see the other post linked in the first paragraph, but both posts are way above my head at this point... as is a lot of what you're saying.

I don't particularly care about what controls the drives and redundancy, so long as I have access to them in Windows and can manage the files on them in Windows. If I have to do the initial installation in a VM of freenas and handle drive replacement in that environment, so be it. I just don't want day to day file management in the VM. My Plex installation is on windows, my ripping software packages are on windows, metadata writing software is windows, etc., so controlling and managing the data in Windows is a must. Similarly, since i use my 2060 KO for Plex transcoding, that also needs to be able to access the data without trouble, whether that data is in a virtual disk or not.

If this can all be done in a VM, great. I just need to know how to configure it, because it sounds to me like there's multiple ways and not all ways are going to work for my use case. I'm getting pretty lost with the terminology, so I'm likely not differentiating the various configurations properly when reading about the stuff.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
So I finally decided to try setting up a VM with FreeNAS...not fucking working right at all. This is why I hate doing this shit, I always run into issues, always.

I went with Hyper-V, got that configured in Windows 10 fine. Got FreeNAS installation media setup with only a little frustration, thanks to thumb drives being a POS. Started runnning through the setup of the VM, and got to the point of launch and it refused to start. I kept getting errors about being unable to make checkpoints. So I rebooted, and it started fine (for the time being).

I ran through the FreeNAS setup, which went smooth. It kicked out a boot IP that was clearly from my VPN server IP, so I reconfigured it to 192.168.1.150 in the FreeNAS command prompt window in the VM. The new IP seemed to configure properly, but I cannot access the Web GUI to save my life. I tried the old IP, new IP, resetting all configurations...nothing works. The VM's network is configured in Hyper-V with "Default Adapter", as that was the only option besides no connection at all. Went to the network adapter settings in Windows 10 and set the ones labelled Virtual or VM to the 192.168.1.150. Also configured the DNS to match my router's configured DNS (1.1.1.1). WTF am I missing here!? Oh and any time I stop the VM, it won't restart without me rebooting the host machine, because of that stupid checkpoint issue. So I just turned off checkpoints for now so I can actually use the VM, but obviously that's not a long term solution.
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
Sounds like some kind of nat or routing issue (wrong gateway in vm or host network adaptor config?), but I havent used hyperv so I'm not sure how that works.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Sounds like some kind of nat or routing issue (wrong gateway in vm or host network adaptor config?), but I havent used hyperv so I'm not sure how that works.
There's nothing to configure in Hyper V for networking really. All the configuration I did in the VM was via the FreeNAS management interface. Alls I did was reconfigure IPv4, per numerous YouTube videos. I don't get why everything works fine for them, but when I follow their exact steps, shit blows up.
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
Add a new network to Hyper-V. You should be able to share it with your PC, such that they're on the same subnet.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Add a new network to Hyper-V. You should be able to share it with your PC, such that they're on the same subnet.
Why wasn't this included in any of the tutorials I watched about setting up a VM in Hyper-V? Ugh. I'll give this a shot in a few minutes.

When you say "share it with your PC", what does that mean? Where would I be going to share it? In Network adapters? Or in Hyper-V settings?
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Yeah, you can have it share the computer's adapter.
OK, I finally got the network working, sort of. I added a new Virtual LAN in Hyper-V. When I tried to add a second network switch for the Wi-Fi, per a TenForums Youtube video, I got an error stating something about a critical failure. So it looks like I cannot use WLAN via my host, which isn't normally a concern as the host is hardwired anyway. But there have been times where I needed to switch to wireless for one reason or another.

Still not sure how to fix the issue with checkpoints
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
OK, I finally got the network working, sort of. I added a new Virtual LAN in Hyper-V. When I tried to add a second network switch for the Wi-Fi, per a TenForums Youtube video, I got an error stating something about a critical failure. So it looks like I cannot use WLAN via my host, which isn't normally a concern as the host is hardwired anyway. But there have been times where I needed to switch to wireless for one reason or another.

Still not sure how to fix the issue with checkpoints
You could create a bridge and connect both the virtual network adapter and your wlan adapter to it, but I think it's difficult, not supported, and you lose some functionality. I tried it myself once when I was fooling around with a VM. I think I eventually succeded, but I don't remember what I did. I was following some written guides and documentation I found online, piecing together what I could to get it to work.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
You could create a bridge and connect both the virtual network adapter and your wlan adapter to it, but I think it's difficult, not supported, and you lose some functionality. I tried it myself once when I was fooling around with a VM. I think I eventually succeded, but I don't remember what I did. I was following some written guides and documentation I found online, piecing together what I could to get it to work.
Well how in the F did this guy do it with ease? I followed every single thing he did step by step. This was just in Hyper-V settings, so it really has nothing to do with FreeNAS until you assign the switch to the FreeNAS VM. But i couldn't even get the switch created in the first place. Whatever, screw it. It's not needed 99% of the time, and I'm sure I can live without Plex the other 1%.

Now i just need to figure out the checkpoint issue. Do either of you have any clue what could cause this? This was like right out of the gate during VM setup, before even running the FreeNAS installer in the VM. So seems to be a Hyper-V issue. See attached errors. Sorry for the ghetto screenshot, it was just easier that way at the time.

IMG_20200628_000901.jpg
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
Well how in the F did this guy do it with ease? I followed every single thing he did step by step. This was just in Hyper-V settings, so it really has nothing to do with FreeNAS until you assign the switch to the FreeNAS VM. But i couldn't even get the switch created in the first place. Whatever, screw it. It's not needed 99% of the time, and I'm sure I can live without Plex the other 1%.

Now i just need to figure out the checkpoint issue. Do either of you have any clue what could cause this? This was like right out of the gate during VM setup, before even running the FreeNAS installer in the VM. So seems to be a Hyper-V issue. See attached errors. Sorry for the ghetto screenshot, it was just easier that way at the time.

View attachment 257185
Dunno, everything I did to get it to work (aside from connecting in the VM) was in a command prompt on the host. It was not easy, but maybe it could have been if I had better documentation.

As far as the checkpoints, the last message in that window is what you're looking for. Looks like checkpoints aren't supported with passthrough disks (physical disks directly controlled by the VM). I imagine it's something they implemented in the disk format they use for their images, and can't be used if the guest OS is controlling the disks directly.

You'd have to either format the disk with a filesystem in the host and create a disk image which can expand to use all the space (this would allow you to use snapshotting in hyperv), or utilize a snapshotting filesystem or backup solution inside the guest OS.
 

IdiotInCharge

[H]F Junkie
Joined
Jun 13, 2003
Messages
14,486
Oh, turn off checkpoints. Useless if you're not running a Windows VM, IMO.

And bridging works well, it just adds another adapter as the Hyper-V bridge.
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
I should mention I was using win7 at the time. I think they've finally made some of those features (bridging, etc) more integrated and accessible from explorer in win10. There are still some configuration hidden from the UI, I'm sure, but the basic stuff works.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Dunno, everything I did to get it to work (aside from connecting in the VM) was in a command prompt on the host. It was not easy, but maybe it could have been if I had better documentation.

As far as the checkpoints, the last message in that window is what you're looking for. Looks like checkpoints aren't supported with passthrough disks (physical disks directly controlled by the VM). I imagine it's something they implemented in the disk format they use for their images, and can't be used if the guest OS is controlling the disks directly.

You'd have to either format the disk with a filesystem in the host and create a disk image which can expand to use all the space (this would allow you to use snapshotting in hyperv), or utilize a snapshotting filesystem or backup solution inside the guest OS.
Only the WDs are passed through. The SSD housing the OS is a virtual disk in the VM.
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
Only the WDs are passed through. The SSD housing the OS is a virtual disk in the VM.
Yeah, I imagine their thinking was "if something changed on the snapshotted disk which depended on something existing (or not) in the physical disk, then bad things will happen if a snapshot is used. Better to disable it altogether if physical disks are present."

And they wouldn't be wrong. But it would be nice if you could override in case you knew it would be fine.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Yeah, I imagine their thinking was "if something changed on the snapshotted disk which depended on something existing (or not) in the physical disk, then bad things will happen if a snapshot is used. Better to disable it altogether if physical disks are present."

And they wouldn't be wrong. But it would be nice if you could override in case you knew it would be fine.
So there's no way to backup the VM?

BTW, I am trying to get these drives setup, but I cannot get through permissions in Windows. I've followed numerous tutorials for setting up guest users/groups and setting up SMB share in FreenNAS, but no matter what I do I still get zero access in Windows. I see the FreeNAS network location, but I cannot even open it. My windows account is an admin too. So confused here. God I hate networking crap.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Looks like the permissions setting have changed in a recent version of FreeNAS, so I had to change the folders for permissions in order for it to work. Seems to be good now. I just added an MKV and directed Plex to it. It is playing perfectly fine.

I noticed some issues in FreeNAS that need to change, like S.M.A.R.T. service is not enabled and I can't turn it on. So I'm going to tweak some of that stuff now.
 

Nobu

Supreme [H]ardness
Joined
Jun 7, 2007
Messages
4,314
For backup, you could do it the old fashioned way: copy disk.img to disk.img.bkup. If you want to backup more frequently, you could create a binary diff file occasionally (using your monthly backup as a base).

That should be fine for the OS disk, since it shouldn't be changing much anyway. For the media disks, you'll have to ask somebody else–I'm not sure the best way to handle backup of large media libraries.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
For backup, you could do it the old fashioned way: copy disk.img to disk.img.bkup. If you want to backup more frequently, you could create a binary diff file occasionally (using your monthly backup as a base).

That should be fine for the OS disk, since it shouldn't be changing much anyway. For the media disks, you'll have to ask somebody else–I'm not sure the best way to handle backup of large media libraries.
Im not really looking to configure a backup. I just want all the checks/monitoring ZFS has to offer to warn me about impending failures. I was watching Chris Titus's video on configuring S.M.A.R.T. tests and Scrub tasks, and I was able to configure them all, except Scrub tasks on the SSD (the SSD doesn't come up as an option). But I cant manually start the S.M.A.R.T. service. I wanted to run a short test on all HDDs, but it fails to start. I'd also like to get scrub tests configured on the SSD, but I have no idea why I can't see that drive as an option.
 

fatryan

[H]ard|Gawd
Joined
Feb 19, 2004
Messages
1,315
Well I tried following the steps below, which match the steps in a YouTube video by a Microsoft engineer. Of course it didn't work, cause why would it work? I couldn't even dismount the HBA. It gave me some BS about the OS not being able to manage it and needing to go into the BIOS or UEFI, but I never even configured this card in the BIOS/UEFI. SO I don't know what the hell it's talking about.

https://devblogs.microsoft.com/scri...er-v-vms-by-using-discrete-device-assignment/

I've about had it with this shit though. If I can't get FreeNAS setup to properly run and protect my drives and their data, whats the point? I might as well just use these in a 36TB non-redundant Drivepool if I'm just going to lose all my data anyway...
 
Top