I humbly approach for FreeNAS problem

I said it before - "ZFS is *NOT* a backup of your data!".

Before doing *ANYTHING* backup the data to storage that isn't connected to FreeNAS.

Even if you screw up and blow up the pools so you have to redo your shares/permissions/jails/etc. at least the data is intact.

Since you are running 4x 1TB in RAID5-style (I assume it is RAIDZ1) you have max 3TB of data to back up. Order an extra 4TB hard drive to act as the backup along with a USB HDD dock. Copy everything over to the 4TB drive.

When you are finished keep that extra 4TB drive around as a cold spare.

If you are flush buy a pair of 8TB drives, that way they are big enough to backup your new pool (assuming you are going RAIDZ2 this time).

I always have on hand enough storage to backup all my pools, I usually buy the drive size I intend to upgrade the pools to so that when I'm ready to upgrade I buy the remaining drives I need and then the new backup drives.

I do this because I prefer to spread my pool drive purchases over time - I hate it when all the drives wear out simultaneously.

Are you keeping the old pool?
I understand about the backup. Seems like sage advice, but I'm tapped as to resources I'm willing to donate to this project.

If I can think of a viable use for the other pool (or, more accurately, the drives that were the other pool) I'd like to use it. The primary purpose of the machine is to be a Plex server. It's a media repository.

I'm not sure how smart Plex is about looking in multiple places.
 
That looks like the best method to migrate filesystems, though I haven't tried it exactly. Note that it works at the zfs, not zpool, level. IOW, first you'll need to configure your destination pool manually. This also means that VDEV type is irrelevant here.

And that last part's good, because most people will say you want RAID-Z2 at this point. I'd agree.


At the very least, use a couple of the new 4TBs to make a temp backup. Probably best to commit to using mirrors for new pool, so you can easily add those drives later.

There's nothing wrong with later using each 1TB to backup a portion of the larger pool, other than inconvenience.


How many simultaneous streams? If it's just you, 6x 4TB in Z2 gives excellent space, safety & sequential bandwidth. For multiple streams (or more IOPs for non-media use), multiple VDEVs are better, so mirrors here.

I think I have a maximum of two network and one out of network streams. Even one stream, though...I have three different bitrates in my media. 720p, 1080p, and some (limited) 4k. It can be significant. I am in the process of running cable in my house now.
 
I want to take a second to thank you all for your guidance. It's very helpful for me. It is appreciated.
 
I understand about the backup. Seems like sage advice, but I'm tapped as to resources I'm willing to donate to this project.

If I can think of a viable use for the other pool (or, more accurately, the drives that were the other pool) I'd like to use it. The primary purpose of the machine is to be a Plex server. It's a media repository.

I'm not sure how smart Plex is about looking in multiple places.

Worse-case scenario you set up Plex with multiple libraries, then partition your data accordingly eg. MP3's on the old pool and video on the new one.

As for the backup, I'd set up 5x 4TB in RAIDZ2 (RAIDZ1 should be avoided as it take only 2 bad drives to wipe out your pool - that sounds unlikely but what happens if you lose 1x drive, then a second one goes bad during the resilvering?) and use the 6th 4TB drive for the backup.

Remember that there are only 2 types of people out there: those who needed a backup and those who will need a backup.
 
Well. I got boned.

I switched some drives around for better rack organization, then all 6 new 4 tb drives were detected (built a raidz2 pool from them) but not all the six 1 the drives were detected, including at least one or more of the original 4. Tried to transfer the data from the remaining drives to the new volume in the hopes only one drive was absent and it could manage - it couldn't. So now I'm going to screw with it again this evening. 2 a.m. nights messing with this crap is killing me.
 
No one ever listens to me...
I listened! I just was gonna do the backup on a new volume inside the machine. New share. New config.

All I did was move drives around. And then shit hit the fan. I think I can fix it by shuffling hdds around. I have 6 drives, four have data on them. I just need to get them in working slots.
 
Last edited:
No one ever listens to me...
Incidentally, I found what I was he problem was. A weird issue. One of my hdds was bad, not one with data on it. I have 8 drives plugged into a HBA adapter (flashed RAID controller) and 4 plugged into onboard SATA. When I plugged this particular drive in, it made itself AND ANOTHER DRIVE not available. I have never seen that happen before. Removed the drive, everything working fine. Slot is empty until my data has been backed up.
 
Glad you got it running! That said, there's a lesson for other visitors here.

I listened! <snip>All I did was move drives around.
No, you didn't listen. He said:
Before doing *ANYTHING* backup the data to storage that isn't connected to FreeNAS.
Moral. Don't touch shit without a backup. You never know if something minor might knock your storage sideways. ALWAYS make the backup first.

There are 10 kinds of computer users: those who make regular backups and those who haven't lost data yet.
 
Glad you got it running! That said, there's a lesson for other visitors here.


No, you didn't listen. He said:

Moral. Don't touch shit without a backup. You never know if something minor might knock your storage sideways. ALWAYS make the backup first.

There are 10 kinds of computer users: those who make regular backups and those who haven't lost data yet.
I *did* listen. I just kinda...gambled...a little bit. I would not recommend the route I took for others. I was lucky, in many ways.

Once it becomes prudent to do so, I'll be setting a periodic backup to an off-system source. As it is now, I'm transferring data at a whopping 5 MB/second. It'll take a few days to transfer everything over. Ugh.
 
Well. I got boned.
I'm guessing you had the vdev setup with the drives from /dev/sda, /dev/sdb etc? The best practice is to use by-id instead. So your devices in the vdev are setup like /dev/disk/by-id/<disk-1-ID-Here> /dev/disk/by-id/<disk-2-ID-Here> etc. etc.

If you did mount by /dev/sd*, then it's likely when you moved the disks that their allocation changed which totally fucked your array.

Edit - stupid forum refresh, I didn't see the rest of the replies. Ignore the above :D
 
I *did* listen. I just kinda...gambled...a little bit. I would not recommend the route I took for others. I was lucky, in many ways.

Once it becomes prudent to do so, I'll be setting a periodic backup to an off-system source. As it is now, I'm transferring data at a whopping 5 MB/second. It'll take a few days to transfer everything over. Ugh.

Oh, so it is worse - you listened, then *IGNORED* (what I consider) sound advice.

I specifically said backup to another storage device *NOT* on FreeNAS, which means you shouldn't have touched FreeNAS to put in the new storage beforehand. Using FreeNAS to back itself up is like making an arsonist the fire chief - you are just asking for trouble. What if FreeNAS shit the bed completely? Then both the primary *AND* backup data stores are toast.

I hope you learned something, never gamble with anything you can't live without (data, money, life, etc.).
 
Oh, so it is worse - you listened, then *IGNORED* (what I consider) sound advice.

I specifically said backup to another storage device *NOT* on FreeNAS, which means you shouldn't have touched FreeNAS to put in the new storage beforehand. Using FreeNAS to back itself up is like making an arsonist the fire chief - you are just asking for trouble. What if FreeNAS shit the bed completely? Then both the primary *AND* backup data stores are toast.

I hope you learned something, never gamble with anything you can't live without (data, money, life, etc.).
Ignored is such a strong word. So the event in question unfolded like this: I was out at the pub until too late, then i returned home and figured it was the perfect time to monkey with my FreeNAS box. So I cracked open a fresh Mountain Dew and went to town. Before I turned the first screw, I thought I should backup the data on an external HDD. But alas! I did not have one available and all the stores were closed. I let not the time of day (or night) delay the project, then everything fell apart. So really it was more like ambition that is to blame, here. Yeah. That's it.
 
  • Like
Reactions: Meeho
like this
Ignored is such a strong word. So the event in question unfolded like this: I was out at the pub until too late, then i returned home and figured it was the perfect time to monkey with my FreeNAS box. So I cracked open a fresh Mountain Dew and went to town. Before I turned the first screw, I thought I should backup the data on an external HDD. But alas! I did not have one available and all the stores were closed. I let not the time of day (or night) delay the project, then everything fell apart. So really it was more like ambition that is to blame, here. Yeah. That's it.

You sure it was a Mountain Dew you cracked open? :)
 
Just in case anyone comes across this thread at a later time, if you are migrating data like I am doing plan to accommodate a LOT of time for it. Even a modest 3 TB at 5ish MB/sec takes several days to transfer. 24 hour days. Like, four. So, might as well grab a Snickers - you're not going anywhere for a while.
 
Just in case anyone comes across this thread at a later time, if you are migrating data like I am doing plan to accommodate a LOT of time for it. Even a modest 3 TB at 5ish MB/sec takes several days to transfer. 24 hour days. Like, four. So, might as well grab a Snickers - you're not going anywhere for a while.
5MB/s is certainly not what you should be getting out of your array. Something is wrong.
 
5MB/s is certainly not what you should be getting out of your array. Something is wrong.
I am using a windows machine between the two. So, Array 1 through SMB share to Array 2 SMB share. It's coming from a RAIDZ1 going to a RAIDZ2.

Edit: should I do this another way?
Edit 2: I really hope there is a better way. I'm only 21% through transfer.
 
Last edited:
3:00 a.m. and many Mountain Dews later, I have done the following:

Cancelled the windows transfer. Because I'm trying to be [H}ard(er). Created a snapshot of the dataset on the old volume, then zfs send <snapshot> | zfs recv -F <destination> . I have no idea if the thing is working, but I'm toast. I've gotta be awake in 3.5 hours. If I am missing a step here, please let me know. It's not for lack of trying on my part. This is very, very alien to me.

Edit: Something is moving. Great. I'll make the new snapshot the rollback snap for the new target dataset in the morning - once it's done.
 
Last edited:
You can zfs send/receive, sure. If you're copying between two ZFS zpools though, just do a normal cp or mv between the two (depending on what you want to do -- copy or move).

Code:
cp /zpool1/mount/ /zfs2/mount

With the windows machine in the middle, you're stuck with the network send/receive in the middle -- that's adding unnecessary overhead.

When I move data between zpools, I saturate my 1gbps network and get most of the way to saturating my 10gbps network.
 
You can zfs send/receive, sure. If you're copying between two ZFS zpools though, just do a normal cp or mv between the two (depending on what you want to do -- copy or move).

Code:
cp /zpool1/mount/ /zfs2/mount

With the windows machine in the middle, you're stuck with the network send/receive in the middle -- that's adding unnecessary overhead.

When I move data between zpools, I saturate my 1gbps network and get most of the way to saturating my 10gbps network.
I started that then.aborted because I thought I messed up. Used exactly that command. Actually lost about .1 tb because of it. I'm not sure how to get at that again.
 
I started that then.aborted because I thought I messed up. Used exactly that command. Actually lost about .1 tb because of it. I'm not sure how to get at that again.
I... am not sure how copy (cp) would ever cause data to go missing. It's not removing the source data at all, just copying it to the destination specified. Sure you did cp and not mv?
 
I... am not sure how copy (cp) would ever cause data to go missing. It's not removing the source data at all, just copying it to the destination specified. Sure you did cp and not mv?
I didn't do -mv. I mean, I used space on the receiving array. It doesn't amount to much.
 
Back
Top