Oh no! Gea, what happened!? 2 replication jobs have deleted all my target snaps and all but one of my source snaps and I think forced me to reseed!! I think maybe given the fragility of WAN replication and the potential for jobs to fail zfs destroy should only fire after replication completes...
bummer, I think WAN link reliability is going to be an issue. Yesterday's manual 152GB send dropped 129GB into it last night. Do we have anything like zfs send --resume in the nappit replication?
And is this normal? Kicked off my nappit replication this morning. Logging these every 15...
Gea, Thankyou ! I'm sure that was getting annoying! It was an http-proxy inspector on our WATCHGUARD set to deny unknown web request methods. Here's a list of the accepted methods. I screen-captured a minilog thought I saw it was a POST request. Know why it might have been failing?
HEAD
GET...
If I understand it correctly netcat is working. It's how I am manually replicating for the time being. Seems like its just the appliance-group recognition piece that's having trouble. I *think* I've got the firewalls passing the traffic and bypassing the IPS, etc.
See the nc send/receive...
Alright! Got my first 92GBs snapshots from Friday manually WAN replicated! Looks like there are 200GBs more as of this morning (Sunday) which means that at ~20Mbps I should just be caught up before the doors open Monday. Phew.
Had to wake up at 3 in the morning to kick off my 2nd stream so...
Good question! But yes, in both directions.
root@nappit0:~# ping nappit1
nappit1 is alive
root@nappit0:~# nc -vzu nappit1 81
Connection to nappit1 81 port [udp/*] succeeded!
root@nappit0:~# nc -vz nappit1 81
Connection to nappit1 81 port [tcp/*] succeeded!
root@nappit1:~# ping...
seems like it's open, see the port scans in both directions below. I clicked the ZFS link for the appliance in the t-AG and see this corresponding output in the jobs monitor. Anything in the fact that the host_ip=nappit1? By the way, there is no question of me buying the extension on Tuesday. No...
I seem to have a handful of issues. The big one is my target-side firewall [sonicwall TZ210] can't seem to handle the traffic. CPU spikes to 100% from about 5% as soon as I kick off a manual replication and latency goes from <30ms to >2000ms!
During a manual remote replication, the target...
hmm, maybe that's not it. ICMP Traceroute completes between both nappit boxes... Do the hosts "register" any Route/MAC or IP info about each other when the appliance group is first established?
81 is open - ran a netcat port scan! handy little feature! - and I've confirmed all traffic is allowed between hosts. I think I've got a host routing issue but I'm not sure what it is yet. Both hosts can ping one another, resolve eachother's hostname with an answer from our domain dns box and...
Thanks Gea.
Source IP unchanged. Target IP has changed. Current source appliance status (as read in target appliance-group) is 'remote call: timeout.' manual nc send/receive test was succesful across vpn.
Confirm delete source from appliance-group and re-add?
I tried to add before...
I'm moving my target replication node tomorrow to our VPN-connected satellite office and will have to change the target-node’s IP address for the new local subnet. Will the existing Napp-it replication extension jobs continue to run?
thanks,
jb
I expected to be network limited based on my Bonnie scores for sequential reads and writes and thought I might be able to increase total throughput by using two source/target NIC-pairs. Disk utilization during replication in the source pool is around 40%, 15% on the target. Processor utilization...
Hi Gea, If source and target have multiple network adapters, can nappit rep take advantage of them and run multiple concurrent replication jobs on discrete network paths?
10.10.2.201->10.10.2.211
10.10.2.202->10.20.2.212
thanks,
jb
gave it the old college try but I'm just gonna reseed over the weekend. Does it look like I got close?
zfs rename -r r10/B1@daily-1365112602_2013.04.19.11.51.24 1366409769_repli_zfs_nappit1_nr_1
- The extension has its own snap naming. While you may rename your snaps its easier to do a new initial sync with deleting or renaming target ZFS
Found the rename command: zfs rename tank/home/cindys@083006 today
Is it just a matter of changing to a supported name? I'm sure it's more...
I've alreay completed an initial seed replication and some incremental replications of two ~2TB iSCSI zvols. Now I'd like to try the replication extension as an easy way to automate ongoing remote replication.
1) Can I start with the seed already in place or does the extension need to...
Well, I didn't actually know about ipadm (on Solaris-based OSes.) I used ifconfig to set up my nics. I *think* if you used the nappit gui to setup network adapters he calls ipadm so you wont have the problem.
Reconfigured w/ ipadm using the instructions here and after a reboot I was full...
Interesting! Significant performance boost after reconfiguring network with ipadm. IO is all ARC driven in this test but what's important is I'm finally pegged out on the nics!
SERVER TYPE: Dell PowerEdge 2900, 48Gb ram
CPU TYPE / NUMBER: 2x Xeon E5410 2.33GHz
STORAGE: 6...
Ah, System-Network info includes values from ipadm? I used ifconfig for everything and now I think that's why one nic doesn't retain it's address on reboot!
Just cleaned out old interfaces. There is one disabled address e1000g0/v4 that seems to be populating the value on the nappit screen...
How does nappit get it System->Network info? It's different on 1 of 2 boxes from what ifconfig -a and/or dladm show-link return. But I happen to have network related IO performance issues on the box where napp-it shows it wrong so I'm wondering if that's a clue to the cause.
thanks,
jb
nevermind, since I don't actually need them mounted on Box2, I think I can just destroy/create my Box1 pool and then clone from the snaps from the iscsi mount. Though I am academically curious how one might access the snaps on Box2 were it necessary ?
I'm coverting my z2 pool to a 3-vdev r10 pool to improve non-sequetial performance. But I have production backup data on two volumes that I don't want to destory. I have an empty pool on a second OI box Box2, so I thought I'd replicate my backup data to Box2, then destroy and create volumes on...
OK - I think I understand. And if I've got this right I appreciate the fact that noone got on and called me a stupid noobie and i should do some reading before wasting peoples time with questions that have already been answered 1000 times.
My pool is Raid Z2 on a single 6 disk vdev. So the...
False positive.
Now the slow non-sequential performance is jumping around to all 3 volumes in the pool so at least it's consistent. I appear to be able to get as high as 95MBps in the non-sequential tests but frequently as low as 1MBps. Scrub is clean. I've turned off dedup. Arc REAL hit...
This appears to have been the result of running:
svcadm disable network/physical:nwam
without the corresponding
svcadm enable network/physical:default
I'd be interested if anyone understands why that config mistake would produce such a narrowly limited effect. Or are the effects of that...
port group: meant lag...
And then what was your test client(s) - or put another way, are you using multiple adapters on both sides of the test? What is the networking between client and storage? Assuming it's a LAN client, I think I've read where you can link two clients into a single IOmeter...
I just grouped two broadcom nics on a standalone OI box. I could help you with the OI commands - basically, google open solaris dladm create-aggr - but I think you've got more pieces to fit together in the all-in-one setup, right? And did you set a corresponding port group on your switch? And...
I setup link aggregation and seem to approach 200MBps on some sequential read tests from IOmeter so I gather that's the better approach. "dladm" will be a useful search string for the next guy looking to set this up. You'll also want to disable NWAM and set your aggr policy to L2,L3 - it...
I took some old hardware and setup OI Napp-it as an iSCSI target to my ESXi5.1 environment. I'm using it as the backup store for Vsphere Data Protection and it's working a treat! I'd like to add two more features to the solution.
1) I've got a second nic I'd like to utilize for iSCSI. Do I...
IO Meter test is run from a ESXi 5.1 guest w an OI hosted iSCSI target. OI server specs are as follows: 6 2TB 7.2K Discs / 48gb RAM, 2x 2.33 Ghz Quad-core processor, 1x1Gb iSCSI NIC.
I think when I figure out how to multipath a second NIC the OI box may dominate conclusively. But the iops and...
Hi,
With a good deal of help from threads on here I have repurposed a couple of old servers as backup storage for my vmware environment. Only have a single 1Gb NIC on each box configured for iSCSI at the moment. Getting good enough results for backups in IOMeter benchmarks but do appear to be...