• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

AES-NI accelerated Solaris 11 ZFS encryption on Xeon E3 CPUs inside a VM (ESXi)

M

meatincereal

Weaksauce
Joined
Jul 17, 2011
Messages
83
Has anyone gotten it to work properly?

I'm getting terrible results when benching my encrypted folder on my 10-disk RAIDZ2 of Samsung F4EG drives on Solaris 11.11.11, with 2 cores assigned to the VM on my Intel Xeon E3-1235:

Code: 
root@odin:/mypool/storage# time dd if=/dev/zero of=/mypool/storage/dd.tst bs=1024000 count=10000
10000+0 records in
10000+0 records out

real    1m52.604s
user    0m0.010s
sys     0m2.715s
root@odin:/mypool/storage# time dd if=/mypool/storage/dd.tst of=/dev/null bs=1024000
10000+0 records in
10000+0 records out

real    3m15.547s
user    0m0.015s
sys     0m4.768s

Which gives us:
Write: 90.94 MB/s
Read: 52.366 MB/s

This is with the CPU maxed at 100% throughout.

Non-encrypted I get 600ish MB/s write and 900ish MB/s read.

prtdiag shows "Pentium(R) Pro", maybe the CPU and its features is not recognized properly?
Code: 
root@odin:/mypool/storage# prtdiag
System Configuration: VMware, Inc. VMware Virtual Platform
BIOS Configuration: Phoenix Technologies LTD 6.00 01/07/2011

==== Processor Sockets ====================================

Version                          Location Tag
-------------------------------- --------------------------
Pentium(R) Pro                   CPU socket #0
Pentium(R) Pro                   CPU socket #1

Could the hypervisor be the culprit? I figured I'd ask here first to see if anyone knows, before formatting everything and trying Solaris bare metal. (Has anyone else tried Solaris 11 encryption bare metal?)

NOTE: Can't get VMware tools to work in Solaris 11.
NOTE2: The disks themselves are "bare metal", as in they are connected to controllers passed through to the VM (vt-d).
 
Nope, it doesn't work. I'm hoping someone will find a fix for it soon.

You can see if AES-NI is getting passed through to the guest by typing 'isainfo -v' in a Solaris console and look for 'aes'. If you see it, then Solaris should know that your CPU has these instructions.
 
I'm gen or two back (Xeon E5620 in a Super Micro X8DT6) on my processors but shows up for me on 11.11.11:

Code: 
tester@solaris11test:~$ isainfo -v | grep -i aes
        pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp ahf cx16 sse3 sse2 sse 
        pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp ahf cx16 sse3 sse2 sse 

tester@solaris11test:~$ prtdiag
System Configuration: VMware, Inc. VMware Virtual Platform
BIOS Configuration: Phoenix Technologies LTD 6.00 01/07/2011

==== Processor Sockets ====================================

Version                          Location Tag
-------------------------------- --------------------------
Pentium(R) Pro                   CPU socket #0

I haven't moved my real disks over from S11 express yet so I can't easily test encryption performance to see if it's used.
 
They hypervisor will pass the AES-NI instruction set (as long as you're not hiding it for vMotion CPU compatability) down to the VM. I've tested this with ESXi and a Linux VM.
 
Yep, there's not an issue with VMWare, there's an issue with Solaris.

firebug@fireserver:~# isainfo -v | grep -i aes
xsave pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp cx16 sse3 sse2 sse
xsave pclmulqdq aes sse4.2 sse4.1 ssse3 popcnt tscp ahf cx16 sse3 sse2
Click to expand...

But.... it's easy to see in benchmarks that aes-NI isn't getting used - on my desktop with a Core I5 I can push as high as 1.2G/sec through AES w/truecrypt.... on Solaris, about 70MB/sec :( When I upgraded from a Athlon X4 to a Xeon E3 my encrypted performance actually went down :(
 
Solaris 11.11.11, on a Core I5-2500K (hardware AES-NI), in a VM - in summary, AES-NI didn't work here either, even though it's detected. Weirder and weirder.

root@solaris:~# openssl speed -engine pkcs11 -evp aes-128-cbc
engine "pkcs11" set.
Doing aes-128-cbc for 3s on 16 size blocks: 31299847 aes-128-cbc's in 2.99s
Doing aes-128-cbc for 3s on 64 size blocks: 19342825 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 7658270 aes-128-cbc's in 2.99s
Doing aes-128-cbc for 3s on 1024 size blocks: 2239061 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 294453 aes-128-cbc's in 3.00s
OpenSSL 1.0.0e 6 Sep 2011
built on: date not available
options:bn(64,64) md2(int) rc4(1x,char) des(ptr,cisc,16,int) aes(partial) blowfi sh(ptr)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 167490.82k 412646.93k 655691.34k 764266.15k 804052.99k
root@solaris:~# LD_HWCAP="-aes" openssl speed -engine pkcs11 -evp aes-128-cbc
engine "pkcs11" set.
Doing aes-128-cbc for 3s on 16 size blocks: 31226973 aes-128-cbc's in 2.99s
Doing aes-128-cbc for 3s on 64 size blocks: 19214598 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 256 size blocks: 7634700 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 1024 size blocks: 2237888 aes-128-cbc's in 3.00s
Doing aes-128-cbc for 3s on 8192 size blocks: 294520 aes-128-cbc's in 3.00s
OpenSSL 1.0.0e 6 Sep 2011
built on: date not available
options:bn(64,64) md2(int) rc4(1x,char) des(ptr,cisc,16,int) aes(partial) blowfi sh(ptr)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192 bytes
aes-128-cbc 167100.86k 409911.42k 651494.40k 763865.77k 804235.95k
root@solaris:~# openssl engine
(aesni) Intel AES-NI engine
(dynamic) Dynamic engine loading support
(pkcs11) PKCS #11 engine support
root@solaris:~#
Click to expand...

Next, I tried a zfs encrypted volume - 50MB/sec encrypted, 120MB/sec unencrypted. I tried `export LD_HWCAP="-aes"` but that didn't make any difference if it was enabled or disabled, still 50MB/sec.

The big hit in performance between encrypted and unencrypted on the I5-2500K makes me think it's not working here either.... and since the Xeon E3 and the Sandy Bridge processors are pretty similar, I'm not all that surprised. Need someone with a different Xeon to check it out and report back.

For refernce, the 2500K w/Truecrypt:

 
Well that's the final word I guess. Solaris has no AES-NI acceleration for ZFS. Not yet at least.

Tempted to go create a thread on Oracle support asking about it.
 
You must log in or register to reply here.
Back
Top