No TLER/ERC/CCTL on WD Gold 12TB + HGST Ultrastar 12TB :-/

Discussion in 'SSDs & Data Storage' started by maxxmusterr, Jan 31, 2018.

  1. maxxmusterr

    maxxmusterr n00b

    Messages:
    8
    Joined:
    Jan 31, 2018
    Hello guys,

    TL;DR - TLER/ERC/CCTL status uf some of the most used/most interesting 12TB HDDs you find in my post below (4th post).

    Well .... I really don't know what I can do now. I'm absolutely desperate...:cry:

    I got here 2 Areca ARC 1260 (with newest Fimware) and 1 ARC-1880(IX-12) (with newest Firmware).
    Plus brand new 7x HGST 12TB SATA Disks (HUH721212ALE600) and 1x WD Gold 12TB SATA.
    The controllers (all 3) show the following:

    [​IMG] (Screenshot of ARC 1880 with 1x HGST)

    No ERC/TLER/CCTL at all - Error Recovery Control (Read/Write) shows Disabled/Disabled.
    At least the WD Gold officialy lists TLER as a feature of their product, so it must support it.

    Btw. I also verified that again via 2 different PCs and with HBAs and smartctl (to check ERC status) under Linux - Same result - No ERC/TLER/CCTL on any of the disks.

    I discovered that by " accident" after i had random dropouts of 1 or 2 disks while building up the Volume or after that. I didn't belive my eyes...

    Can anybody help please? ....
     
    Last edited: Feb 2, 2018
  2. mwroobel

    mwroobel [H]ardness Supreme

    Messages:
    4,937
    Joined:
    Jul 24, 2008
    Unfortunately, this is what happens when you use devices not on the manufacturers Hardware Compatibility List. The Latest Areca HCL lists no tested 12TB drives. The HGST HCL only lists the 1882 & 1883 as tested Areca cards, not the 1880 or especially an older card like the 1260. When you venture into enterprise-level hardware, compatibility issues like this can pop up. Build an array of just the HGST and do some stress testing, and see if the drives drop out or become less stable but it is very possible that you may have issues. I realize this is not the answer you want, but enterprise-level hardware RAID cards can be very picky about the drives they are compatible with.

    Now, that being said, per the latest HGST US12 Tech Bundle it does fully support CCTL, but the card(s) you are using do not seem (in the current firmware) support setting the flags. Either call Areca or send their support people an email and see if they will possibly add your drive support to the next firmware release, but I do not know if they will entertain supporting new/updated features on older/obsolete cards. You may need to update to the 1882/1883 to fully support your drives.
     
    Last edited: Jan 31, 2018
  3. maxxmusterr

    maxxmusterr n00b

    Messages:
    8
    Joined:
    Jan 31, 2018
    Well yes, Areca has indeed not tested it, they also won't update that list anymore for these old controllers i guess.
    In the document you see on the smaller models of this series-> Tested by HGST -> And HGST says in their Compatibility document (in the fine print) that their tests just say that the drive was recognized, writing is possible etc. - just the basic stuff. Which also works here, even on the old Areca 1260 ... The question is just how long .I already had several random dropouts of the disks, 5 times or so. And on 4 times the "Media Error" counter was more than 0 (off course I changed all cables, the whole controller etc. to sort out any controller error). I know now for sure that it won't work together - but the cause would be interesting and is maybe a big thing, which soon will may hit other users.
    I mean you can grab these 1260 with 16 SATA ports and murder performance for not even 100 bucks and flawless driver support, even if the firmware does not get updated anymore. They are rocksolid. Have it runing since release of Server 2012 R2 24/7 with about 30 Disks (3 Controllers) and 100+TB storage.


    But as you sended me the pdf, which i found before... Where do you explicitly see that CCTL is supported?
    I honestly can't really see that confirmed there?


    But the point is:
    Let away all the Arecas completely You basically said that it's old stuff.... Okay agree.... But like I mentioned: I tested it on different HBAs (onboard ones) of 2 or 3 completely different PCs/machines (with fresh FreeNAS install and their smartctl).
    Also directly conntected to a HBA they have TLER/CCTL/ERC Disabled. Same story there. Btw.I also tested it with a harddrive i knew TLER is 100% there and it worked (just to be sure that the reported info is correct).
    So that's not really an Areca problem. It seems more that the HD's firmwares may now do some really strange stuff, which at least most normal/older controllers don't understand (anymore?). Not just my "old" Arecas.

    What is interesting: i got a few Seagate Ironwolfs 10TB hooked up on my Areca 1260 - There TLER/CCTL/ERC works without any problems (7.0 seconds).

    I mean .... if i really could be 100% sure that it will work with an 1883/1882 cool, probably would grab one, even when they are very expensive .... But nobody knows if it works there (like i mentioned the HGST testing is not really testing, unlike Areca which really do test the stuff, which they test themself) ... And then other people will also run into this ... and also people who would like to use TLER/CCTL/ERC on regular HBAs with for example FreeNAS/ZFS or so will have the exact same thing. And that would be a really big problem and would hit a huge mass of people.
    Maybe I am the first one... but for sure not the last one. So it would be great to find new inputs and thoughts of skilled people here, so that we can share our experience/findings and may find the cause/source of the error and make the people aware of it, that they may now can/must forget most older HBA+these 12TB HD's with enabled TLER/CCTL/ERC.
     
    Last edited: Jan 31, 2018
  4. maxxmusterr

    maxxmusterr n00b

    Messages:
    8
    Joined:
    Jan 31, 2018
    So ... I now also ordered also the 2 available SATA Seagates. So basically i now have all interesting available 12TB HDDs here (at the time of writing there is no WD Red 12TB out btw.).
    Since no one seems to have analyzed that so far (which was surprising) and this now seems to be the No1 Google find on 12TB plus TLER/CCTL/ERC I did it myself and wanted to post it for everyone:

    Conclusion/my findings:

    • HGST 12TB (HUH721212ALE600) (SATA) - NO TLER/CCTL/ERC (Disabled) - Not possible to Enable manually (permanently)
    • WD Gold 12TB (WD121KRYZ) (SATA) - NO TLER/CCTL/ERC (Disabled) - Not possible to Enable manually (permanently)
    • Seagate ST12000NM0007 12TB (a.k.a. Exos X12 / a.k.a. Enterprise Capacity 3.5 HDD 512e V.7) - TLER/CCTL/ERC ENABLED, but with 10.0 seconds (most traditional RAID controllers need 7.0 seconds or less) (btw. also not possible to change to 7.0 seconds permanetly)
    • Seagate Ironwolf 12TB (ST12000VN0007) - TLER ENALBED with 7.0 seconds (PERFECT!)- (Will work on nearly every RAID Controller without any problems)

    Like I've written already before:
    THIS is not just valid to the Areca 1260 / 1880 ... It also is true for random HBAs (onboard) which i tested a few (3 completely different machines, all a bit older ones... you know the stuff which is laying around)


    So i hope it helps at least you guys out there ...Unfortunately the shittiest HD of the above 4 (just 1Mio h MTBF, 180TB/year and 2 years warranty) is the only which will work in nearly every setup.
    The others woud offer 2.5Mio h MTBF, 550TB/year and 5 years warranty) -.- Fucking IT
     
    Last edited: Feb 2, 2018
  5. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    This is good information, thanks for posting. I have an Areca 1882ix-16 with 12 X 3TB HGST drives, and would like to upgrade by the usual copy to a second array built on the same card. Given that I am almost full and only have 4 slots left, I need to do it now. I would guess the HGST and Areca firmware eventually are fixed for compatibility, but that clearly has not helped you as an early adopter. I wonder if it is only the Seagate Iron Wold (ST12000VN0007) that has the proper 7-second TLER, or if the Iron Wolf Pro (ST12000NE0007) would work as well? As you mention, these are expensive gambles. And retail in general seems to be completely uncooperative about returns, even legitimate ones, of computer parts.

    I have long been a fan of Toshiba/Hitachi drives, but today they appear questionable. Reading the 12TB HGST datasheet they appear to support configuration of CCTL, but it looks like the details have been re-arranged as firmware versions have evolved. Unfortunately, even a "simple" compatibility problem like this is still a problem.
     
  6. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    I did put a question in to Areca support regarding this issue, but only crickets so far.
     
  7. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    I question if this statement is 100% accurate. Reading the description of HGST firmware updates they describe CCTL functionality in the latest release, although the command structure may not be identical to the older drives. Which to outward appearances, without updated RAID controller support, could look like "no support".

    Unfortunately when I asked Areca tech support I got the usual non-answer, something like it was on their list but they couldn't test every drive.
     
    Last edited: Jul 5, 2018
  8. maxxmusterr

    maxxmusterr n00b

    Messages:
    8
    Joined:
    Jan 31, 2018
    ... I agree - Could also be possible ... But honestly Arecas 1882 are still under support und I tested it with the current firmware...So it seems just not to work with the Arecas. And just to buy a LSI controller also is not a great idea. Especially since i don't even know if they show the TLER/CCTL/ERC thing. I never seen it at least.

    Btw. Could you provide the link with the mention of CCTL?
     
  9. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    Here is the HGST page for the 12TB drives. They have an interesting article on RAID "rebuild assist recovery". There is also a note on the new power disable pin feature. Not sure my old Norco 4220 would be compatible, but they still sell the drive without this feature for legacy chassis.

    I looked for the description of CCTL behavior, and found it in the "Hard disk drive specifications Ultrastar® DC HC520 (previously known as Ultrastar He12)". Just search for "CCTL" in the PDF. It appeared as though there is a "previous" and "current" interface to configure CCTL, maybe Areca hasn't caught up?

    I do get the impression this may be an Areca problem. Too bad they just blew me off when I tried to point out this may be a problem. Sounds like you got the same (lack of) response. This is not just any drive, it is HGST 12TB - they should test it!
     
  10. brouhaha

    brouhaha n00b

    Messages:
    3
    Joined:
    Jul 18, 2018
    The WD Gold 12TB drives (WD121KRYZ) with version 01.01H01 firmware (AFAIK, the only released firmware version to date) definitely do have ERC. They default to having ERC turned off, but you can turn it on with smartctl.

    Code:
    % hdparm -I /dev/sdr
    /dev/sdr:
    
    ATA device, with non-removable media
            Model Number:       WDC WD121KRYZ-01W0RB0
            Serial Number:      [...]
            Firmware Revision:  01.01H01
    [...]
               *    SCT Error Recovery Control (AC3)
    
    To turn on ERC with 7.0 second read and write timeouts:

    Code:
    # smartctl -l scterc,70,70 /dev/sdr
    smartctl 6.5 2016-05-07 r4318 [x86_64-linux-3.10.0-862.2.3.el7.x86_64] (local build)
    Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org
    
    SCT Error Recovery Control set to:
               Read:     70 (7.0 seconds)
              Write:     70 (7.0 seconds)
    
     
  11. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    Does the ERC setting survive a reboot?
     
  12. brouhaha

    brouhaha n00b

    Messages:
    3
    Joined:
    Jul 18, 2018
    Yes, the WD Gold 12TB does preserve the ERC setting across power cycles.

    I'm using nine of these drives on a file server running CentOS 7, with three of the drives in a ZFS mirror, and six using ext4 and SnapRAID. I've had to power-cycle the file server more times than I'd like, due to problems with other drives (Seagate ST3000DM001, which are crap), and I've just verified that the WD Gold 12TB all still have ERC on.

    By tomorrow I will have eliminated the last of the ST3000DM001 drives, and good riddance! The only other drives in the file server now are an Intel Optane 900P PCIe SSD, used for boot and the ZFS slog, and six WD Red 6TB, used in pairs for parity for SnapRAID. I'll probably start replacing those with WD Gold 12TB as well.
     
  13. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    Good info, thanks. I am partial to Toshiba/Hitachi, and now there are 14TB non-shingled drives available. In a perfect world I would start my upgraded RAID array with these babies. But of course there is always the compatibility question, which seems to never be answered by the mfg's. Not sure many Hardforum-ers have tried the combo of the Areca 1882ix-16 and Toshiba 14TB yet either.
     
  14. brouhaha

    brouhaha n00b

    Messages:
    3
    Joined:
    Jul 18, 2018
    WD owns HGST (former Hitachi and IBM drive operations), and WD Gold drives (at 8TB and up, not sure about smaller) are mechanically HGST enterprise Helioseal drives. The Gold 12TB is mechanically the HGST Ultrastar DC HC520 (Ultrastar He12). The Gold 12TB electronics might or might not be the same as the HC520 SATA model, but the firmware is different.

    HGST makes a 14TB PMR (non-SMR) drives (Ultrastar DC HC530), but there isn't yet a WD Gold version of that.

    I used Toshiba laptop drives for many years, and they were fine, but I've never used Toshiba non-laptop drives.

    I expect to continue buying WD Gold 12TB drives for a while. I'm using ZFS mirroring for one array and SnapRAID for the other, and adding new 14TB drives into an existing array of 12TB won't gain me anything. Unless there's some dramatic change in drive pricing, in the next few years I don't think it will make any sense for me to consider buying any higher-capacity drives until they hit 20TB or more.

    However, for my laptop, I'd really love to get a pair of 4TB M.2 NVMe drives to use as a mirror. I'm currently using a single 2TB.

    I'm not thrilled with the WD Red 6TB drives, only because despite having the firmware that was supposed to correct the short idle unload problem, I still see the load cycle count going up a lot faster than I think it should. The WD Gold 12TB do not have that problem.
     
    Lifespeed likes this.
  15. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    Thanks for the clarification. There has been a lot of merger/acquisition activity in the hard drive space, rendering our experiences with particular brand names no longer accurate. Western Digital had a well-deserved reputation of lesser quality and lifespan, supported by extensive data from Backblaze and the user community at large. It sounds like those experiences may no longer apply to the 12TB and 14TB models.
     
  16. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    I'm thinking of pulling the trigger on 4 each 14TB PMR drives, preferably with 4K sectors. Does ARC-1882ix-16 support the larger sectors?
     
  17. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    I see the June 2019 compatibility list now shows the Hitachi HUH721212ALE600 12TB as compatible. I posted in the owner's thread about my imminent upgrade.
     
    Last edited: Sep 6, 2019
  18. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    WUH721414ALE6L4 looks like a hopefully-compatible successor to the 12TB HGST.
     
  19. Lifespeed

    Lifespeed n00b

    Messages:
    49
    Joined:
    May 12, 2013
    maxxmusterr, brouhaha if you're interested in the modern HDD vs. ARC-1882ix-16 saga I'm going to start with 4 of these Western Digital 14TB Ultrastar DC HC530 drives in RAID5 to check Error Recovery Control (ERC) and general compatibility with my ARC-1882ix-16 card. Just updated the FW and RAM from 1GB to 8GB, what in improvement!.

    Do I need to manually enable ERC using a disk utility, or will my RAID card configure it? I don't recall needing to configure my old Hitachi 3TB drives.