Raid Controller required for Home NAS

[laserbeam]

Hi,
I'm building a server at home to function as a NAS, amongst other things. The configuration I want to get to (feel free to comment/advise) is a raid-50 array made up of 10 Seagate 4TB ST4000VM000 drives. I won't do this immediately and plan to start out with a smaller raid-5 array.

Some issues have occurred to me as I've started considering how I'd get to the target config:

- I have not seen any 10-port raid controllers
- Therefore I'll need to span the raid-50 across two raid controllers. Is that a normal config, or even possible with non-enterprise kit?

Seeing as I expect to have to span across raid controllers what controller would people recommend that would allow me to daisy chain other controllers from? And which would allow me to flexibly grow and change my raid config & strategy (if I decide to do things differently) over time? And obviously I'm looking to get as much throughput as I can so performance is a key requirement.

I'm not a storage expert so I apologise if I'm asking naive questions here. But on-mobo raid is not an acceptable option to me so I need to do something a step or two up. If anyone can help me to understand my options on the product front and how the raid-card market is segmented I'll really appreciate it


Regards,
Dave

 

[Justintoxicated]

What do you need enterprise level performance for with a home NAS? Will you be running VMS or something off this storage?

 

[HobartTas]

Greetings

Seagate 4TB ST4000VM000 drives.

Do they have the TLER feature that lets them work with hardware arrays that enterprise drives have and consumer drives do not, I don't think they do? If that's the case you may have to get Enterprise drives and they will cost a lot more, or at least ones with the TLER feature like WD reds or the new NAS capable Seagates.

The configuration I want to get to (feel free to comment/advise) is a raid-50 array made up of 10 Seagate 4TB ST4000VM000 drives. I won't do this immediately and plan to start out with a smaller raid-5 array.

Raid 5 with (I presume five 4TB) drives = effective data storage 16TB

Raid 50 with (I presume ten 4TB) drives = effective data storage 32TB

the problem with this is you can lose at most 2 drives, but if there are 2 lost in one group of 5 then you've lost that Raid 5 group and since this is part of an overall group of Raid 0 (assumming one large volume instead of 2 seperate ones) you will lose everything.

I have not seen any 10-port raid controllers

Thats because they usually come in muliples of 4, so you have 4 porters, 8 porters and 16 porters, etc. So you would have to get a 16 port controller and hook the 10 drives into that and leave the other 6 ports free.

Therefore I'll need to span the raid-50 across two raid controllers. Is that a normal config, or even possible with non-enterprise kit?

No idea, never used hardware raid, guessing probably not but I could be wrong.

And obviously I'm looking to get as much throughput as I can so performance is a key requirement.

How much performance do you need, is it just to saturate a Gb link?

Or do you need a SAN connected with 10Gbe or Infiniband because your running an OLTP database?

But on-mobo raid is not an acceptable option to me

Wise decision.

If anyone can help me to understand my options on the product front and how the raid-card market is segmented I'll really appreciate it

(1) Cheap raid cards usually only have raid 0,1, and 10

(2) moderately expensive cards have same plus raid 5 and 50

(3) expensive cards have same as moderate cards plus raid 6 and 60

(4) cards prices rise with the number of ports and its roughly linear.


Have you considered running ZFS as the native file system?

Advantages are

(1) Don't need raid cards, however, you just use the ports on any card in JBOD mode, e.g. I run two NAS's both 10 drive Raid-Z2 (Raid 6), = 8 data drives + 2 parity drives,

NAS 1, Abit motherboard with 4 onboard satas, Q6600 and 8 port LSI 3081E-R, drives are Samsung 1.5 TB HD154UI's

SATA 0 is OS boot drive
SATA 1 free
SATA 2&3 has 2 drives
3081E-R has the other 8 drives

NAS 2 I7-3820 + Gigabyte X79S-up5, drives are 3TB Toshiba DT01ACA300's
one sata has OS boot drive
2 ZFS drives in two other SATAs
8 ZFS drives in the onboard SAS/SATA ports.

(2) ZFS can use ANY drives including green drives.

(3) ZFS checksums all blocks and re-silvers failed drives with ease.

Disadvantages

main one seems to be that everyone gripes about is that you can't increase the number of drives in a Raid-Z /Z2 /Z3 stripe, I solve this problem by filling my NAS with the maximum number of drives up front.

Performance

The first NAS does 50-70 MB's and the second one saturates Gb links.

I use Solaris 11.1 but I suggest you use napp-it with that or alternatively FreeNAS.


Cheers

 

[dandragonrage]

Note: avoid any 3Gbps/1.5Gbps LSI controller - they all have a 2TB limit. You'll need one of the 6Gb/s LSI controllers. Generally SAS 2008/2108/2208 are what we recommend. IBM ServeRAID M1015 (8 port internal) is popular and I use one myself. Could get 2 of those. If you use ZFS, you can just use it as a HBA and do RAID via ZFS, and if you do that you can even use motherboard ports for your additional 2 drives.

 

[Zedicus]

if you want to go hardware route use a single controller and an expander, or 16port controller (they really just have a built in expander). i like the adaptec stuff. the new 71605 cards, b aware that the E variants are basically HBA's. the adaptec 6xxx series has dropped considerably in price and if you were set on hardware raid i would recommend that line, iether with a built in expander or utilizing a seperate expander, you can end up with hardware raid and 16 - 24 hard drives for under 500$. well under 500$ with a little ebay scouting.

otherwise it has been and will be continually mentioned. ZFS, as far as storage goes its does a lot. basically use commodity hardware and end up with an enterprise SAN. it is a project and there are LOTS of decisions to get an end result though.

 

[dandragonrage]

if you want to go hardware route use a single controller and an expander, or 16port controller (they really just have a built in expander)

I disagree about the expander. They do have uses in large systems, but for 10 drives, every other option in this thread is better. Expanders are not worth it for such modest systems. Even 2 (identical) cards will be better.

 

[laserbeam]

Do they have the TLER feature that lets them work with hardware arrays that enterprise drives have and consumer drives do not, I don't think they do? If that's the case you may have to get Enterprise drives and they will cost a lot more, or at least ones with the TLER feature like WD reds or the new NAS capable Seagates.

Not sure but I take your point. I've also been looking at the ST4000VN000 NAS series drives. I think they'll match up well. In truth I haven't got to the point of looking closely at HDD brands and models yet. So far I've only decided that I'd like:

- quieter, power efficient drives: 5900 spin speeds will probably help there.
- Higher reliability (within reason): The NAS series seems to offer that.

But I've not looked at any brands other than Seagate just yet.

Raid 50 with (I presume ten 4TB) drives = effective data storage 32TB

the problem with this is you can lose at most 2 drives, but if there are 2 lost in one group of 5 then you've lost that Raid 5 group and since this is part of an overall group of Raid 0 (assumming one large volume instead of 2 seperate ones) you will lose everything.

I can tolerate that. That said, I do wonder whether I'd need to have "fallow" drives on-hand so I can swap out failed drives later. It could take a looong time for these drives to fail and this particular model might no longer be available.And then, if that's the case I may aswell go raid-6 - which seems over the top tbh.

How much performance do you need, is it just to saturate a Gb link?

You got it. I'm constrained by my network to Gb ethernet but think that should be fine for some time.

(2) moderately expensive cards have same plus raid 5 and 50

I've been looking at these - adaptec mainly - but am not sure what card hits the price/performance sweet spot cos I don't know enough about the details of the technology.

Have you considered running ZFS as the native file system?

I haven't considered this but may do. Am I right in think that with ZFS you kind of configure layers of cache to make the filesystem super fast? That is, RAM, then SSD and then the slow HDDs?

Can you provide a link which explains how to properly implement ZFS and the issues involved? It seems like it's relatively complex, and probably easy to screw up (suboptimal configs), given what I said above.

Thanks for this suggestion. You've made an excellent point and I'll give it serious consideration. However, I'd still like to understand the merits or otherwise of different raid card options out there should I still choose to go down the hw raid path.


Regards,
Dave

 

[laserbeam]

I disagree about the expander. They do have uses in large systems, but for 10 drives, every other option in this thread is better. Expanders are not worth it for such modest systems. Even 2 (identical) cards will be better.

Could you explain why that is the case?

 

[omniscence]

SATA drives do not adhere to the strict timeouts SAS drives do. I had an occassion where the SAS HBA did reset the whole bus because one SATA drive on an expander stopped responding to any command. The bus reset caused my raid array to go offline. There was no data loss, but all SMB and NFS clients got disconnects.

I used two 9211-8i controllers, but then ran out of PCIe slots. The best solution for me was to buy an expander.

 

[staticlag]

If you do use an expander with the LSI cards and SATA drives, you can only use the intel LSI expander RES2SV240.

or else you will get timeout issues, because the HP and chenbro expanders handle SATA timeout differently than the LSI HBA and will mis-prioritize drives when the array comes under lots of use.


If you use an LSI card with the LSI expander and have current firmware this is no longer an issue.

 

[_Gea]

Can you provide a link which explains how to properly implement ZFS and the issues involved? It seems like it's relatively complex, and probably easy to screw up (suboptimal configs), given what I said above.

To setup a ZFS storage server is not too complicated

I wrote a miniHow To. Maybe it helps for first steps.
http://www.napp-it.org/doc/downloads/napp-it.pdf