Need advice on HDD configuration (a.k.a RAID 5: The Debate)

[Mark Raudonis]

I would buy an additional small SSD and put it in the empty DVD bay. Now you have a total of FIVE drives. Put your OS on the SSD, then you can RAID all FOUR drives in the bay. Maximum raid throughput and storage. Maximum performance and speed off of the SSD for your OS and core programs.

[Douglas K. dempsey]

John, I don’t have advice but I’m anxious to hear responses to your query because I have many of the same questions about what I SHOULD purchase.

I am also wondering about the best config for FCPX, given the library structure. In FCP7 we always kept the project file separate from the media drive. In X, it is very easy to fall into allowing everything to be stored in the Library, because of the ease of copying/transferring the whole shebang intact.

On your system, would you have the Library on the SSD or other single drive, with the “sync” alias pointers to media on your RAID? Or keep the entire Library on the RAID, where it could easily be archived to your other Cine RAID as a single transfer? Does it matter?

BTW in my own home office setup, I tend to prefer RAID 0 because I end up backing up media anyway, so RAID 5 becomes redundant. As I understand it, RAID 5 keeps you working after failure in a time-critical environment … versus stopping to do a major restoration. Is that accurate?

Doug D

[John Rofrano]

[Mark Raudonis] “I would buy an additional small SSD and put it in the empty DVD bay. Now you have a total of FIVE drives. “

Sorry if you missed that in my original post but that’s the first thing that I did. 😀 I have a 512GB SSD in the empty DVD bay with the 4 other bays filled so I have 5 drives.

I am considering getting a OWC PCI SSD. The advantage not only being an extra drive but you get SATA III 6GB/s throughput instead of SATA II 3GB/s throughput. They’re expensive but they also have an eSATA port and I have an eSATA enclosure that I’d like to be able to use as well. I’m wondering if they are worth it performance wise? Perhaps that should be a new thread?

[Mark Raudonis] “… then you can RAID all FOUR drives in the bay. Maximum raid throughput and storage. “

So, that’s something I hadn’t considered. I have always had a separate media drive that contains stock content (royalty free music, images, video, etc.) and a separate drive for editing. You’re suggesting that I make just one big RAID 5 drive for media and projects. That kinda makes more sense because a 4 drive RAID 5 should perform better than a 3 drive RAID 5. Hmmm… That’s something to consider. Thanks!

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Mitch Ives]

I had an Apple RAID card at one point. It was an elegant solution, if not a bit finicky at times. But it never gave me the speed of and external RAID.

As for “is it fast enough”, that’s pretty subjective question… and I suspect that if you ask three people, you’ll get four opinions. For myself, speed has many advantages, in terms of saving me time and saving my clients money. It also lets me experiment more. So for me, whether you ever do 4K or not, speed is always good. That’s why I still use an external TB2 array even now on a screaming new MacPro with the largest SSD.

Mitch Ives
Insight Productions Corp.

“Criticism may not be agreeable, but it is necessary. It fulfills the same function as pain in the human body. It calls attention to an unhealthy state of things.” – Winston Churchill

[John Rofrano]

[Douglas K. Dempsey] “In FCP7 we always kept the project file separate from the media drive. In X, it is very easy to fall into allowing everything to be stored in the Library, because of the ease of copying/transferring the whole shebang intact.”

It’s a different way of thinking about projects. What’s a project without the media? It’s a useless metadata file really. A collection of edit decisions without the media does not a video make! So keeping projects with the media makes a lot of sense when you think about it.

It also depends on how you use Libraries! I use a new library for each “major” collection of projects. Definitely for each client. So, for example, if you are working on a TV show that has episodes, I would create a library for each Season where a normal season is 13 episodes. I would have a Season 1 library and then each Event would be a new episode. Each project is then that episode, or motion graphics used in an episode, or the trailer for the episode, etc. It make sense for me to have everything together like that. I can then backup all of my work by backing up the library and I know I have everything I need to move to another workstation and continue editing (like moving from my MacBook Pro to my Mac Pro) I do this with my Boris TV episodes today.

If, however, you are collaborating on a project where the media is on a shared drive and want your project to be separate on a local drive, then when you create a Library, you should select to leave the media in place. Then your library will only contain your project files with symbolic links to your media files which will be separate on another drive because you left them in place when you imported them. So with FCP X you can work both ways as you suggest. I just don’t work that way.

[Douglas K. Dempsey] “On your system, would you have the Library on the SSD or other single drive, with the “sync” alias pointers to media on your RAID? Or keep the entire Library on the RAID, where it could easily be archived to your other Cine RAID as a single transfer? Does it matter?”

I would keep the entire Library on the internal RAID and then still keep the external CineRAID for archiving finished products.

I just don’t see the need to keep the project separate from the media. I need to be able to move between my MacBook Pro and Mac Pro and so I like having everything in the Library so that I only have to transfer the Library and know that I have everything. Once again, that assumes you use multiple Libraries and not keep all of your work in one big Library! My libraries are small enough that this makes sense for me.

[Douglas K. Dempsey] “BTW in my own home office setup, I tend to prefer RAID 0 because I end up backing up media anyway, so RAID 5 becomes redundant. As I understand it, RAID 5 keeps you working after failure in a time-critical environment … versus stopping to do a major restoration. Is that accurate?”

That’s correct. A RAID 5 (or RAID 1 or RAID 1+0/10, etc.) is NOT a backup. It is purely for continuing to work after a failure. You still must backup any RAID system in case of catastrophic failure where a catastrophe would be loosing more than 1 drive in a RAID 5 or two drives in a RAID 10. Obviously losing a single drive in a RAID 0 would be a catastrophe as well.

~jr

http://www.johnrofrano.com
http://www.vasst.com

[Douglas K. dempsey]

Thanks John.

As I said, I quickly fell into the habit of keeping everything in the Library on external RAID, and just designating the FCPX auto backup to my internal drive, so you theoretically have another copy of the metadata/project if everything goes south before proper RAID backup and you must recreate a media drive and relink everything.

I’ll be interested to see what people say about maximizing effectiveness of your Raid card.

D

Doug D

[Bret Williams]

A raid 5 with 3 drives sucks up 33% of your drive space. I find even a 4 drive Raid 5 kinda sucks that it takes up 25%. 6 drives is sweet as you get full backup but it only eats up 16% of the space if my memory serves me.

[Michael Hadley]

I used to to do Raid 5 and never had problems–but always a little worrisome if the drive went down.

What I do now is:

1. Keep a copy of original cam source media on an external bus powered drive–500gb, 1 tb. I have a library of a couple of dozen of them.

2. I transcode/optimize the media onto a large thunderbolt drive 8-12 tb.

3. I have a second, usual less expensive 8-12 tb drive that I clone from the main drive at the end of each sessions.

If my A drive fails, I can immediately go to my B-drive. If they both fail I’m screwed but then I still have the original camera media. Seemed like a good compromise between cost and peace of mind.

There is no one perfect solution. At least with this workflow, I have two good copies plus cam original. Raid 5, only had 1 good copy.

[Fabrizio D’agnano]

[Michael Hadley] “I used to to do Raid 5 and never had problems–but always a little worrisome if the drive went down.”

I use a similar set. The array I use for daily editing is a RAID 0. The second large array I synch the editing drive onto is a RAID 5. Then I keep cards fcparch’s and copies of the selected media on separate sparse external single drives, divided per year (fcparch’s on one drive, imported media on another). If the main drive goes south I can clone back the RAID 5 without having to walk on thin ice during the process, still keeping on working, as I can still resist one more drive failure. The sparse drives I keep in a second location give me that extra security in case of a severe problem or things like theft or fire. 3Tb hard drives has dropped significantly, so I like the extra security, even if can look paranoid.

Fabrizio D’Agnano
Rome, Italy
early 2008 MacPro, BM Intensity Pro, early 2008 iMac, 2014 MacBook Pro Retina, Blackmagic UltraStudio Mini Monitor, FCP7, FCPX, OSX 10.9.4

[Bret Williams]

For turnkey stuff, we shoot and transfer to portable drive(s) on location. We don’t reformat cards until we have to. Then copy camera footage to raid 5. The library also exists on the raid. Final cut makes backups automatically to the system drive. At any point the raid could lose 1 drive and be ok. Or if it went down completely then the project could be reconstituted from the library backup and the camera backup. When projects are done I mive them to a mirrored usb3 raid 0 which also holds stock media. When that is full I remove the drives and store each at a different physical location for theft/fire protection.