[Kira Hammond]

Sadly this is not an easy question. Do your research and then check in for a free quote with a local video integrator because you need to calculate some stuff and there are some gotchas that will make your purchase a boat anchor.

Here’s my main advice:
– DAS direct attach RAIDs for each station. Thunderbolt is fast and good but expensive see below you might get away with FW 800 depending on bandwidth needs. I like OWC for cheap but good as a supplier. I’d go RAID 5 personally for mix of speed and backups. SSD drives too expensive for your budget, 1TB drives are cheap and plentiful. Look at the size of your average project and pick something that works for each station that truly needs the speed and security. Example your main edit bay has a RAID 5 on thunderbolt. Your producer maybe doesn’t need anything but 1GB ethernet sharing to the RAID 5 knowing they will pull down dailies or reviews locally (don’t play off the editor’s RAID over the network).

– Get a fairly cheap NAS, network attach storage that ONLY holds assets you pull into the direct attach or other stations locally. So just to be clear you aren’t editing off this sucker, or even viewing assets, you are only putting assets there that you need to use for lots o’ projects. This is also the place you put assets you want to archive so it clears up space off your DAS. The NAS could be RAID 1 for full redundancy it’s hella safe on there but slow as a turtle so you aren’t working there you’re just storing there.

– Get an LTO backup system. Part of the reason you are eating up space is you have no secure backup. Get those old projects outta your way. Man what I wouldn’t give to have something better than LTO to recommend but that’s the cheapest, most secure, most universal backup right now. You can always slap a *copy* on a cheap ole external HDD for fast access but that LTO is great for long term storage.

Ok so let’s say you’re like “I want a second opinion, I’m going to talk to a systems integrator.” For sure go for it! I recommend it. Make sure they are considering this stuff and you know the answers. If they don’t ask these questions find a new systems integrator!

— What OS are you on across the board? Do you mix any PCs or Linux or are you all Mac?

— What connections do you have in-house? Thunderbolt probably if new iMacs. 10 gig E will only do so much. Fiber is the bees knees but it’s way too expensive for your budget unless it’s already in your space.

— What CODEC, FPS, do you work with, do you mix em’ up in one project? Use a bandwidth calculator to see how much throughput you’re using. Do it for the max. Don’t forget “streams” which equals layers in your video project. For example if you have a layer of Animation CODEC at 1080p plus ProResLT 3 layers of video plus some audio up in that jam you need to add all that together to come up with bandwidth for one project. If you exceed your bandwidth you’ll hit dropped frames and render errors. Big ole gotcha for performance.

— How many folks will be editing simultaneously? Multiple above by number of simultaneous users.

— Do folks want to share a project (work on it at the same time, share assets)?

The cheapest thing for you is DAS or direct attached storage.
Upside, less IT expertise needed. You can RAID that sucker to meet the demands of that workstation. You can have high speed connection with no worries about sharing. Less of an issue of bandwidth max and easy to remedy because you have 1 user and 1 project. Believe in the power of proxy workflow if you hit a bandwidth issue (example ProResLT in your project fps and frame size as an excellent proxy format).

The next step up is a NAS, storage area network.
Upside hey now we can share. We can also be far away from each other over a network. Save space only having assets in one place.
The downside oh crud now we have to share, you gotta deal with permissions issues and if you have a mixed OS environment how those different platforms deal with permission is a bear. NAS relies on network cables so your bandwidth is limited. It’s also straight up not setup to be a traffic cop and deal with multiple users possibly over writing each others files. You’ll need a central server and some software to deal with the bandwidth allocation and permissions issues. Oh snap now you need a server and probably an IT admin.

Final frontier is a straight up SAN that’s for real what studios use for a reason.
Upside now we have a traffic copy, we have a permissions control center, we have ways to isolate and share openly without fear of overwriting work.
Downside. We have serious complexity at this point and you’re going to want someone with some knowledge and skills to admin it for you. That’s why small shops don’t go this route unless you’re willing to learn networking ins and outs and devote serious time on a regular basis to keeping it humming along.

There you go crash course in video storage.

K

[Kira Hammond]

Hi Folks,
I know this is an old post but some helpful tips. Feel like I’ve seen and dealt with it all when it comes to network storage and video:

All SANs and NAS for that matter suffer from some degree of the following:
– permissions issues
– performance issued due to hitting the max bandwidth
– indexing problems (can’t search for files)

OK so the work around for the following:
– permissions issues
– easiest fix is what the other users say, locally save file. Then move back to the SAN at end of the day for safe keeping. JUST the project file. The data, renders, all of that is A-OK to keep on the SAN. I wanted to note this is NOT just Unity, it’s all SANs, XSan too.

– OK to really fix it you’ll need to get into setting up GUIDs and UIDs and a server side ACL (access control list) which will manage who “owns” a file, and avoid the file locking. It’s something your IT Admin could do for you. It sits in-front of the SAN and plays traffic cop essentially. Xsan was happy with this. Unity you have to be kinda sneaky on, as it’s not supported. Avid dodges the permissions issues through dongle authentication and if you dig down all the files have the same UID and GUID so wa-la no permissions issues. Check the file owner through the command line not the OS, if you find a locked file check the owner. Then you can go to that machine and adjust it’s

– Alright I admit I did a hack once in a small shop that worked, change ALL the machines to the same GUID and UID. It works, avoids 90% of permissions issues. Your IT person will hate you.

– Performance issue
– My personal pet peeve, I haven’t found a SAN or NAS yet that actually publishes bandwidth that’s accurate for the real-world. I used the AJA I/O tester ahead of time to make sure for the CODEC, fps, and audio I’m using the system can handle it. It’s free on AJAs site. Don’t forget to account for ALL the other content you are accessing over the SAN, example: motion graphics which might be in a completely different CODEC esp if you need transparency, alpha stuff only available in a few CODECs.

– Use fiber. Seriously for the amount of headaches you’ll have with Ethernet it’s just not worth it for the common Joe/Jane. Can you get Ethernet to work reliably? Yes, but again time v. money fiber is the way to go.

– Indexing problems
– OK yet another one where there seems to be as of yet no good solution. Issue you’ve got probably hundreds of thousands of files how to index them so you can search for stuff. Neither Unity for Xsan have good solutions. The Unity version is buy this DS system, btw it costs like 50k, or you can buy Interplay it’s around 150k.

– So as of yet there’s no “out of the box” solution to search for files on a SAN that’s reliable.

I will say that I find Unity and FCP very stable in the broad scheme of things, I disagree with the folks saying otherwise.

I would choose Unity over Xsan any day in terms of stability.

In terms of sheer bandwidth performance, flexibility, and cost StorNext is hands-down the winner.

Now if only Xsan would get off Bonjour and DHCP authentication which is about as reliable as the weather man…. I could actually recommend it, as is I cannot in all good faith do so.

I think all three StorNext, Unity, Xsan require an on-site tech guru. Which is lame really. Unity is the easiest to admin of all 3 but the least flexible and lowest sheer bandwidth so yuck there.

For a more hands-off “we have no technical folks on-staff” situation I recommend FiberJet or TerraBlock. Yes I have setup both and left the places to run on their own- no call backs. Amazing!

The Unity niggly things to bite you in the arse:
– VERY dependent on the correct versions of drivers, OS, Ethernet and Fiber Connection clients. It’s worth a support call to Avid, have them use their Log Me In account and go over your Unity with a fine-tooth comb to make sure there’s nothing in conflict.

**Also check your Service Pack on the Windows Unity server, something they forget to mention sometimes.

If you have Ethernet connection issues remember restarting the Port Service is your best friend!

– Naming convention for the Unity drives is super dooper key. You can’t stray even a space or an underscore from what Avid wants. FCP could care less you can call a drive Whoop De Do and FCP is fine, but all your Avid clients will freak out, or have weird stability issues.

– Bandwidth! Remember it’s cumulative too! So if you’ve got one person ingesting HDCAM tapes they are seriously killing your bandwidth. There’s a great chart on the Avid Unity site about what to expect.

I have a feeling what that one person is mentioning with the 30% less deals with the fact that the Avid CODECs (both blessing and curse) is they are specifically meant to work with the Unity architecture to maximize the bandwidth- they are super efficient.

I have a feeling your performance issues are related to two factors:
– not transcoding the footage, so on the fly FCP is chucking over to the video card the real-time conversion. Transcoding to Pro Res is great. Just an FYI the AJA Ki Pro records in Pro Res so if it’s a tape house, just rent one and have em’ dump their tapes directly to Ki Pro. Fast, easy, no rendering involved.

– check your FCP settings. With great range of options comes great complexity. But here’s some settings to keep in mind:
a) Real time tracks, lower it to up performance. More rendering but better playback less stress to video card.

b) CLOSE sequences. FCP has to keep those files up and active if the timeline is open. CLOSE any you aren’t using. This seriously helps.

c) How big is your project file? Project files can creep up in size, I find major instability if a project file is over about 50MB and it happens easily if you’re editor is a digital horder. If this happens convince them to make an “archive project” and move all the old sequences in there. Delete them from the working project file.

For HUGE projects (20+ hours) I use Pro Res Proxy for all the editing (a very picky creative director and client even commented on the high quality considering the heavy compression). Then take that offline, and re-link to the original whatever. Works great on SAN storage that would otherwise be brought to a screeching halt with that much footage in a project, or simply using a less efficient CODEC.

Ya if you have more questions just let me know I have lots of experience here.

Kira