Photo sharing and logging, FCP-style on a SAN

[Matt Geier]

Eric,

Have you considered putting in a 10Gb Ethernet (High Bandwidth / Non Editing) network to do the transfer and logging of all those videos?

What are we talking 50GB-100GB + of files?

At anyrate it could all be set up with Ethernet no doubt — –

In the future these things are perhaps a good candidate to consider something like FCoE when it comes out… Fiber Channel over Ethernet.

It will run on 10Gb Copper or Optical Ethernet wire…

What do you think about that idea?

[Eric Hansen]

thanks matt

but this isnt a bandwidth issue. when we’re logging footage on the Mac Minis, they’re only playing a single stream of ProResHQ. all the footage resides on the SAN volume. it’s not getting transferred anywhere. when we actually transfer footage into the system off external drives, they are just hooked up directly to the SAN server. 10G ethernet wouldn’t help in this situation.

i’m just looking for a way to centralize our photo database. i’ve reached out to a few pro photographer friends, but they are all one-man-bands and don’t have to deal with sharing their libraries among different computers.

any other ideas out there?

thanks

e

Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com

[Dave Klee]

Hey Eric, you’ve found a great problem. There’s a huge disconnect out there between software made for a single photographer at a single computer vs. enterprise level digital asset management systems. This is something we’ve been struggling a lot with over the last few years at our corporate production shop.

The solution we’ve come to rely on uses some ingest scripts that have been custom coded to ensure unique file names and proper image placement on the server (which is a standard OS X server running Leopard with some RAIDs attached). After that, files on the server are annotated in place with Adobe Bridge — keywords, descriptions, headlines, etc. are added to files through AFP file shares as they live on the server. Searches are later performed with Spotlight Server (assuming you’re using Leopard or better everywhere). Permissions are heavily regulated to prevent accidental deletion or overwrites using an Open Directory schema with some reliance on ACLs.

We’re on the fourth or fifth major revision to this workflow, and it seems to be going pretty well for us. Happy to share more details if this sounds helpful — just let me know!

[Eric Hansen]

thanks dave

your mention of Open Directory and ACLs made me cringe. flashbacks to an old Xsan installation i tried to over-manage.

you mention Spotlight. on our SAN setup, i was advised to turn off Spotlight on the SAN volumes. i forget the reason for that. Mr Modica? maybe its the invisible index files that Spotlight creates at the root of the volume. maybe that doesnt work in a shared environment. anyway, Spotlight will see all the metadata you add to the images in Bridge? that’s pretty slick. maybe i should look closer at Bridge.

searching around i found Extensis Portfolio Server. I couldn’t find a price, i’m gonna call them today, but i’m guessing its a few thousand. i don’t know how much i want to spend on this. we do have Final Cut Server, but i haven’t integrated it yet. i’ve been waiting for Peachpit’s book on version 1.5 to come out. it was supposed to be Nov, then Dec, then Feb, now April. i think it can handle images, but i don’t know how well. i guess its time to turn it on and find out. i just ordered Peachpit’s “Getting Started” book.

after i figure out images, i’m moving onto music. i’m thinking iTunes sharing. i like how there’s extra tagging fields so i can add things like Label, Publisher, rights, etc. that’s always been a big hang up for editors i’ve worked with – not knowing what songs are cleared for their edit.

i’ve worked at a lot of production companies where they needed to share this stuff, but never got a system in place. now that i’m working for a company that’s pretty much starting from scratch, i’m implementing the big thing i’ve always wanted – asset management for video, images and audio.

thanks

e

Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com

[Dave Klee]

Hey Eric, sounds like you’ve put some good thought into this! Very cool.

Portfolio isn’t bad, but I’m not a huge fan of the interface (at least I wasn’t on Version 8). Canto Cumulus is another popular choice, and we have an implementation here. Both are in the “thousands to tens of thousands” range for a good client-server setup. We’ve done a lot of shopping for Digital Asset Management (DAM) / Enterprise Content Management (ECM) solutions, and here are the key “sales-pitchy” things that have annoyed me:

1. They show you demos of how easy it is to search for things — search is their key strong point. However, if you regularly need to put new things into the system, or regularly need to make updates to metadata, you’re in trouble. Ingest and initial tagging is often a challenge.

2. Implementation time is longer than they lead you to believe. The demo shows, “Look, you just drag your current library into the software and, BAM, you’re done.” Not true. Sorting out the wreckage and configuring the system to your unique needs will take months — if not years. Plus, don’t forget about teaching your video editors a new interface.

3. They’re heavily print (not video) oriented. Plugins from the big players are becoming fairly common to allow you to import your assets into InDesign and Quark. They’re targeted at graphic design shops that make regular catalogs or magazines — high volume work where big groups of people need regular access to a centralized image database. Once you find an asset in your centralized catalog and want to use it in a video, it’s not as easy as right-clicking to reveal it in the Finder and/or drag it into your FCP timeline. You’re usually looking at an “export” process to give you a copy of the picture that has to be stored somewhere outside your catalog. In general, once you load your assets into a program like this, you’re tied to the ways it allows you to work — which is rarely through the Finder.

4. Licensing can be a challenge. With Canto Cumulus, you can get unlimited “read-only” access through a web interface. I believe Portfolio makes you buy a “web-publish” add on or something like that to get similar functionality. Still, for people to update and have full access to the catalog, you’re usually looking at either per named user or per concurrent user access fees. They add up quickly.

5. Don’t forget about maintenance. Often, there are annual “maintenance fees” associated with these programs. Translation: “Pay us a percentage of your up-front cost every year or we don’t help you with anything and your catalog eventually dies.”

6. Metadata write-back is often claimed to be supported, but rarely adequate. What that means is that when you add keywords to a picture through the interface for these software programs, much like Aperture or Lightroom those keywords live in a propitiatory database that can only be accessed by that particular software (or however the software lets you access it). The keywords are rarely “written back” into the picture itself. So, what happens in five or ten years when you need to transfer your image library out of that propitiatory database to something better? None of your images actually have keywords directly attached to them — they all live inside this software that you want to get away from. Good luck.

7. Updates are frequent enough to be annoying. In general, these systems get updated on a fairly regular basis — small updates take time to install and implement, and big ones cost money. True for life in general, but particularly annoying for what will inevitably become one of your core software packages.

Final Cut Server is a huge exception to many of these problems — which is why I think it has a lot of potential. It’s video oriented, has drag-and-drop import into FCP, isn’t updated too often (both good and bad), and doesn’t charge you extra license or maintenance costs. However, its metadata write-back is fairly immature, meaning most additional information you write into FCS metadata needs to be exported through XML files with some clever coding. And implementation time, as I’m sure you know, isn’t short. Also, while it can certainly handle images, it’s much better at video (in my opinion).

If I sound a little bitter about this stuff, I am. We’ve been through a couple different major pushes to find a DAM or ECM solution that would fit our video production and photo needs, only to end up pretty disappointed at the current batch of offerings. The last major look was a year ago, so it’s possible something has come out since that is worth a look, or that we just totally missed a good option.

Not knowing you and your situation, it’s totally possible there’s something out there that fits your needs. Please let me know what you find — I like looking into this stuff, and would love to see what a fresh set of eyes digs up.

As for spotlight, it hasn’t proven very effective for me on video volumes, but it’s fairly effective for photos given a few pre-requisites:
1. You’re using an actual OS X Server, Leopard or better.
2. You’re primarily accessing your photos via AFP shares from Mac OS X clients, Leopard or better.
3. You’re aware of which IPTC and EXIF (metadata) fields are indexed by Spotlight — it’s not all of them. I’ll look around for a list.
4. A good (gigabit) network is always a plus.

Spotlight Server (built into Leopard+ server) is meant to work in a shared environment — previous versions are not. If you want to give it a try, I’d recommend setting up a separate AFP share on a separate volume dedicated to photos. Personally, I’ve found that systems for photo and video require different optimization techniques that are often incompatible. Photo volumes typically need safety and search-ability, and are willing to sacrifice some speed to get it. Video volumes need maximum throughput with minimum latency. Maybe it’s more of a problem or archive vs. active storage; we treat our photo volume as an archive where we store things permanently, and our video volume as an active scratch disk where things live only for the production cycle. (We’re using Xsan for our video volume.)

Anyway, looking forward to trading thoughts on this stuff and seeing how things go for you! Honestly, I’m surprised video-oriented DAM isn’t a better niche in the marketplace. Hopefully you find something out there that works for you and you can share with me!

Dave

[Eric Hansen]

thanks for all the advice Dave

i’ve been trying a few different solutions. surprisingly, iPhoto sharing works, and the search feature is really fast on a client system. it works just like iTunes sharing. but you can’t access all the metadata in iPhoto the way to can with Aperture, Bridge, et al. i do like with Bridge that you can have keyword categories. we would need to bulk-apply keywords like person, place, etc. i also want to apply things like Copyright and photographer, and only the ‘bigger’ programs let me do that. iPhoto just doesnt have enough tagging ability. but it nails the sharing thing, so it really bugs me that it won’t work. i have Bridge CS3 because i use After Effects CS3. do you know if there’s a difference between CS3 and CS4 with Bridge?

i know that eventually we’re going to have to switch our SAN server over to OS X Server when we add more client systems. maybe i’ll have to make that jump earlier than i thought. it looks like your solution of using Bridge with Spotlight is the best of the inexpensive solutions.

but i still don’t know how well Final Cut Server will work with photos. i’m spilling over all the reading material now. hopefully i’ll kludge something together soon.

thanks

e

Eric Hansen – The Audio Visual Plumber – http://www.avplumber.com

[Dave Klee]

Hey Eric, I haven’t given iPhoto sharing a try — I think I’ll go play around with it. Nice idea for something simple to get going with. But, you’re right — you’ll probably outgrow iPhoto’s tagging capabilities, and it has the same problem of not TRULY embedding all its metadata back into the original files through IPTC write-back, making long term upgrading a little scary.

As for Bridge CS3 to CS4, there’s a pretty decent overview of the updates here:

https://blogs.adobe.com/jnack/2008/09/whats_new_in_bridge_cs4.html

From a cataloging perspective, CS4 has a couple small tweaks that make handling hierarchies of keywords easier (parent & child relationships). And, I can verify CS4 is much faster and much better with searching than CS3.

But, is that worth upgrading your whole suite for that? I’m guessing not. We started our process with CS2, and the CS2 to CS3 upgrade was HUGE. CS2 Bridge was borderline unusable for large-scale cataloging. But, everything you need is in CS3, including the ability to do those bulk-applications (metadata templates) for files. I’d say you can always start with CS3, and if you find you’re doing a lot of work and want a speed bump, CS4 will be nice.

Let me know how things go and if you come up with anything else clever — always looking for new ideas!