-
What Ever Happened to Metadata?
-
Also, I should add that FCPx’s containers aren’t necessarily containers, they are more like saved searches. You can select multiple “containers” and get the results for all of them.
Your smart collection method is very interesting as it essentially creates another UI within FCPX. Events, I think, are still nice because you can keep separate the things you need to keep separate as fcpx’s top view is very hard to give up.
-
[Jeremy Garchow] ” I should add that FCPx’s containers aren’t necessarily containers, they are more like saved searches.”
Very interesting way of looking at it.
But in a sense every container in any application is a saved search, isn’t it? Or do you have a more specific idea in mind?
Simon Ubsdell
tokyo productions
hawaiki -
[Jeremy Garchow] “There’s a different style of cataloging software currently in beta called Kyno. It sits on top of the Finder so there’s no library file. It sends to X, 7, and Pr, exports XMLs for all of those, and is worth checking out: https://lesspain.software/kyno“
Kyno looks very interesting and I’m certainly going to see how that works out for me. I think from the point of view of this discussion though it’s not the kind of thing I was looking for. If one is going to be tagging outside the NLE, I’d want it to be super simple and dedicated to media tagging. Kyno and KeyFlow Pro do too much else, if that makes sense. By which I mean that they’re not simple and fast enough to justify using for media tagging alone.
EDIT: Kyno exports a FCP X Event, which again is not really what I’d be looking for here.
The Apple Finder tagging is so close to what I’d ideally want that I’m hoping to see an option that’s very close to that – or better still that we get a more dedicated, slightly more expanded (but not too much) tagging interface in the next OS …
Simon Ubsdell
tokyo productions
hawaiki -
[Simon Ubsdell] “So my question is: will we always still be using containers of one kind or another, alongside these metadata filtering options, or can you see a time when we might move over entirely to the metadata-driven model?”
“Entirely” is a strong word, but I think it’s the right word to use as a goal.
The issue in the longer run is what happens When Metadata Models Collide. We actually wrote about this way back in 2008, in an interview I did with David Stump, ASC, called Metadata and the Future of Filmmaking. Dave has been a cinematographer specializing in VFX over the years, and (at least as of 2008) the chair of the ASC Camera subcommittee and co-chair of the Metadata subcommittee.
He was also very active in developing metadata recorders for cameras, cranes, lenses, and other on-set systems. He won a SciTech Academy Award for his work on that.
He’s currently one of the big brains behind Lytro, which can be better understood in this context: the highest-grade sensors and optics coupled with a staggering amount of both data and metadata. That’s the reason why Lytro is currently the size of a small car: it takes more computational power than can be contained in even multiple desktop computers.
The question with metadata is what happens to it downstream. For example, it’s very nice that FCPX reads Finder-level tags, but does Motion? (Maybe it does. I’m just asking.) What happens to those Finder-level tags for two clips that are rendered together to form a third piece of media? I assume that the metadata of the original clips is decimated, and you wind up with a clip whose metadata is a tabula erasa — not just a BLANK slate, but a slate formerly full of information that was erased.
Think about something as simple as timecode. It wasn’t that long ago that NO applications preserved timecode for something as simple as a rendered color correction. The only reason that they do now is because Warriors for Metadata Integrity told developers that they WANT that metadata preserved.
The same is true for plug-ins. There are times when their renders overwrite metadata as simple as timecode.
But timecode isn’t always the most relevant metadatum. The fact is that every setting on a modern high-end camera generates a field of metadata that your CAMERA can read. HUNDREDS of fields, just for camera menu settings. But does FCPX read those? Does After Effects? Does Resolve? Where does all that metadata go? Most of it simply disappears.
The goal that Dave and others have been striving for is, if your application doesn’t care about that metadata, don’t destroy the metadata already written to the file.
That’s asking a lot, but on the other hand, not so much. It would be intolerable if applications overwrote file names, creation dates, and other typically high-level metadata. Why would we not expect applications not to overwrite fields that are none of their business? Just leave that stuff alone.
Then there’s the data collision issue that I mentioned. The same finder-level information might be written by multiple applications (including a camera) — so usually the last one in is the one whose metadata “wins,” even though you might have preferred that the original data remained unscathed. So you’d need some higher entity being aware of multiple sets of metadata, and asking you what you want to do when there are conflicts.
And what WOULD you want to do with conflicts? The default would probably be “Which one of these do you want to choose,” but what if you want to save both pieces of information, but only have a single field of metadata to fill?
Avid’s solution is the right one I think, which is to allow the creation of a vast array of custom metadata fields. They’re only readable within the context of Avid editing, but that’s where metadata wrangling comes in. The offline footage goes to Resolve, where good-sized bits of the metadata are overwritten, then back to Symphony, which depending on how workflow was set up, may or may not have access to the original metadata. Somebody has to KNOW in advance where they want the metadata stored in such a way that it’s available where it needs to be.
Dave’s article goes much further into the production side in particular, noting that the most critical metadata on set is frequently only recorded in the script supervisor’s handwritten notes. In 2008, that was kind of a dead end. Today, thanks to relentless advocation by people like Dave, some tablet based applications can capture handwritten notes, but they’re not all that widespread. Acres of mission-critical data and metadata is still captured by hand, and often goes no further than a notebook on a shelf.
This is why I say that “entirely” is a strong word. You may not need Resolve or After Effects to do anything with a script supervisor’s notes, or your camera’s lens settings — but it’d sure be nice if they didn’t destroy those fields if you’ve filled them in. But it’d sure be nice if this plethora of metadata being written by all these different sources had some meaningful way of talking to each other when you wanted them to too.
Even if you’re only nominally interested in this, you won’t find sharper insights into the technical side of filmmaking than Dave Stump shares in this interview. It was actually put together by longtime COW member Gary Adcock, an associate ASC member who’s another Warrior for Metadata Integrity who was able to join us for part of the interview. It’s one of my favorite things to have ever put together in my 10 years here, and I think you’ll really enjoy it too.
Metadata and the Future of Filmmaking
-
[Brett Sherman] “At the moment I’m considering Keyflow Pro since it does seem to work with Finder level tags. And it would allow me to search across projects. One rule for me is not being reliant on any one application because in 20 years it’s likely that application will not exist anymore.”
Brett, as kind of an unofficial PR guy for them, I think you’re on the right track. I’ve been working really closely with KeyFlow Pro’s developers. There are bigger things in development for the app, but also improvements for many other functions, like how they handle metadata. Right now, metadata added to files in FCPX can actually be imported into KeyFlow Pro using the FCPX Agent app. This is pretty limited – but full metadata back and forth with FCPX is planned.
Personally speaking, it’s a more user friendly app for novices. I had our office manager log with in/out and keyword ranges every SOT for about 8 hours of footage we’re working with this month. There’s something amazing about that living outside of the NLE forever. In terms of being reliant on a single app, I’ve expressed the importance of KeyFlow being able to export all data that users put into it in CSV or similar format. I believe this will happen within the year.
By the way, the KeyFlow Pro NAB sale is back online until midnight tonight for $199 because a lot of folks were traveling back from NAB and didn’t get the discount.
I wouldn’t expect to see this price again for a while.
John Davidson | President / Creative Director | Magic Feather Inc.
-
What a fascinating and important article.
“Once we have metadata everywhere, everyone will look around in shock and awe and ask each other, “How did we ever make movies without this stuff?”
And then I looked at the date again and somehow after all this time we’re still not collecting all the metadata, we’re still not preserving it, and in many cases we’re not even using it when it’s there.
And of course we’re no nearer anything resembling open standards.
Simon Ubsdell
tokyo productions
hawaiki -
[David Mathis] “I believe you said in a previous post (feel free to correct me) is that text looks at the clip name. It turns out, my apologies if I missed it in a previous post, that text can look at notes and other metadata that is used.”
Thanks for the correction, David. I did not know that, and yes, it’s very useful as you say.
Simon Ubsdell
tokyo productions
hawaiki -
[Simon Ubsdell] “Very interesting way of looking at it.
But in a sense every container in any application is a saved search, isn’t it? Or do you have a more specific idea in mind?”
No, I don’t think bins are a saved search. That clip will always be in that bin. If I search for a clip, and it’s in a bin, that clip will always be there. If I search for a clip in FCPX, I can then assign another search parameter to it (like a new keyword or other piece of metadata) and that clip will now show up in multiple places, with my original search, and the new search. Both of those “searches” are now saved. I can also select both of those “searches’ in FCPX, and the Browser will display everything inside of it, and then I can go skim around all of the clips. With bins, I can only see what’s inside one bin, not multiple.
With bins, the file is in that bin, and usually nowhere else (besides timelines, of course). I suppose you could duplicate clips and place them in multiple bins, but I never did that with bins, mostly because the master clip relationship can get strange.
With collections (keyword and otherwise) clips can be part of multiple collections. The clips aren’t assigned a location as they can be anywhere (or nowhere). It is much more fluid and dynamic than bins, at least the way bins work in FCP7 and even Pr.
-
OK, yes, I see what you’re saying. That’s a great explanation.
I’d really love to know how this is working at the coding level – we’re stuck with using analogies to describe the process, of course.
Simon Ubsdell
tokyo productions
hawaiki -
John, I know you’ve gone into this before in a lot of detail but could you give me an idiot’s guide to the main selling points of KeyFlow Pro as they might relate to this broader question?
Simon Ubsdell
tokyo productions
hawaiki
Reply to this Discussion! Login or Sign Up