Metadata, P2CMS, P2Log Pro, LogAndTransfer, Raylight and You. A journey into the unseen.

[Jan Crittenden livingston]

Hi Jeremy,

I think one of the most powerful parts of the workflow is the metadata upload feature which you can do from either the P2CMS P2 Viewer or the P2 Log programs. This would give you the ability to come back in from the field with the clips already named like you mention above.

Then by using the Raylight as the interface for ingest and transfer, it can be a very powerful tool in sorting your clips based on User Clip Names. The way they worked the Iditarod this year is that they named the metadata files after each of the checkpoints. Raylight ingesed from 10 different cameras and cards clips that were recorded with the metadata files and it sorted clips into folders based on the metadata.

I think that initially it seems odd to have to use the Raylight but the fact that it maintains the metadata is the most intereting part because that is the power behind the file based workflow.

I am in the process of writing a paper describing how to go about this, and am willing to share it, trust me it is not finished, just contact me offline and I will send you a rough.
crittendenj at us.panasonic.com

Best,

Jan

Jan Crittenden Livingston
Product Manager, HPX500, HVX200, DVX100
Panasonic Broadcast & TV Systems

[Jeremy Garchow]

[Jan Crittenden Livingston] “I think one of the most powerful parts of the workflow is the metadata upload feature which you can do from either the P2CMS P2 Viewer or the P2 Log programs.”

Right, I forgot to mention that. While this seems like THE way to do it, I just can’t trust that every shooter I work with will know what to do with it and that we will be able to foresee exactly what we will be shooting. For instance, if we are shooting Broll for a week, there’s no way we will be able to predict what we are going to get. So in that case, renaming after we shoot is the way to go. I wish every shoot was organized by checkpoint. That’d be sweet. Is there software for that? 😉

[Jan Crittenden Livingston] “I think that initially it seems odd to have to use the Raylight but the fact that it maintains the metadata is the most intereting part because that is the power behind the file based workflow. “

Undoubtedly. That’s what I am trying to harness.

[Jan Crittenden Livingston] “just contact me offline and I will send you a rough. “

Email has been sent. Thanks for your response.

[Bob Woodhead]

I think the most significant aspect not realized in your outline is that working with clips on different edit stations is really very easy with Raylight. There’s an option in Raylight to save the newly created QT reference file in the same location as the MXF. So my workflow is:
– transfer clips (either off camera or off cards) via P2CMS onto a 2 drive RAID 1 array. (Mirrored drives for safety.)
– Run Raylight on those files
– (if desired) copy everything to an edit RAID (raid 5)
– do a simple Finder search for .mov files on the folders. the search results (a QT file for every clip) can be dropped into FCP.
– storing the QT ref files with the MXFs means that you don’t need Raylight after the initial processing (and also eliminates the possibility of changing options in Raylight later, and thus potentially mucking up your FCP references)

So Raylight can be a “run once, use many” operation.

No help here on AVCIntra…. 500 owner.

“Constituo, ergo sum”

Bob Woodhead / Atlanta
http://www.CoolNewMedia.net
Quantel-Avid-FCP-3D-Crayola
Panasonic HPX500

[Bob Woodhead]

[Jeremy Garchow] “For instance, if we are shooting Broll for a week, there’s no way we will be able to predict what we are going to get.”

In a best case, you’d have some PDA-thingy that’d let you edit a text file on a SD card. Wish I had one. (we use a laptop when we’re not out in the boonies) Barring that, you could pre-define broll categories. Shooter selects metadata based on what they’re pointing at. Then rename (if needed) clips once they’re imported as QT refs in FCP.

[Jeremy Garchow]

[Bob Woodhead] “There’s an option in Raylight to save the newly created QT reference file in the same location as the MXF”

Thank you for your response.

The way I understand it, though is that if you move the MXF to another raid with a different name and folder structure, that relationship is now gone and has to be rescanned which means you have to have a copy of Raylight on every potential workstation. I am sure that would make dvfilm.com very happy and for that I don’t blame them, but I am just talking practicality here. Am I wrong about that? From the raylight mac help page:

“Other Locations

Locations other than P2 Cards or volumes containing P2 Card images, for example a P2 folder on your desktop, may be scanned by adding them to the “Other Scan Locations” list.

If you select “Store Links in Same Location as Source Files” then the links will be stored in the same folder as the video MXF files. This can make it easier to organize the links, however, the MXF files must be rescanned if they are copied to another folder or drive.

The links created by Raylight may be copied or moved to any other part of your computer, including other disk drives, and they will function normally.

If the MXF files are copied or moved anywhere, however, then you must remake the links by clicking “Scan Now.” Use the “Overwrite Existing Link Files” option if you need to replace existing links with new ones. In some cases an old link will still function but there is a delay of several seconds while the system searches for the MXF files. If this happens in the Quicktime Player, you can save the link using File->Save and the link will be updated. If it happens in FCP, open the link with QT Player, then File->Save the link.”

And don’t you find P2CMS kind of weird? Do you ingest in the database or just export with verify?

[Jeremy Garchow]

[Bob Woodhead] “Barring that, you could pre-define broll categories. Shooter selects metadata based on what they’re pointing at. Then rename (if needed) clips once they’re imported as QT refs in FCP.”

Yeah, I could see how that would be cool. I just don’t see it happening in reality, but maybe that’s just me and haven’t put it in to practice yet. What we need is a wireless text shooter from the data wrangler to the camera. Logging while shooting.

[Matt Gerard]

Right now I don’t think that there is a workflow that will do everything you want. That’s why you’re posting here, right?:-) You need to be able to have someone on set with a handheld/laptop/iPhone connected directly to the camera, and rename and log clips within the camera. i think this will actually happen, its jsut a matter of when.

In the meantime, what do we do? It depends on your workflow that you need. Its rare that there is a solution that covers every situation. Personally I dont’ like the idea of editing my shows with reference files, its one more thing that can get mucked up. I haven’t done any testing as to relinking files that get archived and refs that get deleted/moved, but right now I have more disk space than time, so I log and transfer. I use a bare drive in my weibetech enclosure and archive the raw P2 data, put it on a shelf. FCP was made to work with quicktime movies. Not MXF files. until Apple opens the platform to recognize other file types, there will not be a reliable way to get metadata from the P2 card into FCP. For some reason, QT’s don’t jive with P2 metadata.

Anyway, enough of my rant. As an editor, I despise logging and digitizing. I’ve always hated it. I can log and transfer, its faster than realtime. I like that. Its one step forward without using some fancy footwork of a workaround that has a greater chance of failing (computer error or human error) and sticking me near my deadline.

These are my OPINIONS, this is the way I work. I know everyone works differently. Most people don’t mid using raylight, and it is a cool idea, and it seems to work. But not for me.

Until log and transfer recognizes the metadata within the mxf files, I don’t think there will be an easy way of doing this.

Matt

Its more fun to ride a slow motorcycle fast than a fast motorcycle slow…

[Matt Gerard]

Sorry for the rant. To summarize, yell at Apple.

Matt

Its more fun to ride a slow motorcycle fast than a fast motorcycle slow…

[Josh Wallace]

Had to come up with a feasible workflow like this for a big, fast turn around, multi P2 Cam shoot in New Zealand earlier this year.

After much testing the best workflow I could find involved;

Using P2 Log Pro to create metadata to load via SD Card to each of 7 Cameras (HVX’s and HPX500’s).
Metadata included Camera Op name, Description of what they were shooting that day but most importantly a UserClipName that ID’d each camera and created incrementing UserClipNames in camera, ie A CAM TUES 001, A CAM TUES 002…
I would do this for all the cameras before the beginning of each days shoot.
( I think there’s also now a free Panasonic tool for creating and uploading metadata to camera so don’t go out and buy P2 Log cos it’s day is done! )

Next step were the Logging and Transferring stations ( PowerBook G4’s ) where full P2 Cards from the field were copied to the edit machine. For this we used P2CMS – this is the one thing it does well and relatively simply, a verified copy of the Contents of each card.
( It does a whole lot of other stuff involving a database or something but forget about all that, it’s confusing as hell.)

Raylight for Mac was installed on the edit machine, you only need it on one machine.
This was setup to scan the edit drive and create Quicktime Ref files for any new MXF files that arrived. It was also set to create new names for these QT Ref files based on the UserClipName from the metadata and the actual MXF file name, this is great, in a matter of seconds Raylight will generate QT Ref clips with meaningful names that are ready to edit in FCP and that you can track back to original material if necessary.
I set it to make these QT ref clips in the same folder as the original MXF Video files so each card still exists as a self contained Folder that could be moved around and still be Imported into FCP using Log and Transfer or can be edited immediately using the Raylight QT Refs.

We actually had another step here where we would open the Raylight QT Refs up in FCP on another networked Logging machine. Here a Logger would make logging notes to help the editor and then save a Batch List back to the CONTENTS Folder of the card in question.
At this point we would use Finder to Label the Folder Green for Go and all the editor had to do was open each Batch List in each Green Folder and do a quick reconnect to the Raylight QT Refs and away he went. Hundreds of Logged, HD Clips with meaningful names in FCP.

Say you transferred a Folder representing a complete P2 Card from one drive to another, the clips would appear as offline in FCP but all you have to do is reconnect to one Raylight QT ref file in the new location and FCP will connect all the files in that folder.
It’s just if you started moving the QT ref files in relation to the MXF files that they are referencing that you would have real problems.

It worked pretty damn well. Occassionally the Raylight clips would open in FCP sans audio. When this happened it was a simple matter of deleting the QT Refs for that card and on it’s next automatic scan Raylight would recreate them and second time round the audio came thru in FCP. ( I think this was due to the network traffic and activity on the drive rather than a Raylight flaw. )

At the end of the job everything was relocated to San Fran for a seperate edit and they did have trouble reconnecting all the QT Ref clips in FCP – In New Zealand we were doing Daily edits with no problem but in San Fran they were dealing with all the media from 7 days so i think it was just too much Quicktime Referencing going on!

I think the answer to my prayers might be just around the corner though…This looks like the business, superceding Raylight, especially if it integrates the camera metedata into FCP the way FCP should!!!

ttp://forums.creativecow.net/thread/8/973195

Last post.

[Jeremy Garchow]

Wow. Thanks a ton for posting this, Josh. Much appreciated.

Unfortunately I haven’t had time to do much more research and testing.

I was alerted to this new product as well, but it doesn’t help with UserClipName within FCP but it allows direct editing of MXFs right in FCP among other features, without the need for reference movies:

https://www.calibratedsoftware.com/

Once I get through this next shoot I am hoping to get into preset metadata, but I know this won’t be a reliable workflow as I won’t be able to teach it to everyone and I personally won’t be on every shoot either.

Raylight seems like a stop gap to me so I can’t commit to it right now.

Thanks again.

Jeremy