FCP P2 workflow – Organization, Storage, and AVID migration

[Scott Sheriff]

Hi,
Have you not hired an editor yet?
If not, it is a real shame to wait until the last minute, a disaster in the making. Seems to be a lot of ‘if’ in your project. I would get the editor position filled and make a decision on platforms. Then, let the editor get the workflow organized.

Sorry this isn’t actually answering your questions directly, but I think the scope of your questions indicate the need for an editor to be present on your project.

Scott Sheriff
Director
SST Digital Media
https://www.sstdigitalmedia.com

[Andrea Scott]

Yes, agreed.

However, as things go in the independent documentary world, that process of picking an editor might be slow. We are still shooting and have not decided when we will start cutting yet – also a problem in terms of the editor’s schedule.

So. Any jump-start I can get on all of this would be great, as a lot of the tasks will fall to me, or to my oversight. If it’s easier – assume we will be cutting in Final Cut (this is 90% certain – the Avid migration question is mostly a peace of mind/what if we MUST switch question), and let me know your thoughts accordingly when/if you have the time.

THANKS!

[Shane Ross]

Well, you just about messed up in every way possible. Sorry to be blunt, but it needs to be said. Doing ANYTHING with the footage before you hired the editor wasn’t wise. Because, editors have their own way of organizing footage, and most of them know the proper workflows for things, like tapeless P2…if you hire someone who is experienced with it. Sorry, but your lack of experience with it, and having gone forward with that lack has really made a mess of things.

[Andrea Scott] “1. Should we be saving our .MXF Files? “

Yes…oh dear lord YES. Do you capture footage from tapes then throw them away? Re-use them? I hope not. No, you backup the P2 cards to an archive drive…the FULL CARD structure. This now, is, in essence, you master tape. Archive this, back it up on two drives..something. This is all the footage you spent time and money getting. You should be treating it with kid gloves.

Please watch this tutorial: P2 Workflow with FCP 6

[Andrea Scott] “2. What is the best practice you have found for naming reels? I was thinking of implementing reel names with the date and a number (ie 100526_001) – with reel number sequential for whole of project. I know from tape days, reel names were very important. Is this still the case with P2 technology? “

Whatever helps you keep track of the footage. Personally, I label the cards either by project, then card number…so Andrew Jackson would be AJ001, AJ002…if A and B camera, AJ_A001, AJ_B001…etc. If for a series, or just generic footage, then by location and date. Los Angeles, May 26, 2010 would be LAX_052610…add an _A or _B for the camera. And YES, it is VERY important with tapeless media too.

[Andrea Scott] “3. What is the best practice you have found for naming files?”

Whatever works for you, to help you find and keep track of the footage. FCP attaches a global ID number to all the footage, and this doesn’t change. So you can change the name when you import, or after you capture. FCP will know what links to what.

[Andrea Scott] “4. As I mentioned earlier, we are currently daisy-chaining 2-TB G-RAID hard drives for the edit. We will likely have somewhere between 10 and 16 TB in the end.”

Do yourself a favor and don’t. G-Raids are all RAID 0…NON-Protected. Meaning a drive dies, you lose everything. And without backups…that’s hose you BIG TIME. No, you need to look into and invest in a RAID 5 or RAID 6 8-12 bay tower. MaxxDigital EVO HD series, or Caldigit HD ONE, or other reputable RAID manufacturer. And daisy chaining drives can get very unreliable after a few. Don’t cheap out now. Not when you put yourself in a mess by not backing up your footage.

[Andrea Scott] “5. There is a small chance we will work with an editor who uses Avid as opposed to FCP. Given that all of the files have been imported using Final Cut, and everything is currently organized in Final Cut – and given that we do NOT have original MXF files for everything that has been shot prior to this week – what would a move like this take? Is it even possible?”

This is why you needed to keep the MXF files. Avid accesses them directly and then imports them in a far different way than FCP does. If you take the footage imported by FCP to an Avid system, it won’t even be able to see the files to convert. The DVCPRO HD codec FCP uses only comes with FCP. Or you can get a third party decoder. Or you can install FCP on the Avid and then use the Avid to import the files as DNxHD…but that will take a lot longer than the direct consolidate of the original files, and you lose quality.

So I hope that you find an FCP editor, and that you back up all of your imported footage, so you have TWO copies of it.

Good luck.

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def

[Andrea Scott]

Okay. Thank you. I sincerely appreciate the help – if not the berating.

In response: Yes, it would have been great to have an editor from the beginning, but when you are working for 2 years to raise the funds to actually make the film – while shooting for a few days here and there once you do get some money – it’s very hard to hold off on doing anything with the footage (let alone hiring an editor) because you need to cut together that footage in order to get more money. Like I said, I just came onto the project, and this is the first time all of us are working with P2 – so, a little slack please.

Now, a few more questions in response: If we are changing file names (both in log & transfer and in the browser), will there be problems when/if we have to re-link to the now-being-saved mxf files? Or does the “Global ID” take care of this? (I can’t remember from tape days, but I do recall problems with reconnecting media…)
Also, if we go in and starting changing reel names, etc. – are we also going to have problems reconnecting media? Granted, for the stuff from the past, we don’t have the MXF files, so perhaps that’s not as much of an issue. These areas are just a bit gray for me and again, we want to make the best decisions moving forward…

Thanks again. And yes, I have already seen your tutorial. Another watch would be a good idea, perhaps.

[Shane Ross]

Well, you don’t need to hire the editor for the full duration. But you needed to hire an editor to get you started, to show you the P2 workflow and just get you pointed in the right direction. ALWAYS consult with someone in post before you start. It is wise to do it even before you SHOOT. Because I have been on so many projects where the camera used isn’t compatible with the desired editing system, or is so new that NO edit system supported it. Post is the last step to get the footage out the door, but since everything ultimately lands there, you need to consult with post before you shoot, to ensure things will go smoothly.

But that is for the future. The next project.

[Andrea Scott] “If we are changing file names (both in log & transfer and in the browser), will there be problems when/if we have to re-link to the now-being-saved mxf files? Or does the “Global ID” take care of this?”

The global ID takes care of this. BUT, you must use FCP for this. Once you transcode to Avid, that global ID can be lost. Timecode might be lost. Reel number too. It all gets reset. So at this point I think you are tied to FCP, so find an editor who knows FCP, or your Avid guy better bone up on FCP.

[Andrea Scott] “Also, if we go in and starting changing reel names, etc. – are we also going to have problems reconnecting media?”

Same answer as above. Global ID takes care of that.

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def

[Jeremy Garchow]

Investing in MXF4mac will save you boat loads of time and disk space. Also, the MXF workflow can transfer right from FCP to Avid. You can also preserve the appropriate metadata, leaving Quicktime in the dust.

Jeremy

[Shane Ross]

[Jeremy Garchow] “Investing in MXF4mac will save you boat loads of time and disk space”

How? Wrapping in QT takes up the same amount of space as the original MXF files when you shoot DVCPRO HD. And since you still need to copy the MXF files to a media drive, something that takes as long as L&T to QT does (OK, maybe a small smidgeon faster)…I don’t see how it saves time nor space. Surely not a BOATLOAD. Not unless you are talking about AVCIntra in which case, wait, FCP doesn’t have sequence settings for AVCI, so you need to use a ProRes timeline. And FCP 7 supports AVCI native, so again, just wrapping and copying to the media drive again…same time, same file size.

THe BIG advantage of MXF4MAC and P2 FLow is METADATA. It retains more metadata,and allows FCP to access that metadata, that importing into FCP does not allow.

[Jeremy Garchow] ” Also, the MXF workflow can transfer right from FCP to Avid. “

If he had the original MXF files, which he does not.

[Jeremy Garchow] “You can also preserve the appropriate metadata, leaving Quicktime in the dust. “

Sorta. I use P2 Flow to make a Batch List for FCP…mapping the metadata to columns in FCP. Then I Batch Import, converting to QT files. Since I am on a SAN with multiple editors, and we don’t want to pay for MXF4MAC for seven FCP systems.

Sorry jeremy, I disagree with these points. Time saving? How? You still need to copy the files to the media drive. And space savings? Not really. Same size, when you don’t transcode.

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def

[Jeremy Garchow]

[Shane Ross] “How? Wrapping in QT takes up the same amount of space as the original MXF files when you shoot DVCPRO HD. “

So now you have multiple copies of the MXF files and multiple copies of the QT files. When using MXF, you just have the MXF files and don’t worry about the QT, or archiving the QT. Plus the MXF structure moves to Avid easier.

[Shane Ross] “And since you still need to copy the MXF files to a media drive, something that takes as long as L&T to QT does (OK, maybe a small smidgeon faster)”

It’s faster than Log and Transfer, as is importing all of the media. Way way faster.

[Shane Ross] “Time saving? How? “

P2 Flow is a much faster metadata editor and organizer than log and transfer, much much faster.

Importing native MXF files in to FCP is way way faster than the rewrapping action of log and transfer. Sorry, but it’s true. Since implementing MXF4mac, our P2 workflow has sped up considerably, and if you need access to metadata and an Avid compatible MXF workflow, it’s a no brainer.

[Shane Ross] “Same size, when you don’t transcode. “

Not when you consider the size of the MXF files and the QT files.

[Shane Ross] “Since I am on a SAN with multiple editors, and we don’t want to pay for MXF4MAC for seven FCP systems. “

You would have paid for the cost of all of the them by the time you finished logging with L&T and then rewrapping.

[Shane Ross]

[Jeremy Garchow] “So now you have multiple copies of the MXF files and multiple copies of the QT files. When using MXF, you just have the MXF files and don’t worry about the QT, or archiving the QT”

No…you have ONE copy of the MXF files that you archived…or two if you do manual RAID 1 on drives like I do. Or the ONE set on your tape backup. And then the QTs that you work with.

With MXF4MAC, you have MXF files backed up, then MXF files on your media drive that you are working with. Are you telling me that you work with the ORIGINALS? Seriously? (and I don’t archive the QT…only the MXF)

[Jeremy Garchow] “It’s faster than Log and Transfer, as is importing all of the media. Way way faster. “

Yes, if you work with the masters…the originals. Which I cannot recommend. Archive the masters, copy them to the media drive and work on the COPIES. The copy process takes as long as L&T…and the same amount of space.

[Jeremy Garchow] “P2 Flow is a much faster metadata editor and organizer than log and transfer, much much faster.”

Faster metadata editor. Well, that’s apples/oranges, as FCP doesn’t really edit the Metadata. Yes, I agree here though. When dealing with metadata, MXF4MAC and P2 Flow are essential. But if you don’t have the need to do that, then it isn’t essential.

[Jeremy Garchow] “Importing native MXF files in to FCP is way way faster than the rewrapping action of log and transfer. Sorry, but it’s true.”

I agree. But then…again…you are talking about working with the originals, NOT copies. I am a bit surprised that you work from the originals. What if your raid dies…even a RAID 5 can go kaput. You have no backup? Or do you back up to LTO. OH! There’s your time. Or copying from the field drives that you bring in to the media storage..there’s your time! Or do you work off the field archive drives?

Can you explain your workflow going from camera to working in FCP? You must be copying something somewhere, and there is your time.

My basic workflow. Shoot…offload P2 to field drive, often two field drives, with ShotPut Pro. Bring drives to office. L&T into FCP, after I use P2 Flow to make a batch list. Begin working in FCP. While I am working, I then copy the P2 offloads from the Field drives to my main archive system…using another machine.

So do you shoot P2, archive to archive drives, then take those to your office, and IMMEDIATELY edit with those drives?

[Jeremy Garchow] “Not when you consider the size of the MXF files and the QT files. “

Same size. All FCP does is put into a QT wrapper. 16GB of P2 media, when log and transferred into FCP, results in 16GB of QT files. When dealing with DVCPRO HD or AVCIntra native. When you transcode AVCintra, yes, the file sizes are larger, but you don’t need to transcode. FCP does import AVCI native.

[Jeremy Garchow] “You would have paid for the cost of all of the them by the time you finished logging with L&T and then rewrapping. “

Again…what time? L&T is still faster than capturing from tape. And you MUST copy the footage to the media drives at some point. Same amount of time.

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def

[Jeremy Garchow]

[Shane Ross] “you have ONE copy of the MXF files that you archived”

One copy? We use two. Then there’s another on the editing raid, so three copies in total. Two are made right at the set, usually with ShotPut Pro.

[Shane Ross] “Faster metadata editor. Well, that’s apples/oranges, as FCP doesn’t really edit the Metadata.”

It’s not apples and oranges. When using L&T, you are editing the metadata of the shoot, albiet that metadata now lives in the FCP project only. Unless you don’t edit the userclipname, but I am sure that most everyone does. When using P2 Flow and MXF4mac, the edited metadata lives with the MXF files themselves, forever, and isn’t locked up in the FCP project. Also, QT does not include all of the metadata that is possible from the P2 structure.

[Shane Ross] ” I am a bit surprised that you work from the originals.”

I don’t. I work from a copy of the files that were backed up on set.

[Shane Ross] “But then…again…you are talking about working with the originals, NOT copies. “

No, I am talking about the process of transferring files in to FCP. Sending files to FCP from P2 Flow is way faster than log and transfer.

[Shane Ross] “Can you explain your workflow going from camera to working in FCP? You must be copying something somewhere, and there is your time. “

Shoot, transfer card to two drives on set, and if lucky edit metadata right there, if not save that for later.

Transfer one of those drives to raid, and if we need to edit metadata, do it then. Send MXf files from P2 Flow to FCP.

Since the shoot drives are usually out shooting, we would transfer the MXF files from the shoot drives to the raid anyway even when we logged and transferred (pre MXF4mac). This way, those MXF files get archived with the project, with the shoot drives also go in the archive. We still end up with at least three copies of the MXF files in the archive somewhere. In our workflow, you aren’t saving time by log and transfer as all the media transferring is happening anyway. We don’t log and transfer from the shoot drives.

[Shane Ross] “Same size.”

Again, when we work, we add the MXF files to the project, so we used to have the MXF files and the QT wrapped files. Now, we just have the MXF files, it’s half of the disk space requirement.

[Shane Ross] “Again…what time? L&T is still faster than capturing from tape.”

Tape? Who’s talking about tape? 🙂

[Shane Ross] “And you MUST copy the footage to the media drives at some point. Same amount of time. “

You are forgetting the time it takes to log the footage and rewrap. I am not talking about file transfers, those are just part of the process. P2 Flow and MXF4mac will save you loads of time as opposed to the extremely clunky L&T interface. You can batch name a selection of clips in P2 Flow faster than it takes to name two clips in L&T, let alone 250 clips.