Import Error from SXS card

[C. Kauffman]

Gotcha. What’s CRC?
I have a file that somehow is out of it’s BPAV folder (it’s from a 2nd camera on a shoot, someone else did the download) that I can’t get Transfer to process. I know it’s there, I’ve used VLC player to watch the footage. Any reco’s on how to restore it?
The card is long since erased.
Thanks

Sony EX1
Canon 5d mark 2
FCP 6.06

[C. Kauffman]

BTW I picked up a 4TB Drobo to back everything up. So far so good.

Sony EX1
Canon 5d mark 2
FCP 6.06

[Craig Seeman]

[Chandler Kauffman] “Gotcha. What’s CRC? “

There’s a pref setting to turn this on in ClipBrowser. It basically checks to make sure the data matches the source. Here’s the explanation in Wiki.
https://en.wikipedia.org/wiki/Cyclic_redundancy_check

[Chandler Kauffman] “I have a file that somehow is out of it’s BPAV folder (it’s from a 2nd camera on a shoot, someone else did the download) that I can’t get Transfer to process.”

Import the raw MP4 into ClipBrowser and it’ll create a new BPAV. Use File/Import.

[Tom Laughlin]

Ok, so Craig, questions, here’s my problem, for a while I have several BPAV folders either not import, or it takes multiple attempts to make sure they copy all the way. Ones that do import, I’ve got several that are not giving me clean video. I get all sorts of green pixel lines and artifacts in my video. What can I do to take these out? I’ve tried converting from the card, no luck. And, with other forum info I’ve tried, using Clip Browser, it shows up in Clip Browser playback. I’ve copied the original BPAV to a card, from the archived external hd, and using Clip Browser, made a copy of the BPAV to see if it fixes the CRC, but the copy is a copy of the corrupted BPAV, it doens’t affect the quality, or am I doing it wrong? I’m running the latest version of Clip Browser, XDCam Log & Transfer won’t work, and the video is not clean when testing it through a mac with just leopard, no snow leopard.? What do think I can do?

Tom Laughlin
Producer/Editor
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel

[Craig Seeman]

[Tom Laughlin] “using Clip Browser, made a copy of the BPAV to see if it fixes the CRC, but the copy is a copy of the corrupted BPAV,”
CRC only ensures it matches the source. You need to use ClipBrowser for your initial copy from the cards. Once a bad copy is made, all things stemming from those copies will continue to be bad.

Play the source clips in ClipBrowser. That’s not using Final Cut Pro’s EX codec. If it’s bad in Clip Browser then the files themselves are corrupted. If it’s good there post back and see if we can figure out where the trouble spot might be.

[Tom Laughlin]

It appears that out of about 300 BPAV folders or more, we have about 77 with corrupted video. We are using a few things that I’m not sure if they are the source of the problem or not. We are often hard-lined SDI into a switcher for events, so I’m not sure if it is a bad SDI-in port in the camera or not, so you have experience with this and if quality suffered?

In the field, we are shooting all over and often inside offices, and we use wireless video transmitters and the IDX battery mounts with IDX batteries powering the wireless system and the camera all at once. I’m not sure if airports and their security systems are ‘frying’ the cards and the footage on them, but many times, we play the video, about 95% of the video is playable with no artifacts, when played inside the camera. Maybe the artifacts are too small to see in the viewfinder window, but we don’t see them.

I don’t see them when the video is played and video is captured back to FCP. It is when the video card hits the computer. I know we had once, it happen where we converted from the card and the artifacts went away, but weren’t able to consistently replicate that fix, it fixed a few cards, but most of the artifacts went away, not all, on the cards where that fix worked.

How can a Sony XDcam record hours and all the video end up corrupted? Can Transcend cards corrupt the media recording capabilities of a XD camera? Again, this is a significant amount of cards that are or had recorded and corrupted. We have used the same cards and backed them up over and over, is it possible for cards to go bad after many uses?

Do you know of anyone that has had or is currently having this problem? We are also having some editors having to actually edit around these artifacts, in FCP, they are editing out frames, and some of the video is not as bad as others, but I’m in a real pickle about this, as we have hundreds of hours that are being logged and so far, into this particular project with over 1TB of video, about 70% is somewhat artifact”y”.

Maybe it is the camera, but we have A and B camera both having issues with this artifact problem. This one project was shot on and off over a period of three months, so, there’s no consistent way to know how the cameras were reacting in the field, when in playback, the video looked fine.

I read a forum about CRC, I think you posted it, in which you talked about not using Mac’s “Finder”, and I’m not sure if that is part of the problem, as most of our work with off-loading was drag and drop and verify files by exact size, we assumed it was a clean transfer and the rest was history. I’m thinking more and more that it is camera recording, because we have always had either clean video, and Finder transferring happened, in every way. I don’t think we’ve ever used Clip Browser, and the concept of CRC checking wasn’t emphasized with the guys we got the cameras from.

I know many others who have had no problems with drag and drop using Finder, and this CRC, how crucially important has it been for you, in terms of what is actually does? And would this cause or contribute to media corruption? Is there any suggestions you have and can I email you some screen-grabs on Monday?

– Tom

Tom Laughlin
Producer/Editor
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel

[Craig Seeman]

I have no clue what your workflow is from your post. It’s all over the place.

Damn I feel like a FRIGIN BROKEN RECORD.
Whether from SxS or SDHC
ClipBrowser with CRC – NO EXCEPTIONS EVER!
Beginning and end of it!

You have no idea where the weak point is in your workflow because you seem to have no system beyond creating multiple weak points.

SDI out is another story but that has NOTHING TO DO WITH BPAV.
If you’re using SDI in the field you better have the field techs to QC ON LOCATION.
Certainly SDI to devices like nanoFlash is sound but you still have to check connections and test.

Hard Drives are a danger zone. Files can corrupt even if drives don’t crash. BACK UP IS IMPERATIVE and IMMEDIATE. I prefer Optical Disk (Flavor is you choice), others use well tested tape drive technology.

When you’ve done ALL THE ABOVE then you might consider the camera at issue. Currently you have NO WAY OF KNOWING.

Yes cards can develop bad sectors but you have so many holes in your workflow you have no idea where the source of your problem. I can’t imagine the cameras themselves to be at issue. Of course if the slot contacts (or card contacts) get dirty that can be an issue.

[Tom Laughlin]

Craig,

Sorry to stir you up, it’s not my intention to upset anyone.

This is our workflow:

1. We shoot using the (2) Sony EX-3 using 32GB Transcend Class 6 Cards. (1080p/24fps SP-mode). We always stop and power off the cameras before we eject the cards. We never let the camera keep rolling and “roll into a new card”.
2. We transfer the cards BPAV Folders to 500GB Rugged Orange drives using Mac’s Finder.
3. We verify the transfer by opening the file’s info and seeing that the exact file size matches the original file on the card.
4. We play the original card and verify no video is corrupted.

Later on,
1. We try using Clip Browser to play the BPAV Folder from off the hard-drives, and also importing them using XD Log and Transfer.
2. The clips playback with several green artifacts that look like static pixel lines, or lines of video that are pixeled. Some lines are green and some are lines that are thicker than others. Some are smaller and longer, etc, etc. The audio is fine, just the video.
3. We have had some projects where there is no artifacts and we have not altered any part of our workflow. We shot on brand new cards, and off-load them once the files are transferred to a HDD.
4. We try using FCP for the conversion process, as well as the XDCAM Log & Transfer to edit the video. The artifacts are not seen in the playback on the camera. They show up after the transfer or the conversion to .mov.

In reference to our workflow, we haven’t changed one thing, as mentioned in your response, we are simply trying multiple ways of trouble-shooting this artifacts problem. Interesting that it is happening with two separate cameras both having the same issues.

Can cards go bad only after 2-3 re-uses? Can a Sony camera’s recording and internal recording mechanisms go bad? Could the hard-drives, as you mentioned, be corrupting the media? Would e-Sata be better than a Firewire 800 Transfer? Do you think that a card can be fixed using CRC checking even after the transfers? How did you learn about CRC checking and has this been something other people have experienced, in terms of artifacting? How can video not be corrupted on the cards, yet with using 3 different transfer test methods, the cards still getting artifacts? You don’t have to respond to this post, I know this is over-kill.

Thanks a ton,

Tom Laughlin
Producer/Editor
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel

[Craig Seeman]

[Tom Laughlin] “We always stop and power off the cameras before we eject the cards. We never let the camera keep rolling and “roll into a new card”. “
There’s no reason to do this and if you power down while still writing to the card you will corrupt the file.

[Tom Laughlin] “We transfer the cards BPAV Folders to 500GB Rugged Orange drives using Mac’s Finder. “
Wrong way to do it. ALWAYS use ClipBrowser with CRC ON.

[Tom Laughlin] “We verify the transfer by opening the file’s info and seeing that the exact file size matches the original file on the card. “
That does NOT verify that data was corrupted. That’s the point of CRC checking.

[Tom Laughlin] “We play the original card and verify no video is corrupted. “
Short of playing the video from end to end you can’t possibly verify everything. The card can be fine. The copy can be bad.

[Tom Laughlin] “We try using Clip Browser to play the BPAV Folder from off the hard-drives, and also importing them using XD Log and Transfer. “
Too late. You need to use ClipBrowser when copying from the card. If the copy is corrupted you’re only maintaining the corruption at this point.

[Tom Laughlin] “The clips playback with several green artifacts that look like static pixel lines, or lines of video that are pixeled. Some lines are green and some are lines that are thicker than others. “
Proves my point. Obviously you haven’t verified your copy in any technically meaningful way.

[Tom Laughlin] “We have had some projects where there is no artifacts and we have not altered any part of our workflow. We shot on brand new cards, and off-load them once the files are transferred to a HDD. “
See my past comments about driving without seat belts. You may have just had a car accident without wearing a set belt.

[Tom Laughlin] “We try using FCP for the conversion process, as well as the XDCAM Log & Transfer to edit the video. The artifacts are not seen in the playback on the camera. They show up after the transfer or the conversion to .mov. “
Sorry but this makes no technical sense. You can’t play the .mov in the camera. Are you copying the BPAV back to the camera? What are you playing in the camera? Are your copied BPAV playing fine in ClipBrowser?

Tom, I’m not trying to be unkind. Professionally speaking your workflow is flawed. It’s high risk. It should not be done that way. It’s an accident waiting to happen and have just happened.

Check COPIED BPAV are flawless in ClipBrowser with clips displayed at full 1920×1080 (Option key + Return) to avoid scaling. This isn’t using the Quicktime EX codec so it eliminates any issues relating to that codec and/or bad re-wrap to mov. Please check a known bad file.

[Tom Laughlin]

If you have a minute: tomyboy342000@yahoo.com. I’ll email you my cell number.

Thanks for your timely responses.

Tom Laughlin
Producer/Editor
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel