-
Render Cache is NOT persistent on network attached drives
For those who read this on the Blackmagic forum, I apologize for the redundancy.
Last week I posted about optimized media becoming unlinked after a restart. Zero response from Blackmagic people. So after a few emails from fellow users who read my post, I tried trouble shooting the issue by developing a work flow around the problem.
What I have found, and it is 100% reproducible, is that Resolve 12 loses it’s link to the render cache if the cache clip is located on a network attached drive. All render files are unlinked. Not just optimized media. I have tried both AFP and SMB protocols with the same results.
This is a major problem for my facility. We have three Resolve 12 Studio systems, connected via 10Gb ethernet to an 80TB shared storage RAID all saving projects to a dedicated Resolve database server. In Resolve 11 our workflow worked splendidly. We could open any project in any room, do updates, save and re-open in another room. We could pre-render RED Dragon source clips on the timeline in one room, then open the project to grade in another. Linking to the cache clip worked fine. It is broken in 12.
If I locate the cache clip to an internal drive or USB/Thunderbolt/FW drive, the render files relink properly.
This basically defeats the purpose of using shared storage or a database server.
Now, yes, we could put the render cache on a locally attached drive. But as soon as I open the project on one of the other systems, the render files unlink and the timeline goes red. This is expected behavior as the cache clip is not mounted. However, when we open the re-saved project up on the original system, the render files stay unlinked and the timeline is red again. Since there is no way to relink the render files, the files in the cache are useless and just taking up space. Even FCP7 could relink render files.
Why is this a problem? Our workflow and scheduling of color suites is very dynamic. We use one system as the primary conform/online/export room. The other two rooms are for grading. Often, after a grading session is over, we wait for VFX shots. Or picture changes. By the time we get them, the grading suites are working on another project. What we do is open the original project in any available room, make the changes and save the project. Then the colorists open the project in their room, regrade the new material and review with clients. Opening the project in the other room breaks the link to the render files for the project permanently.
No more realtime playback for the client. Which they complain about. Which embarrasses us. When we try to explain the problem we look like used car salesmen making up a story. Not good.
If this never worked in 11, I would say “OK Fine, I’m thinking ahead of the curve here. Resolve can’t do this.” But it used to do it. I built my video department they way I did in large part because of this feature.
I can only hope that the fine people at Blackmagic are already aware of this huge bug and a fix is imminent. If not, I would be more than happy to assist them in anyway to pinpoint the issue further. This should be at the top of the list of fixes for 12.1.
Thanks.
Eric Lalicata
Owner
Anarchy Post
1811 Victory Blvd
Glendale, CA 91201
818-334-3300 -
10 Replies
-
I don’t see a solution beyond just manually rendering optimized media. Do that and you can relink very quickly and easily, since the files are in a static place that has a direct path visible in the media pool.
You can complain (with justification) that the feature doesn’t work as advertised, but if you need to get actual billable work done today, then to me, that’s the solution.
-
Please keep us informed with any progress on this.
I’m currently migrating our suites over to shared storage, albeit with local SSD RAIDs for cache (as some do with Fusion IO cards). It would be great to know if this is possible though if we ever want to have a project dynamically switch between suites.
Could it possibly be an issue with symbolic links vs hard links? Or perhaps the permissions of the cache folder? If the other system is a different user, it could certainly be a simple permissions issue.
-
I thought Resolve always lost its caches after a restart, no matter where your storage was.
Glenn
-
Render cache stays linked in Resolve 11. Just tested it again by rolling back one of our systems and duplicating the same timeline as in 12.
Of course you are right Mark. We could manually make transcodes and relink. But that is not the only reason to render a timeline. Often we are laying over vfx with built in alpha channels that need to be rendered for real time playback. Also we would not be able to access the raw settings of the clip without relinking to the raw clip, then re-transcoding. That could get tiresome pretty quickly. We would rather set the debayer settings to full and then lower on playback to achieve real time. But again, this only addresses source related render caching.
I don’t believe it is a permissions issue as the render files lose the link on the same system after restarting Resolve. Not a computer restart with a remount of the RAID, just simply quitting Resolve and then relaunching the app with everything else staying the same. I don’t know much about symbolic vs hard links.
If it is a config issue on our end I would love to know the fix. I can’t imagine we are the only facility using shared storage and trying to share render files. Resolve 11, FCP 7, Premiere 2015 and After Effects 2015 all have dedicated media cache folders on our shared RAID, and all of them remember the render files between rooms. Resolve 12 is the only one that does not.
I hope this is fixed in the next update.
Eric Lalicata
Owner
Anarchy Post
1811 Victory Blvd
Glendale, CA 91201
818-334-3300 -
hmmm
I can report with my Small Tree 10GB SAN (TZ8 with 48TB) I don’t have this problem at all. Render Cache is presistent on same machine through restarts of Resolve or the system and render cache is also presistent after render when opened up on other systems.
We have a mix of Mac, PC, LInux. All operating as expected
Robbie Carman
—————-
Colorist and Author
My Books
Online Training
Twitter
Blog
Mixing Light Powered By The Tao Of Color -
Hi Robbie.
Thanks for the info. Could you elaborate a little regarding which protocols you connect to the TZ8 with? The cache clip is definitely located on this volume? I know that unit is a Linux box using the ZFS file system.
My RAID is connected to a Mac Pro running Mavericks server software. AFP and SMB. Using Small Tree 10G cards. Maybe my problem lies within the Mac Server software. But like I posted earlier, there are no problems with Resolve 11 or any other video software. Only Resolve 12.
I will test further on Monday using a different NAS for the cache location and see if the results are the same.
Thanks.
Eric Lalicata
Owner
Anarchy Post
1811 Victory Blvd
Glendale, CA 91201
818-334-3300 -
Most likely what they need to add is a “refresh connection to Cache drive” or something like that, sort of like a Relink command.
-
Some further testing done today with no solution.
Trying to take the Mac Server out of the equation, I changed the cache clip location to a ReadyNAS Pro that we use for basic file serving. It runs a variant of Linux and is connected over 1Gig ethernet using both AFP and SMB. At first, the render files stayed linked upon restart. So, now I’m thinking it’s a server config problem with my Mac Server. Thinking I have isolated the issue and feeling adventurous, I optimized the three Red Dragon clips to ProRes 4444 using the same cache location. Everything worked until I toggled the “play optimized media if available” off then on again. After the toggle, the clips turned red and went offline even though they were still in the media page. Toggle back to Red Source clips and the media goes online. Clips are not red in color and they showed the correct image. Toggle optimized back on, clips go red and offline. Stays this way after multiple restarts. Then I tried to render the clips as before, and the render files do not stay persistent after restarts.
For fun, I mount the other rooms internal RAID volume via bonjour over our 1 Gig network. I set that as the cache clip location. Same problem with my main Mac Server RAID. Not persistent after restart. So I think I have ruled out a config issue with just the Mac Server. To be sure I double check permissions, propogate these down the directory of the cache location and recheck. Everything is coming up Read/Write for all users/groups.
I do not know what else to try as a test. Some help from BM would be great if possible.
Oh, and btw, the show is grading now and the render cache is set to an internal RAID in the Mac Pro. After a save and restart, all the render files that had been generated this morning were red again. Started the process over, and stopped for the day. Saved, quit and relaunched and all the render files come back blue this time. So, at this point, all I can say is I can’t depend on the new render engine for real world work with clients in the room. Luckily they were very understanding today. We will see how their patience holds up if it happens again.
Eric Lalicata
Owner
Anarchy Post
1811 Victory Blvd
Glendale, CA 91201
818-334-3300 -
What protocol are you using for your network drive? I’d suggest trying NFS instead of AFP (which is really SMB4). NFS is a bit more manual to mount, but generally less prone to disconnects and weirdness than AFP. It’s also quite a bit faster.
—
Joakim Ziegler – Postproduction Supervisor -
Finally found a solution to this issue. After upgrading to Resolve 12.1 this problem persisted. We have finally slowed down a little in preparation for the holidays, so I had a chance to do some investigation. I did not want to mess with a server upgrade while we were grading four projects.
After upgrading the Mac Server app to ver 3.2.2 Resolve began remembering the cache files and optimized media.
I wish I could give a more specific solution, but I’m not sure what the actual issue was or why it is fixed now.
These are the types of things that can drive a person nuts.
Eric Lalicata
Owner
Anarchy Post
1811 Victory Blvd
Glendale, CA 91201
818-334-3300
Reply to this Discussion! Login or Sign Up