Slowwwww sparse bundles?

[Ivan Radovanovic]

I’m not FCP X editor but I wonder why is everybody ignoring NFS as an option when a lot of servers can export NFS share?
I have tested with our NAS couple months ago, 1G connection, 110 MB/s Read and Write. Sparse bundles are always slower.

[Ian Johnson]

I had never considered (or configured) NFS.

Our current setup is a Mac Pro with a set of SAS raids sharing over a LAN. Can NFS be setup on a system like this?

[John Heagy]

The short answer is yes absolutely.

It helps if it’s OS X Server and 10.6 where it’s a simple file sharing setting. In 10.7 and 10.8 Apple has eliminated this setting so a bit of Terminal work is needed or use NFSManager.

https://www.bresink.com/osx/NFSManager.html

John

[Michael Hadley]

Here’s a thought. Ever run Disk Warrior or similar utilities on a sparse bundle? Don’t know if it’s possible but maybe that might help improve speeds?

[Eric Newbauer]

Hi Ian,

Yes, usage can affect the performance of your sparse volume. Counterintuitively, the performance problem can be caused by fragmentation. (This is something we’ve seen in our lab and can reproduce easily by just using generic stuff.)

FWIW, I’d like to point out that our EVO storage server also provides NFS. Please have a look if this is of interest to you:

https://www.studionetworksolutions.com/evo/features/

Eric Newbauer
Studio Network Solutions (SNS)
https://www.studionetworksolutions.com

[Craig Alan]

I ran some tests using both AJA and Black Magic software. All my media is on a 4TB internal raid on a Mac Pro. I have projects stored In Sparse Images. The results were that the sparse image read/write speeds were very close to the parent raid speeds. Sometimes faster. What would be considered good read/write speeds on a software raided media drive? Or more importantly good speeds for editing in FCP X?

And related but a different problem –

I need a way for students in a FCP X lab to NOT have access to each other’s media and projects. The only way I have come up with so far is storing the projects in sparse images and setting up a student user account with simple finder in OS X. In simple finder, sparse images are grayed out when you do an import search in FC. And completely invisible in the finder. I’m hoping there are no gotchas. This would be whole lot easier than setting up tons of user accounts and setting up permissions for every external folder.

If I mount a sparse image using the admin account then the sparse image shows up in FC. They could store all their files in that sparse image and then log out at the end of their session. Then I unmount the image and load the next one.

This will also make help transfer user projects to a different computer or to the trash or to a back-up. The media will be stored on an external raid. I have not tested this on external storage. The media drive I’m using is a software raided drives in a Mac Pro.

My first plan was to use password protected sparse images but copying media to an encrypted image takes way too long.

Do you see any problems with this plan and would it work equally well with a server?

Mac Pro, macbook pro, Imacs (i7); Camcorders: Panasonic AG-HPX170/AG-HPX250P, Canon HV30/40, Sony Z7U, VX2000, PD170; FCP 6 certified; write professionally for a variety of media; teach video production in L.A.

[Caspian Brand]

[Craig Alan] “I need a way for students in a FCP X lab to NOT have access to each other’s media and projects. The only way I have come up with so far is storing the projects in sparse images and setting up a student user account with simple finder in OS X. In simple finder, sparse images are grayed out when you do an import search in FC. And completely invisible in the finder. I’m hoping there are no gotchas. This would be whole lot easier than setting up tons of user accounts and setting up permissions for every external folder.

If I mount a sparse image using the admin account then the sparse image shows up in FC. They could store all their files in that sparse image and then log out at the end of their session. Then I unmount the image and load the next one.

This will also make help transfer user projects to a different computer or to the trash or to a back-up. The media will be stored on an external raid. I have not tested this on external storage. The media drive I’m using is a software raided drives in a Mac Pro.

My first plan was to use password protected sparse images but copying media to an encrypted image takes way too long. “

Hi Craig,

Keep in mind FCP X only allows for one user per Event/Project period. No Read users, regardless of network file system. You can, however, import media w/o copying from another active Event/Project. Not that it sounds like you’re looking for simultaneous users in the same Project like an Avid workflow, but just stating for the record on how FCP X behaves in general.

The workflow you describe actually lends itself very well to a volume managed solution. We have a software product called SANmp which enables workspaces to be assigned per student just as you describe over iSCSI. Permissions can be easily set so that student volumes would only be visible to the assigned student and their instructor, not by other classmates, regardless of which computer the student logs in from. SANmp is very popular in educational media production environments like this for it’s ease of setting up such permission schemes as you described.

Common class media can be centrally stored on it’s own volume(s) for Read Only access by all students and Write access for the instructor. A CLI interface is even included for scripting so you can auto-mount volumes with a specific user’s account. It is available as stand-alone software to manage access to iSCSI targets with your existing storage, as well as with our turnkey Shared Storage platform called EVO.

With our more advanced EVO solution, SANmp is included free at no extra per-seat cost. EVO provides more advanced file level managed workflows with NFS, as well as AFP and SMB in addition to high performance media workspaces using SANmp. With EVO you can:
-configure an all Sparse Disk Image workflow over NAS protocols
-configure an all high performance media sharing workflow with SANmp volumes per Class/Student
-configure a combination of the two, with central/common elements for everyone on a high performance media volume and student specific edits/renders/projects on Sparse Disk Images
-configure XSAN over iSCSI and use the FCP X ‘Add SAN Location’ in conjunction with XSAN volumes/permissions

As has been mentioned by others on this thread, performance of Sparse Disk Images is not as consistent when introduced in a network compared to using them on local disks, so care should be taken on a properly configured RAID as well as keeping your production network ‘clean’ and keeping your storage system over as direct a path to the edit systems as you can without traversing a bunch of routers or switches.

Regards,
-Caspian

Product Specialist
Studio Network Solutions

[Craig Alan]

[Caspian Brand] “Keep in mind FCP X only allows for one user per Event/Project period. No Read users, regardless of network file system”

On the mac pro I have been experimenting with, I set up a student account with a simple finder. If a sparse image is mounted on the teacher account (the account that it was created in), the student account can open it’s events and projects.

Or do you mean allow one user at a time?

Will I be able to use Apple Remote desktop to see what they are doing in an open project and help them if needed?

[Caspian Brand] “performance of Sparse Disk Images is not as consistent when introduced in a network compared to using them on local disks, so care should be taken on a properly configured RAID as well as keeping your production network ‘clean’ and keeping your storage system over as direct a path to the edit systems as you can without traversing a bunch of routers or switches.”

I have 20 computers and external raids for each. Since I had heard that FCP X was not working well in networked facilities, I decided to keep each computer paired with its own media drive. Also I did not get the funding all at once so it was put together piecemeal.

Is there a problem with sparse image performance even on a thunderbolt connected raid to a single computer?

Your solution is definitely‎ networked : so are you using volumes within the server storage instead of sparse images with software to assign ownership to a volume? Are sparse image speeds compromised?

Could we use the thunderbolt raids in such a system? Maybe daisy chained together as a large network storage. They are pegasus raids with four hard drives each.(8TB total, but can be changed out).

What would be a ballpark figure for this type of package for a 20 seat computer lab? Please keep in mind that in our school district we do not have a budget. What I do is apply for grants and I need to know ahead of time how much things cost so I can put in a request long before I get to order anything. It’s a very time consuming, overly complicated way to fund programs. But it is what it is.

Mac Pro, macbook pro, Imacs (i7); Camcorders: Panasonic AG-HPX170/AG-HPX250P, Canon HV30/40, Sony Z7U, VX2000, PD170; FCP 6 certified; write professionally for a variety of media; teach video production in L.A.

[Eric Newbauer]

[Craig Alan] “Your solution is definitely‎ networked : so are you using volumes within the server storage instead of sparse images with software to assign ownership to a volume? Are sparse image speeds compromised?”

Hi Craig, we (SNS) have a few different ways of tackling FCPX workflows with our storage products, but the one Caspian mentioned is based on using our SAN volume sharing software (SANmp), and because of that we don’t have to deal with sparse bundles. It’s very simple this way, and as Caspian pointed out it’s a very popular solution for schools.

[Craig Alan] “Could we use the thunderbolt raids in such a system? Maybe daisy chained together as a large network storage. They are pegasus raids with four hard drives each.(8TB total, but can be changed out).”

Yes. We also have a product (Xtarget) that will enable you to take Thunderbolt storage and present it over Ethernet as SAN storage. You can then use your Thunderbolt storage with our SAN volume sharing software that I mentioned earlier.

[Craig Alan] “What would be a ballpark figure for this type of package for a 20 seat computer lab?”

There are several different ways to solve the problem you’re describing. We could do it all in software and you can use your own storage, or we could put together an entire solution: hardware and all. I’ll add some links below so you can try out our stuff for free, and if you like it please give us a call and chat with one of our reps about costs.

Here’s a link to our volume sharing software (SANmp):
https://www.studionetworksolutions.com/sanmp/

Here’s a link to the software that enables you to present direct attach devices as SAN storage (Xtarget):
https://www.studionetworksolutions.com/globalsan-xtarget/

Eric Newbauer
Studio Network Solutions (SNS)
https://www.studionetworksolutions.com