-
Rendering to Metasan vs Local SCSI storage
-
It can be normal to experience some “lag” when working on a file-level sharing SAN such as metaSAN (as well as Xsan, Unity, Melio FS, and SANergy). The reason is that unlike a local drive, a file-level SAN client must refer to a central authority (the metadata controller or MDC) prior to accessing the storage. The role of the MDC is to maintain the integrity of the file system when multiple computers share access to it. It must therefore authorize each file i/o request. This request is sent over Ethernet, and when authorized, the client can access the file directly over fibre channel.
There is usually only one metadata transaction per file. When working with large files (such as audio and video clips), there is no visible impact. But when working with lots of very small files, you can start to feel it. Opening projects that refer to hundreds of clips or browsing very large folders are two cases involving large amount of metadata processing.
You will experience more sluggishness in environments where there is already lots of traffic on the office LAN. This is why a fast metadata controller and a dedicated network can only improve performances. There are also usually a few settings that can help optimize SAN performances during rendering operations, to minimize the amount of metadata. But in the end, most people feel that the numerous benefits associated with file-level sharing far outweigh this side effect…
Bernard
BTW The reason there is no need for metadata processing on a volume-level SAN (SANmp, Facilis, FibreJet) is because only ONE client can write to the SAN volume at any given time (others can only read from it). There is therefore no risk of file-system corruption. However, the burden is put on the user to manage their data and projects across multiple volumes… File-level sharing eliminates this constraint and allows everyone to simultaneously access the same volume.
-
I agree with Bernard that there will inevitably be a slight difference between local storage and network storage.. but that being said, we regularly work on HUGE projects and have not had any (noticable) slowdowns in After Effects and very little problem with any other app.
Our biggest issue was with 1080 multiple streams to many boxes, but this was an isolated instance, and since the files are HUGE, we were really just pushing the limits of what our storage can deliver.
Now for some questions…
What type of storage are you running? What type of fibre switch are you using and are you sure the ports are configured correctly? How many clients do you have connected? I assume you still have the issue when you only connect one client? Are you running a dedicated Metadata master, and also are you running a dedicated network for your metadata?
-Chris
Chris Irving
Director / Motion Graphics – VFX Artist
https://www.chrisirving.com -
Hi Michael. I Have the same problem like you. When I Open a folder with 20000 TIFF at HD1080 from MetaSan volume, I need more than 1 hour for display the files.
Do you have any news about this? Would you please tell me and help?
Reply to this Discussion! Login or Sign Up