Neil Smith
Forum Replies Created
-
Hi Adam,
Like BobZ points out, there are many shared storage solutions that can easily (and inexpensively) meet your requirements without costing you an arm and a leg.
Where are you based?
We specialize in providing shared storage solutions to Media & Entertainment customers in the LA area and deal with these kinds of requests all the time … there’s quite a bit bit of editing and film based workflow done around here 🙂 … many of our customers are looking to move away from propriety fibre channel based topologies to more open NAS based approaches … 10GbaseT networking has come along in the last few years.
If you haven’t chosen a solution yet, there’s some questions you should be asking of any vendor you talk to in order to ensure that their proposed solution matches your workflow requirements at your budget level.
If you need to talk things through, give us a call and we’ll be happy to give you some pointers. Or just search through the Cow for some of BobZ’s other threads … super smart chap and totally focused on providing no BS advice to any company looking to move away from Isilon or StorNext based SANs.
Cheers,
NeilNeil Smith
CTO
LumaForge Systems
shedding light on storage matters
310-271-7546
http://www.lumaforge.com -
Excellent RFP, Scott … you’ve neatly summarized what’s going to become the standard requirements for modern post-houses.
Our LumaNAS 3.0 solution addresses precisely those requirements you’ve highlighted … a client/server architecture built around a very efficient ZFS file server NAS with a 10GbaseT network to the client workstations (Mac, PC or Linux).
With standard Cat6/7 cabling and 10GbaseT NICs we can deliver a sustained 800MB/s throughput which is plenty fast enough for HD, 2K and even 4K workflows … we can certainly add an 10Gbe SFP+ NIC (or even InfiniBand) for connectivity to your switch and Data Center, but if you haven’t checked out recent developments in 10GbaseT switches and Cat 6/7 cabling you might just want to see how far they’ve come and their amazing the price points.
The cool thing about our LumaNAS 3.0 approach is that we can easily integrate Mac (both Thunderbolt and the old PCIe towers) Windows and Linux clients through a simple NFS mount so you can have Adobe CC running across whatever platform delivers the best price/performance.
Even though you’re not running a 4K workflow at the moment our solution easily scales to handle 4K ProRes and even UHDTV DPX pipelines if you need it.
Open sourced ZFS is the the most advanced file system and volume manager available today … much more efficient and reliable than old fashioned block-based technologies like Xsan and StorNext and in terms of price/performance it’s unbeatable. Its built in data integrity and sophisticated replication and snapshot functionality means that you no longer need to install expensive FC NICs, FC switches and FC RAID controllers … just use 6G or 12 SAS HBAs and JBODs which can be both spinning disks and SSDs … the super smart caching algorithms inherent with ZFS means you can get the best of both worlds – HDDs for cheap large capacity and SSDs for high speed performance!
The other thing about the LumaNAS 3.0 platform is the easy integration with Object-based cloud storage … the M&E world is starting to find ways to incorporate cloud based elastic resources and storage and open sources platforms like OpenStack provide tremendous cost savings over proprietary based solutions like Quantum Lattus.
The future is open .. NAS based storage based around ZFS with efficient connectivity to OpenStack cloud storage is going to become an integral part of many post-production shared storage requirements.
Check out our website for more details on LumaNAS 3.0 … if you’re any where near West Hollywood, we’re actually giving a ‘collaborative workflow’ demonstration this afternoon at 5pm on The Lot …. we show 6K DRAGON and 4K ProRes dailies being processed on 4 clients simultaneously all connected over Cat 7 cabling to the back-end LumaNAS 3.0 file share with 96 TBs of HDD storage and 8 TBs of SSDs.
We can easily scale the solution to handle a petabyte of data if and when you need it – high density JBOD enclosures are really cheap now and 6TB drives are falling in price every week.
Let me know where you’re based and if you need any more information.
Cheers,
NeilNeil Smith
CEO
LumaForge LLC
Advanced Digital Workflow
323-850-3550
http://www.lumaforge.com -
Steve,
I heard an interesting theory a couple of weeks which I don’t know whether to believe or not – appreciate your thoughts on the matter:
1) there’s some kind of SEC rule (don’t quote me on this – needs verifying by someone who knows for sure) that publicly traded companies like Apple Inc. have to go through a four year process if they want to drop support on a product.
2) part of that process is that they have to stop selling the product for four years
3) technically, Apple no longer sell Xsan as a separate product … its included for “free” in OSX.
4) they stopped charging for Xsan a couple of years ago.
5) my conspiracy theory friend then postulated that in two years time Apple will drop Xsan from the OS as well.
6) a year after that they can also formally drop support for the product.
Have you heard of such a process and what do you think of the ‘theory’?
Of course, it would be helpful if Cupertino would give us some clear roadmap of Xsan developments but they won’t of course … but be aware that though Apple don’t ever comment on public forums they do monitor the relevant ones all the time and your comments will be duly noted in the ‘big black book’ … I’m already in the book of naughty boys 🙁
The question kinda goes to the nub of how serious Apple is about the Pro market … think of what they did to ‘Shake’ and FCP 7 … if they’re going to stick with HFS+ (now an antiquated file system) and drop Xsan (essentially CVFS from MountainGate and now owned by Quantum) what are their intentions with shared storage and Thunderbolt networking?
Neil
Neil Smith
CEO
LumaForge LLC
Advanced Digital Workflow
323-850-3550
http://www.lumaforge.com -
For HD 1920×1080 ProRes 4444XQ at 30 fps this is true (with no alpha channel) … but not for other resolutions and frame rates obviously.
Cheers,
The Other NeilNeil Smith
CEO
LumaForge LLC
Advanced Digital Workflow
323-850-3550
http://www.lumaforge.com -
[Steve Modica] “All that being said, I think the SAN complexity will always be a burden. Ethernet is easy and it sucks in and adopts all the best elements of SAN. All your metadata stays in one place. Dead clients don’t kill the system and you only need one simple network. “
Nice summary of why and how things are changing, Steve … it’s funny how the wheel comes around full circle … I was in DEC during the 80’s when we started to implement VAX Clustering and DecNet and deal with emergence of TCP/IP LAN networking versus SNA/Token Ring … when I think back to the size of the machines and the costs involved to do what we now do on a couple of beefy Macs, PCs and Cat 6 10GbaseT networking, it’s kinda unreal.
It seems to me, that you don’t want to bet against Moore’s Law in the M&E industry now that digital acquisition and file-based workflow are the norm … 10Gb/s ethernet is pretty cost/effective now with 40 Gb/s on the horizon … IP over Thunderbolt 2 is also going to be a viable option as soon as certain company sorts out their SMB stack.
Interesting times indeed … are you going to the ‘Creative Storage Conference’ tomorrow? … if so, let’s grab a coffee some time.
Cheers,
NeilNeil Smith
CEO
LumaForge LLC
Advanced Digital Workflow
323-850-3550
http://www.lumaforge.com -
Bravo, Mr Zelin!!!
Chris, please read Bob’s reply carefully and stop being a DickHead … not only are you trying to pick the brains of a highly experienced and talented engineer for free and rob him of his livelihood, you quietly slip in that you’re happily using a Hackintosh on set for mission critical data back-up of production dailies.
Have you checked recently with a little company in Cupertino on how they feel about you using their patented IP in a an illegal way? (read the EULA that comes with OSX) … have you also explained to the producers of the shows that you’re DITing that you’re knowingly using illegal and unsupported systems and that if you loose any of their precious production content they won’t have a leg to stand on?
Get real, dude.
Neil
Neil Smith
CEO
LumaForge LLC
high performance workflow
323-850-3550
http://www.lumaforge.com -
I agree, Bob .. we’re testing NFS and SMB shares over 10GbaseT as we speak … unfortunately both Apple and Microsoft have taken it upon themselves to make multi-vendor integration between Win 7 and OSX as difficult as possible … NFS runs super well under OSX but SMB2 sucks … SMB is great under Win 7 but have you tried setting up a msft NFS share recently? … sheer bloody madness.
The FCP X 10.1.1 Xsan constraint issue is another story .. and while, I understand why Cupertino might want to take such an approach for technical reasons, it sure as hell makes our life difficult.
Really liking ZFS as a shared storage file system … if I recall correctly from our conversation at NAB (amazing that was already two months ago) you also favor ZFS and NFS wherever possible (I may have got that wrong, so don’t quote me) … maybe, next time you’re in LA, you should come over to The Lot and we’ll share notes … we’re building an Open IT PoC (Proof of Concept) demo that leverages ZFS/BSD and 10GbaseT plus Thunderbolt networking … in this day and age, you’d think it’d be easy to connect a Mac to a PC and share some video files in a safe and reliable way without spending an arm and a leg … you’d think.
As well as getting away from Xsan there is another company who also is positioning their proprietary file system as the integration glue for multi platform shared storage in the M&E space (has a lot in common with Xsan – cough, cough) … very robust approach for sure, but again cost is an issue with a FC topology.
At the Creative Storage Conference in Culver City on June 24th we’ll be demoing our Open IT ZFS based approach … if you’re going to attend the Conference be sure to stop by our Booth # 7:
https://www.creativestorage.org/2014Agenda.htm
I’ll also be giving a 10 minute overview in the opening panel in the morning session … plus, they’ll be plenty of heavy-iron vendors will also be present if folks have money to burn 😉
Hope to see you soon, matey … keep up the good work.
Cheers,
NeilNeil Smith
CEO
LumaForge LLC
high performance workflow
323-850-3550
http://www.lumaforge.com -
[Steve Modica] “NFS was working great with 10.1, and then they released 10.1.1 and broke the locking.
With 10.1, you could enable locallocks on the clients and they would adhere to the lock files in the Library directories. The only possible concern would be if two clients tried to create lock files at the exact same moment.
With 10.1.1, apple started using NFS locking (using rpc.lockd) and that messed things up.
First, if you use locallocks, since the client can’t see the files are locked on the server anymore, it just blows through them and overwrites the locks. So clients will corrupt Libraries if they simultaneously access.
(You fix this by removing “locallocks” which makes sure they heed the locking)Second, they seem to have broken things when a second client tries to access a locked Library. Now, the second client hangs. It has to be killed. Nothing gets broken in the library, but the hang on the client isn’t very nice.
I’m not sure if this is our implementation of rpc.lockd or what. I’d like to know if anyone has this working.
Our lockd is pretty standard.Steve
Steve Modica
CTO, Small Tree Communications”That’s what I was trying to hint at in my earlier posting … the chaps from Cupertino may have a different idea on how they want to deploy and manage shared storage under FCP X 10.1 … highly suggest that peeps talk to Apple Support directly before they get too far down un-supported topologies.
Neil
Neil Smith
CEO
LumaForge LLC
high performance workflow
323-850-3550
http://www.lumaforge.com -
Which 10GbE card is installed in the Mac Pro? … exact model number if poss.
NeilNeil Smith
CEO
LumaForge LLC
high performance workflow
323-850-3550
http://www.lumaforge.com -
If you have an Xsan FC SAN set up with StorNext client on the Windows machine it will work … FCP X is only designed to share projects in an Apple Xsan shared storage environment .. it will not work across other protocols … it’s the only way Apple can guarantee performance levels and data integrity.
Xsan (which Apple OEM from Quantum who sell the StorNext SAN system) writes its own File System to the RAID volumes so you will lose your NTFS and HFS+ partitions .. make sure you back up all your data first 😉
metaSAN does not use its own propriety File System but writes a ‘Traffic Cop” bit of code to each SAN Volume which manages the Read/Write priorities and file locking plus maintaining the Volume Directory integrity. If you want to stop using metaSAN you simple disengage the “traffic cop” and your native File System takes over and you keep all your existing data … if you want to stop using Xsan/StorNext you have to completely re-format your RAIDS back to NTFS or HFS+ and lose all your data.
Neil
Neil Smith
CEO
LumaForge LLC
high performance workflow
323-850-3550
http://www.lumaforge.com