[Nat Jencks]

[Neil Smith] ” IP over Thunderbolt 2 is also going to be a viable option as soon as certain company sorts out their SMB stack.
“

Neil, I have been told that IP over thunderbolt will never be a good fast solution, since the Mac has to emulate an ethernet connection via software, and to do this stuff well at line speed requires purpose built hardware in the form of NICs… in other words, all those chips on the NIC are in fact doing something…

Do you feel that the current problems with IP over thunderbolt are just software issues and will be overcome if and when apple does a better job implementing this? I have never had good luck waiting for Apple to implement anything… even fixing critical bugs can take them years.

Curious if you think we will see fast reliable IP over thunderbolt in the near future.

best
-Nat

[Nat Jencks]

A a very basic level a good experiment is to load in a linear gradient and put it on your scopes, it should be a straight diagonal from 0 to 1.

Now when you move lift gamma, gain, offset, contrast, and the log grading tools you can see precicesly what they are doing to the gamma curve.

best
-Nat

[Nat Jencks]

Prores4444 is definitely not that bad, but have worked on VFX shots which had undergone many passes of Prores4444 compression, and you could definitely see the damage. Very subtle blocky patterns began to appear in subtle gradient areas like in mid tone sky.

The inability of Resolve to be able to export “Same as Source” in the way the avid or FCP can is a small feature that becomes bigger as it starts to become a tool that people if not editing are at least taking on some editorial tasks, such as joining a bunch or reels together to build an LP version of a film.

Perhaps a better example is if you rendered out a DCP in reels using easy DCP it would be nice to be able to join these together in a sequence and export an LP with audio without pushing the whole DCP thought a decode re-encode cycle. Clipster does this well.

Exporting “same as source” is a very useful feature for editorial tools and as Resolve becomes and editorial tool it should be one of the things on the list of things to implement. It actually comes up a fair amount with various editorial tasks.

best-
-Nat

[Nat Jencks]

I have experienced this bug many times. Always from timelines that were sourced from XML for some reason.

I copied everything into a new timeline and deleted the old, and that solve the problem.

best-
-Nat

[Nat Jencks]

[Bob Zelin] “you are looking at crazy specs. Nothing is doing 3000 MB/sec “

I’m sure this crazy number comes from filling the 12 drive array with high end 6G SSDs. obviously this is “internal” bandwidth as well not practical bandwidth to a single client. Even so maybe this number is bogus, but i’m not sure. 12x high end SSDs in a RAID0 with 6G backplane might hit 3000MBps. In any case thats not what I would do, I would be using 12x 7200rpm in RAID5 or RAID6, but hopefully that would still cover up to 1000Mbps in internal bandwidth.

[Bob Zelin] “You don’t buy the QNAP, with the 10G card in the QNAP, and then plug it into your client computer’s 1G native ethernet port, and get 275 MB/sec. And these cards cost MONEY !”

Understood, in the hypothetical setup outlined earlier I was using the netgear switch and two sanlink2 thunderbolt to 10G adapters.

[Bob Zelin] “buying
Toshiba desktop drives instead of HGST Ultrastar or Western Digital RE series drives”

FWIW the price I threw out there for the QNAP was based on it being filled with western digital RE enterprise drives.

[Bob Zelin] “your boss will fire you, and you will cry. “

I’m my own boss, but I won’t rule out crying 🙂

Thanks
-Nat

[Nat Jencks]

[Bob Zelin] “Now that I said this – I am sure that all the vendors will do exactly this – “this is what we can offer you for under 10 grand”. And yes – the QNAP solution that you specified will work as well. “

Thanks Bob, I really do appreciate the feedback. Its been a while since I shopped SAN, and I didn’t realize that it was possible to get anywhere near the QNAP or Synology level of performance for the just under $10K level.

Thanks for the feedback, time for me to talk to some vendors.

I’m hoping to be able to hit the SAN/NAS with two clients each pulling up to a single stream of 2K DPX (~275MBps), so an aggregate of ~700MBps with some overhead with up to 275MBps heading out over each 10Gbps connection… Do you think its reasonable to expect this level of performance from a SAN/NAS under $10K? A year or two ago I would have said of course not. Looking at the specs of a box like the QNAP or Synology with a claimed aggregate bandwidth of 3000MBps and dual 10Gbps connections I would hope yes, and perhaps other vendors have solutions that can do the same.

Or maybe this is a pipe dream 🙂

best-
-Nat

[Nat Jencks]

Listen, everyone calm down 🙂 Except Bob. Thats fine.

I am not on a crazed quest to take the bread out of the mouths of the vendors who provide custom post production shared storage. I’ve purchased, used, and supported countless shared storage systems, avid unity’s to fiber jet, facilis, the works, some of them area amazing.

I appreciate the hard work that vendors put into creating solutions.

But i’ve also watched countless areas in post which used to be dominated by purpose built hardware change over time, and get to a point where commodity off the shelf hardware can do the job.

Maybe 10GB NAS solutions are not such an area yet.

But you cannot simply discount commodity hardware unconditionally.

Like I said, there may be specific reasons why one of these types of solutions is a bad idea, but thats exactly what I”m looking for, is the specific reasons why it wont work. E.g. sustained throughput is OK but latency is terrible so there will be annoying pauses when you press play… Or the management interface is horrible… or SMB connections like those used are not suitable and will lead to data corruption… or whatever.

Lets look at the example of the QNAP I linked and which Bob called out:

QNAP TS-1279U-RP populated with 2TB drives (24TB) = ~$6500
LAN-10G2T-U 10GB card for QNAP = $~700
Netgear XS708E = ~$900
Thunderbolt to 10GB Sanlink2 Boxes: ~$600 each.

Being yelled by Bob Zelin for wanting to do things on the cheap with commodity hardware instead of the tried and true gear that will actually work = priceless.

Like I said, I get it that this is probably a bad idea, and there ain’t no free lunch is this business. But what I want is the WHY? Like an annoying 3 year old.

A total of $9300 for entire package including 10GB NICs for clients and NAS, and 24TB of storage is pretty darn good. Its not ridiculously less than the turnkey purpose built post solutions but it is less.

(ducking as infuriated resellers and vendors hurl rotten tomatoes)…

best-
-Nat

[Nat Jencks]

[David Roth Weiss] “You’re looking to take a server designed to share Word docs, MP3s, and JPEG images, and wondering why it won’t deliver shared storage performance for video. If it were that easy, not one of the companies advertising shared storage here would be in business.”

I certainly appreciate that being true in the past, and perhaps your right about these two devices, but it looks to me like in the last year or so some of the “IT class” type devices on the market have really stepped up to the point where they are close to competitive with traditional post production market purpose built solutions.

But maybe I’m wrong!

But both the QNAP and the Synology devices I linked to claim over 3000MBps throughput (aggregate internal) and use SSD caching etc.

Is it really fair to classify these type of devices as something intended for word docs and jpegs?

Specifically what problems could one expect when using a device such as the linked Synology or QNAP rather than a purpose built solution?

Are you certain that an 8 drive Promax Platform would outperform either of these devices in terms of bandwidth and latency?

Does anyone out there have direct experience with either the QNAP or Synology devices in the class I linked to? Please note that obviously QNAP and Synology both make devices with radically lower internal bandwidth, and until recently even their highest end NAS devices capped at 1000MBps internal bandwidth as opposed to the 3000+ available in these type of next-gen devices.

Vendors please take note, I am not attacking the very fine products which you sell on these forums, just looking for specific concrete feedback!

Thanks!
-Nat

[Nat Jencks]

[Bob Zelin] “the way that it typically works ( for performance) is to bond the server ports to the switch (that supports link agg) – then you get more performance from the server to the switch – but the clients (when you bond them) does NOT increase the performance – only gives you redundancy, not increase in speed. The only exception I know of for this, is the trick (that I don’t know) that AVID is doing with the ISIS 5000, where you run 2 cables to the ISIS switch (Dell Force 10 S25), and you get about 150 – 160 MB/sec. This process is NOT link aggregation – I don’t know how they are doing it. “

The process of normal port aggregation is supposed to increase bandwidth, but I 100% agree with you that in practice with Mac OS X it does not in fact seem to.

In Apple’s document here:
https://support.apple.com/kb/PH14045

they say:
“Combining ports provides increased bandwidth by merging the bandwidth of the individual ports. Network traffic is also balanced across the ports.”

But your totally right, it doesn’t seem to work.
Oh well, this was just a rainy day project to bump up my network file copy speeds.

Thanks for your feedback Bob and confirming that what I am seeing is in fact “normal”.

I’ll wait a little and either go with a 10G ethernet SAN if I really need it!
best-
-Nat

[Nat Jencks]

[Bob Zelin] “If you had two new Mac Pro’s or two old Mac Pro’s (or a combination – one new and one old), and setup bonding on the ports, and hooked up TWO Ethernet cables (with or without the switch), your bandwidth would NOT increase – you would get 100 MB/sec transfer rate. “

Thats the part I don’t understand. Why doesn’t bandwidth increase when hooking up two ethernet cables and creating a bond? Using link aggregation typically DOES result in higher bandwidth.

Not so using OS X port bonding? Is OS X port bonding different than typical link aggregation?

Best-
-Nat

p.s.
side note: re my application, I have two workstations which currently have high speed DAS and that works OK for me. The DAS attached to each machine is providing 700MBps which works well for the DI applications I need it for. It not practical for me to try and replicate this performance in a SAN environment at the moment, and I’m not trying to.

But I do occasionally like to move files between the two systems. For example I might render out a film on the primary machine and then move the render to the secondary machine to start an encode of a bluray or DCP while I continue work on the primary machine.

I currently copy files between the two machines using standard gigabit ethernet using SMB protocol. It works fine. However it would be great to have copies go at twice the speed and I thought that was how typical LAG link aggregation worked.

Similarly I already have a cheap synology NAS DS412+ which I just use for nearline backup. Although I done NEED to have higher throughput on this device (or any of these devices) it would be really nice, and it does have two gigabit ethernet ports which can be configured for link aggregation as a single bonded port and which in theory (at least to my understanding) should provide higher bandwidth to the device.

Thanks Bob 🙂