-
iMacs and Fusion Drives
-
[Jeremy Garchow] “Do you, personally, saturate an 8x PCIe bandwidth? Is it even possible?”
The worst disk-bound render I have personally done was a pretty simple composite of a multipass 3D render, but it was a large format 1-minute long animation that weighed in at 972 GB. That’s about 16 GB/s, or about 550 MB per frame. In my case, the main challenge was probably the image sequences, not a pure bandwidth issue, and if it hadn’t been disk-bound, it would have been CPU-bound, but I’m not sure where that cross-over would have occurred. No big deal, because that’s not an every day occurrence, but it’s the worst real production bandwidth story I’ve got.
I can also totally saturate my PCIe SSD RAID thingie for Ae’s disk cache with large raster projects in floating point, but I think that’s only 4x PCIe and the bottleneck may be the RAID controller or the disks. This does kind of happen every day. I should switch it over to full-on flash storage and see if I can fix that.
Editors can comfortably ignore me, because 1 GB/s is a whole lot of data, but for corner cases like these… trust no external bus. The bandwidth is out there.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events -
[Walter Soyka] “The worst disk-bound render I have personally done was a pretty simple composite of a multipass 3D render, but it was a large format 1-minute long animation that weighed in at 972 GB. That’s about 16 GB/s, or about 550 MB per frame. “
In what format? What on Earth needs 16GB/s and how would you realistically play that back?
[Walter Soyka] “I can also totally saturate my PCIe SSD RAID thingie for Ae’s disk cache with large raster projects in floating point, but I think that’s only 4x PCIe and the bottleneck may be the RAID controller or the disks. This does kind of happen every day. I should switch it over to full-on flash storage and see if I can fix that.”
Meaning you need to playback 40Gb/sec or Ae is such a dog that it can’t playback anything in realtime without writing it to cache? See what I did there?
-
When working in heavy 2D or 3D compositions you’ll have multiple passes of each scene. This means you’re working in, normally, 16 or 32bpp and X passes. It’s not hard reaching the insane figures noted above since you’re working lossless compressed at best.
1080p25 @ 16-bpp RGB we’re talking over 300MB/s per pass. Jumping to 4K we’re talking over 1.3 GB/s per pass. A normal scene can easily have 4-5 passes – i.e. saturating a few GB/s isn’t all that hard even at HD resolution.
However, most often other things end up being the bottleneck. And these are quite extreme cases.
-
[Erik Lindahl] “When working in heavy 2D or 3D compositions you’ll have multiple passes of each scene. This means you’re working in, normally, 16 or 32bpp and X passes. It’s not hard reaching the insane figures noted above since you’re working lossless compressed at best.”
Sure, but are these played back in real time, typically?
[Erik Lindahl] “1080p25 @ 16-bpp RGB we’re talking over 300MB/s per pass. Jumping to 4K we’re talking over 1.3 GB/s per pass. A normal scene can easily have 4-5 passes – i.e. saturating a few GB/s isn’t all that hard even at HD resolution. “
Right. As mentioned earlier, when working uncompressed, you will need the bandwidth, SSD will make a difference, and you most likely aren’t working on a Mac, anyway.
-
I don’t think you normally work in real time when doing these kinds of work no. Most comp-apps aren’t built for it. However, i/o can be a limiting factor for sure even if you’re not working in real time. AE for example has very aggressive caching to speed rendering / workflow up which frees the CPU but strains the I/O. If TB2 would be “limiting factor” I’m not sure about though. But seeing you have to potentially load X GB of frames before the CPU or GPU can process anything and some features requires temporal data… It can for sure be a limiting factor.
Going from internal HDDs to SSDs and a new external raid made a HUGE difference on my old 2008 MacPro even with limiting buss- and PCI-speeds. I/O matters but it depends on what you’re doing and how much.
-
[Jeremy Garchow] “In what format? What on Earth needs 16GB/s and how would you realistically play that back?”
What Erik said. It’s not for realtime playback. By outputting multiple passes from a 3D app (lights separate from shadows separate from diffuse materials, depth maps, object buffers for isolating specific elements, etc.), you can exert an incredible degree of control over the look of the shot in post in a couple minutes or hours instead of hours, days, or weeks to re-render in 3D.
As I said, it was disk-bound — CPU was sitting idle, waiting on frames to get read in from disk for processing. (I am sure this would not have been real-time even if I did have the bandwidth, but it was slowed by not having the bandwidth.)
It’s a corner case, but I just wanted to illustrate that big bandwidth needs are not that far off.
[Jeremy Garchow] “Meaning you need to playback 40Gb/sec or Ae is such a dog that it can’t playback anything in realtime without writing it to cache? See what I did there?”
You’re a clever one!
Actually, the cache system is pretty brilliant. Reads from the cache are relatively fast, it’s writes that are slow.
Consider a 4K comp, RGBA, floating point (32 bits per channel). That’s 194 MB per frame. The Ae cache saves not only the final composite of all the layers in a comp, it actually saves out sub-renders of each layer. It’ll write a frame to cache for anything that’s faster to read than it is to calculate again, and it retrieves from the cache according to footage/frame and effects/properties. In other words, once cached, Ae can retrieve “frame 101 of clip A.mov with Blur applied with 1.5 radius” anywhere it’s used in any project on the system.
When a frame is 194 MB, 1 GB/s is not enough to keep up. If you have a 10-layer 4K floating-point comp, that’s about 2 GB per frame, or about 60 GB/s.
Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog – What I’m thinking when my workstation’s thinking
Creative Cow Forum Host: Live & Stage Events -
[Walter Soyka] “What Erik said. It’s not for realtime playback. By outputting multiple passes from a 3D app (lights separate from shadows separate from diffuse materials, depth maps, object buffers for isolating specific elements, etc.), you can exert an incredible degree of control over the look of the shot in post in a couple minutes or hours instead of hours, days, or weeks to re-render in 3D.
As I said, it was disk-bound — CPU was sitting idle, waiting on frames to get read in from disk for processing. (I am sure this would not have been real-time even if I did have the bandwidth, but it was slowed by not having the bandwidth.)
It’s a corner case, but I just wanted to illustrate that big bandwidth needs are not that far off.”
I hear you, but we are speaking about completely separate processes. Sure, you need things to go as fast as they can go, but overall, it’s still a slow process. Thunderbolt, perhaps, may drag you down a little, but it’s not going to throw a wrench in the system, and I’m sure, for you, SSD would be worth the speed/money if you could can manage the decreased capacity.
[Walter Soyka] “Actually, the cache system is pretty brilliant. Reads from the cache are relatively fast, it’s writes that are slow.”
So why can’t I play a simple “ken burns” in real time in Ae without burning it to cache? 😀
I know, I know, it’s too smart, and my needs are too little. If I need 194MB/frame, I’m covered!
No really, I kid Ae. The global cache system is very good and seems to only get better. It is time for a performance enhancement to Ae, though.
Supposedly, it’s in the works.
-
It’s true that AE could use an enhanced engine but on the other hand I’d prefer it remains a useful tool in any circumstance and doesn’t go from amazing real time goodness to unusable after you reach some breaking-point (yes, looking at you Motion!). And the mentioned frame-sizes would be the same in any app being relatively intelligent in it’s rendering.
-
[Erik Lindahl] “It’s true that AE could use an enhanced engine but on the other hand I’d prefer it remains a useful tool in any circumstance and doesn’t go from amazing real time goodness to unusable after you reach some breaking-point (yes, looking at you Motion!). And the mentioned frame-sizes would be the same in any app being relatively intelligent in it’s rendering.”
There’s no question that Ae is the proverbial Jobsian truck.
Ae is the software that I have used the longest during my time in the video industry, even tough I am not a motion graphics designer.
There is no need for it to be Motion (I mean, there’s Motion for that), very true, but there are days when it feels really creaky, despite all of the enhancements. See here:
https://forums.creativecow.net/readpost/378/2618
-
Hi Walter,
Thanks for partaking in this discussion.
I’d like to get some feedback on an iMac editing system in addition to the fusion drives. Do you suggest I post in the “FCPX Techniques forum?”
Reply to this Discussion! Login or Sign Up