Petros Kolyvas
Forum Replies Created
-
[Jeremy Garchow] “I have Neat and it doesn’t slow FCPX down when I am not using it.”
Same.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
Can you define:
[Mitch Ives] ” DOESN’T slow FCPX down badly?”
More clearly?
IE, do you mean it doesn’t require render time and create choppy playback… or simply having it installed slows down the app…
If it’s the former, noise reduction is very expensive, computationally speaking. Until three or four years ago, even audio noise reduction was a largely offline affair for the highest quality noise reduction algorithms.
Having said that, has anyone tried Photon Pro? Cursory research shows it’s as “slow” as neat video, but the results seem acceptable/good and it’s about 3/5ths the price with a much simpler interface.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
[Rainer Wirth] “Raid5, no partitioning, Setting of the raid for optimized read speed.”
Agreed on all counts. Sorry for the tangent.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
One other thing, for Windows arrays this information is probably going to the Event Log, but don’t quote me on that.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
Hi Bob,
So so sorry for the late reply. Life has been crazy.
[Bob Zelin] “1) do you disable heartbeat in the ATTO Configuration Tool, to prevent the firmware reset ? “
I do not. In fact, in my limited experience it was heartbeat’s firmware resets that prevented the array with the not-quite-failed-disk from taking down OS X’s entire I/O subsystem. So instead of an apparent system hang, the firmware reset would release the system for a more controlled/managed crash landing.
Again though, my experience remains anecdotal.
[Bob Zelin] “2) when you say “take a very close look” – exactly what do you mean by this. In the ATTO Configuration Tool, all the drives appear to be ON LINE (as well as the raid group), and then all of a sudden, a drive will be marked offline, or the RAID Group will become degraded, yet no drives are indicated as “failed”. The very amateur test that I do, when I see poor performance or strange behavior, is to run a simple diagnostic (like AJA System Test with the 16 Gig file), and watch the LED lights on the RAID to see if one of them is getting “stuck” while the others are flashing away. This sometimes will point to a failing drive, and I can remove that drive and install another. Sometimes I am lucky and the raid performance increases, and I avoid a disaster, and sometimes I am not so lucky, and the raid degrades. I wish there was a diagnostic that I could use (unless you can tell me how SMART can give you a hint that something bad is happening with a specific drive). “
Yes – they do appear to be ONLINE. And I fully agree with you, there’s this grey zone where it’s hard to figure out what’s going on.
I can only tell you what I did, and maybe it’s something you can add to your toolkit: I used the OS X system Console. I filter for ATTO firmware log entries (you can do so often by filtering by the model number of the HBA/RAID adapter and/or the manufacturer/vendor.)
In the case of the system where I experienced this, filtering by R680, all the disk activities are listed and there it would indicate which disk was pulling the volume offline. SAS firmware will often report disk activity (that is physical disks) and SMART changes directly to the console output. Each line is also coupled to the disk’s serial number so as long as you have a “recall” sheet and/or chart of which disk is which you can co-related the smart issue and or disk firmware feedback to the actual disk itself. For example, in the case I spoke of, the drive’s firmware would time-out, and then a short while later heartbeat would kick in and the controller’s firmware would reset. You could watch it all happen in the console.
I’ve included a sample output here with markup to help/clarify, but I fear I’m often obtuse with these things, especially since some of this was not entirely fresh in my memory, so if you can stand the wait, I’m happy to work through any more issues – I don’t pretend to have anywhere near a complete understanding of this and am happy to pool whatever knowledge I can.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
Partitioning can have a huge effect on RAID performance for spinning disks. However, it’s seek performance, not throughput.
The real trick is throwing away the rest of the space after the first partition. I’ve only ever done it on software arrays and never experimented with hardware arrays in the same way though it’s still possible through various RAID CLIs from what I gather.
The primary technique is called short stroking (let the jokes roll in). Requisite flaccid wikipedia citation: https://en.wikipedia.org/wiki/Hard_disk_drive_performance_characteristics#Short_stroking
Basically, spinning disks read from the outside in. Short stroking limits the heads to a small section of the outer edge of the disk platters, thus dramatically increasing random seeks as the head are limited in their movement; they only need to move a fraction of the entire disk surface.
So, following that, a technique was developed where one would partition a small portion of the outer edge of each disk, leaving the rest unused/unpartitioned. In my use I would partition 100GB of a 1TB disk. This would limit the heads to move a fraction of the space they normally need to move. Again, this is only possible if the rest of the disk remains unpartitioned/unused as any access outside the “shorted” zone will reduce performance back to normal levels.
It also has little effect on single-file reads, but for multi-stream audio and video before the time of SSDs and more affordable large arrays, it was an option.
I only ever used this technique on scratch/cache disks at a time when SSDs of that size (100GB or so) were far more pricy than their spinning-disk brethren. I also haven’t used it in 4 years.
One could, quite feasibly, partition the rest of the space for archival purposes, but make sure those partitions are not mounted when the high performance area is being used. However, in my opinion, SSDs at their current costs can be used for high performance arrays requiring massive IOPS, while spinning disks can be used for large media files.
In most cases though, save some hassle and just use an array of SSDs if need the kind of performance short stroking would offer.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
This response is anything but definitive and almost exclusively anecdotal.
We had an R680 that was repeatedly suffering “firmware resets” and it ended up being a dying drive; something removing the affected disk completely alleviated.
Regarding your specific points:
1. I don’t use auto-rebuild. I keep a spare spun up in each array, but I choose when rebuild occurs so I can’t help there except to say (and this will actually transition to point 2) there was some interaction with OS X causing the resets.2. SMART thresholds don’t, AFAIK, trigger a disk to go offline but rather offer a baseline of where one should “take a very close look.” In our case, the disk was useable to some degree, just incredibly slow and would run into sectors completely unreadable; but it didn’t fail outright and it seems OS X would cause the array to continually try a read or write operation that the failing disk would further affect – the firmware reset would happen almost as if the R680 decided to terminate the operation itself (though this is a wholly anthropomorphized version of the events.) Pulling the affected disk and rebuilding the array on-demand solved the issue.
Furthermore this is from ATTO’s heartbeat documentation:
Heartbeat
Choices: enabled, disabled Default: enabled
When enabled, the RAID controller’s firmware is required to respond to periodic activity. If the firmware does not respond, the system driver resets the firmware on the controller.So indeed, if the array cause an I/O issue such that the controller was stuck in a loop/could not complete the firmware would indeed reset.
I wish there was “working” or “failed” for drives but there’s a grey area a mile wide between the two unfortunately.
Sorry I couldn’t offer you more.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
Isn’t the config tool password an admin-level user account on the OS X host machine? It’s also in the Config Tool user manual.
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
My bad, I assumed due to the sub-forum, we were talking about X.
I’ve found over time, I’ve used Compressor 4 less and less as much of what I need I can do in X and I like Media Encoder better for all batch work that doesn’t require frame-rate changes.
Since you mentioned you don’t get this glitch with Media Encoder; can you use it to create the Proxies you want?
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger -
Have you thought about using GoPro’s CineForm Studio? https://gopro.com/support/cineform-studio-software-support
While I don’t really like the software, it’s what I’ve been using to pre-process any GoPro footage into ProRes before using it in either FCP X or PrPro.
TBH I’ve never used it for ProRes Proxy, but I can’t imagine it’d be much different.
Is there a reason you don’t want to import the GoPro .MP4s directly into X and use the Proxy option therein?
—
There is no intuitive interface, not even the nipple. It’s all learned. – Bruce Ediger
