Chris Murphy
Forum Replies Created
-
Agreed.
It’s possible bad data is coming from the array, but that should manifest as all sorts of problems including failed file system verification and scrubs.
In normal operation raid5/6 do not compare data chunks to parity, so there is no error checking beyond what the drives themselves usually provide. Any drive in the array that has unreported ECC failures (failure to detect or failure to correctly correct problems) will cause upstream problems that will be undetected at the raid layer, the file system layer, and the application layer. Hence the importance of periodic raid scrubs.
-
Almost certainly the offending process writing out that much data to disk is taking up some CPU time as well, so you can use Activity Monitor with the pop-up menu at the top set to All Processes, then click on the % CPU column to sort by process. The cause depends on the process involved. If it’s syslogd, then that means it’s writing out massive log files for some reason, and you’ll need to look at the system.log in the Console application to find out what’s being written and perhaps why.
OmniDiskSweeper may be useful in finding the location of the files taking up all of this storage. It’s a free app.
-
What model drives?
-
I think the responses thus far are heading in the right direction, it’s probably the projector. Have a tech go through the projector’s OSD menu, and video or photograph all of the current settings (cell phone video is fine, do whatever is easiest). Then find a setting to reset everything. Sometimes this is a global reset, sometimes it’s per page, sometimes it’s just for color. I’d reset everything. Then play your video and tweak the settings needed from there. sRGB is a reasonable assumption as it uses the same RGB primaries as Rec 709 for broadcast. But manufacturers are all over the map on hitting the sRGB primaries, and the tone reproduction curve.
If it’s still really bad, well, put up some color bars and see if this is a white point problem, or the primaries are just wrong, or what. If it’s an old projector it’s possible the RGB filters are decaying, and there’s not much that can be done about that. Get it as bright as you can and the white point reasonably neutral. The next step would be some way to characterize the projector, and then do a 3D transform of every single pixels on-the-fly to account for the difference between sRGB and the projector – although I can’t tell you what can do such on-the-fly 3D transforms, maybe someone else knows that. Otherwise the alternative is to re-render the whole film and bake the transform into what’s effectively an projector specific video, and I think Premiere can do that these days with ICC profiles.
-
Questions:
Do all of these problems go away when you point to the internal drive as scratch? And they return when you point to the thunderbolt drive as scratch? Have you tried removing the raid0 on the thunderbolt drive, formatting just one drive (or a partition of one drive), and making that scratch? Does that work, or do you still experience the same list of problems? Do you have example crash reports you can post to pastebin? When the problem occurs is anything reported in the Console application?
FWIW, Thunderbolt is PCIe, so it shouldn’t be seen at a hardware or operating system level as any different than a SATA card with some disks attached to it; which is all your internal drive is: a drive plugged into a SATA port on the motherboard which in turn goes to a controller on the motherboard’s PCI bus. So… the lack of support seems odd.
-
will I not be able to Read media from one and Write to the other?
I don’t understand the question.
This describes striping (raid0). Each read or write request by an application causes a read or write command to each drive in the striped set. In effect each file is chopped up into pieces and the pieces are placed “round robin” on each disk in the raid set. This is both how performance is improved (if the chunk size is set correctly), and why data is lost if one drive in the raid set fails for any reason.
-
Attempt 2 at being more concise and coherent on next steps. What *I* would do is not even mess with the dock anymore. I would call the manufacturer support, tell them you used Apple software raid1 with Disk Utility, and now you’re getting kernel panics on both a rMBP and iMac. And you’re happy to give them the kp report if they want. Questions for them: Is the use of Apple software raid1 supported with this dock, i.e. is it expected to work? Is it expected to work with OS X 10.7.5? Do I have the current dock firmware applied (if applicable)?
If the answers are yes, yes and yes. Then set up an RMA and send it back. I wouldn’t try to troubleshoot this specific unit anymore. All of the conditions are the same as before, so just trying again shouldn’t matter and if it does matter, it’s still unreliable. So I’d swap out the hardware.
-
don’t zero out the data and format each as Mac OS Extended (Journaled), correct ?
Yes. Zeroing isn’t needed, diskutil should wipe the metadata areas first, and then create new ones, then format the resulting logical block device as HFSJ. It shouldn’t matter what the file system is, the kp appears to be occurring before the file system is mounted, at the time the raid is being assembled.
If I configure them as RAID 1 again and get the kp we know its the dock don’t we?
Your kp report points to two things that were in use immediately prior to the kernel panic: apple.iokit.IOStorageFamily and apple.driver.AppleRAID. And in the list of loaded kexts, I see only Apple kexts. I think it’s a kernel bug. It may also be that the dock isn’t conforming to some aspect of the USB spec, and the kernel isn’t prepared to handle it. Or maybe something is being corrupted (again could be dock firmware related) and the kp is occurring as the result of reading some corrupt metdata.
But in any case, the protocol is to treat it as a product defect with the manufacturer. They need to confirm/deny whether they support Apple software RAID 1 being used with their product (I haven’t looked at whether they claim this in marketing material or spec sheets). And if they want to support it, they need to take it up with Apple to work out the details of who fixes what.
I suspect you will get another kp, because a non-deterministic bug where you don’t is actually worse. Without a lot of usage you won’t have confidence if at any future time you will get another kp. And that could cause raid metadata corruption, or worse, file system corruption. So I think that’s not really worth messing around with, I’d go straight to the manufacturer, see what their expectations are for its use. If it’s supposed to work with software RAID, have them swap out the hardware.
Another thing you could try is a much newer kernel. Current is XNU 12.4.0 which is the OS X 10.8.4 kernel. It’s possible the manufacturer will recommend this too. I haven’t check the specs to see what the minimum supported OS is.
-
“Fusion Drive” is a term like “Bootcamp” that encompases several different things, it’s not a single thing or product. A Fusion Drive is made by leveraging CoreStorage, which is sorta like the logical volume manager (LVM2) in linux. CoreStorage is also used to implement full disk encryption.
Either a whole drive, or partition can be designated as a physical volume (PV), and its “space” added to a logical volume group (LVG) to aggregate storage from multiple PVs. From that LVG pool, one or more logical volumes (LV) can be created. So what Apple is doing at the factory is adding one large partition on each of the included SSD and HDDs, as PVs, into a LVG, and then creating a single LV. The LV is formatted HFSJ, so the file system thus extends across two physical devices. So can you do this yourself with your own SSD and HDD if you don’t buy a “Fusion Drive” from Apple? Yes you can. And it can be done with non-Apple branded SSDs and HDDs. But it’s non-obvious how to do this in Disk Utility, and honestly at this point I’ve found enough bugs in Disk Utility once going “off the rails” (it’ll do simple formatting, partitioning, etc of a single disk, OK). But as soon as you start repartitioning, resizing, doing RAID setups, the utility is one of the worst UIs to ever come out of Apple, in my opinion. So I’ve resorted to using the CLI for this.
Much of this is in the coreStorage section of ‘man diskutil’ in Terminal.
The gotcha, of course, is if either the SSD or HDD dies, you lose the whole file system. In effect it’s like the linear or concatenated RAID type. Don’t count on being able to extract any information from the surviving drive. The other thing is that the migration of data from one PV to another, once it’s successfully migrated, is removed from the other PV. Some may wonder what could happen if there were a kernel panic or power failure were to occur during this migration. Only upon successful copy and commit to the PV, is data then removed (technically deallocated) from the other PV.
Also, I’m pretty sure the migration is at an extent level, not a file level. I haven’t read much about this one way or the other, but coreStorage is ignorant of the file system, it’s simply a layer that maps a logical block device to one or more physical block devices, and at least with linux LVM this is done with extents (which are variable size, but default is a 4MB extent).
So if you wanted to intentionally over provision your SSD for wear leveling reasons, or you want a volume exclusively on the SSD or HDD, and a separate volume that’s “fusion” between both drives, yes you can do that. But thus far I haven’t figured out a way to do in the GUI and for other reasons (Bootcamp experiences) I don’t trust Disk Utility aside from really simple tasks on single disks.
-
an entire new raid
Does that mean a new enclosure? Or new enclosure and new modules (drives)?
I can’t get into those. I’ll just buy a solid RAID next time
Unless someone with different hardware can vouch for successfully filling an HFSJ/HFSX volume to 90+% used, on 10.7.4 specifically, there isn’t a way to know that merely buying a different product would have avoided the problem.
While a file system ought to tolerate being filled to 100% of its own reported capacity, experience tells me that file systems start to do weird things when they get beyond 90% capacity, and therefore I tend to avoid that situation unless I’m testing. I’d guess that the user experience of 90% full file systems is comparatively rare, so this may not be heavily tested in real world scenarios.
But only because of the error -36, which is not an out of space error but some kind of IO error, leads me to think more likely it’s either a firmware or driver bug. A full set of kernel messages might be more enlightening.