Chris Gordon
Forum Replies Created
-
Did you update the ReadyNAS firmware/OS? If I remember back several years ago when Mountain Lion came out, it had a new version of AFP and within a couple of weeks all of the NAS venders had updated firmware. There should be an update that addresses this specific issue. Of course there have been plenty of issues over the last several years that you should get to the most current firmware anyway.
Under the covers all of the SOHO NAS vendors use the netatalk package to provide AFP services (like they use Samba for SMB services). You need the updated netatalk server software on the NAS that supports the newer version of AFP. The NAS vendors roll all of this into simple firmware/OS updates.
-
When you hear that a NAS is a “file server”, what that means is that the protocols used (typically AFP, SMB or NFS) provide access to the data at a file level. Your computer says “give me file foo.txt” and the NAS “file server’ hands back the file. The software running on the NAS takes care of file locking, etc. The NAS also has the actual on disk file system (ext*, HFS, ZFS, NTFS, etc).
A SAN on the other hand simply gives a block device to your computer (you often hear of SAN storage called “block storage”). Think of it like a really long SATA or SCSI cable going from your computer to the SAN array. Your computer then puts a file system on the disk. When you access a file on a SAN volume, things work essentially like they do for a local disk. Your computer reads a stream of bits from a specific location on the disk and handles all of the file locking, etc (clustered SAN file systems add some extra parts in here, but that’s another topic).
Files, database files, applications, etc are all just files to the storage system and computer. Files are fundamentally just streams of bits. As far as the storage system is concerned there is no difference between them as it doesn’t really understand the content of the file.
As a general rule you can put any files on any kind of storage (local disks, NAS storage or SAN storage). There are a few times you run into something that is really picky and “smart” about where it is and will only work from a very specific location, but that’s the exception to the rule. The real question is should you put a specific kind of data on a specific kind of storage. That all depends on the specifics of your situation. Without knowing more about your specific environment and uses, it’s hard to say if it’s a good idea or if it will work as you would like.
Hope that helps.
Chris
-
What were the problems you saw with 16GB?
Both LR and FCPX access a lot of big files. It’s very reasonable to expect the file cache to be as big as possible when using these. In fact you generally want this as it helps speed things up a bit.
To monitor, just watch the Memory tab/section of Activity Monitor and see if Swap starts getting used.
-
You don’t have a problem if you’re just seeing memory used by the file cache. In fact, you normally want file cache to be used. The file cache is the OS caching recently accessed files in memory. This is under the theory that the likelihood of a file being accessed is higher if it’s already been accessed recently. Caching it in memory makes the subsequent access faster (read from memory instead of disk). Mac OS X, like pretty much every other UNIX flavor out there, will normally use all available memory for this cache. When memory is actually needed by a running program, it will automatically take that memory for the cache and purge some of the file cache. Any UNIX-like machine running for any non-trivial amount of time doing anything non-trivial work should have a sizable file cache. It is normally to see very little “free” memory since these cache’s SHOULD take up the un-used memory since this can make things faster.
Now, how do you know if you really have a memory problem…
The way to know if you need memory is to see if you are using a lot of swap. If you have a lot of swap used, you probably need more memory for what you’re trying to run. The even better way to know is if you see a lot of swapins and swapouts (from something like vm_stat — a command line tool you run in Terminal). When there is real pressure on memory: the programs you are trying to run need more memory than your system has RAM, the OS will take some things in RAM and move them to disk (swap is sort of like a disk based RAM). When you need to access the program that has been swapped out to disk, it must first be read back into RAM (and probably something else swapped out to make room). In the worst case situation of your system being extremely memory starved, it spends a lot of time swapping things to disk and other things back to memory (thrashing). Obviously this will be excruciatingly slow. The number of “swapins” and “swapouts” shows the rate at which memory pages are moving back and forth. Normally this is 0. Something higher can indicate a need for more memory.
So to your specific case:
– Where you experiencing problems at 16 GB or was it just that you saw a large page cache? What were the problems?
– Are you experiencing problems at 32 GB now or just seeing a large page cache?
– The caches files in ~/Library are completely different than RAM caches and are unrelated to the amount of memory in your system.
– The file cache at 70%-75% of your RAM is a *GOOD* thing. You want this as it will make your system faster in general.
– If you want to purge the file cache, you can use the “purge” command from the command line. Open Terminal and type “sudo purge”. This command is part of the OS and you don’t need any special applications to run it.Hope that helps, but I’m pretty sure you don’t have any actual problems.
-
tar has been around in UNIX since 1979. Though implementations are a little bit different (different options/features) across some platforms, the actual format of the file is a POSIX standard. You should be able to use the commands already installed on OS X (or any other UNIX flavor (*BSD, Linux, etc) to access the data.
The TOLIS website even says they use the tools included with OS X to read the files.
Depending on the drive you have and how it’s all set up, you may also want to look at the other command line tools:
mt, mtio, rmt -
[Alex Gerulaitis] “Cris – great post, much better written than mine.”
Thank you.
[Alex Gerulaitis] “Did you mean “the stripe” on which the updated bits are, rather than “the whole file”? “
Yes. I was just trying to keep the discussion simple and high level. There are probably a couple of other places where I took similar liberties.
-
“Which RAID is better” needs to be bounded by two factors. First, what is your work load. Second, what is your budget.
The typical work load in the video world is dominated by very large sequential reads and writes. These IOs are going to quickly blow through any cache and limit you to what the disks can do. To make things faster, you throw more spindles at the problem.
Budget means you have a set number of disks for whatever RAID configuration you choose. In RAID10, you get essentially half the disks contributing to your IO. In RAID5 you get N-1 disks (RAID6 is N-2) to handle the IO.
In our case (video editing) you’re limited by the throughput of your disks. The more disks, the more throughput. RAID5/6 win in this case.
So what are the “bad” things with RAID5/6 (parity RAID)?
– Parity Calculations. You have to calculate parity for any write. This is work done by some CPU (on a controller card, on some other CPU in the system, etc). The penalty in performing this calculation is negated by the additional disks to handle IO. For large sequential IOs and any half decent hardware/software setup, the additional spindles in a parity RAID more than make up for the parity calculations.
– Updates Require More IO. When you update a file in a parity RAID, you have to read all of the file and recalculate parity in order to make the update even when the update is only a small portion of the file. This just doesn’t appear in the typical work load in a significant way, so it’s not really a problem.
– Recovery from failed disks. Parity RAID is going to have a much more significant impact on performance when recovering from a failed disk and that hit is often going to be much longer than in RAID10. That’s the price you pay. If that hit isn’t acceptable, then you need more money. -
Yes, you can do that. You do need to understand IP networking to configure the networks properly on the machines to make it work right.
Question is why? I assume your LAN1 connections go to a switch (probably a single switch). Unless something is overloaded or you’re already putting heavy load on the the LAN1 connection, you’re not going to see any noticeable change speed changes using your LAN2 cross-over cable idea.
-
SSD vs Spinning disk is a balance between price, capacity and performance. For a given capacity SSD is faster, spinning disk is cheaper. Spinning disks are also available in much larger capacity than SSDs. You need to make your decision on that basis. At this point, I wouldn’t say either is more or less reliable and I expect SSD to eventually be far, far more reliable since it lacks moving parts. I’ve had disks dead on arrival, have some disks from the mid-1990s still running fine and everything in between.
-
It depends on what g-technology has exposed (or hidden). DiskUtility will show what info is available on the exact model. Some makers hide or don’t provide that specific info (very annoying). You can also pull up info in “About This Mac”‘s System Information. I’ve never dealt with G-tech’s enclosures and disks specifically, so I don’t know what they do or don’t provide.
The last resort is the screw driver and hopes there are actual labels on the disks with useful info.
There are only three disk manufacturers in the world now (Seagate, Western Digital and Toshiba). G-tech sources disks from one of these three to use in their enclosures — hopefully they won’t hide the details.