-
Defragging a SAN
one of my ethernet SANs has started to have latency problems during writes. basically, when capturing footage to the SAN, Final Cut will start to beachball and sometimes crash. running AJA System Test, there will be a pause of a few seconds before it starts writing its temp file. on the server directly, running AJA System Test produces the same results. the SAN is 15TB and it had been filled to a little over 14TB. i told the editors that was a no no and we deleted files to get around 6.5TB free. but the AJA System Test has the same initial latency problem, and Final Cut still beachballed during captures.
this system is built with 1.5TB Seagate drives. but they have different firmware numbers than the ones that had issues, and up until now, everything has been running well. so i’m suspecting fragmentation. i’m running Disk Genius 2 which has a “free space” defrag option. and its taking FOREVER. if i could move everything off, delete, then move everything back on, that would be faster. i’m thinking for future SAN installations to create more smaller volumes so they can be rotated – 3 working while 1 is defragging, for example.
on my Xsan systems, i typically delete the entire thing once a year, usually between projects, because Apple never made a decent defragging program for Xsan. if i can’t delete, then i usually run “snfs_defrag” on specific files or folders that are giving me issues until i’m able to delete the volume.
are there any suggestions out there for best practices for defragging SAN volumes? i did some Googling on defragging RAIDs and there seems to be a lot of theory and conjecture out there, but very little agreement.
thanks
e
Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com
-
6 Replies
-
Eric writes –
the SAN is 15TB and it had been filled to a little over 14TB. i told the editors that was a no no and we deleted files to get around 6.5TB free. but the AJA System Test has the same initial latency problem, and Final Cut still beachballed during captures.
REPLY – so you have RESTARTED the server computer, since deleting these files ? And it shows 6.5 TB free ?
this system is built with 1.5TB Seagate drives. but they have different firmware numbers than the ones that had issues, and up until now, everything has been running well.
REPLY – I can tell you from experience that the real experts will blame the Seagate 1.5 TB drives, blaming latency issues on these drives, and will not discuss it further with you. You say “but it worked fine up until now”. I was in the same boat you are in now.
so i’m suspecting fragmentation. i’m running Disk Genius 2 which has a “free space” defrag option. and its taking FOREVER.
REPLY – you are trying to defrag 9 Terabytes, and you are SURPRISED that it is taking forever. This is a job for OVER THE WEEKEND, not while you sit there and wait.
if i could move everything off, delete, then move everything back on, that would be faster. i’m thinking for future SAN installations to create more smaller volumes so they can be rotated – 3 working while 1 is defragging, for example.
REPLY – I agree with this. People love the one big RAID, but I hate it. I have seen RAID 5’s become corrupt anyway and then people get screwed. Two 8TB drives (in a 16 bay chassis) works fine. Getting 600Mb/sec is all anyone needs, and you get this with 8 drives. We can only do 70Mb/sec, so pushing the system to 800Mb/sec total bandwidth with 16 drives means nothing. It’s safer to have 2 or more volumes, than one big mama, and a lot less aggrivation, then you are going thru right now.
are there any suggestions out there for best practices for defragging SAN volumes? i did some Googling on defragging RAIDs and there seems to be a lot of theory and conjecture out there, but very little agreement.
REPLY – I don’t know of any. Let’s see if Mr. Raudonis replies to this one. He is the one with the “big mama” system.
Bob Zelin
-
yes i restarted the server.
i’m not surprised that its taking a long time. i knew it would. in fact the system is running read/writes at about 250MB/s as its moving all these files around. but yes, moving everything off and deleting would be way faster. luckily these guys are out of town for a few days.
The SAN is 2 CalDigit HDElements (6TB raw each) along with 4 internal drives in the Mac Pro all connected to the CalDigit RAID card. if defragging doesnt help this issue and it is in fact the Seagate 1.5TB problem, i’m hoping CalDigit will do something about it since the drives were bundled in their HDElements and they have a 1 year hardware warranty.
but i guess my overall question is, how do i establish some sort of regular defragging schedule? maybe on a system with 4 volumes, each weekend one volume is defragged, thus everything gets defragged once a month. or maybe something less frequent. these guys are just always capturing footage and transferring P2 files. a lot of quick edits and off.
e
e
Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com
-
I have never seen or heard of ANYONE doing a “regular defrag” as scheduled maintenance.
bob Zelin
-
This probably doesn’t help in this thread, but it might help someone evaluating a purchase down the road. The vStor and eStor line of NAS appliances from Apace both automatically defragment the drives.
I have no clue how they do it, but their engineers explained that during periods of inactivity, there are utilities built into their implementation of the Linux OS that run defrag routines. I was concerned about defragmenting the drives when they got about 50% full so I called Tech Support and got their lead engineer (who is very sharp), and he said not to worry about it, it’s built in.
I didn’t believe him at first, so being the cynic I am, I found some utilities to actually scan the drives and show defragmentation, and darn if the things show virtually none.
Pretty sweet thing to build into a shared storage system.
Chris Blair
Magnetic Image, Inc.
Evansville, IN
http://www.videomi.com -
actually Chris that is super helpful. it’s good to know that manufacturers are building this into their products to run automatically. i always felt like this issue didnt get enough attention in the large SAN systems a la Xsan.
the free space defrag i did on this system helped a lot, but we are discussing moving from one large volume to 3 smaller volumes. it will reduce the overall space because we’ll be going from a 12 drive RAID5 to 3 4 drive RAID5s. we’re discussing the Pros and Cons right now.
e
Eric Hansen, The Audio Visual Plumber – http://www.avplumber.com
-
Five years of running X-SAN and I’ve NEVER, EVER defragged it. Performance has remained the same.
We’ve also filled it up too far, but after deleting enough media, performance returns to normal.
There’s plenty of other maintenance procedures that we do on a regular basis, like running “CVFSCK”, but that has nothing to do with “defragging”.
mark
Reply to this Discussion! Login or Sign Up