• Storage & Archiving

  General Data loss during tranfer over network

  Posted by Kasper Sørensen on June 28, 2011 at 8:29 pm

  Hi all out there.

  I’m having an argument with my boss about data loss during data tranfer.
  Let me explain.

  I’m aware about exporting, transcoding of videofiles, no matter what you do, WILL cause some quality loss, but this is not what i’m talking about.

  I’m talking about copying files. From one server to another or one hard drive to another. I’m aware of the risk, that some bits can be misplaced, or not being copied. But that’s what we have hash checks for. And as far as i know, if two, for instance md5 fingerprints, are identical, the file is almost 100% identical.

  My boss arguments, that you have data loss when you copy data from one source to another, and i don’t. Who of us is right?
  I would love some documentation with your answer if possible. I can’t find anything.

  Thanks a lot!
  Kind regards
  Kasper Folman

  Steve Tharp replied 14 years, 10 months ago 2 Members · 1 Reply
• 1 Reply

• 

  Steve Tharp

  July 11, 2011 at 11:28 pm

  Kasper,

  You ~100% correct. If you have two identical MD5 hashes then it is safe to say that both files are identical. (Excluding the possibility of the pseudo-collision bug found in the MD5 compression function).

  If you’re copying file ‘X’ from server ‘A’ to server ‘B’ chances are the MD5 hash from file ‘X’ on server ‘B’ will not match file ‘X’ on server ‘A’ because you are not doing a true byte-for-byte copy. Now that doesn’t mean for all intensive purposes that those two files are not identical, it simply means that at a minimum a single bit (header, timestamp, ACL, etc) was modified. Having said that – there wasn’t technically any data lost in that transmission it just means that server ‘B’ modified a single bit on the file.

  Thanks,

  Steve Tharp
  Art Director
  Point7West
  https://point7west.com

Reply to this Discussion! Login or Sign Up