H4N Zoom corrupt files – One showed 1GB and the other 0KB. Able to get the first but the 0KB i’m having no luck

[Ty Ford]

Hello Mark and welcome to the Cow Audio Forum.

Have you tried looking at the files with Sound Devices Wave Agent? Here’s a link: https://www.sounddevices.com/product/wave-agent-software/

Audacity, a free app also has a RAW import function. You can switch sample rates and mono/stereo on the fly after you get the file on the timeline to make it play properly.

Going forward. The only time I’ve had problems with a file is when I failed to use a newly cleaned and formatted SD card.
Yeah, I know it’s an extra step of two, but less of a pain in the butt. ☺

Sorry for your pain!

Regards,

Ty Ford
Cow Audio Forum Leader

Want better production audio?: Ty Ford\’s Audio Bootcamp Field Guide
Ty Ford Blog: Ty Ford\’s Blog

[Mark Green]

I can’t use Sound Devices Wave Agent because Mac / Apple won’t allow it as it cannot identify the developer.

I am aware Audacity has the same RAW import function and I tried that, too, experimenting with all the settings both in there and Audition. It’s the same results.

The memory card had been newly cleaned and formatted prior to this issue.

My goal now is just to retrieve this file. That’s all. I will be connecting my Zoom to my computer going forward instead of relying on the recorder and memory card, but right now I just want to get this file back.

[Ty Ford]

Mark ,

I think you can use Wave Agent, but there a bit of mucking about in your Mac Security and Privacy panel to let it happen. I’ve forgotten exact;ly where to spread the fairy dust, but Wave Agent works fine here on my Mac. Will it help you in this situation? I don’t know.

Good luck!

Please let us know how you did.

Regards,

Ty Ford
Cow Audio Forum Leader

Want better production audio?: Ty Ford\’s Audio Bootcamp Field Guide
Ty Ford Blog: Ty Ford\’s Blog

[Mark Green]

I downloaded it and opened but didn’t do anything.

Since that file is showing as 0KB, I think I need something to figure out where the original file is.

As I mentioned – I was able to at one point import a raw version that had BOTH files combined, but they were interweaving. So it exists on the card, somewhere, I’m just not sure where it should be and how to get it isolated.

[Mark Green]

(And to clarify, I meant I was able to workaround and get Wave Agent on my Mac – but still didn’t help)

[Mark Green]

One more thing if it helps –

The settings below are what get the 0KB audio file to import as the combined file (jumping back and forth every few seconds) when importing the original memory card DMG file into either Audacity or Adobe Audition with the Import RAW function).

44.1 Hz sample rate
Channels: 1
Encoding: 16-bit PCM in Adobe Audition, Signed 16-bit PCM in Audacity
Bye Order: Little-Endian (Intel Bye Order)
Start Byte Offset: 0 bytes (0x0)

And these are the only settings that work.

But is there a way to import it but getting separate audio tracks? It is pulling from the DMG of the memory card, so that’s why it’s drawing each. I tried making another image copy of the memory card but with deleting the other file (that’s worked fine) but it still brings in both still. As a reminder, I was able to get the other file fine (even though it didn’t play, it said 1GB). But this other file, which displays 0KB, is the problem.

However, based on being able to clearly hear the audio in the combined version, I know it’s in there somewhere. I just don’t know how to locate it or import it as an isolated single track.

[Maarten Andreas]

Guys & gals, I had the same problem and I figured it out!

My Zoom H4n (firmware v1.90) had its power cord pulled while recording internal mics & line inputs (4 channels), which netted me a file 0 bytes in size. And I recovered it!

The bad news: I can say with certainty that no file recovery program on the internet is going to help you in the event of power loss (net adapter pulled, batteries dead), pulling the card too soon, or any other reason for not having closed down the recording properly (which includes a faulty SD card). Don’t spend your time on it, don’t risk installing spyware and what not. Recovery is a manual process, of which I have not (yet?) dreamed up an automated mouseclick mouseclick done process for. Recovery is a manual process involving you.

The good news: The reason I can say the former with certainty is because I figured out how the thing works, and how to recover data.

Summary (TL;DR):
You probably read this before, but to repeat it once again: If you experience a failure of any kind to gracefully stop the recording, your data is still there, somewhere, on the SD card. DO NOT START ANOTHER RECORDING AFTER YOU HAVE EXPERIENCED YOUR FAILURE! If you’re in the situation where your file shows 0 bytes in size when you put the card in your PC, data is still there, somewhere, provided it was not overwritten by you starting another recording. If such happens, swap out the SD card for another, go figure out the recovery later. Or make what’s called a disk image of the first and only partition (or entire SD card if you can’t figure it out) and get into the disk image later. (More on what a disk image is and why further below.)

For the low price of me getting to keep your SD card and you sending two special beers from your country along, I’ll do it for you. Contact me on chef@edgewanderer.com

TL;DR technical details for the leet geeks:
Zoom H4n firmware formats cards as FAT32, uses 512 byte sectors, 64 sector clusters, writes chunks of data alternating between 16 and 17 clusters in size. In 4CH mode it writes two .wav files, which it interleaves/multiplexes 2×16 clusters (16 clusters for each file), 2×17, 2×16, 2×17 etc. sequentially in one lengthy strip of data. Until a proper stop is done, no pointers in the File Allocation Table where the stream of data begins, no RIFF/Wav header prefixed to the stream; all written on hitting the stop button. So to find the start of the stream load the entire image of the partition (not disk, partition! much easier calculating) into for example Audacity by using the load raw data function (44.1kHz., 16bit, little endian, stereo), find the start of the audio you’re after, and calculate the start point in seconds into a byte offset in the disk image. All recordings start on a cluster boundary, so pick the nearest start of a cluster when you calculated your byte offset. Extract the data from your now precise on the cluster boundary-offset to another file (hex editor, dd, sleuth kit whatever), and demultiplex that with the C++ program pasted at the end. You now have your two stereo .wav files.

The story:
There’s a very sad design choice in the firmware of the Zoom H4n (other models are probably very similar) in that it writes all metadata the moment you hit stop. There’s nothing but raw (random looking) data written to the SD card, until you hit stop. No magic recovery program on the internet can help you, as there’s nothing an automated process can go look for. It will find old recordings you deleted before, but not the one you just lost.

By saying metadata, I refer to any sort of location information in the File Allocation Table (FAT) system or any sort of RIFF/Wav header. I spend a lot of time looking for it, there’s none. Zero. All there is, is a filename in the index, with no data assigned to it, hence it being 0 bytes in size. All the metadata stuff is written when you hit stop, or when the file limit size is reached and a new file is created. Your recording is somewhere on the SD card, but no pointers to it, no header in front of it.

Which means it is very hard to write a computer program to recognize where your data is. Writing such a program would involve it being able to differentiate a voice or music instruments from noise and clicks. And being able to do so at different audio settings (sample rate, resolution, channels). But locating the data is possible by means of utilizing a super flexible & programmable neural network! The key tool employed in this recovery is you yourself, your brain.

But first, some on how the writing process works: As mentioned, the Zoom H4n (mine has fw v1.90) basically dumps raw data from the ADC’s (analog-digital converters) straight to the SD card. But, as I found out, it does so in a predictable pattern. When the device is in 4CH mode, it writes two stereo .wav files. Those two files are written almost like the SD card is a tape-deck, in that it writes a chunk of the first file, a chunk of the second file, a chunk of the first file etc. in a continuous strip. So it writes two chunks (one of each file) continuously while recording. Once you hit stop, it writes to the File Allocation Table (FAT) what chunk belongs to the 1st file, what chunks to the 2nd.

If there’s an idea popping up in your mind now going “so, something somehow even numbers is first file, uneven something must be second file!”, then you are absolutely right.

Before we go into this pattern, a few details on how the FAT32 filesystem works: The Zoom H4n wants SD cards formatted with a FAT32 filesystem and it very much likes to do the formatting itself. Reason for this, is that FAT32 can be used with multiple parameters, but the Zoom firmware wants a specific set of them due to how it writes data to SD card. Also it wants a very specific set of directory names to be there before commencing recording, which it makes during formatting.

The main parameters it cares about are sector size and cluster size. Any storage medium, which includes hard disks and SD cards, is essentially like a tape deck in that it has a start and an end, with places in between. All any storage media can do is save & read what’s called bytes. Zeroes and ones. We call a group of 8 zeroes or ones a byte. A byte is the most elementary unit of data in a computer.

What filesystems like FAT32 do is providing a mechanism to manage this enormous strip of bytes, into what we now know as files and directories. A filesystem functions like an index, for example bytes 956 to byte 3065 are /groceries/shoppinglist.txt, and bytes 5364 to 5631 is /school/todo.txt

In FAT16/FAT32, those bytes are not referred to individually, but by what Microsoft has named secors and clusters. How big they are is configurable when you format a drive/card/thumbstick with your PC. The Zoom H4n wants to format SD cards itself because it wants a sector to be precisely 512 bytes and a cluster to be precisely 64 sectors (which makes a cluster 32768 bytes in size).

When the Zoom H4n is recording 4 channels at 44.1kHz samplerate and 16bit resolution, as mentioned above it writes two stereo .wav files interleaved. It writes 16 clusters of data for the first .wav file, 16 clusters of data for the second .wav file, 17 clusters of data for the first file, 17 clusters of data for the second file, then the cycle start over.

16 clusters is 524288 bytes. When sampling at 44100 times a second, with 16bits (two bytes) of resolution times two because stereo, each second of audio takes 2 bytes * 2 channels * 44100 samples = 176400 bytes. If we divide 524288 bytes by 176400 bytes we find that each chunk (16 clusters) is about ~2.97215 seconds worth of data.

For those of you who already managed to load their disk image into Audacity and listen to their lost recording that way, you noticed the audio cuts back and forth between mics and line-in every 3 seconds or so. Now you know why: You are listening to a strip of stored data which is the two .wav files interleaved (multiplexed).

So, how to get this strip of data and demultiplex it?

I’m a little short on time now so I will edit & expand this section later (and deliver on my promise I would tell you what disk images are), but for now, you’ll have to do with the TL;DR (too long; didn’t read) summaries above. Then use the C++ program I wrote below to cut up your multiplexed stream of data into two files, which you can then load as raw data into Audacity. Find a programmer friend or ask your neighbour if you can’t figure it out! For the low price of me getting to keep your SD card and you sending two special beers from your country along, I’ll do it for you. Contact me on chef@edgewanderer.com

I have to get to work now cya later!

#include <iostream>
#include <fstream>
#include <string>
#include <cstring> // memset()

#define ZOOM_FAT32_SECTOR_SIZE 512 // In bytes
#define ZOOM_FAT32_CLUSTER_SIZE 64 // In sectors
#define ZOOM_INTERLEAVE_SIZE 16 // In clusters
#define ZOOM_INTERLEAVE_SIZE_DEVIATION 1 // Zoom H4n v1.90 alternates between writing 16 clusters of each file, writing 17 clusters of each etc.

using namespace std;

int main(int argc, char *argv[])
{
string infile;
string outfile[2];

// Count of arguments is the program name itself, plus the arguments to it;
// from argv[1] and onward is interesting to us
switch(argc)
{
case 2: // One argument in addition to the program name
infile = argv[1];
outfile[0] = infile + “.m.wav”;
outfile[1] = infile + “.i.wav”;
break;
case 4: // Three arguments in addition to the program name
outfile[0] = argv[1];
outfile[1] = argv[2];
infile = argv[3];
break;
default:
cout << “Usage: [outfile_mics outfile_line] infile” << endl;
return 1;
}

cout << “Demuxing file ” << infile << endl;
cout << “to files” << endl << outfile[0] << endl << outfile[1] << endl;

ifstream interleaved(infile);
ofstream deinterleaved1(outfile[0], ios::out | ios::app | ios::binary); // Input-output stream,
ofstream deinterleaved2(outfile[1], ios::out | ios::app | ios::binary); // out, append, binary

for(uint32_t i = 0; !interleaved.eof(); i++)
{
uint32_t chunksize = ZOOM_FAT32_SECTOR_SIZE * ZOOM_FAT32_CLUSTER_SIZE * ZOOM_INTERLEAVE_SIZE;
if((i / 2) % 2 == 1)
chunksize += ZOOM_FAT32_SECTOR_SIZE * ZOOM_FAT32_CLUSTER_SIZE * ZOOM_INTERLEAVE_SIZE_DEVIATION;

char buffer[chunksize];
memset(buffer, 0, chunksize);

cout << “Writing chunk ” << dec << noshowbase << i; // No endl
cout << ” starting at offset ” << hex << showbase << interleaved.tellg() << endl;

interleaved.read(buffer, chunksize);

if(i % 2 == 0)
deinterleaved1.write(buffer, chunksize);
else
deinterleaved2.write(buffer, chunksize);

cout << “Deinterleaved ” << dec << noshowbase << interleaved.gcount() << ” bytes” << endl;
}

return 0;
}

That’s it for now folks! Soon up on github too. But got to get to work now.

[Maarten Andreas]

Hey all,

Took me a while as life got busy, but today I followed up on my promise to deliver a complete story with a complete howto. I put it up on GitHub including the code:

https://github.com/merethan/zoom-H4n-recovery

I’ll update my tool tomorrow or somewhere next week to simplify use.

As described on GitHub is sufficient to get the job done for at the very least all 16-bit resolution recordings. I’m now working on 24-bit, if stuff comes up it will be described & updated in my tool on GitHub. The process however will be similar.

If you’re in need of help, keep an eye on my GitHub page!