OpenFX filter to fix hot pixels

[Kevin Skow]

Hi Mark-

I had a sensor issue with my Canon XH A1s camcorder last year and shot quite a bit of video with bad pixels before I realized the sensor had failed. The sensor has since been fixed (fingers crossed it doesn’t die again), but I had no real easy way of fixing the bad video.

There was a similar program that came out for Vegas Movie Studio 9 (on 32-bit OSes) called the Stuck Video Pixel remover, but the program was never updated to work with 64 bit operating systems or the later versions of Vegas Movie Studio or any version of Vegas Pro. Plus it was a very tedious program to work with, since you had to input the coordinates of each bad pixel into a text file for the program to read. The site that hosted the program has since gone offline, and I no longer use a 32 bit system to edit my videos.

I decided to give your plug-in a whirl. Since I didn’t have a completely black video frame to work with, I created a black image with the same resolution as my video project in Photoshop and transposed the hot pixels from one of my “contaminated” shots onto the image. I subsequently ran your plug-in in Vegas Pro 11 using this image and it wiped out the bad pixels with no problem!! It also doesn’t slow down the rendering process much at all, which I really enjoy.

Now I can spend hours going through my old videos and finally clean them up 😀

Thanks so much for creating this!!
Kevin

[Kevin Skow]

An update to my previous post:

The only issue I’ve run into so far with the plug-in is that it can “smear” the edited area when it contains a high degree of detail, or when there is a sharp cutoff between a bright and dark region within the edited area. Adjusting the blur and blend radii down temporally over these problem areas seems to help a bit. Still, overall a wonderful plug-in.

Do you have any trips or tricks to further reduce this smearing effect? This is likely a far-fetched request, but would there be a way to have the plug-in intelligently map pixels if there is a high contrast scene under the edited region?

For example, in one of my video shots I was panning past a light pole (which was dark against a bright sky). Likewise, the dead pixel also moved across the light pole. When I applied the PixelPatcher plug-in, the region around the pixel became smeared as it moved over the light pole due to the plug-in interpolating the bright and dark areas across the entire edited region.

I just thought I’d throw this suggestion out there in case you wanted to tackle it. I’m not much of a programmer so I don’t know how hard this would be to code up.

Thanks again for making this plug-in!

[Mark Allen]

The first thing I learned while writing PixelPatcher was that it wasn’t that difficult to hide hot pixels in moving video except for large clumps of them. But if you freeze-framed it and knew where to look then I wasn’t ever going to get a perfect result without some kind of intelligent algorithm. And yes, I freeze frame and I know where to look. And I tried to see if I could come up with intelligent algorithms but they just didn’t look to be worth the trouble if your real priority was moving video.

The short answer to eliminating the “smear” is that you set either of the radius params to 0 to stop all the blurring. At that point what it does is start from a bad pixel and then search out to find the closest good pixel and then copy it into the bad pixel. Well, pretty much. There’s a third radius parameter which defaults to a very weak 0.5 pixels. That third radius is only visible if you go into the .ini file and set DebugEnable=1. The .ini stuff is at the end of the documentation. The third radius is an internal parameter and you need to enable debug to see it but it’s always actually there. So if you set all three radius params to 0 then it just finds the closest good pixel and copies it to the bad pixel. It might look a little less harsh if you set the two visible radii to 0 and the debug-only radius to about 0.1.

That’s how PixelPatcher started out. There was no blurring. But my 7D has at times had big clumps of hot pixels and filling in bad ones from the closest good ones looked a little off in big clumps with the kind of high constrast edges you’re describing. That was only when I was freeze framing or specifically looking for them. If it was a low-contrast area it looked fine. But if the clump crossed a clear edge then it often looked bad while freeze-framed. So I added the radius params to try to smear it a little bit for the large clumps and then defaulted to a relatively small 2 pixel radius which seemed a good compromise. There’s little point in automatically turning the blurring off in low-contrast areas because it’s not really changing the result very much. It’s those areas with edges in it where the blurring makes a difference.

If you’re actually noticing the problem in moving video and not just while looking at one frame then I’d just try setting the radius params to 0. For single pixels you don’t need the blurring. It’s really only there for bigger clumps. If that works for you and all you have is single hot pixels then you’re done. It you have both single hot pixels and bigger clumps then you could make two different hot pixel image files: one for the single hot pixels and one for the bigger clumps. Then you hook up two copies of PixelPatcher where the single hot pixel copy has the radius params set to 0 and the big clump copy has radius params set to an appropriate value. For my size clump that would be bigger than 2.

What I did was default the radius params to compromise values which worked okay for both individual hot pixels and large clumps. So you end up with a little blurring applied to both kinds. I figured if you really wanted separate blur radii for different hot pixels then you’d run multiple copies of PixelPatcher on the same video, each with a different amount of blurring. I guess I could have adjusted the blur radii automatically based on the size of the hot pixel clump. Tiny ones get nothing and big ones get something. But I figured anyone who wanted that would just control it manually by adding more than one copy of PixelPatcher.

Oh and by the way, if you actually use more than one copy of PixelPatcher on a video it’s a good idea to set the .ini file var VegasBGR=1. It can actually speed things up quite a bit in some setups. Doubly so if you’re running two copies of PixelPatcher. Unless you have a ton of hot pixels there’s very little processing involved compared to your average filter. Almost all of it is overhead Vegas goes through to pass the data to an OpenFX filter. The VegasBGR switch reduces the overhead a lot. I didn’t want to turn it on by default because it’s not strictly OpenFX compatible with it on.

[Mark Allen]

I remember that other fix you’re talking about. That was the first thing I found while looking for a solution. The original version had you edit the registry to tell it how to fix the pixels. Yikes. Okay, I have experience editing the registry but still, Yikes. And then someone added the less-hostile version where you edited a text file. And then it got abandoned.

[Stephen Mann]

The registry is to programmers as the pancreas is to surgeons.

Ask any doctor what they were taught in medical school about the pancreas. The answer will always be “don’t touch it”.

Steve Mann
MannMade Digital Video
http://www.mmdv.com

[Kevin Skow]

Thanks for the tips and help!! I’ll give these settings a whirl here in the next few days and see how they pan out. It’ll be plenty of trial and error, but that is how we learn :-). The burned pixels on the camcorder video were about 5 x 5 pixels in size, which is pretty bad. There were about 3-4 of these bad pixels in the video. The good news is that I primarily film landscapes and sky scenes, so the plug-in does perfectly fine there.

[Kevin Skow]

Yea, editing the registry was a joyous task. That plug-in wasn’t for the faint of heart. I don’t miss that program much at all, but I do still have the old installer files lying around for nostalgic value.

[Job Brüggen]

Mark Allen: a life saver!
This seems to cleanly work with VEGAS pro.
Nice work. Brilliant and extensive explanation.
It took me a while to understand that I should select ‘patched image’ as option of course. My first video came out black… oh well. Great piece of work!
Greetings,
Job

[Sean Dougherty]

You have saved me a major headache this day.

I’ve had a hot red pixel on my camera sensor for a while, but it was never very noticeable even on dark images. But today while editing a wedding, I found it to be much more glaringly obvious than previously seen. Worried about the repercussions it might have, I looked online to see if anyone found or made a filter for Vegas to fix it.

Lo and behold, I found this thread, and for this I am immensely grateful. It works beautifully with Vegas Pro 13.

Thank you.