"New" Debayering Solution Using AviSynth

[flyer]

EDIT: Please read 2nd and 3rd post first!


A “new” (at least to me) solution for Debayering .avi files has emerged from discussions on the VideoHelp site. Full credit goes to Jagabo for his contributions and great help. See: http://forum.videohelp.com/threads/331647-YUY2-vs-RGB24-vs-Y800?p=2055741#post2055741 The original thread has all of what follows below. I’ve tried to reorganize it to be easier to follow.

The solution allows one to start with Y800 video (grayscale with color info) and convert it into an RGB color .avi with no compression. The resulting .avi will be ca. 3x larger than the Y800 original .avi file, though there is a way to get a smaller file size using the lagarith codec (a so-called lossless codec).

In order to do the Debayering you need the following programs:

1. AviSynth - http://www.videohelp.com/tools/Avisynth

2. AviSynth Debayering filter - http://moodub.free.fr/video/Avisynth_DebayerFilter.zip

(Install AviSynth, then place the Debayering filter in the plugins folder)

3. If not installed, install ffdshow http://www.videohelp.com/tools/ffdshow

Configure as follows:

a. Use Start -> All Programs -> ffdshow -> VFW Configuration
b. Set codec RAW (very bottom) to “all supported.”
c. In column on left highlight “output” and make sure that YV12, YUY2, RGB32, and RGB24 are enabled in the right side. The Debayer filter requires YV12.
d. If ffdshow was installed as part of a codec pack the VFW configuration settings are probably located in a different place. Don’t confuse “VFW” with “DirectShow” settings! The settings look similar.

4. Install VirtualDub - http://www.videohelp.com/tools/Virtualdub


5. AviSynth is frame server. If you have not used it before it can seem complicated, but it is very powerful. It creates a “bridge” between the original file and either a player or encoder. Once the program is installed you set the script in AviSynth by using a text file (either .txt or wordpad) and rename it with the file extension .avs. The .avs file can be loaded directly into many players or encoders as if it were the original video file.

This is the script that can be used to Debayer. Simply copy it to a txt file putting the exact location and name of the .avi file right after AviSource. (The example uses the location on my computer. Obviously, substitute your file’s location and name!)

AviSource("H:\test\Y800.avi").FlipVertical()FlipHorizontal()DebayerFilter(swap =0)FlipHorizontal()
(there should be no spaces in the word Horizontal. I can't eliminate it in this post)

On my computer I could not get “AviSource” to work. It was suggested that I try the following which worked for me:

DirectShowSource("H:\test\Y800.avi").FlipVertical()FlipHorizontal()DebayerFilter(swap =0)FlipHorizontal()

This might also work:

ffVideoSource().("H:\test\Y800.avi").FlipVertical()FlipHorizontal()DebayerFilter(swap =0)FlipHorizontal()

Rename the .txt file to .avs and try to open the file in a video player that accepts AviSynth. (GOM Player does, as does MediaPlayer Classic.

If you want to save a copy of the Debayered avi file open the .avs file in VirtualDub.

a. Go to the drop down menu under “video” and select “direct stream copy.”
b. Then under “file” select “Save as AVI.”

If you want to experiment with different codecs, for example HuffyYUV or Lagarith, first install these codecs. Open VirtualDub, and then from the down menu under video select “full processing mode.” Then select “compression” and select the codec. Save the .avi file as in “b” above.

If anyone experiments with this, I'd like to know how you feel this method compares to using Registax or Ninox for Debayering.

EDIT: I may have been a little hasty in posting this but I was excited to find out about this method. It works for me with 1024x768 output from my DFK31AU.03.AS. However, I tried it on a cropped avi capture (picture dimensions of 392x288) and it generated an error saying Debayer Filter "image pitch and width doesn't match." I'm not sure what this means exactly or if there is a solution with this Debayer filter.

[fquestier]

An easier way which does not require AviSynth or ffdshow is to use VirtualDub with a debayering plugin.

1. Download and install VirtualDub from http://www.virtualdub.org/

2. Download "VirtualDub Debayering Filter" from http://www.inf.ethz.ch/personal/lballan/demo.html. Put the file debayering.vdf in the plugins subdirectory of VirtualDub.

3. Start VirtualDub and open a Y800 AVI file. I have found VirtualDub supports Y800 natively so ffdshow is not required.

4. Go to Video -> Filters and click Add. Select the Debayering plugin and click OK three times. No changes are needed in the plugin settings dialog for the Y800 files produced by our DFK 31AU03.

5. Now VirtualDub should produce debayered output and you can use File -> Save as AVI to save an uncompressed RGB color video.

[flyer]

Thank you, yes, that certainly is much easier. I did not know about that plugin!

I started a similar thread at the Cloudy Nights Forum. As a result I had a reply from Torsten:

"Just for the sake of completeness I'd like to note that another option for Debayer Y800 data would be the new standalone debayer tool in FireCapture. See screenshot here: http://firecapture.wonderplanets.de/help/debayer.html

Based on comparisons Clif Ashcraft did it seems that the Bilinear and Adpative algorithms are producing a much better quality than the standard debayering (NearestNeighbor) in Registax.

Torsten"

--------------------
http://www.wonderplanets.de
http://firecapture.wonderplanets.de

I tried this method out last night on some old avi files and the results were quite strikingly better than anything I had previously outputted from Registax. I can highly recommend this as a superior method for DeBayering.


I also tried out the VirtualDub DeBayering Plugin Filter. It worked well one one Y800 I tried it on, but a second try on a different Y800 gave strange results. It seems to have used the wrong color decoding method and the output was very purple. I will play with it more and it will be interesting to compare results to Torston's tool.

[fquestier]

There are many algorithms for debayering and the results will depend on the algorithm used. NearestNeighbor is a very crude method, I did not know this was the method used by Registax.

The VirtualDub plugin I linked supposedly uses a very advanced algorithm (Demosaicing with Directional Filtering and a Posteriori Decision), see http://www.danielemenon.netsons.org/pub/dfapd/dfapd.php. However, a grid pattern still appeared in Registax after using wavelets on some processed movies debayered with this plugin.

I will try the FireCapture debayering tool, I was not aware of it, thanks.

[flyer]

Yes, the grid pattern ca be a vexing problem. I don't know why I sometimes see the pattern so prominently on application of wavelets in Registax, while other times I don't. I created a different thread on this but got little input. But, I have a suspicion that it might have somerthing to do with upgrading the capture software from the version that came packaged with my camera. I have since removed that software and reinstalled the original version. Alas, the weather has been so terrible for weeks now that I have had no opportunity to experiment further.

Let us know how you liked Torsten's DeBayering tool!

[fquestier]

It's not really working out for me. When I open a Y800 movie with 2000 frames, only 1360 frames are detected and processed. Also, the resulting movie is saved at 10 fps while the original movie is 30 fps. Further, the image is flipped vertically.

On top of that, the SmoothHue and AdaptiveSmoothHue algorithms do not work at all, they do not remove the bayer grid regardless of the pattern setting. NearestNeighbor and Bilinear work with the RG pattern setting. In Registax I have to use the GB pattern setting.

Here are some results:

Raw frame:

http://img109.imageshack.us/img109/2909/rawxw.th.png (http://img109.imageshack.us/img109/2909/rawxw.png)

Debayered with VirtualDub plugin + grayscaled:

http://img714.imageshack.us/img714/6631/pluginb.th.png (http://img714.imageshack.us/img714/6631/pluginb.png)

Debayered with TheImagingSource DirectShow filter through AviSynth + grayscaled:

http://img707.imageshack.us/img707/2031/directshow.th.png (http://img707.imageshack.us/img707/2031/directshow.png)

Debayered with FireCapture Debayer (Bilinear) + grayscaled:

http://img201.imageshack.us/img201/3586/bilinear.th.png (http://img201.imageshack.us/img201/3586/bilinear.png)

Debayered with FireCapture Debayer (AdaptiveSmoothHue) + grayscaled:

http://img146.imageshack.us/img146/7784/smoothhue.th.png (http://img146.imageshack.us/img146/7784/smoothhue.png)

[flyer]

Thanks for your extensive testing! In your opinion, which method has provided you with the best results?

To me it looks like possibly the Vdub solution gives the sharpest image (I assume these are just 1 frame, unprocessed?) Also, can you explain to me about the ImagingSource Direct Show filter, and perhaps the details of the .avs file to use it.

I did not have trouble using Torsten's program to DeBayer my Y800 video using the adaptivesmoothhue filter. Perhaps it is an computer operating system bug? (I'm just using WinXP). I'm also pretty sure that you must use GB (as in Registax) and that using the wrong pattern setting will result in the pattern grid not being removed.

I think I will attempt to point Torsten to this discussion. Perhaps he will contribute something interesting. I know he will probably want to know why it seems not to work for you.

Are your captures done using the latest version of IS software and drivers. In an earlier post Stefan made reference to a little change in the DeBayering. Sometimes I woder if that change is working out. As I said earlier, I've reinstalled an earlier version but cannot check it out due to weather.

BTW, which camera are you using?

[flyer]

I received this reply from Torsten. I am happy to pass along a sample of the lunar video if it is posted somewhere. I think Torsten would appreciate trying to fix any possible bugs in the tool.


Hi Peter,

I can't reply because I'm not registered and I don't wonna so perhaps you can forward my answer:
>>NearestNeighbor and Bilinear work with the RG pattern setting.
>>In Registax I have to use the GB pattern setting.

Regarding to the spec the ICX204 has a GB bayer pattern so you have to choose this one. In the FireCapture-DeBayer tool you need to set y-offset=1 for videos taken with IC Capture.

>>When I open a Y800 movie with 2000 frames, only 1360 frames are detected and processed.
Currently the tool has only be tested using the DFK21/Flea3 and smaller videos. I will check this.

>>Also, the resulting movie is saved at 10 fps while the original movie is 30 fps.
>>Further, the image is flipped vertically
If you could send me a short AVI I could check those issues.

Regards,
Torsten

[fquestier]

I assume these are just 1 frame, unprocessed?

Yes, the samples above are the same raw frame of the movie (see the first sample), debayered with the methods described.


Also, can you explain to me about the ImagingSource Direct Show filter, and perhaps the details of the .avs file to use it.

IC Capture supports two raw formats, Y800 and BY8. Both formats are actually the same, but BY8 movies are automatically debayered during playback by a DirectShow filter installed with IC Capture. To save this debayered output, you can create a simple AviSynth script which allows you to open a BY8 movie in VirtualDub:

DirectShowSource("moon.avi")

The VirtualDub input will automatically be debayered by the DirectShow filter so you can just save it. You can also use this method for Y800 movies by changing all "Y800" identifiers in the movie to "BY8 " (note the space) with a hex editor.


In your opinion, which method has provided you with the best results?

To me it looks like possibly the Vdub solution gives the sharpest image

It appears the VirtualDub filter provides the greatest contrast, although the image seems a bit darker than with the TIS DirectShow filter.


I did not have trouble using Torsten's program to DeBayer my Y800 video using the adaptivesmoothhue filter. Perhaps it is an computer operating system bug? (I'm just using WinXP). I'm also pretty sure that you must use GB (as in Registax) and that using the wrong pattern setting will result in the pattern grid not being removed.
Setting the Y-offset to 1 fixed this problem and allowed me to use all algorithms with the GB pattern, but the number of frames detected is still wrong and the image is still upside down.



Are your captures done using the latest version of IS software and drivers. In an earlier post Stefan made reference to a little change in the DeBayering. Sometimes I woder if that change is working out. As I said earlier, I've reinstalled an earlier version but cannot check it out due to weather
The images were captured with IC Capture 2.0, the version that came with our camera. However, the TIS DirectShow filter I used to debayer some samples is the one that came with IC Capture 2.2, which I have installed on my computer.


BTW, which camera are you using?

We are using the DFK 31AU03.


I am happy to pass along a sample of the lunar video if it is posted somewhere. I think Torsten would appreciate trying to fix any possible bugs in the tool.
A sample can be downloaded here: http://gupl.dk/56360/

[flyer]

Thanks for this update! I'm learning a lot as I had not realized that BY8 is actually Y800..... Things are starting to make a bit of sense. I am also using the DFK31AU.03 but I'm in the process of swapping it for the DBK31 (without the IR filter).

Thanks also for the link to your Y800 sample. I will past the link to Torston and see what he comes back with.

Thanks for making this thread interesting!

Peter

[tea drinker]

Used the virtual dub plugin, firecapture would not run on my system at all, no error. Tried an old jupiter file, attached both versions. this is a very quick processing run, but shows immediate improvements, and gets rid of the pixelated effect on registax. I will try it on some moon runs, I previously noticed same pixelated effect in high contrast areas...
Thanks for this :-)