Image processing - built a second image

[Droise]

Hello,
I'm trying to write a software to recognize some image elements, in a frame video from a DFK21BF04.H cam.
I'm using visual basic language on visual studio 2010.
I open the device setting and I start the live video. For each available frame i read an image with IcImagingControl1.ImageActiveBuffer istruction, and then i read al pixel of the image, to find the pixel that meet my recognition parameters.
The software work regularly. This is the code of the processing:

Private Sub IcImagingControl1_ImageAvailable_1(ByVal sender As System.Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs) Handles IcImagingControl1.ImageAvailable

Frame = IcImagingControl1.ImageActiveBuffer
IcImagingControl1.DisplayImageBuffer(Frame)

For y = 0 To Frame.Lines - 1
For x = 0 To Frame.PixelPerLine - 1
...
pixel research
...
Next
Next

I'd like to mark on the show image the pixel found with a color (yellow for example). I tried to use OverlayBitmap istruction on the video, but in this mode i compromise the successive frame processing. I tried to create a second IcImagingControl in my form, to put in it the same image, and doing OverlayBitmap in the second control, but it don't work... I use this code:

Private Sub IcImagingControl1_ImageAvailable_1(ByVal sender As System.Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs) Handles IcImagingControl1.ImageAvailable

Frame = IcImagingControl1.ImageActiveBuffer
IcImagingControl1.DisplayImageBuffer(Frame)
IcImagingControl2.DisplayImageBuffer(Frame)

For y = 0 To Frame.Lines - 1
For x = 0 To Frame.PixelPerLine - 1
...
pixel research and OverlayBitmap on ICImagingControl2
...
Next
Next

It show regularly the video in the both image, but don't show the overlay on the second image.

I tried a second solution, working directly on the frame buffer, setting the pixel i recognize with this code:

Private Sub IcImagingControl1_ImageAvailable_1(ByVal sender As System.Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs) Handles IcImagingControl1.ImageAvailable

Frame = IcImagingControl1.ImageActiveBuffer
Frame2 = Frame
IcImagingControl1.DisplayImageBuffer(Frame)

For y = 0 To Frame.Lines - 1
For x = 0 To Frame.PixelPerLine - 1
...
pixel research and setting pixel on Frame2 buffer
...
Next
Next

IcImagingControl2.DisplayImageBuffer(Frame2)

The software work, i find setting pixel in the second image, but in the first image i see flickering in the pixel i set in the second image, and i cannot understand why...
Where is the problem?
To mark some pixel, or write text on the video frame without compromise the image analysis, which is the correct way to do it?

Thank you!
Best regards.
Droise Paolo.
Sirio Systems.

[Stefan Geissler]

Hello,

I suppose you see the flickering, because you did not disable the live display, therefore, you see the unmarked and marked image in sequence. Simply set Ic.LiveDisplay to false.

[Droise]

These are the setting i do at startlive:

IcImagingControl1.LiveDisplayDefault = False
IcImagingControl1.LiveDisplayHeight = IcImagingControl1.Height
IcImagingControl1.LiveDisplayWidth = IcImagingControl1.Width

IcImagingControl1.LiveCaptureContinuous = True
IcImagingControl1.LiveCaptureLastImage = False
IcImagingControl1.LiveDisplay = False

IcImagingControl1.OverlayBitmap.Enable = True
IcImagingControl1.OverlayBitmap.ColorMode = TIS.Imaging.OverlayColorModes.Color

IcImagingControl2.LiveDisplayDefault = False
IcImagingControl2.LiveDisplayHeight = IcImagingControl1.Height
IcImagingControl2.LiveDisplayWidth = IcImagingControl1.Width

IcImagingControl2.LiveCaptureContinuous = True
IcImagingControl2.LiveCaptureLastImage = False
IcImagingControl2.LiveDisplay = False

IcImagingControl2.OverlayBitmap.Enable = True
IcImagingControl2.OverlayBitmap.ColorMode = TIS.Imaging.OverlayColorModes.Color

IcImagingControl1.LiveStart()

IcLiveDisplay is setting to false...
The flickering show the pixel modified in the second image. For example, i search red pixel, and i set them to yellow in the secondi image. In the second image i regularly see the pixel marked yellow, but in the first image, instead to see original pixels, they blink from original color to yellow...

[Stefan Geissler]

Well,

then I think, the frames share the memory. Therefore, you use the ImageBuffer.Bitmap object and pass it to a PictureBox instead. Then you do not need two frames. I am not sure, whether they share memory, but you can do your drawings on the PictureBox instead on the frame.

[Droise]

I understand...

So, for the second windows i will use a simply picturebox...
How can I transfer the image from Frame to picturebox? The format seem not directly compatible...
Frame is declared as TIS.Imaging.ImageBuffer.

Thank you!

[Droise]

Oh,
I find how transfer image from buffer to picturebox:

PictureBox1.Image = Frame.Bitmap

Now i will try to draw directly on ghe picturebox.

Thank you!
Best regards!

[Droise]

I post evolution of my software...

I tried to use a picturebox for show the image with marked pixels, but it is a little heavy to manage, and it compromises optimal frame rate analysis.
I was able to use only the first IcImagingControl window for the analysis and the to show of marked pixels, setting directly the pixels on the frame buffer, and after I show it in the IcImagingControl window. In this mode i see that there aren't problem for the analysis.

The code is this... it find a laser point in real time. I tested it until 30 fps.

Private Sub IcImagingControl1_ImageAvailable_1(ByVal sender As System.Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs) Handles IcImagingControl1.ImageAvailable
Dim Frame As TIS.Imaging.ImageBuffer

Dim indexX As Integer
Dim indexY As Integer
Dim red As Byte
Dim x As Integer
Dim y As Integer
Dim xLaser As Long
Dim yLaser As Long
Dim redPixel As Integer

Frame = IcImagingControl1.ImageActiveBuffer

redPixel = 0
xLaser = 0
yLaser = 0
For y = 0 To Frame.Lines - 1
For x = 0 To Frame.PixelPerLine - 1
'read red value of pixel
indexX = x * 3
indexY = Frame.Lines - y - 1
red = Frame(indexX + 2, indexY)
'recognize red pixel
If red > 100 Then
redPixel = redPixel + 1
xLaser = xLaser + x
yLaser = yLaser + y
'mark recognized pixel to cyan color
Frame(indexX + 2, indexY) = 0
Frame(indexX + 1, indexY) = 255
Frame(indexX + 0, indexY) = 255
End If
Next
Next
'calculate center of laser point
If (redPixel > 0) Then
xLaser = xLaser / redPixel
yLaser = yLaser / redPixel
'mark laser point with a green cross
For x = xLaser - 15 To xLaser + 15 Step 2
If x >= 0 And x < Frame.PixelPerLine And yLaser >= 0 And yLaser < Frame.Lines Then
indexX = x * 3
indexY = Frame.Lines - yLaser - 1
Frame(indexX + 2, indexY) = 0
Frame(indexX + 1, indexY) = 255
Frame(indexX + 0, indexY) = 0
End If
Next
For y = yLaser - 15 To yLaser + 15 Step 2
If xLaser >= 0 And xLaser < Frame.PixelPerLine And y >= 0 And y < Frame.Lines Then
indexX = xLaser * 3
indexY = Frame.Lines - y - 1
Frame(indexX + 2, indexY) = 0
Frame(indexX + 1, indexY) = 255
Frame(indexX + 0, indexY) = 0
End If
Next
End If

'show the frame
IcImagingControl1.DisplayImageBuffer(Frame)

End Sub

I notice also another thing: with OverlayBitmap instructions I see a little delay between the original video and the overlay marked. With directly write to frame buffer there aren't delay between laser point movement and marker movement.

[Stefan Geissler]

Hello,

Nice done.

The Overlay delay occurs, because it will be drawn on the next frame. Not on the current.

[Droise]

I'm trying to optimize performance of my software. With the bayer filter of color camera, read all 3 color of all pixels is not significant. Each pixel has only one significant color. I did a test with camera setted on Y800 mode, reading red color (to find the laser point) only from real red pixel of the image. The resulted max frame rate is now 60 fps, with increase of performance. The reading of red color from other pixels is useless, and it slows process.

Is my argument right?

Another question: in my application i'd like to save some frames in bmp format, but i prefer to have these image in color format.
I tried to do a software debuyering, and it work well, rebuilding the original color image, but obviously it slows the process. I see that i can set the camera in YB8 format, and i have a color image without less frame rate (camera can run to 60 fps, in the same of Y800 format).
In this mode, debuyering calculation don't weigh to the computer software. I can read only red-significant pixels, like in Y800 mode, to have a light and fast process. Can I read the same pixels like in Y800 mode? How this camera do color interpolation?

I see that color maps in Y800 mode is this:

G B G B
R G R G
G B G B
R G R G

Is it correct?
Is the real color pixel map the same in BY8 format?

Thank you!

[Stefan Geissler]

Hello,

I'm trying to optimize performance of my software. With the bayer filter of color camera, read all 3 color of all pixels is not significant. Each pixel has only one significant color. I did a test with camera setted on Y800 mode, reading red color (to find the laser point) only from real red pixel of the image. The resulted max frame rate is now 60 fps, with increase of performance. The reading of red color from other pixels is useless, and it slows process.

Is my argument right?

Yes.

Another question: in my application i'd like to save some frames in bmp format, but i prefer to have these image in color format.
I tried to do a software debuyering, and it work well, rebuilding the original color image, but obviously it slows the process. I see that i can set the camera in YB8 format, and i have a color image without less frame rate (camera can run to 60 fps, in the same of Y800 format).
In this mode, debuyering calculation don't weigh to the computer software. I can read only red-significant pixels, like in Y800 mode, to have a light and fast process. Can I read the same pixels like in Y800 mode? How this camera do color interpolation?

I see that color maps in Y800 mode is this:

G B G B
R G R G
G B G B
R G R G

Is it correct?
Is the real color pixel map the same in BY8 format?

BY8 format is the same as Y800 in the camera, but DirectShow will automatically insert the DeBayertransform filter. The DeBayertransform filter transforms the incoming Y800 raw data into the colored RGB image you see.

You are right, you can access the pixels, that are red in the Y800 raw data, as you did on Y800 memory buffer. The values of the other pixels are interpolated. So you can step in row and column by two pixels each time.

[Droise]

Thank you!

Best regards
Droise Paolo
Sirio Systems

[Murat]

"Private Sub IcImagingControl1_ImageAvailable_1(ByVal sender As System.Object, ByVal e As TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs) Handles IcImagingControl1.ImageAvailable
Dim Frame As TIS.Imaging.ImageBuffer"

But it doesn't work, colud you help me please

[Stefan Geissler]

Hello Murat,

What does not work? If the event is not called, then you may set the "LiveCaptureContinuous" property of IC Imaging Control to true.