PDA

View Full Version : image processed live video



javorszk
March 18, 2004, 17:49:31
Hello!

I downloaded the imagedataaccess.zip (for the trial version of imaging-control), which contains a sample that grab an image from the live video and inverts it. My question is: How can it be done continously? I mean: "real time image processing" instead of live video. For example color segmentation.

Thanks, Robert.

Stefan Geissler
March 19, 2004, 10:46:15
Hello,

To perform continuous image processing, you should derive a class from GrabberListener. See Listener.* in the callback sample.
An object of this class must be added as listener to the grabber.
The sinktype must be set to eGRAB (See FrameGrabberSink). If all if this is done, the frameReady() ,method is called for every frame, that is delivered by the grabber. Please see the remarks in the callback sample.

javorszk
March 20, 2004, 15:57:00
Ok. I see, but I'd rather write down what I did until now. (sorry)

I added the listener.* files to the firststep project and removed the saveImage and the SetBufferSize methods from the .h file.

I did it because it's not necessarry for me to save the images. Next from the .cpp I removed the 2 lines from the contstructor of Clistener, and from the desctructor, too, and the 3 lines from the frameReady method which call the saveImage. And of course I removed the saveImage and the SetBufferSize methods from the cpp file.

And finally I added lines to the OnInitialUpdate() with the help of the user's guide and the callback.cpp below the "if( formatIndex < pVideoFormats->size()) {" line:

CListener *pcListener = new CListener();
m_pGrabber->getOverlay()->setEnable(true);
m_pGrabber->addListener( pcListener,
DShowLib::GrabberListener::eFRAMEREADY|
DShowLib::GrabberListener::eOVERLAYCALLBACK );

- next is setting the SinkColorformat like in the callback.cpp)
- SetSinkType with eGrab
- newMemBufferCollection with parameter 1. (I didn't know what number to
write here)
- SetActiveMemBufferCollection
- setVideoFormat
- setHWND
- startLive(true) // has to be true?

That's all. And it isn't work, it does nothing. I know i made wrong that'swhy I wrote it down. Could you help me in this problem and in reaching the pixels' RGB data?

Thank you very much.

Stefan Geissler
March 22, 2004, 08:46:37
Hello Javorszk

First of all i can not see, what you have done wrong. I suggest to use the DemoApp sample and enhance it as you have posted. The OnInitialUpdate() is not a good method to initialize the grabber.

I will now describe the same steps you have done for the DemoApp sample. I included and changed the CListener class as you did it too.

I added a member “m_cListener” to the CMainFrame class (mainframe.h). At the end of the constructor of CMainFrame the line


m_Grabber.addListener( &m_cListener);

is added.

If you would add the following line, you would see the Clistener::overlayCallback() be called.


m_Grabber.getOverlay()->setEnable(true);



Then i change all frame grabber modes from eSNAP to eGRab the CMainFrame class:


// Set the sinktype to 24 bit. See OnSettingsDevice() too.
m_Grabber.setSinkType(DShowLib::FrameGrabberSinkDS howLib::FrameGrabberSink::tFrameGrabberMode::eGRAB , DShowLib::eRGB24 ) );

This tells the the Grabber to call Clistener::frameReady every time a new frame is delivered.

We want to see the processed image, not the original live video. Therefore, all occurrences of “startLive()” must be changed to “startLive(false)” in CMainFrame. The display of the processed images will be done in the CListener class.

This is all to be done in CMainFrame.

Now lets do the image processing. The pixel format of the frames, that are delivered by the Grabber, are determined by the sink type, not by the video format of the camera. In our case, it has been set to
DShowLib::eRGB24, that means three bytes per pixes. The first byte is blue, the second green and third one is red.
To process the images, you need to enhance the CListener::frameReady( ) method. It is very important to lock the current processed image buffer, so it will not be overwritten by the Grabber:


void CListener::frameReady( Grabber& param, smart_ptr<MemBuffer> pBuffer, DWORD currFrame)
{
pBuffer->lock();
// Here do the image processing
pBuffer->unlock();
}


We also need to know the image dimensions and the bits per pixel. Even if we know we use a RGB 24 sink type, we should make this function as flexible as possible. Therefore we query the BITMAPINFOHEADER from the image buffer:


smart_ptr<BITMAPINFOHEADER> pInf = pBuffer->getBitmapInfoHeader();

We also need a pointer to the image data:


BYTE *pPixel = pBuffer->getPtr();

Now the Clistener::frameReady() looks as follows:


void CListener::frameReady( Grabber& param, smart_ptr<MemBuffer> pBuffer, DWORD currFrame)
{
pBuffer->lock();
smart_ptr<BITMAPINFOHEADER> pInf = pBuffer->getBitmapInfoHeader();
BYTE *pPixel = pBuffer->getPtr();
// Here do the image processing
pBuffer->unlock();
}


With the BITMAPINFOHEADER we can calculate the number of bytes to be processed:


unsigned int iBytecount = pInf->biWidth * pInf->biBitCount / 8 * pInf->biHeight;

Now we can setup the image processing loop:


for( unsigned int i = 0; i < iBytecount; i++)
{
*pPixel = (BYTE)(0xFF - (*pPixel));
pPixel++;
}

The result of your image processing is not displayed yet, therefore we need to write some code doing this. We need a device context for drawing. Therefore we take the Grabber's HWND and get it's HDC:


HDC hDC = ::GetDC(param.getHWND());

Then we perform the drawing on this hDC:


int nLines = SetDIBitsToDevice( hDC,// Handle to the device
0,
0,
pInf->biWidth, // Source rectangle width
pInf->biHeight, // Source rectangle height
0, // X-coordinate of lower-left corner of the source rect
0, // Y-coordinate of lower-left corner of the source rect
0, // First scan line in array
pInf->biHeight, // Number of scan lines
pBuffer->getPtr(), // Modified address of array with DIB bits
reinterpret_cast<LPBITMAPINFO>( &*pInf ),
DIB_RGB_COLORS // RGB or palette indices
);

Do not forget to release the DC:


::ReleaseDC(param.getHWND(),hDC);


Now we are finished. The CListener::frameReady( ) method for image inverting looks like follows:


void CListener::frameReady( Grabber& param, smart_ptr<MemBuffer> pBuffer, DWORD currFrame)
{
pBuffer->lock();

smart_ptr<BITMAPINFOHEADER> pInf = pBuffer->getBitmapInfoHeader();
BYTE *pPixel = pBuffer->getPtr();
unsigned int iBytecount = pInf->biWidth * pInf->biBitCount / 8 * pInf->biHeight;

for( unsigned int i = 0; i < iBytecount; i++)
{
*pPixel = (BYTE)(0xFF - (*pPixel));
pPixel++;
}

HDC hDC = ::GetDC(param.getHWND());
int nLines = SetDIBitsToDevice( hDC,// Handle to the device
0,
0,
pInf->biWidth, // Source rectangle width
pInf->biHeight, // Source rectangle height
0, // X-coordinate of lower-left corner of the source rect
0, // Y-coordinate of lower-left corner of the source rect
0, // First scan line in array
pInf->biHeight, // Number of scan lines
pBuffer->getPtr(), // Modified address of array with DIB bits
reinterpret_cast<LPBITMAPINFO>( &*pInf ),
DIB_RGB_COLORS // RGB or palette indices
);

::ReleaseDC(param.getHWND(),hDC);
pBuffer->unlock();
}


The complete sample project is attached to this post.

javorszk
March 22, 2004, 11:12:33
Thank you for your help, it helped a lot. But I have still a little question. How can I get the index of the pPixel in the "for" loop?

I change the "for" loop, namely: instead of the iBytecount I put the iPixelCount, defined as:

int iPixelCount = iBytecount / pInf->biBitCount * 8;

And after that I can reach the individual BGR data, but it could be easier if I knew the index in each loop.

Stefan Geissler
March 22, 2004, 11:56:58
Hello,

The index of the pixel is calculated as follows:


int getIndex( int iX, int iY, smart_ptr<MemBuffer> pBuffer)
{
int iIndex
int iBytesPerLine;
smart_ptr<BITMAPINFOHEADER> pInf = pBuffer->getBitmapInfoHeader();
iBytesPerLine = pInf->biWidth * pInf->biBitCount /8;
iIndex = iY * iBytesPerLine + iX * pInf->biBitCount /8;
return iIndex;
}

printf(“Blue value = %d\n” pPixel[getIndex(10,20,pBuffer)];
printf(“Green value = %d\n” pPixel[getIndex(10,20,pBuffer)+ 1];
printf(“Red value = %d\n” pPixel[getIndex(10,20,pBuffer)+ 2];

Not very efficient, but should work.