PDA

View Full Version : "Synchronizing" of ICImagingControl and µC



Matthias88
August 23, 2010, 11:20:46
Hello,

I am currently working on my bachelor project using the DMK 21BF04 from IC Imaging Source.

First I would like to describe the running hardware:
The camera is running untriggered (freemode) at 60Hz. The camera's strobe out is connected to a microcontroller's gpio. Each time a strobe impulse is detected by the controller he starts flashing one of twelve leds (from led_1 to led_12). When he reached the twelfth led he starts again with the first led, so the sequence is always the same. So the camera continuously sends a sequence of twelve different images, flashed by twelve different leds, to the pc.

On my pc I am using the IC Imaging Control 3.0 software (C#). The software is running in LiveContinuousMode (ImageBufferSize = 12). So the image ring buffer always holds all different images of all different leds in correct order.
When I press a special button on my gui, I would like to immediately stop this process, lock the buffers and save the twelve images (holded in buffer) to my hdd. Up to here everything works fine...

Now my problem:
I must save them to my hdd in the correct led order, e.g.:
pic_0.bmp (flashed by led_1)
pic_1.bmp (flashed by led_2)
pic_2.bmp (flashed by led_3)
... and so on
To do this I send a message to my controller (after I pressed the button to stop the continuous process) asking for the last led he flashed (the controller sends the led number synchronously to the next strobe pulse). This way I know that ICImagingControl.ImageActiveBuffer has been flashed by the requested led and then the other images are easy to calculate...

The problem is, when using an exposure > ~3-4ms, there is always a difference of one image between the last flashed led and the last image in the image buffer. So lets say the controller says the last flashed led was led_3, then the ImageActiveBuffer will hold the image of led_2.
If I use a shorter exposure the last led number and the ImageActiveBuffer correspond to each other corretly.

Why does this happen?
Is it possible that, when using a short exposure, the images will arrive on the pc before the next exposure has been started?
Is there any better way to "synchronize" the pc and the controller?

I am using a loop to lock the buffers


for(int i = 0; i < icImagingControl1.ImageRingBufferSize; i++)
{
ImageBuffers[i].Lock();
}
Is it possible that there will arrive a new frame while locking the buffers in the ring buffer?

I hope my English was quite understandable...

Thank you very much in advance!

With kind regards,
Matthias

Stefan Geissler
August 24, 2010, 13:41:34
Hello Matthias,

My fist suspect is, that there is a problem with your micro controller detecting the strobe signal, if you use very short exposure times. As far as I know, the default setting for strobe is "exposure". That means, the duration of the high signal of strobe is as long as the exposure time is. Maybe this is too short for your micro controller. In this case, I suggest to set the strobe mode to duration and the strobe duration to 10ms for 60 fps.

You can test this with IC Capture.

If you want to continue in German language, you may create a support case at http://www.imagingcontrol.com/en_US/support/case/

Matthias88
August 24, 2010, 14:55:06
Hello Mr. Geissler,

there shouldn't be any problem detecting the strobe signal. I've check this many times using an oscilloscope, the micro controller detects each strobe signal correctly and starts flashing the correct led for the specified duration. I am using a Cortex-M3 micro controller (@75MHz) and he was also able to detect the strobe signal when using a fixed duration of 100µs.

I tried to sketch my guess how the problem occurs (attached file). Is that a possible scenario?

With kind regards,
Matthias

Stefan Geissler
August 24, 2010, 15:46:18
Hello,

ok, if the micro controller an der LEDs are fast enough, then we must look for the software.

It is possible, that a new frame is copied into the image while your locking loop runs, but this should be a very random incident. You loop is very fast and should be executed within µs. In this time no new frame is provided. The frames are provided within clock of the frame rate, not faster. That means, the software should behave in the same way, regardless of the exposure time being below 4ms or above. The images are send each time with the same speed. You must imagine, for short exposure times, the begin of exposure is moved forward in the 60 fps time intervall. That means, short exposures start later than long exposurers. The delivery of the frame to the computer starts at the same point of time in the clock.

Thus, the image delivery can not be the problem.

Now again a guess: Is it possible, that there is a general problem, because the ImageRingBuffer does not start with index 0 but with index 1 (I am not sure about this right now), thus using short exposure times shows the last LED because of inertia (Trägheit)? May be it is a good idea to find the oldest image in the ringbuffer (see the sample start time of the image buffers) and start saving the images from this buffer? Please see the sample at http://www.imagingcontrol.com/en_US/library/dotnet/iterating-an-image-sequence/ how to do this.

I hope, I made my point a little bit clear.

Matthias88
August 24, 2010, 18:20:55
You must imagine, for short exposure times, the begin of exposure is moved forward in the 60 fps time intervall. That means, short exposures start later than long exposurers.
Does that mean that when I use short exposure times the camera won't start the exposure at the same time the strobe signal is sent?
Or to put it in other words: When I start flashing the leds at the same time the strobe signal is sent, the camera might not have started to expose yet? So that I would "loose" some of the led light?
That would be terrible because I must know the exact start of an exposure to flash the leds for example exactly 5ms within the exposure!?


Is it possible, that there is a general problem, because the ImageRingBuffer does not start with index 0 but with index 1 (I am not sure about this right now), thus using short exposure times shows the last LED because of inertia (Trägheit)?
I didn't get your point exactly here... I know that the ImageRingBuffer starts with index 1 so that the last frame, before overwriting, will be at index 0. But at the moment I am not looking for the indexes of the images when saving them to hdd, but I am looking for the "youngest" image (what I supposed to be the last flashed by the micro controller). Is there any difference when looking for the oldest one?
I didn't get your point about the inertia... could you explain it a little bit more detailed please?

With kind regards,
Matthias

Stefan Geissler
August 25, 2010, 10:03:04
Hello Matthias,

First of all, the camera tries to fire the strobe as near as possible to the exposure start.

Secondly, you can not predict the count of exposures are strobe signals, after you have clicked you special buttons. For long exposure times this means, the exposure and strobe are fired, but the camera got the signal to stop image delivery. Thus no image was delivered, but your micro controller got a strobe signal and switches to the next LED. This is the theory after a longer discussion with the firmware programmer. In summary, the problem is, you cannot know, what happens immediately after you clicked your special button.

Thus we should check, whether we can find a well determined way to expose and provide images. This depends on your task. If you do not need to run with exactly 60 fps, but e.g. 59, then I suggest to upgrade the firmware in your DMK 21BF04, thus you can use software trigger. That means, your program commands the camera to expose and send exactly one frame each time, you "push" the software trigger. (What is the serial number of your camera? Maybe it already supports the software trigger.) For using the software trigger, see below.



// Function for enabling the trigger
public static bool SetTrigger(ICImagingControl ic, bool OnOff )
{

VCDSwitchProperty Trigger;
Trigger = (VCDSwitchProperty)ic.VCDPropertyItems.FindInterfa ce(VCDIDs.VCDID_TriggerMode + ":" + VCDIDs.VCDElement_Value + ":" + VCDIDs.VCDInterface_Switch);

if (Trigger != null)
{
Trigger.Switch = OnOff;
return true;

}
return false;
}

// Enable the trigger
SetTrigger(_ic, true);
// Start LiveVideo
_ic.LiveStart();

//Gather the softtrigger property
VCDButtonProperty SoftTrigger = (VCDButtonProperty)_ic.VCDPropertyItems.FindInterf ace(VCDIDs.VCDID_TriggerMode + ":{FDB4003C-552C-4FAA-B87B-42E888D54147}:" + VCDIDs.VCDInterface_Button);

// Generate the softtriggerpulse
if (SoftTrigger != null)
{
for( int i = 0; i < 13; i++ )
{
SoftTrigger.Push();
System.Threading.Thread.Sleep(17);
}
}


This is a simple approach. Better is to use the ImageAvailable event for checking, whehter an image was receive after the softtrigger pulse was fired.

What do you think about this?

Matthias88
August 25, 2010, 10:36:21
Hello Mr. Geissler,

my first thought for a possible alternative was to run the camera in trigger mode and trigger it with the micro controller... but I wasn't sure if this would really solve the problem!?

But I had no idea that it is also possible to trigger the camera by using a pc-sided software trigger. I really guess that this could solve the problem because I always know when an exposure starts and when the corresponding image arrives in the ImageRingBuffer, that would be great! (I guess the camera would be still able to send strobe signals when starting an exposure!?)

Is the software trigger a special feature of the DMK 21BF04 camera or is it also available on other ImagingSource cameras? Because it could be possible that the final camera used for this project could be a different ImagingSource camera (I don't know which one at the moment).
The serial number of the camera I am using right now is: 15800062

If this could solve the problem that would be a wonderful solution!

Thank you very much!

Best wishes,
Matthias

Stefan Geissler
August 25, 2010, 10:45:11
Hello


my first thought for a possible alternative was to run the camera in trigger mode and trigger it with the micro controller... but I wasn't sure if this would really solve the problem!?

It solves the problem, because in this case, you know what the micro controller does and the micro controller controls the camera.


But I had no idea that it is also possible to trigger the camera by using a pc-sided software trigger. I really guess that this could solve the problem because I always know when an exposure starts and when the corresponding image arrives in the ImageRingBuffer, that would be great! (I guess the camera would be still able to send strobe signals when starting an exposure!?)

Yes, the strobe corelates with the exposure.

The software trigger was invented in the last firmware, thus you will have to upate the firmware. This can be done by software using a Windows XP system. I do not provide the firmware update through the forum, thus please generate a simple support case here: http://www.imagingcontrol.com/en_US/support/case/

Matthias88
August 25, 2010, 11:00:25
Ok, then I will first try to solve it using the micro controller to trigger the camera (to stay independent of the software trigger)
and if this doesn't work (for what reason ever) I am going to use the software trigger!

One last question for the moment:
When do the images/frames arrive in the ImageRingBuffer? Do they always arrive within the same period the camera exposed the related image, or do they always arrive within the next period when the camera starts the next exposure? Is there any documentation available to have a closer look at the camera's timing mechanism?

Best wishes,
Matthias

Stefan Geissler
August 25, 2010, 11:08:18
Hello,

The images arrive in the computer, after the exposure has ended. The time they arrive in your software depends highly on your computer and the CPU load. In best case, the images arrive in the computer in the frame rate's clock a few milliseconds after they have they arrived in the driver's buffer. However, in the worst case, this can last some time. As I wrote above, this highly depends on the computer.

No, there is no more information available. You may look in the sensor specification at http://www.theimagingsource.com/downloads/icx098bl.en_US.pdf

Matthias88
August 25, 2010, 11:24:39
Ok, thank you very much for the wonderful support!

I will update this thread as soon as I can report any success or when I have new questions! ;-)

Thanks again!

With kind regards,
Matthias

Matthias88
August 26, 2010, 10:05:09
Hello Mr. Geissler,

I updated the firmware of the camera and tried to use the software trigger example you posted above. The SetTrigger()-function seems to work absolutely fine but


ICImagingControl3.VCDButtonProperty SoftTrigger = (ICImagingControl3.VCDButtonProperty)_ic1.VCDPrope rtyItems.FindInterface(VCDIDs.VCDID_TriggerMode + ":{FDB4003C-552C-4FAA-B87B-42E888D54147}:" + VCDIDs.VCDInterface_Button);

always returns a null reference. How can I solve this problem and what does the code
FDB4003C-552C-4FAA-B87B-42E888D54147 mean?


My function to use the trigger:

private void button2_Click(object sender, EventArgs e)
{
ICImagingControl3.VCDButtonProperty SoftTrigger = (ICImagingControl3.VCDButtonProperty)_ic1.VCDPrope rtyItems.FindInterface(VCDIDs.VCDID_TriggerMode + ":{FDB4003C-552C-4FAA-B87B-42E888D54147}:" + VCDIDs.VCDInterface_Button);

// Generate the softtriggerpulse
if (SoftTrigger != null)
{
SoftTrigger.Push();
}
else
{
MessageBox.Show("No Trigger!");
}
}

With kind regards,
Matthias

Stefan Geissler
August 26, 2010, 15:14:51
Please excuse the long delay of my answer, I was a little bit busy.

I would like you to uninstall the camera driver from the Device Manager and install it again. The version should be 4.1.1.4 or higher. The latest driver versions can be found at http://www.theimagingsource.com/de_DE/support/downloads/

The driver saves some properties in the registry, thus new properties sometimes are not displayed after a firmware upgrade.

Matthias88
August 27, 2010, 09:25:04
Ok, I uninstalled the camera driver, reinstalled the new (latest) driver v4.1.1.4 but it still get a null reference...

Is it possible that this code is wrong?

FDB4003C-552C-4FAA-B87B-42E888D54147

Or is it possible that I need the latest version of ICImagingControl?

With kind regards,
Matthias

Stefan Geissler
August 27, 2010, 09:50:33
You also need the latest version of IC Imaging Control. I send the download by email to you.

Matthias88
August 27, 2010, 10:01:00
I tried to install the new software but it says that both keys are invalid.
Edit: Sorry, it seems to work now!

Stefan Geissler
August 27, 2010, 10:21:43
Well, I was fast by sending an email to you :-)

Text was:
Both keys are correct. Please keep in mind, they are case sensitive, there are not blanks within
and also make sure, there are no extra characters, which may are added by using copy & paste.

Matthias88
August 27, 2010, 12:09:18
Ok, thanks! I guess I added a space character to each key when using copy & paste :curl-lip:

The software trigger works absolutely fine now!

Now I have got another question regarding the software...

I would like to "simulate" the free running mode of the camera.
So I
1. push the software trigger
2. use the ImageAvailableEvent to wait untill the image arrived in the ImageRingBuffer
then push the software trigger again and so on.
This should be an infinite loop that starts when pressing LiveStart() and stops when pressing LiveStop().

The problem is when using this loop in my gui, the gui is blocked for the whole time.
When using it in a backgroundworker thread I always get the following exception after pressing LiveStart():

Unknown error occurred
Unknown error occurred
Base Library Error : An unexpected DShowLibException occured: Error = startLive failed.
VideoCaptureDevice = DMx 21BF04
VideoFormat = Y800 (640x480)

The loop and the ImageAvailbleEvent:


private void PulseCamera() {
while (true)
{
if (icImagingControl1.LiveVideoRunning)
{
VCDButtonProperty SoftTrigger = (VCDButtonProperty)icImagingControl1.VCDPropertyIt ems.FindInterface(VCDIDs.VCDID_TriggerMode + ":{FDB4003C-552C-4FAA-B87B-42E888D54147}:" + VCDIDs.VCDInterface_Button);

// Generate the softtriggerpulse
if (SoftTrigger != null)
{
waiting_for_frame = true;
SoftTrigger.Push();
while (waiting_for_frame)
{
Thread.Sleep(1);
}
}
}
}
}

private void icImagingControl1_ImageAvailable(object sender, TIS.Imaging.ICImagingControl.ImageAvailableEventAr gs e)
{
waiting_for_frame = false;
}


What would be the best way to realise the simulation? What would be the best "place" to put the loop?

Thanks again!

Best wishes,
Matthias

Stefan Geissler
August 27, 2010, 13:26:26
Multithreading can cause problems, thus why don't you use a timer event instead of a loop?

Matthias88
August 27, 2010, 16:14:59
Multithreading can cause problems, thus why don't you use a timer event instead of a loop?
Some days I really feel to be a blockhead :doh:

I am going to try this on Monday... ;-)

Thanks again and have a nice weekend!

Best wishes,
Matthias

Matthias88
September 13, 2010, 16:07:32
Hello again,

after trying the software trigger I now would like to test to trigger the camera with the micro controller. I set the camera to 60Hz and generated a trigger pulse at an interval of 17ms. I used the Image Capture software to display the images on pc but I noticed that the images came in with about 30Hz, so only half of the possible frequency.

Is this because after having made a picture the camera needs another 16ms to send the image to the pc before accepting another trigger?
If so, the maximal frequency in trigger mode would be 30Hz... so it should be sufficient to trigger the camera with 30Hz for the maximal frequency!? But when doing so the camera only runs at about 15Hz, why?

Thanks!

Best wishes,
Matthias

Stefan Geissler
September 14, 2010, 12:00:11
Hello Matthias,

this sounds for me, that the expsoure time is too long. Please try shorter times like e.g. 1/1000 second, otherwise the camera will switch down to half frame rate. Please keep in mind, the camera exposes and sends the image in sequences. Exposure time and delivery time must fit into the 16.666 ms frame rate time intervall. (The image deliverey needs a little bit less than 16.666 ms.)

Matthias88
September 27, 2010, 13:02:12
Hello,

in fact it depends on the exposure time. When running the camera @60Hz and using an exposure time <= 1ms the images will be delivered @60Hz too. When using longer exposure times the images will be delivered only ~@30Hz.

In the previous post you wrote
The image deliverey needs a little bit less than 16.666 ms. So I thought when running the camera @30Hz I could use exposure times up to ~15ms before the camera will switch down to half frame rate. But I just could use ~2ms before noticing the camera switching down. So does the delivery of images take longer when running the camera @30Hz or @15Hz?

Another question:
What does exactly happen when invoking ICImagingControl.LiveStart()? I noticed that, when running the camera in free running mode, the strobe pulse will first be send after I invoked LiveStart().
Does LiveStart() just start the delivery of images to the pc or does it even start the camera making exposures and so on?
Or in other words: Does the first strobe pulse that is send after invoking LiveStart always belong to the first image that is delivered to the internal ring buffer (at ICImagingControl.ImageBuffers[1]) when ICImagingControl.LiveCaptureContinuous = true?

Thanks a lot!

Best wishes,
Matthias

Stefan Geissler
September 27, 2010, 15:28:05
You are right, if the frame rate is set to 30 fps, then the clock of the camera is halved. Also the transfer speed is reduced. That means, the data transfer of image needs 33ms instead of 60ms at 60 fps.

The call to liveStart starts the camera, that means, the camera starts exposing and providing images. That means, the sensor is reset, exposure time is set. For some properties, the sensor accepts the values after a frame has been sent. However, this is firmware internal stuff, that will not be discussed in public.


Or in other words: Does the first strobe pulse that is send after invoking LiveStart always belong to the first image that is delivered to the internal ring buffer. (at ICImagingControl.ImageBuffers[1])?
I have to admit, that I am not able to answer the question reliably, because I never tested this on my own.

Matthias88
September 27, 2010, 23:02:51
Thank you very much for the reply!


I have to admit, that I am not able to answer the question reliably, because I never tested this on my own.
Hm, ok too bad.

I realised the synchronisation using the micro controller to trigger the camera. The synchronisation works absolutely fine this way, but very often an error occurs (see attachement) after invoking my own StartLiveVideo() or StopLiveVideo() function. Maybe I do something wrong when starting/stopping the LiveMode or the camera trigger!? I use the following functions:


private UInt16 GetFramePeriod()
{
return (UInt16)(Math.Ceiling(10000.0f / icImagingControl1.DeviceFrameRate));
}

private void StartLiveVideo()
{
icImagingControl1.LiveStart();
btCamStart.Text = LBL_STOP;
ControllerControl.SetCameraTrigger(GetFramePeriod( )); // sends an command to the micro controller to start triggering the camera with the defined frame rate
}
private void StopLiveVideo()
{
ControllerControl.SetCameraTrigger(0); // sends an command to the micro controller to stop triggering the camera
icImagingControl1.LiveStop();
btCamStart.Text = LBL_START;
}

Is it ok to start the trigger after invoking LiveStart()?
Is it ok to stop the trigger before invoking LiveStop()?
What would be a common way to start/stop the camera trigger signal generated by the micro controller?

Regards,
Matthias

Stefan Geissler
September 28, 2010, 09:27:32
Is it ok to start the trigger after invoking LiveStart()?
Yes, if you use the VCD Properties.


Is it ok to stop the trigger before invoking LiveStop()?
Yes, if you use the VCD Properties.

For the error message: "Details anzeigen" may give some more information. Also surrounding the LiveStart by a try...catch block with error handling can help to narrow down the problem. Usually LiveStart fails, if the live video is already running.

Matthias88
September 29, 2010, 12:28:56
Ok, I am going to try this!

Another question:

When setting ICImagingControl.LivePause = true; the documentation says:
If LivePause is set to True, no images are delivered.
Does this mean that the camera will also stop to exposure and therefore doesn't send any strobe pulse while ICImagingControl.LivePause is set to true?

[I am still trying to find a proper solution when using the camera in free running mode]
What I am trying to do is to realise something like a synchronisation point between the camera and the micro controller. Because the internal image ring buffer is as long as the led sequence to flash, the only thing I would need is to know one image buffer and its correlating led, something like an "entry point" for the sequence. Then all the other leds could be calculated easily because they wouldn't change their absolute position in the buffer.
But to realise this I would need a possibility to start the camera making images and the controller flashing the leds at the same time and to make sure that the first image which arrives in the internal ring buffer was flashed by the first led.

Could LivePause be such a possibility?

EDIT: I also tried to use ICImagingControl.DeviceCurrentActualFrameRate but it always says that there is no definition for "DeviceCurrentActualFrameRate" in "TIS.Imaging.ICImagingControl". Do I have to include a special namespace to use this property?


Best regards,
Matthias