View Full Version : Video synchronization with 2 DFG/USB2-lt devices

February 24, 2010, 16:01:46

I am trying to synchronize video input from 2 DFG/USB2-lt devices, with relative success so far. I rely on manual synchronization at the beginning to get an offset between the respective FrameNumber of each device received through the frameReady() callback. I store frames with their index in their repsective vector, and then grab dual frames from those 2 vectors. If I stop grabbing

When running on a powerful computer I have no desync issue at all. However when running the app on a lighter computer (netbook), I sometimes notice that frames start desynchronizing, meaning frames with the corresponding FrameNumber value are not synchronized any more...

I am thus wondering: how could the Grabber "miss" an internal frame copy? Would a frameReady callback method that takes too long be able to cause such a problem? And if so, is there a way to know if frames have been "dropped"? (apparently the device does not support the getCountOfFramesDropped method)

Stefan Geissler
February 24, 2010, 16:56:35
You can use the the sample times and check the time differences from frame to the next. If the time difference is bigger than the usual frame rate intervall, e.g. near 80ms, then you missed a frame.

February 25, 2010, 16:29:51
What do you mean by sample time? I did not find any form of frame timestamping in the documentation yet but I may have missed it.

So far I only saw the getCurReferenceTime method, which from the documentation looks like a QueryPerformanceCounter call... or does it return the "timestamp" of the last grabbed frame?

Stefan Geissler
February 25, 2010, 16:42:33
The image buffer has a property samplestartime. You can call getSampleDesc() on the ImageBuffer, that is passed to the frameReady() function.


February 26, 2010, 08:38:52
Ok great news! I'll give it a try and I can remove my own timestamping method then.

Thank you!

March 17, 2010, 14:53:13
Ok, using the timestamps I think I found the problem (and I correct what I said in my previous post, I have the same problem on a powerful computer).

To perform synchronization, I basically match frame indexes from one device to frame indexes from the other with a fixed offset. From what I understand, these indexes are built by the driver layer based on the Sample Time (I assume sample time is set by the USB device, which probably has an internal clock synchronized with the host, but only guessing here).

After taking a closer look at those sample times, I noticed they are consistently over 400'000 (which I assume is time in clock ticks, so 40ms), in the vicinity of 410'000 (so 41ms). I also noticed that each 20 frames (more or less), a frame has a Sample time of only around 20ms, which looks like an internal mechanism to account for drift due to computation (also guessing here).

I also noticed that when there is a gap in frame indexes from 2 successive notifications of the frameReady() callback, there is a corresponding gap in the successive sample time (so if I get framenumber 10, then framenumber 12, the sampletime difference between both frames is around 80 ms), which means the driver layer (still guessing) already has a mechanism to account for frame loss, probably based on sample time.

However when using 2 devices, there is drift somewhere. I tried to look at timestamps of frames coming from both devices. I recorded around 15 minutes of video from 2 devices using my synchronization mechanism. At the beginning the video was synchronized (visually), and looking at timestamps from 2 frames displayed simultaneously, the difference is of around 57 ms (not perfect synchro but visually acceptable). After 15 minutes of recording, 2 simultaneous frames had a timestamp difference of around 200 ms, which corresponds to the desynchronization I can observe visually watching the video (around 4 frames of delay).

From this experience I guess I will have to handle synchronization myself based on the sample times coming from both devices, and ignoring the frame indexes returned, which is not very convenient. I have a question though: is the clock of the devices really synchronized with the host's clock, as I guess it is?