Side Note: First Draft on Apr 13 2011.
Video boundary detection is a foundamental technique for video retrieval. There’re two different types of transitions, abrupt transition (also called cut) and gradual transitions, which includes fade in/fade out, dissolve, wipe, etc.
This article covers abrupt video transition detection and provide a simple implementation in Matlab.
Abrupt transitions are relatively easy to detect, as there’re normally a big difference between the two transition frames. The problem is equivalent to detect this big difference.
This difference can be measured on a pixel by pixel basis, in block-based manner, or based on some global characteristics of the frames, for example, color histogram and intensity histogram.
One of the effective way is intensity histogram. According to NTSC standard, the intensity for a RGB frame can be calculated as,
I = 0.299R + 0.587G + 0.114B
where R, G and B are Red, Green and Blue channel of the pixel.
For the intensity histogram difference we’re looking for, it can be expressed as,
where Hi(j) is the histogram value for ith frame at level j. G denotes the total number of levels for the histogram.
In a continuous video frame sequence, the histogram difference is small, whereas for abrupt transition detection, the intensity histogram difference spikes. Even there is a notable movement or illumination changes between neighboring frames, the intensity histogram difference is relatively small compared with those peaks caused by abrupt changes. Therefore, the difference of intensity histogram with a proper threshold is effective in detecting abrupt transitions.
The threshold value to determine whether the intensity histogram difference indicates a abrupt transition can be set to,
where mu and sigma are the mean value and standard deviation of the intensity histogram difference. The value of alpha typically varies from 3 to 6.
Implementation in Matlab
The implementation of the above method in Matlab can be found here. Note that sometimes gradual transitions can have spikes that even higher than those in abrupt transitions. In order to differentiate the abrupt transitions from graudal transitions, the neighboring frames of a detected spike are also tested, if there’re multiple spikes nearby, the transition is more likely to be gradual transition, and we simply drop this detection.
Test and Result
One of the video sequence I’ve tested present the following graph,
The straight line is the threshold, it’s clear that there’re two abrupt transitions in this video sequence.