Multi-modal Visual Features Based Video Shot Boundary Detection

Abstract

OAPA One of the essential pre-processing steps of semantic video analysis is the video shot boundary detection (SBD). It is the primary step to segment the sequence of video frames into shots. Many SBD systems using supervised learning have been proposed for years; however, the training process still remains its principal limitation. In this paper, a multi-modal visual features based SBD framework is employed that aims to analyse the behaviours of visual representation in terms of the discontinuity signal. We adopt a candidate segment selection that performs without the threshold calculation but uses the cumulative moving average of the discontinuity signal to identify the position of shot boundaries and neglect the non-boundary video frames. The transition detection is structurally performed to distinguish candidate segment into a cut transition and a gradual transition including fade in/out and logo occurrence. Experimental results are evaluated using the golf video clips and the TREC2001 documentary video dataset. Results show that the proposed SBD framework can achieve good accuracy in both types of video dataset compared with other proposed SBD methods.