Ebook: Joint Key-frame Extraction and Object Segmentation for Content-based Video Analysis

This work is supported in part by the National Science Foundation (NSF) under Grant IIS-0347613 (CAREER) and the Department of Defense EPSCoR (DEPSCoR) under Grant W911NF-04-1-0221;. This work is partially published in IEEE Workshop on Motion and Video Computing, Breckenridge, Colorado, Jan. 5-6, 2005, and IEEE International Conference on Acoustics, Speech and Signal Processing, Philadelphia, PA, March 18-23, 2005.Joint Key-frame Extraction and Object Segmentation for Content-based Video AnalysisAbstractKey-frame extraction and object segmentation are usually implemented independently and separately due to the fact that they are on different semantic levels and involve different features. In this work, we propose a joint key-frame extraction and object segmentation method by constructing a unified feature space for both processes, where key-frame extraction is formulated as a feature selection process for object segmentation in the context of Gaussian mixture model (GMM)-based video modeling. Specifically, two divergence-based criteria are introduced for key-frame extraction. One recommends key-frame extraction that leads to the maximum pairwise interclass divergence between GMM components. The other aims at maximizing the marginal divergence that shows the intraframe variation of the mean density. The proposed methods can extract representative key-frames for object segmentation, and some interesting characteristics of key-frames are also discussed. This work provides a unique paradigm for content-based video analysis.Index Terms — Key-frame extraction, object segmentation, Gaussian mixture model, feature selection, cluster divergence.Joint Key-frame Extraction and Object Segmentation
Problem Formulation
Maximum Average Interclass Kullback Leibler Distance (MAIKLD)
Maximum Marginal Diversity
Proposed Algorithm
Key-frame Characteristics
Simulations and Discussions
Object Segmentation
Key-frame Extraction
LimitationsXiaomu Song, Member, IEEE, and Guoliang Fan, Senior Member, IEEE.
X. Song was with the School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK 74078, USA. He is now with Northwestern University, Evanston, IL 60208, USA, email: [email protected]. G. Fan is with the School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, OK 74078, USA, email: [email protected]. Guoliang Fan is the contact author.