Time Mapping Using Space-Time Saliency
Sampling and filtering using our system. The sampling is denser in the middle to stretch out a little the more interesting part of the video.
People
- Feng Zhou (CMU)
- Sing Bing Kang (Microsoft Research)
- Michael Cohen (Microsoft Research)
Introduction
We describe a new approach for generating regular-speed, low-frame-rate (LFR) video from a high-frame-rate (HFR) input while preserving the important moments in the original. We call this time-mapping, a time-based analogy to high dynamic range to low dynamic range spatial tone-mapping. Our approach makes these contributions:
- A robust space-time saliency method for evaluating visual importance;
- A re-timing technique to temporally resample based on frame importance;
- Temporal filters to enhance the rendering of salient motion.
Results of our space-time saliency method on a benchmark dataset show it is state-of-the-art. In addition, the benefits of our approach to HFR-to-LFR time-mapping over more direct methods are demonstrated in a user study.
Videos
This video spotlight summarizes the main problem and our contributions.
Download the [Video 28MB] .
This video shows the results on the saliency benchmark dataset and the high-speed video dataset.
Download the [Video 24MB]
or its [High-Quality Version 154MB].
Code
The space-time saliency implementation is available at here.
Publications
-
[1]Time-Mapping Using Space-Time SaliencyIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
F. Zhou, S.-B. Kang and M. Cohen
Acknowledgements
Our initial investigation with Eduardo Gastal on processing high-speed videos helped to provide focus on our time-mapping project. We would also like to thank him and Patrick Meegan for capturing the high-speed video clips used in this paper.