Evaluating Film Model For Controlled Missing-Frame Reconstruction In Low-Light Smartphone Video
DOI:
https://doi.org/10.5281/zenodo.23795151Keywords:
Video Frame Interpolation, Missing-Frame Reconstruction, FILM Model, Low-Light Video, Smartphone Video, Frame Loss Simulation, Structural Similarity Index Measure, SSIM, Optical Flow, Bidirectional Warping, Video Restoration, Deep LearningAbstract
This study evaluates Frame Interpolation for Large Motion (FILM) for the controlled reconstruction of intentionally withheld frames from a researcher-owned low-light video recorded with a smartphone camera. The source video was standardized to 1280 x 720 pixels and prepared in four editorial transition conditions: no transition, cross dissolve, dip to black, and dip to white. Ten target frames were selected from each condition and evaluated at frame gaps of 1, 3, and 5, producing 120 primary Structural Similarity Index Measure (SSIM) observations. For each observation, FILM synthesized an intermediate frame from two retained boundary frames; the synthesized image was compared with the withheld original frame. The aggregate mean SSIM was 0.9504 at frame gap 1, 0.9354 at frame gap 3, and 0.9152 at frame gap 5. These results indicate high structural similarity throughout the evaluated conditions, while also showing the expected degradation as temporal separation increases. The study establishes a traceable, narrowly scoped baseline for a single low-light smartphone asset. It does not evaluate network packet loss, end-to-end video quality of service.
References
[1] F. Reda et al., “FILM: Frame Interpolation for Large Motion,” ECCV, 2022. doi: 10.1007/978-3-031-20047-2_3.
[2] W. Bao et al., “Depth-Aware Video Frame Interpolation,” CVPR, 2019, pp. 3703-3712. doi: 10.1109/CVPR.2019.00383.
[3] Z. Huang et al., “Real-Time Intermediate Flow Estimation for Video Frame Interpolation,” ECCV, 2022, pp. 624-642. doi: 10.1007/978-3-031-19772-7_36.
[4] L. Lu et al., “Video Frame Interpolation with Transformer,” CVPR, 2022, pp. 3532-3542. doi: 10.1109/CVPR52688.2022.00352.
[5] D. M. Argaw and I. S. Kweon, “Long-Term Video Frame Interpolation via Feature Propagation,” CVPR, 2022, pp. 3543-3552.
[6] H. Jiang et al., “Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation,” CVPR, 2018, pp. 9000-9008.
[7] S. Niklaus, L. Mai, and F. Liu, “Video Frame Interpolation via Adaptive Separable Convolution,” ICCV, 2017, pp. 261-270.
[8] H. Lee et al., “AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolation,” CVPR, 2020, pp. 5316-5325.
[9] M. Choi et al., “Channel Attention Is All You Need for Video Frame Interpolation,” AAAI, 2020, pp. 10663-10671.
[10] Q. Hou, A. Ghildyal, and F. Liu, “A Perceptual Quality Metric for Video Frame Interpolation,” ECCV, 2022.
[11] Z. Wang et al., “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Trans. Image Process., 13(4), pp. 600-612, 2004. doi: 10.1109/TIP.2003.819861.
[12] Google Research, “Google Colab.” https://colab.research.google.com/.
[13] OpenCV, “OpenCV Documentation.” https://docs.opencv.org/.
[14] FFmpeg, “FFmpeg Documentation.” https://ffmpeg.org/documentation.html.
[15] TensorFlow, “TensorFlow Documentation.” https://www.tensorflow.org/api_docs.
[16] TensorFlow, “TensorFlow Hub.” https://www.tensorflow.org/hub.
[17] NumPy, “NumPy Documentation.” https://numpy.org/doc/.
[18] Matplotlib, “Matplotlib Documentation.” https://matplotlib.org/stable/.
[19] Google Research, “MediaPy Documentation.” https://google.github.io/mediapy/.
[20] ImageIO, “ImageIO Documentation.” https://imageio.readthedocs.io/.
[21] scikit-image, “skimage.metrics Documentation.” https://scikit-image.org/docs/stable/ api/skimage.metrics.html
[22] Google Research, “frame-interpolation: FILM implementation.” https://github.com/ google-research/frame-interpolation
[23] Z. Zhong et al., “Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame Interpolation,” ECCV, 2024.
[24] Y. Zhang et al., “LADDER: An Efficient Framework for Video Frame Interpolation,” arXiv:2404.11108, 2024.
[25] G. Wu et al., “Perception-Oriented Video Frame Interpolation via Asymmetric Blending,” CVPR, 2024, pp. 2753-2762.
[26] D. Sun, X. Yang, M.-Y. Liu, and J. Kautz, “PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8934-8943.
