应用残差生成对抗网络的路况视频帧预测模型

袁帅; 秦贵和; 晏婕

doi:10.7652/xjtuxb201810020

您当前的位置：

首页 >

文章列表页 >

应用残差生成对抗网络的路况视频帧预测模型

更新时间：2025-07-09

- 应用残差生成对抗网络的路况视频帧预测模型
- A Prediction Model for Video Frames of Road Conditions Using Residual Blocks Generating Adversarial Networks
- 西安交通大学学报 2018年52卷第10期页码：146-152+166
- 作者机构：
  
  吉林大学计算机科学与技术学院,长春,130000
- 作者简介：
- 基金信息：
  
  : 2018-03-28。作者简介: 袁帅(1992-),男,硕士生;秦贵和(通信作者),男,教授,博士生导师。基金项目: 吉林省重点科技攻关资助项目(20150204034GX)
- DOI：10.7652/xjtuxb201810020
  中图分类号： TP38
- 网络首发：2018-10-10，
  
  纸质出版：2018
- 稿件说明：
移动端阅览
袁帅, 秦贵和, 晏婕. 应用残差生成对抗网络的路况视频帧预测模型[J]. 西安交通大学学报, 2018,52(10):146-152+166.

A Prediction Model for Video Frames of Road Conditions Using Residual Blocks Generating Adversarial Networks[J]. 2018, 52(10): 146-152+166.
袁帅, 秦贵和, 晏婕. 应用残差生成对抗网络的路况视频帧预测模型[J]. 西安交通大学学报, 2018,52(10):146-152+166. DOI： 10.7652/xjtuxb201810020.

A Prediction Model for Video Frames of Road Conditions Using Residual Blocks Generating Adversarial Networks[J]. 2018, 52(10): 146-152+166. DOI： 10.7652/xjtuxb201810020.

摘要

在路况视频帧的预测领域中

针对当前大部分模型所存在的预测图像分辨率低、图像模糊和局部细节缺失等问题

提出了一种应用残差生成对抗网络的路况视频帧预测模型(RB-GAN)。该模型用于在给定一段路况视频流的情况下更好地预测未来的一帧路况图像

应用多个级联的残差模块初步提取输入视频流的图像特征; 利用感知网络强化对视频流中物体运动特征的提取; 通过使用双重判别器提高生成对抗网络生成的图像的质量; 用Adam方法来优化网络权值的深度学习过程。基于生成对抗网络这种半监督的学习框架

训练后的模型可以预测出一段路况视频流下一时刻的同输入视频流具有时空一致性的帧图像。应用车辆检测领域常用的KITTI数据集对生成对抗网络模型进行训练和测试

实验结果表明:与主要依赖于像素均值的方法相比

RB-GAN模型预测图像的分辨率提高了2～4倍

达到256像素×512像素

在图像锐度标准上提高了1～2个数量级

同时图像也更加符合人眼视觉的主观感受

所预测得到的路况视频帧图像质量更高

更具有实用性价值

可以更好地为诸如检测算法等其他下游算法提供有效的特征信息。

Abstract

A novel model to predict video frames of road conditions is proposed to solve the problem that in the prediction area of road condition video frames

most of predicting models are of such as low resolution

image blurring and lack of local details

and the model uses residual blocks to generate adversarial networks. Multi-cascade residual units are used to extract feature maps preliminarily and two discriminators are adopted to raise the quality of images generated by the GANs. The model power to extract feature maps is improved by using perceptual loss network

and the Adam optimizer is used to update network weights. The proposed model produces future frames in the next moment which have great temporal coherency with the input frames through using semi-supervised deep learning framework. The model is trained and tested using datasets KITTI

and the results show that the generated frames by the proposed model have the resolution of 256×512

which are 2 to 4 times higher than the results from pixel mean based functions and past foreign works

the sharpness accuracy increases by 1 to 2 magnitudes

and the generated frames are more responsive to human visual perception. The frame images predicted by the model are of higher quality and more practical value

and provide more effective features for other downstream models such as detection algorithms.

关键词

Keywords

references

SRIVASTAVA N, MANSIMOV E, SALAKHUDINOV R. Unsupervised learning of video representations using LSTMS [C]∥Proceedings of the 32nd International Conference on Machine Learning. New York, USA: ACM, 2015: 843-852.

HOCHREITER S, SCHMIDHUBER J. Long short-term memory [J]. Neural Computation, 1997, 9(8): 1735-1780.

BHATTACHARJEE P, DAS S, BHATTACHARJEE P, et al. Temporal coherency based criteria for predicting video frames using deep multi-stage generative adversarial networks [C]∥Proceedings of the Advances in Neural Information Processing Systems. New York, USA: NIPS, 2017: 4271-4280.

MATHIEU M, COUPRIE C, LECUN Y. Deep multi-scale video prediction beyond mean square error [EB/OL].(2015-11-17)[2018-02-28]. https:∥arxiv.org/abs/1511.05440.

LIANG X, LEE L, DAI W, et al. Dual motion GAN for future-flow embedded video prediction [C]∥Proceedings of the IEEE International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2017: 1762-1770.

XIONG W, LUO W, MA L, et al. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks [EB/OL].(2017-09-22)[2018-02-28]. https:∥arxiv.org/abs/1709.07592.

LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2017: 105-114.

CHEN Q, KOLTUN V. Photographic image synthesis with cascaded refinement networks [C]∥Proceedings of the IEEE International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2017: 1520-1529.

YANG C, LU X, LIN Z, et al. High-resolution image inpainting using multi-scale neural patch synthesis [C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2017: 4076-4084.

GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks [C]∥Proceedings of the Advances in Neural Information Processing Systems. New York, USA: NIPS, 2014: 2672-2680.

MIRZA M, OSINDERO S. Conditional generative adversarial nets [EB/OL].(2014-11-06)[2018-02-28]. https:∥arxiv.org/abs/1411.1784.

ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks [C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2017: 5967-5976.

RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation [C]∥Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Germany: Springer, 2015: 234-241.

HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition [C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognitio. Piscataway, NJ, USA: IEEE, 2016: 770-778.

WANG T C, LIU M Y, ZHU J Y, et al. High-resolution image synthesis and semantic manipulation with conditional GANs [EB/OL].(2017-11-30)[2018-02-25]. https:∥arxiv.org/abs/1711.11585.

JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution [C]∥Proceedings of the European Conference on Computer Vision. Berlin, Germany: Springer, 2016: 694-711.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL].(2014-09-04)[2018-02-24]. https:∥arxiv.org/abs/1409.1556.

GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the KITTI dataset [J]. Journal of Robotics Research, 2013, 32(11): 1231-1237.

ULYANOV D, VEDALDI A, LEMPITSKY V, et al. Instance normalization: the missing ingredient for fast stylization [EB/OL].(2016-07-27)[2018-03-10]. https:∥arxiv.org/abs/1607.08022.

ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks [C]∥Proceedings of the International Conference on Machine Learning. New York, USA: ACM, 2017: 214-223.

KINGMA D P, BA J. Adam: a method for stochastic optimization [EB/OL].(2014-12-22)[2018-02-20]. https:∥arxiv.org/abs/1412.6980.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

深度学习驱动的碳纤维管壁缺陷镜面红外检测无损定量评估方法

面向模型预测控制的风电机组深度Koopman全局线性建模方法

主蒸汽管道破裂事故下蒸汽发生器的动态液位预测

融合U-net网络的纯卷积视频预测模型

一种双阻尼小波赋能的可解释卷积神经网络在轴承故障诊断中的应用