An Encoder-Decoder-Based Super Resolution Network for License Plate Images

XU Shengjun; DENG Bowen; SHI Ya; MENG Yuebo; LIU Guanghui; HAN Jiuqiang

doi:10.7652/xjtuxb202210010

您当前的位置：

首页 >

文章列表页 >

An Encoder-Decoder-Based Super Resolution Network for License Plate Images

更新时间：2025-07-09

- An Encoder-Decoder-Based Super Resolution Network for License Plate Images
- Vol. 56, Issue 10, Pages: 101-110(2022)
- 作者机构：
  
  1. 西安建筑科技大学信息与控制工程学院,西安,710055
  2. 西安市建筑制造智动化技术重点实验室,西安,710055
  3. 西安交通大学电子与信息学部,西安,710049
- 作者简介：
- 基金信息：
- DOI：10.7652/xjtuxb202210010
  CLC： TP391
- Online First：10 October 2022，
  
  Published：2022
- 稿件说明：
移动端阅览
XU Shengjun, DENG Bowen, SHI Ya, et al. An Encoder-Decoder-Based Super Resolution Network for License Plate Images[J]. 2022, 56(10): 101-110.
DOI：

XU Shengjun, DENG Bowen, SHI Ya, et al. An Encoder-Decoder-Based Super Resolution Network for License Plate Images[J]. 2022, 56(10): 101-110. DOI： 10.7652/xjtuxb202210010.

摘要

针对复杂实际场景中模糊、污损、扭曲、倾斜等车牌图像关键信息缺失以及新能源车牌背景与字符对比度低难以识别的问题

提出了一种编解码结构的车牌图像超分辨率网络。首先

构建一种基于编解码结构的车牌重构生成器网络

利用编码器对车牌图像的纹理、字符等特征进行提取

解码器对车牌特征进行重构; 然后

设计一种基于语义监督的判别器网络

在网络损失中引入了对抗损失与CTC(connectionist temporal classification)损失

增强生成器网络对车牌图像语义特征的表征能力; 最后

基于VGG16网络提取车牌顶角点特征

利用坐标变换方法对车牌图像进行矫正

进一步提高重构清晰度与识别准确率。采用所提网络在自建XAUAT-Parking数据集和公开CCPD数据集上进行超分辨率重构与识别实验

结果表明:所提网络在CCPD数据集上的平均峰值信噪比可达25.5 dB

结构相似性(SSIM)可达0.989; 在XAUAT-Parking数据集上峰值信噪比可达26.6 dB

结构相似性可达0.997。研究结果表明

该网络有较好的车牌图像超分辨率重建效果

而且对车牌关键信息缺失问题具有较强的鲁棒性。

Abstract

An encoder-decoder-based super resolution network for license plate images was proposed for the lack of key information on license plate images caused by blur

stain

damage

distortion

and tilt in complex actual scenes and for the recognition difficulty due to low contrast between the license plate background and characters of new-energy vehicles. Firstly

a license plate reconstruction generator network based on the encoder-decoder structure is constructed. The texture and characters of the license plate image are extracted by an encoder

and the license plate features are reconstructed by a decoder. Then

a discriminator network based on semantic supervision is designed

and the adversarial loss and CTC loss are introduced into the network loss to enhance the ability of the generator network to represent the semantic features of license plate images. Finally

the features of the vertex points of the license plate are extracted based on VGG16 network

and a coordinate transformation method is utilized to correct the license plate image and further improve the reconstruction quality and recognition accuracy. Super-resolution reconstruction and recognition tests were performed on the self-built XAUAT-Parking dataset and the public CCPD dataset with the proposed network. The test results showed that the proposed network has an average peak signal to noise ratio(PSNR)of 25.5 dB and a structural similarity(SSIM)of 0.989 on the CCPD dataset. The PSNR and SSIM on the XAUAT-Parking dataset can reach 26.6 dB and 0.997 respectively. According to the research results

the proposed network has a good super-resolution reconstruction effect of license plate images and a strong robustness to the missing of key license plate information.

关键词

Keywords

references

刘济林, 宋加涛, 丁莉雅, 等. 高性能的车牌识别系统 [J]. 自动化学报, 2003, 29(3): 457-465.

LIU Jilin, SONG Jiatao, DING Liya, et al. Vehicle license plate recognition system with high performance [J]. Acta Automatica Sinica, 2003, 29(3): 457-465.

PAN Meisen, XIONG Qi, YAN Junbiao. A new method for correcting vehicle license plate tilt [J]. International Journal of Automation and Computing, 2009, 6(2): 210-216.

LIN C H, LI Ying. A license plate recognition system for severe tilt angles using mask R-CNN [C]//2019 International Conference on Advanced Mechatronic Systems(ICAMechS). Piscataway, NJ, USA: IEEE, 2019: 229-234.

左龙, 张鹏, 荆树旭, 等. 用于图像超分辨率重建的双通道残差网络 [J]. 西安交通大学学报, 2022, 56(1): 158-164.

ZUO Long, ZHANG Peng, JING Shuxu, et al. Dual-channel residual network for image super-resolution reconstruction [J]. Journal of Xi'an Jiaotong University, 2022, 56(1): 158-164.

吴俊峰, 牟轩沁. L1范数字典约束的感兴趣区域CT图像重建算法 [J]. 西安交通大学学报, 2019, 53(2): 163-169.

WU Junfeng, MU Xuanqin. A reconstruction algorithm of CT images of interested regions based on an L1 norm dictionary sparse constraint [J]. Journal of Xi'an Jiaotong University, 2019, 53(2): 163-169.

DUAN T D, DUC D A, DU T L H. Combining Hough transform and contour algorithm for detecting vehicles' license-plates [C]//Proceedings of 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing. Piscataway, NJ, USA: IEEE, 2004: 747-750.

HOSSEN M K, DEB K. Vehicle license plate detection and tilt correction based on HSI color model and SUSAN corner detector [J]. Smart Computing Review, 2014, 4(5): 371-388.

SILVA S M, JUNG C R. License plate detection and recognition in unconstrained scenarios [C]//Computer Vision-ECCV 2018. Cham, Germany: Springer International Publishing, 2018: 593-609.

IRANI M, PELEG S. Improving resolution by image registration [J]. CVGIP: Graphical Models and Image Processing, 1991, 53(3): 231-239.

RASTI P, DEMIREL H, ANBARJAFARI G. Improved iterative back projection for video super-resolution [C]//2014 22nd Signal Processing and Communications Applications Conference(SIU). Piscataway, NJ, USA: IEEE, 2014: 552-555.

HE Dongliang, ZHOU Zhichao, GAN Chuang, et al. StNet: local and global spatial-temporal modeling for action recognition [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 8401-8408.

LIN Mianfen, LIU Liangxin, WANG Fei, et al. License plate image reconstruction based on generative adversarial networks [J]. Remote Sensing, 2021, 13(15): 3018.

LIU Wu, LIU Xinchen, MA Huadomg, et al. Beyond human-level license plate super-resolution with progressive vehicle search and domain priori GAN [C]//Proceedings of the 25th ACM International Conference on Multimedia. New York, NY, USA: ACM, 2017: 1618-1626.

ZHANG Minghui, LIU Wu, MA Huadong. Joint license plate super-resolution and recognition in one multi-task Gan framework [C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Piscataway, NJ, USA: IEEE, 2018: 1443-1447.

LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2017: 105-114.

GRAVES A, FERNÁNDEZ S, GOMEZ F, et al. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks [C]//Proceedings of the 23rd International Conference on Machine Learning. New York, NY, USA: ACM, 2006: 369-376.

ZOU Yongjie, ZHANG Yongjun, YAN Jun, et al. A robust license plate recognition model based on Bi-LSTM [J]. IEEE Access, 2020, 8: 211630-211641.

XU Zhenbo, YANG Wei, MENG A, et al. Towards end-to-end license plate detection and recognition: a large dataset and baseline [C]//Computer Vision-ECCV 2018. Cham, Germany: Springer International Publishing, 2018: 261-277.

PONOMARENKO N, IEREMEIEV O, LUKIN V, et al. Modified image visual quality metrics for contrast change and mean shift accounting [C]//2011 11th International Conference on the Experience of Designing and Application of CAD Systems in Microelectronics(CADSM). Piscataway, NJ, USA: IEEE, 2011: 305-311.[20] WANG Zhou, LI Qiang. Information content weighting for perceptual image quality assessment [J]. IEEE Transactions on Image Processing, 2011, 20(5): 1185-1198.

PENG Xi, FERIS R S, WANG Xiaoyu, et al. RED-net: a recurrent encoder-decoder network for video-based face alignment [J]. International Journal of Computer Vision, 2018, 126(10): 1103-1119.

KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2016: 1646-1654.

REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector [C]//Computer Vision-ECCV 2016. Cham, Germany: Springer International Publishing, 2016: 21-37.

REDMON J, FARHADI A. YOLO9000: better, faster, stronger [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2017: 6517-6525.

ZHERZDEV S, GRUZDEV A. LPRNet: license plate recognition via deep neural networks [EB/OL].[2021-12-12]. https://arxiv.org/abs/1806.10447v1.

ZHANG Kaipeng, ZHANG Zhanpeng, LI Zhifeng, et al. Joint face detection and alignment using multitask cascaded convolutional networks [J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Speech Adversarial Sample Generation Method Based on Class-Conditional Generative Adversarial Networks

Black-Box Transferable Adversarial Attack Method Based on Generative Adversarial Networks for Lung Disease Diagnosis Models

Multi-Scale Regional Attention InfoGAN License Plate Recognition Network

Energy Efficiency Optimization of the Cryogenic Air Separation Process for Electronic-Grade Ultra Pure Nitrogen

Study on the Thermodynamic Properties in the Anomalous Region of Supercritical Water

Related Author

YU Zhenhua

HU Xufei

YE Ou

WANG Xiaoyin

WANG Dan

SUN Jiaze

YANG Yikang

DU Miao

Related Institution

西安科技大学计算机科学与技术学院

School of Computer Science, Xi'an University of Posts and Telecommunications

Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi'an University of Posts and Telecommunications

School of Automation Science and Engineering, Xi'an Jiaotong University

⁰