面向多功能张量加速器的细粒度结构化稀疏设计

赵桦筝; 庞善民; 赵英海; 华高晖; 李晨阳; 段战胜; 梅魁志

doi:10.7652/xjtuxb202411017

您当前的位置：

首页 >

文章列表页 >

面向多功能张量加速器的细粒度结构化稀疏设计

—————————————————————— | 更新时间：2025-07-25

- 面向多功能张量加速器的细粒度结构化稀疏设计
- Fine-Grained Structured Sparse Design for Versatile Tensor Accelerator
- 西安交通大学学报 2024年58卷第11期页码：176-184
- 作者机构：
  
  1. 西安交通大学软件学院,西安,710049
  2. 北京华航无线电测量研究所,北京,100013
  3. 西安交通大学自动化科学与工程学院,西安,710049
  4. 西安交通大学人工智能学院,西安,710049
- 作者简介：
  
  : 2024-04-13。作者简介: 赵桦筝(1999—),女,硕士生
  梅魁志(通信作者),男,教授,博士生导师。基金项目: 新疆维吾尔自治区重点研发计划资助项目(2022B01008-1)
- 基金信息：
  
  : 2024-04-13。作者简介: 赵桦筝(1999—),女,硕士生;梅魁志(通信作者),男,教授,博士生导师。基金项目: 新疆维吾尔自治区重点研发计划资助项目(2022B01008-1);国家自然科学基金资助项目(62076193)。
- DOI：10.7652/xjtuxb202411017
  中图分类号： TP31
- 网络首发：2024-11-10，
  
  纸质出版：2024
- 稿件说明：
移动端阅览
赵桦筝, 庞善民, 赵英海, 等. 面向多功能张量加速器的细粒度结构化稀疏设计[J]. 西安交通大学学报, 2024,58(11):176-184.

ZHAO Huazheng, PANG Shanmin, ZHAO Yinghai, et al. Fine-Grained Structured Sparse Design for Versatile Tensor Accelerator[J]. 2024, 58(11): 176-184.
赵桦筝, 庞善民, 赵英海, 等. 面向多功能张量加速器的细粒度结构化稀疏设计[J]. 西安交通大学学报, 2024,58(11):176-184. DOI： 10.7652/xjtuxb202411017.

ZHAO Huazheng, PANG Shanmin, ZHAO Yinghai, et al. Fine-Grained Structured Sparse Design for Versatile Tensor Accelerator[J]. 2024, 58(11): 176-184. DOI： 10.7652/xjtuxb202411017.

摘要

为解决模型压缩算法与多功能张量加速器(VTA)的适配性问题

通过改进经典的YOLObile分块剪枝方法

完成面向该加速器的自适应细粒度结构化稀疏设计及性能评估。针对VTA的多重循环维度展开特性

对模型的权重张量进行32×32大小的分块; 结合时间维度的自蒸馏与空间维度的教师蒸馏

进行多维度特征对齐; 通过一阶段式迭代训练方式

改进原有的ADMM算法计算流程

在提升模型部署精度的同时减少训练成本; 提出自适应层剪枝率模块

进行总剪枝率的自适应分配

实现端到端的自动化剪枝。实验结果表明:改进方法有效减少了约2.4%的浮点计算量

并在图像分类、目标检测等多项任务中提升了压缩模型的精度

最大增长百分比为2.6%。该方法为深度学习模型在VTA上的稀疏化部署提供了一种高效、轻量级的软件解决方案。

Abstract

In order to address the compatibility issue between model compression algorithms and the versatile tensor accelerator(VTA)

an adaptive fine-grained structured sparse design tailored for this accelerator is proposed by enhancing the classical YOLObile block-wise pruning method and evaluates its performance. In light of the multi-dimensional loop unfolding characteristics of VTA

the model's weight tensors are divided into 32×32 blocks. This approach integrates temporal distillation and spatial distillation to align multidimensional features. Through a single-stage iterative training method

the calculation process of the original ADMM algorithm is refined to improve model deployment accuracy while reducing training costs. An adaptive layer pruning rate module is introduced to dynamically allocate the total pruning rate

facilitating end-to-end automated pruning. The experimental results demonstrate that this improved method effectively reduces floating-point computations by approximately 2.4% and enhances the accuracy of compressed models across various tasks such as image classification and object detection

with a maximum growth percentage of 2.6%. This method offers an efficient and lightweight software solution for the sparse deployment of deep learning models on VTAs.

关键词

Keywords

references

高晗, 田育龙, 许封元, 等. 深度学习模型压缩与加速综述 [J]. 软件学报, 2021, 32(1): 68-92.

GAO Han, TIAN Yulong, XU Fengyuan, et al. Survey of deep learning model compression and acceleration [J]. Journal of Software, 2021, 32(1): 68-92.

BERTHELIER A, CHATEAU T, DUFFNER S, et al. Deep model compression and architecture optimization for embedded systems: a survey [J]. Journal of Signal Processing Systems, 2021, 93(8): 863-878.

符惠桐, 王鹏, 李晓艳, 等. 面向移动目标识别的轻量化网络模型 [J]. 西安交通大学学报, 2021, 55(7): 124-131.

FU Huitong, WANG Peng, LI Xiaoyan, et al. Lightweight network model for moving object recognition [J]. Journal of Xi'an Jiaotong University, 2021, 55(7): 124-131.

BA L J, CARUANA R. Do deep nets really need to be deep? [C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2014: 2654-2662.

KIM T, KWON Y, LEE J, et al. CPrune: compiler-informed model pruning for efficient target-aware DNN execution [C]//Computer Vision -ECCV 2022. Cham, Switzerland: Springer Nature, 2022: 651-667.

LI Zhengang, YUAN Geng, NIU Wei, et al. NPAS: A compiler-aware framework of unified network pruning and architecture search for beyond real-time mobile acceleration [C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2021: 14250-14261.

GUAN Hui, LIU Shaoshan, MA Xiaolong, et al. CoCoPIE: enabling real-time AI on off-the-shelf mobile devices via compression-compilation co-design [J]. Communications of the ACM, 2021, 64(6): 62-68.

CHEN Tianqi, MOREAU T, JIANG Ziheng, et al. TVM: an automated end-to-end optimizing compiler for deep learning [C]//Proceedings of the 13th USENIX conference on Operating Systems Design and Implementation. USA: USENIX Association, 2018: 579-594.

MOREAU T, CHEN Tianqi, VEGA L, et al. A hardware-software blueprint for flexible deep learning specialization [J]. IEEE Micro, 2019, 39(5): 8-16.

Mudigere D, HAO Yuchen, HUANG Jianyu, et al. Software-hardware co-design for fast and scalable training of deep learning recommendation models [C]//Proceedings of the 49th Annual International Symposium on Computer Architecture. New York, USA: ACM, 2022: 993-1011.

VADERA S, AMEEN S. Methods for pruning deep neural networks [J]. IEEE Access, 2022, 10: 63280-63300.

HUANG Sitao, PEARSON C, NAGI R, et al. Accelerating sparse deep neural networks on FPGAs [C]//2019 IEEE High Performance Extreme Computing Conference(HPEC). Piscataway, NJ, USA: IEEE, 2019: 1-7.

ZHOU Aojun, MA Yukun, ZHU Junnan, et al. Learning N:M fine-grained structured sparse neural networks from scratch [EB/OL].(2021-04-18)[2024-04-01]. https://arxiv.org/abs/2102.04010.

CHANG S E, LI Yanyu, SUN Mengshu, et al. Mix and match: a novel FPGA-centric deep neural network quantization framework [C]//2021 IEEE International Symposium on High-Performance Computer Architecture(HPCA). Piscataway, NJ, USA: IEEE, 2021: 208-220.

林景栋, 吴欣怡, 柴毅, 等. 卷积神经网络结构优化综述 [J]. 自动化学报, 2020, 46(1): 24-37.

LIN Jingdong, WU Xinyi, CHAI Yi, et al. Structure optimization of convolutional neural networks: a survey [J]. Acta Automatica Sinica, 2020, 46(1): 24-37.

CAI Yuxuan, LI Hongjia, YUAN Geng, et al. YOLObile: real-time object detection on mobile devices via compression-compilation co-design [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2021: 955-963.

FARHADI A, REDMON J. Yolov3: an incremental improvement [C]//Computer Vision and Pattern Recognition. Berlin/Heidelberg, Germany: Springer, 2018: 1-6.

HINTON G, VINYALS O, DEAN J. Distilling the knowledge in a neural network [EB/OL].(2015-03-09)[2024-04-01]. https://arxiv.org/abs/1503.02531.

NIU Wei, LI Zhengang, MA Xiaolong, et al. GRIM: a general, real-time deep learning inference framework for mobile devices based on fine-grained structured weight sparsity [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10): 6224-6239.

BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection [EB/OL].(2020-04-23)[2024-04-01]. https://arxiv.org/abs/2004.10934.

BAE W, YOO J, YE J C. Beyond deep residual learning for image restoration: persistent homology-guided manifold simplification [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Piscataway, NJ, USA: IEEE, 2017: 1141-1149.

SANDLER M, HOWARD A, ZHU Menglong, et al. MobileNetV2: inverted residuals and linear bottlenecks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 4510-4520.

HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 7132-7141.

HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2017: 2261-2269.

FANG Gongfan, MA Xinyin, SONG Mingli, et al. DepGraph: towards any structural pruning [C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR). Piscataway, NJ, USA: IEEE, 2023: 16091-16101.

SUN Yongshuai, GUO Mengyu, LIANG Dacheng, et al. Exploiting dynamic bit sparsity in activation for deep neuralnetwork acceleration [C]//2021 IEEE 14th International Conference on ASIC(ASICON). Piscataway, NJ, USA: IEEE, 2021: 1-4.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

深度学习驱动的碳纤维管壁缺陷镜面红外检测无损定量评估方法

面向模型预测控制的风电机组深度Koopman全局线性建模方法

主蒸汽管道破裂事故下蒸汽发生器的动态液位预测

融合U-net网络的纯卷积视频预测模型

一种双阻尼小波赋能的可解释卷积神经网络在轴承故障诊断中的应用