Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets

Wei Cheng Chou; Cheng Wei Huang; Juinn Dar Huang

doi:10.1109/VLSI-DAT54769.2022.9768087

Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets

Wei Cheng Chou, Cheng Wei Huang, Juinn Dar Huang

Institute of Electronics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Pattern-based weight pruning on CNNs has been proven an effective model reduction technique. In this paper, we first present how to select hardware-friendly pruning pattern sets that are universal to various models. We then propose a progressive pruning framework, which produces more globally optimized outcomes. Moreover, to the best of our knowledge, this is the first paper dealing with the pruning issue of the first and also the most sensitive layer of a CNN model through a two-staged pruning strategy. Experiment results show that the proposed framework achieves 2.25x/2x computation/model reduction while minimizing the accuracy loss.

Original language	English
Title of host publication	2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665409216
DOIs	https://doi.org/10.1109/VLSI-DAT54769.2022.9768087
State	Published - 2022
Event	2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Hsinchu, Taiwan Duration: 18 Apr 2022 → 21 Apr 2022

Publication series

Name	2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings

Conference

Conference	2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022
Country/Territory	Taiwan
City	Hsinchu
Period	18/04/22 → 21/04/22

Keywords

convolutional neural network (CNN)
first layer pruning
model compression
pattern pruning
universal pattern sets
weight pruning

Access to Document

10.1109/VLSI-DAT54769.2022.9768087

Cite this

Chou, W. C., Huang, C. W., & Huang, J. D. (2022). Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets. In 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings (2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VLSI-DAT54769.2022.9768087

Chou, Wei Cheng ; Huang, Cheng Wei ; Huang, Juinn Dar. / Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets. 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. (2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings).

@inproceedings{e66e9d7d4546486786c3549b78649ffd,

title = "Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets",

abstract = "Pattern-based weight pruning on CNNs has been proven an effective model reduction technique. In this paper, we first present how to select hardware-friendly pruning pattern sets that are universal to various models. We then propose a progressive pruning framework, which produces more globally optimized outcomes. Moreover, to the best of our knowledge, this is the first paper dealing with the pruning issue of the first and also the most sensitive layer of a CNN model through a two-staged pruning strategy. Experiment results show that the proposed framework achieves 2.25x/2x computation/model reduction while minimizing the accuracy loss.",

keywords = "convolutional neural network (CNN), first layer pruning, model compression, pattern pruning, universal pattern sets, weight pruning",

author = "Chou, {Wei Cheng} and Huang, {Cheng Wei} and Huang, {Juinn Dar}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 ; Conference date: 18-04-2022 Through 21-04-2022",

year = "2022",

doi = "10.1109/VLSI-DAT54769.2022.9768087",

language = "English",

series = "2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings",

address = "United States",

}

Chou, WC, Huang, CW & Huang, JD 2022, Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets. in 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings. 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022, Hsinchu, Taiwan, 18/04/22. https://doi.org/10.1109/VLSI-DAT54769.2022.9768087

Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets. / Chou, Wei Cheng; Huang, Cheng Wei; Huang, Juinn Dar.
2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. (2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets

AU - Chou, Wei Cheng

AU - Huang, Cheng Wei

AU - Huang, Juinn Dar

PY - 2022

Y1 - 2022

N2 - Pattern-based weight pruning on CNNs has been proven an effective model reduction technique. In this paper, we first present how to select hardware-friendly pruning pattern sets that are universal to various models. We then propose a progressive pruning framework, which produces more globally optimized outcomes. Moreover, to the best of our knowledge, this is the first paper dealing with the pruning issue of the first and also the most sensitive layer of a CNN model through a two-staged pruning strategy. Experiment results show that the proposed framework achieves 2.25x/2x computation/model reduction while minimizing the accuracy loss.

AB - Pattern-based weight pruning on CNNs has been proven an effective model reduction technique. In this paper, we first present how to select hardware-friendly pruning pattern sets that are universal to various models. We then propose a progressive pruning framework, which produces more globally optimized outcomes. Moreover, to the best of our knowledge, this is the first paper dealing with the pruning issue of the first and also the most sensitive layer of a CNN model through a two-staged pruning strategy. Experiment results show that the proposed framework achieves 2.25x/2x computation/model reduction while minimizing the accuracy loss.

KW - convolutional neural network (CNN)

KW - first layer pruning

KW - model compression

KW - pattern pruning

KW - universal pattern sets

KW - weight pruning

UR - http://www.scopus.com/inward/record.url?scp=85130481720&partnerID=8YFLogxK

U2 - 10.1109/VLSI-DAT54769.2022.9768087

DO - 10.1109/VLSI-DAT54769.2022.9768087

M3 - Conference contribution

AN - SCOPUS:85130481720

T3 - 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings

BT - 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022

Y2 - 18 April 2022 through 21 April 2022

ER -

Chou WC, Huang CW, Huang JD. Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets. In 2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2022. (2022 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2022 - Proceedings). doi: 10.1109/VLSI-DAT54769.2022.9768087

Hardware-Friendly Progressive Pruning Framework for CNN Model Compression using Universal Pattern Sets

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this