Compressing DNN Parameters for Model Loading Time Reduction

Yang Ming Yeh; Jennifer Shueh Inn Hu; Yen Yu Lin; Yi Chang Lu

doi:10.1109/ICCE-Asia46551.2019.8942192

Compressing DNN Parameters for Model Loading Time Reduction

Yang Ming Yeh, Jennifer Shueh Inn Hu, Yen Yu Lin, Yi Chang Lu

Institute of Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Deep neural network (DNN) has been applied to a variety of computer vision tasks these days. However, DNN often suffers from its enormous execution time even with the aid of GPU. In this paper, we argue that the bandwidth bottleneck between GPU and GDRAM has to be addressed. To reduce loading time, we propose a DNN acceleration approach which compresses DNN parameters before loading model information to GPU and performs decompressing on GPU. Using JPEG compression as an example, the loss of the test accuracy can be kept within 4%, while an 8 × parameter-size reduction is achieved for VGG16.

Original language	English
Title of host publication	2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	78-79
Number of pages	2
ISBN (Electronic)	9781728133362
DOIs	https://doi.org/10.1109/ICCE-Asia46551.2019.8942192
State	Published - Jun 2019
Event	4th IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019 - Bangkok, Thailand Duration: 12 Jun 2019 → 14 Jun 2019

Publication series

Name	2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019

Conference

Conference	4th IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019
Country/Territory	Thailand
City	Bangkok
Period	12/06/19 → 14/06/19

Keywords

DNN
architecture
compression

Access to Document

10.1109/ICCE-Asia46551.2019.8942192

Cite this

Yeh, Y. M., Hu, J. S. I., Lin, Y. Y., & Lu, Y. C. (2019). Compressing DNN Parameters for Model Loading Time Reduction. In 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019 (pp. 78-79). Article 8942192 (2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCE-Asia46551.2019.8942192

@inproceedings{86577731d8b7481882c1093cfe8f8eb1,

title = "Compressing DNN Parameters for Model Loading Time Reduction",

abstract = "Deep neural network (DNN) has been applied to a variety of computer vision tasks these days. However, DNN often suffers from its enormous execution time even with the aid of GPU. In this paper, we argue that the bandwidth bottleneck between GPU and GDRAM has to be addressed. To reduce loading time, we propose a DNN acceleration approach which compresses DNN parameters before loading model information to GPU and performs decompressing on GPU. Using JPEG compression as an example, the loss of the test accuracy can be kept within 4%, while an 8 × parameter-size reduction is achieved for VGG16.",

keywords = "DNN, architecture, compression",

author = "Yeh, {Yang Ming} and Hu, {Jennifer Shueh Inn} and Lin, {Yen Yu} and Lu, {Yi Chang}",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE.; 4th IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019 ; Conference date: 12-06-2019 Through 14-06-2019",

year = "2019",

month = jun,

doi = "10.1109/ICCE-Asia46551.2019.8942192",

language = "English",

series = "2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "78--79",

booktitle = "2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019",

address = "美國",

}

Yeh, YM, Hu, JSI, Lin, YY & Lu, YC 2019, Compressing DNN Parameters for Model Loading Time Reduction. in 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019., 8942192, 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019, Institute of Electrical and Electronics Engineers Inc., pp. 78-79, 4th IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019, Bangkok, Thailand, 12/06/19. https://doi.org/10.1109/ICCE-Asia46551.2019.8942192

Compressing DNN Parameters for Model Loading Time Reduction. / Yeh, Yang Ming; Hu, Jennifer Shueh Inn; Lin, Yen Yu et al.
2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 78-79 8942192 (2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Compressing DNN Parameters for Model Loading Time Reduction

AU - Yeh, Yang Ming

AU - Hu, Jennifer Shueh Inn

AU - Lin, Yen Yu

AU - Lu, Yi Chang

PY - 2019/6

Y1 - 2019/6

N2 - Deep neural network (DNN) has been applied to a variety of computer vision tasks these days. However, DNN often suffers from its enormous execution time even with the aid of GPU. In this paper, we argue that the bandwidth bottleneck between GPU and GDRAM has to be addressed. To reduce loading time, we propose a DNN acceleration approach which compresses DNN parameters before loading model information to GPU and performs decompressing on GPU. Using JPEG compression as an example, the loss of the test accuracy can be kept within 4%, while an 8 × parameter-size reduction is achieved for VGG16.

AB - Deep neural network (DNN) has been applied to a variety of computer vision tasks these days. However, DNN often suffers from its enormous execution time even with the aid of GPU. In this paper, we argue that the bandwidth bottleneck between GPU and GDRAM has to be addressed. To reduce loading time, we propose a DNN acceleration approach which compresses DNN parameters before loading model information to GPU and performs decompressing on GPU. Using JPEG compression as an example, the loss of the test accuracy can be kept within 4%, while an 8 × parameter-size reduction is achieved for VGG16.

KW - DNN

KW - architecture

KW - compression

UR - http://www.scopus.com/inward/record.url?scp=85077999948&partnerID=8YFLogxK

U2 - 10.1109/ICCE-Asia46551.2019.8942192

DO - 10.1109/ICCE-Asia46551.2019.8942192

M3 - Conference contribution

AN - SCOPUS:85077999948

T3 - 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019

SP - 78

EP - 79

BT - 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 4th IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019

Y2 - 12 June 2019 through 14 June 2019

ER -

Yeh YM, Hu JSI, Lin YY, Lu YC. Compressing DNN Parameters for Model Loading Time Reduction. In 2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 78-79. 8942192. (2019 IEEE International Conference on Consumer Electronics - Asia, ICCE-Asia 2019). doi: 10.1109/ICCE-Asia46551.2019.8942192

Compressing DNN Parameters for Model Loading Time Reduction

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this