A Hadoop-based Principle Component Analysis on embedded heterogeneous platform

Sheng Yen Chen; Chia I. Wei; Yu Chen Chiu; Bo-Cheng Lai

doi:10.1109/VLSI-DAT.2017.7939667

A Hadoop-based Principle Component Analysis on embedded heterogeneous platform

Sheng Yen Chen, Chia I. Wei, Yu Chen Chiu, Bo-Cheng Lai

Institute of Electronics

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Hadoop is a widely adopted distributed processing framework which assumes each computing node a CPU-based system with local memory. This design scheme cannot effectively take full advantage of an embedded heterogeneous many-core platform due the mismatch of data collection and management paradigms between the Hadoop environment and embedded heterogeneous systems. This paper proposes a Hadoop-based design of Principle Component Analysis (PCA) to efficiently leverage the distributed embedded heterogeneous many-core systems. By taking the same data layout of conventional Hadoop applications, the proposed design introduces efficient manners to collect and manage the fine-grained data chunks. The experiments on a Tegra K1 has achieved 5.9× performance enhancement.

Original language	English
Title of host publication	2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781509039692
DOIs	https://doi.org/10.1109/VLSI-DAT.2017.7939667
State	Published - 5 Jun 2017
Event	2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017 - Hsinchu, Taiwan Duration: 24 Apr 2017 → 27 Apr 2017

Publication series

Name	2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017

Conference

Conference	2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017
Country/Territory	Taiwan
City	Hsinchu
Period	24/04/17 → 27/04/17

Access to Document

10.1109/VLSI-DAT.2017.7939667

Cite this

Chen, S. Y., Wei, C. I., Chiu, Y. C., & Lai, B-C. (2017). A Hadoop-based Principle Component Analysis on embedded heterogeneous platform. In 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017 [7939667] (2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VLSI-DAT.2017.7939667

@inproceedings{92f84828efbb4359ac4427f3bd5f0b45,

title = "A Hadoop-based Principle Component Analysis on embedded heterogeneous platform",

abstract = "Hadoop is a widely adopted distributed processing framework which assumes each computing node a CPU-based system with local memory. This design scheme cannot effectively take full advantage of an embedded heterogeneous many-core platform due the mismatch of data collection and management paradigms between the Hadoop environment and embedded heterogeneous systems. This paper proposes a Hadoop-based design of Principle Component Analysis (PCA) to efficiently leverage the distributed embedded heterogeneous many-core systems. By taking the same data layout of conventional Hadoop applications, the proposed design introduces efficient manners to collect and manage the fine-grained data chunks. The experiments on a Tegra K1 has achieved 5.9× performance enhancement.",

author = "Chen, {Sheng Yen} and Wei, {Chia I.} and Chiu, {Yu Chen} and Bo-Cheng Lai",

year = "2017",

month = jun,

day = "5",

doi = "10.1109/VLSI-DAT.2017.7939667",

language = "English",

series = "2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017",

address = "United States",

note = "null ; Conference date: 24-04-2017 Through 27-04-2017",

}

Chen, SY, Wei, CI, Chiu, YC & Lai, B-C 2017, A Hadoop-based Principle Component Analysis on embedded heterogeneous platform. in 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017., 7939667, 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017, Institute of Electrical and Electronics Engineers Inc., 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017, Hsinchu, Taiwan, 24/04/17. https://doi.org/10.1109/VLSI-DAT.2017.7939667

A Hadoop-based Principle Component Analysis on embedded heterogeneous platform. / Chen, Sheng Yen; Wei, Chia I.; Chiu, Yu Chen et al.

2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017. Institute of Electrical and Electronics Engineers Inc., 2017. 7939667 (2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Hadoop-based Principle Component Analysis on embedded heterogeneous platform

AU - Chen, Sheng Yen

AU - Wei, Chia I.

AU - Chiu, Yu Chen

AU - Lai, Bo-Cheng

PY - 2017/6/5

Y1 - 2017/6/5

N2 - Hadoop is a widely adopted distributed processing framework which assumes each computing node a CPU-based system with local memory. This design scheme cannot effectively take full advantage of an embedded heterogeneous many-core platform due the mismatch of data collection and management paradigms between the Hadoop environment and embedded heterogeneous systems. This paper proposes a Hadoop-based design of Principle Component Analysis (PCA) to efficiently leverage the distributed embedded heterogeneous many-core systems. By taking the same data layout of conventional Hadoop applications, the proposed design introduces efficient manners to collect and manage the fine-grained data chunks. The experiments on a Tegra K1 has achieved 5.9× performance enhancement.

AB - Hadoop is a widely adopted distributed processing framework which assumes each computing node a CPU-based system with local memory. This design scheme cannot effectively take full advantage of an embedded heterogeneous many-core platform due the mismatch of data collection and management paradigms between the Hadoop environment and embedded heterogeneous systems. This paper proposes a Hadoop-based design of Principle Component Analysis (PCA) to efficiently leverage the distributed embedded heterogeneous many-core systems. By taking the same data layout of conventional Hadoop applications, the proposed design introduces efficient manners to collect and manage the fine-grained data chunks. The experiments on a Tegra K1 has achieved 5.9× performance enhancement.

UR - http://www.scopus.com/inward/record.url?scp=85021394050&partnerID=8YFLogxK

U2 - 10.1109/VLSI-DAT.2017.7939667

DO - 10.1109/VLSI-DAT.2017.7939667

M3 - Conference contribution

AN - SCOPUS:85021394050

T3 - 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017

BT - 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 24 April 2017 through 27 April 2017

ER -

Chen SY, Wei CI, Chiu YC, Lai B-C. A Hadoop-based Principle Component Analysis on embedded heterogeneous platform. In 2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017. Institute of Electrical and Electronics Engineers Inc. 2017. 7939667. (2017 International Symposium on VLSI Design, Automation and Test, VLSI-DAT 2017). doi: 10.1109/VLSI-DAT.2017.7939667

A Hadoop-based Principle Component Analysis on embedded heterogeneous platform

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this