VidLife: A Dataset for Life Event Extraction from Videos

Tai Te Chu; An Zi Yen; Wei Hong Ang; Hen Hsen Huang; Hsin Hsi Chen

doi:10.1145/3459637.3482022

VidLife: A Dataset for Life Event Extraction from Videos

Tai Te Chu, An Zi Yen, Wei Hong Ang, Hen Hsen Huang, Hsin Hsi Chen

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Filming video blogs, which is shortened to vlog, becomes a popular way for people to record their life experiences in recent years. In this work, we present a novel task that is aimed at extracting life events from videos and constructing personal knowledge bases of individuals. In contrast to most existing researches in the field of computer vision that focus on identifying low-level script-like activities such as moving boxes, our goal is to extract life events where high-level activities like moving into a new house are recorded. The challenges to be tackled include: (1) identifying which objects in a given scene related to the life events of the protagonist we concern, and (2) determining the association between an extracted visual concept and a more high-level description of a video clip. To address the research issues, we construct a video life event extraction dataset VidLife by exploiting videos from the TV series The Big Bang Theory, in which the plot is around the daily lives of several characters. A pilot multitask learning model is proposed to extract life events given video clips and subtitles for storing in the personal knowledge base.

Original language	English
Title of host publication	CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	4436-4444
Number of pages	9
ISBN (Electronic)	9781450384469
DOIs	https://doi.org/10.1145/3459637.3482022
State	Published - 26 Oct 2021
Event	30th ACM International Conference on Information and Knowledge Management, CIKM 2021 - Virtual, Online, Australia Duration: 1 Nov 2021 → 5 Nov 2021

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings

Conference

Conference	30th ACM International Conference on Information and Knowledge Management, CIKM 2021
Country/Territory	Australia
City	Virtual, Online
Period	1/11/21 → 5/11/21

Keywords

life event extraction
lifelogging
personal knowledge base construction

Access to Document

10.1145/3459637.3482022

Cite this

Chu, T. T., Yen, A. Z., Ang, W. H., Huang, H. H., & Chen, H. H. (2021). VidLife: A Dataset for Life Event Extraction from Videos. In CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management (pp. 4436-4444). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3459637.3482022

@inproceedings{21cbd02e44014c6798c5f72efab19f10,

title = "VidLife: A Dataset for Life Event Extraction from Videos",

abstract = "Filming video blogs, which is shortened to vlog, becomes a popular way for people to record their life experiences in recent years. In this work, we present a novel task that is aimed at extracting life events from videos and constructing personal knowledge bases of individuals. In contrast to most existing researches in the field of computer vision that focus on identifying low-level script-like activities such as moving boxes, our goal is to extract life events where high-level activities like moving into a new house are recorded. The challenges to be tackled include: (1) identifying which objects in a given scene related to the life events of the protagonist we concern, and (2) determining the association between an extracted visual concept and a more high-level description of a video clip. To address the research issues, we construct a video life event extraction dataset VidLife by exploiting videos from the TV series The Big Bang Theory, in which the plot is around the daily lives of several characters. A pilot multitask learning model is proposed to extract life events given video clips and subtitles for storing in the personal knowledge base.",

keywords = "life event extraction, lifelogging, personal knowledge base construction",

author = "Chu, {Tai Te} and Yen, {An Zi} and Ang, {Wei Hong} and Huang, {Hen Hsen} and Chen, {Hsin Hsi}",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 30th ACM International Conference on Information and Knowledge Management, CIKM 2021 ; Conference date: 01-11-2021 Through 05-11-2021",

year = "2021",

month = oct,

day = "26",

doi = "10.1145/3459637.3482022",

language = "English",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "4436--4444",

booktitle = "CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management",

}

Chu, TT, Yen, AZ, Ang, WH, Huang, HH & Chen, HH 2021, VidLife: A Dataset for Life Event Extraction from Videos. in CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 4436-4444, 30th ACM International Conference on Information and Knowledge Management, CIKM 2021, Virtual, Online, Australia, 1/11/21. https://doi.org/10.1145/3459637.3482022

VidLife: A Dataset for Life Event Extraction from Videos. / Chu, Tai Te; Yen, An Zi; Ang, Wei Hong et al.
CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2021. p. 4436-4444 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - VidLife

T2 - 30th ACM International Conference on Information and Knowledge Management, CIKM 2021

AU - Chu, Tai Te

AU - Yen, An Zi

AU - Ang, Wei Hong

AU - Huang, Hen Hsen

AU - Chen, Hsin Hsi

PY - 2021/10/26

Y1 - 2021/10/26

N2 - Filming video blogs, which is shortened to vlog, becomes a popular way for people to record their life experiences in recent years. In this work, we present a novel task that is aimed at extracting life events from videos and constructing personal knowledge bases of individuals. In contrast to most existing researches in the field of computer vision that focus on identifying low-level script-like activities such as moving boxes, our goal is to extract life events where high-level activities like moving into a new house are recorded. The challenges to be tackled include: (1) identifying which objects in a given scene related to the life events of the protagonist we concern, and (2) determining the association between an extracted visual concept and a more high-level description of a video clip. To address the research issues, we construct a video life event extraction dataset VidLife by exploiting videos from the TV series The Big Bang Theory, in which the plot is around the daily lives of several characters. A pilot multitask learning model is proposed to extract life events given video clips and subtitles for storing in the personal knowledge base.

AB - Filming video blogs, which is shortened to vlog, becomes a popular way for people to record their life experiences in recent years. In this work, we present a novel task that is aimed at extracting life events from videos and constructing personal knowledge bases of individuals. In contrast to most existing researches in the field of computer vision that focus on identifying low-level script-like activities such as moving boxes, our goal is to extract life events where high-level activities like moving into a new house are recorded. The challenges to be tackled include: (1) identifying which objects in a given scene related to the life events of the protagonist we concern, and (2) determining the association between an extracted visual concept and a more high-level description of a video clip. To address the research issues, we construct a video life event extraction dataset VidLife by exploiting videos from the TV series The Big Bang Theory, in which the plot is around the daily lives of several characters. A pilot multitask learning model is proposed to extract life events given video clips and subtitles for storing in the personal knowledge base.

KW - life event extraction

KW - lifelogging

KW - personal knowledge base construction

UR - http://www.scopus.com/inward/record.url?scp=85119203466&partnerID=8YFLogxK

U2 - 10.1145/3459637.3482022

DO - 10.1145/3459637.3482022

M3 - Conference contribution

AN - SCOPUS:85119203466

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 4436

EP - 4444

BT - CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

Y2 - 1 November 2021 through 5 November 2021

ER -

Chu TT, Yen AZ, Ang WH, Huang HH, Chen HH. VidLife: A Dataset for Life Event Extraction from Videos. In CIKM 2021 - Proceedings of the 30th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery. 2021. p. 4436-4444. (International Conference on Information and Knowledge Management, Proceedings). doi: 10.1145/3459637.3482022

VidLife: A Dataset for Life Event Extraction from Videos

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this