Are You Killing Time? Predicting Smartphone Users' Time-killing Moments via Fusion of Smartphone Sensor Data and Screenshots

Yu Chun Chen, Yu Jen Lee, Kuei Chun Kao, Jie Tsai, En Chi Liang, Wei Chen Chiu, Faye Shih, Yung Ju Chang*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Time-killing on smartphones has become a pervasive activity, and could be opportune for delivering content to their users. This research is believed to be the first attempt at time-killing detection, which leverages the fusion of phone-sensor and screenshot data. We collected nearly one million user-annotated screenshots from 36 Android users. Using this dataset, we built a deep-learning fusion model, which achieved a precision of 0.83 and an AUROC of 0.72. We further employed a two-stage clustering approach to separate users into four groups according to the patterns of their phone-usage behaviors, and then built a fusion model for each group. The performance of the four models, though diverse, yielded better average precision of 0.87 and AUROC of 0.76, and was superior to that of the general/unified model shared among all users. We investigated and discussed the features of the four time-killing behavior clusters that explain why the models' performance differ.

Original languageEnglish
Title of host publicationCHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450394215
DOIs
StatePublished - 19 Apr 2023
Event2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 - Hamburg, Germany
Duration: 23 Apr 202328 Apr 2023

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2023 CHI Conference on Human Factors in Computing Systems, CHI 2023
Country/TerritoryGermany
CityHamburg
Period23/04/2328/04/23

Keywords

  • Deep Learning
  • Mobile Devices
  • Opportune Moment
  • Screenshot
  • Time-killing

Fingerprint

Dive into the research topics of 'Are You Killing Time? Predicting Smartphone Users' Time-killing Moments via Fusion of Smartphone Sensor Data and Screenshots'. Together they form a unique fingerprint.

Cite this