Recognizing human actions with outlier frames by observation filtering and completion

Shih Yao Lin, Yen Yu Lin, Chu Song Chen, Yi Ping Hung

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


This article addresses the problem of recognizing partially observed human actions. Videos of actions acquired in the realworld often contain corrupt frames caused by various factors. These frames may appear irregularly, and make the actions only partially observed. They change the appearance of actions and degrade the performance of pretrained recognition systems. In this article, we propose an approach to address the corrupt-frame problem without knowing their locations and durations in advance. The proposed approach includes two key components: outlier filtering and observation completion. The former identifies and filters out unobserved frames, and the latter fills up the filtered parts by retrieving coherent alternatives from training data. Hidden Conditional Random Fields (HCRFs) are then used to recognize the filtered and completed actions. Our approach has been evaluated on three datasets, which contain both fully observed actions and partially observed actions with either real or synthetic corrupt frames. The experimental results show that our approach performs favorably against the other state-of-the-art methods, especially when corrupt frames are present.
Original languageAmerican English
JournalACM Transactions on Multimedia Computing, Communications and Applications
Issue number3
StatePublished - Aug 2017


  • And conditional random fields
  • Early prediction
  • Gap filling
  • Human action recognition
  • Observation completion
  • Outlier filtering

Cite this