Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation

Yun Chun Chen*, Yen-Yu Lin, Ming Hsuan Yang, Jia Bin Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

27 Scopus citations

Abstract

We present an approach for jointly matching and segmenting object instances of the same category within a collection of images. In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks. The key insights of our method are two-fold. First, the estimated dense correspondence fields from semantic matching provide supervision for object co-segmentation by enforcing consistency between the predicted masks from a pair of images. Second, the predicted object masks from object co-segmentation in turn allow us to reduce the adverse effects due to background clutters for improving semantic matching. Our model is end-to-end trainable and does not require supervision from manually annotated correspondences and object masks. We validate the efficacy of our approach on five benchmark datasets: TSS, Internet, PF-PASCAL, PF-WILLOW, and SPair-71k, and show that our algorithm performs favorably against the state-of-the-art methods on both semantic matching and object co-segmentation tasks.

Original languageAmerican English
Article number9057736
Pages (from-to)3632-3647
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume43
Issue number10
DOIs
StatePublished - 1 Oct 2021

Keywords

  • Semantic matching
  • object co-segmentation
  • weakly-supervised learning

Fingerprint

Dive into the research topics of 'Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation'. Together they form a unique fingerprint.

Cite this