Unsupervised semantic feature discovery for image object retrieval and tag refinement

Yin Hsi Kuo*, Wen-Huang Cheng, Hsuan Tien Lin, Winston H. Hsu


研究成果: Article同行評審

42 引文 斯高帕斯(Scopus)


We have witnessed the exponential growth of images and videos with the prevalence of capture devices and the ease of social services such as Flickr and Facebook. Meanwhile, enormous media collections are along with rich contextual cues such as tags, geo-locations, descriptions, and time. To obtain desired images, users usually issue a query to a search engine using either an image or keywords. Therefore, the existing solutions for image retrieval rely on either the image contents (e.g., low-level features) or the surrounding texts (e.g., descriptions, tags) only. Those solutions usually suffer from low recall rates because small changes in lighting conditions, viewpoints, occlusions, or (missing) noisy tags can degrade the performance significantly. In this work, we tackle the problem by leveraging both the image contents and associated textual information in the social media to approximate the semantic representations for the two modalities. We propose a general framework to augment each image with relevant semantic (visual and textual) features by using graphs among images. The framework automatically discovers relevant semantic features by propagation and selection in textual and visual image graphs in an unsupervised manner. We investigate the effectiveness of the framework when using different optimization methods for maximizing efficiency. The proposed framework can be directly applied to various applications, such as keyword-based image search, image object retrieval, and tag refinement. Experimental results confirm that the proposed framework effectively improves the performance of these emerging image retrieval applications.

頁(從 - 到)1079-1090
期刊IEEE Transactions on Multimedia
發行號4 PART1
出版狀態Published - 27 7月 2012


深入研究「Unsupervised semantic feature discovery for image object retrieval and tag refinement」主題。共同形成了獨特的指紋。