An intelligent content-based image retrieval methodology using transfer learning for digital IP protection

Trademarks are used by companies to help customers identify products or services using images or logos in addition to slogans, words, names, sounds, smells, color, and motions. Trademark logos are widely distributed through advertising and published through online media websites and social networks such as Facebook, Pinterest, and Flicker. The intellectual property (IP) rights of the trademark owners have strong legal protection when registered with international intellectual property platforms such as the US Patent and Trademark Office and the World Intellectual Property Office. Using a registered trademark without prior consent of the owner may result in intellectual property infringement with severe legal consequences under civil or criminal law. Companies invest large capital resources in protecting their trademark from being copied or misused in ways that confuse the customers or steal market share. This research focuses on trademark (TM) logo image retrieval systems used in the cyber marketplaces to identify similar TM logo images online automatically and intelligently. The methodology developed for TM logo similarity measurement is based on content-based image retrieval. Content retrieval reduces the gap between high-level semantic interpretation of human vision and the low-level features processed by the machine. The proposed transfer learning methodology uses embedded learning with triplet loss to fine-tune a pre-trained convolutional neural network model. The Logo-2K+ large-scale logo dataset is re-organized and divided into the top 70% as the training set and the remaining 30% as the testing set. The results show that the novel transfer learning approach is developed and demonstrated in this research for the intelligent automatic detection of similar TM logo images with high accuracy. The verification experiments (trained with 7625 logos and tested with 3221 logos) demonstrates that the Recall@10 of the test set can reach 95% using the advanced convolutional neural network model (VGG19) adjusted with the novel transfer learning methodology.