Measuring and Predicting Visual Importance of Similar Objects

• Yan Kong¹
• Weiming Dong¹
• Xing Mei¹
• Chongyang Ma²


• Tong-Yee Lee³
• Siwei Lyu⁴
• Feiyue Huang⁵
• Xiaopeng Zhang¹


• ¹Institute of Automation, Chinese Academy of Sciences
• ²University of Southern California
• ³National Cheng Kung University
• ⁴University at Albany, SUNY
• ⁵Tencent

Prediction of object importance importance values which are normalized to (0, 1)

Abstract

Similar objects are ubiquitous and abundant in both natural and artificial scenes. Determining the visual importance of several similar objects in a complex photograph is a challenge for image understanding algorithms. This study aims to define the importance of similar objects in an image and to develop a method that can select the most important instances for an input image from multiple similar objects. This task is challenging because multiple objects must be compared without adequate semantic information. This challenge is addressed by building an image database and designing an interactive system to measure object importance from human observers. This ground truth is used to define a range of features related to the visual importance of similar objects. Then, these features are used in learning-to-rank and random forest to rank similar objects in an image. Importance predictions were validated on 5922 objects. The most important objects can be identified automatically. The factors related to composition (e.g., size, location, and overlap) are particularly informative, although clarity and color contrast are also important. We demonstrate the usefulness of similar object importance on various applications, including image retargeting, image compression, image re-attentionizing, image admixture, and manipulation of blindness images.

Keywords

Similar objects, visual importance, listwise ranking

Paper

Links