GlideNet: Global, Local and Intrinsic based Dense Embedding NETwork for Multi-category Attributes Prediction

Attaching attributes (such as color, shape, state, action) to object\ncategories is an important computer vision problem. Attribute prediction has\nseen exciting recent progress and is often formulated as a multi-label\nclassification problem. Yet significant challenges remain in: 1) predicting\ndiverse attributes over multiple categories, 2) modeling attributes-category\ndependency, 3) capturing both global and local scene context, and 4) predicting\nattributes of objects with low pixel-count. To address these issues, we propose\na novel multi-category attribute prediction deep architecture named GlideNet,\nwhich contains three distinct feature extractors. A global feature extractor\nrecognizes what objects are present in a scene, whereas a local one focuses on\nthe area surrounding the object of interest. Meanwhile, an intrinsic feature\nextractor uses an extension of standard convolution dubbed Informed Convolution\nto retrieve features of objects with low pixel-count. GlideNet uses gating\nmechanisms with binary masks and its self-learned category embedding to combine\nthe dense embeddings. Collectively, the Global-Local-Intrinsic blocks\ncomprehend the scene's global context while attending to the characteristics of\nthe local object of interest. Finally, using the combined features, an\ninterpreter predicts the attributes, and the length of the output is determined\nby the category, thereby removing unnecessary attributes. GlideNet can achieve\ncompelling results on two recent and challenging datasets -- VAW and CAR -- for\nlarge-scale attribute prediction. For instance, it obtains more than 5\\% gain\nover state of the art in the mean recall (mR) metric. GlideNet's advantages are\nespecially apparent when predicting attributes of objects with low pixel counts\nas well as attributes that demand global context understanding. Finally, we\nshow that GlideNet excels in training starved real-world scenarios.\n