Exploiting attribute dependency for attribute assignment in crowded scenes

Abstract

Attributes now play a vital role for characterizing a crowded scene. Compared to low-level visual features, processing informed by attributes can capture rich semantic information. However, to effectively assign attributes to a crowded scene still remains a challenging task. In this letter, inspired by a recently proposed zero-shot learning framework, a novel attribute assignment method that maps low-level features to predefined attributes is proposed. In particular, we propose to exploit the attribute dependency during the phase of attribute assignment, which can be regarded as our main contribution. In addition, to further enhance the performance, an effective low-level feature extraction mechanism is also proposed. More precisely, appearance and motion features are first simultaneously extracted from several sampled video frames and corresponding optical flow fields via deep convolutional neural network and then, respectively, aggregated by using Fisher vector encoding to form the low-level representation of crowded scenes. Experimental results on the challenging WWW dataset demonstrate that both the proposed attribute assignment method and the low-level feature extraction mechanism outperform the state of the art.