Please use this identifier to cite or link to this item:
|Scopus||Web of Science®||Altmetric|
|Title:||Two-stream 3-D convNet Fusion for action recognition in videos with arbitrary size and length|
|Citation:||IEEE Transactions on Multimedia, 2018; 20(3):634-644|
|Xuanhan Wang, Lianli Gao , Peng Wang, Xiaoshuai Sun and Xianglong Liu|
|Abstract:||3-D convolutional neural networks (3-D-convNets) have been very recently proposed for action recognition in videos, and promising results are achieved. However, existing 3-D-convNets has two “artificial” requirements that may reduce the quality of video analysis: 1) It requires a fixed-sized (e.g., 112 × 112) input video; and 2) most of the 3-D-convNets require a fixed-length input (i.e., video shots with fixed number of frames). To tackle these issues, we propose an end-to-end pipeline named Two-stream 3-D-convNet Fusion, which can recognize human actions in videos of arbitrary size and length using multiple features. Specifically, we decompose a video into spatial and temporal shots. By taking a sequence of shots as input, each stream is implemented using a spatial temporal pyramid pooling (STPP) convNet with a long short-term memory (LSTM) or CNN-E model, softmax scores of which are combined by a late fusion. We devise the STPP convNet to extract equal-dimensional descriptions for each variable-size shot, and we adopt the LSTM/CNN-E model to learn a global description for the input video using these time-varying descriptions. With these advantages, our method should improve all 3-D CNN-based video analysis methods. We empirically evaluate our method for action recognition in videos and the experimental results show that our method outperforms the state-of-the-art methods (both 2-D and 3-D based) on three standard benchmark datasets (UCF101, HMDB51 and ACT datasets).|
|Keywords:||Action recognition; 3D convolution neural networks|
|Rights:||© 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.|
|Appears in Collections:||Electrical and Electronic Engineering publications|
Files in This Item:
There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.