About Elastic

Scale variation has been a challenge from traditional to modern approaches in computer vision. Most solutions to scale issues have a similar theme: a set of intuitive and manually designed policies that are generic and fixed (e.g. SIFT or feature pyramid). We argue that the scaling policy should be learned from data. In this paper, we introduce ELASTIC, a simple, efficient and yet very effective approach to learn a dynamic scale policy from data. We formulate the scaling policy as a non-linear function inside the network's structure that (a) is learned from data, (b) is instance specific, (c) does not add extra computation, and (d) can be applied on any network architecture. We applied ELASTIC to several state-of-the-art network architectures and showed consistent improvement without extra (sometimes even lower) computation on ImageNet classification, MSCOCO multi-label classification, and PASCAL VOC semantic segmentation. Our results show major improvement for images with scale challenges.

Elastic Scale Policies
Instance-specific scale policy. Scaling policy in CNNs are typically integrated in the network architecture manually in a pyramidal fashion. The color bar in this figure (second row) shows the scales at different blocks of the ResNext50 architecture. The early layers receive eXtralarge resolutions and in the following layers resolutions decrease as Large, Medium, and Small. We argue that scaling policy in CNNs should be instance-specific. Our Elastic model (the third row) allows different scaling policy for different input images and it learns from the training data how to pick the best policy. For scale challenging images e.g. images with lots of small(or diverse scale) objects, it is crucial that network can adapt its scale policy based on the input. As it can be seen in this figure, Elastic gives a better prediction for these scale challenging images.

Model

Elastic Model
Multi-scaling model structures. This figure illustrates different approaches to multi-scaling in CNN models and our Elastic model. The solid-line rectangles show the input size and the dashed-line rectangles shows the filter size. ELASTIC layers: 1. down-sample the input feature map to multiple resolutions. 2. Process them all with 3×3 kernels. 3. Up-sample them to the input resolution and add them together.

Elatic Paper

ELASTIC: Improving CNNs with Dynamic Scaling Policies

ELASTIC: Improving CNNs with Dynamic Scaling Policies

Huiyu Wang, Aniruddha Kembhavi, Ali Farhadi, Alan Yuille, and Mohammad Rastegari • CVPR • 2019

CODE AND MODELS

Our PyTorch implementation and pre-trained models are publicly available.