Deep Multi-Patch Aggregation Network
for Image Style, Aesthetics, and Quality Estimation
Xin Lu
The Pennsylvania State University
Zhe Lin, Xiaohui Shen, Radomir Mech
Adobe Research
James Z. Wang
The Pennsylvania State University
Abstract:
This paper investigates problems of image style, aesthetics,
and quality estimation, which require fine-grained
details from high-resolution images, utilizing deep neural
network training approach. Existing deep convolutional
neural networks mostly extracted one patch such as a downsized
crop from each image as a training example. However,
one patch may not always well represent the entire image,
which may cause ambiguity during training. We propose
a deep multi-patch aggregation network training approach,
which allows us to train models using multiple patches generated
from one image. We achieve this by constructing
multiple, shared columns in the neural network and feeding
multiple patches to each of the columns. More importantly,
we propose two novel network layers (statistics and sorting)
to support aggregation of those patches. The proposed deep
multi-patch aggregation network integrates shared feature
learning and aggregation function learning into a unified
framework. We demonstrate the effectiveness of the deep
multi-patch aggregation network on the three problems, i.e.,
image style recognition, aesthetic quality categorization,
and image quality estimation. Our models trained using
the proposed networks significantly outperformed the state
of the art in all three applications.
Full Paper
(PDF, 1.8MB)
Citation:
Xin Lu, Zhe Lin, Xiaohui Shen, Radomir Mech and James Z. Wang, ``Deep
Multi-Patch Aggregation Network for Image Style, Aesthetics, and
Quality Estimation,'' International Conference on Computer Vision,
pp. 990-998, Santiago, Chile, IEEE, 2015.
© 2015 IEEE. Personal use of this material is permitted. However,
permission to reprint/republish this material for advertising or
promotional purposes or for creating new collective works for resale
or redistribution to servers or lists, or to reuse any copyrighted
component of this work in other works must be obtained from the IEEE.
Last Modified:
September 25, 2015
© 2015