Beyond Saliency: Assessing Visual Balance with High-level Cues

Baris Kandemir, Zihan Zhou, Jia Li, James Z. Wang
The Pennsylvania State University, USA


Automatic composition optimization is a vital technique for computational photography systems. Balance in composition is one of the agreed-upon principles of aesthetics and is commonly employed as a visual feature in many computational aesthetics studies. It refers to an equilibrium of visual weights within composition. Existing composition optimization and aesthetic quality assessment systems utilize the saliency map to represent balance. However, saliency map methods fail to account for high-level visual features that are important for compositional balance. Our work establishes a framework for the purpose of evaluating the relationship between visual features and compositional balance. This provides a better understanding of compositional balance and help improve composition optimization performance. A dataset based on a human subject study was created with photos representing main balance concepts such as symmetric, dynamic balance, and imbalance. We take the visual center given by human subjects as the dependent variable and the center-of-mass for each type of visual features as the predictor variable. Based on a linear regression model, we can assess how much each type of visual features contributes to the prediction of the visual center. Our findings show that highlevel visual elements can help increase prediction accuracy with significance on top of saliency maps. Specifically, extra information provided through human and dominant vanishing point detection is statistically significant for assessing balance in the composition.

Full color PDF file (10.5 MB)

Citation: Baris Kandemir, Zihan Zhou, Jia Li and James Z. Wang, ``Beyond Saliency: Assessing Visual Balance with High-level Cues,'' Proceedings of the Thematic Workshops of ACM Multimedia, in conjunction with the ACM Multimedia Conference, pp. 26-34 , Mountain View, California, October 2017.

Copyright 2017 ACM. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works, must be obtained from ACM.

Last Modified: August 10, 2017
© 2017