hrvatski jezikClear Cookie - decide language by browser settings

A Deep Learning Perspective on Beauty, Sentiment, and Remembrance of Art

Cetinić, Eva; Lipić, Tomislav; Grgić, Sonja (2019) A Deep Learning Perspective on Beauty, Sentiment, and Remembrance of Art. IEEE Access, 7 . pp. 73694-73710. ISSN 2169-3536

[img]
Preview
PDF - Published Version - article
Download (23MB) | Preview

Abstract

With the emergence of large digitized fine art collections and the successful performance of deep learning techniques, new research prospects unfold in the intersection of artificial intelligence and art. In order to explore the applicability of deep learning techniques in understanding art images beyond object recognition and classification, we employ convolutional neural networks (CNN) to predict scores related to three subjective aspects of human perception: aesthetic evaluation of the image, sentiment evoked by the image, and memorability of the image. For each concept, we evaluate several different CNN models trained on various natural image datasets and select the best performing model based on the qualitative results and the comparison with existing subjective ratings of artworks. Furthermore, we employ different decision treebased machine learning models to analyze the relative importance of various image features related to the content, composition, and color in determining image aesthetics, visual sentiment, and memorability scores. Our findings suggest that content and image lighting have significant influence on aesthetics, in which color vividness and harmony strongly influence sentiment prediction, while object emphasis has a high impact on memorability. In addition, we explore the predicted aesthetic, sentiment, and memorability scores in the context of art history by analyzing their distribution in regard to different artistic styles, genres, artists, and centuries. The presented approach enables new ways of exploring fine art collections based on highly subjective aspects of art, as well as represents one step forward toward bridging the gap between traditional formal analysis and the computational analysis of fine art.

Item Type: Article
Uncontrolled Keywords: convolutional neural networks ; image aesthetics ; image memorability ; fine art ; visual sentiment
Subjects: TECHNICAL SCIENCES > Computing
Divisions: Center for Informatics and Computing
Division of Electronics
Projects:
Project titleProject leaderProject codeProject type
Napredne metode i tehnologije u znanosti o podatcima i kooperativnim sustavima [Advanced methods and technologies in Data Science and Cooperative SystemsUNSPECIFIEDKK.01.1.1.01.0009UNSPECIFIED
Depositing User: Tomislav Lipić
Date Deposited: 23 Jul 2019 14:02
Last Modified: 23 Jul 2019 14:03
URI: http://fulir.irb.hr/id/eprint/4914
DOI: 10.1109/ACCESS.2019.2921101

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year