hrvatski jezikClear Cookie - decide language by browser settings

Fine-tuning Convolutional Neural Networks for fine art classification

Cetinić, Eva; Lipić, Tomislav; Grgić, Sonja (2018) Fine-tuning Convolutional Neural Networks for fine art classification. Expert Systems with Applications, 114 . pp. 107-118. ISSN 0957-4174

| Request a personal copy from author
[img] PDF - Accepted Version - article
Restricted to Registered users only until 1 December 2020.

Download (8MB) | Request a personal copy from author

Abstract

The increasing availability of large digitized fine art collections opens new research perspectives in the intersection of artificial intelligence and art history. Motivated by the successful performance of Convolutional Neural Networks (CNN) for a wide variety of computer vision tasks, in this paper we explore their applicability for art-related image classification tasks. We perform extensive CNN fine-tuning experiments and consolidate in one place the results for five different art- related classification tasks on three large fine art datasets. Along with addressing the previously explored tasks of artist, genre, style and time period classification, we introduce a novel task of classifying artworks based on their association with a specific national artistic context. We present state-of- the-art classification results of the addressed tasks, signifying the impact of our method on computational analysis of art, as well as other image classification related research areas. Furthermore, in order to question transferability of deep representations across various source and target domains, we systematically compare the effects of domain- specific weight initialization by evaluating networks pre-trained for different tasks, varying from object and scene recognition to sentiment and memorability labelling. We show that fine-tuning networks pre-trained for scene recognition and sentiment prediction yields better results than fine-tuning networks pre- trained for object recognition. This novel outcome of our work suggests that the semantic correlation between different domains could be inherent in the CNN weights. Additionally, we address the practical applicability of our results by analysing different aspects of image similarity. We show that features derived from fine-tuned networks can be employed to retrieve images similar in either style or content, which can be used to enhance capabilities of search systems in different online art collections.

Item Type: Article
Additional Information: This research has been partly supported by the European Regional Development Fund under the grant KK.01.1.1.01.0009 (DATACROSS).
Uncontrolled Keywords: Painting classification ; Deep learning ; Convolutional Neural Networks ; Fine-tuning strategies
Subjects: TECHNICAL SCIENCES > Computing
Divisions: Center for Informatics and Computing
Division of Electronics
Depositing User: Eva Cetinić
Date Deposited: 25 Nov 2019 14:39
Last Modified: 02 Dec 2019 13:38
URI: http://fulir.irb.hr/id/eprint/5146
DOI: 10.1016/j.eswa.2018.07.026

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year