Learning Latent Representations of Style with Archetypal Style Analysis
| Authors: |
Daan Wynen Cordelia Schmid Julien Mairal |
|---|---|
| Contact: | firstname.lastname@inria.fr |
| Status: | Manuscript submitted for publication. |
Abstract
In this paper, we introduce an unsupervised learning approach to automatically discover, summarize, and manipulate artistic styles from large collections of paintings. Our method is based on archetypal analysis, which is an unsupervised learning technique akin to sparse coding with a geometric interpretation. When applied to deep image representations from a collection of artworks, it learns a dictionary of archetypal styles, which can be easily visualized. After training the model, the style of a new image, which is characterized by local statistics of deep visual features, is approximated by a sparse convex combination of archetypes. This enables us to interpret which archetypal styles are present in the input image, and in which proportion. By leveraging large sets of annotations obtained from the WikiArt website, we show that archetypal styles - which are learned from raw paintings only - are often meaningful and are able to automatically recover some of well-known artists's or groups of artists' periods. Finally, our approach allows us to manipulate the coefficients of the latent archetypal decomposition, and achieve various special effects on a given painting or photograph, such as style enhancement, transfer, and interpolation between multiple archetypal styles.
Overview
We extend on our work on archetypal style analysis, applying it to the WikiArt collection, a bigger subset of the artworks collected on wikiart.org. Namely, we analyze the works of Vincent van Gogh, Salvador Dalí, Pablo Picasso, four famous painters, and the Venetian school. We also apply archetypal style analysis to different subsets of artworks, examining the properties of the underlying representations of style. It turns out that although the two representations we use capture some aspects of artistic style in an art-historic sense, despite both being based on imagenet pretrained CNNs. These pages contain examples of the archetypes computed on the different subsets, the decompositions of artworks' styles into archetypes, as well as many more results of the style enhancement method described in the paper.
Acknowledgements
This work was supported by a grant from ANR (MACARON project under grant number ANR-14-CE23-0003-01), by the ERC grant number 714381 (SOLARIS project) and the ERC advanced grant Allegro.