Improvement and measurement of neural style transfer

Yeh, Mao-Chuang

Permalink

https://hdl.handle.net/2142/102527 Copy

Description

Title

Improvement and measurement of neural style transfer

Author(s)

Yeh, Mao-Chuang

Issue Date

2018-12-14

Director of Research (if dissertation) or Advisor (if thesis)

Forsyth, David A.

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• style transfer
• gram matrix
• texture synthesis

Abstract

Style transfer methods produce a transferred image which is a rendering of a content image in the manner of a style image. There is a rich literature of variant methods. We seek to understand how to improve style transfer: in particular, there is some evidence that cross-layer losses are helpful, and some evidence that optimization problems might present difficulties. To do so requires quantitative evaluation procedures, but current evaluation is qualitative, mostly involving user studies. We describe a novel quantitative evaluation procedure. Our procedure relies on two statistics: the Effectiveness (E) statistic measures the extent that a given style has been transferred to the target, and the Coherence (C) statistic measures the extent to which the original image's content is preserved. Our statistics are calibrated to human preference: targets with larger values of E (resp C) will reliably be preferred by human subjects in comparisons of style (resp. content). We use these statistics to investigate relative performance of a number of recent style transfer methods, revealing a number of intriguing properties. {Our experiments pool multiple style transfers from many different styles to many different content images using many different style weights, allowing us to make general statements about what influences style transfer. }Admissible methods lie on a Pareto frontier (i.e. improving E reduces C, or vice versa). Three methods are admissible: Universal style transfer produces very good C but weak E; modifying the optimization used for Gatys' loss produces a method with strong E and strong C; and a modified cross-layer method has slightly better E at strong cost in C. While the histogram loss improves the E statistics of Gatys' method, it does not make the method admissible. Surprisingly, style weights have relatively little effect, and most variability in transfer is explained by the style itself (meaning experimenters can be misguided by selecting styles).

Graduation Semester

2018-12

Type of Resource

text

Permalink

http://hdl.handle.net/2142/102527

Copyright and License Information

Copyright 2018 Mao-Chuang Yeh

Owning Collections