Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations

Mahmud, M.M. Hassan; Ray, Sylvian R.

Permalink

https://hdl.handle.net/2142/11343 Copy

Description

Title

Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations

Author(s)

• Mahmud, M.M. Hassan
• Ray, Sylvian R.

Issue Date

2007-06

Keyword(s)

computer science

Abstract

In transfer learning we aim to solve new problems quicker by using information gained from solving related problems. Transfer learning has been successful in practice, and extensive PAC analysis of these methods has been developed. However it is not yet clear how to define relatedness between tasks. This is considered as a major problem as, aside from being conceptually troubling, it makes it unclear how much information to transfer and when and how to transfer it. In this paper we propose to measure the amount of information one task contains about another using conditional Kolmogorov complexity between the tasks. We show how existing theory neatly solves the problem of measuring relatedness and transferring the .right. amount of information in sequential transfer learning in a Bayesian setting. The theory also suggests that, in a very formal and precise sense, no other transfer method can do much better than the Kolmogorov Complexity theoretic transfer method, and that sequential transfer is always justified. We also develop a practical approximation to the method and use it to transfer information between 8 arbitrarily chosen databases from the UCI ML repository.

Type of Resource

text

Permalink

http://hdl.handle.net/2142/11343

Copyright and License Information

You are granted permission for the non-commercial reproduction, distribution, display, and performance of this technical report in any format, BUT this permission is only for a period of 45 (forty-five) days from the most recent time that you verified that this technical report is still available from the University of Illinois at Urbana-Champaign Computer Science Department under terms that include this permission. All other rights are reserved by the author(s).

Owning Collections