Cumulative Learning Using Functionally Similar States

Mahmud, M.M. Hassan; Ray, Sylvian R.

Permalink

https://hdl.handle.net/2142/10958 Copy

Description

Title

Cumulative Learning Using Functionally Similar States

Author(s)

• Mahmud, M.M. Hassan
• Ray, Sylvian R.

Issue Date

2005-01

Keyword(s)

Machine Learning Artificial Intelligence

Abstract

In this paper we propose a Cumulative Learning System for artificial agents that ses the idea of Functional Similarity between states. The general idea of Cumulative Learning is to build a cognitive architecture for an artificial agent that 'lives' for a long time and solves many related tasks during its lifetime. Two states (or situations) are said to be functionally similar (FS) with respect to an action if the action induces the same change on both the states. We define the notion of FS for Markov Environments, and then use that to develop a Predictive Model (PM) that given states and actions observed so far, predicts the next state when an action is taken in some novel state (state never observed before or often) - i.e. the PM is a novel type of forward model. We also describe a planning mechanism for goal directed MDPs with multiple goals that uses the PM to solve tasks quicker using information from solutions to similar tasks solved previously by the agent. After establishing some necessary theoretical properties of both we perform experiments that shows the efficacy of our method. We also outline how the current system, which can actually be categorized as a Lifelong Learning system, may be extended to a complete cumulative learning system.

Type of Resource

text

Permalink

http://hdl.handle.net/2142/10958

Copyright and License Information

You are granted permission for the non-commercial reproduction, distribution, display, and performance of this technical report in any format, BUT this permission is only for a period of 45 (forty-five) days from the most recent time that you verified that this technical report is still available from the University of Illinois at Urbana-Champaign Computer Science Department under terms that include this permission. All other rights are reserved by the author(s).

Owning Collections