Record Details

Feature Search in the Grassmanian in Online Reinforcement Learning

DSpace at IIT Bombay

View Archive Info
 
 
Field Value
 
Title Feature Search in the Grassmanian in Online Reinforcement Learning
 
Creator BHATNAGAR, S
BORKAR, VS
PRABUCHANDRAN, KJ
 
Subject Feature adaptation
Grassman manifold
online learning
residual gradient scheme
stochastic approximation
temporal difference learning
FUNCTION APPROXIMATION
 
Description We consider the problem of finding the best features for value function approximation in reinforcement learning and develop an online algorithm to optimize the mean square Bellman error objective. For any given feature value, our algorithm performs gradient search in the parameter space via a residual gradient scheme and, on a slower timescale, also performs gradient search in the Grassman manifold of features. We present a proof of convergence of our algorithm. We show empirical results using our algorithm as well as a similar algorithm that uses temporal difference learning in place of the residual gradient scheme for the faster timescale updates.
 
Publisher IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
 
Date 2014-10-17T04:46:08Z
2014-10-17T04:46:08Z
2013
 
Type Article
 
Identifier IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING, 7(5)746-758
http://dx.doi.org/10.1109/JSTSP.2013.2255022
http://dspace.library.iitb.ac.in/jspui/handle/100/15984
 
Language en