[agents] NEW BOOK on relational Markov decision processes, reinforcement learning, and decision-theoretic planning

[Martijn Van Otterlo]

Dear colleagues,

Learning and acting in large, probabilistic, relational worlds is
investigated in research fields such as machine learning, intelligent
agents, knowledge representation and optimization. This research has
been ongoing for many years, and below follows an *announcement* for the
first *book* that describes and surveys these developments in a unified
manner.

It starts with 'learning sequential decision making problems under
uncertainty' and surveys important developments related to knowledge
representation, generalization and abstraction. The core of the book is
a detailed and complete study of 'relational representations' in this
field. In addition to introducing several new methodological, technical
and algorithmic advances, the book contains complete surveys of
relational reinforcement learning, first-order decision-theoretic
planning, and matters related to world models, hierarchies and knowledge
transfer.

The book provides a complete and self-contained reference work, and is
aimed at (PhD) students and researchers working on matters related to
learning and acting in large, probabilistic, (relational) worlds.

Best regards,

Martijn van Otterlo

----------------------------------------------------------
Book Announcement
----------------------------------------------------------

THE LOGIC OF ADAPTIVE BEHAVIOR: 
    Knowledge Representation and Algorithms for
    Adaptive Sequential Decision Making under Uncertainty
    in First-Order and Relational Domains
    
by Martijn van Otterlo

2009 -- IOS Press, 
  Amsterdam, Berlin, Oxford, Tokyo, Washington D.C.
ISBN 978-1-58603-969-1
Hardcover, 500+pp, 800++refs 
(also available in electronic version)

--- See for more information:
http://www.iospress.nl/html/9781586039691.php

--- Preface and TOC: (Online)
http://www.booksonline.iospress.nl/Content/View.aspx?piid=11738

Contents: Chapter 1: Introduction / Chapter 2: Markov decision processes: concepts and algorithms / Chapter 3: Generalization and abstraction in MDPs / Chapter 4: Reasoning, learning and acting in first-order worlds / Chapter 5: Model-free algorithms for relational MDPs / Chapter 6: Model-based algorithms for relational MDPs / Chapter 7: Sapience, models and hierarchy / Chapter 8: Conclusions and future directions

---------------------------------------------------------
Back Cover text:
---------------------------------------------------------
Learning and reasoning in large, structured, probabilistic worlds is at the heart of artificial intelligence. Markov decision processes have become the de facto standard in modeling and solving sequential decision making problems under uncertainty. Many efficient reinforcement learning and dynamic programming techniques exist that can solve such problems. Until recently, the representational state-of-the-art in this field was based on propositional representations.

However, it is hard to imagine a truly general, intelligent system that does not conceive of the world in terms of objects and their properties and relations to other objects. To this end, this book studies lifting Markov decision processes, reinforcement learning and dynamic programming to the first-order (or, relational) setting. Based on an extensive analysis of propositional representations and techniques, a methodological translation is constructed from the propositional to the relational setting. Furthermore, this book provides a thorough and complete description of the state-of-the-art, it surveys vital, related historical developments and it contains extensive descriptions of several new model-free and model-based solution techniques.  


-- 
--------------------------------------------------------------
Dr. Ir. Martijn van Otterlo (room 02.151)

      Department of Computer Science
      Katholieke Universiteit Leuven
      Celestijnenlaan 200A
      B3001 Heverlee, Belgium.

      TeLePhOnE ... +32 +16 32 7741
      WeB ... http://www.cs.kuleuven.be/~martijn
      eMaIl ... Martijn.vanOtterlo at cs.kuleuven.be

'When I grow up, I'd like to become a librarian'
--------------------------------------------------------------