A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games
type: Dataset
contact point: Organization
  fn: CCSD
keyword: preprints-working-papers-
keyword: gambling-houses
keyword: characterization-of-the-value
keyword: uniform-value
keyword: limit-value
keyword: general-values
keyword: infoeu-reposemanticspreprint
keyword: repeated-games
keyword: markov-decision-process
keyword: distance-for-belief-spaces
keyword: mathmath-ocmathematics-mathoptimization-and-control-mathoc
keyword: kantorovich-rubinstein-duality
keyword: pomdp
landing page: https://hal.science/hal-00674998
description: Given a finite set $K$, we denote by $X=\Delta(K)$ the set of probabilities on $K$ and by $Z=\Delta_f(X)$ the set of Borel probabilities on $X$ with finite support. Studying a Markov Decision Process with partial information on $K$ naturally leads to a Markov Decision Process with full information on $X$. We introduce a new metric $d_*$ on $Z$ such that the transitions become $1$-Lipschitz from $(X, \|.\|_1)$ to $(Z,d_*)$. In the first part of the article, we define and prove several properties of the metric $d_*$. Especially, $d_*$ satisfies a Kantorovich-Rubinstein type duality formula and can be characterized by using disintegrations. In the second part, we characterize the limit values in several classes of ''compact non expansive" Markov Decision Processes. In particular we use the metric $d_*$ to characterize the limit value in Partial Observation MDP with finitely many states and in Repeated Games with an informed controller with finite sets of states and actions. Moreover in each case we can prove the existence of a generalized notion of uniform value where we consider not only the Cesàro mean when the number of stages is large enough but any evaluation function $\theta \in \Delta(\N^*)$ when the impatience $I(\theta)=\sum_{t\geq 1} |\theta_{t+1}-\theta_t|$ is small enough.
issued: 2026-05-26T11:34:53.419645
language: en
modified: 2026-05-26T11:34:53.419650

test_moissonnage_selune
type: Agent

A distance for probability spaces, and long-term values in Markov Decision Processes and Repeated Games
type: Distribution
access URL: https://hal.science/hal-00674998
format: HTML
issued: 2026-05-26T11:34:53.455751
modified: 2026-05-26T11:34:53.386409