ESAIM Control Optimisation and Calculus of Variations· 2026Q1
Absorbing markov decision processes: geometric properties and sufficiency of finite mixtures of deterministic policies
- 0citations
- Q1SCImago
- 2026year
Short summary
This paper characterizes geometric properties of occupancy measures in absorbing Markov Decision Processes (MDPs), showing these structures are fully defined by MDP parameters.
AI-generated from the title and abstract; the full text is not read.
Key points
- Geometric properties of occupancy measures in absorbing MDPs are analyzed, including faces and their relative algebraic interiors.
- These geometric structures are fully characterized by the parameters defining the MDP.
- Finite mixtures of deterministic stationary policies are shown to be a sufficient class of policies for uniformly absorbing MDPs.
- A condition is provided for an occupancy measure to be generated by a finite mixture of deterministic policies.
AI-generated from the title and abstract; the full text is not read.
Abstract
In this paper we investigate several geometric properties of the set of occupancy measures of an absorbing Markov decision process (MDP). In particular, we analyse the structure of the faces generated by a given occupancy measure, together with their relative algebraic interior. We also determine the affine hulls of these faces and describe the associated parallel linear subspaces. It is shown that these structures can be fully characterised in terms of the parameters that define the underlying Markov decision process. Moreover, we establish that the class of finite mixtures of deterministic stationary policies constitutes a sufficient class of policies for uniformly absorbing MDPs with a measurable state space and multiple criteria. We also provide a sufficient condition, stated in terms of the MDP's parameters and a given occupancy measure, under which that measure can be generated by a finite mixture of deterministic policies.
The authors' abstract, as published at the source. ESAIM Control Optimisation and Calculus of Variations, 2026 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Management Information SystemsBusiness, Management and Accounting