Discrete-Time Markov Jump Linear Systems (eBook)

Preface 6
Contents 8
1 Markov Jump Linear Systems 12
1.1 Introduction 12
1.2 Some Examples 15
1.3 Problems Considered in this Book 19
1.4 Some Motivating Remarks 22
1.5 A Few Words On Our Approach 23
1.6 Historical Remarks 24
2 Background Material 26
2.1 Some Basics 26
2.2 Auxiliary Results 29
2.3 Probabilistic Space 31
2.4 Linear System Theory 32
2.5 Linear Matrix Inequalities 38
3 On Stability 40
3.1 Outline of the Chapter 40
3.2 Main Operators 41
3.3 MSS: The Homogeneous Case 47
3.4 MSS: The Non-homogeneous Case 59
3.5 Mean Square Stabilizability and Detectability 68
3.6 Stability With Probability One 74
3.7 Historical Remarks 80
4 Optimal Control 82
4.1 Outline of the Chapter 82
4.2 The Finite Horizon Quadratic Optimal Control Problem 83
4.3 In.nite Horizon Quadratic Optimal Control Problems 89
4.4 The H2-control Problem 93
4.5 Quadratic Control with Stochastic 2-input 101
4.6 Historical Remarks 110
5 Filtering 112
5.1 Outline of the Chapter 112
5.2 Finite Horizon Filtering with .(k) Known 113
5.3 Infinite Horizon Filtering with (k) Known 120
5.4 Optimal Linear Filter with .(k) Unknown 124
5.5 Robust Linear Filter with .(k) Unknown 130
5.6 Historical Remarks 139
6 Quadratic Optimal Control with Partial Information 142
6.1 Outline of the Chapter 142
6.2 Finite Horizon Case 143
6.3 Infinite Horizon Case 147
6.4 Historical Remarks 152
7 H-Control 154
7.1 Outline of the Chapter 154
7.2 The MJLS H-like Control Problem 155
7.3 Proof of Theorem 7.3 159
7.4 Recursive Algorithm for the H-control CARE 173
7.5 Historical Remarks 177
8 Design Techniques and Examples 178
8.1 Some Applications 178
8.2 Robust Control via LMI Approximations 184
8.3 Achieving Optimal H-control 199
8.4 Examples of Linear Filtering with .(k) Unknown 208
8.5 Historical Remarks 212
A Coupled Algebraic Riccati Equations 214
A.1 Duality Between the Control and Filtering CARE 214
A.2 Maximal Solution for the CARE 219
A.3 Stabilizing Solution for the CARE 227
A.4 Asymptotic Convergence 237
B Auxiliary Results for the Linear Filtering Problem with .(k) Unknown 240
B.1 Optimal Linear Filter 240
B.2 Robust Filter 247
C Auxiliary Results for the H2-control Problem 260
References 268
Notation and Conventions 282
Index 288

1 Markov Jump Linear Systems (p.1)

One of the main issues in control systems is their capability of maintaining an acceptable behavior and meeting some performance requirements even in the presence of abrupt changes in the system dynamics. These changes can be due, for instance, to abrupt environmental disturbances, component failures or repairs, changes in subsystems interconnections, abrupt changes in the operation point for a non-linear plant, etc.

Examples of these situations can be found, for instance, in economic systems, aircraft control systems, control of solar thermal central receivers, robotic manipulator systems, large flexible structures for space stations, etc. In some cases these systems can be modeled by a set of discrete-time linear systems with modal transition given by a Markov chain.

This family is known in the specialized literature as Markov jump linear systems (from now on MJLS), and will be the main topic of the present book. In this first chapter, prior to giving a rigorous mathematical treatment and present specific definitions, we will, in a rather rough and nontechnical way, state and motivate this class of dynamical systems.

1.1 Introduction
Most control systems are based on a mathematical model of the process to be controlled. This model should be able to describe with relative accuracy the process behavior, in order that a controller whose design is based on the information provided by it performs accordingly when implemented in the real process.

As pointed out by M. Kac in [148], "Models are, for the most part, caricatures of reality, but if they are good, then, like good caricatures, they portray, though perhaps in a distorted manner, some of the features of the real world."

This translates, in part, the fact that to have more representative models for real systems, we have to characterize adequately the uncertainties. Many processes may be well described, for example, by time-invariant linear models, but there are also a large number of them that are subject to uncertain changes in their dynamics, and demand a more complex approach.

If this change is an abrupt one, having only a small influence in the system behavior, classical sensitivity analysis may provide an adequate assessment of the effects.

On the other hand, when the variations caused by the changes significantly alter the dynamic behavior of the system, a stochastic model that gives a quantitative indication of the relative likelihood of various possible scenarios would be preferable.

Over the last decades, several different classes of models that take into account possible different scenarios have been proposed and studied, with more or less success. To illustrate this situation, consider a dynamical system that is, in a certain moment, well described by a model G1.

Suppose that this system is subject to abrupt changes that cause it to be described, after a certain amount of time, by a di.erent model, say G2. More generally we can imagine that the system is subject to a series of possible qualitative changes that make it switch, over time, among a countable set of models, for example, {G1, G2, . . . , GN}.