The construction of a discontinuous near optimal feedback control law universal on a prescribed set first appeared in Krasovskii [31] and was elaborated upon in [32] and [33]; see also Krasovskii and Subbotin [34], [35]. Unlike these motivating seminal works, we take a proximal analytic approach in our constructions, in line with the results of the previous section on stabilizability.

Suppose that
is a compact set which is *weakly invariant* (or in alternate terminology, *viable* or
*holdable*);
that is, for any
there exists a control such that
for
all . Let
be continuous,
and let
be fixed. For an initial phase
,
consider the following fixed time endpoint cost optimal control problem
with state constraint :

minimize

subject to

.

Let us now add the condition

- (F3)
- The velocity set is convex for all .

Then, by standard ``sequential compactness of trajectories'' arguments, the following facts are readily verified:

*The minimum in**is attained*; we denote this minimum by and refer to as the*value function*.*The value function is lower semicontinuous**on*.

As before, by a *feedback* we simply mean any selection of
of the form
. We will
commence to sketch the method in Clarke, Rifford, and Stern
[18] for producing a feedback which generates a
near optimal trajectory which nearly satisfies the state
constraint, with respect to the -trajectory discretized
solution concept; complete details can be found in that reference.
This feedback will be operative universally for all initial phases
in a specified bounded subset of
, and it is
robust with respect to measurement and external errors. The main
idea is to adapt the arguments employed in [12] in
proving the stabilizability result given by Theorem 2.4
above to the present problem, with the value function taking over
the role played by the CLF in our prior feedback
stabilizability considerations. *Note well, however, that a
serious technical difficulty must be overcome in achieving this:*
The method in Theorem 2.4 required local Lipschitzness
of the CLF (in obtaining (2.11)), but the value function
in the present problem may not even be continuous, as was pointed
out in Remark 3.1.

We require the following notation for ``enlarged'' dynamics. For , we denote

Our result is the following.

also satisfies

Hence, it is asserted that the feedback produces a -trajectory for the enlarged dynamics (3.1), which is -optimal and which remains -near in a manner which is robust and effective universally for any initial phase in the generalized rectangle .

Without loss of generality, we shall for notational ease assume and take in the statement of Theorem 3.2.

Define a lower semicontinuous extended real valued function
as

(3.8) |

(1.1) with
, satisfies

where is the

We also denote by the augmented Hamilton obtained from the enlarged dynamics (3.1).

Given , we now define the lower semicontinuous extended
real valued function
as

A proximal calculus argument implies that satisfies the

One also has the obvious boundary conditions

For a parameter value , we denote by
the *quadratic inf-convolution* of
; that is

- is locally Lipschitz on .

The idea of using the quadratic inf-convolution in order to construct near optimal strategies goes back to Subbotin and his coworkers, where it was employed in a differential games context; see, e.g., Subbotin [51].

Since is compact in the present case, we have

Denote

and

These extrema are attained due to the compactness of , continuity of , lower semicontinuity of
, and continuity of
. The fact that the second equality involving holds for any is evident from (3.14). Note also
that

Suppose at some . Basic proximal analytic facts about the quadratic inf-convolution (see Clarke, Ledyaev, and Wolenski [17] as well as the exposition in [20]) are that

- there is a unique minimizer
in (3.14);
- is a singleton and is contained in .

In addition, we will require some elementary lemmas.

We now fix ; subsequently it is required that be taken sufficiently small. The next lemma follows easily from the previous one and (3.11).

We now introduce notation for the sublevel sets of
and

We shall also require the following lemma, which asserts how the sublevel sets of are approximated by those of its quadratic inf-convolution. (We denote the Hausdorff metric by ``haus''.)

Now fix ; we will not require the smallness of this parameter. It is
easy to see that for any and one has

Let be given. In view of the preceding lemma, it therefore follows that can be taken large enough to guarantee

This puts us in a position to adapt the general technique used in proving Theorem 2.4 to the function with chosen as above. For the given in the statement of Theorem 3.2, the parameters and are taken sufficiently small, near , and near , in such a way that further estimates lead to the required conclusion. The idea of the proof is to use (3.21) and (3.22) in order to show that achieves appropriate nonincrease while never leaving the set ; a shell based construction is employed, as described in connection with Theorem 2.4.

- When
(no state constraint) and the error functions and are both zero (no measurement or external errors), the
above result was proven in Nobakhtian and Stern [42]
without enlarging the dynamics. (Euler polygonal arcs were
employed in [42] as opposed to -trajectories here; we
need not dwell upon the distinction.) In that less general version
of Theorem 3.2, (3.3) is replaced by the one-sided
condition

As was pointed out earlier, it is the presence of the state measurement error which necessitates the lower bound in (3.3) of Theorem 3.2.Berkovitz [6] provided a method of universal feedback construction for optimal control, quite different from those mentioned above, but one which also relies upon a nonsmooth Hamilton-Jacobi approach. In the context of the present article, Berkovitz's approach can be described as follows. Since the value function of the problem is known to satisfy the generalized Hamilton-Jacobi inequality

one approach (which is known to work when is smooth) is to consider a set-valued ``feedback map'' such that

One is then led to consider the differential inclusion

It transpires that under the present hypotheses, any solution of this differential inclusion corresponds to an optimal trajectory of the optimal control problem. On the other hand, as is noted in [6], the multifunction on the right-hand-side of (3.26) in general lacks sufficient regularity (most notably, convexity, compactness, and upper semicontinuity) for the existence of solutions to hold in general, or, for that matter, for discretized solution procedures to be applicable. However, it is known (see Subbotina [53]) that under sufficient smoothness of the dynamics and cost functional , the feedback map is compact valued and upper semicontinuous, but convexity of can still fail.An approach to feedback construction related to [6] and [53] was undertaken by Cannarsa and Frankowska in [8]; in that work, additional conditions on the cost functional and dynamics were given which provide the requisite regularity in Berkovitz's original procedure, namely, smoothness of .

In Rowland and Vinter [45], a modification of Berkovitz's method is given which overcomes the lack of regularity of without imposing extra conditions. Rowland and Vinter provided a discretization procedure (but not a feedback law) which in the limit produces an optimal trajectory for any initial phase.

- If is known, then a special case of Theorem 4.8.1 of [20] (which first appeared as Theorem 10.1 in [14]) provides
a proximal aiming method for constructing a feedback, such that all its
limiting discretized (in this case, Euler polygonal arcs) solutions are
optimal (that is,
), for a
*given*initial data pair . Actually, the invariance-based proof shows that a somewhat better result holds: the feedback produces optimal limit solutions for any initial data in the set

The universality property of the feedback produced in Theorem 3.2 is an important distinction, and in a sense, the weakening of ``optimal'' to ``-optimal for any given '' in Theorem 3.2 can be viewed as the price paid for universality, albeit a small one in any practical sense. Whether this price is truly unavoidable is an open question, since we do not at present have a counterexample to the case (either for -trajectories or for limiting -trajectories). On the other hand, Subbotina [52] (see also Krasovskii and Subbotin [35]) has provided an example of a fixed duration differential game which does not possess a universal saddle point, under hypotheses which imply the existence of a saddle point for each*individual*startpoint. - In Theorem 10.2 of [14], a sufficient condition is
given for the existence of a universal -optimal feedback, in
the classical ordinary differential equations (as opposed to the discretized
or limiting discretized) solution sense. This condition requires finding a
Lipschitz semisolution to a strict Hamilton-Jacobi inequality, but with the
proximal subdifferential
replaced by the generalized
subdifferential
of Clarke, which is in general a larger
object than the -subdifferential. Because of this, the value function in
general does not satisfy this condition, so there is the difficulty of
finding an appropriate semisolution if one seeks to apply this result.
- In Clarke, Ledyaev, and Subbotin [15], a proximal
analytic method is given for constructing universal -optimal
feedback controls in differential games of pursuit, in the Krasovskii-Subbotin framework; see also [16]. This work is related
to that of Garnysheva and Subbotin [27],
[26], who constructed suboptimal discontinuous feedback
by using what they called aiming at ``quasi-gradients''; see also
Subbotin [51]. The feedbacks in [15] were
constructed with the aid of the quadratic inf-convolutions of a
not necessarily continuous proximal semisolution to a
Hamilton-Jacobi inequality; this lack of continuity is a natural
feature of the value function in time-optimal and pursuit
problems, as it is in the fixed duration state constrained control
problem considered above.
- For maximum principle based approaches to the general problem of optimal control in the presence of state constraints, see Ferreira, Fontes, and Vinter [23] as well as Vinter and Zheng [55].

Let us now posit the following additional geometric assumptions on the state constraint set :

- (S1)
- is compact.
- (S2)
- is
*wedged*. (This means that at each point , the Clarke normal cone is pointed.) - (S3)
- is
*regular*. (This is the condition that at every point in , the Clarke tangent cone agrees with the Bouligand or D-tangent cone

at each .) - (S4)
- The following ``strict inwardness'' condition holds: there
exists such that

In Clarke, Rifford, and Stern [18], the following result is proven by means of a state constrained tracking lemma.

We denote
, and for ,
we denote the *r-inner approximation* of by

Inner approximations of this type were extensively studied in Clarke, Ledyaev, and Stern [19]. In [18], the following result is given.

In other words, under the strengthened hypotheses on , for a given tolerance , if one considers any sufficiently tight inner approximation of , there exists a robust feedback effective universally for initial phases in , such that for each such initial phase, the -trajectory produced (for the original, i.e. not enlarged dynamics) is -optimal and remains in . This is in contrast to Theorem 3.2, where the -trajectory only remains -near under enlarged dynamics.

Further results,including a Hamilton-Jacobi characterization of the state constrained value, are to appear in [18].