83:: that the optimal control solution in this case is the same as would be obtained in the absence of the additive disturbances. This property is applicable to all centralized systems with linear equations of evolution, quadratic cost function, and noise entering the model only additively; the quadratic assumption allows for the optimal control laws, which follow the certainty-equivalence property, to be linear functions of the observations of the controllers.
969:
In the literature, there are two types of MPCs for stochastic systems; Robust model predictive control and
Stochastic Model Predictive Control (SMPC). Robust model predictive control is a more conservative method which considers the worst scenario in the optimization procedure. However, this method,
110:
In the discrete-time case with uncertainty about the parameter values in the transition matrix (giving the effect of current values of the state variables on their own evolution) and/or the control response matrix of the state equation, but still with a linear state equation and quadratic objective
924:
matrices. But if they are so correlated, then the optimal control solution for each period contains an additional additive constant vector. If an additive constant vector appears in the state equation, then again the optimal control solution for each period contains an additional additive constant
986:
chosen at any time, the determinants of the change in wealth are usually the stochastic returns to assets and the interest rate on the risk-free asset. The field of stochastic control has developed greatly since the 1970s, particularly in its applications to finance. Robert Merton used stochastic
102:
In a discrete-time context, the decision-maker observes the state variable, possibly with observational noise, in each time period. The objective may be to optimize the sum of expected values of a nonlinear (possibly quadratic) objective function over all the time periods from the present to the
111:
function, a
Riccati equation can still be obtained for iterating backward to each period's solution even though certainty equivalence does not apply. The discrete-time case of a non-quadratic loss function but only additive disturbances can also be handled, albeit with more complications.
103:
final period of concern, or to optimize the value of the objective function as of the final period only. At each time period new observations are made, and the control variables are to be adjusted optimally. Finding the optimal solution for the present time may involve iterating a
58:
affects the evolution and observation of the state variables. Stochastic control aims to design the time path of the controlled variables that performs the desired control task with minimum cost, somehow defined, despite the presence of this noise. The context may be either
970:
similar to other robust controls, deteriorates the overall controller's performance and also is applicable only for systems with bounded uncertainties. The alternative method, SMPC, considers soft constraints which limit the risk of violation by a probabilistic inequality.
952:
If the model is in continuous time, the controller knows the state of the system at each instant of time. The objective is to maximize either an integral of, for example, a concave function of a state variable over a horizon from time zero (the present) to a terminal time
900:
627:
79:. Here the model is linear, the objective function is the expected value of a quadratic form, and the disturbances are purely additive. A basic result for discrete-time centralized systems with only additive uncertainty is the
1038:), dynamic programming is used. There is no certainty equivalence as in the older literature, because the coefficients of the control variables—that is, the returns received by the chosen shares of assets—are stochastic.
243:
354:
675:
982:
context, the state variable in the stochastic differential equation is usually wealth or net worth, and the controls are the shares placed at each time in the various assets. Given the
90:
of the model, or decentralization of control—causes the certainty equivalence property not to hold. For example, its failure to hold for decentralized control was demonstrated in
916:
The optimal control solution is unaffected if zero-mean, i.i.d. additive shocks also appear in the state equation, so long as they are uncorrelated with the parameters in the
667:
913:
matrices is the expected value and variance of each element of each matrix and the covariances among elements of the same matrix and among elements across matrices.
469:
50:
that deals with the existence of uncertainty either in observations or in the noise that drives the evolution of the system. The system designer assumes, in a
1047:
905:
which is known as the discrete-time dynamic
Riccati equation of this problem. The only information needed regarding the unknown parameters in the
1138:(1976). "Optimal Stabilization Policies for Stochastic Linear Systems: The Case of Correlated Multiplicative and Additive disturbances".
1413:
1290:
1116:
1034:
is the main tool of analysis. In the case where the maximization is an integral of a concave function of utility over an horizon (0,
104:
76:
961:. As time evolves, new observations are continuously made and the control variables are continuously adjusted in optimal fashion.
91:
1020:
1442:
992:
125:
1432:
895:{\displaystyle X_{t-1}=Q+\mathrm {E} \left-\mathrm {E} \left\left^{-1}\mathrm {E} \left(B^{\mathsf {T}}X_{t}A\right),}
277:
1140:
1204:
55:
996:
1437:
1062:
87:
31:
86:
Any deviation from the above assumptions—a nonlinear state equation, a non-quadratic objective function,
1067:
119:
A typical specification of the discrete-time stochastic linear quadratic control problem is to minimize
1175:
Mitchell, Douglas W. (1990). "Tractable Risk
Sensitive Control Based on Approximate Expected Utility".
1246:
1135:
51:
1085:
1008:
1030:, is subject to stochastic processes on the components of wealth. In this case, in continuous time
1016:
1236:
1213:
1157:
1052:
17:
1409:
1359:
1286:
1112:
1031:
988:
389:
639:
1392:
1349:
1184:
1149:
1012:
983:
460:
264:
1231:
Hashemian; Armaou (2017). "Stochastic MPC Design for a Two-Component
Granulation Process".
622:{\displaystyle u_{t}^{*}=-\left^{-1}\mathrm {E} \left(B^{\mathsf {T}}X_{t}A\right)y_{t-1},}
1004:
64:
42:
1383:(1991). "A Simplified Treatment of the Theory of Optimal Regulation of Brownian Motion".
457:
distributed through time, so the expected value operations need not be time-conditional.
1250:
1057:
253:
47:
445:) are known symmetric positive definite cost matrices. We assume that each element of
1426:
1396:
1380:
1188:
60:
1202:
Turnovsky, Stephen (1974). "The stability properties of optimal economic policies".
1104:
1280:
1354:
1337:
1026:
The maximization, say of the expected logarithm of net worth at a terminal date
1363:
944:
is characterized by removing the time subscripts from its dynamic equation.
1217:
1161:
1000:
979:
75:
An extremely well-studied formulation in stochastic control is that of
936:
goes to infinity, can be found by iterating the dynamic equation for
1153:
1241:
932:(if it exists), relevant for the infinite-horizon problem in which
1003:
literature. Influential mathematical textbook treatments were by
463:
can be used to obtain the optimal control solution at each time,
1406:
Stochastic
Controls : Hamiltonian Systems and HJB Equations
1338:"Blockchain Token Economics: A Mean-Field-Type Game Perspective"
957:, or a concave function of a state variable at some future date
454:
107:
backwards in time from the last period to the present period.
632:
with the symmetric positive definite cost-to-go matrix
1323:
Stochastic
Optimal Control and the US Financial Crisis
678:
642:
472:
280:
128:
238:{\displaystyle \mathrm {E} _{1}\sum _{t=1}^{S}\left}
1308:
Controlled Markov
Processes and Viscosity Solutions
271:is the time horizon, subject to the state equation
894:
661:
621:
348:
237:
1109:Analysis and Control of Dynamic Economic Systems
349:{\displaystyle y_{t}=A_{t}y_{t-1}+B_{t}u_{t},}
54:-driven fashion, that random noise with known
8:
1282:Deterministic and Stochastic Optimal Control
1130:
1128:
367:Ă— 1 vector of observable state variables,
1353:
1240:
1048:Backward stochastic differential equation
875:
864:
863:
849:
840:
817:
806:
805:
793:
773:
762:
761:
747:
730:
719:
718:
704:
683:
677:
647:
641:
604:
586:
575:
574:
560:
551:
526:
515:
514:
500:
482:
477:
471:
337:
327:
308:
298:
285:
279:
224:
210:
209:
204:
191:
177:
176:
171:
156:
145:
135:
130:
127:
1385:Journal of Economic Dynamics and Control
1336:Barreiro-Gomez, J.; Tembine, H. (2019).
1078:
1099:
1097:
1095:
1093:
865:
807:
763:
720:
576:
516:
211:
178:
88:noise in the multiplicative parameters
1404:Yong, Jiongmin; Zhou, Xun Yu (1999).
928:The steady-state characterization of
7:
940:repeatedly until it converges; then
1015:. These techniques were applied by
978:In a continuous time approach in a
965:Stochastic model predictive control
421:matrix of control multipliers, and
850:
794:
748:
705:
561:
501:
131:
25:
375:Ă— 1 vector of control variables,
77:linear quadratic Gaussian control
1279:Fleming, W.; Rishel, R. (1975).
636:evolving backwards in time from
1306:Fleming, W.; Soner, M. (2006).
18:Certainty equivalence principle
832:
798:
413:realization of the stochastic
81:certainty equivalence property
1:
455:independently and identically
92:Witsenhausen's counterexample
27:Probabilistic optimal control
1397:10.1016/0165-1889(91)90037-2
1189:10.1016/0264-9993(90)90018-Y
263:, superscript T indicates a
1355:10.1109/ACCESS.2019.2917517
1086:Definition from Answers.com
1021:financial crisis of 2007–08
461:Induction backwards in time
1459:
1141:Review of Economic Studies
999:changed the nature of the
991:of safe and risky assets.
29:
1205:American Economic Review
256:operator conditional on
56:probability distribution
1266:Continuous Time Finance
1264:Merton, Robert (1990).
662:{\displaystyle X_{S}=Q}
398:state transition matrix
105:matrix Riccati equation
1408:. New York: Springer.
1063:Multiplier uncertainty
896:
663:
623:
350:
239:
161:
32:Stochastic programming
1321:Stein, J. L. (2012).
1068:Stochastic scheduling
1011:, and by Fleming and
897:
664:
624:
351:
240:
141:
71:Certainty equivalence
1443:Stochastic processes
676:
640:
470:
278:
126:
52:Bayesian probability
1325:. Springer-Science.
1251:2017arXiv170404710H
1111:. New York: Wiley.
487:
388:realization of the
216:
183:
1433:Stochastic control
1177:Economic Modelling
1136:Turnovsky, Stephen
1053:Stochastic process
989:optimal portfolios
892:
659:
619:
473:
346:
235:
200:
167:
46:is a sub field of
37:Stochastic control
987:control to study
16:(Redirected from
1450:
1419:
1400:
1368:
1367:
1357:
1333:
1327:
1326:
1318:
1312:
1311:
1303:
1297:
1296:
1276:
1270:
1269:
1261:
1255:
1254:
1244:
1233:IEEE Proceedings
1228:
1222:
1221:
1199:
1193:
1192:
1172:
1166:
1165:
1132:
1123:
1122:
1105:Chow, Gregory P.
1101:
1088:
1083:
984:asset allocation
901:
899:
898:
893:
888:
884:
880:
879:
870:
869:
868:
853:
848:
847:
839:
835:
822:
821:
812:
811:
810:
797:
786:
782:
778:
777:
768:
767:
766:
751:
743:
739:
735:
734:
725:
724:
723:
708:
694:
693:
668:
666:
665:
660:
652:
651:
628:
626:
625:
620:
615:
614:
599:
595:
591:
590:
581:
580:
579:
564:
559:
558:
550:
546:
545:
541:
531:
530:
521:
520:
519:
504:
486:
481:
355:
353:
352:
347:
342:
341:
332:
331:
319:
318:
303:
302:
290:
289:
265:matrix transpose
244:
242:
241:
236:
234:
230:
229:
228:
215:
214:
208:
196:
195:
182:
181:
175:
160:
155:
140:
139:
134:
21:
1458:
1457:
1453:
1452:
1451:
1449:
1448:
1447:
1423:
1422:
1416:
1403:
1379:
1376:
1374:Further reading
1371:
1348:: 64603–64613.
1335:
1334:
1330:
1320:
1319:
1315:
1305:
1304:
1300:
1293:
1278:
1277:
1273:
1263:
1262:
1258:
1230:
1229:
1225:
1201:
1200:
1196:
1174:
1173:
1169:
1154:10.2307/2296614
1134:
1133:
1126:
1119:
1103:
1102:
1091:
1084:
1080:
1076:
1044:
976:
967:
950:
948:Continuous time
871:
859:
858:
854:
813:
801:
792:
788:
787:
769:
757:
756:
752:
726:
714:
713:
709:
679:
674:
673:
643:
638:
637:
600:
582:
570:
569:
565:
522:
510:
509:
505:
499:
495:
494:
468:
467:
408:
383:
333:
323:
304:
294:
281:
276:
275:
262:
251:
220:
187:
166:
162:
129:
124:
123:
117:
100:
73:
65:continuous time
43:optimal control
34:
28:
23:
22:
15:
12:
11:
5:
1456:
1454:
1446:
1445:
1440:
1438:Control theory
1435:
1425:
1424:
1421:
1420:
1414:
1401:
1391:(4): 657–673.
1381:Dixit, Avinash
1375:
1372:
1370:
1369:
1328:
1313:
1298:
1291:
1271:
1256:
1223:
1212:(1): 136–148.
1194:
1183:(2): 161–164.
1167:
1124:
1117:
1089:
1077:
1075:
1072:
1071:
1070:
1065:
1060:
1058:Control theory
1055:
1050:
1043:
1040:
1032:ItĂ´'s equation
975:
972:
966:
963:
949:
946:
903:
902:
891:
887:
883:
878:
874:
867:
862:
857:
852:
846:
843:
838:
834:
831:
828:
825:
820:
816:
809:
804:
800:
796:
791:
785:
781:
776:
772:
765:
760:
755:
750:
746:
742:
738:
733:
729:
722:
717:
712:
707:
703:
700:
697:
692:
689:
686:
682:
658:
655:
650:
646:
630:
629:
618:
613:
610:
607:
603:
598:
594:
589:
585:
578:
573:
568:
563:
557:
554:
549:
544:
540:
537:
534:
529:
525:
518:
513:
508:
503:
498:
493:
490:
485:
480:
476:
404:
379:
357:
356:
345:
340:
336:
330:
326:
322:
317:
314:
311:
307:
301:
297:
293:
288:
284:
260:
254:expected value
249:
246:
245:
233:
227:
223:
219:
213:
207:
203:
199:
194:
190:
186:
180:
174:
170:
165:
159:
154:
151:
148:
144:
138:
133:
116:
113:
99:
96:
72:
69:
48:control theory
26:
24:
14:
13:
10:
9:
6:
4:
3:
2:
1455:
1444:
1441:
1439:
1436:
1434:
1431:
1430:
1428:
1417:
1415:0-387-98723-1
1411:
1407:
1402:
1398:
1394:
1390:
1386:
1382:
1378:
1377:
1373:
1365:
1361:
1356:
1351:
1347:
1343:
1339:
1332:
1329:
1324:
1317:
1314:
1309:
1302:
1299:
1294:
1292:0-387-90155-8
1288:
1284:
1283:
1275:
1272:
1267:
1260:
1257:
1252:
1248:
1243:
1238:
1235:: 4386–4391.
1234:
1227:
1224:
1219:
1215:
1211:
1207:
1206:
1198:
1195:
1190:
1186:
1182:
1178:
1171:
1168:
1163:
1159:
1155:
1151:
1148:(1): 191–94.
1147:
1143:
1142:
1137:
1131:
1129:
1125:
1120:
1118:0-471-15616-7
1114:
1110:
1106:
1100:
1098:
1096:
1094:
1090:
1087:
1082:
1079:
1073:
1069:
1066:
1064:
1061:
1059:
1056:
1054:
1051:
1049:
1046:
1045:
1041:
1039:
1037:
1033:
1029:
1024:
1022:
1018:
1014:
1010:
1006:
1002:
998:
997:Black–Scholes
994:
990:
985:
981:
973:
971:
964:
962:
960:
956:
947:
945:
943:
939:
935:
931:
926:
923:
919:
914:
912:
908:
889:
885:
881:
876:
872:
860:
855:
844:
841:
836:
829:
826:
823:
818:
814:
802:
789:
783:
779:
774:
770:
758:
753:
744:
740:
736:
731:
727:
715:
710:
701:
698:
695:
690:
687:
684:
680:
672:
671:
670:
669:according to
656:
653:
648:
644:
635:
616:
611:
608:
605:
601:
596:
592:
587:
583:
571:
566:
555:
552:
547:
542:
538:
535:
532:
527:
523:
511:
506:
496:
491:
488:
483:
478:
474:
466:
465:
464:
462:
458:
456:
452:
448:
444:
440:
436:
432:
428:
424:
420:
416:
412:
407:
403:
399:
397:
393:
387:
382:
378:
374:
370:
366:
362:
343:
338:
334:
328:
324:
320:
315:
312:
309:
305:
299:
295:
291:
286:
282:
274:
273:
272:
270:
266:
259:
255:
231:
225:
221:
217:
205:
201:
197:
192:
188:
184:
172:
168:
163:
157:
152:
149:
146:
142:
136:
122:
121:
120:
114:
112:
108:
106:
98:Discrete time
97:
95:
93:
89:
84:
82:
78:
70:
68:
66:
62:
61:discrete time
57:
53:
49:
45:
44:
38:
33:
19:
1405:
1388:
1384:
1345:
1341:
1331:
1322:
1316:
1307:
1301:
1281:
1274:
1268:. Blackwell.
1265:
1259:
1232:
1226:
1209:
1203:
1197:
1180:
1176:
1170:
1145:
1139:
1108:
1081:
1035:
1027:
1025:
995:and that of
977:
968:
958:
954:
951:
941:
937:
933:
929:
927:
921:
917:
915:
910:
906:
904:
633:
631:
459:
450:
446:
442:
438:
434:
430:
426:
422:
418:
414:
410:
409:is the time
405:
401:
395:
391:
385:
384:is the time
380:
376:
372:
368:
364:
360:
358:
268:
257:
247:
118:
109:
101:
85:
80:
74:
40:
36:
35:
1342:IEEE Access
1310:. Springer.
453:is jointly
390:stochastic
41:stochastic
1427:Categories
1242:1704.04710
1074:References
974:In finance
30:See also:
1364:2169-3536
842:−
745:−
688:−
609:−
553:−
492:−
484:∗
313:−
143:∑
1107:(1976).
1042:See also
993:His work
925:vector.
1247:Bibcode
1218:1814888
1162:2296614
1019:to the
1005:Fleming
1001:finance
980:finance
252:is the
248:where E
115:Example
1412:
1362:
1289:
1216:
1160:
1115:
1009:Rishel
433:) and
363:is an
359:where
267:, and
1237:arXiv
1214:JSTOR
1158:JSTOR
1017:Stein
1013:Soner
371:is a
1410:ISBN
1360:ISSN
1287:ISBN
1113:ISBN
1007:and
920:and
909:and
449:and
1393:doi
1350:doi
1185:doi
1150:doi
63:or
39:or
1429::
1389:15
1387:.
1358:.
1344:.
1340:.
1285:.
1245:.
1210:64
1208:.
1179:.
1156:.
1146:43
1144:.
1127:^
1092:^
1023:.
441:Ă—
429:Ă—
417:Ă—
400:,
394:Ă—
94:.
67:.
1418:.
1399:.
1395::
1366:.
1352::
1346:7
1295:.
1253:.
1249::
1239::
1220:.
1191:.
1187::
1181:7
1164:.
1152::
1121:.
1036:T
1028:T
959:T
955:T
942:X
938:X
934:S
930:X
922:B
918:A
911:B
907:A
890:,
886:)
882:A
877:t
873:X
866:T
861:B
856:(
851:E
845:1
837:]
833:)
830:R
827:+
824:B
819:t
815:X
808:T
803:B
799:(
795:E
790:[
784:]
780:B
775:t
771:X
764:T
759:A
754:[
749:E
741:]
737:A
732:t
728:X
721:T
716:A
711:[
706:E
702:+
699:Q
696:=
691:1
685:t
681:X
657:Q
654:=
649:S
645:X
634:X
617:,
612:1
606:t
602:y
597:)
593:A
588:t
584:X
577:T
572:B
567:(
562:E
556:1
548:]
543:)
539:R
536:+
533:B
528:t
524:X
517:T
512:B
507:(
502:E
497:[
489:=
479:t
475:u
451:B
447:A
443:k
439:k
437:(
435:R
431:n
427:n
425:(
423:Q
419:k
415:n
411:t
406:t
402:B
396:n
392:n
386:t
381:t
377:A
373:k
369:u
365:n
361:y
344:,
339:t
335:u
329:t
325:B
321:+
316:1
310:t
306:y
300:t
296:A
292:=
287:t
283:y
269:S
261:0
258:y
250:1
232:]
226:t
222:u
218:R
212:T
206:t
202:u
198:+
193:t
189:y
185:Q
179:T
173:t
169:y
164:[
158:S
153:1
150:=
147:t
137:1
132:E
20:)
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.