342:
of the state variable's value at that time, and the resulting optimal value of the objective function is thus expressed in terms of that value of the state variable. Next, the next-to-last period's optimization involves maximizing the sum of that period's period-specific objective function and the optimal value of the future objective function, giving that period's optimal policy contingent upon the value of the state variable as of the next-to-last period decision. This logic continues recursively back in time, until the first period decision rule is derived, as a function of the initial state variable value, by optimizing the sum of the first-period-specific objective function and the value of the second period's value function, which gives the value for all the future periods. Thus, each period's decision is made by explicitly acknowledging that all future decisions will be optimally made.
3677:) for approximating the Bellman function. This is an effective mitigation strategy for reducing the impact of dimensionality by replacing the memorization of the complete function mapping for the whole space domain with the memorization of the sole neural network parameters. In particular, for continuous-time systems, an approximate dynamic programming approach that combines both policy iterations with neural networks was introduced. In discrete-time, an approach to solve the HJB equation combining value iterations and neural networks was introduced.
25:
229:. For instance, given their current wealth, people might decide how much to consume now. Choosing the control variables now may be equivalent to choosing the next state; more generally, the next state is affected by other factors in addition to the current control. For example, in the simplest case, today's wealth (the state) and consumption (the control) might exactly determine tomorrow's wealth (the new state), though typically other factors will affect tomorrow's wealth too.
82:
341:
by writing down the relationship between the value function in one period and the value function in the next period. The relationship between these two value functions is called the "Bellman equation". In this approach, the optimal policy in the last time period is specified in advance as a function
187:
Dynamic programming breaks a multi-period planning problem into simpler steps at different points in time. Therefore, it requires keeping track of how the decision situation is evolving over time. The information about the current situation that is needed to make a correct decision is called the
164:
In discrete time any multi-stage optimization problem can be solved by analyzing the appropriate
Bellman equation. The appropriate Bellman equation can be found by introducing new state variables (state augmentation). However, the resulting augmented-state multi-stage optimization problem has a
3754:
describe stochastic and nonstochastic dynamic programming in considerable detail, and develop theorems for the existence of solutions to problems meeting certain conditions. They also describe many examples of modeling theoretical problems in economics using recursive methods. This book led to
3205:
177:
To understand the
Bellman equation, several underlying concepts must be understood. First, any optimization problem has some objective: minimizing travel time, minimizing cost, maximizing profits, maximizing utility, etc. The mathematical function that describes this objective is called the
2650:
3732:). The solution to Merton's theoretical model, one in which investors chose between income today and future income or capital gains, is a form of Bellman's equation. Because economic applications of dynamic programming usually result in a Bellman equation that is a
109:. It writes the "value" of a decision problem at a certain point in time in terms of the payoff from some initial choices and the "value" of the remaining decision problem that results from those initial choices. This breaks a dynamic optimization problem into a
4236:
1139:
Principle of
Optimality: An optimal policy has the property that whatever the initial state and initial decision are, the remaining decisions must constitute an optimal policy with regard to the state resulting from the first decision. (See Bellman, 1957, Chap.
907:
3623:
4038:
3416:
3839:
arising from the vast number of possible actions and potential state variables that must be considered before an optimal strategy can be selected. For an extensive discussion of computational issues, see
Miranda and Fackler, and Meyn 2007.
113:
of simpler subproblems, as
Bellman's “principle of optimality" prescribes. The equation applies to algebraic structures with a total ordering; for algebraic structures with a partial ordering, the generic Bellman's equation can be used.
1051:
280:
Finally, by definition, the optimal decision rule is the one that achieves the best possible value of the objective. For example, if someone chooses consumption, given wealth, in order to maximize happiness (assuming happiness
169:”. Alternatively, it has been shown that if the cost function of the multi-stage optimization problem satisfies a "backward separable" structure, then the appropriate Bellman equation can be found without state augmentation.
2525:
3074:
2839:
2127:
1903:
2709:
1501:
3834:
Using dynamic programming to solve concrete problems is complicated by informational difficulties, such as choosing the unobservable discount rate. There are also computational issues, the main one being the
2004:
1608:
3696:'. Standard techniques for the solution of difference or differential equations can then be used to calculate the dynamics of the state variables and the control variables of the optimization problem.
2536:
1710:
So far it seems we have only made the problem uglier by separating today's decision from future decisions. But we can simplify by noticing that what is inside the square brackets on the right is
3033:
2996:
2448:
762:
232:
The dynamic programming approach describes the optimal plan by finding a rule that tells what the controls should be, given any possible value of the state. For example, if consumption (
476:
1774:
1700:
2267:
2371:
751:
4076:
2902:
539:
188:"state". For example, to decide how much to consume and spend at each point in time, people would need to know (among other things) their initial wealth. Therefore, wealth
3230:
1156:, although what constitutes an optimal policy in this case is conditioned on the decision-maker's opponents choosing similarly optimal policies from their points of view.
4294:
3452:
3066:
2932:
1090:
165:
higher dimensional state space than the original multi-stage optimization problem - an issue that can potentially render the augmented problem intractable due to the “
4264:
3253:
716:
621:
1638:
1188:
1121:
566:
503:
427:
380:
4061:
3878:
2319:
2207:
316:
271:
212:
3657:
on a computer. Numerical backwards induction is applicable to a wide variety of problems, but may be infeasible when there are many state variables, due to the
2956:
2411:
2391:
2339:
2287:
2178:
2154:
681:
661:
641:
586:
400:
918:
4837:
Abu-Khalaf, Murad; Lewis, Frank L. (2005). "Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach".
3886:
4872:
Al-Tamimi, Asma; Lewis, Frank L.; Abu-Khalaf, Murad (2008). "Discrete-Time
Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof".
4557:
Jones, Morgan; Peet, Matthew M. (2020). "Extensions of the
Dynamic Programming Framework: Battery Scheduling, Demand Charges, and Renewable Integration".
1702:. That new state will then affect the decision problem from time 1 on. The whole future decision problem appears inside the square brackets on the right.
4316:
3200:{\displaystyle \max _{\left\{c_{t}\right\}_{t=0}^{\infty }}\mathbb {E} {\bigg (}\sum _{t=0}^{\infty }\beta ^{t}u({\color {OliveGreen}c_{t}}){\bigg )}.}
2456:
158:
4296:
refers to the value function of the optimal policy. The equation above describes the reward for taking the action giving the highest expected return.
2009:
This is the
Bellman equation. It may be simplified even further if the time subscripts are dropped and the value of the next state is plugged in:
1092:
to denote the optimal value that can be obtained by maximizing this objective function subject to the assumed constraints. This function is the
5201:
5174:
5035:
4926:
4821:
4474:
2724:
2015:
1789:
135:
2661:
1196:
1163:, we will consider the first decision separately, setting aside all future decisions (we will start afresh from time 1 with the new state
289:
function and is something defined by wealth), then each level of wealth will be associated with some highest possible level of happiness,
46:
3635:
2851:
Now, if the interest rate varies from period to period, the consumer is faced with a stochastic optimization problem. Let the interest
1908:
1512:
5135:
5089:
5064:
5010:
4796:
4665:
4526:
4431:
4380:
4340:
68:
5212:
3440:
For a general stochastic sequential optimization problem with
Markovian shocks and where the agent is faced with their decision
3247:
is governed by a Markov process, dynamic programming simplifies the problem significantly. Then the
Bellman equation is simply:
3639:
2645:{\displaystyle {\color {Red}a_{t+1}}=(1+r)({\color {Red}a_{t}}-{\color {OliveGreen}c_{t}}),\;{\color {OliveGreen}c_{t}}\geq 0,}
273:
that gives consumption as a function of wealth. Such a rule, determining the controls as a function of the states, is called a
4349:
225:
4604:"A Generalization of Bellman's Equation with Application to Path Planning, Obstacle Avoidance and Invariant Set Estimation"
2958:. In this model the consumer decides their current period consumption after the current period interest rate is announced.
1190:). Collecting the future decisions in brackets on the right, the above infinite-horizon decision problem is equivalent to:
4305:
3729:
2714:
The first constraint is the capital accumulation/law of motion specified by the problem, while the second constraint is a
154:
5239:
2845:
2715:
3764:
3713:. Martin Beckmann also wrote extensively on consumption theory using the Bellman equation in 1959. His work influenced
3693:
902:{\displaystyle V(x_{0})\;=\;\max _{\left\{a_{t}\right\}_{t=0}^{\infty }}\sum _{t=0}^{\infty }\beta ^{t}F(x_{t},a_{t}),}
1153:
39:
33:
3001:
2964:
2416:
326:
102:
5145:
3755:
dynamic programming being employed to solve a wide range of theoretical problems in economics, including optimal
3670:
2160:. Recall that the value function describes the best possible value of the objective, as a function of the state
50:
432:
5244:
4322:
3849:
3836:
3784:
3658:
2222:
166:
2238:
For a specific example from economics, consider an infinitely-lived consumer with initial wealth endowment
2227:
In the deterministic setting, other techniques besides dynamic programming can be used to tackle the above
3650:
1717:
1643:
4447:
Szcześniak, Ireneusz; Woźna-Szcześniak, Bożena (2023), "Generic Dijkstra: Correctness and tractability",
2241:
4231:{\displaystyle V^{\pi *}(s)=\max _{a}\left\{{R(s,a)+\gamma \sum _{s'}P(s'|s,a)V^{\pi *}(s')}\right\}.\ }
3689:
3674:
149:. The term "Bellman equation" usually refers to the dynamic programming equation (DPE) associated with
4943:
4603:
2344:
724:
4733:
4334:
3780:
1145:
2862:
508:
4310:
3760:
3737:
3733:
3685:
3680:
By calculating the first-order conditions associated with the Bellman equation, and then using the
3646:
2935:
2133:
145:
106:
98:
4396:
3618:{\displaystyle V(x,z)=\max _{c\in \Gamma (x,z)}\{F(x,c,z)+\beta \int V(T(x,c),z')d\mu _{z}(z')\}.}
3213:
5234:
5115:
4983:
4897:
4854:
4633:
4615:
4584:
4566:
4480:
4452:
3654:
338:
94:
4518:
Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management
1131:
The dynamic programming method breaks this decision problem into smaller subproblems. Bellman's
4788:
4269:
3411:{\displaystyle V(a,r)=\max _{0\leq c\leq a}\{u(c)+\beta \int V((1+r)(a-c),r')Q(r,d\mu _{r})\}.}
3038:
2907:
5197:
5170:
5131:
5085:
5060:
5031:
5006:
4922:
4916:
4889:
4817:
4792:
4761:
4661:
4522:
4516:
4470:
4427:
4376:
4370:
3828:
3662:
1059:
318:. The best possible value of the objective, written as a function of the state, is called the
130:
4545:
4500:
4421:
4244:
3831:. Anderson adapted the technique to business valuation, including privately held businesses.
153:
optimization problems. In continuous-time optimization problems, the analogous equation is a
5107:
4975:
4881:
4846:
4751:
4741:
4693:
4625:
4576:
4462:
4043:
This equation describes the expected reward for taking the action prescribed by some policy
3721:
3714:
3681:
2290:
686:
591:
429:. At any time, the set of possible actions depends on the current state; we express this as
126:
1616:
1166:
1099:
544:
481:
405:
358:
5216:
5149:
4328:
4046:
3863:
3816:
3808:
3796:
3788:
3756:
3751:
3747:
3706:
3666:
2295:
2228:
2183:
719:
292:
247:
122:
3684:
to eliminate the derivatives of the value function, it is possible to obtain a system of
1046:{\displaystyle a_{t}\in \Gamma (x_{t}),\;x_{t+1}=T(x_{t},a_{t}),\;\forall t=0,1,2,\dots }
191:
121:
and to other topics in applied mathematics, and subsequently became an important tool in
4737:
4033:{\displaystyle V^{\pi }(s)=R(s,\pi (s))+\gamma \sum _{s'}P(s'|s,\pi (s))V^{\pi }(s').\ }
2718:
that the consumer does not carry debt at the end of their life. The Bellman equation is
756:
Under these assumptions, an infinite-horizon decision problem takes the following form:
5163:
5053:
4781:
4756:
4721:
3856:
for expected rewards. For example, the expected reward for being in a particular state
3824:
3792:
3768:
2941:
2856:
2396:
2376:
2324:
2272:
2163:
2139:
666:
646:
626:
571:
385:
216:
118:
2231:
problem. However, the Bellman Equation is often the most convenient method of solving
5228:
5209:
5119:
4850:
4637:
4629:
4588:
4512:
4484:
3820:
3812:
3800:
3776:
330:
180:
150:
4901:
4858:
4466:
3736:, economists refer to dynamic programming as a "recursive method" and a subfield of
2938:
governing the distribution of interest rate next period if current interest rate is
4966:
3743:
3710:
2844:
Alternatively, one can treat the sequence problem directly using, for example, the
2413:. Then the consumer's utility maximization problem is to choose a consumption plan
140:
4352: – 1957 technique for modelling problems of decision making under uncertainty
1144:
In computer science, a problem that can be broken apart like this is said to have
81:
5191:
2520:{\displaystyle \max \sum _{t=0}^{\infty }\beta ^{t}u({\color {OliveGreen}c_{t}})}
2209:
that describes the optimal action as a function of the state; this is called the
3638:, also known as 'guess and verify', can be used to solve some infinite-horizon,
1149:
4697:
4885:
3772:
3434:
4580:
4331: – Mathematical way of attaining a desired output from a dynamic system
3853:
1780:
334:
4893:
4765:
4684:
Dreyfus, S. (2002). "Richard Bellman on the birth of dynamic programming".
4325: – Mathematical model for sequential decision making under uncertainty
402:. For a decision that begins at time 0, we take as given the initial state
4964:
Merton, Robert C. (1973). "An Intertemporal Capital Asset Pricing Model".
4746:
3804:
3795:
apply dynamic programming to study a variety of theoretical questions in
3705:
The first known application of a Bellman equation in economics is due to
110:
5098:— (2009). "The Value of Private Businesses in the United States".
4874:
IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics
3421:
Under some reasonable assumption, the resulting optimal policy function
5142:
4987:
3442:
2834:{\displaystyle V(a)=\max _{0\leq c\leq a}\{u(c)+\beta V((1+r)(a-c))\},}
2341:
denotes consumption and discounts the next period utility at a rate of
2122:{\displaystyle V(x)=\max _{a\in \Gamma (x)}\{F(x,a)+\beta V(T(x,a))\}.}
1898:{\displaystyle V(x_{0})=\max _{a_{0}}\{F(x_{0},a_{0})+\beta V(x_{1})\}}
286:
5111:
4343: – economic equilibrium concept associated with a dynamic program
3232:
is taken with respect to the appropriate probability measure given by
2704:{\displaystyle \lim _{t\rightarrow \infty }{\color {Red}a_{t}}\geq 0.}
1496:{\displaystyle \max _{a_{0}}\left\{F(x_{0},a_{0})+\beta \left\right\}}
16:
Necessary condition for optimality associated with dynamic programming
223:
The variables chosen at any given point in time are often called the
125:; though the basic concepts of dynamic programming are prefigured in
4979:
4449:
NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium
2180:. By calculating the value function, we will also find the function
1123:, since the best value obtainable depends on the initial situation.
505:
represents particular values for one or more control variables, and
4944:"On the Solution to the 'Fundamental Equation' of inventory theory"
4620:
4571:
4457:
3725:
80:
3068:
in such a way that their lifetime expected utility is maximized:
1999:{\displaystyle a_{0}\in \Gamma (x_{0}),\;x_{1}=T(x_{0},a_{0}).}
1603:{\displaystyle a_{0}\in \Gamma (x_{0}),\;x_{1}=T(x_{0},a_{0}).}
18:
1640:, knowing that our choice will cause the time 1 state to be
3720:
A celebrated economic application of a Bellman equation is
5001:
Stokey, Nancy; Lucas, Robert E.; Prescott, Edward (1989).
4066:
The equation for the optimal policy is referred to as the
4319: – An optimality condition in optimal control theory
3661:. Approximate dynamic programming has been introduced by
643:
is taken, and that the current payoff from taking action
285:
can be represented by a mathematical function, such as a
2136:, because solving it means finding the unknown function
4375:(2nd ed.). Oxford University Press. p. 164.
117:
The Bellman equation was first applied to engineering
4521:(Second ed.). Amsterdam: Elsevier. p. 261.
4272:
4247:
4079:
4049:
3889:
3866:
3455:
3256:
3216:
3077:
3041:
3004:
2967:
2944:
2910:
2865:
2727:
2664:
2539:
2459:
2419:
2399:
2379:
2347:
2327:
2298:
2275:
2244:
2186:
2166:
2142:
2018:
1911:
1792:
1720:
1646:
1619:
1515:
1199:
1169:
1102:
1062:
921:
765:
727:
689:
669:
649:
629:
594:
574:
547:
541:
is the set of actions available to be taken at state
511:
484:
435:
408:
388:
361:
295:
250:
194:
4345:
Pages displaying wikidata descriptions as a fallback
1714:
of the time 1 decision problem, starting from state
4812:Bertsekas, Dimitri P.; Tsitsiklis, John N. (1996).
2393:carries over to the next period with interest rate
5162:
5052:
4780:
4288:
4258:
4230:
4055:
4032:
3872:
3827:showed the value of the method for thinking about
3617:
3410:
3224:
3199:
3060:
3027:
2990:
2950:
2926:
2896:
2833:
2703:
2644:
2519:
2442:
2405:
2385:
2365:
2333:
2313:
2281:
2261:
2201:
2172:
2148:
2121:
1998:
1897:
1768:
1694:
1632:
1602:
1495:
1182:
1115:
1084:
1045:
901:
745:
718:. Finally, we assume impatience, represented by a
710:
675:
655:
635:
615:
580:
560:
533:
497:
470:
421:
394:
374:
310:
265:
206:
3446:, the Bellman equation takes a very similar form
3189:
3128:
1096:. It is a function of the initial state variable
568:. It is also assumed that the state changes from
4106:
3478:
3279:
3079:
2744:
2666:
2460:
2035:
1816:
1266:
1201:
1152:, this principle is analogous to the concept of
791:
101:for optimality associated with the mathematical
3028:{\displaystyle \{{\color {OliveGreen}c_{t}}\}}
2991:{\displaystyle \{{\color {OliveGreen}c_{t}}\}}
2961:Rather than simply choosing a single sequence
2443:{\displaystyle \{{\color {OliveGreen}c_{t}}\}}
4779:Ljungqvist, Lars; Sargent, Thomas J. (2004).
2373:. Assume that what is not consumed in period
1779:Therefore, the problem can be rewritten as a
8:
5161:Miranda, Mario J.; Fackler, Paul L. (2004).
4651:
4649:
4647:
3609:
3508:
3402:
3300:
3055:
3042:
3022:
3005:
2985:
2968:
2825:
2765:
2437:
2420:
2113:
2059:
1892:
1832:
5165:Applied Computational Economics and Finance
4337: – Property of a computational problem
5026:Ljungqvist, Lars; Sargent, Thomas (2012).
4679:
4677:
2998:, the consumer now must choose a sequence
2618:
1947:
1551:
1470:
1415:
1012:
957:
789:
785:
173:Analytical concepts in dynamic programming
4755:
4745:
4619:
4570:
4456:
4277:
4271:
4248:
4246:
4194:
4173:
4148:
4119:
4109:
4084:
4078:
4048:
4004:
3974:
3949:
3894:
3888:
3865:
3726:intertemporal capital asset pricing model
3589:
3481:
3454:
3393:
3282:
3255:
3218:
3217:
3215:
3188:
3187:
3176:
3170:
3158:
3148:
3137:
3127:
3126:
3122:
3121:
3113:
3102:
3092:
3082:
3076:
3049:
3040:
3014:
3008:
3003:
2977:
2971:
2966:
2943:
2918:
2909:
2885:
2864:
2747:
2726:
2687:
2681:
2669:
2663:
2625:
2619:
2604:
2598:
2587:
2581:
2546:
2540:
2538:
2506:
2500:
2488:
2478:
2467:
2458:
2429:
2423:
2418:
2398:
2378:
2346:
2326:
2297:
2274:
2251:
2245:
2243:
2185:
2165:
2141:
2038:
2017:
1984:
1971:
1952:
1935:
1916:
1910:
1883:
1858:
1845:
1824:
1819:
1803:
1791:
1757:
1744:
1725:
1719:
1683:
1670:
1651:
1645:
1624:
1618:
1588:
1575:
1556:
1539:
1520:
1514:
1458:
1445:
1420:
1403:
1384:
1368:
1355:
1333:
1323:
1312:
1300:
1289:
1279:
1269:
1245:
1232:
1209:
1204:
1198:
1174:
1168:
1107:
1101:
1073:
1061:
1000:
987:
962:
945:
926:
920:
887:
874:
858:
848:
837:
825:
814:
804:
794:
776:
764:
726:
688:
668:
648:
628:
593:
573:
552:
546:
522:
510:
489:
483:
459:
440:
434:
413:
407:
387:
366:
360:
294:
249:
193:
69:Learn how and when to remove this message
5051:Dixit, Avinash; Pindyck, Robert (1994).
4942:Beckmann, Martin; Muth, Richard (1954).
4602:Jones, Morgan; Peet, Matthew M. (2021).
2132:The Bellman equation is classified as a
32:This article includes a list of general
5193:Control Techniques for Complex Networks
5080:Anderson, Patrick L. (2004). "Ch. 10".
4951:Cowles Commission Discussion Paper 2116
4423:Optimal Control Theory: An Introduction
4361:
1126:
471:{\displaystyle a_{t}\in \Gamma (x_{t})}
5003:Recursive Methods in Economic Dynamics
4722:"On the Theory of Dynamic Programming"
4559:IEEE Transactions on Automatic Control
3645:The Bellman equation can be solved by
220:, but there would probably be others.
3171:
3009:
2972:
2859:with probability transition function
2682:
2620:
2599:
2582:
2541:
2501:
2424:
2246:
1056:Notice that we have defined notation
136:Theory of Games and Economic Behavior
7:
4787:(2nd ed.). MIT Press. pp.
4541:
4496:
3740:is now recognized within economics.
1769:{\displaystyle x_{1}=T(x_{0},a_{0})}
1695:{\displaystyle x_{1}=T(x_{0},a_{0})}
4397:"Bellman's principle of optimality"
4313: – Problem optimization method
3636:method of undetermined coefficients
3035:for each possible realization of a
2262:{\displaystyle {\color {Red}a_{0}}}
4918:Economic Dynamics in Discrete Time
3488:
3149:
3114:
2676:
2479:
2045:
1925:
1783:definition of the value function:
1529:
1471:
1393:
1324:
1301:
1013:
935:
849:
826:
512:
449:
38:it lacks sufficient corresponding
14:
4341:Recursive competitive equilibrium
1127:Bellman's principle of optimality
5082:Business Economics & Finance
4851:10.1016/j.automatica.2004.11.034
4630:10.1016/j.automatica.2021.109510
4317:Hamilton–Jacobi–Bellman equation
3860:and following some fixed policy
2366:{\displaystyle 0<\beta <1}
746:{\displaystyle 0<\beta <1}
159:Hamilton–Jacobi–Bellman equation
23:
5128:Economics of Business Valuation
4467:10.1109/NOMS56928.2023.10154322
4372:Optimization in Economic Theory
3724:'s seminal 1973 article on the
5196:. Cambridge University Press.
5059:. Princeton University Press.
5028:Recursive Macroeconomic Theory
4783:Recursive Macroeconomic Theory
4350:Stochastic dynamic programming
4214:
4203:
4187:
4174:
4162:
4135:
4123:
4099:
4093:
4021:
4010:
3997:
3994:
3988:
3975:
3963:
3936:
3933:
3927:
3915:
3906:
3900:
3606:
3595:
3579:
3565:
3553:
3547:
3532:
3514:
3503:
3491:
3471:
3459:
3399:
3377:
3371:
3357:
3345:
3342:
3330:
3327:
3312:
3306:
3272:
3260:
3184:
3167:
2897:{\displaystyle Q(r,d\mu _{r})}
2891:
2869:
2822:
2819:
2807:
2804:
2792:
2789:
2777:
2771:
2737:
2731:
2673:
2612:
2578:
2575:
2563:
2514:
2497:
2308:
2302:
2196:
2190:
2110:
2107:
2095:
2089:
2077:
2065:
2054:
2048:
2028:
2022:
1990:
1964:
1941:
1928:
1905:, subject to the constraints:
1889:
1876:
1864:
1838:
1809:
1796:
1763:
1737:
1689:
1663:
1594:
1568:
1545:
1532:
1464:
1438:
1409:
1396:
1374:
1348:
1251:
1225:
1079:
1066:
1006:
980:
951:
938:
893:
867:
782:
769:
705:
693:
610:
598:
534:{\displaystyle \Gamma (x_{t})}
528:
515:
465:
452:
325:Bellman showed that a dynamic
305:
299:
260:
254:
201:
195:
1:
5130:. Stanford University Press.
4515:; Schwartz, Nancy L. (1991).
4426:. Prentice-Hall. p. 55.
4306:Bellman pseudospectral method
2289:. They have an instantaneous
337:, step-by-step form known as
155:partial differential equation
5055:Investment Under Uncertainty
5005:. Harvard University Press.
3225:{\displaystyle \mathbb {E} }
1148:. In the context of dynamic
478:, where a particular action
5208:Appendix contains abridged
5030:(3rd ed.). MIT Press.
4068:Bellman optimality equation
3653:in a few special cases, or
1506:subject to the constraints
1154:subgame perfect equilibrium
912:subject to the constraints
5261:
4921:. MIT Press. p. 134.
4720:Bellman, R (August 1952).
4698:10.1287/opre.50.1.48.17791
4369:Dixit, Avinash K. (1990).
4266:is the optimal policy and
3880:has the Bellman equation:
3852:, a Bellman equation is a
3730:Merton's portfolio problem
3671:artificial neural networks
2235:optimal control problems.
2220:
351:A dynamic decision problem
4886:10.1109/TSMCB.2008.926614
4814:Neuro-dynamic Programming
4710:Bellman, 1957, Ch. III.2.
4289:{\displaystyle V^{\pi *}}
3850:Markov decision processes
3701:Applications in economics
3061:{\displaystyle \{r_{t}\}}
2927:{\displaystyle d\mu _{r}}
1135:describes how to do this:
244:), we would seek a rule
4726:Proc Natl Acad Sci U S A
4581:10.1109/TAC.2020.3002235
4420:Kirk, Donald E. (1970).
3765:principal–agent problems
2716:transversality condition
1085:{\displaystyle V(x_{0})}
4656:Bellman, R.E. (2003) .
4323:Markov decision process
4259:{\displaystyle {\pi *}}
3837:curse of dimensionality
3785:industrial organization
3659:curse of dimensionality
2223:Markov decision process
2217:In a stochastic problem
1161:principle of optimality
1133:principle of optimality
167:curse of dimensionality
53:more precise citations.
4915:Miao, Jianjun (2014).
4329:Optimal control theory
4290:
4260:
4232:
4057:
4034:
3874:
3690:differential equations
3675:multilayer perceptrons
3619:
3412:
3226:
3201:
3153:
3062:
3029:
2992:
2952:
2928:
2898:
2835:
2705:
2646:
2521:
2483:
2444:
2407:
2387:
2367:
2335:
2315:
2283:
2263:
2203:
2174:
2150:
2123:
2000:
1899:
1770:
1696:
1634:
1604:
1497:
1328:
1184:
1142:
1117:
1086:
1047:
903:
853:
747:
712:
711:{\displaystyle F(x,a)}
677:
657:
637:
617:
616:{\displaystyle T(x,a)}
582:
562:
535:
499:
472:
423:
396:
376:
312:
267:
214:would be one of their
208:
86:
4816:. Athena Scientific.
4747:10.1073/pnas.38.8.716
4291:
4261:
4233:
4058:
4035:
3875:
3620:
3413:
3227:
3202:
3133:
3063:
3030:
2993:
2953:
2929:
2899:
2846:Hamiltonian equations
2836:
2706:
2647:
2522:
2463:
2445:
2408:
2388:
2368:
2336:
2316:
2284:
2264:
2204:
2175:
2151:
2124:
2001:
1900:
1771:
1697:
1635:
1633:{\displaystyle a_{0}}
1613:Here we are choosing
1605:
1498:
1308:
1185:
1183:{\displaystyle x_{1}}
1137:
1118:
1116:{\displaystyle x_{0}}
1087:
1048:
904:
833:
748:
713:
678:
658:
638:
618:
583:
563:
561:{\displaystyle x_{t}}
536:
500:
498:{\displaystyle a_{t}}
473:
424:
422:{\displaystyle x_{0}}
397:
382:be the state at time
377:
375:{\displaystyle x_{t}}
313:
268:
209:
84:
4335:Optimal substructure
4270:
4245:
4077:
4056:{\displaystyle \pi }
4047:
3887:
3873:{\displaystyle \pi }
3864:
3686:difference equations
3453:
3254:
3236:on the sequences of
3214:
3075:
3039:
3002:
2965:
2942:
2908:
2863:
2725:
2662:
2537:
2457:
2417:
2397:
2377:
2345:
2325:
2314:{\displaystyle u(c)}
2296:
2273:
2242:
2202:{\displaystyle a(x)}
2184:
2164:
2140:
2016:
1909:
1790:
1718:
1706:The Bellman equation
1644:
1617:
1513:
1197:
1167:
1159:As suggested by the
1146:optimal substructure
1100:
1060:
919:
763:
725:
687:
667:
647:
627:
592:
572:
545:
509:
482:
433:
406:
386:
359:
311:{\displaystyle H(W)}
293:
266:{\displaystyle c(W)}
248:
192:
5240:Dynamic programming
5190:Meyn, Sean (2008).
4738:1952PNAS...38..716B
4686:Operations Research
4658:Dynamic Programming
4311:Dynamic programming
3761:resource extraction
3738:recursive economics
3734:difference equation
3647:backwards induction
3118:
2936:probability measure
2134:functional equation
1305:
830:
333:can be stated in a
207:{\displaystyle (W)}
157:that is called the
146:sequential analysis
107:dynamic programming
99:necessary condition
85:Bellman flow chart.
5215:2007-10-12 at the
5210:Meyn & Tweedie
5148:2013-08-08 at the
5100:Business Economics
4286:
4256:
4228:
4158:
4114:
4053:
4030:
3959:
3870:
3642:Bellman equations.
3615:
3507:
3408:
3299:
3222:
3197:
3182:
3120:
3083:
3058:
3025:
3020:
2988:
2983:
2948:
2924:
2894:
2831:
2764:
2701:
2693:
2680:
2642:
2631:
2610:
2593:
2558:
2517:
2512:
2440:
2435:
2403:
2383:
2363:
2331:
2311:
2279:
2259:
2257:
2199:
2170:
2146:
2119:
2058:
1996:
1895:
1831:
1766:
1692:
1630:
1600:
1493:
1307:
1270:
1216:
1180:
1113:
1082:
1043:
899:
832:
795:
743:
708:
673:
653:
633:
613:
578:
558:
531:
495:
468:
419:
392:
372:
339:backward induction
308:
263:
204:
181:objective function
95:Richard E. Bellman
87:
5203:978-0-521-88441-9
5176:978-0-262-29175-0
5112:10.1057/be.2009.4
5037:978-0-262-01874-6
4928:978-0-262-32560-8
4823:978-1-886529-10-6
4513:Kamien, Morton I.
4476:978-1-6654-7716-1
4227:
4144:
4105:
4029:
3945:
3829:capital budgeting
3477:
3278:
3078:
2951:{\displaystyle r}
2743:
2665:
2406:{\displaystyle r}
2386:{\displaystyle t}
2334:{\displaystyle c}
2282:{\displaystyle 0}
2173:{\displaystyle x}
2149:{\displaystyle V}
2034:
1815:
1265:
1200:
790:
676:{\displaystyle x}
656:{\displaystyle a}
636:{\displaystyle a}
581:{\displaystyle x}
395:{\displaystyle t}
226:control variables
131:Oskar Morgenstern
79:
78:
71:
5252:
5220:
5207:
5187:
5181:
5180:
5168:
5158:
5152:
5141:
5126:— (2013).
5123:
5095:
5077:
5071:
5070:
5058:
5048:
5042:
5041:
5023:
5017:
5016:
4998:
4992:
4991:
4961:
4955:
4954:
4948:
4939:
4933:
4932:
4912:
4906:
4905:
4869:
4863:
4862:
4834:
4828:
4827:
4809:
4803:
4802:
4786:
4776:
4770:
4769:
4759:
4749:
4717:
4711:
4708:
4702:
4701:
4681:
4672:
4671:
4653:
4642:
4641:
4623:
4599:
4593:
4592:
4574:
4565:(4): 1602–1617.
4554:
4548:
4539:
4533:
4532:
4509:
4503:
4494:
4488:
4487:
4460:
4451:, pp. 1–7,
4444:
4438:
4437:
4417:
4411:
4410:
4408:
4407:
4393:
4387:
4386:
4366:
4346:
4295:
4293:
4292:
4287:
4285:
4284:
4265:
4263:
4262:
4257:
4255:
4237:
4235:
4234:
4229:
4225:
4221:
4217:
4213:
4202:
4201:
4177:
4172:
4157:
4156:
4113:
4092:
4091:
4062:
4060:
4059:
4054:
4039:
4037:
4036:
4031:
4027:
4020:
4009:
4008:
3978:
3973:
3958:
3957:
3899:
3898:
3879:
3877:
3876:
3871:
3722:Robert C. Merton
3717:, among others.
3715:Edmund S. Phelps
3682:envelope theorem
3669:with the use of
3667:J. N. Tsitsiklis
3629:Solution methods
3624:
3622:
3621:
3616:
3605:
3594:
3593:
3578:
3506:
3417:
3415:
3414:
3409:
3398:
3397:
3370:
3298:
3242:
3231:
3229:
3228:
3223:
3221:
3210:The expectation
3206:
3204:
3203:
3198:
3193:
3192:
3183:
3181:
3180:
3163:
3162:
3152:
3147:
3132:
3131:
3125:
3119:
3117:
3112:
3101:
3097:
3096:
3067:
3065:
3064:
3059:
3054:
3053:
3034:
3032:
3031:
3026:
3021:
3019:
3018:
2997:
2995:
2994:
2989:
2984:
2982:
2981:
2957:
2955:
2954:
2949:
2933:
2931:
2930:
2925:
2923:
2922:
2903:
2901:
2900:
2895:
2890:
2889:
2840:
2838:
2837:
2832:
2763:
2710:
2708:
2707:
2702:
2694:
2692:
2691:
2679:
2651:
2649:
2648:
2643:
2632:
2630:
2629:
2611:
2609:
2608:
2594:
2592:
2591:
2559:
2557:
2556:
2526:
2524:
2523:
2518:
2513:
2511:
2510:
2493:
2492:
2482:
2477:
2449:
2447:
2446:
2441:
2436:
2434:
2433:
2412:
2410:
2409:
2404:
2392:
2390:
2389:
2384:
2372:
2370:
2369:
2364:
2340:
2338:
2337:
2332:
2320:
2318:
2317:
2312:
2291:utility function
2288:
2286:
2285:
2280:
2268:
2266:
2265:
2260:
2258:
2256:
2255:
2208:
2206:
2205:
2200:
2179:
2177:
2176:
2171:
2155:
2153:
2152:
2147:
2128:
2126:
2125:
2120:
2057:
2005:
2003:
2002:
1997:
1989:
1988:
1976:
1975:
1957:
1956:
1940:
1939:
1921:
1920:
1904:
1902:
1901:
1896:
1888:
1887:
1863:
1862:
1850:
1849:
1830:
1829:
1828:
1808:
1807:
1775:
1773:
1772:
1767:
1762:
1761:
1749:
1748:
1730:
1729:
1701:
1699:
1698:
1693:
1688:
1687:
1675:
1674:
1656:
1655:
1639:
1637:
1636:
1631:
1629:
1628:
1609:
1607:
1606:
1601:
1593:
1592:
1580:
1579:
1561:
1560:
1544:
1543:
1525:
1524:
1502:
1500:
1499:
1494:
1492:
1488:
1487:
1483:
1463:
1462:
1450:
1449:
1431:
1430:
1408:
1407:
1389:
1388:
1373:
1372:
1360:
1359:
1344:
1343:
1327:
1322:
1306:
1304:
1299:
1288:
1284:
1283:
1250:
1249:
1237:
1236:
1215:
1214:
1213:
1189:
1187:
1186:
1181:
1179:
1178:
1122:
1120:
1119:
1114:
1112:
1111:
1091:
1089:
1088:
1083:
1078:
1077:
1052:
1050:
1049:
1044:
1005:
1004:
992:
991:
973:
972:
950:
949:
931:
930:
908:
906:
905:
900:
892:
891:
879:
878:
863:
862:
852:
847:
831:
829:
824:
813:
809:
808:
781:
780:
752:
750:
749:
744:
717:
715:
714:
709:
682:
680:
679:
674:
662:
660:
659:
654:
642:
640:
639:
634:
622:
620:
619:
614:
587:
585:
584:
579:
567:
565:
564:
559:
557:
556:
540:
538:
537:
532:
527:
526:
504:
502:
501:
496:
494:
493:
477:
475:
474:
469:
464:
463:
445:
444:
428:
426:
425:
420:
418:
417:
401:
399:
398:
393:
381:
379:
378:
373:
371:
370:
317:
315:
314:
309:
272:
270:
269:
264:
213:
211:
210:
205:
127:John von Neumann
105:method known as
91:Bellman equation
74:
67:
63:
60:
54:
49:this article by
40:inline citations
27:
26:
19:
5260:
5259:
5255:
5254:
5253:
5251:
5250:
5249:
5225:
5224:
5223:
5217:Wayback Machine
5204:
5189:
5188:
5184:
5177:
5160:
5159:
5155:
5150:Wayback Machine
5138:
5125:
5124:
5097:
5096:
5092:
5079:
5078:
5074:
5067:
5050:
5049:
5045:
5038:
5025:
5024:
5020:
5013:
5000:
4999:
4995:
4980:10.2307/1913811
4963:
4962:
4958:
4946:
4941:
4940:
4936:
4929:
4914:
4913:
4909:
4871:
4870:
4866:
4836:
4835:
4831:
4824:
4811:
4810:
4806:
4799:
4778:
4777:
4773:
4719:
4718:
4714:
4709:
4705:
4683:
4682:
4675:
4668:
4655:
4654:
4645:
4601:
4600:
4596:
4556:
4555:
4551:
4540:
4536:
4529:
4511:
4510:
4506:
4495:
4491:
4477:
4446:
4445:
4441:
4434:
4419:
4418:
4414:
4405:
4403:
4395:
4394:
4390:
4383:
4368:
4367:
4363:
4359:
4344:
4302:
4273:
4268:
4267:
4243:
4242:
4206:
4190:
4165:
4149:
4115:
4080:
4075:
4074:
4045:
4044:
4013:
4000:
3966:
3950:
3890:
3885:
3884:
3862:
3861:
3846:
3817:labor economics
3809:economic growth
3797:monetary policy
3789:Lars Ljungqvist
3757:economic growth
3752:Edward Prescott
3748:Robert E. Lucas
3707:Martin Beckmann
3703:
3694:Euler equations
3663:D. P. Bertsekas
3631:
3598:
3585:
3571:
3451:
3450:
3389:
3363:
3252:
3251:
3240:
3212:
3211:
3172:
3154:
3088:
3084:
3073:
3072:
3045:
3037:
3036:
3010:
3000:
2999:
2973:
2963:
2962:
2940:
2939:
2914:
2906:
2905:
2881:
2861:
2860:
2723:
2722:
2683:
2660:
2659:
2621:
2600:
2583:
2542:
2535:
2534:
2502:
2484:
2455:
2454:
2425:
2415:
2414:
2395:
2394:
2375:
2374:
2343:
2342:
2323:
2322:
2294:
2293:
2271:
2270:
2247:
2240:
2239:
2229:optimal control
2225:
2219:
2211:policy function
2182:
2181:
2162:
2161:
2156:, which is the
2138:
2137:
2014:
2013:
1980:
1967:
1948:
1931:
1912:
1907:
1906:
1879:
1854:
1841:
1820:
1799:
1788:
1787:
1753:
1740:
1721:
1716:
1715:
1708:
1679:
1666:
1647:
1642:
1641:
1620:
1615:
1614:
1584:
1571:
1552:
1535:
1516:
1511:
1510:
1454:
1441:
1416:
1399:
1380:
1364:
1351:
1329:
1275:
1271:
1264:
1260:
1241:
1228:
1221:
1217:
1205:
1195:
1194:
1170:
1165:
1164:
1129:
1103:
1098:
1097:
1069:
1058:
1057:
996:
983:
958:
941:
922:
917:
916:
883:
870:
854:
800:
796:
772:
761:
760:
723:
722:
720:discount factor
685:
684:
665:
664:
645:
644:
625:
624:
590:
589:
588:to a new state
570:
569:
548:
543:
542:
518:
507:
506:
485:
480:
479:
455:
436:
431:
430:
409:
404:
403:
384:
383:
362:
357:
356:
353:
348:
291:
290:
275:policy function
246:
245:
217:state variables
190:
189:
175:
123:economic theory
75:
64:
58:
55:
45:Please help to
44:
28:
24:
17:
12:
11:
5:
5258:
5256:
5248:
5247:
5245:Control theory
5242:
5237:
5227:
5226:
5222:
5221:
5202:
5182:
5175:
5153:
5143:Stanford Press
5136:
5090:
5072:
5065:
5043:
5036:
5018:
5011:
4993:
4974:(5): 867–887.
4956:
4934:
4927:
4907:
4880:(4): 943–949.
4864:
4845:(5): 779–791.
4829:
4822:
4804:
4797:
4771:
4712:
4703:
4673:
4666:
4643:
4594:
4549:
4534:
4527:
4504:
4489:
4475:
4439:
4432:
4412:
4401:www.ques10.com
4388:
4381:
4360:
4358:
4355:
4354:
4353:
4347:
4338:
4332:
4326:
4320:
4314:
4308:
4301:
4298:
4283:
4280:
4276:
4254:
4251:
4239:
4238:
4224:
4220:
4216:
4212:
4209:
4205:
4200:
4197:
4193:
4189:
4186:
4183:
4180:
4176:
4171:
4168:
4164:
4161:
4155:
4152:
4147:
4143:
4140:
4137:
4134:
4131:
4128:
4125:
4122:
4118:
4112:
4108:
4104:
4101:
4098:
4095:
4090:
4087:
4083:
4052:
4041:
4040:
4026:
4023:
4019:
4016:
4012:
4007:
4003:
3999:
3996:
3993:
3990:
3987:
3984:
3981:
3977:
3972:
3969:
3965:
3962:
3956:
3953:
3948:
3944:
3941:
3938:
3935:
3932:
3929:
3926:
3923:
3920:
3917:
3914:
3911:
3908:
3905:
3902:
3897:
3893:
3869:
3845:
3842:
3825:Robert Pindyck
3793:Thomas Sargent
3769:public finance
3702:
3699:
3698:
3697:
3678:
3643:
3630:
3627:
3626:
3625:
3614:
3611:
3608:
3604:
3601:
3597:
3592:
3588:
3584:
3581:
3577:
3574:
3570:
3567:
3564:
3561:
3558:
3555:
3552:
3549:
3546:
3543:
3540:
3537:
3534:
3531:
3528:
3525:
3522:
3519:
3516:
3513:
3510:
3505:
3502:
3499:
3496:
3493:
3490:
3487:
3484:
3480:
3476:
3473:
3470:
3467:
3464:
3461:
3458:
3419:
3418:
3407:
3404:
3401:
3396:
3392:
3388:
3385:
3382:
3379:
3376:
3373:
3369:
3366:
3362:
3359:
3356:
3353:
3350:
3347:
3344:
3341:
3338:
3335:
3332:
3329:
3326:
3323:
3320:
3317:
3314:
3311:
3308:
3305:
3302:
3297:
3294:
3291:
3288:
3285:
3281:
3277:
3274:
3271:
3268:
3265:
3262:
3259:
3220:
3208:
3207:
3196:
3191:
3186:
3179:
3175:
3169:
3166:
3161:
3157:
3151:
3146:
3143:
3140:
3136:
3130:
3124:
3116:
3111:
3108:
3105:
3100:
3095:
3091:
3087:
3081:
3057:
3052:
3048:
3044:
3024:
3017:
3013:
3007:
2987:
2980:
2976:
2970:
2947:
2921:
2917:
2913:
2893:
2888:
2884:
2880:
2877:
2874:
2871:
2868:
2857:Markov process
2842:
2841:
2830:
2827:
2824:
2821:
2818:
2815:
2812:
2809:
2806:
2803:
2800:
2797:
2794:
2791:
2788:
2785:
2782:
2779:
2776:
2773:
2770:
2767:
2762:
2759:
2756:
2753:
2750:
2746:
2742:
2739:
2736:
2733:
2730:
2712:
2711:
2700:
2697:
2690:
2686:
2678:
2675:
2672:
2668:
2653:
2652:
2641:
2638:
2635:
2628:
2624:
2617:
2614:
2607:
2603:
2597:
2590:
2586:
2580:
2577:
2574:
2571:
2568:
2565:
2562:
2555:
2552:
2549:
2545:
2528:
2527:
2516:
2509:
2505:
2499:
2496:
2491:
2487:
2481:
2476:
2473:
2470:
2466:
2462:
2439:
2432:
2428:
2422:
2402:
2382:
2362:
2359:
2356:
2353:
2350:
2330:
2310:
2307:
2304:
2301:
2278:
2254:
2250:
2218:
2215:
2198:
2195:
2192:
2189:
2169:
2158:value function
2145:
2130:
2129:
2118:
2115:
2112:
2109:
2106:
2103:
2100:
2097:
2094:
2091:
2088:
2085:
2082:
2079:
2076:
2073:
2070:
2067:
2064:
2061:
2056:
2053:
2050:
2047:
2044:
2041:
2037:
2033:
2030:
2027:
2024:
2021:
2007:
2006:
1995:
1992:
1987:
1983:
1979:
1974:
1970:
1966:
1963:
1960:
1955:
1951:
1946:
1943:
1938:
1934:
1930:
1927:
1924:
1919:
1915:
1894:
1891:
1886:
1882:
1878:
1875:
1872:
1869:
1866:
1861:
1857:
1853:
1848:
1844:
1840:
1837:
1834:
1827:
1823:
1818:
1814:
1811:
1806:
1802:
1798:
1795:
1765:
1760:
1756:
1752:
1747:
1743:
1739:
1736:
1733:
1728:
1724:
1707:
1704:
1691:
1686:
1682:
1678:
1673:
1669:
1665:
1662:
1659:
1654:
1650:
1627:
1623:
1611:
1610:
1599:
1596:
1591:
1587:
1583:
1578:
1574:
1570:
1567:
1564:
1559:
1555:
1550:
1547:
1542:
1538:
1534:
1531:
1528:
1523:
1519:
1504:
1503:
1491:
1486:
1482:
1479:
1476:
1473:
1469:
1466:
1461:
1457:
1453:
1448:
1444:
1440:
1437:
1434:
1429:
1426:
1423:
1419:
1414:
1411:
1406:
1402:
1398:
1395:
1392:
1387:
1383:
1379:
1376:
1371:
1367:
1363:
1358:
1354:
1350:
1347:
1342:
1339:
1336:
1332:
1326:
1321:
1318:
1315:
1311:
1303:
1298:
1295:
1292:
1287:
1282:
1278:
1274:
1268:
1263:
1259:
1256:
1253:
1248:
1244:
1240:
1235:
1231:
1227:
1224:
1220:
1212:
1208:
1203:
1177:
1173:
1128:
1125:
1110:
1106:
1094:value function
1081:
1076:
1072:
1068:
1065:
1054:
1053:
1042:
1039:
1036:
1033:
1030:
1027:
1024:
1021:
1018:
1015:
1011:
1008:
1003:
999:
995:
990:
986:
982:
979:
976:
971:
968:
965:
961:
956:
953:
948:
944:
940:
937:
934:
929:
925:
910:
909:
898:
895:
890:
886:
882:
877:
873:
869:
866:
861:
857:
851:
846:
843:
840:
836:
828:
823:
820:
817:
812:
807:
803:
799:
793:
788:
784:
779:
775:
771:
768:
742:
739:
736:
733:
730:
707:
704:
701:
698:
695:
692:
672:
652:
632:
612:
609:
606:
603:
600:
597:
577:
555:
551:
530:
525:
521:
517:
514:
492:
488:
467:
462:
458:
454:
451:
448:
443:
439:
416:
412:
391:
369:
365:
352:
349:
347:
344:
320:value function
307:
304:
301:
298:
262:
259:
256:
253:
203:
200:
197:
174:
171:
119:control theory
93:, named after
77:
76:
31:
29:
22:
15:
13:
10:
9:
6:
4:
3:
2:
5257:
5246:
5243:
5241:
5238:
5236:
5233:
5232:
5230:
5218:
5214:
5211:
5205:
5199:
5195:
5194:
5186:
5183:
5178:
5172:
5169:. MIT Press.
5167:
5166:
5157:
5154:
5151:
5147:
5144:
5139:
5137:9780804758307
5133:
5129:
5121:
5117:
5113:
5109:
5106:(2): 87–108.
5105:
5101:
5093:
5091:1-58488-348-0
5087:
5084:. CRC Press.
5083:
5076:
5073:
5068:
5066:0-691-03410-9
5062:
5057:
5056:
5047:
5044:
5039:
5033:
5029:
5022:
5019:
5014:
5012:0-674-75096-9
5008:
5004:
4997:
4994:
4989:
4985:
4981:
4977:
4973:
4969:
4968:
4960:
4957:
4952:
4945:
4938:
4935:
4930:
4924:
4920:
4919:
4911:
4908:
4903:
4899:
4895:
4891:
4887:
4883:
4879:
4875:
4868:
4865:
4860:
4856:
4852:
4848:
4844:
4840:
4833:
4830:
4825:
4819:
4815:
4808:
4805:
4800:
4798:0-262-12274-X
4794:
4790:
4785:
4784:
4775:
4772:
4767:
4763:
4758:
4753:
4748:
4743:
4739:
4735:
4731:
4727:
4723:
4716:
4713:
4707:
4704:
4699:
4695:
4691:
4687:
4680:
4678:
4674:
4669:
4667:0-486-42809-5
4663:
4659:
4652:
4650:
4648:
4644:
4639:
4635:
4631:
4627:
4622:
4617:
4613:
4609:
4605:
4598:
4595:
4590:
4586:
4582:
4578:
4573:
4568:
4564:
4560:
4553:
4550:
4547:
4543:
4538:
4535:
4530:
4528:0-444-01609-0
4524:
4520:
4519:
4514:
4508:
4505:
4502:
4498:
4493:
4490:
4486:
4482:
4478:
4472:
4468:
4464:
4459:
4454:
4450:
4443:
4440:
4435:
4433:0-13-638098-0
4429:
4425:
4424:
4416:
4413:
4402:
4398:
4392:
4389:
4384:
4382:0-19-877211-4
4378:
4374:
4373:
4365:
4362:
4356:
4351:
4348:
4342:
4339:
4336:
4333:
4330:
4327:
4324:
4321:
4318:
4315:
4312:
4309:
4307:
4304:
4303:
4299:
4297:
4281:
4278:
4274:
4252:
4249:
4222:
4218:
4210:
4207:
4198:
4195:
4191:
4184:
4181:
4178:
4169:
4166:
4159:
4153:
4150:
4145:
4141:
4138:
4132:
4129:
4126:
4120:
4116:
4110:
4102:
4096:
4088:
4085:
4081:
4073:
4072:
4071:
4069:
4064:
4050:
4024:
4017:
4014:
4005:
4001:
3991:
3985:
3982:
3979:
3970:
3967:
3960:
3954:
3951:
3946:
3942:
3939:
3930:
3924:
3921:
3918:
3912:
3909:
3903:
3895:
3891:
3883:
3882:
3881:
3867:
3859:
3855:
3851:
3843:
3841:
3838:
3832:
3830:
3826:
3822:
3821:Avinash Dixit
3818:
3814:
3813:search theory
3810:
3806:
3802:
3801:fiscal policy
3798:
3794:
3790:
3786:
3782:
3778:
3777:asset pricing
3774:
3770:
3766:
3762:
3758:
3753:
3749:
3745:
3741:
3739:
3735:
3731:
3727:
3723:
3718:
3716:
3712:
3708:
3700:
3695:
3691:
3687:
3683:
3679:
3676:
3672:
3668:
3664:
3660:
3656:
3652:
3648:
3644:
3641:
3637:
3633:
3632:
3628:
3612:
3602:
3599:
3590:
3586:
3582:
3575:
3572:
3568:
3562:
3559:
3556:
3550:
3544:
3541:
3538:
3535:
3529:
3526:
3523:
3520:
3517:
3511:
3500:
3497:
3494:
3485:
3482:
3474:
3468:
3465:
3462:
3456:
3449:
3448:
3447:
3445:
3444:
3438:
3436:
3432:
3428:
3424:
3405:
3394:
3390:
3386:
3383:
3380:
3374:
3367:
3364:
3360:
3354:
3351:
3348:
3339:
3336:
3333:
3324:
3321:
3318:
3315:
3309:
3303:
3295:
3292:
3289:
3286:
3283:
3275:
3269:
3266:
3263:
3257:
3250:
3249:
3248:
3246:
3239:
3235:
3194:
3177:
3173:
3164:
3159:
3155:
3144:
3141:
3138:
3134:
3109:
3106:
3103:
3098:
3093:
3089:
3085:
3071:
3070:
3069:
3050:
3046:
3015:
3011:
2978:
2974:
2959:
2945:
2937:
2919:
2915:
2911:
2886:
2882:
2878:
2875:
2872:
2866:
2858:
2854:
2849:
2847:
2828:
2816:
2813:
2810:
2801:
2798:
2795:
2786:
2783:
2780:
2774:
2768:
2760:
2757:
2754:
2751:
2748:
2740:
2734:
2728:
2721:
2720:
2719:
2717:
2698:
2695:
2688:
2684:
2670:
2658:
2657:
2656:
2639:
2636:
2633:
2626:
2622:
2615:
2605:
2601:
2595:
2588:
2584:
2572:
2569:
2566:
2560:
2553:
2550:
2547:
2543:
2533:
2532:
2531:
2507:
2503:
2494:
2489:
2485:
2474:
2471:
2468:
2464:
2453:
2452:
2451:
2430:
2426:
2400:
2380:
2360:
2357:
2354:
2351:
2348:
2328:
2305:
2299:
2292:
2276:
2252:
2248:
2236:
2234:
2230:
2224:
2216:
2214:
2212:
2193:
2187:
2167:
2159:
2143:
2135:
2116:
2104:
2101:
2098:
2092:
2086:
2083:
2080:
2074:
2071:
2068:
2062:
2051:
2042:
2039:
2031:
2025:
2019:
2012:
2011:
2010:
1993:
1985:
1981:
1977:
1972:
1968:
1961:
1958:
1953:
1949:
1944:
1936:
1932:
1922:
1917:
1913:
1884:
1880:
1873:
1870:
1867:
1859:
1855:
1851:
1846:
1842:
1835:
1825:
1821:
1812:
1804:
1800:
1793:
1786:
1785:
1784:
1782:
1777:
1758:
1754:
1750:
1745:
1741:
1734:
1731:
1726:
1722:
1713:
1705:
1703:
1684:
1680:
1676:
1671:
1667:
1660:
1657:
1652:
1648:
1625:
1621:
1597:
1589:
1585:
1581:
1576:
1572:
1565:
1562:
1557:
1553:
1548:
1540:
1536:
1526:
1521:
1517:
1509:
1508:
1507:
1489:
1484:
1480:
1477:
1474:
1467:
1459:
1455:
1451:
1446:
1442:
1435:
1432:
1427:
1424:
1421:
1417:
1412:
1404:
1400:
1390:
1385:
1381:
1377:
1369:
1365:
1361:
1356:
1352:
1345:
1340:
1337:
1334:
1330:
1319:
1316:
1313:
1309:
1296:
1293:
1290:
1285:
1280:
1276:
1272:
1261:
1257:
1254:
1246:
1242:
1238:
1233:
1229:
1222:
1218:
1210:
1206:
1193:
1192:
1191:
1175:
1171:
1162:
1157:
1155:
1151:
1147:
1141:
1136:
1134:
1124:
1108:
1104:
1095:
1074:
1070:
1063:
1040:
1037:
1034:
1031:
1028:
1025:
1022:
1019:
1016:
1009:
1001:
997:
993:
988:
984:
977:
974:
969:
966:
963:
959:
954:
946:
942:
932:
927:
923:
915:
914:
913:
896:
888:
884:
880:
875:
871:
864:
859:
855:
844:
841:
838:
834:
821:
818:
815:
810:
805:
801:
797:
786:
777:
773:
766:
759:
758:
757:
754:
740:
737:
734:
731:
728:
721:
702:
699:
696:
690:
670:
650:
630:
607:
604:
601:
595:
575:
553:
549:
523:
519:
490:
486:
460:
456:
446:
441:
437:
414:
410:
389:
367:
363:
350:
345:
343:
340:
336:
332:
331:discrete time
328:
323:
321:
302:
296:
288:
284:
278:
276:
257:
251:
243:
239:
235:
230:
228:
227:
221:
219:
218:
198:
185:
183:
182:
172:
170:
168:
162:
160:
156:
152:
151:discrete-time
148:
147:
142:
138:
137:
132:
128:
124:
120:
115:
112:
108:
104:
100:
96:
92:
83:
73:
70:
62:
52:
48:
42:
41:
35:
30:
21:
20:
5192:
5185:
5164:
5156:
5127:
5103:
5099:
5081:
5075:
5054:
5046:
5027:
5021:
5002:
4996:
4971:
4967:Econometrica
4965:
4959:
4950:
4937:
4917:
4910:
4877:
4873:
4867:
4842:
4838:
4832:
4813:
4807:
4782:
4774:
4732:(8): 716–9.
4729:
4725:
4715:
4706:
4692:(1): 48–51.
4689:
4685:
4657:
4611:
4607:
4597:
4562:
4558:
4552:
4537:
4517:
4507:
4492:
4448:
4442:
4422:
4415:
4404:. Retrieved
4400:
4391:
4371:
4364:
4240:
4067:
4065:
4042:
3857:
3847:
3833:
3783:supply, and
3744:Nancy Stokey
3742:
3728:. (See also
3719:
3711:Richard Muth
3704:
3692:called the '
3651:analytically
3441:
3439:
3430:
3426:
3422:
3420:
3244:
3237:
3233:
3209:
2960:
2934:denotes the
2852:
2850:
2843:
2713:
2654:
2529:
2450:that solves
2237:
2232:
2226:
2210:
2157:
2131:
2008:
1778:
1711:
1709:
1612:
1505:
1160:
1158:
1143:
1138:
1132:
1130:
1093:
1055:
911:
755:
623:when action
354:
327:optimization
324:
319:
282:
279:
274:
241:
237:
233:
231:
224:
222:
215:
186:
179:
176:
163:
144:
141:Abraham Wald
134:
116:
103:optimization
90:
88:
65:
56:
37:
3771:, business
3655:numerically
3243:s. Because
2530:subject to
1150:game theory
329:problem in
240:on wealth (
51:introducing
5229:Categories
4839:Automatica
4621:2006.08175
4614:: 109510.
4608:Automatica
4572:1812.00792
4544:, p.
4499:, p.
4458:2204.13547
4406:2023-08-17
4357:References
3773:investment
3640:autonomous
3435:measurable
2269:at period
2233:stochastic
2221:See also:
346:Derivation
236:) depends
59:April 2018
34:references
5235:Equations
5120:154743445
4660:. Dover.
4638:222350370
4589:119622206
4542:Kirk 1970
4497:Kirk 1970
4485:248427020
4282:∗
4279:π
4253:∗
4250:π
4199:∗
4196:π
4146:∑
4142:γ
4089:∗
4086:π
4051:π
4006:π
3986:π
3947:∑
3943:γ
3925:π
3896:π
3868:π
3854:recursion
3649:, either
3587:μ
3542:∫
3539:β
3489:Γ
3486:∈
3391:μ
3352:−
3322:∫
3319:β
3293:≤
3287:≤
3156:β
3150:∞
3135:∑
3115:∞
2916:μ
2883:μ
2855:follow a
2814:−
2784:β
2758:≤
2752:≤
2696:≥
2677:∞
2674:→
2634:≥
2596:−
2486:β
2480:∞
2465:∑
2355:β
2084:β
2046:Γ
2043:∈
1926:Γ
1923:∈
1871:β
1781:recursive
1712:the value
1530:Γ
1527:∈
1478:≥
1472:∀
1394:Γ
1391:∈
1338:−
1331:β
1325:∞
1310:∑
1302:∞
1258:β
1041:…
1014:∀
936:Γ
933:∈
856:β
850:∞
835:∑
827:∞
735:β
663:in state
513:Γ
450:Γ
447:∈
335:recursive
5213:Archived
5146:Archived
4902:14202785
4894:18632382
4859:14757582
4766:16589166
4300:See also
4211:′
4170:′
4154:′
4018:′
3971:′
3955:′
3805:taxation
3603:′
3576:′
3368:′
111:sequence
4988:1913811
4757:1063639
4734:Bibcode
3844:Example
3443:ex-post
1140:III.3.)
287:utility
97:, is a
47:improve
5200:
5173:
5134:
5118:
5088:
5063:
5034:
5009:
4986:
4925:
4900:
4892:
4857:
4820:
4795:
4764:
4754:
4664:
4636:
4587:
4525:
4483:
4473:
4430:
4379:
4241:where
4226:
4028:
3815:, and
3781:factor
3750:, and
2904:where
2321:where
36:, but
5116:S2CID
4984:JSTOR
4947:(PDF)
4898:S2CID
4855:S2CID
4791:–90.
4634:S2CID
4616:arXiv
4585:S2CID
4567:arXiv
4481:S2CID
4453:arXiv
3433:) is
3241:'
5198:ISBN
5171:ISBN
5132:ISBN
5086:ISBN
5061:ISBN
5032:ISBN
5007:ISBN
4923:ISBN
4890:PMID
4818:ISBN
4793:ISBN
4762:PMID
4662:ISBN
4523:ISBN
4471:ISBN
4428:ISBN
4377:ISBN
3823:and
3791:and
3709:and
3665:and
3634:The
2655:and
2358:<
2352:<
738:<
732:<
355:Let
238:only
139:and
129:and
5108:doi
4976:doi
4882:doi
4847:doi
4752:PMC
4742:doi
4694:doi
4626:doi
4612:127
4577:doi
4463:doi
4107:max
3848:In
3688:or
3479:max
3280:max
3080:max
2745:max
2667:lim
2461:max
2036:max
1817:max
1267:max
1202:max
792:max
683:is
143:'s
133:'s
5231::
5114:.
5104:44
5102:.
4982:.
4972:41
4970:.
4949:.
4896:.
4888:.
4878:38
4876:.
4853:.
4843:41
4841:.
4789:88
4760:.
4750:.
4740:.
4730:38
4728:.
4724:.
4690:50
4688:.
4676:^
4646:^
4632:.
4624:.
4610:.
4606:.
4583:.
4575:.
4563:66
4561:.
4546:88
4501:70
4479:,
4469:,
4461:,
4399:.
4070::
4063:.
3819:.
3811:,
3807:,
3803:,
3799:,
3787:.
3779:,
3775:,
3767:,
3763:,
3759:,
3746:,
3437:.
2848:.
2699:0.
2213:.
1776:.
753:.
322:.
277:.
184:.
161:.
89:A
5219:.
5206:.
5179:.
5140:.
5122:.
5110::
5094:.
5069:.
5040:.
5015:.
4990:.
4978::
4953:.
4931:.
4904:.
4884::
4861:.
4849::
4826:.
4801:.
4768:.
4744::
4736::
4700:.
4696::
4670:.
4640:.
4628::
4618::
4591:.
4579::
4569::
4531:.
4465::
4455::
4436:.
4409:.
4385:.
4275:V
4223:.
4219:}
4215:)
4208:s
4204:(
4192:V
4188:)
4185:a
4182:,
4179:s
4175:|
4167:s
4163:(
4160:P
4151:s
4139:+
4136:)
4133:a
4130:,
4127:s
4124:(
4121:R
4117:{
4111:a
4103:=
4100:)
4097:s
4094:(
4082:V
4025:.
4022:)
4015:s
4011:(
4002:V
3998:)
3995:)
3992:s
3989:(
3983:,
3980:s
3976:|
3968:s
3964:(
3961:P
3952:s
3940:+
3937:)
3934:)
3931:s
3928:(
3922:,
3919:s
3916:(
3913:R
3910:=
3907:)
3904:s
3901:(
3892:V
3858:s
3673:(
3613:.
3610:}
3607:)
3600:z
3596:(
3591:z
3583:d
3580:)
3573:z
3569:,
3566:)
3563:c
3560:,
3557:x
3554:(
3551:T
3548:(
3545:V
3536:+
3533:)
3530:z
3527:,
3524:c
3521:,
3518:x
3515:(
3512:F
3509:{
3504:)
3501:z
3498:,
3495:x
3492:(
3483:c
3475:=
3472:)
3469:z
3466:,
3463:x
3460:(
3457:V
3431:r
3429:,
3427:a
3425:(
3423:g
3406:.
3403:}
3400:)
3395:r
3387:d
3384:,
3381:r
3378:(
3375:Q
3372:)
3365:r
3361:,
3358:)
3355:c
3349:a
3346:(
3343:)
3340:r
3337:+
3334:1
3331:(
3328:(
3325:V
3316:+
3313:)
3310:c
3307:(
3304:u
3301:{
3296:a
3290:c
3284:0
3276:=
3273:)
3270:r
3267:,
3264:a
3261:(
3258:V
3245:r
3238:r
3234:Q
3219:E
3195:.
3190:)
3185:)
3178:t
3174:c
3168:(
3165:u
3160:t
3145:0
3142:=
3139:t
3129:(
3123:E
3110:0
3107:=
3104:t
3099:}
3094:t
3090:c
3086:{
3056:}
3051:t
3047:r
3043:{
3023:}
3016:t
3012:c
3006:{
2986:}
2979:t
2975:c
2969:{
2946:r
2920:r
2912:d
2892:)
2887:r
2879:d
2876:,
2873:r
2870:(
2867:Q
2853:r
2829:,
2826:}
2823:)
2820:)
2817:c
2811:a
2808:(
2805:)
2802:r
2799:+
2796:1
2793:(
2790:(
2787:V
2781:+
2778:)
2775:c
2772:(
2769:u
2766:{
2761:a
2755:c
2749:0
2741:=
2738:)
2735:a
2732:(
2729:V
2689:t
2685:a
2671:t
2640:,
2637:0
2627:t
2623:c
2616:,
2613:)
2606:t
2602:c
2589:t
2585:a
2579:(
2576:)
2573:r
2570:+
2567:1
2564:(
2561:=
2554:1
2551:+
2548:t
2544:a
2515:)
2508:t
2504:c
2498:(
2495:u
2490:t
2475:0
2472:=
2469:t
2438:}
2431:t
2427:c
2421:{
2401:r
2381:t
2361:1
2349:0
2329:c
2309:)
2306:c
2303:(
2300:u
2277:0
2253:0
2249:a
2197:)
2194:x
2191:(
2188:a
2168:x
2144:V
2117:.
2114:}
2111:)
2108:)
2105:a
2102:,
2099:x
2096:(
2093:T
2090:(
2087:V
2081:+
2078:)
2075:a
2072:,
2069:x
2066:(
2063:F
2060:{
2055:)
2052:x
2049:(
2040:a
2032:=
2029:)
2026:x
2023:(
2020:V
1994:.
1991:)
1986:0
1982:a
1978:,
1973:0
1969:x
1965:(
1962:T
1959:=
1954:1
1950:x
1945:,
1942:)
1937:0
1933:x
1929:(
1918:0
1914:a
1893:}
1890:)
1885:1
1881:x
1877:(
1874:V
1868:+
1865:)
1860:0
1856:a
1852:,
1847:0
1843:x
1839:(
1836:F
1833:{
1826:0
1822:a
1813:=
1810:)
1805:0
1801:x
1797:(
1794:V
1764:)
1759:0
1755:a
1751:,
1746:0
1742:x
1738:(
1735:T
1732:=
1727:1
1723:x
1690:)
1685:0
1681:a
1677:,
1672:0
1668:x
1664:(
1661:T
1658:=
1653:1
1649:x
1626:0
1622:a
1598:.
1595:)
1590:0
1586:a
1582:,
1577:0
1573:x
1569:(
1566:T
1563:=
1558:1
1554:x
1549:,
1546:)
1541:0
1537:x
1533:(
1522:0
1518:a
1490:}
1485:]
1481:1
1475:t
1468:,
1465:)
1460:t
1456:a
1452:,
1447:t
1443:x
1439:(
1436:T
1433:=
1428:1
1425:+
1422:t
1418:x
1413:,
1410:)
1405:t
1401:x
1397:(
1386:t
1382:a
1378::
1375:)
1370:t
1366:a
1362:,
1357:t
1353:x
1349:(
1346:F
1341:1
1335:t
1320:1
1317:=
1314:t
1297:1
1294:=
1291:t
1286:}
1281:t
1277:a
1273:{
1262:[
1255:+
1252:)
1247:0
1243:a
1239:,
1234:0
1230:x
1226:(
1223:F
1219:{
1211:0
1207:a
1176:1
1172:x
1109:0
1105:x
1080:)
1075:0
1071:x
1067:(
1064:V
1038:,
1035:2
1032:,
1029:1
1026:,
1023:0
1020:=
1017:t
1010:,
1007:)
1002:t
998:a
994:,
989:t
985:x
981:(
978:T
975:=
970:1
967:+
964:t
960:x
955:,
952:)
947:t
943:x
939:(
928:t
924:a
897:,
894:)
889:t
885:a
881:,
876:t
872:x
868:(
865:F
860:t
845:0
842:=
839:t
822:0
819:=
816:t
811:}
806:t
802:a
798:{
787:=
783:)
778:0
774:x
770:(
767:V
741:1
729:0
706:)
703:a
700:,
697:x
694:(
691:F
671:x
651:a
631:a
611:)
608:a
605:,
602:x
599:(
596:T
576:x
554:t
550:x
529:)
524:t
520:x
516:(
491:t
487:a
466:)
461:t
457:x
453:(
442:t
438:a
415:0
411:x
390:t
368:t
364:x
306:)
303:W
300:(
297:H
283:H
261:)
258:W
255:(
252:c
242:W
234:c
202:)
199:W
196:(
72:)
66:(
61:)
57:(
43:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.
