For an optimization problem with multiple states ($x$), controls ($y$), and random disturbances ($z$), the Euler equation for a stochastic dynamic programming problem is:
$D_yU(x,y,z)+\beta E D_xU(x,y,z)=0$
where $D_x$ and $D_y$ are the gradients with respect to the states and controls, respectively, $\beta$ is the discount factor, $U$ is the objective function, and $E$ is the expectation operator. This is easy enough to do in the single-state case, but I'm having problems getting a clear picture of the multivariate case, and none of the textbooks I have include very lucid examples.
My problem is:
$\max \quad E_0 \sum_{t=0}^\infty\beta^t u(c_t,N_t)$
subject to:
$c_t+K_{t+1}+G_t=f(K_t,N_t,\zeta_t)$
and
$N_t\leq1$
About the notation – $u$ is the objective function (a utility function), $K$ is the capital stock, $N$ is the proportion of available time supplied as labor, $\zeta$ is a random productivity disturbance, and $G$ is the government's budget (exogenous). $f$ is a production function, and total production depends on the two inputs ($N,K$), as well as the disturbance $\zeta$. Decisions are made after the disturbance is realized, so the budget restriction is always binding. $G$ and $\zeta$ both follow Markov processes. Lastly, capital completely depreciates every period, which is why $K_t$ doesn't show up on the left hand side of the constraint.
OK, with that out of the way, what I really want to know is how to set the problem up so that I can get first order conditions for a solution. As I see it, if I substitute the budget constraint into the objective function, the states for the problem are $\{K,G,\zeta\}$, the controls are $\{K^+,N\}$, and the random disturbances are $\{G,\zeta\}$. If I substitute these into the Euler equation I get:
$\left[\begin{matrix} U_{K^+} \\ U_N \end{matrix}\right]+\beta E(U_{K^+})|G,\zeta=0,$
where $U_{K^+}$ and $U_N$ are the partial derivatives of the objective function with respect to $K^+$ (i.e., $K_{t+1}$) and $N$ (i.e., $N_t$) respectively. Does this look correct, or am I missing something? My biggest worry is that the dimensions of the gradients don't match — is that a problem? I'd appreciate any help at all.
Best Answer
On closer inspection, it seems that the Euler equation at the bottom of the question is the correct expression for the necessary conditions for a solution. It's composed of two conditions – an intertemporal condition on the growth of the state variable, $K$:
$U_{K^+}+\beta E\left[(U_{K^+})|G,\zeta \right]=0$
In other words, the sacrifice in utility gained by increasing $K^+$ has to be compensated for by the increase in expected future utility that is gained. In other words, increasing $K^+$ this period decreases current utility, but increases the expected value of future utility flows, and an optimum is achieved (provided an interior solution exists) when the two of these exactly offset one another.
The second condition is the intratemporal condition on $N$:
$U_N=\frac{\partial u}{\partial c}\times\frac{\partial c}{\partial N} + \frac{\partial u}{\partial N}=0$
$u$ is increasing in $N$ in its first argument (higher $N$, more production), but decreasing in its second argument (lower $N$, more utility from leisure). Therefore, it's a necessary condition for an optimum that the marginal increase in utility gained by increasing $N$ be offset by the marginal decrease in utility lost.