You can derive the probability in a manner similar to that for the usual derivation of the derangement probabilities (the probability that none of the $N$ men get their own hat back).
There are a total of $N!^n$ ways that all of the items can be distributed among the $N$ men so that each has exactly one of each type of item. Let $A_i$ denote the event that the $i$th man obtains the $n$ items that belong to him. The number of ways this can happen is $|A_i| = (N-1)!^n$, as this involves distributing all items but those belonging to the $i$th man among the other $N-1$ men. Similarly, $|A_i \cap A_j| = (N-2)!^n$, and, in general, $|A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_j}| = (N-j)!^n$. There are also $\binom{N}{j}$ ways to choose which $j$ men will receive their own $n$ items.
Let $D(N,n,k)$ denote the number of ways that exactly $k$ of the $N$ men receive all $n$ of their items back. By the principle of inclusion/exclusion, $$D(N,n,0) = \sum_{j=0}^N (-1)^j \binom{N}{j} (N-j)!^n = N! \sum_{j=0}^N (-1)^j \frac{(N-j)!^{n-1}}{j!}.$$
Now, $D(N,n,k) = \binom{N}{k} D(N-k,n,0)$, as this is the number of ways of choosing $k$ of the $N$ men to receive all of their items back times the number of ways that none of the remaining $N-k$ men receive all of their items back.
Thus $$D(N,n,k) = \binom{N}{k} (N-k)! \sum_{j=0}^{N-k} (-1)^j \frac{(N-k-j)!^{n-1}}{j!} = \frac{N!}{k!} \sum_{j=0}^{N-k} (-1)^j \frac{(N-k-j)!^{n-1}}{j!}.$$
Dividing by $N!^n$, we have that the probability that exactly $k$ of the $N$ men receive all $n$ of their items back is
$$\frac{1}{k! N!^{n-1}} \sum_{j=0}^{N-k} (-1)^j \frac{(N-k-j)!^{n-1}}{j!}.$$
Note that this formula agrees with the values obtained by Henry for the case $N = 4$, $n=2$.
Added: In fact, the Poisson approximation suggested by Henry appears to match up well with the exact values provided by the formula given here for small values of $k$. The accuracy of the Poisson approximation appears to deteriorate, relatively speaking, as $k$ increases. However, the Poisson approach still gives a good absolute approximation when $k$ is large because the probabilities are extremely small.
This is a standard conceptual misunderstanding. What's the probability that the second person gets the right hat? Lots of people respond instantly that it depends on whether the first person got the right hat. That's the right answer to the wrong question.
The conditional probability that the second person gets the right hat given that the first person got the right hat, or that the first person got the wrong hat, does depend on whether the first person got the right hat or not.
But that's the wrong question. What's the probability that the second person gets the right hat, in the absence of any information about the first person's fate? It's $1/n$ because there are $n$ hats and none is more likely than another to be the one that that person gets.
But if you insist on thinking about those conditional probabilities, here's how to do it:
\begin{align}
& \Pr(\text{2nd right}) = \Pr(\text{(1st right and 2nd right) or (1st wrong and 2nd right)}) \\[10pt]
= {} & \Pr(\text{1st right})\Pr(\text{2nd right}\mid\text{1st right}) \\
& {} + \Pr(\text{1st wrong and got 2nd one's hat}) \cdot \Pr(\text{2nd right}\mid\text{1st wrong and got 2nd one's hat}) \\
& {} + \Pr(\text{1st wrong and didn't get 2nd one's hat}) \cdot\Pr(\text{2nd right} \mid \text{1st wrong and $\cdots$ [etc.]}) \\[10pt]
= {} & \left(\frac1n\cdot\frac{1}{n-1}\right) + \left(\frac 1 n \cdot 0\right) + \left( \frac{n-2} n \cdot \frac 1 {n-1} \right) \\[10pt]
= {} & \frac 1 n.
\end{align}
Best Answer
Not quite. $X = I_1+\dotsb+I_n$, so $$E[X] = E[I_1+\dotsb+I_n] =E[I_1]+\dotsb+E[I_n] = n\cdot E[I_1] = n\cdot\frac{1}{n} = 1,$$ since \begin{align*} E[I_k] &= 0\cdot P(I_k = 0)+1\cdot P(I_k = 1) \\ &= P(I_k = 1), \end{align*} and $P(I_k = 1) = \frac{1}{n}$.
Think about it this way, pretend each person is sitting in a line and the each hat has been labeled to match the correct person. With indicators, it is like you distribute each ticket back randomly face down to each person. So the chance that person $k$ got the correct ticket is $\frac{1}{n}$. In this manner, we bar people from choosing the same hat.