normal distributionpoisson distributionprobability
I noticed that in the Normal distribution, the probability $P(x=c)$ equals zero, while for the Poisson distribution, it will not equal zero when $c$ is a non-negative integer.
My question is: Does the probability of any constant in the normal distribution equal zero because it represents the area under any curve? Or it is just only a rule to memorize?
That Wiki page is abusing language by referring to this number as a probability. You are correct that it is not. It is actually a probability per foot. Specifically, the value of 1.5789 (for a height of 6 feet) implies that the probability of a height between, say, 5.99 and 6.01 feet is close to the following unitless value:
$$1.5789\, [1/\text{foot}] \times (6.01 - 5.99)\, [\text{feet}] = 0.0316$$
This value must not exceed 1, as you know. (The small range of heights (0.02 in this example) is a crucial part of the probability apparatus. It is the "differential" of height, which I will abbreviate $d(\text{height})$.) Probabilities per unit of something are called densities by analogy to other densities, like mass per unit volume.
Bona fide probability densities can have arbitrarily large values, even infinite ones.
This example shows the probability density function for a Gamma distribution (with shape parameter of $3/2$ and scale of $1/5$). Because most of the density is less than $1$, the curve has to rise higher than $1$ in order to have a total area of $1$ as required for all probability distributions.
This density (for a beta distribution with parameters $1/2, 1/10$) becomes infinite at $0$ and at $1$. The total area still is finite (and equals $1$)!
The value of 1.5789 /foot is obtained in that example by estimating that the heights of males have a normal distribution with mean 5.855 feet and variance 3.50e-2 square feet. (This can be found in a previous table.) The square root of that variance is the standard deviation, 0.18717 feet. We re-express 6 feet as the number of SDs from the mean:
$$z = (6 - 5.855) / 0.18717 = 0.7747$$
The division by the standard deviation produces a relation
$$dz = d(\text{height})/0.18717$$
The Normal probability density, by definition, equals
$$\frac{1}{\sqrt{2 \pi}}\exp(-z^2/2)dz = 0.29544\ d(\text{height}) / 0.18717 = 1.5789\ d(\text{height}).$$
(Actually, I cheated: I simply asked Excel to compute NORMDIST(6, 5.855, 0.18717, FALSE). But then I really did check it against the formula, just to be sure.) When we strip the essential differential $d(\text{height})$ from the formula only the number $1.5789$ remains, like the Cheshire Cat's smile. We, the readers, need to understand that the number has to be multiplied by a small difference in heights in order to produce a probability.
Suppose $X$ is Poisson with parameter $\lambda$, and $Y$ is normal with mean and variance $\lambda$. It seems to me that the appropriate comparison is between $\Pr(X = n)$ and $\Pr(Y \in [n-\frac12,n+\frac12])$. Here for simplicity I write $n = \lambda + \alpha \sqrt\lambda$, that is, we are interested when $n$ corresponds to $\alpha$ standard deviations from the mean.
So I cheated. I used Mathematica. So both $\Pr(X = n)$ and $\Pr(Y \in [n-\frac12,n+\frac12])$ are asymptotic to $$ \frac 1{\sqrt{2\pi \lambda}} e^{-\alpha^2/2} $$ as $\lambda \to \infty$. But their difference is asymptotic to $$ \frac{\alpha \left(\alpha ^2-3\right) e^{-\alpha ^2/{2}}}{6 \sqrt{2 \pi } \lambda } $$ If you plot this as a function of $\alpha$, you will get the same curve as is shown in the second to last figure in http://www.johndcook.com/blog/normal_approx_to_poisson/.
Here are the commands I used:
n = lambda + alpha Sqrt[lambda];
p1 = Exp[-lambda] lambda^n/n!;
p2 = Integrate[1/Sqrt[2 Pi]/Sqrt[lambda] Exp[-(x-lambda)^2/2/lambda], {x, n-1/2, n+1/2}];
Series[p1, {lambda, Infinity, 1}]
Series[p2, {lambda, Infinity, 1}]
Also, with a bit of experimentation, it seems to me that a better asymptotic approximation to $\Pr(X = n)$ is $\Pr(Y \in [n-\alpha^2/6,n+1-\alpha^2/6])$. Then the error is $$ -\frac{\left(5 \alpha ^4-9 \alpha ^2-6\right) e^{-{\alpha ^2}/{2}} }{72 \sqrt{2 \pi } \lambda ^{3/2} } $$ which is about $\sqrt\lambda$ times smaller.
Best Answer
Perhaps the following thought-experiment helps you to understand better why the probability $Pr(X=a)$ is zero in a continuous distribution: Imagine that you have a wheel of fortune. Normally, the wheel is partitioned in several discrete sectors, perhaps 20 or so. If all sectors have the same area, you would have a probability of $1/20$ to hit one specific sector (e.g. the main price). The sum of all probabilities is 1, because $20\cdot 1/20 = 1$. More general: If there are $m$ sectors evenly distributed on the wheel, every sectors has a probability of $1/m$ of being hit (uniform probabilities). But what happens if we decided to partition the wheel into a million sectors. Now the probability of hitting one specific sectors (the main prize), is extremely small: $1/10^{6}$. Further, note that the pointer can theoretically stop at an infinite number of positions of the wheel. If we wanted to make a separate prize for each possible stopping point, we would have to partition the wheel in an infinite number of "sectors" of equal area (but each of those would have an area of 0). But what probability should we assign to each of these "sectors"? It must be zero because if the probabilities for each "sectors" would be positive and equal, the sum of infinitely many equal positive numbers diverges, which creates a contradiction (the total probability must be 1). That's why we only can assign a probability to an interval, to a real area on the wheel.
More technical: In a continuous distribution (e.g. continuous uniform, normal, and others), the probability is calculated by integration, as an area under the probability density function $f(x)$ (with $a\leq b$): $$ P(a\leq X \leq b) = \int_{a}^{b} f(x) dx $$ But the area of an interval of length 0 is 0.
See this document for the analogy of the wheel of fortune.
The Poisson distribution on the other hand is a discrete probability distribution. A random Poisson variable can only take discrete values (i.e. the number of children for one family cannot be 1.25). The probability that a family has exactly 1 child is certainly not zero but is positive. The sum of all probabilities for all values must be 1. Other famous discrete distributions are: Binomial, negative binomial, geometric, hypergeometric and many others.