Suppose that you use $p$ pennies, $n$ nickels, $d$ dimes, and $q$ quarters; then $p,n,d$, and $q$ must satisfy the system
$$\left\{\begin{align*}
&p+n+d+q=50\\
&p+5n+10d+25q=150\;.
\end{align*}\right.\tag{1}$$
Thus, you want to count the solutions to $(1)$ in non-negative integers.
The problem is small enough that you can easily solve by hand mostly using brute force. First, subtracting the first equation in $(1)$ from the second gives us the condition $$4n+9d+24q=100\;,\tag{2}$$
from which it’s clear that $q\le 4$.
Suppose that $q=4$; then $4n+9d=100-24\cdot4=4$, which clearly has only the solution $n=1,d=0$. At this point we’ve used four quarters, no dimes, and one nickel, for a total of $105$ cents from $5$ coins, and setting $p=45$ clearly gives the unique solution with $q=4$.
Now suppose that $q=3$, so that $(2)$ reduces to $4n+9d=28$. Clearly $d\le 3$. But both $4n$ and $28$ are divisible by $4$, while $9d$ is not for $0<d\le 3$, so we must have $d=0$ and $n=7$. We’ve now used $10$ coins that total $110$ cents, and it’s clear that setting $p=40$ gives us the unique solution with $q=3$.
Now suppose that $q=2$, so that $(2)$ reduces to $4n+9d=52$. Then $d\le 5$, and we can argue as in the previous case that $9d$ must be a multiple of $4$, so either $d=0$ and $n=13$, or $d=4$ and $n=4$. In the first case we have $15$ coins that total $115$ cents, and we must set $p=35$; in the second we have $10$ coins that total $110$ cents, and we must set $p=40$. These are clearly the only solutions when $q=2$.
If $q=1$, $(2)$ reduces to $4n+9d=76$, so that $d\le 8$. As in the previous two cases we must have $4\mid d$, so $d$ must be $0,4$, or $8$. These lead to the following solutions: $q=1,d=0,n=19,p=30$; $q=1,d=4,n=10,p=35$; and $q=1,d=8,n=1,p=40$.
Finally, if $q=0$, $(2)$ becomes $4n+9d=100$, the feasible values of $d$ are again $0,4$, and $8$, and the resulting solutions are: $q=0,d=0,n=25,p=25$; $q=0,d=4,n=16,p=30$; and $q=0,d=8,n=7,p=35$.
Thus, there are precisely $10$ solutions, and we’ve not just counted them, but also found them.
Here's some intuition:
Suppose you have 15 coins as follows:
O O O O O O O O O O O O O O O O
How many ways can you divide up these 15 coins into 4 different categories of coins? For instance:
O | O O | O O O O O O O O O | O O O
(Pennies | Nickels | Dimes | Quarters)
Hope this is helpful, let me know if you need another hint :)
Best Answer
The answer is not $\binom84$. Instead, it is found by stars and bars: in a line of 8 coins, you can put three separators among those coins to partition them into four groups, which can be identified with pennies, nickels, dimes and quarters. Each way of placing the separators corresponds one-to-one with a possible collection.
There are three separators to be placed among $8+3=11$ coins plus separators, so the correct answer is $\binom{11}3=165$.