Why is binary base-2 but decimal is base 10?
This is quite a trivial question, since those are just the terms we use for those bases. It's like asking "why are humans people?"; it's just a word we use. Binary means base-2 and decimal means base-10. There's nothing complicated there.
Why do we use powers of $2$ for binary numbers?
Base-10 and Carrying
Think about how you represent numbers (integers). It's easy for the first $10$, since we just make a new symbol each time: $0,1,2,3,4,5,6,7,8,9$. But, of course, we don't want to make a new symbol for every new number since it'd get very messy and complicated. So for the number after $9$, we start counting in a new place. So, we put a $1$ (shown in blue) in the place on the left, to get $\color{blue}{1}\text{_}$. Then, in the place on the right, we reset our $9$ to $0$ and start counting all over again (shown in red). Hence, after $9$, we get $\color{blue}{1}\color{red}{0},\color{blue}{1}\color{red}{1},\color{blue}{1}\color{red}{2},\ldots$. The $1$ that we put in the left place is called a carry. Of course, when we get to $19$, we change the $1$ on the left to a $2$ and again, reset the $9$ to a $0$. This may seem obvious, but we need to think about exactly what we're doing.
The whole point of doing this is making it simpler and easier to read numbers. With no loss of clarity, we've managed to represent every integer as a sequence of $10$ symbols. This is the system that we almost always use, and is called base-ten or decimal.
To represent bigger numbers in decimal notation, we'd keep putting a new number on the left every time we can't advance our numbers any further. Hence, $9\mapsto10$, and $99\mapsto100$ and $999\mapsto1000$ and so on. Like I mentioned before, we're carrying the $1$ in each case. What's the common feature among $10,100,100\ldots$? Well, they're all powers of $10$. Think about this, since it's important. We started with $10$ symbols (i.e. $0,1,2,3,\ldots$) but whenever we carry a $1$ (to a new position on the left), we do it at a power of $10$. This isn't a coincidence. This happens because we carry the $1$ every time we've maxed out our count in the other columns, which happens at $9$ and $99$ and so on.
This means that when we write the number $1729$, what we actually mean is $$1\times1000\quad+\quad7\times100\quad+\quad2\times10\quad+\quad9\times1$$
This shows us why base-ten is so fundamentally linked with powers of $10$, because the digits of a number tell you how many $1$'s, $10$'s, $100$'s, etc. there are in it.
Base-2
In base $10$, we started with $10$ symbols. But why? Is there a special reason for choosing $10$? It turns out that no, there is no fundamental, mathematical reason to prefer $10$ symbols to a different number (say $8$ or $12$). Notice that the number of symbols is the base. There are cultural, historical and practical arguments for choosing $10$ but these aren't relevant here. In fact, the Native American Yuki People use base $8$, while the Babylonians used base $12$.
Let's say we wanted to use base $2$ for our counting (i.e. only using $2$ symbols). For simplicity, we use the first $2$ symbols we were using before (i.e. $0$ and $1$). If we were using this system, how would we count? We'd start with $0,1,???$ and then what? Well, we'd do the same thing we did before and put a $1$ to the left of our numbers. Hence $\color{red}{0},\color{red}{1}\mapsto\color{blue}{1}\color{red}{0},\color{blue}{1}\color{red}{1}$.
In fact, just like how we max out our base-$10$ count at $9$ or $99$ or $999\ldots$, etc., we max out our base $2$ count at $1_B$ or $11_B$ or $111\ldots_B$ (the subscript B indicates that these numbers are in base-2). To make this clearer, look at how we count in binary:
$$0,1,10,11,100,101,110,111,1000\ldots$$
Every time we get a number only made of ones, we carry over a $1_B$ to the left. For instance, after $111_B$ we get $1000_B$.
But when do we get $1_B$ or $10_B$ or $100\ldots_B$? Looking at these numbers in decimal form should give a clearer indication: $1,2,4,8,16,\dots$. These are the powers of $2$. Since we've only got $2$ different symbols, we max out our count after $2$ numbers or $2\times2$ numbers, or $2\times2\times2\ldots$.
Hence, base-$2$ is fundamentally related to powers of $2$. When we write the base-$2$ number $1011_B$, we really mean:
$$1\times8\quad+\quad0\times4\quad+\quad1\times2\quad+\quad1\times1$$.
Where $1,2,4,8,\ldots$ are the powers of $2$. In other words, $2^0,2^1,2^2,\ldots$
In general, for any system with $n$ symbols, when you write $31415_{\text{(base $n$)}}$, you mean the following, in base-$10$: $$(3\times n^4)+(1\times n^3)+(4\times n^2)+(1\times n^1)+(5\times n^0)$$
Hopefully this clears up why base $10$ has a sum of powers of $10$ but base $2$ has powers of $2$. It's a natural consequence of using any base.
Why do we count with base-$10$ for most things but with base-$2$ for computers
As I wrote earlier, using base $10$ for representing numbers is heavily influenced by history and society. Most often, the numbers you'll deal with in your life will be between $1$ and $200$. For example, how many years old you are, how many cars are on your street or how many eggs you ate this week. Of course, these numbers are unlikely to be massive. So it makes sense to need a base approximately $10$ since small or large bases aren't practical.
For practical purposes, base-$2$ is too cumbersome since it uses so many numbers (the number $100$ is $1100100_B$ in base-$2$) but base $1000$ is completely unfeasible since you'd need $1000$ different symbols. Can you think of $1000$ symbols that look significantly different? I can't.
Hence, we use a base around $8-16$. Some people have used $8$ before, some $12$ and some $16$. It doesn't really matter too much which one you choose. It's handy for the base to have a lot of divisors, since it lets you write simple expressions for fractions (like how $\frac15=0.2$ in base-$10$). Hence $11$ and $13$ aren't used. It's also handy to have the base divisible by $2$ so that $\frac12$ can have a simple expression. So $7,9,15$ aren't typically used. This leaves us with $8,10,12,14$ and $16$. Base $14$ isn't often used since one of its divisors is $7$, and why should we include $7$ as a divisor but not $3$ or $5$? Nevertheless, $8,10,12$ and $16$ have been widely used as bases. This rule doesn't cover every culture since even base $64$ has been used (though it needed a lot of symbols!). Also, as @ Luís Henrique points out, the mesoamerican Maya civilization used an interesting combination of base-$5$ and base-$20$ in their number system.
Other commenters have pointed out the fact that we have $10$ fingers/toes so it's easy to count base-$10$ numbers with your fingers. But this doesn't have a huge amount of bearing (in written math) since you don't often count with your fingers. As @Luís Henrique points out, this may have been much more important in more primitive societies where numbers could be communicated by hand-signs (instead of with written or spoken words), so it may give a historical basis for using base-$10$.
On the other hand, computers use binary (base-$2$) since they represent numbers in electronic states (i.e. a lightbulb being on or off). It makes computers a lot simpler and easier to work with (particularly in the design and manufacture of transistors) for them work in binary. Though there have been computers that use base-$3$ (ternary computers), they are less common.
Hopefully this answers your questions.
Best Answer
I don't know that this counts as an answer, but it is too much to say in a comment:
Actually, the change does help considerably to understand the question. But the answer is still dependent on the particular implementation. Different systems represent floating point numbers in different ways (even without considering Endianess). Most common (I believe - I am not a computer scientist), is to represent -37.125 exactly the same as 37.125, except for the sign bit $D$ being set. $37.125 = 0b100101.001$. The Exponent $E$ is determined by shifting this until only the leading 1 is to left of the radix $37.125 = 0b1.00101001 \times 2^5$, so $E = 5$. (I'll let you translate it to binary. Note that $E$ is a signed integer and normally written in 2s-complement - the only part of the expression that is.) Since the leading bit is always 1, there is no need to actually store it, so the significand $A = 0b00101001$, followed by however many other 0s you have room for.
However, whether this is the exact form you need it in, I cannot say.