This is an interesting problem. I thought about it for some time and I believe I have a complete solution.
First, let's see how we would answer the question if we had only 2 people. Say, Alice believes that an event will happen with probability $90\%$, while Bob believes that it will happen with probability $80\%$. How do you make a bet out of this event? It is not obvious.
Intuitively we can think that the two people must take opposite positions, otherwise they will both be winners or losers at the same time. Moreover, the one with more confidence in the event (Alice) should take the for position and the other person should take the against position. We will prove this later, but there is also another major question: on what odds should they bet? Let's explore the problem:
Assume that Alice bets $\$1$ on the for position (i.e. the event actually happening) How much should she make back to consider this a favourable bet for her? Let's name the return (i.e. the amount won) as $x$. Just to be clear, $x$ is the amount returned above the $\$1$ she's already put in. In other words, if Alice is betting $\$1$ and wins, she is getting back $\$1+\$x$. To find $x$ of a favourable bet, we just make the expected value of the bet positive:
$$0.9\cdot x - 0.1\cdot 1 > 0 \iff x > \frac{1}{9}$$
So if Alice bets $\$1$, she would like her opponent to bet at least $\$1/9$. If we write this in the typical form of odds as opponent_bet:your_bet then Alice is looking for a $1/9:1 \iff 1:9$ bet or better.
What if Alice wants to bet on the against position (i.e., the event not happening)? A favourable bet of $\$1$ would be one with return $x$ where
$$0.1\cdot x - 0.9\cdot 1 > 0 \iff x > 9$$
The odds now are $9:1$. It is the reciprocal as we expected.
Doing the analysis for Bob we find that
- Bob betting on the
for position wants odds $1:4$ or better
- Bob betting on the
against position wants odds $4:1$ or better
How would we create a fair bet between the two? Let's make a table with all the possible position combinations and conditions for each combination
$$
\begin{array}{l|c|c}
& \text{Alice for} & \text{Alice against} \\
\hline
\text{Bob for} & \text{not a bet}
&
\begin{array}{c}
& \text{Alice wants}\space x:1, \space \color{red}{x>9}\\
& \text{Bob wants}\space 1:x, \space \color{red}{x<4} \\
\end{array}\\
\hline
\text{Bob against}
& \begin{array}{c}
& \text{Alice wants}\space 1:x, \space \color{green}{x<9}\\
& \text{Bob wants}\space x:1, \space \color{green}{x>4} \\
\end{array}
& \text{not a bet} \\
\end{array}
$$
As we see from the table, the only possible bet is when Bob bets against and Alice bets for. This is inline with our intuition that the person most confident should take the for position. We also know that an $x$ such that $4<x<9$ is acceptable for both Alice and Bob (in the sense that they both expect a positive gain). What is the most fair value however? Should we just take the middle value ($x=6.5$)? This might seem reasonable, but remember that odds are "multiplicative" entities, and taking "additive" averages might not work. Indeed the most fair selection of $x$ is the one that yields the same expected values for both parties:
$$0.2\cdot x -0.8 \cdot 1 = 0.9 \cdot 1 - 0.1 \cdot x\iff 0.3 \cdot x = 1.7 \iff x = \frac{17}{3} \approx 5.666$$
So Alice should make the for bet with odds $3:17$ and Bob make (the opposite) bet with odds $17:3$.
And if we wanted for both of them to bet $\$100$ in total, the bets would be $100\cdot 17/(17+3) = \$85$ for Alice and $100\cdot 3/(17+3) = \$15$ for Bob. Notice how Alice's bet is the average of their perceived likelihoods.
So if we know the perceived likelihoods for each of the two players, we can construct a fair bet. It's interesting to note that we can always construct a fair bet for two people, and that the closer the likelihoods are to each other, the smaller the gain both players expect. So if both Alice and Bob believe that the event happens with probability $80\%$, then the only fair bet is $1:4 \space\space(4:1)$ and the expected gain for both of them is $0$. If, on the other hand, Alice believes the event will happen $90\%$ and Bob believes it will happen $10\%$ of the time, then the fair bet is $1:1$ and the expected gain of both players is $\$0.8$ for each $\$1$ played. Just to be clear, this is the expected gain based on a player's perceived likelihood, it does not have to agree with the actual expected gain (in other words: people can have mistaken beliefs)
The case for 3 players
Let's examine what happens with 3 players. Let's take the particular example given in the question.
- Alice believes the event happens with probability $35\%$
- Bob believes the event happens with probability $50\%$
- Charlie believes the event happens with probability $40\%$
This gives us the following favourable odds for each person:
- Alice wants to bet
for with $13:7$ (or better), or against with $7:13$ (or better)
- Bob wants to bet either
for or against with $1:1$ (or better)
- Charlie wants to bet
for with $3:2$ (or better), or against with $2:3$ (or better)
Let's build the betting table again. With $3$ people we have $2^3=8$ combinations. Similarly to the 2-people case, combinations where everyone bets for or everyone bets against are not valid bets. So we are left with 6 betting combinations. What should the conditions be for each combination? Things are more complicated now that we have 3 people. First let's revisit the conditions for the 2-person case. In the analysis above, I stated without much explanation that conditions for the 2-person case look like this: $x:1$ for Alice and $1:x$ for Bob. It seems intuitive, but it would be helpful to derive it so we can understand the restrictions we are taking and the assumptions we are making. The initial conditions for the 2-person example should have been:
$$
\begin{array}{rc}
\text{Alice wants odds } \space x:a, & \frac{x}{a}>\frac{1}{9} \\
\text{Bob wants odds } \space y:b, & \frac{y}{b}>\frac{1}{4} \\
\end{array}
$$
But now we have 4 variables instead of one! This is because these two conditions describe only what the favourable bets are for each player. So if Alice and Bob were betting any amount against a bookmaker, these are the conditions they would use. But if we want to make them bet against each other then we need to impose more restrictions. We need to restrict these quantities so that whatever one player bets is the (potential) winning of the other player. This means that $a=y$ and $b=x$. With these restrictions we are down to two variables. How do we eliminate another variable? Since odds are essentially ratios, we can simply normalise one of them by setting $a=1$. This is what we have implicitly done when analysing the 2-person case above. Another (perhaps better) option is to treat the $x,y,a,b$ variables as the actual quantities being betted and won. In this case, if we know the total amount we want our players to bet (say $\$100$) we simply set $a+b=100 \iff a+x = 100$ which together with the equation of making the expected gains equal, yields a $2\times 2$ system. Its solution for the 2-person example we gave earlier results in $a=85$ and $x=15$ as we have found before.
How does all these translate to the 3-person case? Let's take one particular combination where Alice bets against, while Bob and Charlie bet for. Also assume, as the question states, that all together are betting $\$100$. Here are the initial three conditions for favourable bets, along with the three extra restrictions that make this a valid bet among Alice, Bob, and Charlie :
$$
\begin{array}{rl}
\text{Alice wants odds } \space x:a, & \frac{x}{a}>\frac{7}{13} \\
\text{Bob wants odds } \space y:b, & \frac{y}{b}>\frac{1}{1} \\
\text{Charlie wants odds } \space z:c, & \frac{z}{c}>\frac{3}{2} \\
a &= y+z\\
x &= b +c\\
a+b+c &= 100
\end{array}
$$
If we make the expected gains to be equal to each other in order to ensure the fairness criterion, we will get $2$ equations which along with the $3$ restriction make an underdetermined system with $5$ equations and $6$ variables. We could bring another restriction to force the system to have a unique solution (unique for a given betting combination out of the possible $6$), but there is no strong restriction I can think of. There are desirable properties such as maximising the product $a\cdot b\cdot c$ which means that we are trying to make the 3 bets as equal as possible (in other words, make it unlikely to have one person betting $\$98$ and the other two betting $\$1$). Or we can maximise the (equalised) expected gain, but note that this does not mean that everyone will get their best odds. In any case, I do not see these as necessary restrictions. We can have one or not. If not, we will have a infinite number of solutions (under the remaining restrictions) for each betting combination. The solutions ensure that everyone has a perceived positive gain, and that all the perceived gains are equal, so we have a fair solution.
Here are the 5 equations and the parametrised solutions for the case where Alice bets against, while Bob and Charlie bet for, and all together are betting $\$100$:
$$
\left.
\begin{array}{rl}
0.65\cdot x - 0.35\cdot a &= 0.5\cdot y - 0.5\cdot b \\
0.5\cdot y - 0.5\cdot b &= 0.4\cdot z - 0.6\cdot c \\
a &= y+z\\
x &= b +c\\
a+b+c &= 100
\end{array}
\right\} \iff
\begin{array}{rl}
x &= 39.62 - 0.077 \cdot t\\
y &= 48.85 - 1.231 \cdot t\\
z &= 11.54 + 1.310 \cdot t\\
a &= 60.38 + 0.077 \cdot t\\
b &= 39.62 - 1.077 \cdot t\\
c &= t
\end{array}
$$
(Solution provided by this online tool)
Our parameter (free variable) is $t$, and we can notice that in this particular case it mostly affects $b,c,y,z$ and minimally affects $a,x$. By changing the parameter we are essentially changing the relative "weight" between Bob and Charlie as they both take on Alice. For example, try $t=19$ for a fairly balanced "weight" between Bob and Charlie (where Bob and Charlie bet almost equal amounts). This results in the following bet:
- Alice bets $\$61.85$
against. If she wins she'll earn $\$38.15$ (so she'll receive back $\$38.15 + \$61.85 = \$100$)
- Bob bets $\$19.15$
for. If he wins he'll earn $\$25.46$ (so he'll receive back $\$25.46 + \$19.15 = \$44.61$)
- Charlie bets $\$19.00$
for. If he wins he'll earn $\$36.39$ (so he'll receive back $\$36.39 + \$19.00 = \$55.39$)
Notice how the sum of players' bets is $\$100$, and what winners get back also totals $\$100$ (as it should).
We can find similar solutions (families of solutions) for the five remaining betting combinations. It turns out that with this example (these likelihood beliefs) all six combinations can yield fair solutions, albeit some combinations have a smaller range of solutions than others. You can explore the other families of solutions yourself, doing the same analysis and solving equivalent $5\times6$ systems.
I believe this is a quite interesting result and it fully solves this betting problem.
Addendum
I include here my earlier work on the 3-people case in order to show some of the progress of my ideas. I created a table with all the betting combinations and tried to provide the conditions for each combination. I arbitrarily restricted 3 of the variables to take specific values (but still abiding by all restrictions), and this resulted in a $3\times3$ equation system. For example, for the case we have been analysing above I took $x = 7$, $b = 3$, $c=4$, satisfying the condition $x=b+c$. I do not assume that $a+b+c=100$. This simplified some expressions and I could check the conditions of all the combinations manually. For three of the combinations there are no solutions with this arbitrary choice that I made (impossible conditions are written in red at the table below). Here's the table (note: I changed the original variable names used, so that they agreed with the names of the rest of the analysis above.)
$$
\begin{array}{l|c|c}
& \text{Alice for} & \text{Alice against} \\
\hline
\text{Bob for, Charlie for} & \text{not a bet}
&
\begin{array}{c}
& \text{Alice wants}\space 7:a, \space \color{green}{a<13}\\
& \text{Bob wants}\space y:3, \space \color{green}{y>3} \\
& \text{Charlie wants}\space z:4, \space \color{green}{z>6} \\
& \color{green}{a = y + z}
\end{array}\\
\hline
\text{Bob for, Charlie against}
&
\begin{array}{c}
& \text{Alice wants}\space x:7, \space \color{red}{x>13}\\
& \text{Bob wants}\space y:1, \space \color{red}{y>1} \\
& \text{Charlie wants}\space 8:c, \space \color{red}{c<12} \\
& \color{red}{x+y = c}
\end{array}
&
\begin{array}{c}
& \text{Alice wants}\space 7:a, \space \color{green}{a<13}\\
& \text{Bob wants}\space y:9, \space \color{green}{y>9} \\
& \text{Charlie wants}\space 2:c, \space \color{green}{c<3} \\
& \color{green}{a+c=y}
\end{array}\\
\hline
\text{Bob against, Charlie for}
&
\begin{array}{c}
& \text{Alice wants}\space x:7, \space \color{red}{x>13}\\
& \text{Bob wants}\space 9:b, \space \color{red}{b<9} \\
& \text{Charlie wants}\space z:2, \space \color{red}{z>3} \\
& \color{red}{x+z=b}
\end{array}
&
\begin{array}{c}
& \text{Alice wants}\space 7:a, \space \color{green}{a<13}\\
& \text{Bob wants}\space 1:b, \space \color{green}{b<1} \\
& \text{Charlie wants}\space z:8, \space \color{green}{z>12} \\
& \color{green}{a+b=z}
\end{array}\\
\hline
\text{Bob against, Charlie against}
& \begin{array}{c}
& \text{Alice wants}\space x:7, \space \color{red}{x>13}\\
& \text{Bob wants}\space 3:b, \space \color{red}{b<3} \\
& \text{Charlie wants}\space 4:c, \space \color{red}{c<6} \\
& \color{red}{x=b+c}
\end{array}
& \text{not a bet} \\
\end{array}
$$
Applying these arbitrary restrictions we can see that only three combinations (the ones with conditions in green) offer viable solutions. A minor side note: these restrictions were not so arbitrary. They were chosen so that we get integer values in the conditions, so we can check easily and quickly if the conditions hold.
If we take the first of these valid betting combinations: Alice against, Bob for, and Charlie for, and apply the rule of equal expected gains we get:
$$
\left.
\begin{array}{rl}
0.65\cdot 7 - 0.35\cdot a &= 0.5\cdot y - 0.5\cdot 3\\
0.5\cdot y - 0.5\cdot 4 &= 0.4 \cdot z - 0.6 \cdot 4\\
a = y + z
\end{array}
\right\} \iff
\begin{array}{rl}
a &= \frac{1179}{103} \approx 11.45\\
y &= \frac{421}{103} \approx 4.09\\
z &= \frac{758}{103} \approx 7.36
\end{array}
$$
(Solution to the above $3\times3$ system provided by this online tool)
Which means Alice is betting against with odds $7:11.45$, Bob betting for with odds $4.09:3$,and Charlie betting for with odds $7.36:4$. How much should each person put in, if the total initial bet is $\$100$?
For Alice it should be $\frac{100 \times 11.45}{11.45+3+4} = \$62.06 $, for Bob $\frac{100\times 3}{11.45+3+4} = \$16.26 $, and for Charlie $\frac{100\times 4}{11.45+3+4} = \$21.68$.
This solution corresponds to our general/parametric solution where $t = 21.68$
Best Answer
Addendum added to the end of my answer, to respond to the comment of Dmitrii I.
The confusion is actually between (for example) the quoted odds of
To a Mathematician, the first syntax above is the most rational, and expresses the amount that you receive upon winning, not including the return of your original bet.
Casinos like to seduce the betting public, who are generally non-Mathematicians. $6$ for $1$ sounds more attractive than its mathematically equivalent expression of $5$ to $1$.
When reading a book on odds, you have to be careful to decipher the author's intent. Authors (unfortunately) may be careless in not explicitly specifying whether the quoted odds of $n::1$ represent $n$ to $1$ or $n$ for $1$.
Addendum
You are assuming that what you read accurately expressed the intent. It is my contention that the text that you quoted intends a Mathematically valid exchange but is very poorly written.
It is my contention that the most likely intent is that you are the bookmaker. The bettor automatically gives you either $bS$ or $(1-b)S$, before the outcome of the event is determined.
Then, if the bettor wins, you pay the bettor $S$.
This means that if the bettor wins, he will receive $S$ back total after first giving you either $bS$ or $(1-b)S$. This is in fact standard practice with sportsbooks in Casinos. That is, the Sportsbooks never trust the bettor, but instead make the bettor put up the money that he is risking, in advance.
As an example, suppose that a sportbook requires you to lay $11$ to $10$ on a pick-em bet. This means that regardless of which side that you bet, you have to put up $11\$$ to win $10\$$.
Then, if you win, you receive $21\$$, which would represent a net profit to you of $10\$$, if you win. Horse racing is the exact same. You have to purchase the ticket in advance. If the horse that you bet on wins, then you can cash your ticket.
So, my answer reflected my surmise that the author that you quoted is not Mathematically inept, but (more likely) simply writes very poorly (or carelessly).
That is, I am surmising that the intent was that the bettor gives you either $bS$ or $(1-b)S$, in advance. Then, if he wins, he gets back $S$. Otherwise, he gets back nothing.
Assuming that my surmise is correct, the effect is the same as the bettor giving you the $bS$ or $(1-b)S$ only if he loses, and collecting from you $(1-b)S$ or $bS$ if he wins.
This is because (for example) the bettor paying $bS$ and then getting back $S$ is equivalent to the bettor winning $S - bS = (1-b)S.$
So, the theory indicated in my original answer is tenable and likely. That is, it is more likely that the author writes poorly, than that the author does not understand arithmetic.