I was reading a book on deep learning [1] and came across an equation like this:
$$ C_{i,j} = A_{i,j} + b_{j} $$
This is an equation for showing how an addition of a matrix and a vector is done. But I think instead of $b_j$ it should be $b_i$ since vectors doesn't have a $j$ index. Please tell me if I am wrong and why? Thanks.
Quote from the book:
In the context of deep learning, we also use some less conventional notation.
We allow the addition of matrix and a vector, yielding another matrix $$C = A + b$$ where $ C_{i,j} = A_{i,j} + b_{j} $. In other words, the vector $b$ is added to each row of the matrix. This shorthand eliminates the need to define a matrix with $b$ copied into each row before doing the addition. This implicit copying of $b$ to many locations is called broadcasting.
[1] Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron, Deep learning, Adaptive Computation and Machine Learning. Cambridge, MA: MIT Press (ISBN 978-0-262-03561-3/hbk; 978-0-262-33743-4/ebook). xxii, 775 p. (2016). ZBL1373.68009. Free Online Version and Errata on https://www.deeplearningbook.org
Best Answer
I suspect that the author is thinking in Python. The proper matrix notation would be $$ {\bf C} = {\bf A} + {\bf 1} {\bf b}^\top $$