I'm looking at some formulas involving matrices (in the context of machine learning, but I'm not sure it's relevant) and I came across $\odot$. What could this mean? The context is $M \odot N$, where $M$ is a matrix and $N$ might be a vector, or a matrix, or a scalar, it's a bit dense so it's hard to tell. I have reason to believe it may be the Hadamard product, is there anything else it could mean?
[Math] What does a dot in a circle mean
linear algebranotation
Related Solutions
By asking "What is important about homogeneous equations?" You come pretty close to asking "Why is linear algebra important?" Most every problem in linear algebra no matter how abstract at some point boils down to solving linear systems.
In general a linear system can be written in the form $A{\bf x} = {\bf b}$. If you take any two solutions ${\bf x}_1$ and ${\bf x}_2$ then ${\bf x}_1-{\bf x}_2$ is a solution of the corresponding homogeneous system $A{\bf x}={\bf 0}$. This is turn implies that if you find one solution ${\bf x}_p$ (a particular solution) of $A{\bf x} = {\bf b}$ and then find the general solution of the corresponding homogeneous system $A{\bf x}={\bf 0}$, say ${\bf x}_h$, then ${\bf x}={\bf x}_p+{\bf x}_h$ is the general solution of $A{\bf x}={\bf b}$. So in some sense the homogeneous solutions account for all of the redundant solutions of $A{\bf x}={\bf b}$ once you've found a particular solution.
If you have a linear transformation, say $T:V \to W$, then the kernel (or nullspace) of $T$ is the subspace $\mathrm{Ker}(T)=\{ v \in V \;|\; T(v)={\bf 0} \}$ (everything in $V$ that maps to the zero vector in $W$). If your linear transformation is $T(v)=Av$ for some matrix $A$, then the kernel of $T$ is nothing more than the null space of the matrix $A$. The range of $T$ is all of the vectors of $W$ that get mapped to: $\mathrm{Range}(T) = T(V) = \{ T(v) \;|\; v\in V\}$. Again if your transformation is $T(v)=Av$, then the range of $T$ is nothing more than the column space of the matrix $A$.
Again the kernel (a set of solutions of a homogeneous linear system) accounts for redundancies. If $T(v_1)=w=T(v_2)$ (i.e. $v_1$ and $v_2$ both map to the same output $w$), then $v_1-v_2 \in \mathrm{Ker}(T)$. So if $w \in T(V)$ (the range of $T$) and $v_p \in V$ is a vector such that $T(v_p)=w$, then $T(v_p+k)=w$ for any $k \in \mathrm{Ker}(T)$. Briefly, let $K=\mathrm{Ker}(T)$ and $v_p+K=\{v_p+k\;|\; k\in K\}$. Then $v_p+K$ is the set of all vectors which map to $w$.
In general, each element in the range of $T$ corresponds to a set of the form $v+K=v+\mathrm{Ker}(T)$ (these are called cosets of the kernel). So if we take $V$ and quotient out $K$ (whatever that means), denoted $V/K$, then we are left with a collection of sets which exactly correspond (i.e. isomorphic) with the range $T(V)$. This is written: $V/\mathrm{Ker}(T) = \mathrm{Range}(T)$. This result is known as the first ismorphism theorem. A quick consequence is that "rank plus nullity equals the dimension of the domain".
I know that doesn't complete answer your question, but maybe it'll get you started.
- I'm not sure what the fact that it's an endomorphism tells me. I understand that endomorphism means that it is mapping to the same vector space and that it is linear. Does that mean it's mapping to a vector subspace of this vector space? (since it's not bijective) But the transformation matrix seems to say otherwise.
But in this example, $\Phi$ is bijective (since the rank is 3, and the kernel is $0$). It's exactly as you said: "endomorphism" just means a linear map from a vector space to itself (there is no assumption on whether or not it is bijective...it might be, but it might not).
- I don't understand what the "(with respect to standard basis in R^3)" part means. Does it mean, the 2 basis sets we are mapping are wrt. to the standard basis, or that we are mapping the standard basis to a new basis? Or something else?
That's right: it means both the domain and range are written in the standard basis. If we call them $$e_1 = \begin{pmatrix}1\\0\\0\end{pmatrix}, \quad e_2 = \begin{pmatrix} 0\\1\\0 \end{pmatrix}, \quad e_3 = \begin{pmatrix}0\\0\\1\end{pmatrix} $$ then saying $A_\Phi$ represents $\Phi$ in the standard basis means $$ \begin {align*} \Phi(e_1) &= e_1+e_2+e_3 \\ \Phi(e_2) &= e_1-e_2+e_3 \\ \Phi(e_3) &= e_3 \end{align*} $$ This is just obtained from looking at the columns of $A_\Phi$ (the coefficients of $e_1,e_2,e_3$ on the right-hand sides are the columns of the matrix).
- Finally I don't understand what the b. part is asking me to do. I don't understand the "with respect to the basis B" part either.
Now, if we write the new basis as $$ v_1 = \begin{pmatrix}1\\1\\1\end{pmatrix}, \quad v_2 = \begin{pmatrix}1\\2\\1 \end{pmatrix}, \quad v_3 = \begin{pmatrix} 1\\0\\0\end{pmatrix} $$ Then the new matrix $\widetilde{A}_\Phi$ for $\Phi$ in the new basis is $$ \widetilde{A}_\Phi = \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} $$ where these coefficients satisfy $$ \begin{align*} \Phi(v_1) &= a_{11}v_1 + a_{21}v_2 + a_{31}v_3 \\ \Phi(v_2) &= a_{12}v_1 + a_{22}v_2 + a_{32} v_3 \\ \Phi(v_3) &= a_{13}v_1 + a_{23}v_2 + a_{33}v_3 \end {align*} $$
So part $(b)$ is asking you to find these coefficients $a_{ij}$ of this matrix $\widetilde{A}_\Phi$.
The formula you gave is the correct one: $\widetilde{A}_\Phi = T^{-1} A_\Phi T$, where $T$ is the matrix whose columns are $v_1,v_2,v_3$: $$ T = \begin{pmatrix} 1&1&1\\1&2&0\\1&1&0 \end{pmatrix} $$
Best Answer
In the LSTM equations, the circled dot operator is typically used to represent element-wise multiplication.