Element-wise Operations Notation – Understanding Pointwise Operations

hadamard-productnotation

Is there a notation for element-wise (or pointwise) operations?

For example, take the element-wise product of two vectors x and y (in Matlab, x .* y, in numpy x*y), producing a new vector of same length z, where $z_i = x_i * y_i$ .

In mathematical notation, there doesn't seem to be a standard for this, am I wrong?

There is $x \cdot y$, the dot product. There is $x*y$, which is usually considered the cross product. I need to find a notation for element-wise multiplication. I was aiming at maybe using the . as is done in Matlab, but it looks a little off :

$z = x .* y$

What do you think?

Best Answer

I've seen several conventions, including $\cdot$, $\circ$, $*$, $\otimes$, and $\odot$. However, most of these have overloaded meanings (see http://en.wikipedia.org/wiki/List_of_mathematical_symbols).

  • $\times$ (\times) -- cross product or cartesian product.
  • $*$ (*) -- convolution.
  • $\cdot$ (\cdot) -- dot product
  • $\bullet$ (\bullet) -- dot product
  • $\otimes$ (\otimes) -- tensor product.
  • $\circ$ (\circ) -- function composition. Not a problem for vectors, but can be ambiguous for matrices.

Thus, in my personal experience, the best choice I've found is:

  • $\odot$ (\odot) -- to me the dot makes it look naturally like a multiply operation (unlike other suggestions I've seen like $\diamond$) so is relatively easy to visually parse, but does not have an overloaded meaning as far as I know.

Also:

  • This question comes up often in multi-dimensional signal processing, so I don't think just trying to avoid vector multiplies is an appropriate notation solution. One important example is when you map from discrete coordinates to continuous coordinates by $x = i \odot\Delta + b$ where $i$ is an index vector, $\Delta$ is sample spacing (say in mm), $b$ is an offset vector, and $x$ is spatial coordinates (in mm). If sampling is not isotropic, then $\Delta$ is a vector and element-wise multiplication is a natural thing to want to do. While in the above example I could avoid the problem by writing $x_k = i_k \Delta_k + b_k$, having a symbol for element-wise multiplication lets us mix and match matrix multiplies and elementwise multiplies, for example $y = A(i \odot \Delta) + b$.
  • Another alternative notation I've seen for $z = x \odot y$ for vectors is $z = $ diag$(x) y$. While this technically works for vectors, I find the $\odot$ notation to be far more intuitive. Furthermore, the "diag" approach only works for vectors -- it doesn't work for the Hadamard product of two matrices.
  • Often I have to play nicely with documents that other people have written, so changing the overloaded operator (like changing dot products to $\left< \cdot , \cdot \right>$ notation) often isn't an option, unfortunately.

Thus I recommend $\odot$, as it is the only option I have yet to come across that has seems to have no immediate drawbacks.

Related Question