Convolution

The Definition

The integral

$$\int g(t-\tau )h(\tau) d\tau = g \otimes h$$

is called the convolution of \(g\) and \(h\). Here the \(\otimes\) denotes
convolution, it is sometimes represented by a \(\star\). This is representing the
amount of overlap of function \(h\) as \(g\) is shifted over it. It in effect
blends one function with another. This crops up a lot in physics and in
statistics.

The Fourier Transform of a Convolution

If \(F(u)\) is the Fourier transform of \(f(x)\) and \(G(u)\) is the Fourier
transform of \(g(x)\), multiplying both Fourier transforms together we have

$$F(u)G(u) = \int_{-\infty}^{\infty}f(x)e^{-2 \pi iu x} dx \int_{-\infty}^{\infty}g(x)e^{-2 \pi i u x} dx$$

So that we can rewrite this as a double integral we replace the \(x\) with
two dummy variables and rewrite as

$$F(u)G(u) = \int_{-\infty}^{\infty}f(\tau)e^{-2 \pi i u \tau} d \tau \int_{-\infty}^{\infty}g(\upsilon)e^{-2 \pi i u \upsilon} d
\upsilon$$

$$= \int_{-\infty}^{\infty} \int_{-\infty}^{\infty}e^{-2 \pi i u(\tau + \upsilon)}f(\tau)g(\upsilon)d \tau d
\upsilon$$

Next, making a change of variables so \(x= \tau+\upsilon ,dx=d \upsilon\)

$$F(u)G(u) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(\tau) g(x-\tau) d \tau dx$$

$$=\int_{-\infty}^{\infty} e^{-i2 \pi x }[ \int_{-\infty}^{\infty}f(\tau) g(x-\tau) d\tau ] dx$$

$$=\int_{-\infty}^{\infty}e^{-i2 \pi x} f(x) \otimes g(x) dx$$

i.e., \(F(u)G(u)\) and \(f(x) \otimes g(x)\) are Fourier transform pairs. This
result is very important in optical processing, and signal processing in
general. It means a convolution can be calculated by Fourier transforming
both signals, multiplying the result and inverse transforming back. This is
often quicker than calculating the convolution directly and can be performed
using a simple system of lenses and filters.

Properties

Commutativity

$$f \otimes g = g \otimes f$$

Associativity

$$f \otimes (g \otimes h) = (f \otimes g) \otimes h$$

Distributivity

$$f \otimes (g+h)= (f \otimes g) + (f \otimes h)$$

Scalar multiplication

$$a(f \otimes g) = (af) \otimes g = f \otimes (ag)$$

where \(a \in \mathbb{C}\)

Differential rule

If \(\partial\) is a differential operator

$$\partial (f \otimes g) = \partial f \otimes g = f \otimes \partial g$$

A graphical example of convolution/correlation