Dual numbers and hyperdual space

Dual number theory

Dual numbers are a type of complex numbers. The ubiquitous set of complex numbers, $\mathbb{C}$, may be defined as follows, where $i$ is the imaginary number:

$$\mathbb{C}=\mathbb{R}[i]=\{z=a+bi|(a,b)\in {\mathbb{R}}^2,i^2=-1\}$$

Similarly, we may define the set of dual numbers as follows, where $ϵ$ is the dual number:

$$\mathbb{D}=\mathbb{R}[ϵ]=\{z=a+bϵ|(a,b)\in {\mathbb{R}}^2,{ϵ}^2=0\text{~and~}ϵ\ne 0\}$$

Moreover, for $z=a+bϵ$ where $z\in \mathbb{D},(a,b)\in \mathbb{R}$, let us define the $real$ and $dual$ parts of a dual number such as

$$\{\begin{array}{l}real(z)=a\\ dual(z)=b\end{array}$$

An auto-differentiation property emerges from the addition of this nilpotent element, as is obvious from a Taylor series expansion. Evidently, this result is only valid for values of $a$ where the function is differentiable.

$$\begin{array}{rl}f:\mathbb{D}\to \mathbb{D},(a,b)\in {\mathbb{R}}^2& \\ f(a+b\epsilon )& =\sum _{n=0}^{\mathrm{\infty }}\frac{f^{(n)}(a)b^n{\epsilon }^n}{n!}\\ & =f(a)+b\frac{df(a)}{da}\epsilon \\ & \{\begin{array}{l}real(f(a+bϵ))=f(a)\\ dual(f(a+bϵ))=b\frac{df(a)}{da}\end{array}\end{array}$$

By choosing $b=1$, the first derivative comes out for free by simply evaluating the function $f$.

Hyperdual spaces

We can further extend the dual numbers to a hyperdual space. Let us define a hyperdual space of size 2 as follows, where ${ϵ}_j$ is the $j$-th dual number:

$${\mathbb{D}}^2=\mathbb{R}[{ϵ}_x,{ϵ}_y]=\{z=a+b{ϵ}_x+c{ϵ}_y+d{ϵ}_x{ϵ}_y|(a,b,c,d)\in {\mathbb{R}}^4,{ϵ}_{\gamma }^2=0,{ϵ}_{\gamma }\ne 0,\gamma \in \{x,y\},{ϵ}_x{ϵ}_y\ne 0\}$$

This mathematical tool enables auto-differentiation of multi-variate functions as follows, where $dua{l}_{\gamma }$ corresponds to the $\gamma$-th dual number, i.e. the number associated with ${ϵ}_{\gamma }$.

$$\begin{array}{rl}f:{\mathbb{D}}^2\to {\mathbb{D}}^2,(x,y)\in {\mathbb{R}}^2& \\ & \{\begin{array}{l}real(f(x+{ϵ}_x,y+{ϵ}_y))=f(x,y)\\ dua{l}_x(f(x+{ϵ}_x,y+{ϵ}_y))=\frac{\partial }{\partial x}f(x,y)\\ dua{l}_y(f(x+{ϵ}_x,y+{ϵ}_y))=\frac{\partial }{\partial y}f(x,y)\end{array}\end{array}$$

Example

Let us detail a computation example of a smooth multivariate polynomial function defined over all reals.

$$\begin{array}{rl}f:\mathbb{R}\to \mathbb{R},(x,y)\in {\mathbb{R}}^2& \\ & f(x,y)=2x^3-0.2y^2+x\\ & \frac{\partial }{\partial x}f(x,y)=6x^2+1\\ & \frac{\partial }{\partial y}f(x,y)=-0.4y\end{array}$$

Let us extend the definition of this function to ${\mathbb{D}}^2$.

$$\begin{array}{rl}g:\mathbb{D}\to \mathbb{D},(x,y)\in {\mathbb{R}}^2& \\ & g(x+{ϵ}_x,y+{ϵ}_y)=2(x+{ϵ}_x{)}^3-0.2(y+{ϵ}_y{)}^2+(x+{ϵ}_x)\end{array}$$

Trivially,

$$\begin{array}{rl}(x+{ϵ}_x{)}^2& =x^2+2x{ϵ}_x\\ (x+{ϵ}_x{)}^3& =x^3+3x^2{ϵ}_x\end{array}$$

Hence, $g$ may be written as follows:

$$\begin{array}{rl}g(x+{ϵ}_x,y+{ϵ}_y)& =2(x+{ϵ}_x{)}^3-0.2(y+{ϵ}_y{)}^2+(x+{ϵ}_x)\\ & =2(x^3+3x^2{ϵ}_x)-0.2(y^2+2y{ϵ}_y)+(x+{ϵ}_x)\\ & =(2x^3-0.2y^2+x)+(6x^2+1){ϵ}_x-0.4y{ϵ}_y\end{array}$$

As expected, the $dua{l}_x$ part of $g$ corresponds to the partial of $f$ with respect to $x$, the $dua{l}_y$ part of $g$ corresponds to the partial of $f$ with respect to $y$, and the $real$ part of the $g$ corresponds to $f$.

Computation

All hyperdual computations are handled by the hyperdual Rust crate, co-authored by Chris Rabotin.