The so-called completing the square is a method for solving quadratic equations of the form
\[ \begin{align} ax^2 + bx + c = 0 \end{align} \]
with $a \not= 0$. With the definitions
\[ \begin{align} p \coloneqq \frac{b}{a},& {} & q \coloneqq \frac{c}{a} \end{align} \]
one can write this in the form
\[ \begin{align} x^2 + px + q = 0 \end{align} \]
Completing the square gets its name from the fact that a quadratic term is now added to this equation and immediately subtracted again:
\[ \begin{align} x^2 + px + \left(\frac{p}{2}\right)^2 - \left(\frac{p}{2}\right)^2 + q &= 0\nonumber\\ \Leftrightarrow\left(x + \frac{p}{2}\right)^2 - \left(\frac{p}{2}\right)^2 + q &= 0\nonumber\\ \Leftrightarrow\left(x + \frac{p}{2}\right)^2 &= \left(\frac{p}{2}\right)^2 - q \end{align} \]
From this it follows
This formula is called the pq-formula.
First, the Gaussian sum formula is examined. It reads
\[ \begin{align} \sum_{i = 0}^{n}i = \frac{n\left(n + 1\right)}{2}\tag{A.6}\label{eq:kleiner_gauss}. \end{align} \]
One proves this by complete induction. For $n = 0$, $\sum_{i = 0}^{n = 0}i = 0 = \frac{1}{2}n\left(n + 1\right)$. Suppose the statement already holds for $n\in\mathbb{N}$. Then it follows
\[ \begin{align} \sum_{i = 0}^{n + 1}i = \frac{n\left(n + 1\right)}{2} + n + 1 = \frac{n^2 + 3n + 2}{2} = \frac{\left(n + 1\right)\left(n + 2\right)}{2}. \end{align} \]
This proves the statement.
The theorem on the geometric series reads
\[ \begin{align} \sum_{n = 0}^{\infty}q^n = \frac{1}{1 - q}\tag{A.8}\label{eq:geometr_reihe} \end{align} \]
for $q\in \mathbb{C}$ with $\left|q\right| <1$.
First, for $N\in \mathbb {N}$ one has
\[ \begin{align} \sum_{n = 0}^{N}q^n = \frac{1 - q^{N + 1}}{1 - q}. \end{align} \]
This is shown by complete induction. For $N = 0$ this is clear; suppose the statement holds for $N$. Then
\[ \begin{align} \sum_{n = 0}^{N + 1}q^n = \frac{1 - q^{N + 1}}{1 - q} + q^{N + 1} = \frac{1 - q^{N + 1}}{1 - q} + \frac{q^{N + 1} - q^{N + 2}}{1 - q} = \frac{1 - q^{N + 2}}{1 - q}. \end{align} \]
With the arithmetic rules for limits of sequences and
\[ \begin{align} \sum_{n = 0}^{\infty}q^n = \lim\limits_{N\to\infty}\sum_{n = 0}^{N}q^n \end{align} \]
the statement follows. Furthermore, according to [11], one has
\[ \begin{align} \sum_{n = 1}^{\infty}\frac{1}{n^4} = \frac{\pi^4}{90}.\tag{A.12}\label{eq:reihe_1} \end{align} \]
One defines the binomial coefficient
\[ \begin{align} \left(\begin{array}{c} n\\ k \end{array}\right) \coloneqq \frac{n!}{k!\left(n - k\right)!} \end{align} \]
with the factorial
\[ \begin{align} n! \coloneqq 1\cdot 2\cdot\dotsc\cdot n \end{align} \]
with $n, k\in \mathbb{N}$ and $n\geq k$. The number of ways to arrange $n\in \mathbb{N}$ elements is $n!$. For the first element there are $n$ possibilities, for the second only $n - 1$, and so on; for the last element there is exactly one way to arrange it. A permutation is a bijection on the set $\{1, 2, \dotsc, N\}$ with $1 \leq N \in \mathbb{N}$. The set of all these mappings is denoted by $S_N$. By the statement just proven, the number $\left|S_N\right|$ of all permutations satisfies
\[ \begin{align} \left|S_N\right| = N!. \end{align} \]
One further defines the sign of a permutation $\pi \in S_N$ by $\sign\left(\pi\right) \coloneqq \left(-1\right)^M$, where $M$ is the number of all pairwise transpositions required to transform $\id \to \pi$. If $M$ is even, $\pi$ is called an even permutation, otherwise odd.
$\left(\begin{array}{c} n\\ k \end{array}\right)$ is the number of $k$-element subsets of an $n$-element set. This is easy to verify. Let an $n$-element set $M$ be given and let $k\in\mathbb{N}$ with $k\leq n$. In the case $k = 0$, one has $\left(\begin{array}{c} n\\ k \end{array}\right) = 1$, because the only possible set is the empty set $\emptyset$, which is a subset of every set, including itself. For $n, k>0$, there are $n$ possibilities for choosing the first element of the subset, $n-1$ for the second, and so on. This yields $n\cdot\left(n - 1\right)\cdot\dotsc\cdot\left(n - k + 1\right) = \frac{n!}{\left(n - k\right)!}$. However, each subset is counted multiple times this way. Since there are $k!$ ways to order $k$ elements, the number of $k$-element subsets of an $n$-element set is
\[ \begin{align} \frac{n!}{k!\left(n - k\right)!} = \left(\begin{array}{c} n\\ k \end{array}\right). \end{align} \]
Let $n, n_1, \dotsc, n_k\in \mathbb{N}$ with $\sum_{i = 1}^{k}n_i = n$. One defines the multinomial coefficient by
\[ \begin{align} \left(\begin{array}{c} n\\ n_1, \dotsc, n_k \end{array}\right) \coloneqq \frac{n!}{n_1!\dotsc n_k!}.\tag{A.17}\label{def:multinomialkoeffizient} \end{align} \]
Let a set with $k$ elements be given. Draw from it $n$ times at random (where either $n\geq k$ or $n\leq k$ is possible). Record the counts $n_1, \dotsc, n_k$, which specify how often each element was drawn; the outcome is therefore the tuple $\left(n_1, \dotsc, n_k\right)$. The multinomial coefficient
\[ \begin{align} \left(\begin{array}{c} n\\ n_1, \dotsc, n_k \end{array}\right) \end{align} \]
is the number of draws producing the outcome $\left(n_1, \dotsc, n_k\right)$. The quantity $n!$ counts all permutations of the $n$ drawn elements. For each such arrangement, identical elements can be permuted among themselves without changing the outcome. The number of such internal permutations is $n_1!\dotsc n_k!$.
The binomial theorem, also known as the general binomial formula, is
\[ \begin{align} \left(a + b\right)^n = \sum_{k = 0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)a^kb^{n - k}.\tag{A.19}\label{eq:gen_bin_formula} \end{align} \]
For $n = 0$ this is clear. Furthermore, one has
\[ \begin{align} \left(a + b\right)^{n + 1} &= \left(a + b\right)^n\left(a + b\right) = \left(a + b\right)\sum_{k = 0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)a^kb^{n - k}\nonumber\\ &= \sum_{k = 0}^{n}\frac{n!}{k!\left(n - k\right)!}a^{k + 1}b^{n - k} + \sum_{k = 0}^{n}\frac{n!}{k!\left(n - k\right)!}a^kb^{n - k + 1}\nonumber\\ &= \sum_{k = 1}^{n + 1}\frac{n!}{\left(k - 1\right)!\left(n + 1 - k\right)}a^kb^{n - k + 1} + \sum_{k = 0}^{n}\frac{n!}{k!\left(n - k\right)!}a^kb^{n - k + 1}\nonumber\\ &= \sum_{k = 1}^{n + 1}\frac{\left(n + 1\right)!}{k!\left(n + 1 - k\right)!}\frac{k}{n + 1}a^kb^{n - k + 1}\nonumber\\ & + \sum_{k = 0}^{n}\frac{\left(n + 1\right)!}{k!\left(n + 1 - k\right)!}\frac{n + 1 - k}{n + 1}a^kb^{n - k + 1}\nonumber\\ &= \sum_{k = 0}^{n + 1}\frac{\left(n + 1\right)!}{k!\left(n + 1 - k\right)!}a^kb^{n - k + 1}\left(\frac{k + n + 1 - k}{n + 1}\right)\nonumber\\ &= \sum_{k = 0}^{n + 1}\frac{\left(n + 1\right)!}{k!\left(n + 1 - k\right)!}a^kb^{n + 1 - k} = \sum_{k = 0}^{n + 1}\left(\begin{array}{c} n + 1\\ k \end{array}\right)a^kb^{n + 1 - k}. \end{align} \]
Let $\left(x_1, x_2, x_3\right)$ be three Cartesian coordinates and $\left(\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\right)$ be the corresponding basis. One defines the Levi-Civita symbol $\epsilon_{i, j, k}$ by
\[ \begin{align} \epsilon_{i, j, k} \coloneqq \begin{cases} 1, \:\left(i, j, k\right)\text{ even permutation of }\left(1, 2, 3\right),\\ - 1, \:\left(i, j, k\right)\text{ odd permutation of }\left(1, 2, 3\right),\\ 0, \:\text{otherwise}. \end{cases} \end{align} \]
This means in particular
\[ \begin{align} \epsilon_{i, j, k} &= \epsilon_{k, i, j} = \epsilon_{j, k, i} = -\epsilon_{j, i, k} = -\epsilon_{k, j, i} = -\epsilon_{i, k, j},\\ \epsilon_{i, i, j} &= \epsilon_{i, j, i} = \epsilon_{j, i, i} = 0. \end{align} \]
The following holds
\[ \begin{align} & \sum_{n = 1}^{3}\epsilon_{l, n, k}\epsilon_{n, o, j} - \epsilon_{l, n, j}\epsilon_{n, o, k} = \sum_{n = 1}^{3}\epsilon_{l, n, k}\epsilon_{j, n, o} + \epsilon_{l, n, j}\epsilon_{k, o, n} = \sum_{n = 1}^{3}\epsilon_{j, o, n}\epsilon_{l, k, n} + \epsilon_{j, l, n}\epsilon_{k, o, n}\nonumber\\ &= \sum_{n = 1}^{3}\epsilon_{j, o, n}\epsilon_{l, k, n} - \epsilon_{j, l, n}\epsilon_{o, k, n}.\tag{A.24}\label{eq:levi-civita_prop_1} \end{align} \]
Here one must distinguish cases. For the left-hand expression to be nonzero, either $j = l$ and $o = k$ or $j = k$ and $o = l$ must hold. The left-hand expression can therefore be written as $\delta_{j, l}\delta_{o, k} - \delta_{j,k}\delta_{o,l}$. For the right-hand expression to be nonzero, either $j = o$ and $l = k$ or $j = k$ and $l = o$ must hold. The right-hand expression can therefore be written as $-\delta_{j, o}\delta_{l, k} + \delta_{j,k}\delta_{o,l}$. One can therefore continue
\[ \begin{align} \sum_{n = 1}^{3}\epsilon_{j, o, n}\epsilon_{l, k, n} - \epsilon_{j, l, n}\epsilon_{o, k, n} &= \delta_{j, l}\delta_{o, k} - \delta_{j, o}\delta_{l, k} = \sum_{n = 1}^{3}\epsilon_{j, k, n}\epsilon_{l, o, n}, \end{align} \]
as one readily verifies.
Let a square matrix $A \in \mathbb {C}^{N \times N}$ be given with $1 \leq N \in \mathbb {N}$. The determinant $\det\left(A\right)$ is defined by
\[ \begin{align} \det\left(A\right) \coloneqq \sum_{\pi \in S_N}\sign\left(\pi\right)A_{1, \pi\left(1\right)}\cdot\dotsc\cdot A_{N, \pi\left(N\right)}. \end{align} \]
Occasionally, the notation is also used
\[ \begin{align} \left|\begin{array}{ccc} A_{1, 1} & \dots & A_{1, N} \\ \vdots & \ddots & \vdots \\ A_{N, 1} & \dots & A_{N, N} \end{array}\right| \coloneqq \det\left(A\right). \end{align} \]
First, one has
\[ \begin{align} \det\left(A^T\right) &= \sum_{\pi \in S_N}\sign\left(\pi\right)A^T_{1, \pi\left(1\right)}\cdot\dotsc\cdot A^T_{N, \pi\left(N\right)} = \sum_{\pi \in S_N}\sign\left(\pi\right)A_{\pi\left(1\right), 1}\cdot\dotsc\cdot A_{\pi\left(N\right), N}. \end{align} \]
By sorting each individual summand by ascending row indices, one obtains
\[ \begin{align} \det\left(A^T\right) &= \sum_{\pi \in S_N}\sign\left(\pi\right)A_{\pi\left(1\right), 1}\cdot\dotsc\cdot A_{\pi\left(N\right), N} = \sum_{\pi \in S_N}\sign\left(\pi\right)A_{1, \pi\left(1\right)}\cdot\dotsc\cdot A_{N, \pi\left(N\right)} = \det\left(A\right). \end{align} \]
All following statements are therefore only proven for rows; the statement for columns then follows analogously. If two rows $1 \leq i, j \leq N$ of $A$ are equal, then already
\[ \begin{align} \det\left(A\right) = 0. \end{align} \]
To this end, sort the elements $\pi \in S_N$ into $N!/2$ pairs $\left(\pi, \pi'\right)$ such that $\pi'$ arises from $\pi$ by interchanging $\pi\left(i\right)$ and $\pi\left(j\right)$. In this case, $\sign\left(\pi\right) = -\sign\left(\pi'\right)$, which proves the statement. If the $j-$th row arises from the $i-$th row through multiplication by a complex constant $C \in \mathbb{C}$, i.e. $A_{j, k} = CA_{i, k}$ for all $1 \leq k \leq N$, then it likewise follows
\[ \begin{align} \det\left(A\right) = 0, \end{align} \]
by simply pulling $C$ out in front of the sum defining the determinant. If the $j-$th row is a linear combination of the other $N - 1$ rows, i.e. there exist constants $C_l \in \mathbb{C}$ for $1 \leq l \leq N$ with $l \not= j$ such that $A_{j, k} = \sum_{\substack{l = 1,\\l \not = j}}^NC_lA_{l, k}$ for all $1 \leq k \leq N$, then in this case too one has
\[ \begin{align} \det\left(A\right) = 0. \end{align} \]
To see this, one simply applies the previously proven statement $N - 1$ times.
The concept of a function is assumed to be known. Let $K\in\{\mathbb{R}, \mathbb{C}\}$ and $a_n\in K$ for all $n\in\mathbb{N}$. A power series $p$ about the expansion point $x_0\in K$ is a series of the form
\[ \begin{align} p\left(x\right) = \sum_{n = 0}^{\infty}a_n\left(x - x_0\right)^n. \end{align} \]
For its derivatives the following holds (recall the sum rule)
\[ \begin{align} p'\left(x\right) &= \sum_{n = 0}^{\infty}a_nn\left(x - x_0\right)^{n - 1} = \sum_{n = 1}^{\infty}a_nn\left(x - x_0\right)^{n - 1}\sum_{n = 0}^{\infty}a_{n + 1}(n + 1)\left(x - x_0\right)^{n},\tag{A.34}\label{eq:1st_derivative_series}\\ \frac{d^2}{dx^2}\sum_{n = 0}^{N}a_ix^i &= \sum_{n = 0}^{\infty}\left(n + 2\right)\left(n + 1\right)a_{n + 2}\left(x - x_0\right)^n,\\ \frac{d^m}{dx^m}\sum_{n = 0}^{N}a_ix^i &= \sum_{n = 0}^{\infty}\frac{\left(n + m\right)!}{n!}a_{n + m}\left(x - x_0\right)^n\tag{A.36}\label{eq:nth_derivative_series} \end{align} \]
These formulas are also applicable to terminating power series, so-called polynomials.
Let $K\in\{\mathbb{R}, \mathbb{C}\}$ and $f:K\to K$ be a function. The function $f$ is called locally analytic at a point $x_0\in K$ if there is a neighborhood M of $x_0$ and a power series $p$ about the expansion point $x_0$ such that for all $x\in M$ one has
\[ \begin{align} f\left(x\right) = p\left(x\right). \end{align} \]
A function is called analytic if it is locally analytic at every point. There are some standard power series called the basic functions:
Polynomials.
Rational functions (fractions of polynomials).
The exponential function
\[ \begin{align} \exp\left(x\right) \coloneqq \sum_{n = 0}^{\infty}\frac{x^n}{n!}. \end{align} \]
The natural logarithm function
\[ \begin{align} \ln \coloneqq \exp^{-1}. \end{align} \]
The hyperbolic functions
\[ \begin{align} \sinh\left(x\right) \coloneqq \frac{\exp\left(x\right) - \exp\left(-x\right)}{2} & {} & \cosh\left(x\right) \coloneqq \frac{\exp\left(x\right) + \exp\left(-x\right)}{2} & {} & \tanh \coloneqq \frac{\sinh}{\cosh}. \end{align} \]
The correspondingly restricted inverse functions of the hyperbolic functions, the so-called area functions
\[ \begin{align} \arsinh \coloneqq \sinh^{-1}, & {} & \arcosh \coloneqq \cosh^{-1}, & {} & \artanh \coloneqq \tanh^{-1}. \end{align} \]
The trigonometric functions
\[ \begin{align} \sin\left(x\right) \coloneqq -i\sinh\left(ix\right), & {} & \cos\left(x\right) \coloneqq \cosh\left(ix\right), & {} & \tan \coloneqq \frac{\sin}{\cos}. \end{align} \]
The correspondingly restricted inverse functions of the trigonometric functions, the so-called arcus functions
\[ \begin{align} \arcsin \coloneqq \sin^{-1}, & {} & \arccos \coloneqq \cos^{-1}, & {} & \arctan \coloneqq \tan^{-1} \end{align} \]
The exponential function satisfies the computational rule
\[ \begin{align} \exp\left(x + y\right) & \stackrel{\href{#eq:gen_bin_formula}{\text{Eq. (A.19)}}}{=} \sum_{k = 0}^\infty\frac{1}{k!}\sum_{l = 0}^k\left(\begin{array}{c} k\\ l \end{array}\right)x^ly^{k - l} = \sum_{k = 0}^\infty\frac{1}{k!}\sum_{l = 0}^k\frac{k!}{l!\left(k - l\right)!}x^ly^{k - l} = \sum_{k = 0}^\infty\sum_{l = 0}^k\frac{1}{l!\left(k - l\right)!}x^ly^{k - l}\nonumber\\ &= \sum_{l = 0}^\infty\sum_{k = l}^\infty\frac{1}{l!\left(k - l\right)!}x^ly^{k - l} = \sum_{l = 0}^\infty\frac{x^l}{l!}\sum_{k = 0}^\infty\frac{y^k}{k!} = \exp\left(x\right)\exp\left(y\right).\tag{A.44}\label{eq:exp_prop_1} \end{align} \]
From this it follows
\[ \begin{align} 1 = \exp\left(x - x\right) = \exp\left(x\right)\exp\left(-x\right) \Rightarrow \exp\left(-x\right) = \frac{1}{\exp\left(x\right)}.\tag{A.45}\label{eq:exp_prop_2} \end{align} \]
One has
\[ \begin{align} \exp\left(x\right) > 0\tag{A.46}\label{eq:exp_prop_3} \end{align} \]
for all $x \in \mathbb{R}.$ For $x \geq 0$ this is clear. For $x < 0$ one has
\[ \begin{align} \exp\left(x\right) = \frac{1}{\exp\left(-x\right)} > 0. \end{align} \]
Thus, the following hold
\[ \begin{align} \cosh^2\left(x\right) &= \frac{1}{4}\left(e^{2x} + e^{-2x} + 2\right) = \frac{1}{4}\left(e^{2x} + e^{-2x} - 2\right) + 1 = \sinh^2\left(x\right) + 1,\\ \sin^2\left(x\right) + \cos^2\left(x\right) &= -\sinh^2\left(ix\right) + \cosh^2\left(ix\right) = 1,\\ \tanh'&= \frac{\cosh^2 - \sinh^2}{\cosh^2} = 1 - \tanh^2 = \frac{1}{\cosh^2},\\ \tan' &= \frac{\cos^2 + \sin^2}{\cos^2} = 1 + \tan^2 = \frac{1}{\cos^2}. \end{align} \]
The basic functions satisfy the following symmetry properties:
\[ \begin{align} \sinh\left(-x\right) &= \frac{e^{-x} - e^x}{2} = -\frac{e^{x} - e^{-x}}{2} = -\sinh\left(x\right)\\ \cosh\left(-x\right) &= \frac{e^{-x} + e^x}{2} = \frac{e^{x} + e^{-x}}{2} = \cosh\left(x\right)\\ \Rightarrow \tanh\left(-x\right) &= -\tanh\left(x\right)\\ \sin\left(-x\right) &= -i\sinh\left(-ix\right) = i\sinh\left(ix\right) = -\left(-i\sinh\left(ix\right)\right) = -\sin\left(x\right)\\ \cos\left(-x\right) &= \cosh\left(-ix\right) = \cosh\left(ix\right) = \cos\left(x\right)\\ \Rightarrow \tan\left(-x\right) &= -\tan\left(x\right) \end{align} \]
The so-called addition theorems hold
\[ \begin{align} \sinh\left(x + y\right) &= \frac{e^xe^y - e^{-x}e^{-y}}{2} = \frac{1}{4}\left(2e^xe^y - 2e^{-x}e^{-y}\right) = \frac{1}{4}\left(\left(e^x - e^{-x}\right)\left(e^y + e^{-y}\right) + \left(e^y - e^{-y}\right)\left(e^x + e^{-x}\right)\right)\nonumber\\ &= \sinh\left(x\right)\cosh\left(y\right) + \cosh\left(x\right)\sinh\left(y\right), \tag{A.58}\label{eq:sinh_add_1}\\ \cosh\left(x + y\right) &= \frac{e^xe^y + e^{-x}e^{-y}}{2} = \frac{1}{4}\left(2e^xe^y + 2e^{-x}e^{-y}\right) = \frac{1}{4}\left(\left(e^x + e^{-x}\right)\left(e^y + e^{-y}\right) + \left(e^y - e^{-y}\right)\left(e^x - e^{-x}\right)\right)\nonumber\\ &= \cosh\left(x\right)\cosh\left(y\right) + \sinh\left(x\right)\sinh\left(y\right), \tag{A.59}\label{eq:cosh_add_1}\\ \tanh\left(x + y\right) &= \frac{\sinh\left(x + y\right)}{\cosh\left(x + y\right)} = \frac{\sinh\left(x\right)\cosh\left(y\right) + \cosh\left(x\right)\sinh\left(y\right)}{\cosh\left(x\right)\cosh\left(y\right) + \sinh\left(x\right)\sinh\left(y\right)} = \frac{\tanh\left(x\right)\cosh\left(y\right) + \sinh\left(y\right)}{\cosh\left(y\right) + \tanh\left(x\right)\sinh\left(y\right)}\nonumber\\ &= \frac{\tanh\left(x\right) + \tanh\left(y\right)}{1 + \tanh\left(x\right)\tanh\left(y\right)}.\tag{A.60}\label{eq:tanh_add_1} \end{align} \]
Applying this to the trigonometric functions gives
\[ \begin{align} \sin\left(x + y\right) &= -i\sinh\left(ix + iy\right) = -i\sinh\left(ix\right)\cosh\left(iy\right) - i\cosh\left(ix\right)\sinh\left(iy\right)\nonumber\\ &= \sin\left(x\right)\cos\left(y\right) + \cos\left(x\right)\sin\left(y\right),\\ \cos\left(x + y\right) &= \cosh\left(ix + iy\right) = \cosh\left(ix\right)\cosh\left(iy\right) + \sinh\left(ix\right)\sinh\left(iy\right)\nonumber\\ &= \cos\left(x\right)\cos\left(y\right) - \sin\left(x\right)\sin\left(y\right). \end{align} \]
From this it follows
\[ \begin{align} \cos\left(\alpha\right) = \cos^2\left(\frac{\alpha}{2}\right) - \sin^2\left(\frac{\alpha}{2}\right) = 1 - 2\sin^2\left(\frac{\alpha}{2}\right) = 2\cos^2\left(\frac{\alpha}{2}\right) - 1.\tag{A.63}\label{eq:trigo_add_lemma_1} \end{align} \]
Furthermore, the so-called Euler identity holds
\[ \begin{align} \exp\left(ix\right) = \frac{\exp\left(ix\right) + \exp\left(-ix\right) + \exp\left(ix\right) - \exp\left(-ix\right)}{2} = \cosh\left(ix\right) + \sinh\left(ix\right) = \cos\left(x\right) + i\sin\left(x\right). \end{align} \]
The following holds for the derivatives of the basic functions
\[ \begin{align} \exp'\left(x\right) & \stackrel{\href{#eq:1st_derivative_series}{\text{Eq. (A.34)}}}{=} \sum_{n = 0}^{\infty}\frac{n + 1}{\left(n + 1\right)!}x^n = \sum_{n = 0}^{\infty}\frac{x^n}{n!} = \exp\left(x\right),\\ \sinh'\left(x\right) &= \cosh\left(x\right),\\ \cosh'\left(x\right) &= \sinh\left(x\right),\\ \sin'\left(x\right) &= \cosh\left(ix\right) = \cos\left(x\right),\\ \cos'\left(x\right) &= -\sin\left(x\right). \end{align} \]
Because of Eq. (A.46), $\exp$ is strictly monotonically increasing on $\mathbb{R}$. Because $\cosh\left(x\right) > 0$ for all real $x$, $\sinh$ is strictly monotonically increasing; moreover,
\[ \begin{align} \lim\limits_{x\to\infty}\sinh\left(x\right) = \infty, & {} & \lim\limits_{x\to - \infty}\sinh\left(x\right) = -\infty. \end{align} \]
Thus,
\[ \begin{align} \arsinh:\mathbb{R}\to\mathbb{R} \end{align} \]
is defined; furthermore, one has
\[ \begin{align} \arsinh'\left(x\right) = \frac{1}{\cosh\left(\arsinh'\left(x\right)\right)} = \frac{1}{\sqrt{1 + x^2}}. \end{align} \]
Furthermore, the following hold
\[ \begin{align} \lim\limits_{x\to\infty}\cosh\left(x\right) = \infty, & {} & \lim\limits_{x\to - \infty}\cosh\left(x\right) = \infty, \end{align} \]
the minimum is at $x = 0$, where $\cosh\left(0\right) = 1$, so one defines
\[ \begin{align} \arcosh\left[1, \infty\right)\to\left[0, \infty\right). \end{align} \]
For the derivative one has
\[ \begin{align} \arcosh'\left(x\right) = \frac{1}{\sinh\left(\arcosh\left(x\right)\right)} = \frac{1}{\sqrt{x^2 - 1}}. \end{align} \]
The arcus functions have the domains and ranges
\[ \begin{align} \arcsin: & \left[-1, 1\right]\to\left[-\frac{\pi}{2}, \frac{\pi}{2}\right],\\ \arccos: & \left[-1, 1\right]\to\left[0, \pi\right],\\ \arctan: & \left(-\infty, \infty\right)\to\left(-\frac{\pi}{2}.\frac{\pi}{2}\right). \end{align} \]
Their derivatives are calculated as:
\[ \begin{align} \arcsin'\left(x\right) &= \frac{1}{\cos\left(\arcsin\left(x\right)\right)} = \frac{1}{\sqrt{1 - x^2}},\\ \arccos'\left(x\right) &= -\frac{1}{\sin\left(\arccos\left(x\right)\right)} = \frac{1}{\sqrt{1 - x^2}},\\ \arctan'\left(x\right) &= \frac{1}{\tan'\left(\arctan\left(x\right)\right)} = \frac{1}{1 + x^2}. \end{align} \]
For the first-order Taylor expansions of the basic functions about the expansion point zero, one obtains
\[ \begin{align} \exp\left(x\right) \approx 1 + x, & {} & \sinh\left(x\right) \approx x, & {} & \cosh\left(x\right) \approx 1, & {} & \tanh\left(x\right) \approx x,\\ \sin\left(x\right) \approx x, & {} & \cos\left(x\right) \approx 1, & {} & \tan\left(x\right) \approx x. \end{align} \]
Let $p\left(x\right) = \sum_{n = 0}^{\infty}a_nx^n$ be a power series $\mathbb{C}\to\mathbb{C}$ with $p\left(x\right) = 0$ for all $x\in \mathbb{C}$; then $a_n = 0$ already holds for all $n\in\mathbb{N}$. This is first shown for a polynomial, i.e. for a terminating power series, by contradiction. So assume that $a_n = 0$ for all $N < n \in \mathbb{N}$ and further that $a_N \not = 0$. As $\left|x\right|\to\infty$, every polynomial diverges; since polynomials are continuous, there exists in particular a $b \in \mathbb{R}$ with $b > 0$ and $\left|p\left(x\right)\right| > 0$ for all $x \in \mathbb{C}$ with $\left|x\right| > b$, contradicting the assumption. Thus the statement is shown for polynomials.
At this point, a remark about complex numbers is in order. The intuitive hurdle with complex numbers is to accept the existence of the imaginary unit $i =\sqrt{-1}$. One cannot picture a number $i$ that satisfies this equation. Nevertheless, one can compute with it; it is thus a tool of the theory. Before comparing with measurements, however, the result must be projected onto the real axis. Even with real numbers, one may ask what to intuitively make of a number that never terminates, i.e. an irrational number such as $e$. The limits of intuition have, in essence, already been exceeded here.
The delta distribution $\delta\left(x\right)$ is defined by the property that for every continuous function $f = f\left(x\right)$
\[ \begin{align} \int_{a<0}^{b>0}\delta\left(x\right)f\left(x\right)dx = f\left(0\right) \end{align} \]
holds. According to [20], one has
\[ \begin{align} \lim_{t\to \infty}\frac{4\sin^2\left(\frac{\Omega t}{2}\right)}{\Omega ^2t} = 2\pi\delta\left(\Omega\right)\tag{A.85}\label{eq:delta_distribution_prop_1}. \end{align} \]
According to [19], the following holds
\[ \begin{align} \delta\left(x - x_0\right) = \frac{1}{2\pi}\int_{-\infty}^{\infty}\exp\left(ik\left(x - x_0\right)\right)dk\tag{A.86}\label{eq:delta_distribution_prop_2}. \end{align} \]
Let $f, g:\mathbb{R}\to\mathbb{R}$ be Riemann integrable. Then, by the product rule,
\[ \begin{align} \left(fg\right)' &= f'g + fg'\Leftrightarrow fg = \int f'gdx + \int fg'dx\Leftrightarrow \int f'gdx = fg - \int fg'dx. \end{align} \]
This is called partial integration.
Let $a, b\in \mathbb {R}$ with $a<b$, $f:\left[a, b\right]\to \mathbb {R}$ be Riemann integrable, $I\subseteq\mathbb {R}$ an interval, $\varphi:I\to\left[a, b\right]$ continuously differentiable and bijective, and $F$ an antiderivative of $f$. Then
\[ \begin{align} \int_{a}^{b}f\left(x\right)dx = \left[F\left(x\right)\right]_a^b = \left[F\left(\varphi\left(x\right)\right)\right]_{\varphi^{-1}\left(a\right)}^{\varphi^{-1}\left(b\right)} = \int_{\varphi^{-1}\left(a\right)}^{\varphi^{-1}\left(b\right)}\frac{d}{dx}\left(F\left(\varphi\left(x\right)\right)\right)dx = \int_{\varphi^{-1}\left(a\right)}^{\varphi^{-1}\left(b\right)}f\left(\varphi\left(x\right)\right)\varphi'\left(x\right)dx.\tag{A.88}\label{eq:substitutionsregel} \end{align} \]
This is called the substitution rule.
The method of separation of variables is derived. Let $a, b\in \mathbb {R}$ with $a<b$ be given, let $h:\left[a, b\right]\to \mathbb {R}$ be Riemann integrable, $f:\left[a, b\right]\to \mathbb {R}$ continuously differentiable and bijective, and $g:f\left(\left[a, b\right]\right)\to \mathbb {R}$ Riemann integrable with $g\left(y\right)\not = 0$ for all $y\in f\left(\left[a, b\right]\right)$. One wishes to solve the differential equation
\[ \begin{align} \frac{df}{dx} = g\left(f\right)h\left(x\right) \end{align} \]
on $\left[a, b\right]$. One computes
\[ \begin{align} \frac{1}{g\left(f\right)}\frac{df}{dx} = h\left(x\right). \end{align} \]
This is now integrated from $a$ to a desired $x\in\left[a, b\right]$:
\[ \begin{align} \int_{a}^x\frac{1}{g\left(f\left(x'\right)\right)}\frac{df}{dx'}dx' &= \int_{a}^{x}h\left(x'\right)dx'\Leftrightarrow \int_{f\left(a\right)}^{f\left(x\right)}\frac{1}{g\left(f\right)}df = \int_{a}^{x}h\left(x'\right)dx' \end{align} \]
The substitution rule was used here. If these two integrals are solvable, one obtains an algebraic expression for $f\left(x\right)$.
Let a continuously differentiable function $f: \mathbb{R}^2 \to \mathbb{R}$ be given. Furthermore, let two functions $a \leq b$ be given, which depend on a parameter $t$, $a = a\left(t\right)$, $b = b\left(t\right)$. Define the function $F = F\left(t\right)$ by
\[ \begin{align} F\left(t\right) \coloneqq \int_{a\left(t\right)}^{b\left(t\right)}f\left(x, t\right)dx. \end{align} \]
The Leibniz rule is a statement about the derivative of $F$. To derive it, one first defines an auxiliary function
\[ \begin{align} \newtilde{F}\left(a, b, t\right) \coloneqq \int_{a}^{b}f\left(x, t\right)dx. \end{align} \]
Then, for the derivative of $\newtilde{F}$, one has
\[ \begin{align} \newtilde{F}'\left(a, b, t\right)&= \nabla\newtilde{F}\left(a, b, t\right) = \left(\begin{array}{c} \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial a}\\ \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial b}\\ \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial t} \end{array}\right). \end{align} \]
Using the fundamental theorem of calculus, this can be rewritten as
\[ \begin{align} \left(\begin{array}{c} \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial a}\\ \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial b}\\ \frac{\partial\int_{a}^{b}f\left(x, t\right)dx}{\partial t} \end{array}\right) &= \left(\begin{array}{c} -f\left(a, t\right)\\ f\left(b, t\right)\\ \int_{a}^{b}\frac{\partial f\left(x, t\right)}{\partial t}dx \end{array}\right) \end{align} \]
Using the multidimensional chain rule, one obtains
\[ \begin{align} \frac{dF\left(t\right)}{dt} &= \left(\begin{array}{c} \frac{da}{dt}\\ \frac{db}{dt}\\ 1 \end{array}\right)\cdot\nabla\newtilde{F}\left(a, b, t\right) \end{align} \]
This is the Leibniz rule.
We now wish to establish
\[ \begin{align} \int_{0}^{\infty}x^ne^{-\beta x}dx = \frac{n!}{\beta^{n + 1}}\tag{A.98}\label{eq:int_prop_1} \end{align} \]
with $\beta\in \mathbb {R}$, $\beta>0$, $n\in \mathbb {N}$. This is best done by complete induction. For $n = 0$ one has
\[ \begin{align} \int_{0}^{\infty}e^{-\beta x}dx = \frac{1}{\beta}, \end{align} \]
so the statement is true for $n = 0$. Now suppose the statement already holds for $n$. Then, by partial integration, it follows
\[ \begin{align} \int_{0}^{\infty}x^{n + 1}e^{-\beta x} &= \left[-\frac{1}{\beta}x^{n + 1}e^{-\beta x}\right]_0^{\infty} + \frac{n + 1}{\beta}\int_{0}^{\infty}x^ne^{-\beta x}dx = \frac{n + 1}{\beta}\frac{n!}{\beta^{n + 1}} = \frac{\left(n + 1\right)!}{\beta^{n + 2}}. \end{align} \]
The statement therefore holds for all $n\in\mathbb{N}$. Furthermore,
\[ \begin{align} \int_{0}^{\infty}\frac{x^3}{e^x - 1}dx = \frac{\pi^4}{15}.\tag{A.101}\label{eq:stefanboltzmann_help} \end{align} \]
To this end, one first computes with Eq. (A.8)
\[ \begin{align} \frac{1}{e^x - 1} = \frac{e^{-x}}{1 - e^{-x}} = \frac{e^{-x} + 1 - 1}{1 - e^{-x}} = \frac{1}{1 - e^{-x}} - 1 = \sum_{n = 0}^{\infty}e^{-nx} - 1 = \sum_{n = 1}^{\infty}e^{-nx}. \end{align} \]
It then follows, using Eqs. (A.98) and (A.12), that
\[ \begin{align} \int_{0}^\infty\frac{x^3}{e^x - 1}dx &= \sum_{n = 1}^{\infty}\int_{0}^{\infty}x^3e^{-nx}dx = 6\sum_{n = 1}^{\infty}\frac{1}{n^4} = \frac{\pi^4}{15}.\tag{A.103}\label{eq:int_prop_3} \end{align} \]
The so-called gamma function is defined by
\[ \begin{align} \Gamma\left(n+1\right)\coloneqq\int_{0}^{\infty}x^ne^{-x}dx \end{align} \]
For it, using Eq. (A.98), it follows that
\[ \begin{align} \Gamma\left(n\right) = \left(n - 1\right)!. \end{align} \]
The error function $\text{erf}$ is defined by
\[ \begin{align} \text{erf}\left(x\right) \coloneqq \frac{2}{\sqrt{\pi}}\int_{0}^{x}e^{-t^2}dt.\tag{A.106}\label{eq:def_fehlerfunktion} \end{align} \]
According to [11], one has
\[ \begin{align} \lim\limits_{x\to\infty}\text{erf}\left(x\right) = 1\tag{A.107}\label{eq:integral_fehlerfunktion}. \end{align} \]
For $C>0$, it follows that
\[ \begin{align} \lim\limits_{x\to\infty}\int_{0}^{\infty}e^{-Cx^2}dx = \frac{1}{\sqrt{C}}\lim\limits_{x\to\infty}\int_{0}^{\infty}e^{-x^2}dx = \frac{1}{2}\sqrt{\frac{\pi}{C}}. \end{align} \]
One obtains
\[ \begin{align} \int_{0}^{\infty}e^{-Cx^2}x^2dx &= \left[x\left(-\frac{1}{2C}e^{-Cx^2}\right)\right]_0^\infty + \frac{1}{2C}\int_{0}^{\infty}e^{-Cx^2}dx = \frac{1}{4}\sqrt{\frac{\pi}{C^3}}\tag{A.109}\label{eq:int_prop_5},\\ \int_{0}^{\infty}e^{-Cx^2}x^3dx &= \left[x^2\cdot\left(-\frac{1}{2C}e^{-Cx^2}\right)\right]_0^\infty + \int_{0}^{\infty}\frac{x}{C}e^{-Cx^2}dx\nonumber\\ &= \int_{0}^{\infty}\frac{x}{C}e^{-Cx^2}dx = -\frac{1}{2C^2}\left[e^{-Cx^2}\right]_0^\infty = \frac{1}{2C^2}.\tag{A.110}\label{eq:int_prop_4} \end{align} \]
One defines the gamma function for $x>0$ by
\[ \begin{align} \Gamma\left(x\right) \coloneqq \int_{0}^{\infty}t^{x - 1}e^{-t}dt. \end{align} \]
For $n\in\mathbb{N}$, because of Eq. (A.98), one has
\[ \begin{align} \Gamma\left(n + 1\right) = \int_{0}^{\infty}t^{n}e^{-t}dt = n!.\tag{A.112}\label{eq:gamma_prop_0} \end{align} \]
From this, in particular, it follows that
\[ \begin{align} \Gamma\left(n + 1\right) &= n! = n\left(n - 1\right)! = n\Gamma\left(n\right)\tag{A.113}\label{eq:gamma_prop_1}. \end{align} \]
This can be used to derive the Stirling formula, which provides an approximation of $n!$ for large $n$ and is very useful when, for example, one asks how large $\left(10^{23}\right)!$ is. To this end, one first computes
\[ \begin{align} n! = \Gamma\left(n + 1\right) = \int_{0}^{\infty}t^ne^{-t}dt = \int_{0}^{\infty}e^{-t + n\ln\left(t\right)}dt = \int_{0}^{\infty}e^{-n\varphi\left(t\right)}dt \end{align} \]
with
\[ \begin{align} \varphi\left(t\right) \coloneqq \frac{t}{n} - \ln\left(t\right). \end{align} \]
The following hold
\[ \begin{align} \lim\limits_{t \to 0}\varphi\left(t\right) = \infty, & {} & \lim\limits_{t \to \infty}\varphi\left(t\right) = \infty. \end{align} \]
For the first two derivatives, one has
\[ \begin{align} \varphi'\left(t\right) = \frac{1}{n} - \frac{1}{t}, & {} & \varphi''\left(t\right) = \frac{1}{t^2}. \end{align} \]
The unique minimum is therefore at $t = n$. The second-order Taylor expansion about the point $x = n$ is thus
\[ \begin{align} \varphi\left(t\right) = 1 - \ln\left(n\right) + \frac{1}{2}\frac{\left(t - n\right)^2}{n^2} + \mathcal{O}\left[\left(t - n\right)^3\right]. \end{align} \]
The expression $e^{-n\varphi}$ is only relevant for negative and small $\varphi$; therefore, the Taylor expansion about the minimum is a good approximation within the integral of the gamma function:
\[ \begin{align} n!\approx \int_{0}^{\infty}e^{-n + n\ln\left(n\right) - \frac{1}{2}\frac{\left(t - n\right)^2}{n}}dt = n^ne^{-n}\int_{0}^{\infty}e^{-\frac{t^2}{2n}}dt = n^{n}e^{-n}\sqrt{2\pi n} = \sqrt{2\pi}n^{n + 1/2}e^{-n}\tag{A.119}\label{eq:stirling} \end{align} \]
In the last step, Eq. (A.107) was used. Eq. (A.119) is the Stirling formula.
Let $n\in \mathbb{N}$ with $n \geq 1$. Let $\Omega \subseteq \mathbb {R}^3$ be connected and $\psi: \Omega\to \mathbb {R}^n$ and $f: \mathbb {R}^n\to \mathbb {R}$ be continuously differentiable. Define the functional
\[ \begin{align} F\left[\psi\right] \coloneqq \int_\Omega f\left(\psi\left(\mathbf{r}\right)\right)d^3r \in \mathbb{R}. \end{align} \]
Now define a set of auxiliary functions
\[ \begin{align} D \coloneqq \lbrace h:\Omega\to\mathbb{R}^n \vert \text{continuously differentiable, }h\left(\partial\Omega\right) = 0\rbrace. \end{align} \]
Building on this, a further set of difference quotients
\[ \begin{align} H: D \times \mathbb{R}_{> 0}: \left(h, \epsilon\right) \mapsto \frac{F\left[\psi + \epsilon h\right] - F\left[\psi\right]}{\epsilon} \end{align} \]
is defined. The functional derivative
\[ \begin{align} \frac{\delta F}{\delta\psi}: \Omega \to \mathbb{R}^n \end{align} \]
is the function that satisfies
\[ \begin{align} \frac{dH\left(h, \epsilon\right)}{d\epsilon} = \lim_{\epsilon\to 0}\frac{F\left[\psi + \epsilon h\right] - F\left[\psi\right]}{\epsilon} = \lim_{\epsilon\to 0}\frac{1}{\epsilon}\int_\Omega f\left(\psi\left(\mathbf{r}\right) + \epsilon h\left(\mathbf{r}\right)\right) - f\left(\psi\left(\mathbf{r}\right)\right)d^3r = \int_\Omega\frac{\delta F}{\delta\psi}\cdot h\left(\mathbf{r}\right)d^3r\tag{A.124}\label{eq:def_functional_derivative} \end{align} \]
for all $h \in D$. The functional derivative can be understood as the derivative of $F$ in the direction of $\psi$.
Let $K\in\{\mathbb{R}, \mathbb{C}\}$, then one defines a vector space $V$ over $K$ in the following way:
There is a map
\[ \begin{align} + :V\times V\to V;\left(\mathbf{x}, \mathbf{y}\right)\mapsto \mathbf{x} + \mathbf{y} \end{align} \]
as well as another map
\[ \begin{align} \cdot :K\times V\to V;\left(\lambda, \mathbf{x}\right)\mapsto\lambda \mathbf{x}. \end{align} \]
These two functions have the following properties:
\[ \begin{align} \left(\mathbf{x} + \mathbf{y}\right) + \mathbf{z} = \mathbf{x} + \left(\mathbf{y} + \mathbf{z}\right), & {} & \mathbf{x} + \mathbf{y} = \mathbf{y} + \mathbf{x}, & {} & \mathbf{0} + \mathbf{x} = \mathbf{x}. \end{align} \]
For every $\mathbf{x}\in V$ there exists a $\left(-\mathbf{x}\right)\in V$ with
\[ \begin{align} \left(-\mathbf{x}\right) + \mathbf{x} = 0. \end{align} \]
Furthermore, the following shall hold:
\[ \begin{align} \lambda_1\cdot\left(\lambda_2\mathbf{x}\right) = \left(\lambda_1\lambda_2\right)\cdot\mathbf{x}, & {} & 1\cdot\mathbf{x} = \mathbf{x},\\ \lambda\left(\mathbf{x} + \mathbf{y}\right) = \lambda\mathbf{x} + \lambda\mathbf{y}, & {} & \left(\lambda_1 + \lambda_2\right)\mathbf{x} = \lambda_1\mathbf{x} + \lambda_2\mathbf{x}. \end{align} \]
Vectors that can be understood as arrows in space are written in this book in $\mathbf{bold italic}$.
A matrix $A$ over $K\in\{\mathbb{R}, \mathbb{C}\}$ is a rectangular array of elements from $K$ with $m\geq 1$ rows and $n\geq 1$ columns. One writes $A_{i, j}$ for the element in the $j-$th column of the $i-$th row. One defines the transpose $A^T$ of $A$ by $A_{j, i} = A_{i, j}$ for all $1\leq i\leq m$, $1\leq j\leq n$. The multiplication of such a matrix by a vector $\mathbf{b} = \left(b_1, \dotsc, b_n\right)^T$ is defined by
\[ \begin{align} A\mathbf{b} = \sum_{i = 1}^{m}\sum_{j = 1}^{n}A_{i, j}b_j\mathbf{e}_i, \end{align} \]
so that an $m-$dimensional column vector results. More generally, let a matrix $B$ over $K$ with $n$ rows and $l\geq 1$ columns be given; then one defines the matrix product $A\cdot B$ by
\[ \begin{align} \left(A\cdot B\right)_{i, j} \coloneqq\sum_{\alpha = 1}^{n}A_{i, \alpha}B_{\alpha, j}. \end{align} \]
One has
\[ \begin{align} \left(A\cdot B\right)^T_{i, j}&= \left(A\cdot B\right)_{j, i} = \sum_{\alpha = 1}^{n}A_{j, \alpha}B_{\alpha, i} = \sum_{\alpha = 1}^{n}B_{\alpha, i}A_{j, \alpha} = \sum_{\alpha = 1}^{n}B^T_{i, \alpha}A^T_{\alpha, j} = \left(B^T\cdot A^T\right)_{i, j} \end{align} \]
and thus, in this case,
\[ \begin{align} \left(A\cdot B\right)^T = B^T \cdot A^T. \end{align} \]
One further defines the scalar product of two vectors $\mathbf{a}, \mathbf{b}\in \mathbb{R}^n$ by
\[ \begin{align} \mathbf{a}\cdot\mathbf{b} = \left\langle\mathbf{a}|\mathbf{b}\right\rangle \coloneqq \mathbf{a}^T\mathbf{b}. \end{align} \]
If $A$ is a real matrix, then it follows
\[ \begin{align} \left\langle\mathbf{a}|A\mathbf{b}\right\rangle = \mathbf{a}^TA\mathbf{b} = \left(A^T\mathbf{a}\right)^T\mathbf{b} = \left\langle A^T\mathbf{a}\big|\mathbf{b}\right\rangle. \end{align} \]
Matrices with as many rows as columns are called square.
Operators are functions between functions. In meteorology, only two function spaces are relevant:
\[ \begin{align} F_1 &= \{A\times\mathbb{R}\to\mathbb{R}, \:\text{infinitely often continuously differentiable}\}\\ F_2&= \{A\times\mathbb{R}\to\mathbb{R}^3, \:\text{infinitely often continuously differentiable}\} \end{align} \]
Here $A$ is the atmosphere. Operators are marked with a hat symbol. One defines
\[ \begin{align} & \newhat{A}\:\text{linear}\:\text{operator}:\nonumber\\ &\Leftrightarrow&\newhat{A}\left(\lambda\psi\right) = \lambda\newhat{A}\left(\psi\right)\land\newhat{A}\left(\psi_1 + \psi_2\right) = \newhat{A}\left(\psi_1\right) + \newhat{A}\left(\psi_2\right) \end{align} \]
for every $\lambda\in \mathbb{C}$. Let $\newhat{A}$ and $\newhat{B}$ be operators; then define
\[ \begin{align} \newhat{A}\newhat{B} \coloneqq \newhat{A}\circ\newhat{B} \end{align} \]
If $\newhat{A}$ and $\newhat{B}$ are linear, then the following holds
\[ \begin{align} \newhat{A}\left(\newhat{B}\left(\lambda\psi\right)\right) = \newhat{A}\left(\lambda\newhat{B}\left(\psi\right)\right) = \lambda\newhat{A}\left(\newhat{B}\left(\psi\right)\right)\Leftrightarrow \newhat{A}\newhat{B}\left(\lambda\psi\right) = \lambda\newhat{A}\newhat{B}\left(\psi\right) \end{align} \]
Furthermore,
\[ \begin{align} \newhat{A}\newhat{B}\left(\psi_1 + \psi_2\right) = \newhat{A}\left(\newhat{B}\left(\psi_1\right) + \newhat{B}\left(\psi_2\right)\right) = \newhat{A}\newhat{B}\left(\psi_1\right) + \newhat{A}\newhat{B}\left(\psi_2\right). \end{align} \]
Compositions of linear operators are therefore linear. Furthermore, for operators the brackets around the argument are often omitted:
\[ \begin{align} \newhat{A}\psi \coloneqq \newhat{A}\left(\psi\right) \end{align} \]
Moreover,
\[ \begin{align} \left(\lambda_1\newhat{A} + \lambda_2\newhat{B}\right)\lambda\psi &= \lambda_1\newhat{A}\lambda\psi + \lambda_2\newhat{B}\lambda\psi = \lambda_1\lambda\newhat{A}\psi + \lambda_2\lambda\newhat{B}\psi = \lambda\left(\lambda_1\newhat{A}\psi + \lambda_2\newhat{B}\psi\right)\nonumber\\ &= \lambda\left(\lambda_1\newhat{A} + \lambda_2\newhat{B}\right)\psi \end{align} \]
as well as
\[ \begin{align} \left(\lambda_1\newhat{A} + \lambda_2\newhat{B}\right)\left(\psi_1 + \psi_2\right) &= \lambda_1\newhat{A}\left(\psi_1 + \psi_2\right) + \lambda_2\newhat{B}\left(\psi_1 + \psi_2\right)\nonumber\\ &= \lambda_1\newhat{A}\psi_1 + \lambda_1\newhat{A}\psi_2 + \lambda_2\newhat{B}\psi_1 + \lambda_2\newhat{B}\psi_2\nonumber\\ &= \left(\lambda_1\newhat{A} + \lambda_2\newhat{B}\right)\psi_1 + \left(\lambda_1\newhat{A} + \lambda_2\newhat{B}\right)\psi_2. \end{align} \]
Linear combinations of linear operators are therefore linear. Since the usual differentiation rules apply to partial derivatives, one obtains
\[ \begin{align} \frac{\partial}{\partial x_i}\lambda\psi &= \lambda\frac{\partial}{\partial x_i}\psi\\ \frac{\partial}{\partial x_i}\left(\psi_1 + \psi_2\right) &= \frac{\partial}{\partial x_i}\psi_1 + \frac{\partial}{\partial x_i}\psi_2, \end{align} \]
where time is taken to be one of the $x_i$. Partial derivatives, as well as linear combinations of partial derivatives (also with basis vectors), are therefore linear.
Let $\left(x_1, x_2, x_3\right)$ be three Cartesian coordinates and $\left(\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\right)$ be the corresponding basis. The vector product $\mathbf{A}\times\mathbf{B}$ of two vectors $\mathbf{A} = \left(A_1, A_2, A_3\right)^T$, $B = \left(B_1, B_2, B_3\right)^T$ is defined by
\[ \begin{align} \mathbf{A}\times\mathbf{B} \coloneqq \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_j\mathbf{e}_k. \end{align} \]
This is linear, since for $\lambda\in \mathbb{R}$ and $\mathbf{C}\in \mathbb{R}^3$ the following hold:
\[ \begin{align} \mathbf{A}\times\left(\mathbf{B} + \mathbf{C}\right) &= \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_i\left(B_j + C_j\right)\mathbf{e}_k = \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_j\mathbf{e}_k + \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iC_j\mathbf{e}_k\nonumber\\ &= \mathbf{A}\times\mathbf{B} + \mathbf{A}\times\mathbf{C}, \tag{A.149}\label{eq:vektorprodukt_linear_summe}\\ \mathbf{A}\times\left(\lambda\mathbf{B}\right) &= \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_i\lambda B_j\mathbf{e}_k = \lambda\mathbf{A}\times\mathbf{B}.\tag{A.150}\label{eq:vektorprodukt_linear_faktor} \end{align} \]
Furthermore, it is anticommutative,
\[ \begin{align} \mathbf{A}\times\mathbf{B} = \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_j\mathbf{e}_k = \sum_{i, j, k = 1}^{3} - \epsilon_{j,i,k}B_jA_i\mathbf{e}_k = -\mathbf{B}\times\mathbf{A}.\tag{A.151}\label{eq:cross_product_anticommutative} \end{align} \]
For the $l-$th component, $1 \leq l \leq 3$, of the vector product, one finds
\[ \begin{align} \left(\mathbf{A}\times\mathbf{B}\right)_l &= \left(\sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_j\mathbf{e}_k\right)\cdot\mathbf{e}_l = \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_j\delta_{k,l} = \sum_{i, j = 1}^{3}\epsilon_{i, j, l}A_iB_j. \end{align} \]
The vector product has the orthogonality properties
\[ \begin{align} \mathbf{A}\cdot\left(\mathbf{A}\times\mathbf{B}\right) &= \sum_{i = 1}^3A_i\left(\mathbf{A}\times\mathbf{B}\right)_i = \sum_{i = 1}^3A_i\sum_{j, k = 1}^{3}\epsilon_{j, k, i}A_jB_k = \sum_{k = 1}^3B_k\sum_{i, j = 1}^3\epsilon_{k, i, j}A_iA_j\nonumber\\ &= \sum_{k = 1}^3B_k\sum_{i, j = 1}^3A_iA_j - A_jA_i = 0 \end{align} \]
as well as
\[ \begin{align} \mathbf{B}\cdot\left(\mathbf{A}\times\mathbf{B}\right) & \stackrel{\href{#eq:cross_product_anticommutative}{\text{Eq. (A.151)}}}{=} -\mathbf{B}\cdot\left(\mathbf{B}\times\mathbf{A}\right) = -0 = 0. \end{align} \]
Furthermore, the so-called Grassmann identity applies:
\[ \begin{align} \mathbf{A}\times\left(\mathbf{B}\times\mathbf{C}\right) &= \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_i\left(\mathbf{B}\times\mathbf{C}\right)_j\mathbf{e}_k = \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_i\left(\sum_{l, m, n = 1}^3\epsilon_{l, m, n}B_lC_m\mathbf{e}_n\right)_j\mathbf{e}_k\nonumber\\ &= \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_i\sum_{l, m = 1}^3\epsilon_{l, m, j}B_lC_m\mathbf{e}_k = \sum_{k = 1}^3\mathbf{e}_k\sum_{i = 1}^3A_i\sum_{j, l, m = 1}^{3}\epsilon_{j, k, i}\epsilon_{j, l, m}B_lC_m\nonumber\\ &= \sum_{k = 1}^3\mathbf{e}_k\sum_{i = 1}^{3}A_i\left(B_kC_i - B_iC_k\right) = \mathbf{B}\left(\mathbf{A}\cdot\mathbf{C}\right) - \mathbf{C}\left(\mathbf{A}\cdot\mathbf{B}\right)\tag{A.155}\label{eq:bac-cab} \end{align} \]
This is also memorably known as the BAC-CAB rule. One can further write
\[ \begin{align} \mathbf{B} \equiv \mathbf{B}_\parallel + \mathbf{B}_\perp, \end{align} \]
where $\mathbf{B}_\parallel$ denotes the component of $\mathbf{B}$ parallel to $\mathbf{A}$, and
\[ \begin{align} \mathbf{B}_\perp \coloneqq \mathbf{B} - \mathbf{B}_\parallel \end{align} \]
the remaining component of $\mathbf{B}$, which is perpendicular to $\mathbf{A}$, as will be shown. One has
\[ \begin{align} \mathbf{B}_\parallel = \left(\frac{1}{\left|\mathbf{A}\right|^2}\mathbf{A}\cdot\mathbf{B}\right)\mathbf{A}. \end{align} \]
From this it follows
\[ \begin{align} \mathbf{A}\cdot\mathbf{B}_\perp = \mathbf{A}\cdot\left(\mathbf{B} - \mathbf{B}_\parallel\right) = \mathbf{A}\cdot\mathbf{B} - \mathbf{A}\cdot\left(\frac{1}{\left|\mathbf{A}\right|^2}\mathbf{A}\cdot\mathbf{B}\right)\mathbf{A} = 0, \end{align} \]
so $\mathbf{B}_\perp$ is indeed perpendicular to $\mathbf{A}$. Using Eq. (A.155), one can further write
\[ \begin{align} \mathbf{A}\times\left(\mathbf{B}\times\mathbf{A}\right) &= \mathbf{B}A^2 - \mathbf{A}\left(\mathbf{A}\cdot\mathbf{B}\right) = A^2\mathbf{B} - A^2\mathbf{B}_\parallel = A^2\mathbf{B}_\perp\nonumber\\ \Rightarrow \mathbf{B}_\perp &= \frac{1}{A^2}\mathbf{A}\times\left(\mathbf{B}\times\mathbf{A}\right). \end{align} \]
One has
\[ \begin{align} \mathbf{A}\cdot\left(\mathbf{B}\times\mathbf{C}\right) &= \sum_{i = 1}^3A_i\left(\mathbf{B}\times\mathbf{C}\right)_i = \sum_{i = 1}^3A_i\sum_{j, k = 1}^{3}\epsilon_{j, k, i}B_jC_k = \sum_{i, j, k = 1}^{3}\epsilon_{j, k, i}A_iB_jC_k = \sum_{i, j, k = 1}^{3}\epsilon_{i, j, k}A_iB_jC_k\nonumber\\ &= \sum_{k = 1}^{3}C_k\sum_{i, j = 1}^{3}\epsilon_{i, j, k}A_iB_j = \sum_{i = 1}^3C_k\left(\mathbf{A}\times\mathbf{B}\right)_k = \mathbf{C}\cdot\left(\mathbf{A}\times\mathbf{B}\right).\tag{A.161}\label{eq:spat_prod_prop_0} \end{align} \]