Variational derivative

See @olver86 page 244.

Informal approach

For functionals, the variational derivative plays the role of the gradient of functions of several variables.
Given a smooth function $f (x), x = (x^{1}, \dots, x^{p})$ , the gradient is a 1-form $d f$ such that for a vector $V$

d f_{x} (V) = lim_{ϵ \to 0} \frac{f (x + ϵ V) - f (x)}{ϵ} = \frac{d}{d ϵ} f (x + ϵ V) |_{ϵ = 0},

that is, it tells us how much the function varies along the direction specified by $V$ .

If we think of $x$ not like a finite dimensional vector $(x^{1}, \dots, x^{p})$ but like a function $x : {1, \dots, p} \to R$ we can generalize this to the case $x : R \to R$ , $t \mapsto x (t)$ , being now $f$ not a function but a functional. The question is: for a functional $f$ , is there any mathematical object $d f_{x}$ such that for a function $V (t)$ gives us the number

d f_{x} (V) = lim_{ϵ \to 0} \frac{f (x + ϵ V) - f (x)}{ϵ} = \frac{d}{d ϵ} f (x + ϵ V) |_{ϵ = 0}

?
Observe that in the usual case of $x$ being a vector and $f$ being a function we can interpret $d f$ as the gradient vector $\nabla f$ (assuming the standard inner product, see here for more information) which satisfies

d f_{x} (V) = ⟨ (\nabla f)_{x}, V ⟩ = \frac{d}{d ϵ} f (x + ϵ V) |_{ϵ = 0} .

We can replace $⟨ -, - ⟩$ with the usual inner product $\int d t$ for the Hilbert space $L^{2}$ , so we can alternatively require a new object $δ f_{x}$ to be a function of $t$ satisfying

d f_{x} (V) = \int δ f_{x} (t) \cdot V (t) d t = \frac{d}{d ϵ} f (x + ϵ V) |_{ϵ = 0}

There are lots of technical details we are missing here, but this is the idea.

Partial derivatives

What would be the analogous of partial derivatives $\frac{\partial}{\partial x^{k}}$ ? In the same way that for functions the symbol $\frac{\partial}{\partial x^{k}}$ , for a particular value of $k$ , corresponds to $V = (0, \dots, 1, \dots, 0)$ , for functionals we would have, for a particular value $t_{0}$ ,

\frac{\partial}{\partial x (t_{0})}

which corresponds to the Dirac delta

V (t) = δ (t - t_{0}) .

This way,

\frac{\partial}{\partial x (t_{0})} f (x) = d f_{x} (V) = \int δ f_{x} (t) \cdot δ (t - t_{0}) d t = δ f_{x} (t_{0})

It is interpreted as the measure of the variation of the functional $f (x)$ when we uniquely modify the value of the function $x (t)$ at $t = t_{0}$ .

Technical definition

Definition (@olver86 page 245)
Let $J [u]$ be a variational problem (for functions $u : R^{p} \to R^{q}$ ). The variational derivative of $J$ is the unique $q$ -tuple

δ J [u] = (δ_{1} J [u], \dots, δ_{q} J [u])

such that given functions $f, η : Ω \subset R^{p} \to R^{q}$ , $η$ with compact support, satisfies:

\begin{matrix} (1) & \frac{d}{d ϵ} J [f + ϵ η] |_{ϵ = 0} = \int_{Ω} δ J [f (x)] \cdot η (x) d x . \end{matrix}

$◼$
Indeed, it is a kind of 1-form, which applied to a "vector" $η$ yields the derivative of $J$ at $f$ along the direction $η$ .

Some facts

Remarks
a) So far I know, it doesn't have to exist...

b) I think that the quantity (1) should also be denoted by

δ J_{f} (η)

to agree with $d f_{x} (V)$ . In this sense $δ J$ would be something similar to differential of a function. The expression $δ J_{f} = δ J [f (x)]$ can be thought as another function of $x$ or, alternatively, as something eating functions $η$ and turning back numbers, i.e., something analogous to the 1-form $d f_{x}$ in the usual case of functions of $R^{n}$ . Something like the duality between the gradient $\nabla f_{x}$ and $d f_{x}$ (vectors correspond to functions).

c) Since the functional $J$ is coming from a variational problem we can compute an explicit formula for its variational derivative. Let's focus in the case of functions $u : R \to R$ :

\int_{Ω} δ J [f (x)] \cdot η (x) d x = \frac{d}{d ϵ} J [f + ϵ η] |_{ϵ = 0} =

= \frac{d}{d ϵ} |_{ϵ = 0} \int_{Ω} L (x, p r^{n} (f + ϵ η)) d x =

= \int_{Ω} \frac{\partial L}{\partial u} \cdot η + \frac{\partial L}{\partial u_{1}} \cdot η^{'} + \dots + \frac{\partial L}{\partial u_{n}} \cdot η^{n)} d x

being $η^{j)}$ the $j$ th derivative of $η$ .
We can apply integration by parts, leaving apart the first term, and obtain that the expression above equals

\int_{Ω} \frac{\partial L}{\partial u} \cdot η d x + \int_{Ω} - D_{x} \frac{\partial L}{\partial u_{1}} \cdot η d x + \dots + \int_{Ω} - D_{x} \frac{\partial L}{\partial u_{n}} \cdot η^{n - 1)} d x

and after $n$ steps:

= \int_{Ω} \frac{\partial L}{\partial u} \cdot η d x + \int_{Ω} - D_{x} \frac{\partial L}{\partial u_{1}} \cdot η d x + \dots + \int_{Ω} {(- D_{x})}^{n} \frac{\partial L}{\partial u_{n}} \cdot η d x =

= \int_{Ω} \sum_{j = 0}^{n} {(- D_{x})}^{j} \frac{\partial L}{\partial u_{j}} \cdot η d x

So it should be

δ J [f (x)] = \sum_{j = 0}^{n} {(- D_{x})}^{j} \frac{\partial L}{\partial u_{j}}

This last expression is called the Euler operator.
$◼$

Proposition (@olver86 page 246)
If $f (x)$ is an extrema of $J [u]$ then $δ J [f (x)] \equiv 0$ , the null function on $Ω$ .
$◼$

The Euler-Lagrange equations then appear from this proposition together with Remark c) above.

Old stuff (integrate with above)

Functional derivative. Gradient flow

Given a functional $F : H \mapsto R$ , being $H$ a Hilbert space of functions, the associated gradient flow is given by the equation

\begin{matrix} (1) & \frac{\partial ρ}{\partial t} = - \frac{\partial F}{\partial ρ} . \end{matrix}

for $ρ (t) \in H$ .

In other words, $ρ$ decreases along the gradient of $F$ . The terminology stems from the 'finite dimensional' case, where a function $f (x, y, z)$ produces a vector field $V = \nabla f$ , which is called its 'gradient vector field'. Then, as with any vector field, one can study the flow induced by that vector field, i.e. the flow of the dynamical system given by $\dot{x} = V (x)$ .

In $(1)$ , the notation $\frac{\partial F}{\partial ρ}$ denotes the so-called functional derivative of $F$ to $ρ$ , which generalises the 'gradient' notion for functions. There exist multiple versions of the functional derivative, mainly because its definition depends on the function space on which $F$ acts. Anyway, the idea is to perturb $ρ$ a bit, i.e. to substitute $ρ \to ρ + ϵ ϕ$ with $0 < ϵ ≪ 1$ , and work out the resulting expression.

Example

Let $H = L^{2} (R^{n})$ , and $F$ such that for $u \in L^{2} (R^{n})$ :

F (u) = \frac{1}{2} \int | \nabla u (x) |^{2} d x

That is, $F$ is the Dirichlet energy. Then the Heat Equation

\partial_{t} u = \nabla^{2} u

is the associated gradient flow problem.