Is there free lunch in this world?

Since you were toddlers, almost surely, your parents have nagged you many times never to expect a free lunch; Whatever you get for “free”, you pay for it in some other way or form. But is there anything stopping us from getting lucky once in a while, say, a shawarma falls from the sky right into our mouths?

Consider the following scenario: your favourite Chinese restaurant is doing a special promotion, where every customer who walks into the shop will get a plate of succulent dumplings. You look at this offer and think to yourself: “I don’t mind some free dumplings!”. You thought to yourself that the restaurant owners must be warm-hearted individuals, and decided to visit the restaurant more often in the future.

In essence, the illusion of a “free lunch” is an observation on a local scale, while the lack of “free lunch” is a global property. On the local scale of the day and time in which you ate that succulent Chinese meal for free, you indeed had a free lunch, and there is no doubt about it. On average, however, customers like you may visit said restaurant more often in the coming months above their average frequency, effectively giving the money of that free meal back. We can describe these as bookkeeping illusions.

In the past months, I’ve run into this so-called “no free lunch” theorem in one way or another. It is quite interesting to see how fundamental this idea can be, or rather, how useful it is to carry this mindset around, beyond just daily life interactions and avoiding crypto scams (Don’t send Elon Musk your Bitcoin, he will give you double in return!). From conservation laws in physics to arbitrage-free asset pricing in finance, we see this philosophy in action close to an axiomatic level. Hence, I thought it would be a good idea to collect a few of such “no free lunch” theorems here and look at how they come about, and maybe spot some connections and learn something new along the way.

Conservation, conservation, and conservation

In physics, the conservation laws are naturally the things that come into mind when we think about “no free lunch”. Locally, a region can be more “energetic” than others, but globally, the energy of an entire system is conserved. It is stated in the infamous Noether’s Theorem that for every symmetry of a physical system (with respect to the Lagrangian), there is a corresponding conserved quantity.

For a brief introduction to the Lagrangian, we can think of it as an infinitesimal action, i.e. it is a function taking the paths of objects within a physical system as input, and returns the cost of the objects taking that path, in a balance between kinetic and potential energy. Equations can be derived from the Lagrangian by asserting that objects take the path of the least cost, and this set of equations is called the Euler-Lagrange equations.

For Noether’s theorem, suppose the Lagrangian $L(q(t),\dot{q}(t), t)$ is symmetric under some infinitesimal transformation $q(t) \to Q(s,t)$, (we can treat $q(t)$ as the collection of paths of objects within a physical system, and $Q(s,t)$ as the deformation of all the paths via a parameter $s$), we mean

(Remark 1) A caveat here is that we assert the deformation $Q(s,t)$ to be smooth and continuous. This makes an infinitesimal transformation (local nudge) equivalent to a global transformation, as we can “generate” a global transformation via these small nudges. Furthermore, since the path is arbitrarily chosen, and for all paths, such a transformation is smooth and continuous, we can describe the deformation as a vector field that assigns a deformation continuously and smoothly to all the points in the space. Then a transformation is characterised by the flows of the vector field.

(Remark 2) In term’s of the language of vector fields and Lie derivatives, we write the symmetric condition as $\mathcal{L}_{X}L = \dfrac{d}{dt} f(q, \dot{q}, t)$, meaning under the vector field, the Lagrangian changes according to a total time derivative of some function $f(q, \dot{q}, t)$. There is a difference between $\mathcal{L}_{X}L = 0$ and $\mathcal{L}_{X}L = \dfrac{d}{dt}f(q, \dot{q}, t)$, where the Lagrangian being symmetric is no neccesarily being invariant. The big thing to notice is that we can modify the Lagrangian by a total time derivative and the physics (governed by Euler-Lagrangian equation) remains the same, i.e. for

\[L' = L + \dfrac{d}{dt}f(q, \dot{q}, t)\]

L’ and L gives the same set of Euler-Lagrange equations. To see this, since the Euler-Lagrange equation comes from the first order variation of the action being $0$, we have

\[\delta S'[q] = \delta \int_{t_1}^{t_2} L'(q,\dot{q},t) dt = \delta \int_{t_1}^{t_2} L + \dfrac{d}{dt}f(q, \dot{q}, t) dt = \delta S[q] + \delta \int_{t_1}^{t_2} \dfrac{d}{dt}f(q, \dot{q}, t) dt = \delta S[q] + \delta [f(q, \dot{q}, t_1) - f(q, \dot{q}, t_2)] = \delta S[q]\]

Where the total time derivative leads to boundary terms that vanishes under first order variation.This is why we say the Lagrangian is symmetric, not invariant.

Then, Noether’s theorem tells us we are guaranteed to find a conserved quantity (over time) under this transformation, i.e. there is some $F(q, \dot{q}, t)$ such that $\dfrac{d}{dt}F(q, \dot{q}, t) = 0$.

Ok, maybe I just really liked Noether’s theorem and want to waffle about it more. But what does this tell us? Well, at least for physical systems, all forms of “no free lunch”, i.e. conservation of energy, momentum, angular momentum, etc., are consequences of symetries. Then, what if symmetry is broken? Well, we can construct such (local) systems, for example, for a particle in a gravitational field:

and the momentum is not conserved:

Of course, if we expand the system (the Lagrangian) to include the generator of the gravitational field, say the earth, the momentum of the total system is conserved again. i.e. just like our succulent Chinese meal, on a local scale, the dumpling/momentum is free, but on a global scale it is not.

One could further ask what if we include forces like friction in the system? Well, we can no longer model such a system using a Lagrangian on its own. For example, in the case of friction, the conserved quantity energy “leaks” out of the closed system. The lost conserved quantity is dissipated as heat, which we can describe as entropy, where energy spreads into microscopic degrees of freedom that are intractable by our macroscopic description of the Lagrangian. Hence, energy is still conserved in a sense.

The limitation of learning algorithms

In the field of machine learning, we have exactly a theorem named No Free Lunch theorem (NFL). It states that for any two optimization algorithms, averaged over all possible problems, they have the same performance.

Let $\mathcal{X}$ be the input space, $\mathcal{Y}$ the output space, and let $f: \mathcal{X} \to \mathcal{Y}$ represent a target function drawn from the set of all possible functions $\mathcal{F} = \mathcal{Y}^{\mathcal{X}}$. Consider a learning algorithm $A$ that, given training data, outputs a hypothesis $h_A$ to predict unseen inputs.

Assume that all functions $f \in \mathcal{F}$ are picked uniformly. Then, for any two algorithms $A$ and $B$, the NFL theorem states:

where $L(h,f)$ is a loss function measuring the error of hypothesis $h$ on target function $f$.

Equivalently,

Averaged over all possible functions, all algorithms have identical expected performance.

Essentially, any algorithm’s advantage on some problems must be exactly offset by its disadvantage on others. The “free lunch” of a universally superior learning algorithm does not exist.

This doesn’t mean that it is worthless to find learning algorithms that are good on some problems, because in practice, we often have a priori knowledge of the problem at hand, and design algorithms that are tailored to the problem (We enjoy the benefit of a local “free lunch”).

One thing that is interesting is to consider what exactly is being conserved quantity here? Well, suppose we pick a learning algorithm $A_t$ continuously over time, then the expression

is conserved over time. This could be a result of the symmetry of the uniform distribution over all choices of $f$, but we don’t have a formal phase space here. Maybe it would be interesting to introduce some functional space, and see if we can find a corresponding Lagrangian that gives rise to this conserved quantity. We could possibly discover some other forms of weaker NFL theorems for specific choices of classes of functions $f$, who’s distribution exhibits some symmetry.

Arbitrage: the lunch that is really free:

For market makers, they actually get real, physical free lunches by running arbitrage strategies. The whole point of arbitrage is to make risk-free profit; hence, even on a global scale, market makers are not taking any risk (in theory).

Let these arbitrageurs (yes, this is what they are called) make their bread. For the rest of us, we enjoy consistent prices across markets. What can we do then?

Well, in the field of asset pricing, the most central result (as the name implies) is the Fundamental Theorem of Asset Pricing (FTAP). The most simplistic version of this theorem can be summarised as:

“An asset is arbitrage free if and only if there is a risk neutral probability measure $\mathbb{Q}$, under which discounted asset price are martingales.”

In a maths setting, how do we interpret the idea of no arbitrage? We first construct a strategy $\phi = (\phi_{i})_{i=0}^{k}$ we define as “self-financing”, which basically means there is no cost of trading as time goes on (change of strategy is only due to change in asset price). The present value of the strategy at time $t$ is given by:

where $\tilde{S_{i}} = \frac{S_{i}}{B}$ is the discounted asset prices, and $B = e^{rt}$ is the risk free asset price.

In a world of No arbitrage (NA), all strategies that is self-financing, with $V_{0} = 0$, cannot have a positive payoff, i.e. $\mathbb{P}[V_{T} > 0] = 0$.

Under this definition No arbitrage (NA), We then want:

\[\text{No Arbitrage (NA)} \iff \mathbb{Q} \sim \mathbb{P} \text{ and } \tilde{S_{t}} \text{ is a } \mathbb{Q} \text{-martingale}\]

This is a very non-trivial result to prove, and for a full proof with all the details and nuances (warning: functional analysis ahead), see here.

Under such theoretical jargon, this theorem may sound a bit out of reach. However, there is an intuitive way to explain the equivalence.

Suppose we are in a world of no arbitrage. In this world, suppose an asset has some drift, i.e. $S_{t} = \mu dt + \sigma dW_{t}$. Will an investor prefer to hold this asset over a risk-free one? Well, in a world with arbitrage, risk-averse investors will price $S_{t}$ differently than risk-seeking ones.

This different price on the same asset allows “free lunch”, where an arbitrageur can buy the asset from the risk-averse investor at a lower price and sell it to the risk-seeking investor at a higher price. The no-arbitrage condition forces investors to agree on the same price, which is equivalent to them not having a risk preference, hence the word risk-neutral.

In a nutshell, FTAP gives us two equivalent lenses to price an asset. The first is the more intuitive perspective of no arbitrage. For any asset at a given time $t$, it has a price $S_{t}$ agreed by all market participants. This can be seen as a “canonical” price for an asset. If an asset is risk-free, whether by definition or by construction (i.e. combining risky assets together, we call this hedging), its price evolution will be identical to that of the risk-free rate. In this lens, we don’t require any expectation of future value to price an asset.

The second perspective, which is equivalant to the first via FTAP, is using the risk neutral probability measure $\mathbb{Q}$.

Notice there is no explicit hedging here. It is all done implicitly via the existence of $\mathbb{Q}$, instead of dragging $\mu$ around and hedging it away every time, we just shift to the measure $\mathbb{Q}$ where drift is the risk free rate $r$. In measure $\mathbb{P}$, The excess $\mu - r$ is exactly the compensation for investors holding a risky asset.

(I’m guessing this is why people name those who price/sell derivatives as $\mathbb{Q}$ quants, and those who purchase said derivatives as $\mathbb{P}$ quants. )

We can also treat FTAP as a conservation law. In the space of possible strategies, the risk-adjusted return is conserved: The best you can do is the risk-free rate. This is because after discounting, wealth processes are martingales in a specific measure, meaning we can think of one’s wealth as the result of repeated, independent fair games, and fair games have expected payoff of 0. We can even write down the conserved quantity, where

Perhaps we have too many conserved quantities here, as any discounted wealth process leads to such a conservation! This is perhaps the strictest no free lunch of the 3, as here, we are looking at every strategy on their own, and individually on a local level, there is no free lunch to be found.

Under measure $\mathbb{P}$, perhaps there aren’t that many such wealth processes, maybe if your strategy is to invest in a risk free asset, then the conservation holds.

It may be harder to find symmetry in the underlying space of possible strategies that correspond to this conserved quantity, but still, we have managed to locate some form of conservation in accordance to no free lunch.

What have we learned about free lunches?

It is only appropriate to end this discussion with some of the insights we have gained from free lunches. As we have mentioned in the introduction, Free lunch tends to be a local illusion that vanishes on a global scale. From the perspective of Noether’s theorem, we start to associate conservation of certain quantities (lack of free lunch) to symmetries, and can precisely identify when a local break of symmetry (leading to local free lunch) is nonexistent at a global scale.

In more complicated scenarios, we are still able to identify forms of conservation. It still seems that there is a lot of room to move the Noether machinery to these scenarios, but it is quite clear that, fundamentally, conservation or the lack of conservation are dependent on perspective.

This recurring idea suggests that “gainz” in any domain is never absolute; it is merely the rebalancing of trade-offs under different assumptions. When we find a free lunch, it only means we have not yet looked far enough to see who is paying the bill.