Why can’t we forecast the weather a month ahead?

A common belief is that forecasts fail because the weather is partly random, or because our computers are not yet fast enough. Neither is the real reason. The deeper problem shows up in something far simpler than the sky: a double pendulum, which is just a pendulum with a second pendulum hanging from its end.

Nothing about it is random. Two rods, two weights, gravity, and Newton’s laws of motion. If you know exactly where the weights start and how fast they are moving, the laws fix everything that happens next. Physicists call a system like this deterministic: the present completely determines the future.

A pendulum with a joint

In the figure below each rod is 1 metre long and each weight is 1 kilogram. Drag either weight to a new starting position and let go. Start with a small nudge (press Small swing): the weights rock back and forth in a steady, nearly repeating rhythm. Then try Big swing, or lift the lower weight above the pivot, and watch the trail of the lower weight.

The small swing looks like an ordinary pendulum: regular and predictable. You could measure it for a minute and confidently say where the weights will be an hour from now. The big swing is different. The lower weight flips over the top, hesitates, reverses, whirls the other way. It never quite repeats. This irregular behaviour, produced by exact laws with no chance in them, is called chaos.

But irregular is not the same as unpredictable. A complicated dance is still perfectly predictable if you can calculate it. The real trouble is subtler.

Twins that don’t stay twins

No measurement is perfect. Suppose you measure the starting angle of a real pendulum and are off by a millionth of a radian. (A radian is the angle unit mathematicians prefer: about 57°. A millionth of a radian is about the width of a human hair seen from 100 metres away.) Surely such a tiny error cannot matter?

The next figure starts 20 pendulums, each one tilted a little further than its neighbour, by the amount you choose. At first they sit exactly on top of each other and look like one. Press Play and watch the moment they come apart. Then make the difference a thousand times smaller and try again.

With a difference of a millionth of a radian between neighbours, the pendulums move as one for about six seconds, and then within a few more seconds they are scattered across every possible position. Make the difference a thousand times smaller and they hold together for only about four seconds longer. Another thousand times smaller: about three seconds more. This is called sensitive dependence on initial conditions, and it is the defining feature of chaos.

How fast do errors grow?

To see the pattern, measure the gap between two twins as time passes. The next figure plots that gap for ten pairs of twins whose starting difference ranges from 10⁻³ radians down to 10⁻¹² radians (a trillionth). The vertical axis is logarithmic: each gridline is ten times the one below, so something that multiplies by the same factor every second shows up as a straight line.

Choose a starting difference with the slider, or tap a curve. The dashed line marks the gap at which we say the forecast has failed: 0.5 radians, about 29°. Look at where each curve crosses it.

For the big swing, each curve climbs in a rough straight line (wobbly, because the pendulum speeds up and slows down) until the gap is about as large as it can get, and then levels off: two pendulums cannot be more different than “anywhere versus anywhere else”. A straight line on this scale means exponential growth: the error multiplies by a fixed factor in each equal stretch of time. For this pendulum it grows about tenfold every one to two seconds.

Now read the table. Making the starting measurement ten times more precise does not make the forecast ten times longer. It adds a roughly fixed amount of time, here one to two seconds. Going from 10⁻³ to 10⁻¹² radians, a billion times better, buys only about twelve seconds. If the error grows by a factor of e^λt, where λ is the growth rate, then:

forecast time ≈ (1/λ) · ln(tolerance ÷ starting error)

In words: the forecast time depends on the logarithm of how precise you are, so every extra digit of precision buys the same small extra slice of time. Switch to the small swing and the picture changes completely: the gap grows only slowly, never reaches the failure line, and prediction works as far ahead as you like. Regular motion forgives small errors. Chaotic motion punishes them.

From pendulums to the sky

In 1961 the meteorologist Edward Lorenz at MIT was running a 12-variable model of the atmosphere on a small valve computer. Wanting to look at one run more closely, he retyped some numbers from a printout and restarted it. The printout had rounded the numbers to three decimal places (so a value like 0.506127 went back in as 0.506), while the computer worked with more digits. At first the new run matched the old one, then the differences “more or less steadily doubled in size every four days or so”, and by the second simulated month there was no resemblance at all.

In 1963 Lorenz published Deterministic Nonperiodic Flow, showing that even a three-variable model of convection behaves this way, and concluded that very-long-range weather prediction may be impossible. The atmosphere, like the big swing, is chaotic. Small errors in today’s measurements — a gust not observed over the Pacific, a temperature reading off by a tenth of a degree — double in a matter of days. Errors in the smallest motions, such as single thunderstorms, grow fastest of all and spread to larger and larger scales.

Lorenz’s three-variable model describes a slowly rolling cell of fluid heated from below: x is how fast it turns (its sign is the direction), and y and z describe how the temperature inside is arranged. The next figure runs two copies of it whose x differs at the start by the amount you choose, and draws their paths. Press Play. The famous two-lobed “butterfly” appears as the fluid flips unpredictably between turning one way and the other, and the two paths, at first one line, eventually go their own ways.

With a difference of 10⁻⁵ the paths part after about 13 units of model time; each factor of ten closer adds only about 2½ units. It is the same pattern as the pendulum’s table, in a model built to imitate the atmosphere.

How far ahead can we hope to go? A 2019 study by Fuqing Zhang and colleagues ran two of the world’s best forecasting models at very high resolution with almost perfect starting data. Their answer for day-to-day weather in the mid-latitudes (where most people live) was an intrinsic limit of roughly two weeks. Useful skill today runs out nearer ten days. Cutting today’s errors in the starting data tenfold, an enormous task, would add up to about five days: still far short of a month. That is the table of Figure 3, written in the sky.

Forecasters have learned to work with chaos rather than against it. Instead of one forecast, centres such as the European Centre for Medium-Range Weather Forecasts run an ensemble: 51 forecasts, 50 of them started from slightly altered versions of today’s best estimate, exactly like the fan of pendulums in Figure 2. When the members stay together, the forecast is confident. When they fan out, the forecast says so, as a probability: “60% chance of rain”.

So the answer to the opening question is that “deterministic” and “predictable” are different things. A system can obey exact laws and still be impossible to forecast far ahead, because it multiplies the error in what we know, and no measurement is perfect. Better computers and better observations still help: forecasts have gained about one day of useful range per decade since the 1980s. But each gain is paid for in exponentially harder work, against a wall that sits somewhere around two weeks.