Drawdown: Signal or Noise

Drawdown: Signal or Noise

The fourth article in the validation series. It follows The Best of N Is Not a Result. Every number is reproducible — script linked at the end.

---

If you read nothing else: an alert threshold set at a round number like “minus twenty percent” carries no information about the state of your strategy. The only informative threshold is one derived from the drawdown distribution of that specific strategy — and it can be computed in advance, before you launch.

Two weeks of losses

You launched a bot. Two weeks have passed. The result is negative, with a drawdown of around ten percent.

Has the strategy broken?

Let us check by calculation. But first, some arithmetic, without which the numbers below cannot be read.

How many trades is that, actually

Fifteen-minute markets appear four times an hour, which is 96 per day.

2 days    =   192 markets
2 weeks   = 1,344 markets

But a market and a trade are not the same thing. A strategy does not enter everywhere: it has conditions, and it skips whatever fails them. So the number of trades depends on selectivity:

Two hundred trades is either two days of continuous trading or two weeks of selective trading. This matters below, because the question “is two weeks a lot or a little” has no answer without a trade count.

How the numbers below are computed

Take a strategy with a genuine edge: it forecasts two percentage points better than the market, enters at 90¢, position size 2% of capital.

On each trade it is right with probability 92% (that is the 90% implied by the price plus two points of edge). If right, it collects the payout minus the fee; if wrong, it loses its stake. The exact figures at this entry price: a win returns +10.4% of the stake, a loss −100.7%.

Then comes the simulation. Play out two hundred such trades in sequence, track the capital, record the final result and the deepest drawdown along the way. Repeat twenty thousand times. What emerges is the distribution of what an observer can possibly see over two hundred trades of a perfectly functional strategy.

Here it is:

Every fourth such stretch of a working strategy ends in a loss. A five-percent drawdown you will see almost certainly. A ten-percent one in a quarter of cases.

The picture you are looking at is exactly this one, and it is entirely consistent with a strategy that is working correctly. Switching the bot off now means switching it off because of an event that was bound to happen.

But the converse holds too, and it matters more. Had those same two hundred trades ended in profit, that would not have been evidence either. You would simply have landed in the remaining 75%.

And immediately — why “two weeks” means nothing

The same calculations at different selectivity:

The first two rows are practically identical, though they differ sevenfold on the calendar. And two weeks of active trading versus two weeks of selective trading are entirely different situations: a 5% chance of finishing in the red in one, 25% in the other.

Calendar time by itself means nothing. What matters is the trade count. Everything further in this article counts trades, not days.

---

The claim, stated so it can be broken

I claim: a decision to stop a strategy based on a drawdown expressed in round percentages contains no information about whether the strategy is working. The only informative comparison is between the observed drawdown and the distribution of drawdowns expected for that strategy at its measured quality and track length.

I am wrong if: a strategy’s drawdown distribution does not depend on its Sharpe ratio and trade count — that is, if one threshold is equally meaningful for different strategies. Checkable with the attached script in about a minute.

---

Why drawdowns are unavoidable even when everything is fine

The reason is simple but counterintuitive.

A strategy with positive expectation still loses a share of its trades. The losses do not arrive evenly — they cluster, because that is what randomness looks like. A long enough run of them and the capital goes down.

The question is not whether there will be a drawdown. There will be. The question is how large a drawdown is normal for your strategy.

Here is the answer, computed for a strategy with a two-point edge over a thousand trades:

Read this carefully, because there are three separate conclusions here.

First. The forecasting quality is identical in every row — a two-point edge throughout. And the median drawdown differs sixfold. One alert threshold cannot be meaningful for the top and bottom rows at the same time.

Second. The strategy at 50¢ has a median drawdown of half the capital. Not in the worst case — in the typical one. The reason is in the first article: the fee eats seven eighths of the edge there, the Sharpe falls to 0.005, and the capital wanders almost blind.

Third, and the most unpleasant. Look at rows 0.25 and 0.50: a strategy with a genuine edge has a negative median result. It really does forecast better than the market — and over a thousand trades it more often than not loses money, because the fee and the dispersion overwhelm the advantage.

---

How long you have to watch before anything is knowable

Since short windows show nothing, the question becomes: what window does show something?

(A strategy with a genuine edge entering at 90¢ — the best case from the previous table.)

Twenty trades is a coin flip: nearly half of such stretches end in the red. Even at a thousand trades a working strategy finishes in the red seven percent of the time.

Note the second pattern too: the longer you watch, the larger the drawdown you will see. That is not the strategy deteriorating. The longer the track, the more chances there are for a bad run to occur somewhere in it. A month’s drawdown cannot be compared directly with a year’s — they have to be brought to a common length.

---

The main case: live drawdown exceeded the backtest maximum in three days

Everything above is about patience. Now the opposite: when a drawdown really is a signal.

Let us work through a specific case, because it comes up more often than any other and looks equally alarming regardless of what is behind it.

The backtest was done carefully — with walk-forward, with an out-of-sample check. Maximum drawdown in the report: 30%. Position size 1% of capital. The bot runs on fifteen-minute markets, which is 96 markets a day. Three days in, the live drawdown is already past thirty percent.

The usual first reaction: “well, that is only thirty losing trades in a row, and out of three hundred over three days that is easy.” Start with the fact that this intuition is wrong.

Thirty losses in a row do not happen

Compute the probability of a run of thirty losses within 288 trades:

At a 50% win rate the worst run over three days is around seven losses, not thirty. Thirty in a row is a one-in-four-million event.

But a drawdown accumulates not through a consecutive run but through a net excess of losses over wins. That part is real, and that is where the main story begins.

The answer depends entirely on where you trade

I computed the maximum drawdown over 288 trades at 1% position size:

The spread is enormous. The same observed picture — thirty percent in three days — means:

• at 90¢: this does not happen at all. Not even without an edge. Look for a mistake in position size, in execution, or in the code;
• at 50¢: an event of 6% to 18% probability — rare, warrants investigation, but not fatal;
• at 25¢: this is the median. It is supposed to look like that, and there is no signal here at all.

The reason is what the first article covered:

At cheap entries you win rarely and large. Thirty losses in a row is not an anomaly there but an ordinary Tuesday: the average worst run at a 10¢ entry is thirty-two. A strategy that earns from rare large wins is obliged to spend a lot of time in drawdown — otherwise it would not be that kind of strategy.

So the first question is not “why the drawdown” but “what is my average entry price.” Without that number, discussing drawdown is pointless.

A general rule you can apply to your own strategy

The tables above are built for specific entry prices, which is convenient for understanding the mechanism. But your strategy does not trade at exactly 0.25 or exactly 0.90, so you need a way to carry all of this over to yourself.

It is simple, and it rests on one fact.

A losing trade costs you your position size — always, at any entry price. You lose everything you put in: bought with 1% of capital, lost, minus 1%. Entry price does not affect this.

So the depth of a drawdown is determined not by entry price directly but by how many losses cluster together. And that depends on the win rate — a number you already have from your backtest.

Average length of the worst consecutive losing run:

From which a one-step estimate:

drawdown ≈ position size × worst run length × 2

The factor of two is needed because a drawdown accumulates not through one consecutive run but through a net excess of losses over a longer stretch. I checked against simulations — the actual drawdown comes out 1.5 to 2.5 times the product:

How to apply it to yourself, in three minutes:

1. Take the win rate and trade count from your backtest, find your row in the table.
2. Multiply by position size as a percentage of capital.
3. Multiply by two.
4. Compare against what you are seeing live.

It comes out rough, but rough enough to separate “expected” from “something is wrong.” The exact distribution is computed by reshuffling trades, as described further down in the alerts section — but for a decision in the moment, this estimate is enough.

Hold on, didn’t the first article say win rate was useless?

It did — and this is not a contradiction, but it needs explaining, because the question is a fair one.

The first article argued that win rate does not tell you whether you are making money. Profitability is the difference between your accuracy and the entry price. A single number cannot give you a difference: 60% accuracy is profitable at an entry of 0.58 and unprofitable at 0.62.

Here the win rate is being used for something else — for the shape of the distribution. The length of a losing run depends only on the probability of a loss. Entry price has no effect on it whatsoever: a run of five losses is equally likely at 25¢ and at 75¢ for the same win rate.

Difference and level are different quantities. The win rate gives you the level, not the difference.

But the rule has an assumption, and it should be named. The depth of a drawdown depends not only on how many losses there are but on how much the wins recover — and that is determined by entry price. The same 60% win rate can be reached in different ways:

As long as the edge stays within one to five points — and this whole series is about the fact that real edges are exactly that size — the drawdown changes little and the rule works. The mechanism is simple: at a small edge the win rate is nearly equal to the entry price. A two-point edge means a 60% win rate was achieved at an entry of 0.58; knowing one tells you the other.

If your strategy has an edge larger than ten points, the win-rate rule will start overstating the drawdown — use the entry-price tables above instead, they are more accurate.

And a neat closure with the first article: the win rate cannot tell you about profitability precisely because it is nearly equal to the entry price — the difference between them is small and invisible inside the win rate itself. And for the same reason it can tell you about the shape of the distribution, which depends on the level rather than the difference.

And straight away, the crucial sanity check on sizing. If you set position size to 5% at a 50% win rate, then the worst run of six losses over a hundred trades gives you minus thirty percent, and with the multiplier a full sixty. That is not risk management, that is a matter of time. Position size is the only variable in this formula you control directly.

Depth is not the only measure. There is also duration

A shallow drawdown can be more punishing than a deep one, and here is why.

How many wins it takes to recover one loss:

And what that produces in practice:

At high entry prices the drawdown is shallow — and climbing out of it takes almost ten wins per loss. The curve grinds gently upward in small steps, then a cliff, then another long gentle climb.

Psychologically that is harder than a deep fast drawdown: the account barely grows, and every setback throws you back a dozen trades. People switch off strategies like this not because of the depth but because of the sense that nothing is happening.

Practical consequence: measure time underwater, not only depth. A strategy that sat at the bottom for a month and one that fell and recovered in two days can have the same maximum drawdown and be entirely different to live with.

The fragility of edge at high entry prices

And one last thing worth knowing about the top of the scale, because it runs against the feeling of safety that a shallow drawdown creates.

Breaking even at a 0.90 entry requires 90.63% accuracy. Here is what happens near that boundary:

Two percentage points of accuracy flip the sign of the result.

This is the real danger of high entry prices — not the size of the drawdown but the fact that the margin between “works” and “does not work” is a couple of points. A small error in estimating the edge, slightly worse execution, a market that shifted a little, and a profitable strategy becomes an unprofitable one — and you will barely notice from the drawdown, because it stays shallow either way.

Quiet unprofitability is more dangerous than loud unprofitability.

One more correction: the backtest maximum is the least reliable number in the report

Even if the entry price says your drawdown is anomalous, there is another correction to apply.

The maximum drawdown in a backtest is one observation of an extreme on one path. Not a property of the strategy but the outcome of a particular set of circumstances.

I ran the same strategy three thousand times over five thousand trades:

 5th percentile of observed max drawdown:   7.5%
50th percentile:                           11.2%
95th percentile:                           17.9%
range across all runs:                5.2% — 28.9%

A fivefold spread. The same backtest on an adjacent slice of history would have shown a number twice different.

So your 30% from the report might be a typical value, an unlucky outlier, or — most importantly — a lucky understatement. If the strategy was selected for the beauty of its backtest, you almost certainly picked a configuration with an understated observed drawdown. That is the same selection the third article was about, applied to risk rather than return.

What to do: reshuffle the backtest trades a thousand times and look at the distribution rather than at one number. If 30% turns out to be the median, compare against thirty. If it turns out to be the 95th percentile, the real reference is around 18%, and the live excess is more serious than it looks.

And the last one: compare per unit of track length

Drawdown grows with the number of trades. You cannot compare 288 live trades directly with five thousand backtest ones.

Growth is slower than the square root — positive drift pulls the capital back up and shortens the troughs.

Hence the rule: if the backtest ran over 5,000 trades, then over 288 live ones you should expect around 40% of the backtest drawdown, not 100%. Reaching the backtest maximum in one seventeenth of the time is an excess of roughly two and a half times.

The direct probability that a drawdown over 288 trades will beat the maximum accumulated over 5,000: 0.4% at a 50¢ entry and 2.4% at 90¢. That is a signal.

The order of diagnosis

If after all the corrections above the drawdown still looks anomalous, work through it in this order. It runs from the most common and cheapest to the rarest and most expensive.

First — position size. Make sure the live bot is staking the same 1% the backtest was. This is the most common mistake and the most galling: double the size gives double the drawdown with a perfectly sound strategy.

Second — execution. Compare actual entry prices against what the backtest assumed. If the live bot is entering worse, the entire drawdown may be explained by slippage while the signal is fine.

Third — fees. Check what rate was actually charged against what the run assumed. The market’s fee category is not always the one shown in the interface — the first article covered this.

Fourth — entry price. Compare the distribution of entry prices between live and backtest. If live consistently enters more expensively, it has a different asymmetry and a different normal drawdown at the same signal.

Fifth, and only now — the strategy itself. If the first four check out, you are looking at either overfitting or a regime change.

Note that four of the five causes have nothing to do with the quality of the strategy. That is why switching the bot off on day three is premature — not because the drawdown is normal, but because you do not yet know what broke.

What order to check in: by speed of detection

The list above runs from common to rare. But there is a second criterion, more important and rarely considered: how fast you can get an answer at all.

I computed how many trades it takes to notice each kind of divergence:

That last row is a verdict on every attempt to work this out quickly. At a 92% win rate, statistically confirming a two-point drop in accuracy takes on the order of three thousand trades. If you have 288, you cannot establish signal degradation. Neither confirm it nor rule it out.

Hence the rule: check in order of speed of detection, not in order of importance. Start with what is visible today.

The main fork — one question

The whole diagnosis reduces to a single comparison:

Does your live win rate match the backtest one?

It matches — then the forecast is working the same as in the backtest, and the money is being lost somewhere between signal and result. The cause is in costs or execution: fees, spread, slippage, entry price, position size. All of it is measurable today, on the trades you already have, and all of it is fixable.

It does not match — then the quality of the forecast itself has changed. But before drawing that conclusion, look at the table above: do you have enough trades to distinguish a genuine drop in win rate from a random fluctuation? Usually you do not.

The practical sequence that follows:

1. Compare the live and backtest win rates. If the difference is smaller than what you are able to detect at your trade count, treat the win rate as matching and take the costs branch.
2. In the costs branch, decompose the difference: how much was eaten by fees beyond what was assumed, how much went to worse entry prices, how much was forgone through unfilled orders.
3. If the difference is now explained, the problem is found — and it is not the strategy.
4. If it is not explained and you have enough trades, then you are looking at the signal.

Step 3 closes most cases. Which is why switching the bot off without working through this list is premature: with high probability you are switching off a sound strategy with badly configured execution.

Separately, on “but I did everything right”

Walk-forward and out-of-sample checks lower the risk of overfitting but do not eliminate it. Two reasons, both covered in the third article.

Re-optimising in each walk-forward window is itself multiple testing — you sweep configurations in every window, and the best of them make it into the final result.

And an out-of-sample check stops being a check if you went back to it. Looked, disliked it, adjusted, looked again — the window has been spent.

So “I did everything right” is not a guarantee but a reduction in probability. And the statistics of your live drawdown know nothing about it: they look identical under overfitting and under a mistake in position size.

---

The formal answer: how many trades before you can claim anything

There is an exact calculation of how many observations are required to claim “the edge is above zero” with ordinary confidence. It is called the minimum track record length, and it accounts not only for the Sharpe ratio but for the skew of the distribution:

A hundred and eight thousand trades at a 50¢ entry — and that is with a genuine two-point edge. Until then, any claim about whether the strategy works is statistically unsupported.

The figures roughly match those in the first article, and that is no coincidence: the same calculation was done there through the t-statistic. Here a correction for skew has been added, so the values at the ends of the scale differ slightly.

---

How to build an alert, then

Everything above implies a construction with three levels. None of them is a round number.

Level 1 — observation. The drawdown has passed the 95th percentile of the drawdown distribution for this strategy.

The distribution is computed in advance, before launch: take the trades from the backtest, reshuffle them many times, compute the maximum drawdown for each shuffle. The resulting curve is where the threshold comes from.

For the strategy in our tables (0.90 entry, a thousand trades) it looks like this:

So the threshold for this particular strategy is around 24%, not “twenty” and not “ten.” For a strategy at a different entry price it will be different.

What a trigger means: a rare event, but not an impossible one — it happens once in twenty times even with a perfectly sound strategy. Grounds for looking, not for switching off.

Level 2 — verification. The empirical edge after costs has moved outside the confidence interval obtained out of sample.

This is about forecast quality rather than the path of capital. It is computed from the calibration table in the second article: if price buckets that used to be profitable have dropped below the breakeven line, something substantive has changed.

Level 3 — stop. One of two things:

• the market is in a regime the strategy was never validated in (volatility changed, market structure changed, the participants changed);
• the track length has already exceeded the minimum, and the lower bound of the confidence interval on the edge still has not separated from zero.

The second point is the most underrated. It means: you gave the strategy enough time and it did not prove that it works. Not “proved that it does not work” — simply did not prove it. That is sufficient grounds to stop, because you are paying fees the whole time.

---

The symmetric rule everyone forgets

All of the above has to be applied in both directions.

If you are prepared to switch a bot off after two bad weeks, you are obliged to be equally sceptical of two good ones. Otherwise you have a filter that admits only pleasant news: failures are attributed to a broken strategy, successes to its quality.

It follows directly from the tables above: at twenty trades the probability of finishing in profit is about the same as finishing in loss. A profitable week is exactly as uninformative as a losing one.

Practical consequence: kill criteria are written before launch, together with the success criteria, and are not revised along the way. A decision made after you have seen the result is not a decision but a reaction.

---

What would refute this

The entire calculation rests on the drawdown distribution being derivable from the Sharpe ratio and the track length. Run the script with your own strategy’s parameters. If drawdowns did not depend on the Sharpe, every row of the first table would be identical and the article would be wrong.

A more serious check: reshuffling trades assumes they are independent. If your trades are autocorrelated — and on short-horizon markets they often are — reshuffling understates the drawdown, because it breaks up the real bad runs. This is directly testable: compute the autocorrelation of your trade returns. If it differs noticeably from zero, every threshold in this article is optimistic, and instead of plain reshuffling you need a block bootstrap — reshuffling in chunks, which preserves runs.

This is not a minor caveat. It is probably the method’s principal limitation, and I cannot say in advance how large it is for your strategy.

---

Limitations — the complete list

Independence of trades. Covered above; under autocorrelation every threshold is understated.

Stationarity. The drawdown distribution is computed from past behaviour. When the market regime changes it stops describing the future, and any threshold calibrated on the past errs with a lag.

Fixed position size. All calculations assume a constant 2% of capital per trade. A varying size produces a different distribution, usually with a fatter left tail.

Hold to resolution. As throughout the series, the calculations are for trades that survived to market resolution. Stops and take-profits change the shape of the distribution, and it has to be computed from actual trades rather than from this model.

Drawdown is not the only risk. It measures depth but not duration. A strategy that sat at the bottom for six months can formally have the same maximum drawdown as one that fell and recovered within a week. These are different situations, and time underwater is worth measuring separately.

---

What to do with this tomorrow

1. Estimate your normal drawdown in three minutes. Win rate and trade count from the backtest → your row in the runs table → multiply by position size → multiply by two. Rough, but enough to separate “expected” from “something is wrong.”
2. Compute your strategy’s drawdown distribution before launch. Reshuffle the backtest trades a thousand times, build the percentile curve. The alert threshold is the 95th percentile, not a round number.
3. If live diverges from backtest, start with the win rate. Matches → look in costs and execution, measurable today. Does not match → first check whether you have enough trades to tell a genuine drop from a fluctuation.
4. Check the autocorrelation of your trade returns. If there is any, raise the thresholds and use block reshuffling.
5. Write your kill criteria before launch and date them. Anything decided after looking at results is not a decision.
6. Compare your track length against the minimum. If you have two hundred trades and need ten thousand, the question “does the strategy work” has no answer yet — positive or negative.
7. Measure time underwater, not just depth. Two strategies with the same maximum drawdown can be entirely different to live with.
8. Apply scepticism symmetrically. Every time you are about to act on a result, ask: would I make the opposite decision on the opposite result of the same size?

---

FAQ

My drawdown is 30% and the calculated threshold is 24%. Switch off? Look, do not switch off. A level-1 trigger means a rare event, not a breakage — it happens once in twenty times with a perfectly sound strategy. Check level 2: has the edge after costs changed?

How long should I wait if the strategy is not producing? Until the minimum track record length. If it has been exceeded and the confidence interval on the edge still includes zero, that is your answer: the strategy has not proved that it works, and there is nothing left to pay fees for.

Can I reduce the drawdown with a smaller position size? As a percentage of capital, yes, proportionally. But the required track length does not change: it depends on the Sharpe ratio, and the Sharpe does not depend on position size. Shrinking the position makes the risk tolerable, not the result more reliable.

Why can a strategy with a genuine edge have a negative median result? Because the median and the mean of a skewed distribution do not coincide. A positive expectation can be composed of rare large wins and frequent small losses. Which is exactly why looking at the result of one run is meaningless — you need the distribution.

---

Not financial advice. Past results do not predict future results. Trading prediction markets carries risk up to and including total loss of capital. Verify that trading on these venues is legal in your jurisdiction.

Sources. Magdon-Ismail & Atiya, Maximum Drawdown, Risk 17(10), 2004 — the analytical link between expected maximum drawdown, mean return and the Sharpe ratio. Bailey & López de Prado, The Sharpe Ratio Efficient Frontier, Journal of Risk 15(2), 2012 — the Probabilistic Sharpe Ratio and minimum track record length. Politis & Romano (1994) — the block bootstrap for dependent series.