An edge. Or a fluke?

A beautiful backtest can hide a lucky guess. We’re here to ask better questions.

Look beyond the backtest

When you try enough models, one can look extraordinary by chance. Fluke starts with a simple idea: to understand the winner, look at the whole search.

One winner. A bigger story.

It’s easy to fall in love with a line. Let’s see what sits behind it.

Twenty-four versions of a model were tried and only the best one was shown. Seen together, the winner is simply the luckiest of the lot. Illustrative demo: the curves are synthetic, not market data.

Fluke is research on how to judge an AI trading model by more than its best backtest. Three things come out of it.

Research notesPublishing

An edge. Or a fluke?

  1. Look at the search, not just the selection.
  2. Give uncertainty a seat at the table.
  3. Make the evidence easier to inspect.
Read note No. 1
The Fluke checkOn this page

Six questions to put to any backtest before trusting it.

Try the check
Research logIn progress

Every attempt on record: the idea, the data, the settings, and why it was kept or dropped.

    Illustrative rows.

    Three tools
    for one question.

    Each one answers the same question: is this result an edge, or a fluke?

    Click the three icons on the computer

    Is your result real?

    Paste your strategy’s returns and say how many versions you tried. Fluke Check gives back the Deflated Sharpe Ratio and one plain sentence:

    “There is a 34% chance this result is real.”

    The more versions you try, the likelier the best one is luck. Registration opens soon: tell me when it does.

    A record nobody can edit.

    Every signal is stamped on-chain, hash and time, before the trade. After enough months the strategy earns a public badge, with its Deflated Sharpe Ratio on live data.

    Sellers of bots and signals prove they are honest. Buyers check before they pay. Registration opens soon: tell me when it does.

    A bot you run yourself.

    Quantitative finance, not quantum computing. It runs on your own exchange account with a trade-only key, withdrawals off: Fluke never touches your money. You set the parameters and the risk.

    On sale only once its own live record is verified by Fluke Verified. Not on sale yet: join the waitlist.

    0 of 3 opened · click the icons on the computer

    Continue

    Conviction.
    With a paper trail.

    Our research direction is built around three habits. Simple to say. Worth getting right.

    the search.

    The winner is only one part of the story. Keep the attempts, choices and assumptions that led to it.

    beyond the fit.

    Ask how a model behaves on unseen data, with realistic costs and a process that respects time.

    the result.

    Look at uncertainty and the number of trials. A strong backtest earns attention, not automatic trust.

    Six questions. Switch on the ones the result can answer.

    Every question a result can answer moves the needle away from luck. A result that answers none of them is most likely a fluke.

    • Do you know how many versions were tried before this one?
    • Was some data kept aside and used only once, at the very end?
    • Does the test only move forward in time, never peeking ahead?
    • Are trading costs and slippage included?
    • Is the result adjusted for the number of trials, for example with a Deflated Sharpe Ratio?
    • Could someone reproduce it from the record of attempts?

    Probably a fluke. Ask more questions.

    A conversation starter, not a verdict. Nothing here is investment advice.

    It started with a question. Not “which AI has the best backtest?” but “what would make that result worth trusting?”

    Read the thinking behind Fluke

    Is Fluke investment advice?

    No. Fluke is research about how trading models are evaluated. Nothing on this site is a recommendation to buy or sell anything.

    Are the charts on this site real results?

    No. Every chart here is illustrative and drawn from synthetic data, to show how picking the best of many trials can flatter a result.

    What are the Deflated Sharpe Ratio and the Probability of Backtest Overfitting?

    Two published methods for judging a result in light of the search that produced it. The Deflated Sharpe Ratio adjusts a performance measure for the number of trials and the shape of returns. The Probability of Backtest Overfitting estimates how often the best choice on past data falls behind on new data.

    Who is behind Fluke?

    Fluke is an independent, early-stage research project led by its founder. The notes come first; the tools follow the research.

    How can I follow the work?

    Start with the research note on this page. New notes will be published here as the research moves forward.

    Read the research note