Point-in-Time Truth: What Did the AI Know, and When?

Here's a question that sounds easy: when your strategy made a trade, what did it know?

Most people never ask it. But if you can't answer it, you can't trust your results. This post is about why, and how we handle it at Pickled Lizard.

The quiet way tests lie

Imagine a strategy that buys every time the daily candle closes green. In a backtest, it looks great. But there's a catch. The test let the strategy "see" the close before it bought. In real life, you don't know the close until the day is over.

This is called look-ahead bias. The test used information from the future. It does not take much. One field updated late. One price that got fixed after the fact. One indicator that uses the full candle. The chart looks smart, but it was cheating, and nobody meant it to.

Look-ahead bias is hard to spot because it doesn't throw an error. Everything runs. The numbers look clean. You only find out when you trade it for real and the edge is gone.

Point-in-time truth

We use a simple idea to fight this. We call it point-in-time truth.

For every decision a strategy makes, we want to answer three questions:

1. What did it know? Every piece of data it used.
2. When did it know it? The exact time each piece of data was available.
3. Can we prove it? A record we can check later, that nobody changed.

If we can't answer all three, we don't trust the result.

Why AI makes this more important

AI models are good at finding patterns. That includes patterns that only exist because of a data leak. If future data slips into a test, a smart model will find it and use it. It will look like a genius. It's not. It found a crack in the test.

As more people use ChatGPT, Claude, Grok, or Cursor to build trading ideas, this gets bigger. More ideas, built faster, tested on data that may not be clean. That's a lot of fake winners.

We're not building an AI that tells you what to buy. We're building the infrastructure that forces AI trading ideas to prove themselves. Point-in-time truth is one of the main parts of that.

How we do it

A few rules we follow:

• Timestamps on everything. Each piece of market data is stored with the time it was actually available, not just the time it is "about."
• Strategies only see the past. At each step, a strategy gets the data that existed at that moment. Nothing later.
• Exchange-native data. We pull data straight from the exchange, with the exchange's own timing, so we're not relying on a cleaned-up copy made later.
• Immutable records. We never rewrite history. If something failed, the record stays. If a number was wrong, we add a correction. We don't erase the old one.

That last rule matters more than it sounds. If you can edit the past, you can always make it look better. So we made it so we can't.

Observed vs predicted

There's one more piece. Some of our market intelligence is observed. It's what actually happened: a price, a trade, a funding rate. Some of it is predictive. It's a guess about what might happen next.

We keep these apart and hold them to different standards. An observed fact needs a clean source and timestamp. A prediction needs a track record. Mixing them up is another way to fool yourself, because a guess starts to look like a fact.

What this looks like for you

If you test a strategy on Pickled Lizard, every trade has a time and a reason. You can look back and see what it knew when it acted. Paper trades use real fees, margin rules, and timestamps.

When we share results, it's in one format: "X of N net positive (simulated)." The full count, every time. No hiding the losers.

A quick test for your own setup

Ask yourself:

• Does my strategy ever use a value from the end of a bar to act at the start of that bar?
• Is any of my data "fixed" or updated after the fact?
• Can I show, for one trade, exactly what data the strategy had at that second?
• If I re-run the test next month, will I get the same answer from the same data?

If you're not sure about any of these, your results might be more hopeful than real.

Why I care about this

I started Pickled Lizard after the October 10, 2025 crash. I watched emotion and slow information wipe people out. I don't want to build another tool that makes people feel sure for the wrong reasons. I'd rather build one that shows you the truth, even when it's boring. You can read more about me and why I'm building this.

Try it

The beta is free. Bring your own AI and test your ideas on paper with point-in-time records.

Join the Founding 1,000

Paper trading only. Results are simulated. Not financial advice.