How to question an AI trading signal

Define the claim

A model output needs a specific meaning. Is it forecasting the underlying's direction, the probability of a positive trade result, or the net outcome of a particular option strategy? Those questions are different. A forecast for the next hour does not automatically support a position held for three weeks.

Ask which information was available when the prediction was made. A result that depends on later data cannot represent the decision available at the time.

Compare it with something simpler

A complex model should be tested against a simpler approach on the same evaluation period and under the same execution assumptions. Include fees, realistic fills, losses and periods in which the model did not trade. Separate training data from later evaluation data.

Confidence should be checked against observed outcomes. An explanation that sounds convincing is not, by itself, evidence of calibrated probabilities.

Keep the risk decision independent

Even a useful forecast can propose a trade that a portfolio cannot afford. Position size, liquidity, concentration and the need to preserve cash remain separate constraints. In our curriculum, ‘no trade’ is a legitimate conclusion, not a failed exercise.