Your Six AI Models Are Really Two: The Math of Correlated Mistakes

What changed

Multiple AI models are often pooled together in the hope that combining their answers improves accuracy through a voting system. But testing six open AI models on 1,500 exam questions shows that six models’ opinions are not really six independent voices. Instead, they behave more like the equivalent of two independent models because their errors correlate strongly. A simple mathematical formula predicts how many independent opinions the pool truly represents by accounting for error overlap.

Why builders should care

This insight directly affects how AI-powered decision-making systems rely on ensembles or model voting. Developers often expect that adding more models lowers error proportionally through diversity of opinions. But if models make correlated mistakes, the number of models needed to reach a target accuracy is higher than anticipated. This raises costs and complicates reasoning about reliability, forcing a rethink of how many models really add value versus just redundancy.

The practical takeaway

Operators pooling six different AI models cannot assume those six answers are six independent data points. Instead, the pool’s effective size is closer to two collaborators yielding truly independent insights. This means strategies relying on majority voting to de-risk outputs face significantly diminishing returns as model counts increase unless new methods to reduce error correlation emerge. Budgeting and confidence estimates should adjust to these limits to avoid overestimating model diversity.

What to watch next

Future research or product efforts that reduce correlation of failures across models could unlock better ensemble performance and more reliable multi-model voting. Watch for new approaches to training, fine-tuning, or model architecture that tackle overlapping blind spots. Investors, founders, and builders should also track tools that quantify or visualize error correlation to optimize model portfolios effectively and cut wasted costs on redundant opinions.

AI Quick Briefs Editorial Desk