Why profit-and-loss leaderboards lie
Ranking by profit is ranking by luck, with skill as a rounding error at the top.
Every trading venue publishes one and every trader reads it as a ranking of ability. It is not. It is a ranking of outcomes over an unstated period with an unstated denominator, which is a different thing and mostly a worse one.
Distortion one: no denominator
A wallet up $50,000 on $2,000,000 staked returned 2.5%. A wallet up $5,000 on $8,000 staked returned 62%. The leaderboard puts them ten rows apart in the wrong order. Profit without the money at risk behind it is not a performance figure, and no major leaderboard shows the denominator.
Distortion two: survivor selection
Take enough wallets, rank them by profit, and the top of the list will be populated even if nobody in it has any skill at all — that is what randomness does to a large sample. The wallets that made the same bets and lost are not shown next to them. You are reading the winners of a lottery and being invited to infer a system.
Distortion three: the single-market fortune
One enormous position on one binary question, resolved once, is a coin-flip with a large stake. It produces the largest numbers on any prediction-market leaderboard and it contains the least information of any result on the board. Breadth — distinct markets, over time — is the thing that distinguishes a process from an anecdote, and leaderboards never show it.
Distortion four: open positions
Depending on the window and the venue, a leaderboard figure may include unrealised marks on positions that have not resolved. Those can and do evaporate. Only settled results are results — which is why our own scoring uses settled trips exclusively, and says so on the methodology page.
What to read instead
Realised return on money actually risked. Number of distinct markets. Consistency across them rather than one outlier. Staking discipline — does the size move with conviction or with mood. Drawdown survived. None of those are on a leaderboard, all of them are computable from a public wallet, and reading one is not hard.
Questions
Are trading leaderboards accurate?
The dollar figures are usually accurate. The inference people draw from them — that the top of the list is the most skilled — is not, because ranking by profit ranks by outcome and outcome is what luck is best at faking.
What should I look at instead of profit?
Realised return on the money actually risked, the number of distinct markets, consistency across them rather than one outlier, staking discipline, and how the wallet behaved in drawdown.
Does a big number on a leaderboard mean anything at all?
It means a big outcome happened. Whether it means skill depends entirely on how many distinct settled results are behind it.