Why Indexing Fails

The recent paper “Why Indexing works” [1] gives a probabilistic explanation of the futility of the Active process and why Passive Indexing is hard to beat. For every 1000 people who read the Wall Street Journal, maybe 10 read the Bloomberg Markets (BM) magazine and for every 10 who read the last month’s issue of BM maybe 1 read this research paper cited in the article [2]. And you don’t need a geologist to tell you that the chances to dig and find are small. This is why making a mathematical case against the underperformance of the USD 11 trillion plus active market using hypothetical probabilities is not easy.

Agreed that probabilities don’t lie but it’s the assumptions that generate those probabilities that could be flimsy. Probabilities can indicate or anticipate underperformance or outperformance but they cannot claim to own the basis for poor Active underperformance. Manufacturing hypothetical probabilities are an easy and rather effective way to feed back inefficiency into the system. The more the quant researchers, the more the probability manufacturing and hence more the inefficiency. The farcical aspect is that the probabilities cited by the authors of the paper in discussion assume efficiency.

The paper in question makes more than a few assumptions. First, it makes the assumption of Geometric Brownian Motion (GBM) [3] as a realistic statistical distribution to understand markets. Random walk, efficiency, normality is stone age thinking in today’s quantum world. There is enough literature out there on power law and its natural and stock market expression. Despite such prominence, the power law has an army of detractors. In such a context, the GBM hypothesis has limited relevance. Even if there was a perfect statistical distribution, thinkers like Eugene Stanley [4] and his global academic followers would have determined it and Mandelbrot [5] would have attained Wall Street cult. Second, the authors assume that Active managers are still working with redundant statistical distributions to generate alpha. The third assumption is a Trojan horse that turns the argument against Indexing itself. Think about it, is selection just for the Active manager? Can the Passive manager do without selection? Are the Indexing companies building baskets insulated from the selection process? Selection is everywhere and it blurs the line between what’s Active and what’s Passive.

Indexing is not just about a certain Market Capitalized weighted basket [6]. Indexing is about factors today and a few claim to have discovered 458 factors [7] like the 486 cryptocurrencies [8], overrated and ticking like time bombs. This means that the probability which the authors cite as working against stock selecting Active managers also works against the Beta and Smart Beta companies which are at the mercy of the same probabilities that grace the various Indexing methodologies, giving them a year of success (outperformance) and a few years of failure (underperformance). So just like Active managers, Indexing as an idea is destined to fail periodically not only because of its construction (which requires selection) but also because investors and asset managers have a few million choices to “Select” from.

This means that the positive skew problem highlighted by the authors is not just the bane for the Active manager but even disastrous for the Passive industry. Hence the probabilistic argument made by the authors in favor of Passive and against Active falls apart. The surviving Active manager’s search for ‘gold in garbage’ (a data science expression) has advanced beyond the use of generic statistical distributions. It’s time we saw some new age mathematical research that makes a real case to solve the alpha problems rather than the same banal smarter Passive, dumb Active overused themes.


[1] Heaton. J. B, Polson, N. G, Witte. J. H., “Why Indexing Works”, October 2015, SSRN.
[2] Another Reason for Active’s Decline, April/May 2017, Bloomberg Markets.
[3] Saichev, A. I., Malevergne, Y., Sornette, D. “Theories of Zipf’s Law and beyond”, 2009, Springer.
[4] Matia, K. Pal, M, Stanley, E. Salunkay, H. “EPL – Scale dependent price fluctuations for the Indian stock market”, 2004, Europhysics Letters.
[5] Mandelbrot, B., “A Multifractal Walk down Wall Street”, February 1999, Scientific American.
[6] Pal, M., “Is Smart Beta Dumb”, March 2017, SSRN.
[7] Authers, J. “A clinical test of the 5 most popular Smart Beta factors”, March 2017, Financial Times
[8] Osterrieder, J., Chan, S., Chu, J., Nadarajah, S. “A Statistical Analysis of Cryptocurrencies”, April 2017, SSRN.