Legend has it, a pharmacist named John Pemberton was searching for a headache cure when he tried blending Coca leaves with Cola nuts. Who knew his recipe was destined to become such a smashing success, even if Coca-Cola® never did become the medicine Pemberton had in mind?

In similar vein, when Charles Dow launched the Dow Jones Industrial Average (the Dow), his aim was to better assess stock prices and market trends, hoping to determine when the market’s tides had turned by measuring the equivalent of its incoming and outgoing “waves.” He chose industrials (mostly railroads) because, as he proposed in 1882, “The industrial market is destined to be the great speculative market of the United States.”

While the actively minded Dow never did achieve market-timing clairvoyance (and neither has anyone else we’re aware of), he did devise the world’s first index. We’d like to think his creation turned into something even greater than what he’d intended – especially when Vanguard founder John Bogle and other pioneers leveraged Dow’s early work to create among the most passive ways to invest in today’s markets: the index fund.

Bogle launched the first publicly available index fund in 1976. Initially dismissed by many as “Bogle’s folly,” its modern-day rendition, the Vanguard 500 Index Fund, remains among the most familiar funds of any type.

Index Investing Is Born

In defense of Dow’s quest to forecast market movements, it’s worth remembering that his was a world in which electronic ticker tape was the latest technology, there were no open-ended mutual funds or fee-only financial advisors, and safeguards and regulations were few and far between. Essentially, speculating was the only way one could invest in late-nineteenth century markets.

Compared to actively managed funds that seek to “beat” the market by engaging in these now-outdated speculative strategies, passively managed index funds offer a more solid solution for sensibly capturing available market returns. As the name implies, an index fund buys and holds the securities tracked by a particular index, which is seeking to represent the performance of a particular slice of the market. For example, the Vanguard 500 Index Fund tracks the popular S&P 500 Index, which in turn approximately tracks the asset class of U.S. large-company stocks.

Compared to actively managed solutions, index funds lend themselves well to helping investors more efficiently and effectively target these three pillars of sensible investing:

  1. Asset allocation – How you allocate your portfolio across various market asset classes plays a far greater role in varying your long-term portfolio performance than does the individual securities you hold.
  2. Global diversification – Through broad and deep diversification, the sum of your whole risk can actually be lower than its individual parts.
  3. Cost control – The less you spend implementing a strategy, the more you get to keep.

Index Investing: Room for Improvement

As we’ve described throughout this series, indexes weren’t specifically devised to be invested in. There’s often a lot going on underneath their seemingly simple structures that can lead to inefficiencies by those trying to retrofit their investment products on top of popular indexes.

Index Dependence – Whenever an index “reconstitutes” by changing the underlying stocks it is following, any funds tracking that index must change its holdings as well – and relatively quickly if it’s to remain true to its stated goals. In a classic display of supply-and-demand pricing, this can generate a “buy high, sell low” environment as index fund managers hurry to sell stocks that have been removed from the index and buy stocks that have been added.

Compromised Composition – Asset allocation is based on the premise that particular market asset classes exhibit particular risk and return characteristics over time. That’s why your investment “pie” should be carefully managed to include the right asset class “slices” for your financial goals and risk tolerances. As we described in Part III of this series, if you’re invested in an index fund and you aren’t sure what its underlying index is precisely tracking, you may end up with off-sized pieces of pie. For example, the S&P 500 and the Russell 3000 are both positioned as U.S. stock market indexes, but both also track some real estate. If you don’t factor that into your plans, you can end up with a bigger helping of real estate than you had in mind.

Introducing Evidence-Based Investing

So, yes, index investing has its advantages … It also has inherent challenges. No wonder academically minded innovators from around the globe soon sought to improve on index investing’s best traits and minimize its weaknesses. In fact, many of these thought leaders were the same early adapters who introduced index fund investing to begin with. Building on index investing, they devised evidence-based investment funds, to offer several more advantages:

Index-independence – Instead of tracking an index that tracks an asset class … why not just directly capture the asset class itself as effectively as possible? Evidence-based fund managers have freed themselves from tracking popular indexes by establishing their own parameters for cost-effectively investing in most of the securities within the asset classes being targeted. This reduces the need to place unnecessary trades at inopportune times simply to track an index. It also allows more patient trading strategies and scales of economy to achieve better pricing.

Improved Concentration – Untethering themselves from popular indexes also enables evidence-based fund managers to more aggressively pursue targeted risk factors; for example, an evidence-based small-cap value fund often has more flexibility to hold smaller and more value-tilted holdings than a comparable index fund. This provides more refined control for building your personal investment portfolio according to your unique risk/return goals.

Focusing on Innovative Evidence – Evidence-based investing shifts the emphasis from tracking an index, to continually improving our understanding of the market factors that contribute to the returns we are seeking. By building portfolios using fund managers who apply this same evidence to their funds, you can make best use of existing academic insights, while efficiently incorporating credible new ones as they emerge.

An Index Overview, Revisited

From describing an index’s basic functions, to exploring some of the intricacies of their construction, we’ve covered a lot of ground in this four-part series on indexing. To recap, indexes can help us explore what is going on in particular slices of our capital markets. In the right context, they also can help you compare your own investment performance against a common benchmark. Last but not least, you can invest in funds that track particular indexes.

Equally important, remember that indexes do not help us forecast what to expect next in the markets, nor do high-water markets such as “Dow 20,000” foretell whether it’s a good or bad time to buy, hold or sell your own market holdings. And, while low-cost, well-managed index funds may still play a role in your overall investment portfolio, it’s worth ensuring that you select them when they are the best fit for your evidence-based investment strategy, not simply because they are a popular choice at the time.

What else can we tell you about indexes or index investing? Let’s take a look at your unique financial goals, and see how indexing fits into your globally diversified world of investments.