There’s a reason we refer to our strategy for building durable, long-term wealth as evidence-based investing. There are a number of other terms we could use instead: Structured (getting warm), low-cost (definitely), passive (sometimes), smart beta (maybe), indexing (close, but) … the list goes on. But because the evidence is at the root of what we do, we believe it should also be at the root of what we call it.

That said, there is a great deal of evidence to consider, and some purported findings seem to contradict others. How do we know which evidence to take seriously and which are false leads?

Evidence-Based Investing: A Never-Ending Story

First, it’s worth noting that academic inquiry is never fully final, nor does it allow for absolutes in our application of it. As University of Chicago professor of finance and Nobel laureate Eugene F. Fama has said, “You should use market data to understand markets better, not to say this or that hypothesis is literally true or false. No model is ever strictly true. The real criterion should be: Do I know more about markets when I’m finished than I did when I started?” [Source]

With this caveat, there are still a number of important qualities to seek when assessing the validity of a body of academic evidence.

A Disinterested Outlook – Rather than beginning with a point to prove, ideal academic inquiry is conducted with no agenda other than to explore intriguing phenomena and report the results. It is then up to us practitioners to apply the useful findings.

Robust Data Analysis – The analysis should be free from weaknesses such as data that is too short-term or too small of a sampling; survivorship bias (wherein returns from funds that went under during the analysis period are disregarded); apples-to-oranges benchmark comparisons; or plain, old-fashioned faulty math.

 Repeatability and Reproducibility – Results should be repeatable in additional studies across multiple environments and timeframes. This helps demonstrate that the results weren’t just random luck or “data mining.” As AQR fund manager and founding principal Clifford Asness describes, “If a researcher discovered an empirical result only because she tortured the data until it confessed, one would not expect it to work outside the torture zone.” [Source]

Peer Review – To ensure that all of the above and more is taking place as required, scholars are expected to publish their detailed data sets, methodologies and findings in a credible academic journal or similar forum, so their credentialed peers can review their work and either agree that the results appear to be valid or refute them if they are not.

The Alternative: Data Foolery

If our emphasis on deep and diligent peer review sounds like it doesn’t really apply to you and your tangible wealth, think again. Echoing the sentiment about lies, damned lies and statistics, when faulty conclusions are inappropriately applied, the results can send countless investors astray, with real dollars lost.

Consider this fascinating exposé by journalist John Bohannon, “I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.” The evidence-based sting operation happened to take place within healthcare, but similar lessons apply to finance.

Bohannon began by conducting a deliberately flawed “study” to serve as a glaring example of poorly done research. He intentionally used a paltry data set of 15 participants in a one-shot clinical trial, and then heavily tortured the resulting data to extract a technically accurate if essentially meaningless conclusion that chocolate consumption contributed to weight loss.

Next, Bohannon mined his familiarity with the scientific publishing industry to submit his study to several journals that had questionable reputations with respect to their screening processes. Despite full disclosure of the study’s many weaknesses, Bohannon observed that his paper “was published less than 2 weeks after [our] credit card was charged.” He and his cohorts then launched an aggressive PR campaign to a global media who was apparently more interested in printing exciting sound bites than substantiating the validity of the sources involved.

The title of his resulting exposé, tells us how the story ended. A blitz of television, Internet and print media coverage suggested to audiences around the globe that they might want to go on a chocolate diet to lose weight. Faulty evidence in, garbage conclusions out.

The Preeminence of Peer Review

The point is not to dismiss all evidence as bogus. The point is that substantive, meaningful peer review remains an essential component in separating real academic evidence from the wider glut of sloppy work that all too often occupies headline-grabbing news. Peer review also enables scholars to reference and build on their colleagues’ best work, which enables collective insights into important subjects to deepen and expand over time.

That’s why the evidence that survives the gamut of academic peer review, and has withstood the test of time is the evidence that we are most interested in applying to a set of orderly (if never certain) principles to guide our practical, evidence-based investment strategies.

The Adviser’s Essential Role: Separating Fact from Fiction

As evidence-based advisers, we are continuously scanning the work being presented to the public, filling in the due diligence that might have been missed, and translating the results with an eye toward helping investors appropriately view the big picture that emerges.

Of all the ways we can go about investing, which ones are expected to best serve our clients’ personal interests and financial goals? Equally as important, which are more likely to distract or detract from our efforts? The evidence-based answers to these vital questions explain why we pay so much attention to qualities such as fund structure, cost management, patient trading, and global market exposure (beta) according to index-like asset classes. It’s also why we advise against trying to chase or flee current market trends or pick popular stocks, despite what seemingly erudite talking heads seem to forever be recommending.

This is evidence-based investing. This is the essential difference between building a manageable investment strategy that reflects your personalized goals versus succumbing to chaotic, nerve-wracking guess-work in an ever-noisy world of market mayhem. We choose the evidence.

Learn more about Hiley Hunt Wealth Management and who we serve in Omaha, NE – Financial Planning and Investment Management