Featured Research

from universities, journals, and other organizations

Should you trust your financial advisor? Pseudo-mathematics and financial charlatanism

Date:
April 10, 2014
Source:
American Mathematical Society
Summary:
Your financial advisor calls you up to suggest a new investment scheme. Drawing on 20 years of data, he has set his computer to work on this question: If you had invested according to this scheme in the past, which portfolio would have been the best? His computer assembled thousands of such simulated portfolios and calculated for each one an industry-standard measure of return on risk. Out of this gargantuan calculation, your advisor has chosen the optimal portfolio. After briefly reminding you of the oft-repeated slogan that "past performance is not an indicator of future results," the advisor enthusiastically recommends the portfolio, noting that it is based on sound mathematical methods. Should you invest?

Your financial advisor calls you up to suggest a new investment scheme. Drawing on 20 years of data, he has set his computer to work on this question: If you had invested according to this scheme in the past, which portfolio would have been the best? His computer assembled thousands of such simulated portfolios and calculated for each one an industry-standard measure of return on risk. Out of this gargantuan calculation, your advisor has chosen the optimal portfolio. After briefly reminding you of the oft-repeated slogan that "past performance is not an indicator of future results," the advisor enthusiastically recommends the portfolio, noting that it is based on sound mathematical methods. Should you invest?

The somewhat suprising answer is, probably not. Examining a huge number of sample past portfolios -- known as "backtesting" -- might seem like a good way to zero in on the best future portfolio. But if the number of portfolios in the backtest is so large as to be out of balance with the number of years of data in the backtest, the portfolios that look best are actually just those that target extremes in the dataset. When an investment strategy "overfits" a backtest in this way, the strategy is not capitalizing on any general financial structure but is simply highlighting vagaries in the data.

The perils of backtest overfitting are dissected in the article "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance," which will appear in the May 2014 issue of the Notices of the American Mathematical Society. The authors are David H. Bailey, Jonathan M. Borwein, Marcos Lopez de Prado, and Qiji Jim Zhu.

"Recent computational advances allow investment managers to methodically search through thousands or even millions of potential options for a profitable investment strategy," the authors write. "In many instances, that search involves a pseudo-mathematical argument which is spuriously validated through a backtest."

Unfortunately, the overfitting of backtests is commonplace not only in the offerings of financial advisors but also in research papers in mathematical finance. One way to lessen the problems of backtest overfitting is to test how well the investment strategy performs on data outside of the original dataset on which the strategy is based; this is called "out-of-sample" testing. However, few investment companies and researchers do out-of-sample testing.

The design of an investment strategy usually starts with identifying a pattern that one believes will help to predict the future value of a financial variable. The next step is to construct a mathematical model of how that variable could change over time. The number of ways of configuring the model is enormous, and the aim is to identify the model configuration that maximizes the performance of the investment strategy. To do this, practitioners often backtest the model using historical data on the financial variable in question. They also rely on measures such as the "Sharpe ratio," which evaluates the performance of a strategy on the basis of a sample of past returns.

But if a large number of backtests are performed, one can end up zeroing in on a model configuration that has a misleadingly good Sharpe ratio. As an example, the authors note that, for a model based on 5 years of data, one can be misled by looking at even as few as 45 sample configurations. Within that set of 45 configurations, at least one of them is guaranteed to stand out with a good Sharpe ratio for the 5-year dataset but will have a dismal Sharpe ratio for out-of-sample data.

The authors note that, when a backtest does not report the number of configurations that were computed in order to identify the selected configuration, it is impossible to assess the risk of overfitting the backtest. And yet, the number of model configurations used in a backtest is very often not revealed -- neither in academic papers on finance, nor by companies selling financial products. "[W]e suspect that a large proportion of backtests published in academic journals may be misleading," the authors write. "The situation is not likely to be better among practitioners. In our experience, overfitting is pathological within the financial industry." Later in the article they state: "We strongly suspect that such backtest overfitting is a large part of the reason why so many algorithmic or systematic hedge funds do not live up to the elevated expectations generated by their managers."

Probably many fund managers unwittingly engage in backtest overfitting without understanding what they are doing, and their lack of knowledge leads them to overstate the promise of their offerings. Whether this is fraudulent is not so clear. What is clear is that mathematical scientists can do much to expose these problematic practices -- and this is why the authors wrote their article. "[M]athematicians in the twenty-first century have remained disappointingly silent with regard to those in the investment community who, knowingly or not, misuse mathematical techniques such as probability theory, statistics, and stochastic calculus," they write. "Our silence is consent, making us accomplices in these abuses."


Story Source:

The above story is based on materials provided by American Mathematical Society. Note: Materials may be edited for content and length.


Journal Reference:

  1. David H. Bailey, Jonathan M. Borwein, Marcos López de Prado, Qiji Jim Zhu. Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance. Notices of the American Mathematical Society, 2015; 61 (5): 458 DOI: 10.1090/noti1105

Cite This Page:

American Mathematical Society. "Should you trust your financial advisor? Pseudo-mathematics and financial charlatanism." ScienceDaily. ScienceDaily, 10 April 2014. <www.sciencedaily.com/releases/2014/04/140410103005.htm>.
American Mathematical Society. (2014, April 10). Should you trust your financial advisor? Pseudo-mathematics and financial charlatanism. ScienceDaily. Retrieved July 22, 2014 from www.sciencedaily.com/releases/2014/04/140410103005.htm
American Mathematical Society. "Should you trust your financial advisor? Pseudo-mathematics and financial charlatanism." ScienceDaily. www.sciencedaily.com/releases/2014/04/140410103005.htm (accessed July 22, 2014).

Share This




More Science & Society News

Tuesday, July 22, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Courts Conflicted Over Healthcare Law

Courts Conflicted Over Healthcare Law

AP (July 22, 2014) — Two federal appeals courts issued conflicting rulings Tuesday on the legality of the federally-run healthcare exchange that operates in 36 states. (July 22) Video provided by AP
Powered by NewsLook.com
What Self-Made Women Need to Know Financially Before Getting Hitched

What Self-Made Women Need to Know Financially Before Getting Hitched

TheStreet (July 21, 2014) — Halle Berry was recently ordered to pay her ex-boyfriend Gabriel Aubry $16,000 a month in child support by a California judge for their daughter Nahla. As women make strides in the workforce, they are increasingly left holding the bag when relationships end regardless of marital status. 'What Monied Women Need to Know Before Getting Married or Cohabitating' discusses information such as debt incurred during the marriage is both spouse's responsibility at divorce, whether after ten years of marriage spouses are entitled to half of everything and why property acquired within the marriage is fair game without a pre-nup. Video provided by TheStreet
Powered by NewsLook.com
$23.6 Billion Awarded To Widow In Smoking Lawsuit

$23.6 Billion Awarded To Widow In Smoking Lawsuit

Newsy (July 20, 2014) — Cynthia Robinson claims R.J. Reynolds Tobacco Company hid the health and addiction risks of its products, leading to the death of her husband in 1996. Video provided by Newsy
Powered by NewsLook.com
Clock Ticks Down on Internet Speed Debate

Clock Ticks Down on Internet Speed Debate

Reuters - US Online Video (July 18, 2014) — The FCC received more than 800,000 comments on whether and how internet speeds should be regulated, even crashing its system. Lily Jamali reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins