Testing strategies

 

You can use the free screener in many sites including your broker’s web site and Finviz.com. However, most do not provide a historical database for back testing. Some include historical data for technical indicators such as SMA (Simple Moving Average) and/or P/E.

 

Here are some services that have historical data for back testing: Vector Vest, Zacks and Portfolio 123 at low prices. This article outlines some of the hints for evaluating their services.  You should list your requirements. They are expensive and time-consuming to learn. However, they should give you valuable tools if they are properly used.

                                                                                                     

·         Price. Most would charge $500 and up for a yearly subscription. If you cannot afford it, use Finviz.com which is free. It does not have a historical database for fundamental data and you need to save the test results.

·         Most sites do not compensate for survivorship bias by taking out the unlisted companies. There are more bankrupt companies than merged or acquired companies hence making your strategy looking better than it is actually is. If you take out Enron and/or Lehman Brothers, your test would look better.

 

Since more small companies bankrupt than large companies, try to include companies with a market cap greater than 500 million (100 million for strategies for small companies) to reduce this bias.

·         I prefer the historical database which starts from year 2000 so we have two full market cycles from 2000 to 2019.

·         I prefer dividing the test periods into phases of the market cycle. Some screens perform better in the down market than the up market for example.

·         Most have a comparison to the S&P500 index or SPY for simplicity. In the last few years (as of 2019) SPY has been doing great because SPY is weighed by market cap with a lot of high-flyers. If these high-flyers fall, this index and SPY would fall accordingly. Use related ETF for comparison. For example, if you are testing a strategy on dividend stocks, your yardstick should be one related to dividend stocks and most likely you need to add dividends to your results and the ETF. Personally I would buy SPY for the money left over, and hence SPY is a logical choice for me.  Do not use DOW index: 1. It has only 30 stocks, and 2. It is price weighed that is not practical.

·         I prefer to enter the filter parameters via ‘click-and-select’.

·         It is easier to understand annualized return; a 1% return per month is equivalent to 12% in a year especially for short-term trades. However, if the period is less than 15 days, the annualized return would be amplified too much.

·         Most do not include dividends and it is fine when you do not include the same in your benchmark such as the SPY. For screens specialized on dividends, add the extra dividends to the performance.

·         Many vendors group several parameters such as P/E and Debt/Equity into a value score. The other popular score is momentum score with the simple moving average for example. The combining of these two scores is a summary score. There are many other scores such as a safety score using beta.

·         Determine that whether the search is for value stocks or momentum stocks. You need to hold value stocks longer (one year for me) and momentum stocks shorter (1 month for me). Then do the same for testing.

 

To illustrate, SMA-20, Sales Q-to-Q and Earnings Q-to-Q are momentum parameters while P/E, Debt/Equity and P/B are value parameters. If you test a value strategy, the holding period should be more than 6 months.

·         Some screens find more volatile stocks than others. It is measured by the maximum drawback, which is defined as the loss from the recent peak. For stocks from volatile screens, use a higher stop loss otherwise they would be stopped out during stock price fluctuations.

·         Read the evaluations of the service you are interested in by googling it. It could save you a lot of money by learning from others’ experiences.

·         A screen has many criteria such as P/E. My basic parameters are:  in one of the 3 major exchanges, U.S. stocks, Market Cap within a specified range, Earnings Yield > 5% and Stock Price > 1. Vary it for your requirements for the specific screen.

·         Sorting. You may want to sort Earnings Yield (E/P) in descending order.

·         Testing. You may want to test the top 5 stocks according to your sort specified.

·         Testing period. For value screens, you may want to hold the stocks for 6 months or a year.

 

The following uses the Year-End screen as an example. You have the following start date: 11/1, 11/15, 12/1, 12/15, 1/15 and 1/30 and hold the screened stocks for 1, 2, 4 and 6 months.

 

The number of simulations from 2007 to 2017: 10 years * 6 start dates * 4 (1, 2, 4 and 6 months holding) = 240 simulations. For each sort variation, you double the number of simulations to 480.

 

Adding every variation to the screen such as Market Cap > 1000 would again double the number of simulations to 960.

 

It is a time-consuming process. Some sites may have strategies simulating what gurus such as Buffett would buy.

·         In reality, after you have found a handful of stocks, be sure to start further research using Debt/Equity, Q-Q sales / earnings, short % and Insider Transaction from Finviz.com; also use Fidelity.com’s Equity Summary Score and average 5-year P/E and Yahoo!Finance.com’s EV/EBITDA.

 

·         If you have found too many stocks, restrict your criteria and vice versa for too few stocks.

·         Ensure the calculations are correct. When you compare the returns to SPY, the negative values could give you wrong interpretations.

·         Define your tests according to the phases of the market cycle. Market Peak (a phase defined by me) should have different strategies rather than Early Recovery.

·         The last 5 years is better than the last 10 years as it is more similar to the current market.

·         A spreadsheet is the tool to summarize all your test results.

·         There has been no evergreen strategy, which is defined as a strategy that works well in all market conditions. High Insider Purchases is close to one.  Recently, the following was the actual averages for my six strategies. I interpret that the period “2007 to 2014” does not favor stock pickers.

 

	Beat SPY by
From 2000 – 2006	490%
From 2007 – 2014	-96%
From 2000 – 2014	180%

 

More on strategies

 

A strategy tells you how to find a stock and when to sell a stock. The most important task is how to screen potentially profitable stocks to limit your selection and then evaluate each screened stock.

 

Today, there are many pre-defined screens available and we can define our own screens. Most of the financial ratios such as P/E have been defined for us to reduce the need of the time-consuming task of finding them from the financial statements. There are certain items in the financial statements that we should be concerned with after the initial evaluation.

 

To me, the most important advance for retail investors is the availability of historical data bases at prices we can afford. It shows us the performance of our strategies for the last 10 or so years in an hour or two. The following are my suggestions to make a better use of the findings from these databases and their pitfalls.

 

·         Start with proven metrics as demonstrated by research (the metrics used in this book and other sources such as from SSRN.com) and your own trading.

 

·         Select parameters as close to your trading style as possible. For example, if you only deal with small stocks, select market caps from 100 M to 500 M.

 

·         Some screens perform better in the short term while some perform better in the long term. I recommend testing the performances for 3 months, 6 months, 12 months and 18 months for short-term screens (1 month for momentum stocks). If you believe the strategy is for a long term only, get the performances for 6, 12 and 18 months.

 

·         In 2015 (use a date more current), I start with the year 2000 and end with 2014  for a total of 15 sets. It covers the two market cycles. The market before 2000 may not be relevant as today’s market is quite different from that time. Actually I prefer the tests for the last 10 years.

·         Run the screen in the first part of the year and test the performance at year end. Repeat the test for the rest of the intervals: 3, 6, 12 and 18 (adding 1 for momentum and 2 years for long term).

 

·         I do not start with an amount (say $10,000) and see the results at the end of a period (say 10 years). It could be misleading when your screen performs exceptionally well (or bad) in the first few years. I call the one I use “window testing” for lack of a better term. To illustrate this, the start date is Jan. 1, 2000 (actually Jan. 2 due to holiday) and end date is Jan. 1, 2001. The next set is Feb. 1, 2000 to the start, the next set is Feb. 1, 2001 and so on.

 

·         I use the next available date if the date falls on a weekend or a holiday for consistency. If you have more time, try another set starting in mid year. For some strategies and when time allows, I test the strategy monthly.

·         If my screen makes 10% and the market makes 20%, it is 100% worse off. I compare how much it beats the market.

 

·         For simplicity I use the SPY to simulate the market. No index is ideal. The S&P500 is capitalization cap weighed. In 2015, the bubble stocks dominated this index. The better selection is to choose the index according to the type of stocks that you usually have such as the Russell 2000 for small stocks.

 

·         Consider safety such as the Sharpe ratio, maximum drawdown (peak-to-trough loss) and a winner percent. A winner percent at 55% (target for 52%) is very good. If your strategy has a very high winner percent, most likely it is safe but just not only performing.

 

·         Why your actual performance with real money may be worse than the tested strategy:

·         Survivor bias. Most databases take out the stock when they’re bankrupt, merged or acquired. Hence the performance of the screen is better than it really is as there are more bankrupt companies than merged /acquired. From my experience I have at least one such stock in a year. Hence, the test results are not correct if the database does not take care of this bias.

·         Emotions do not allow you to stick with the strategy.

·         The strategy does not work in the current market; that’s why we need to test the performance of the last 6 months.

·         There is idle money between trades.

 

·         I use the top 5 stocks (hence sorting is important). In addition, if you do not buy foreign stocks, deselect them. I use the top 2 for some strategies such as sector rotation.

·         If you cannot find data in some screens in any month, just leave it blank (actually null), which will not be included in calculating averages / totals. If the market is risky, you may not find any value stocks.

·         The most recent tests would resemble the performance of the strategy better. Hence, I have an extra average of performance on more recent tests.

·         Return = (B-A)/A, while A is the return of SPY for me. It does not handle the negative numbers. Use (B-A)/ABS(A). In any case, check the results to see they make sense. When B or A is zero or very small number, the results could be misleading. I usually delete the test results on huge returns in both directions.

·         Using sector rotation strategy as an example, the holding periods are 1, 2 and 3 months.

Some strategies work better on different phases of the market cycle such as growth stocks in the market trending up and staple stocks during market plunges.