If you’ve studied finance in school, or read books by finance professors, you’ve probably learned that technical analysis “doesn’t work.” In one important sense, that is correct, because when it does work (and it does in the right hands), it tends to produce negative tax consequences (see the end of this post for a solution). But it misses the significant benefits that technical analysis does provide.

Most of the major brokerage firms that cater to retail traders pay attention to, and/or provide resources for, technical analysis.

In this post, we look at what technical analysis is, whether it adds value, and whether it is a useful tool for the retail investor.

Our answers might surprise you.

What is Technical Analysis?

For most of the 20^th century, technical analysis meant “chart reading.” There are many “chart patterns” in the technical analysis literature. One basic pattern is the uptrend. For example, consider this chart:

Suppose you had been charting this stock, and noticed that it followed this nice, linear trend. At point B, the technical analysis would suggest a buy point.

Perhaps the trend is violated at around day 105. Maybe you sold, or maybe you held. It would depend on your particular chart reading skills. Either way, this technical-analysis-based trade would have been profitable.

Let’s look at another common pattern, the “head and shoulders.”

In this pattern, when the prices form a “head and shoulders” pattern (A is the left shoulder, the head is in the middle, and the right shoulder is on the right) and the stock price then moves above the “shoulder line” at point B, a buy signal is given.

This type of technical analysis is still quite common. Traditional technical analysis is also called “charting” because it relies on identifying visual patterns in charts. When I was young and starting to learn about the stock markets and the futures markets, I made my own charts, and then, for a while, I subscribed to one of the commercially available chart services. They’d send me charts, in the mail, about once a week (as I recall), and I’d then manually update them, looking for these patterns.

After a while, I decided that I didn’t have the knack, or whatever it is, to read the charts correctly. In this post, I describe an alternative hypothesis to explain why I (and I believe many others) cannot properly read the charts.

New Technology

Since the advent of the personal computer, and especially since the internet, traditional technical analysis techniques have been augmented by the availability of both low-cost computing power, and widespread availability of stock (and other traded markets) price data.

When I was in graduate school at Stanford, I wrote my Master’s thesis on the price behavior of the US Treasury bond futures contract. I needed to get the data. In those days, there was no internet. Just obtaining the data was a major undertaking. I had to first find a source for the data, which I was able to do with the help of my professor, Anne Peck.

I then had to visit the vendor, and negotiate with them to provide the data. The data was then, and is now, a valuable commodity. With the help of my professor, Anne Peck, I was able to work out a deal, and the vendor agreed to provide me with the data. The vendor provided it on old- fashioned computer tapes. Here’s a picture of that kind of tape mounted in an IBM machine.

When I got the data back to campus, I discovered that the data on the tapes was in a format called EBCDIC (extended binary coded-decimal interchange code) which was the standard then in use by IBM. However, Stanford’s computers were DEC (Digital Equipment Corporation) machines and used data in ASCII (American Standard Code for Information Interchange).

It was quite a bit of work (and gave me a great sense of accomplishment) to convert the data from EBCD to ASCII. Only then could I begin the analysis. I don’t think that in those days my experience was too out of the ordinary.

Today, all you need is a desktop computer, an internet connection, and a few bucks to subscribe to a service that will give you access to more data than you know what to do with.

Data Mining

The easy availability of all that data has given rise, whether all traders realize it or not, to many traders engaging in what is pejoratively called “data mining.”

Data mining is the process of searching through data sets not with a particular hypothesis in mind, but rather with the goal of finding patterns that repeat in the data.

If you are skilled with data analysis, you can hunt through past price data for patterns that are much harder to see than the visual examples above.

In fact, the patterns can be so hard for a human to spot that even the very smart people who develop the software to find the patterns don’t always understand what the computer is really doing.

Does Technical Analysis Work?

To answer this question, we must be clearer about what we mean. For example, an article on Investopedia[1] concludes “for those who diligently practice the concepts, it does provide a realistic possibility of trading success.”

Perhaps the author had in mind people like Jim Simons, the multi-billionaire mathematician and founder what might be the most successful hedge fund ever, Renaissance Technologies.

But for the rest of us, let’s interpret “trading success” as “trading profits.” We are close to a testable hypothesis. Let us formalize that hypothesis as “Technical analysis tools can be used by retail traders to generate consistent trading returns, net of transactions costs, that exceed the market returns on a risk-adjusted basis.”

In other words, the hypothesis is that a retail investor using technical analysis can over time “beat the market” without taking greater-than-market risk.

There is a great deal of finance literature that attempts to answer this question. And the answers, perhaps not surprisingly, are mixed. There are a number of academic studies that have found this or that signal that worked in the period studied. However, for the retail investor seeking to personally profit, for the most part, the answer is “no,” unless we count the so-called “momentum” factor as a technical analysis tool. And even then, it’s challenging for a retail investor to exploit successfully. Two big challenges are competition, and false signals. Let’s look at why that’s the case.

Competition

For a technical analysis tool to “work” it must give signals that allow an investor to buy stocks that will outperform, and/or sell those that will underperform. Let us assume that there are tools that allow investors to outperform. Even then, if investors are using tools that are available to the general public, e.g. from Schwab, or Fidelity, or many other sources of tools such as “relative strength index,” “channel index”, “stochastics”, “Bollinger Bands” and similar indicators, then only a small number of early actors will be able to outperform, because the very action of buying will drive up the price and bid away the potential profit. Similarly, early actors on the sell signal will drive prices down, again driving away the potential profit.

This difficulty of profiting from widely known public information is a limit on technical analysis working even if the signals are in fact valid.

Overfitting

One downside of lots of data and lots of computing power is that it becomes possible, even likely, to find patterns in data even when that data is random.

Relatively few retail traders have formal training in statistics, econometrics, data analysis, or a similar field.

If they had such training, they might be more wary of the dangers of “curve fitting” or data-mining.

Curve fitting, roughly, is the mathematical fact that, given enough variables, it is possible to fit a curve (such as a polynomial) to any set of real-number pairs.[2]

Here’s an example. Everyone knows that if you have two points, (e.g. two closing prices), you can perfectly fit a line. If the closing price on Monday was 10, and on Tuesday it was 11, a line that perfectly fit those two points would “predict” that on Wednesday the closing price would be 12.

Every trader realizes that such a prediction would be worth very little.

But not every consumer of technical analysis fully appreciates that many more sophisticated statistical analyses can end up making predictions that are just more complicated versions of that kind of prediction.

In fact, the chart patterns shown at the beginning of this post are not actually stocks at all. They are series that I generated using Excel’s random number generator. I then found two that illustrate the technical patterns shown.

But there is empirical reason to doubt that most technical signals are valid. There is a great deal of research showing that common forms of technical analysis do not in fact yield signals that persist outside of the data set used to create the model. (In machine learning terminology, the model doesn’t work on data outside the training set.)

Technical Trading is Rational?

Let us suppose that for the retail trader (non-professional), the expected return on technical trading is less than the expected return on the market. For the sake of this discussion, let’s assume that the technical analysis signals are random. If we make a few more assumptions (mainly about market efficiency, but also some technical statistical assumptions), we can estimate that the trader’s expected returns will be the market return, less the transactions costs of his[3] trading.

Transactions Costs

Now, we need to estimate those trading costs. In today’s world, the commission per trade can be very low, close to zero. But even if a trader’s commission cost is zero, he will, on average, incur the bid-offer spread with every trade. The size of that bid-offer spread varies a great deal depending on the asset being traded. For a large, actively traded stock the spread might be only a cent or two per share. For smaller, less actively traded stocks it can be ten times that amount.

Let’s say that on average the bid-offer spread costs the trader one cent, on an average share price of $50. Suppose the trader is a day trader, and makes 252 trades a year, each one that costs $.01 per $50 of shares traded. Over the whole year, he’ll pay $.01 252 times, or a cost of $2.52 per $50. That works out to 5.04%.

However, suppose he only averages one trade per week. Now that cost is $.52 per year on $50, or just over 1%.

Whatever the transaction costs, the trader (given the assumptions that there is no information, positive or negative in the signals) will on average underperform the market.

So, how can that be rational?

Rationality Found

Most traders, and most of your clients, are not stupid. Most traders, if they do not beat the market, will eventually figure out that they are not beating the market. What will they do then?

That depends. Many keep trading. When competent people are informed and do things that appear irrational, we should stop and ask ourselves whether it is we who are missing something.

In the case of active traders, we might very well find that we are missing the fact that traders derive enjoyment from the activity of trading.

Is it Entertainment?

When someone spends $200 and a day to go to a sporting event, or a concert, we do not generally assume that the activity is irrational, despite the fact that it clearly costs the person both money and time. We understand that those are consumption activities.

We do not complain that the person is irrational because he could get almost the same experience, for a fraction of the cost in time and money, by watching a game, or concert, on TV. If we look just at the financial aspect of the activity, we must conclude that the action is irrational because the expected financial outcome is negative.

Why, then, should we assume that a trader is irrational because he persists in an activity – trading – that (we believe) has a negative expected cost? 

Probably it is the case that most retail traders derive significant experiential enjoyment from their trading, or they would desist.

“Play Money” Portfolio

Many successful people who have most of their money invested “rationally” also keep a relatively small amount aside for this kind of entertainment value. Few explicitly call it entertainment, though many would admit that’s at least partially what it is. And perhaps most believe, in some way, that they can in fact “beat the market.”

Taxes

Unfortunately, success in investing draws the attention of an unwanted partner – the tax man. In my 35+ years in business, I have never spoken to anyone who thought he wasn’t paying enough taxes. (I have spoken to many people who thought other people weren’t paying enough, but that’s obviously completely different.)

Our hypothetical trader, because he trades so often, generates taxes at the highest short-term capital gain rates on his gains. And that could be considerable. Suppose that the average return on the market is 10% (about where it’s been over the very long term in the US). And suppose our trader incurs total costs of 1%, meaning his pretax return is 9%.

If his net gains are all short term (likely if he’s actively trading), those short-term gains are going to be taxed at the same rates as ordinary income. At this writing, the federal tax on short term gains goes up to 40.8%, and if the trader lives in a high tax state (such as Massachusetts, California, New York, Maryland, etc.) the total tax rate on gains can be around 50%.

Active Trading Trust

You may not have known about a powerful solution to the short-term capital gains problem: an Active Trading Trust. An Active Trading Trust can protect you or your clients from paying short-term capital gains tax when trading stocks. Please call us to see whether an Active Trading Trust might be a good solution. You can also sign up here for a copy of our forthcoming book, The Investor’s Dilemma Decoded. can also call 703 437 9720 and ask for Connor or Katherine, or email us at [email protected].

---

[1] https://www.investopedia.com/articles/active-trading/062215/debunking-8-myths-about-technical-analysis.asp

[2] Pairs of numbers for 2-space, and generally “n-tuples” of numbers in n-dimensional space.

[3] The great majority of such traders are male.