Podcast: Interview of Perry Kaufman, author of the bible of the trading systems
Perry is also the director of Kaufman Analytics which provides trading strategies and consulting to institutions and individual investors.
There is a lot of meat in this interview so I hope you will appreciate it. Thanks a lot to Perry who managed to stay passionate and humble despite more than 40 years on the markets…
If you do not understand french, you can skip the very first seconds of the podcast where I present Perry Kaufman in french. You can find the transcript below or the french translated version here.
Below is the interview in plain text.
NV (Nicolas Vitale) – Hi Perry, how are you?
Perry Kaufman – Fine, thank you. A pleasure to hear from you.
NV – Thank you a lot for being with us today! I know you have been in the market for many many years, but you started as what we can call a rocket scientist. Could you explain us that a bit?
Perry Kaufman – I found when I make presentations to clients, they are surprisingly interested in my background in aerospace. It was a very interesting time because I worked on the navigation for Gemini which was a two man space vehicle, and the navigation for that was used in Apollo, because you always develop the technology in advance. And you know, I was quite young and very excited, and at that time we were at the state-of-art in mathematics, but now, I think anyone getting out of college knows more than we did. But it was all very exciting! And I worked on this matter for quite until, in 1969, I founded my own computer company.
And after that, someone came to me with a problem in options. It was London options, a long time ago. And it was so interesting that we never stopped. We just switched over what we were doing, which was medical reimbursement. Of course, that would have turned out to be highly profitable. But we switched to options and then into futures and other instruments in the financial markets… and we just kept going! It was a field that you had to fall into. No training was available, and everybody in the field came from different backgrounds and different skills. It was a fascinating time! Perhaps it still is…, but with much more competition then there used to be.
NV – And did you find some similarities between what you were doing in the space sector and financial engineering? That is, in term of mathematics.
Perry Kaufman – Yes, actually, it may seem very simple now, but in the old days, besides the space program, I also worked in missiles and reconnaissance. And you know, we used exponential smoothing to figure out where a missile would land, what would be its trajectory. And a lot of fellows in the aerospace program had time to trade their own stock portfolio. And I used to sit there and apply exponential averages and moving averages to the stock market. And the stock market was so much smoother then, not as noisy as it is now. Many of the engineers made substantial amounts of money with very simple methods. I know even in 1980, which was even a long time for you (ndlr – yes indeed I was not born yet 😉 when I started consulting for an oil company and we were trading a lot of money, the oil market was not used by anyone other than commercials. And you could use a 3-day moving average… and make a lot of money. When prices started going up, they would kept going up and when they went down they would kept going down. The simplest methods trending methods were highly profitable.
NV – Do you think the market today is more difficult than what it used to be?
Perry Kaufman – Much more difficult. With all of the pension programs and all of the different types of investors and competition, the volume is much higher. People have different opinions whether it is going up or down and all these orders placed at different times for different reasons cause a substantial amount of market noise. And that noise makes it difficult to recognize the trend. And so you find that you can not apply trend following in the short term. It is just too noisy. The only trend following that is successful is macro-trend, where you try to take longer-term position that represents a fundamental view of the stock market or interest rate policy. Something that would develop over time. Because in the short term, prices just move up and down too much to be able to really identify a trend with reliability.
NV – I know you are more into strategies with important underlying economic reason, something linked to the market, rather than the new kind of artificial intelligent, computerized methods. Could you just explain why?
Perry Kaufman – Well, you’re not completely correct. My methods are all algorithmic. I only like methods that have what I call a sound premise. I use the computer to explore solutions, but I need to have a premise which I believe is true before I start working on a system.
I like seasonality in agriculture markets and we can develop a program that says “if prices are low in the spring before plantation and high in the summer, then the market is seasonal.” Once you have decided that this year has a seasonal pattern, then you can figure out how to trade it. And I can explore that algorithmically. Also, I like arbitrage and I do a lot of pairs trading. Pairs trading also have a sound premise. It takes markets that have some relationships to one another, and when they move apart, I sell one and buy the other expecting them to move back together. Those are very, very popular methods that can be fully automated. And each one has some reason that you believe is true.
Just taking an indicator like a Relative Strength (cf RSI), and trying to figure out where to buy and sell the market based on overbought or oversold levels, is just not a successful approach. Perhaps it’s just too simple. But mostly, there is no sound premise. Just because prices are overbought at some arbitrary level doesn’t mean they will reverse within a few days. They can continue to go up for a longer period of time. So they are not good on their own as strategies. And finding one that appears to work is generally a formula for failures.
NV – And speaking about computerized methods, do you optimize your strategies?
Perry Kaufman – That is a very difficult question. Everybody optimizes to a certain degree. For example, when I am looking to take profits in a particular system, I use the Average True Range as an indicator of volatility. I know, because I have done that for too long, that targeting an average true range times a factor of 3 or 4 from the trade entry will work. I don’t even need to test it to find out this is the right answer. So when you have enough experience, you always optimize. You also know that long-term moving averages work but short-term moving averages don’t work. You do not even need to bother trying them. So my first guess as to how to specify the parameters of the system are going to be pretty close of what somebody might find when they optimize. And so I can never say that I do not optimize.
Of course I will take my final strategy and run it through some backtests on data. Because if it does not work, or the profit per trade is too small, then I have to look for an alternative. It may be that the strategy is sound but I need more profit per trades to overcome cost of slippage and commissions. As you know, if a strategy is robust then you should have the choice of slowing it down or making more per trades, or speeding it up and making less per trades. I think those are valid choices without calling optimizations.
But it is certainly tuning it to your needs.
NV – Do you use computerized methods like walk–forward analysis, or you do not because as you explained, you base your decisions on experience to prevent over optimization.
Perry Kaufman – When I do a test of a system, let’s say I am interested into finding out if the method I just created is robust. And that is a very important issue. What I do is, in advance, I decide a range of parameters that makes sense for that method. For example, looking at a macro trend system, I am going to test trend speeds of say 40 to 120 days. Or I might test different factors for taking profits. For any optimization to be successful, I want about 70% of all of the tests to be profitable. So I could just close my eyes and pick one combination and expect it to be profitable.
But I do not pick the parameters with the best test result. What I do is I pick a sample of a number of combinations and I trade all of them. What I am really doing is looking for an average performance. If I do an optimization on a thousand combinations, the only result in which I am really interested is how many of these combinations are successful and what is the average return of all of the tests. And that is my target goal, the average return. And so when I choose to trade a number of different parameters, I only expect the average return.
That is going to be disappointing for a lot of people, but I think it is realistic. We really don’t know what we will get in the future. But we can be pretty sure we will get the average of a number of these reasonable tests.
NV – Thank you for all the details. When it comes to finding ideas of new trading systems, do you have a method or starting point? Where do you find inspiration globally? I’m sure every new trader is interested.
Perry Kaufman – An other difficult question… Finding trading ideas is perhaps the most difficult of all things (ndlr Perry’s book would help you definitely…). Because once you have an idea, you can make it a successful system (ndlr… and Alpha Novae can help you for that…). So first you have to be an observer of the market. Then, there are so many things that happen in economics or in seasonality, or things that you see in the newspaper, about why the price of oil goes up or why the dollar has gone down. You need to be able to identify them and say aha!, here is an idea that I can do something with. Or you see that there are many books on pairs trading and you know believe that arbitrage is a sound approach, but people have already done that already. So you have to study it and decide how your ideas may differ from others.
For example, everybody knows you can arbitrage a stock within the pharmaceutical sector or auto industry. But so many others are doing this that the profit per trade is very small and not profitable enough. However, there is a bigger picture. The entire market has become more correlated because of index programs. An index often is a group of stocks within the S&P. Investors will place an order to by or sell the index. The index company goes in and buys all of the stocks in that sector, and that drives the entire group in the same direction. But there are some companies in that group don’t deserve to be driven up along of the others, and so there is an opportunity to sell that company and buy the sector or a different stock in that sector. In a broader sense, you can now arbitrage nearly any two items in the S&P because index markets drive them back together and then apart and then back together. There are plenty of opportunities.
NV – Overall are you more into intraday or swing, mid-term strategies?
Perry Kaufman – I prefer daily strategies because they have larger profits per trade. There is a lot of competition everywhere but I think it is easier to deal with daily data. And I suspect for most investors, even many professional traders, daily is much easier to deal with. With intraday trading you need to have much more data. You need to be able to place orders during the day and that is inconvenient for many people; and, profits per trade are much smaller. So intraday trading is subject to slippage and other issues that become a real problem. I believe there are plenty of opportunities in daily. In fact, I do intraday arbitrage, but I don’t do it the way high frequency does it, which is in microseconds. I do it in hours.
NV – Speaking about high frequency trading, everyone is talking about it, even at the political level. Do you have an opinion about that? Should it be banned or regulated more? (ndlr you can see an here an extract of our conference on High Frequency Trading at the Salon du Trading, and more extracts from the Salon du Trading here).
Perry Kaufman – If high frequency trading makes money, it does extract profits from the market that somebody else could possibly get. I don’t really care who makes money as long as it is done honestly. It’s fine to find a method to arbitrage in micro seconds. They will create their own competition. Up to now, the high-frequency people have made an arrangement with the exchanges to place their computers as close as possible to the transmission source in order to beat competition by a fraction of a millisecond. The exchanges are now rejecting that and say no, you cannot be closer than, say 100 feet, and cannot share the same space as the exchange. As with all of these great methods of trading, competition will eventually minimize the profits and many companies will stop. I think the market will take care of itself.
NV – To change the subject a bit, I know, having read some of your books, than you emphasize risk and money management. Is it at the core of your systems? And do you build specific money management modules for each strategy? Can you detail a bit how you handle money management?
Perry Kaufman – Risk management became even more important in the new book that is just coming out within the next weeks or two. I have really gone through it and made sure that risk is explained almost everywhere throughout the book. It is mostly because we were horribly surprised by the subprime crisis. I am one of those who did not really expect the magnitude of the risk in 2008. I kept saying that it gone far enough, it will stop and recover and things will be back to normal. We did not expect anything this big and this long. And that requires you rethink all of your concepts of risk. You don’t want to be wiped out because you managed the risk incorrectly.
So I have spent at least five years focusing on risk and decided that risk is as important as the underlying system. When the market becomes more volatile, we must reduce our position size. But even more important, when the market becomes less volatile, you need to increase your position size because there are long periods of low volatility which make nice money. If you do not have a big enough position during periods of low volatility, you can’t get the return that you want. So for both high and low volatility, you want to do what we call “volatility stabilization.” It is a concept which is familiar to most professionals, where you keep adjusting your size, always conscious of the cost of doing this. Because whenever you switch position size there is a cost of what we call “rebalancing.” Then you have to be careful how often you rebalance, but you still must rebalance.
Investors that put money into a program deserve risk control. Without it, there is a much higher chance being wiped out. Diversification is also important, but almost everyone already understands that. Traders should also look at Value at Risk. Value at Risk is a formula that tells you the probability of a loss greater than some value, given the positions you are holding. It’s not perfect, but it is a help controlling risk. I know the formula is available in many books, and it may also be in Excel. If you are smart, you will reduce you position when VaR shows extreme risk.
Besides portfolio risk, you have risk on individual trades. Some people deal with that with stop losses but I prefer taking profit and getting out of the market whenever possible. Taking profits has the added advantage of keeping you out of the market. For example, if you are not in the market one third of the time, then you have avoided unexpected price shocks for one third of the time, and your overall risk will go down. Then one of the rules of choosing the best trading method should be that you are out of the market as much as possible. By doing that you avoid unexpected an unpleasant situation.
(ndlr – By not being in the market, you also get extra liquidity for other strategies that would need being in the market at this time. So you get a better cash flexibility for optimal allocation.)
NV – Thank you for everything. To speak more technically now, do you use specific platforms or tools to automate your strategy or to backtest?
Perry Kaufman – Well, I do, mostly a platform that I developed myself, and so I have 40 years now of developing that.
One of the problems with the platforms that you buy, and some of them are quite nice and very convenient, is that they can be slow, especially if you’re working with a lot of intraday data. On the other hand, I often use TradeStation to get a fast look at whether a new idea seems reasonable. I learned to use TradeStation because it was the first platform you could really program and it’s very convenient. So that is why there are many example of TradeStation in my book (ndlr we recommend MultiCharts too as an equivalent multi broker and actively developed). We have also started using MetaStock because Thomson Reuters has done a great job upgrading it. Of course, there are many other platforms that will work and a far less expensive.
NV – Yes, there are hundreds and hundreds platforms of course. (ndlr, most used platforms from our forum)
Perry Kaufman – Yes of course, but I am not in the business of building platforms. The greatest advantage of any platform is to see if a new idea make any sense at all, I would use TradeStation or MetaStock to see where the signals occur on the screen and make sure they do not look stupid.
NV – Do you try to optimize your execution quality when you are trading? Or is it not really the most important point because you do not really trade a lot in small timeframes?
Perry Kaufman – That is correct, but I found that when using trend following, I prefer not entering on a trend signal because the markets tend to be pushed and they are many orders clustering at the trend change area. I found that my best approach is to wait until the market reverses after the signal. That turns out to be a good technique because sometimes the trend reverses again and you never have to actually change your position. The other method would be averaging in over whatever the timeframe you’re using.
For example, if you are a macro trend trader and you’re going to hold the trade for 3 to 6 months, you can take a week to get in. And if you are a shorter-term trader, you can take an hour or two to get in. When you average in, you find that your results are much more stable than if you just take the original trading signal.
NV – I know you trade in stocks, futures, and commodities. Are you also trading in Foreign Exchange (forex)?
Perry Kaufman – Yes, I have a major program in FX Carry. But again, FX Carry is a sound premise. It is not the concept that there is money in this arbitrage, but that investors put money into countries with higher interest rates net of inflation. And so it is the movement of money that makes FX Carry successful. But it is also a complicated program because it is all about controlling risk. You need to avoid begin caught in the Japanese intervention which turns your entire profits over the year into a loss. So that is my focus in FX, although FX is part of all the other trend programs and everything else I do. But specifically I like the Carry program.
NV – You just spoke before about your book and a lot of people especially know the “New Trading Systems and Methods“. We refer quite often to it as the bible of the trading systems (ndlr more than 1200 pages!). You were telling us we would have a new edition soon?
Perry Kaufman – A new edition within weeks! It should be released soon (ndlr begining february according to Amazon). It is a good upgrade. All the examples and charts have been upgraded to current examples. There is a lot more on arbitrage, more trading systems, and a lot more on risk throughout the book. It is also a better organized. I have started to give more analysis on whether or not particular methods are good for certain users. In the past I treated it more like an encyclopedia. You could pick what you wanted and see how it worked. Now I am trying to give you a better understanding on when to use it, when it does not work, and when other methods which are simpler may better work. So I hope the users will like those changes.
NV – Yes I think they will. Personally, I became interested in automated trading strategies some years ago partly thanks to your book, so I will be happy to see the new version. And do you have other projects for the future?
Perry Kaufman – Yes, I am working on some retails programs for both UBS and Thomson Reuters and I have a program running at RBS. So I am trying to stay what we call “relevant,” staying in the mainstream with these companies. The program for UBS will be a relative value arbitrage similar to pair trading, while the one with Thomson Reuters will be both proprietary trading strategies and a toolbox of strategies. So I hope investors will find them interesting.
NV – Thank you very much for all these details and for the time you have spent with us today. I know that you are a very busy man so thanks a lot!