Can a Chatbot Pick Stocks?
Ask five chatbots which stocks to buy and you’ll get five answers. All of them the same. A line-up of the usual suspects. I’ve done it. I’m sure you have done it too.
But – the research!
I’m sure you have seen all the adverts. They claim to be backed by research. Over the past couple of years there’s been a flood of research papers showing AI trading systems beating the market. Impressive charts. Big percentages. The usual.
But does AI stock-picking really work? Researchers (details below, we’ll call it the Edinburgh study) designed a proper experiment and found the answer: no, not yet.
Let me explain.
Yes, Minister
Remember the old UK satire? Sir Humphrey’s advice:
‘. . . never set up an inquiry unless you know in advance what its findings will be.’
Well, that could be applied to many of these studies.
When the Edinburgh team looked closely, they found most of these studies tested their AI over a few months, on a handful of stocks.
One popular system was tested over six months on five stocks, mostly household names like Tesla and Amazon. Now, if you pick five of the most successful companies of the last decade and test over a convenient six months, of course the results will be stellar.
So instead, the Edinburgh study tested it properly. No Sir Humphrey tricks.
20 years, 100+ stocks, no cherry-picking
They rebuilt the backtests over 20 years (2004 to 2024), across more than 100 stocks chosen without hindsight – that’s the important bit. They included companies that later went bust or dropped out of the S&P 500. (Quietly leaving the dead ones out is a classic way to make any strategy look good – it even has a name: survivorship bias). And they compared the AI against good old-fashioned strategies, including plain buy-and-hold.
The surprising results:
OK, they weren’t that surprising:
- The AI’s advantage evaporated. Over the longer period and the wider set of stocks, the market-beating results mostly disappeared.
- Buy-and-hold beat the AI. Under the fairest tests, simply buying and holding outperformed both AI systems, and the difference was statistically significant, not a fluke.
- Short tests were wildly unstable. Extend one system’s original test by just two months, and its Microsoft result flipped from a gain of about 23% to a loss of about 22%. Two months!
- Bigger brains didn’t help. Larger, fancier AI models didn’t reliably beat smaller ones, and a simple statistical forecasting method dating back to the 1970s often beat them both.
- No skill. Once the luck was stripped out, neither AI system showed any genuine ability to beat the market.
Oh, and the researchers estimated around US$700 in AI fees for the broader comparison tests alone.
Bulls & Bears
This is the interesting part: when the researchers split the results into bull markets and bear markets, they found the AI was getting it exactly backwards. It was too timid in bull markets, sitting out the gains, and too aggressive in bear markets, piling on the losses. Sound familiar?
In the 2008 crash, one AI system’s loss on a single stock (Deere) approached 75%, compared with around 50% for the S&P 500. It didn’t protect anyone. It made things worse.
How to fix it?
So what was their recommendation for fixing AI trading? Not more computing power. Not bigger models. They said future AI strategies need two things:
- Better trend detection, so they at least keep up with the market when it’s rising.
- Risk controls that recognise what kind of market they’re in, so they pull back when conditions turn.
In plain English: work out whether you’re in a bull market or a bear market, and act accordingly.
Well, duh!!
AI is emotional?
I bet you’ve all figured out where I’m going with this. The AI isn’t failing because it isn’t clever enough. It’s failing because it’s watching the waves (every headline, every earnings report, every daily wobble) and trying to react to all of them. Which is exactly what humans do when they trade emotionally. AI learned from us, after all, and it seems to have picked up our worst habits. An emoting robot!
AI can't see the tide.
That’s what The ITM Plan is built on. It doesn’t read the news. It doesn’t try to predict next month. It asks one boring question: is the tide coming in or going out? Bull-market strategy when it’s coming in, bear-market strategy when it’s going out, and sit on your hands the rest of the time.
Test it yourself
The researchers’ other big message was about backtesting honestly: long periods, lots of data, no hindsight, no cherry-picking. I couldn’t agree more. It’s why I’ve published the full rules and data so anyone can rebuild my backtests from scratch in the ITM SPY Replication Pack. If my results don’t stand up, I want to be the first to know – and you’ll be the second.
Caveat Emptor
So next time an AI hands you a confident list of hot stocks, remember this: the people who build these things for a living tested them for 20 years, and the best fix they could come up with was… watch the tide.
Trade the tide, not the waves!
To the markets
It has been an OK week – ordinary was how I described it last week. It has been ordinary again this week. Yes we have had new highs, but they seem very lethargic – a kind of grudging, creeping new high, not a nice bouncy full-of-enthusiasm high.
But maybe I’m looking a gift bull in the mouth.
SPY Charts
SPY is still bouncing around in the trading channel. It really is directionless right now.
Longer term it is still on the same uptrend.
SPYG Charts
SPYG is slightly more cheering: a new high! Yeah! But I would be happier if there was slightly more conviction behind it.
Longer term, it is still in the top half of the trading channel.
QQQ Charts
Another new high! Nice. But – as for SPYG – I wish there was more conviction behind it – it is a pretty half-hearted effort.
Longer term it is still at the top of the trading channel.
VIX Chart (Volatility)
The VIX is similarly lethargic, but under 20 (low volatility) which is where we like it.
ITMeter
The week ahead
Services, interest rates and consumer sentiment take the spotlight this week. We get the services surveys on Monday, the Fed’s meeting minutes on Wednesday, and consumer sentiment on Friday. PepsiCo and Delta head the earnings calendar.
Monday 5 October
Announcements: ISM Services Index and the final September S&P Global services survey.
Earnings: No major announcements.
Tuesday 6 October
Announcements: August trade balance.
Earnings: Constellation Brands, Lamb Weston, RPM.
Wednesday 7 October
Announcements: Minutes from the Fed’s September meeting, August consumer credit.
Earnings: Applied Digital, Levi Strauss.
Thursday 8 October
Announcements: Weekly jobless claims, August wholesale inventories.
Earnings: PepsiCo.
Friday 9 October
Announcements: Preliminary October consumer sentiment from the University of Michigan, including inflation expectations.
Earnings: Delta Air Lines.
Wednesday’s Fed minutes and Friday’s consumer survey are the two releases I’ll be watching most closely. One offers a window into the Fed’s thinking; the other shows how households feel about the economy and inflation. Together, they could shift expectations for the Fed’s next move.
The futures
The futures are looking quite happy, evidently the jobs data was weaker than expected so expectations for another Fed rate hike are reduced. The usual bad-news-is-good-news syndrome.
Next week . . .
I’m monitoring AskITM – and so far he seems to be doing a good job – what are your thoughts? I must say I am missing the contact with readers – the comments have tapered off, everyone is asking the bot instead! I am feeling a bit redundant, so please keep the comments coming.
For next week – I haven’t thought of a topic yet, if anyone has an idea they would like me to explore then please let me know.
Hoping for a great week!
Heather
Trade the tide, not the waves
Comments
Sources: A group of researchers from the University of Edinburgh, with colleagues from UCLA, Oxford and Sungkyunkwan University in Korea, decided to find out properly. Their paper was presented in August at KDD, one of the big international computer science conferences.
Li, Kim, Cucuringu & Ma, “Can LLM-based Financial Investing Strategies Outperform the Market in Long Run?”, KDD 2026. Read the paper.
4 Responses
Hi Heather, I’ve been using the Turbo method and your book states that you should roll up when strike drops to 55%. However, the rollup calculator for QQQ has a 47% threshold. For SPY and SPYG, threshold is 48%. Please let me know which threshold is correct.
Hi Jason – honestly, I can’t remember offhand – I have to look it up too!
AskITM is trained on the latest book and it will give you the correct answer.
Not handballing this – just trying to make sure that there is one source of facts so that no-one gets mixed up.
The new book is now on Amazon – I am going to see if I can reduce the price for a special offer for readers here and will announce in next week’s blog. Just tidying up a few loose ends, and going to unpublish the old books so that people don’t get mixed up.
So AskITM will give you the right answer.
h
Hello, thank you for this amazing website. I have a question, being that the ITM signal are so infrequent (1-3 signals a year), how would I be able to make a living on just this strategy? Am I missing something? Thank you in advance.
John
Hi John
ITM isn’t an income strategy – its a capital amassing strategy.
We have looked at income strategies – and concluded that they don’t work – here’s a link: https://heathercullen.com/blog/selling-your-upside/
I also would love a weekly income (my sole income is trading, and has been for many years) – but I have concluded it isn’t possible without giving away a large part of your gains.
If you have a big enough portfolio then you could allocate a portion to weekly strategies – but again youare selling yourself short.
I just make sure that the money I take out to live on is less than my profits and doesn’t deplete my capital.
And it has been working well for quite a few years now.
h