Common Algorithmic Trading Pitfalls and Why Backtests Fail

I am currently compiling a list of the most basic strategies used in algorithmic trading.

• Trend Following (+Momentum)
  • MA Cross
  • EMA Cross
  • MACD (ingpawat)
  • RSI / LRSI (Mike_Trdw)
  • New Low or High
  • ATR (Away-Box793)
  • VWAP
• Seasonal
  • Sell in May and Stay away
• Mean Reversion (Mike_Trdw)
  • Mean Reversion To Trend
  • Mean Reversion in Range (The-Goat-Trader)
  • Reverting Market (The-Goat-Trader)
• Momentum Rotation (Tactical Allocation) (The-Goat-Trader)
• Grid Trading (Mike_Trdw)
• Arbitrage
• Offset Trades / Trading Pairs
• Index fund rebalancing
• Market timing
• Scalping
• Price Pattern / Candle Stick
• Price Forecasting
  • Neural Networks
• News-based
• Market Sentiment
• Trend line
  • Break
  • Bounce
• Standard SMA
  • break (SMA 20D, 50D, 100D, 150D, 200D)
  • bounce
• Range Breakout
  • Open Range Break Out
  • Horizontal Compression Breakout
  • Wedge Compression Breakout
• Options
  • Covered calls (Chuyito)
  • "The Wheel" on LargeCaps (faancy5050)
  • Short Iron Condor on SPY 0DTEs (faancy5050)
  • Long Straddle or Long Strangle on any earnings (faancy5050)
• Smart Money Concepts (good read, Franco_Love)
• "Martingale" (reckless_homicide)
  • Me: It is risky but it is a classic and basic strategy for you to play with. There are good papers on it too, so it made the list.

---

• BT_2112 mentioned that most of those strategies can profit from applying those on multiple timeframes simultanously. Since I do this too when it comes to my manual trading, I can see how this can be a very good improvement/factor when it comes to algorithmic trading.
• EntrepreV Just dropped the content bomb on us. He mentioned the paper 151 Trading Strategies which is just 370 pages of 151 trading strategies explained including formulas and stuff. I have yet to start reading, but I guess, if you want to have some more strategies to choose from, this might take home the whole cake. Many thanks!
• disaster_story_69 Mentioned having a visit of Trading Views library of pine scripts. I took a look, and it looks like a ton of content one can mine, especially if the pine scripts open source.
  • Link to the library (filter = strategies):
  • Link to the editor's picks (filter = strategies):
  • While some is noise, some others look like driving legit strategies
• artemiusgreat mentioned several sites dedicated to algorithmic trading (I briefly checked the costs)
  • Free: MQL5 Codebase: 
  • 600$-1200$/year:
    • Papers with backtests: 
    • Quantpedia: 
    • QC codebase: 
• The-Goat-Trader (got his comment removed for unknown reasons) mentioned Momentum Rotation and pointed to the sources Gary Antonacci GEM, Ned Davis or Meb Faber 3-Way Model, Andreas Clenow for further informations.

---

If you want to add to the list, just drop a comment and I will edit the post and add it together with an honorary mention of your username. (If two suggest the same strategy twice, time of comment will be the deciding factor).

--

I simply want to implement different strategies and see which is performing which way to test my software and also broaden my knowledge.

Thanks for participating!

Built this in about 4 weeks, results from tradingview strategies starting Jan 1 (as much data as I could pull from TV)

(Edit: this system/backtest is trading only 1 ES contract)

Overfitting: most people optimize their strategies to fit past noise perfectly. The moment they go live, disappointment sets in

Algo traders usually do backtesting and only go live after getting positive results with proper confirmation. If the backtesting results are good, then logically, once live, there should be many successful traders, since there's no human emotion involved that leads to overtrading.

So why don't we see many successful algo traders in reality? What am I missing here?

For one thing its a pretty tough nut to crack. Took me years. For another thing- do we even know how many there are? Not sure how you would know that.

it did 200 percent in 4 months on es? that’s wild

So I’m currently trying to design a strategy at the moment. A lot of people here will have way more experience in terms of developing an algorithm than I do. I just wanted to ask, so I can learn from them, what mistakes did you make? If you could do it again, what would you change etc?

Mistakes? It's one big learning curve, every day is a school day.

I think the biggest issue was converting my "strategy in my head" into an algo that mimicked it. Takes time, it's easy to look at a chart and think "I can see what is happening here" as opposed to converting that mind process into actual code which replicates what you are thinking.

Hmm what else, you can never have enough derived data to confirm your thoughts.

Trying to make it do too many things at once. Also not knowing the market enough.

Just because you can code doesn't mean, it'll work.

I’ve been diving deeper into algorithmic trading recently mostly focusing on strategy development, back testing discipline, and execution logic.

One thing I’ve realized is that it’s really easy to overcomplicate things early on.

Curious to hear from more experienced traders here:

What’s one mistake that slowed your progress when you started with algotrading?

ML? Rule-based? Traditional indicators or alternative data?

Hey everyone! I'm primarily a day trader and just decided to try algorithmically trading one of my profitable strategies for the first time. The challenge was translating all my manual conditions into code, and honestly, I couldn't figure out a clean way to include everything. But the backtest results still came out pretty solid, so I thought I'd share.

Backtest Summary:

• Total P&L: +$70,278.56 USD (+70.28%)
• Total Trades: 3,349
• Win Rate: 59.87% (2,005 wins / 3,349 trades)
• Profit Factor: 2.434
• Max Drawdown: 1.71% ($2,803)
• Equity Curve: Steady, consistent growth over the backtest period

I'm happy with how the equity curve looks—no wild swings or catastrophic drawdowns. The profit factor is solid too. That said, I know there are some nuances to my manual strategy that didn't make it into the code, so the real-world results might differ.

I'd love to hear your feedback, especially if anyone has tips on translating complex trading logic into code. Also curious if there are any glaring red flags in these metrics I should be watching for.

Thanks!

Hi!

I'm a software developer and I'm almost 40 years old. I always wanted to create a systems-dependent business, i. e. one where revenue is not tied to me working (where I don't sell my time for dollars).

By "business" I mean a source of income that would generate enough money for two people living in a Western country, with one of them (me) working for 40 hours per week (that would be 4000-6000 Euros after taxes). Ideally, that source of income would be location-independent so that I can live wherever I want.

I also prefer to work as a freelancer without a boss and without employees to manage. Also, I'm getting older and I want to have a source of income where nobody can fire me (or make me retire) because of my age. I was looking for a business idea that would provide money for me when I'm too old for the labor market (I don't want to rely on the state pension system).

Recently I had a business idea that seems perfect for me. Therefore, I want to do a pre-mortem -- I would appreciate, if you wrote in the comments all possible reasons this strategy could fail. I value honesty over politeness, therefore be as direct as possible.

What is the product I want to create?

Trading algorithms.

How much do I want to earn?

Between 48,000 and 72,000 Euros per month after taxes (55,000 -- 83,000 dollars) per year.

How do I want to achieve that goal?

The road to that goal consists of two phases:

1. Learning
2. Operating

Phase 1: Learning (1-5 years)

First, I would learn the basics of quantitative finance and try out the concepts in IPython notebooks. This is important because I learn things through experimentation.

Second, and most important: I would develop trading algorithms on Quantopian (or any other platform with similar functionality), run backtests on them, and submit them to their contest. In scope of that contest the performance of trading algorithms is evaluated over a period over 60 days on current (non-historical) data.

Latter two items (backtests and the contest) would give me feedback, which is crucial for learning. Especially the backtesting looks like a way to get feedback on the quality of my algorithms fast. It almost looks like I could use deliberate practice#Deliberate_practice) to become better.

I don't expect to earn any profits during that time. Therefore I will work full-time to support myself during this time and work for about 20 hours per week on trading algorithm business (20 hours per week in addition to a 40-hour job is definitely doable according to my time tracking statistics).

Phase 2: Operating (after phase 1 till my death)

This phase starts when I have developed several well-performing algorithms on a regular basis.

Now I could

• license them to other people or institutions (i. e. they buy the code of my algorithm and use it to invest their money) and
• use them myself (I invest my own money) via a platform like QuantConnect.

Why do I like this idea?

• Fun: I could earn money by following my interests. I always wanted to know how the economy works. In order to develop successful trading algorithms, I will need to deepen my knowledge in economics and finance. Also, if I manage to create profitable trading algorithms, this will be a proof that my understanding of how the economy works, is correct. In fact, I came up with this idea after asking myself "How can I make money doing macroeconomics?". Learning about economics (with finance being part of it) is fun. This can help push through the learning phase. Also, I tried to get into finance in my youth, but it failed (I approached the whole thing from the wrong angle). Building this business is like making my childhood's dreams come true.
• Fast and easy feedback: It's very easy to get feedback on the quality of my product (the algorithm). At least one platform gives you several metrics on how well your algorithm performed in real life (tells you where you can improve). This is different from most other products. For example, if the product is an app or a web site, it's challenging to find out whether or not it will be useful for the target audience. Even finding people who would give meaningful feedback on a product idea is hard because most products and services are so multi-faceted.
• Tailored to my personality: I like situations, where I can make money based on my skills. Like, I come to a company, they ask me to do a test task, and then hire me based on results. Some of the companies that hired me that way didn't care about my personality, whether I can present myself, whether I'm likable, whether I had experience etc. I assume that when I sell trading algorithms, it's the same -- my skills matter, my personality doesn't. Analogy: Imagine a weight-lifting competition. A shy person whom nobody knows walks in and lifts a record weight. He or she will get the prize, regardless of his personality and whether or not they have connections. (And if that person is too weak, they will know exactly by how much.)
• Contrary to most other products, an algorithm can sell itself. That's good because I'm not a particularly good communicator.
• I can re-use my programming skills to generate a semi-passive income.
• I assume you can do this business from any place in the world with a good enough Internet connection.
• Value for the customer is easily visible. I can see how much money an algorithm can generate for its owner. It is much harder to estimate the financial upside of an app, a web site, a gadget, or a software development service. This means it is easier to sell a good trading algorithm than a good app, web site membership, gadget, or software development service.

What can go wrong?

I am aware of the following risks to this plan. Feel free to comment on them (especially, if you consider a particular risk very, very unlikely) and add your own ideas of what can fail.

• I fail to acquire the knowledge and skills required to create quality algorithms (fail to complete the learning phase).
  • Counter-argument: See the part about fun. On several occasions in my life I did things related to economics and finance out of pure interest. The earliest time when I got interested in economics was at an age of 5-7 years. It's easier to push through hard times when the thing you are doing is fun.
• My math skills are not good enough.
  • Counter-argument: In the past I co-operated or hired people to help me with math problems that were too hard to me. When I can't learn a mathematical concept from a book, I usually can learn it with the help of another human.
• Due to a catastrophic event, such as Dollar losing its status as the reserve currency, the stock market collapses and nobody needs any trading algorithms any more.
• There are not enough buyers for trading algorithms in Europe (especially in its German-speaking parts such as Germany, Switzerland, Austria) apart from platforms like Quantopian and CloudQuant.
• There are potential buyers for the algorithms, but they don't want to work with me for cultural reasons. I can imagine that a big bank would never buy anything from a one-man-company, regardless how good the product is.
• I need to become certified to sell my trading algorithms (or buy-sell signals based on them) and the bureaucratic hurdles are insurmountable (I'm not sure whether it applies to selling algorithms, but this list of requirements is frightening).
• Algorithmic trading becomes outlawed.
• Trading platforms (Quantopian, CloudQuant, QuantConnect and others) go out of business and I won't be able to sell my algorithms to anybody.
• Revolution in AI makes human algorithm developers obsolete.
• Using my own algorithms with own money on platforms like QuantConnect turns out to be not viable because of too high transaction fees. I heard such complaint about Robin Hood trading platform (the fees are so high that an average investor can't make profits trading on this platform).

Thanks in advance

Franz Drollig

You learn the most when the bullets are live.

building only one algo could also be viewed as a mistake. the more uncorrelated edges you can combine the smoother the ride. the portfolio approach - trading multiple strategies - is often lauded as the way to go

When I over rule my algorithmic signal, I always get into trouble!

I spent months building pattern detectors that looked incredible on historical data and then watched them fall apart in live markets. The core mistake was confusing "fitting the past" with "predicting the future."

A few specific ways overfitting burned me:

"Textbook perfect" is a trap. I assumed the more geometrically perfect a pattern looked, the better it would perform. Turns out, textbook-perfect setups are crowded trades: everyone sees them, they're already priced in. Moderate-quality patterns with less obvious characteristics often outperform. This was counterintuitive and took a long time to accept.

Feature fishing without correction. When you test 17 features looking for significance at p < 0.05, you're almost guaranteed to find "winners" that are pure noise. The fix was applying Bonferroni correction (alpha/number_of_features) and requiring consistent direction across chronological train/validation splits. If a feature says "higher is better" in one time period and "lower is better" in the next then it's noise, no matter how strong the p-value.

Optimizing thresholds to the decimal. Early on I'd tune a threshold to 14.73 because that's where the backtest peaked. That's memorizing your dataset. Now I use simple, round thresholds and require that performance doesn't collapse when I perturb them ±20%. If your edge disappears with a small parameter shift, you never had an edge.

The validation discipline that actually helped: chronological splits (60/20/20), never touching the holdout set until you're ready to commit, minimum sample sizes per bucket, and effect size requirements (Cohen's d > 0.3); not just statistical significance. A "significant" result with a tiny effect size is useless in practice.

The humbling part: even with all these protections, I still catch myself occasionally. Overfitting is the default state of quantitative work. You have to actively fight it at every step.

Hey so I've recently been working on a backtesting engine for a personal project and I've been reading up on survivorship bias, fees, slippage, taxes, margin rules, regulations (Reg T) etc to try and get a very precise result on my strategies.

Currently the workflows start with you doing some research, then testing out strategies, then layering it out in a portfolio with a beta layer and one or many alpha layers to improve your risk/return profile, and finally stress testing it ideally since 1990 but I would prefer going back to 1945 if possible.

Recently I've been putting in a bigger focus on US equities although I am planning on working for Canadian equities too but I then have to read up on all their regulation and tax systems too. What are some more niche but highly important costs or workflows you think should be included for good research and backtesting?

Currently the project uses Norgate since they seem to have the best survivorship bias and their updater downloads all updates every hour, but I'd be interested to know what other sources anyone uses to go back as close as possible to 1945. The one downside here is that rebalancing can only be once a day at most since the granularity stops there (for now). It also uses IB Gateway to send orders via API since I can hook up a paper and live account here.

The current project can be found here:

I'd appreciate any feedback anyone may have :).

Building a backtesting engine and designing strategies are two very different things.

If you wanna build a house, you shouldn't be designing nail guns. You should design a house.

Processing my data using databases like SQL / SQL lite. For years of data it would sometimes take more than a day (like 30 hours) for a single backtest to get my results

I have since then went to my own Dataframe which makes more since knowing the Dataframe runs on RAM and SQL runs on cloud / hard storage.

That switch alone 10x or more my speed and can do many years like 5+ in about an hour or two (walk-forward) backtest

The only other regret I have is not writing in c++ yet (don't know enough of it to rewrite my program) . I know that it would be way faster than Python what Im writing in.

That's a reasonable place to start. Just build a good enough MVP so that you can start testing strategies and improve it as you go. If you're a software engineer it's easy to get lost in designing the backtesting engine and make zero progress doing the thing that actually makes you money - designing strategies and running them live. Some people here spend 1900 hours designing the backtester and 100 hours designing strategies when it should be the other way around.

Take something live to the paper server quickly so that you can troubleshoot your backtester assumptions.

One of the biggest issues is false confidence from a clean backtest. data leakage can be subtle, like fitting a scaler on the full dataset, using the current candle's close to enter at that same close, repeatedly tuning against the same test period, or excluding delisted assets. then there's the backtest to live a gap, spreads, fees, slippage, partial fills, latency, missing candles, time zone mismatches, and changing market conditions. a strategy trained during a low volatility trend can quickly lose its edge when the market becomes choppy. with AI generated code, the dangerous part is that it can run without errors while the trading logic is still wrong. i'd trust a modest result that survives walk forward testing and realistic costs more than a nearly perfect equity curve. are you seeing more issues with the data/model side or when moving from backtesting to paper or live trading?

The one that cost me most recently was a timestamp mismatch that never threw anything. Daily bars came stamped at the session open, the option chain rows were stamped at midnight, and the lookup joined the two on exact equality. Zero of 121 days matched. Every strategy came back with no entries. The runs completed cleanly and reported nothing kept, which is indistinguishable from an honest result about a ticker that simply had no edge. Finding it meant reading the code path rather than the output. That is the shape I watch for now. A crash is free information. The expensive failures are the ones that hand you a number you have no reason to doubt.

I’m not sure how common these issues are, but they were significant learning moments in my development process.

Avoiding leakage when training ml models. It seems so basic but when dealing with sub ms, it’s easy to accidentally include something that isn’t known until after the decision you are trying to predict.

I switched computers and forgot that can change the way floats are calculated.

Breaking apart the program so that I had dozens of programs that would spread out across all the processors and reprogramming some things in rust to cut down on memory usage. Helped with hardware limitations I was facing