Open-Sourcing market-data-normalizer v0.4.0: Guardrails for Dirty Data

Every quant has been burned by a bad tick. We are releasing v0.4.0 of our pure-Python market data normalizer to protect your backtests from cascading data failures and naive resampling.

A single 200x print. A gap where the WebSocket feed dropped. A timestamp that runs backwards. These are the silent killers of quantitative research. When these anomalies leak into your dataset, your backtest quietly starts lying to you, presenting a beautiful equity curve built on physical impossibilities.

Today, the HarvestGroup360 engineering team is excited to announce the release of market-data-normalizer v0.4.0, a lightweight, zero-dependency Python library designed to build robust data-quality layers directly into your research pipeline.

Anchored Outlier Detection

In v0.4.0, we've introduced a robust data-quality guardrail system that actively flags or drops price outliers, gaps, out-of-order records, and non-positive fields. Most naive outlier detection algorithms fail during extreme volatility because a legitimate price spike is treated as an error, corrupting the subsequent state.

Our outlier detection anchors on the last known good price, ensuring that a single bad print doesn't cascade into the next. It produces a structured report of everything it caught, providing a boring but essential guardrail that saves you a day of debugging a "highly profitable strategy" that was actually just dirty data.

The Danger of Naive Resampling

Beyond dirty ticks, another major focus of this release is proper time-series resampling. Consider the transition from 1-minute bars to 5-minute bars. It sounds trivial, but naive resampling just averages the Volume-Weighted Average Price (VWAP) of the 1-minute bars, which quietly hands you the wrong number.

To calculate it correctly, you have to weight by volume—a bar's VWAP multiplied by its volume is its traded notional. That is the metric that actually adds up. This is a tiny correctness detail, but it makes a massive difference three steps downstream in a backtest when your execution simulator assumes liquidity that doesn't exist.

Pure Python, Zero Dependencies

We believe foundational infrastructure should be accessible. The `market-data-normalizer` remains a pure Python repository with zero external dependencies, making it incredibly easy to integrate into any existing Jupyter workflow or Airflow pipeline.

Check out the code and documentation on our GitHub, and start building strategies on data you can actually trust.

← Back to Blog