Run the same strategy over the same date range with historical price data from two different vendors and the equity curves rarely match. Sometimes the gap is a few basis points of drift that nobody notices. Sometimes one run shows a working system and the other shows a losing one, with not a single line of strategy code changed. The instinct is to debug signal logic. The better first move is to interrogate the bars, because historical datasets disagree in specific, repeatable ways, and every one of them changes backtest results silently.
Adjusted or unadjusted: the overnight move that never happened#
A 4-for-1 split cuts a stock's price by 75 percent overnight. In unadjusted bars that looks like a crash. AAPL split 4-for-1 in August 2020; a momentum system reading unadjusted bars sees a catastrophic gap down and sells, a mean-reversion system sees an extreme dip and buys, and both are reacting to an event in which no holder gained or lost anything. Dividends do the same at smaller scale: the price drops by roughly the dividend on the ex-date, and a strategy that shorts overnight weakness will harvest phantom profits from it.
Adjusted bars rescale history so the series is continuous, but "adjusted" is a family of methods, split-only or split-plus-dividend, and adjustment factors are recomputed every time a new dividend is paid. Adjusted history is rewritten retroactively by design. Two vendors can both truthfully call their bars adjusted and still hand you different numbers for the same day in 2019. A backtest that mixes the two is wrong around every corporate action in its date range.
The high and low depend on which venue printed them#
An OHLC bar looks objective. For anything traded on more than one venue, it's an editorial product. US equities trade across many venues at once, and a mid-cap crypto pair trades on dozens, with real dispersion between them. The open and close of a minute tend to agree across sources because they're anchored to a dense stream of trades. The high and the low are single prints, and single prints are exactly where a thin venue, a fat-fingered order, or a stale feed leaves its mark.
The backtest consequence is concrete: a stop-loss 1 percent below entry fires in the dataset whose source venue printed a momentary low, and never fires in the dataset built from a different venue. One run is stopped out into a loss, the other rides the position to profit, and the divergence compounds from there. Intraday extremes drive stops, breakout entries, and ATR sizing, so this is a first-order effect rather than noise.
This is the fault a blended price exists to fix. SiftingIO publishes one price per instrument, formed as a volume-and-reputation-weighted median across independent venues; outlier prints are scored with median absolute deviation before they can enter the aggregate, and feeds that go stale or freeze are excluded. The method is public, on the data methodology page. The property that matters for backtesting: a single venue's bad print can't become the bar's low, so simulated stops fire on prices the market broadly reached. The bound is stated there too. A median holds while a minority of venues are wrong; no aggregation survives a majority erring together.
Gaps, weekends, and bars that don't exist#
Crypto trades through the weekend. Stocks don't, and spot FX pauses from Friday evening to the Sunday session open. Inside a session, thin hours and trading halts produce minutes in which nothing printed at all. Vendors resolve this differently: some emit no bar, some forward-fill the last close as a flat bar with zero volume, some interpolate. None of these choices is announced in the data itself.
Indicator math changes under each policy. A 20-period moving average on 1m bars spans 20 minutes of market time in a gapless dataset and possibly hours in one that skips empty minutes. ATR decays toward zero across a run of forward-filled flat bars, which shrinks stops right before the market reopens and moves. Auditing the policy takes a few lines:
curl -s --compressed -H "X-API-Key: $SIFTING_KEY" \
"https://api.sifting.io/v1/hist/crypto/BTCUSD/bars?interval=1m&limit=2000"
import pandas as pd
# bars: one row per 1m bar, timestamp column "t" (adjust to your source's field name)
bars["t"] = pd.to_datetime(bars["t"], utc=True)
expected = pd.date_range(bars["t"].min(), bars["t"].max(), freq="1min")
missing = expected.difference(bars["t"])
print(len(missing), "missing minutes")
Run that against every candidate source for the same symbol and week. The counts will differ, and the backfill policy becomes a known quantity instead of a hidden variable. One gotcha while doing it: SiftingIO's historical bar endpoints require gzip and return 406 gzip_required when the client doesn't send Accept-Encoding: gzip, so curl needs its, compressed flag. A bare Accept-Encoding header returns a 200 but writes raw gzip bytes to stdout.
Where the day ends#
A daily bar needs a rule for where the day ends. Equities inherit a session close, but FX and crypto run around the clock, so the daily candle boundary is a vendor decision: UTC midnight, 5 pm New York, or something else. Every daily close a strategy trades on shifts with that choice, and so does every indicator built from those closes. Two daily datasets for EURUSD can disagree on the close of every single day while each stays internally consistent. Daylight saving makes it worse, since a dataset cut on a local clock moves its boundary twice a year, and weekly bars inherit whichever convention the dailies used. SiftingIO buckets FX and crypto bars in UTC. Whatever the source, write the boundary into the backtest's stated assumptions: a strategy validated on 5 pm closes and deployed against UTC closes is trading prices it was never tested on.
History that changes after you tested on it#
Vendors correct bad prints. That's good hygiene, and it also means history is mutable: a backtest run today and the identical run from last month may have consumed different bars, while dividend-driven adjustment recomputation rewrites history on a schedule of its own. If results feed a decision, reproducibility has to be engineered. Snapshot the exact bars a run consumed, hash the file and store the hash next to the results, record the response's as_of metadata where the API provides it, and rerun the hash check before trusting an old result. When two runs disagree, the first question is whether they read the same bytes.
A checklist to run against any historical price data source#
- Pull daily bars across a known split (AAPL, August 2020) and confirm from the numbers, not the marketing page, whether bars are adjusted and by which method.
- Diff one month of 1m bars for one symbol against a second source: count mismatched highs and lows, and missing timestamps.
- Establish how empty periods are handled (absent bar, zero-volume flat bar, interpolation) and verify with the gap script across a weekend and a halt.
- Identify the clock the daily boundary cuts on, and whether DST moves it.
- Confirm whether a bar's timestamp labels its open or its close; off-by-one-bar joins between prices and signals come from mixing the two.
- Pull the same range twice, weeks apart, and hash both responses; ask the vendor how corrections are published.
For reference on coverage: SiftingIO serves historical OHLCV bars for US stocks, forex, crypto, and commodities under one schema and one API key, with intervals from 1m to 1mo for stocks and 1m to 1h for forex and crypto. History depth is plan-dependent: one month on the free tier, one year on Builder, full depth from Pro upward. If this checklist becomes part of your data-source evaluation, the endpoint details are in the docs.
