Intraday stock data for student research: how Quantitative Trading at Brown uses the SiftingIO API

Where does a university quant club get historical intraday stock data for student research? Free public sources mostly stop at daily bars, and stitching together per-venue feeds means a different schema, auth scheme, and cleaning job for every source. Quantitative Trading at Brown, a student organization at Brown University, runs its member research on the SiftingIO API instead: each member has their own account and API key, and pulls historical intraday US equities OHLCV bars that arrive normalized under a single schema. The setup is described on SiftingIO's education page for the club, part of a program that offers free and discounted access for academic research, classrooms, and student projects.

The research pattern is worth copying well beyond a campus. Members combine their own alternative datasets with the market data to study how those signals associate with returns, volatility, and trading volume at intraday granularity. Alternative data on one side, minute bars on the other, a timestamp join in the middle: that is the same workflow a professional quant team or a weekend researcher runs, and the plumbing below applies to all of them.

Historical intraday stock data from one endpoint#

US equity bars live under /v1/hist/stocks/{ticker}/bars, with intervals from 1m up to 1mo. Historical endpoints require gzip: a request without Accept-Encoding: gzip returns 406 gzip_required rather than data. With curl that means adding --compressed:

curl --compressed -H "X-API-Key: $SIFTING_KEY" \
  "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1m&limit=200"

Pagination is cursor-based. Each response carries meta.next_cursor; pass it back as ?cursor= until it returns null. A Python loop that backfills minute bars looks like this (the requests library negotiates gzip on its own):

import os
import requests
import pandas as pd

BASE = "https://api.sifting.io/v1"
HEADERS = {"X-API-Key": os.environ["SIFTING_KEY"]}

def minute_bars(ticker):
    params = {"interval": "1m", "limit": 200}
    rows = []
    while True:
        r = requests.get(f"{BASE}/hist/stocks/{ticker}/bars",
                         headers=HEADERS, params=params)
        r.raise_for_status()
        payload = r.json()
        rows.extend(payload["data"])  # see /docs for the exact response envelope
        cursor = payload["meta"]["next_cursor"]
        if cursor is None:
            return pd.DataFrame(rows)
        params["cursor"] = cursor

bars = minute_bars("AAPL")

Because every asset class shares one schema, the identical loop pointed at /v1/hist/crypto/BTCUSD/bars or /v1/hist/forex/EURUSD/bars returns the same shape. A student who has written the equities version has already written the crypto and FX versions, which matters when a semester project drifts across asset classes.

One account and API key per member#

The default failure mode for any group project is a shared credential. Thirty students behind one key means one member's runaway backfill loop rate-limits the whole club, nobody can tell whose script caused the 429s, and rotating the key breaks everyone at once. Quantitative Trading at Brown avoids this: each member has their own account and API key, so every student gets an independent quota and a rate-limit budget that belongs to them alone.

The free tier is shaped for individual student work: 10,000 REST calls per month at 60 requests per minute, one API key, no credit card required. For work that needs more depth, SiftingIO's Education program offers free and discounted access for academic research, classrooms, and student projects. Every API response also carries X-RateLimit-Limit and X-RateLimit-Remaining headers, so a script can watch its own budget and pace itself rather than discovering the ceiling mid-backfill.

Joining alternative data to minute bars#

The club's stated research question is a general one: does a given alternative dataset associate with returns, volatility, or trading volume at intraday granularity? The mechanics reduce to a careful timestamp join. Bar timestamps arrive as Unix epoch milliseconds in UTC; convert them, sort, and compute the outcome variables first:

import numpy as np

bars["ts"] = pd.to_datetime(bars["t"], unit="ms", utc=True)
bars = bars.sort_values("ts").reset_index(drop=True)

bars["log_ret"] = np.log(bars["c"]).diff()
bars["fwd_ret_30m"] = np.log(bars["c"].shift(-30) / bars["c"])
bars["rvol_30m"] = bars["log_ret"].rolling(30).std() * np.sqrt(390)

alt = pd.read_csv("alt_signal.csv", parse_dates=["ts"])
merged = pd.merge_asof(alt.sort_values("ts"), bars,
                       on="ts", direction="backward")

merge_asof with direction="backward" pins each alternative-data observation to the most recent completed bar, which is the lookahead-safe choice: the signal is only ever compared against prices that existed when it fired. From merged you can regress 30-minute forward returns on the signal, bucket realized volatility by signal quantile, or test whether volume moves before price does. None of it needs infrastructure beyond pandas.

Common pitfalls#

• Forgetting gzip on historical pulls. The bars endpoints return 406 gzip_required without Accept-Encoding: gzip. The Python requests library and browsers send it by default; curl needs --compressed, and Go's http.Client handles it automatically unless you set a custom Accept-Encoding header, which silently disables transparent decompression.
• Timezone drift in the join. Bar timestamps are UTC epoch milliseconds. Alternative datasets are very often stamped in local time, and merge_asof will join a series that is four or five hours misaligned without complaint. The output looks plausible and the study is wrong. Localize both sides to UTC explicitly before joining, and re-check around daylight-saving transitions.
• Backfill loops that ignore the rate limiter. A cursor loop across a ticker list will eventually hit 429 rate_limit_exceeded. The response includes a Retry-After header in seconds; sleep for that long instead of retrying immediately, and watch X-RateLimit-Remaining to pace the loop before the 429 ever arrives.

A reproducible setup for any student group#

Nothing in the Quantitative Trading at Brown setup depends on the club itself: individual accounts, one normalized schema across asset classes, cursor pagination for backfills, and a timestamp join against whatever dataset a member brings. Any course, research group, or student organization can assemble the same stack, and the Education program exists to keep the access side approachable for academic work. Read the docs for the full endpoint reference.