406 Not Acceptable gzip API error: send Accept-Encoding: gzip and decode the reply

If SiftingIO returns HTTP 406 Not Acceptable with error: "gzip_required", the request did not advertise an accepted compression setting. Send Accept-Encoding: gzip and make sure your client also decodes the reply. For curl, use --compressed to do both. A 406 from another service may have a different cause, so check the response body before applying this fix.

This often appears when a request that worked in one tool is moved to another. Python Requests and Node's built-in fetch handle compression automatically. Low-level clients may need both a request header and an explicit decompression step. Fixing the encoding does not guarantee that the symbol, parameters or data permissions are correct; check the next response too.

Request negotiation and response decoding are separate jobs. The client table below shows which parts each library handles for you.

The response captures and payload measurements below come from the recorded live-API check on 2026-09-19. Counts and payload sizes describe that check, not guaranteed future responses. The code examples have since been tightened to make error handling explicit.

What the 406 gzip_required response looks like#

This is a daily bars request for AAPL with a valid key and no Accept-Encoding header:

curl -i -H "X-API-Key: $SIFTING_KEY" \
  "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12"

The response below was captured on 2026-09-19 at 23:03 UTC (status, selected headers, full body):

HTTP/1.1 406 Not Acceptable
content-type: application/json
content-length: 150
x-ratelimit-limit: 500
x-ratelimit-remaining: 499

{"error":"gzip_required","hint":"responses are large; clients must accept compressed bodies","message":"this endpoint requires Accept-Encoding: gzip"}

The documented error envelope is { "error": "error_code", "message": "details" }, and the errors page lists 406 gzip_required as "Heavy endpoint called without Accept-Encoding: gzip." The live body also carried a hint field that the docs don't list, so branch on error and treat hint as optional.

The same request with the header succeeds:

curl --compressed -H "X-API-Key: $SIFTING_KEY" \
  "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12"

{"data":[{"t":1788220800000,"o":316.986,"h":327.2698,"l":314.7528,"c":325.1578,"v":22960579}],"meta":{"as_of":"2026-09-19T23:03:16Z","symbol":"AAPL","interval":"1d"}}

That is the real reply from the same session, trimmed from eight bars to one.

Which endpoints return 406 without gzip#

The docs mark Accept-Encoding as a required header on the historical bars pages and on three XBRL endpoints: the full financials bundle, the single-concept series, and the screener. A check on 2026-09-19 sent each of the following with a valid key and no Accept-Encoding header. These returned 406 gzip_required:

• GET /v1/hist/stocks/AAPL/bars
• GET /v1/hist/forex/EURUSD/bars
• GET /v1/hist/crypto/BTCUSD/bars
• GET /v1/fnd/stocks/AAPL/financials/Revenues
• GET /v1/fnd/stocks/screener/Revenues/CY2024

These returned 200 with an uncompressed body in the same check: /v1/last/quote/forex/EURUSD, /v1/fnd/stocks/AAPL/profile, /v1/fnd/stocks/AAPL/ratios, /v1/fnd/stocks/AAPL/filings and /v1/fnd/economic-calendar. Commodities and DEX bars were not part of the check; their docs pages are the reference for those. So a quote poller built on the single-symbol quote endpoint, like the one in Live bid and ask price API: build a correct EURUSD and BTCUSD quote client, does not need this header for that quote endpoint, and the first 406 arrives the day someone adds a bars backfill to the same HTTP wrapper.

The requirement exists because of payload size. The docs describe the financials bundle as "5+ MB for a mature filer." In the 2026-09-19 check, GET /v1/fnd/stocks/AAPL/financials transferred 346,475 bytes compressed and decoded to 4,972,987 bytes of JSON, roughly a 14 to 1 ratio. Your bundle sizes will differ by filer.

Sending Accept-Encoding: gzip, client by client#

Choose the recipe for your client. Setting Accept-Encoding tells the server what you can receive; it does not configure every client's decoder.

Custom transports, proxies and disabled compression options can change the defaults.

curl#

By default, plain curl does not offer gzip, so SiftingIO's gzip-required endpoints return 406. The --compressed flag does both jobs: it sends the header and decodes the reply. Setting the header with -H does only the first job:

# Prints binary: the header was sent, nothing decodes the body
curl -s -H "Accept-Encoding: gzip" -H "X-API-Key: $SIFTING_KEY" \
  "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12"

# Works: decode it yourself
curl -s -H "Accept-Encoding: gzip" -H "X-API-Key: $SIFTING_KEY" \
  "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12" | gunzip

In the check (curl 8.14.1), the first command's output started with the bytes 1f 8b, the gzip magic number. If a JSON parser ever complains about an unexpected character at position 0 and a hex dump shows 1f 8b, you are parsing a compressed body. Use --compressed and drop the manual header.

Python#

Requests (2.32.3 in the recorded check) advertises supported compression methods and decodes gzip automatically. The exact advertised list depends on the installed dependencies. A plain requests.get with only the X-API-Key header returned 200. Setting Accept-Encoding: gzip by hand also worked, because urllib3 decodes based on the response's Content-Encoding header regardless of who set the request header. Two things break it. Passing 'Accept-Encoding': None removes Requests' default compression offer and produced a 406 in the recorded check. The underlying HTTP layer may send identity instead. Reading response.raw with stream=True bypasses automatic decoding by default and returned bytes starting with 1f 8b. Use .json(), .content or iter_content() when you want decoded content. The Requests quickstart documents the difference.

The standard library is different. urllib.request does not offer gzip or decompress the body automatically, so handle both steps:

import gzip, json, os, urllib.request

url = 'https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12'
req = urllib.request.Request(url, headers={
    'X-API-Key': os.environ['SIFTING_KEY'],
    'Accept-Encoding': 'gzip',
})
with urllib.request.urlopen(req, timeout=20) as resp:
    raw = resp.read()
    encoding = resp.headers.get('Content-Encoding', '').strip().lower()
    if encoding == 'gzip':
        raw = gzip.decompress(raw)
    elif encoding not in ('', 'identity'):
        raise ValueError(f'Unsupported Content-Encoding: {encoding}')
bars = json.loads(raw)['data']
print(len(bars), 'bars')

Run on 2026-09-19 this printed 8 bars. Without the Accept-Encoding line, urlopen raised HTTPError 406.

Node#

The built-in fetch (Node 22.22.0 in the check) offers compression by default and decodes the reply, including when you set Accept-Encoding: gzip yourself. Both forms returned parsed JSON. Overriding the header with identity produced the 406, which is the case to look for in a wrapper that sets default headers for every request.

The node:https module is the by-hand case. It sends no Accept-Encoding and returns the body as it arrived:

import https from 'node:https';
import zlib from 'node:zlib';

const url = 'https://api.sifting.io/v1/hist/crypto/BTCUSD/bars?interval=1h&start=2026-09-10&end=2026-09-11';
function reportError(err) {
  console.error(err.message);
  process.exitCode = 1;
}

const req = https.get(url, {
  headers: { 'X-API-Key': process.env.SIFTING_KEY, 'Accept-Encoding': 'gzip' },
  signal: AbortSignal.timeout(20_000),
}, (res) => {
  const chunks = [];
  res.on('error', reportError);
  res.on('data', (chunk) => chunks.push(chunk));
  res.on('end', () => {
    try {
      if (res.statusCode !== 200) throw new Error('HTTP ' + res.statusCode);
      let body = Buffer.concat(chunks);
      const encoding = String(res.headers['content-encoding'] || '').trim().toLowerCase();
      if (encoding === 'gzip') body = zlib.gunzipSync(body);
      else if (encoding && encoding !== 'identity') {
        throw new Error('Unsupported Content-Encoding: ' + encoding);
      }
      const payload = JSON.parse(body);
      if (!Array.isArray(payload.data)) throw new Error('Expected a data array');
      console.log(payload.data.length, 'bars');
    } catch (err) {
      reportError(err);
    }
  });
});
req.on('error', reportError);

The recorded API check returned 25 bars for this request. The revised example adds a 20-second abort signal and handles network, response-stream and decoding errors. It does not follow redirects. Use a supported Node.js LTS release. The Node HTTPS documentation describes the underlying request API.

Many Node HTTP libraries sit on top of node:https, but their decompression settings differ. Check the library's behaviour before adding a manual decoder, or you may try to decompress content that is already plain JSON.

Go#

The automatic-decompression behaviour here follows the Go net/http documentation. Go's Transport adds Accept-Encoding: gzip on its own and decodes the reply transparently, but only when the request has no Accept-Encoding value already. The DisableCompression field documentation distinguishes a header added by the transport from one supplied by your code. So the simplest correct Go client sets only X-API-Key and leaves Accept-Encoding alone. If you set the header yourself, or a shared helper sets it for you, resp.Body holds gzip bytes and you decode them:

package main

import (
	"compress/gzip"
	"fmt"
	"io"
	"net/http"
	"os"
	"strings"
	"time"
)

func main() {
	url := "https://api.sifting.io/v1/hist/stocks/AAPL/bars?interval=1d&start=2026-09-01&end=2026-09-12"
	req, err := http.NewRequest("GET", url, nil)
	if err != nil {
		panic(err)
	}
	req.Header.Set("X-API-Key", os.Getenv("SIFTING_KEY"))
	// Setting this by hand turns off Go's transparent decoding.
	req.Header.Set("Accept-Encoding", "gzip")

	client := &http.Client{
		Timeout: 20 * time.Second,
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
			return http.ErrUseLastResponse
		},
	}
	resp, err := client.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		panic(fmt.Errorf("HTTP %d", resp.StatusCode))
	}

	var reader io.Reader = resp.Body
	encoding := strings.ToLower(strings.TrimSpace(resp.Header.Get("Content-Encoding")))
	if encoding == "gzip" {
		gz, err := gzip.NewReader(resp.Body)
		if err != nil {
			panic(err)
		}
		defer gz.Close()
		reader = gz
	} else if encoding != "" && encoding != "identity" {
		panic(fmt.Errorf("unsupported Content-Encoding: %s", encoding))
	}
	body, err := io.ReadAll(reader)
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.StatusCode, len(body))
}

The Content-Encoding check works with both automatic and manual decompression because Go removes that header after automatic decoding. The example also sets a 20-second request timeout and refuses redirects so the API-key header is not carried to a different URL. The same source has two more conditions worth knowing: the transport does not add gzip when the request carries a Range header or when DisableCompression is true. By that logic a Go client with either one would send no Accept-Encoding and receive the 406. Those configurations were not tested against the live API.

Send exactly gzip when you decode by hand#

The value of the header matters as much as its presence. On the AAPL bars request in the 2026-09-19 check, Accept-Encoding: gzip returned Content-Encoding: gzip. The values identity, br alone and deflate alone each returned 406. The value gzip, deflate, br returned 200 with Content-Encoding: br, a brotli body.

That last case is harmless for clients that decode on their own. Node's fetch received brotli by default and parsed it without any extra code. It is a trap for hand-written decoders: if you copy a browser's Accept-Encoding line into a client and then call a gunzip function on the reply, the gunzip fails on a brotli body. When you decode by hand, offer only gzip, and branch on the response's Content-Encoding header, as the snippets above do, instead of assuming the encoding.

When the client is right and a proxy is wrong#

If the code sends the header and the 406 persists, look at what sits between the client and the API. A forward proxy, an API gateway or a service mesh sidecar can strip Accept-Encoding or rewrite it to identity, often so that the intermediary can inspect bodies. The API then sees a request without gzip and refuses it. Two checks separate the cases. First, run the same request with curl --compressed from the same host without the proxy. Second, inspect the outbound Accept-Encoding at the proxy itself, since the client's own log only shows what it meant to send. Redact X-API-Key and other credentials from diagnostics. The reverse fault also exists: an intermediary that decodes the body but leaves Content-Encoding: gzip in place makes the client try to decode plain JSON. The 1f 8b check on the first two bytes tells you which situation you are in.

Handle gzip_required without a retry loop#

The errors page gives retry guidance for 429 (wait for Retry-After) and for 502 and 503 upstream errors (retry shortly). A confirmed 406 gzip_required needs a request change, not repeated retries of the same headers. Give that specific error an actionable message; do not label every 406 as a gzip problem:

import os
import requests

class RequestBug(Exception):
    pass

def get_json(url, headers=None):
    h = {'X-API-Key': os.environ['SIFTING_KEY']}
    h.update(headers or {})
    r = requests.get(url, headers=h, timeout=20)
    if r.status_code == 406:
        try:
            body = r.json()
        except ValueError:
            body = None
        if isinstance(body, dict) and body.get('error') == 'gzip_required':
            raise RequestBug('Offer gzip and ensure the client decodes the response')
        # A different 406, including an HTML proxy response, remains an HTTP error.
    r.raise_for_status()
    return r.json()

url = 'https://api.sifting.io/v1/hist/forex/EURUSD/bars?interval=1h&start=2026-09-10&end=2026-09-11'
print(len(get_json(url)['data']), 'bars')

In the recorded live check this URL returned 25 bars, and explicitly requesting identity produced 406 gzip_required. The revised wrapper turns that error into RequestBug: Offer gzip and ensure the client decodes the response. Other 406 responses still go through raise_for_status().

If one wrapper serves both snapshots and history, as in Real-time FX and crypto quotes: REST snapshots and WebSocket streams, keep compression handling consistent across both. With Requests or Node fetch, their defaults normally suffice. With curl, use --compressed. For a low-level client, explicitly offer gzip and decode according to Content-Encoding. Test one bars request through your application's real HTTP wrapper and proxy configuration, using a controlled test account or a local fixture. A quote-only test will not catch a missing gzip header on a bars request.

The revised Node, Python and Go decoding and error-handling examples passed 23 offline fixture checks on 2026-09-21. Those checks used simulated responses, not a real API key, and do not revalidate the dated live measurements above.

The per-endpoint header requirements and the full error table are in the API reference. Read the docs