Why curl Works but Rust’s reqwest Gets Blocked by Cloudflare

I hit a puzzling problem while building a small price tracker in Rust. The same CoinGecko API call, with the same URL and the same API key, behaved completely differently depending on the client:

  1. curl returned the correct JSON with the correct values.
  2. A Rust program using tokio and reqwest failed with a Cloudflare error.

If the request is logically identical, why does one succeed and the other fail? Because Cloudflare doesn’t decide based only on the URL and the key. It looks at how the request is made.

Why curl passes and reqwest doesn’t

1. Missing or different User-Agent (the most common cause)

curl automatically sends User-Agent: curl/8.x and Accept: */*. reqwest sends no User-Agent at all unless you set one.

Cloudflare’s WAF and Bot Fight Mode frequently block empty or suspicious User-Agents, producing errors such as:

  • Error 1010: browser signature banned
  • Error 1020: access denied

2. TLS fingerprint (JA3/JA4)

Cloudflare fingerprints the TLS handshake: cipher suite order, extensions, curve preferences, and more. curl typically uses OpenSSL or similar, while reqwest uses rustls or native-tls depending on the enabled features. The fingerprints differ, and some Cloudflare configurations score unfamiliar ones as bot-like.

3. HTTP/2 fingerprint and header ordering

Beyond TLS, Cloudflare can inspect HTTP/2 SETTINGS frames, header order, and pseudo-header order. These differ between curl (which uses nghttp2) and Rust’s hyper/h2 stack.

4. Header differences

curl sends a specific set of headers in a specific order. Your Rust client may omit Accept, send a different Accept-Encoding, or set Content-Type differently, any of which can trip a rule.

5. Environment differences

If the Rust program runs somewhere other than where you ran curl (a VPS, a container, CI, or a different network), IP reputation and rate limits may differ. That can surface as Error 1015 (rate limited) or a challenge page.

6. Redirect and auth handling

reqwest strips the Authorization header when following a redirect to a different host. If the API redirects, you may silently lose authentication.

Diagnosing the problem

Before changing anything, read the actual response rather than just the error:

let resp = client.get(url).send().await?;
println!("status: {}", resp.status());
println!("headers: {:#?}", resp.headers());
println!("body: {}", resp.text().await?);

The HTML body of a Cloudflare error page usually names the error code (1010, 1020, 1015, and so on), which tells you which of the causes above applies. It also helps to compare curl -v output against your Rust request to see which headers curl sends that you don’t.

The fix

The original code

My first version used reqwest::get, which creates a throwaway default client with no User-Agent:

let url = format!(
    "https://api.coingecko.com/api/v3/simple/price?ids=cardano,midnight-3,blockdag,tron,dogecoin,binancecoin,ethereum,tether,usd-coin&vs_currencies=btc&x_cg_demo_api_key={api_key}"
);
let resp = reqwest::get(&url).await?;
let data = resp.json::<PriceMultiResponse>().await?;
Ok(data)

Building a proper client

Build a Client with a User-Agent and an Accept header, then send the request through it:

use reqwest::header::{HeaderMap, HeaderValue, ACCEPT};

let url = format!(
    "https://api.coingecko.com/api/v3/simple/price?ids=cardano,midnight-3,blockdag,tron,dogecoin,binancecoin,ethereum,tether,usd-coin&vs_currencies=btc&x_cg_demo_api_key={api_key}"
);

let mut headers = HeaderMap::new();
headers.insert(ACCEPT, HeaderValue::from_static("application/json"));

let client = reqwest::Client::builder()
    .user_agent(concat!(env!("CARGO_PKG_NAME"), "/", env!("CARGO_PKG_VERSION")))
    .default_headers(headers)
    .build()?;

let resp = client.get(&url).send().await?;
let data = resp.json::<PriceMultiResponse>().await?;
Ok(data)

A few notes on this version:

  • concat!(env!("CARGO_PKG_NAME"), ...) builds a User-Agent like my-app/0.1.0 from your Cargo.toml. Any fixed string such as "my-app/1.0" works too.
  • Build the Client once and reuse it. Store it in a struct or pass it in rather than rebuilding it on every call. It pools connections internally.
  • If it still fails, add .error_for_status()? after .send().await?, or print resp.status() and the body before calling .json().

Bonus: keep the API key out of the URL

CoinGecko also accepts the demo key as a header (x-cg-demo-api-key) instead of a query parameter. That keeps the key out of URLs and logs, and .query() handles encoding for you:

let resp = client
    .get("https://api.coingecko.com/api/v3/simple/price")
    .query(&[
        ("ids", "cardano,midnight-3,blockdag,tron,dogecoin,binancecoin,ethereum,tether,usd-coin"),
        ("vs_currencies", "btc"),
    ])
    .header("x-cg-demo-api-key", &api_key)
    .send()
    .await?;

If it’s still blocked

Work through these in order:

  1. Set a User-Agent and Accept header. This fixes the problem most of the time.
  2. Switch TLS backends by toggling between rustls-tls and native-tls in your Cargo.toml features.
  3. Force HTTP/1.1 with .http1_only() on the builder, to rule out HTTP/2 fingerprinting.
  4. Run from the same machine and network as your successful curl test.
  5. Contact the API provider. Ask them to allowlist your client or tell you which Cloudflare rule is blocking it.

A word of caution on that last point: spoofing a browser’s fingerprint to evade bot protection is fragile and may violate the provider’s terms. An honest, identifiable User-Agent is the better long-term approach.

Takeaway

When “curl works but my code doesn’t,” the difference is rarely the URL or the key. It’s usually the invisible parts of the request: headers, TLS, and HTTP/2 behavior. Start by setting a proper User-Agent, read the real error body, and only go deeper if you need to.

Leave a Response