Inference API Abuse Prevention

First Principle

LLM inference is expensive to generate and cheap to abuse. Every unenforced limit is a subsidy for attackers.

A single inference request can cost orders of magnitude more than a traditional API call. Rate limits keyed only on API key are bypassed by credential stuffing or key rotation. Quota systems that track only request count miss cost-based attacks that maximize token consumption per request. Abuse that would cost an attacker $1 to perform can cost the provider $100 to serve.

Attack Mental Model

1. Credential stuffing — leaked API keys from public GitHub repos, pastebins, or prior breaches are tested in bulk. Valid keys are consumed to exhaustion or sold.
2. Key rotation bypass — an attacker with access to a paid account generates many virtual keys, distributes them across IPs, and exceeds aggregate quota while staying under per-key limits.
3. Quota arbitrage — an attacker with a free-tier key uses max_tokens manipulation to maximize output per request, burning disproportionate GPU resources against a fixed token quota.
4. Account takeover via prompt exfiltration — system prompts containing session tokens, API keys, or configuration secrets are exfiltrated via prompt injection, enabling account takeover without credential theft.

Control Lens

Principle	What It Means Here
Validate	Every API key is validated against a current revocation list on each request — not just at authentication time.
Scope	Rate limits are multi-dimensional: per-key, per-account, per-IP, per-organization, and per token budget — all enforced at a shared counter store.
Isolate	Credential stuffing attempts are isolated to an authentication pre-check that does not reach inference infrastructure, preventing probe-based key validation at inference cost.
Enforce	Anomalous usage patterns trigger automated key suspension with human review queue, not just throttling.

IAP.1 Credential Abuse Detection and Key Hygiene

The core vulnerability: API keys distributed to users eventually leak. Without rotation enforcement, revocation speed, and stuffing detection, a leaked key remains valid indefinitely and can be used at full quota.

Check

• Is there automated scanning of public code repositories (GitHub, GitLab, npm) for leaked API keys using your key format signature?
• When a leaked key is detected, what is the revocation latency — minutes or hours?
• Does the authentication layer check a revocation list on every request, or only at session start?

Action

• Scan for leaked keys continuously. Register your API key prefix pattern with GitHub Secret Scanning and GitGuardian. Implement a webhook that triggers immediate revocation when a key is detected.

# Webhook handler for GitHub secret scanning alerts
def handle_secret_alert(alert: dict):
    key_value = alert["secret"]
    key_record = lookup_key(key_value)
    if key_record:
        revoke_key(key_record.id, reason="public_leak_detected")
        notify_owner(key_record.owner_email, event="key_revoked_leak")
        log_security_event("key_leaked", key_id=key_record.id, source="github_scanning")

• Enforce mandatory key rotation. Keys older than 90 days generate a deprecation warning; keys older than 180 days are suspended pending owner confirmation. This limits the window of exposure for undetected leaks.
• Detect credential stuffing at the authentication layer — not the inference layer. A high rate of authentication failures against distinct key values from a single IP or ASN is a stuffing signal. Block at auth, not after compute is spent.

Failure Modes

• A developer commits an API key to a public repository. The key is scraped within minutes by automated tooling and consumed to quota exhaustion before the developer notices.
• Key revocation updates a database record but the inference layer caches validated keys in memory for 5 minutes. A revoked key continues to work during the cache TTL.

IAP.2 Rate Limiting and Quota Enforcement

The core vulnerability: Rate limits that measure only request count miss cost-based attacks. A system that allows 60 requests/minute with uncapped max_tokens can be driven to generate 60 × 4096 tokens per minute — identical request count, 8× the compute.

Check

• Is quota tracked in token units (prompt + completion tokens), not just request count?
• Are rate limit counters stored in a shared distributed store (Redis, DynamoDB) so horizontal scaling cannot be exploited for per-instance limit bypass?
• Is there a cost anomaly detector that flags accounts whose cost-per-request significantly exceeds the account tier's expected profile?

Action

• Track and limit by token budget, not request count:

class TokenBudgetEnforcer:
    def __init__(self, redis_client, limits: dict):
        self.redis = redis_client
        self.limits = limits  # {"per_minute": 50000, "per_day": 2_000_000}

    def check_and_consume(self, account_id: str, estimated_tokens: int) -> bool:
        key_minute = f"budget:{account_id}:minute:{current_minute()}"
        key_day = f"budget:{account_id}:day:{current_day()}"

        pipe = self.redis.pipeline()
        pipe.incrby(key_minute, estimated_tokens)
        pipe.expire(key_minute, 60)
        pipe.incrby(key_day, estimated_tokens)
        pipe.expire(key_day, 86400)
        results = pipe.execute()

        return (results[0] <= self.limits["per_minute"]
                and results[2] <= self.limits["per_day"])

• Implement multi-dimensional rate limits. Per-key limits prevent single-key abuse; per-account limits prevent key proliferation bypass; per-IP limits catch distributed credential attacks; per-org limits catch insider key sharing.
• Apply cost anomaly detection. Compute a rolling cost-per-request baseline per account tier. Accounts exceeding 3× their tier baseline trigger an automatic hold with owner notification.

Minimum Deliverable Per Review

• Leaked key scanning: automated detection with webhook-triggered revocation
• Revocation propagation: max 60-second lag from revocation event to enforcement at inference layer
• Token-based quota: request quota tracked in token units with per-minute and per-day limits
• Multi-dimensional rate limits: per-key, per-account, per-IP, per-org counters in shared store
• Stuffing detection: authentication failure rate threshold with IP/ASN block at auth layer
• Cost anomaly alerting: account cost-per-request vs. tier baseline monitoring

Quick Win

Add token-based quota tracking alongside request-count limits today. If you only track requests, an attacker with a valid key can maximize max_tokens to extract disproportionate compute. Track prompt_tokens + completion_tokens per account per window and enforce a hard cap.

References

• Model serving layer controls → model-serving-security/SKILL.md
• LiteLLM proxy rate limiting → litellm-proxy-security/SKILL.md
• Severity wording → severity-and-reporting.md