リポジトリの原文
見出し、例、コード、表、リンク、参照画像を含む原文を表示しています。
Scraping Twitter Profiles
Bulk profile data export for any set of Twitter/X usernames. Returns account-level metadata: follower counts, bios, join dates, verification status.
Prerequisites
APIFY_TOKENenvironment variable set- Optional: Apify MCP server installed
Inputs
| Parameter | Type | Required | Default | Notes |
|---|---|---|---|---|
startUrls | array | Optional | [] | Twitter profile or tweet URLs |
twitterHandles | array | Optional | [] | Twitter usernames (without @) |
twitterUserIds | array | Optional | [] | Twitter user IDs |
getFollowers | boolean | Optional | false | Extract follower lists |
getFollowing | boolean | Optional | false | Extract following lists |
getRetweeters | boolean | Optional | false | Extract retweeters of a tweet URL |
includeUnavailableUsers | boolean | Optional | false | Include unavailable/suspended users |
maxItems | number | Optional | Unlimited | Maximum users to return |
customMapFunction | string | Optional | — | JavaScript function to transform each output object |
Workflow
Progress:
- [ ] Step 1: Normalize username list
- [ ] Step 2: Run twitter-user-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Deliver profile datasetStep 1: Normalize
Remove @ symbols, deduplicate, validate no empty strings.
Step 2: Run the Actor
Recommended — run_actor.js (handles waiting, output, and file saving automatically):
bash
# Quick answer (prints table to chat)
node scripts/run_actor.js \
--actor "apidojo~twitter-user-scraper" \
--input '{"param": "value"}'
# Save as CSV
node scripts/run_actor.js \
--actor "apidojo~twitter-user-scraper" \
--input '{"param": "value"}' \
--output YYYY-MM-DD_results.csv --format csv
# Save as JSON
node scripts/run_actor.js \
--actor "apidojo~twitter-user-scraper" \
--input '{"param": "value"}' \
--output YYYY-MM-DD_results.json --format jsonAPIFY_TOKENmust be set in environment or.envfile.
If Apify MCP is available:
Tool: apify:run-actor
Actor: "apidojo~twitter-user-scraper"
Input:
{
"usernames": ["handle1", "handle2", "handle3"]
}REST API fallback:
bash
curl -X POST \
"https://api.apify.com/v2/acts/apidojo~twitter-user-scraper/runs?token=$APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"usernames": ["handle1", "handle2"]}'Save id as RUN_ID. Poll until status = SUCCEEDED:
bash
curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'Fetch results:
bash
curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"Step 3: Handle Edge Cases
- Missing accounts in results: Account suspended, deleted, or handle changed. Note missing handles in output.
- Private accounts: Returns profile metadata but no tweet content — flag
isPrivate: truerows. - Deactivated accounts: Returns null or empty object — exclude from final dataset.
Output Format
# Twitter Profile Dataset
Accounts requested: N | Accounts returned: N | Missing: N
| Username | Display Name | Followers | Following | Tweets | Verified | Bio (truncated) | Joined |
|----------|-------------|-----------|-----------|--------|----------|-----------------|--------|
| ... | ... | ... | ... | ... | ... | ... | ... |
Available fields: username, displayName, followersCount, followingCount, tweetCount,
isVerified, isBlueVerified, description, location, url, profileImageUrl, createdAt, isPrivateTroubleshooting
Some handles missing from results: Accounts may be suspended or renamed — cross-reference manually. Large lists (> 1000): Break into batches of 500 and run sequentially.
