Sign in

Create a Crawl

documented

Create and start a crawl job for a site.

POST /v3/crawlCrawlno implementation yet0 runs
No implementation yet. Run it to have the agent write one, or open a pull request with your own.

Seed example (keychains curl)

npx -y keychains@latest curl "https://api.diffbot.com/v3/crawl" \
  -X POST \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -H "Accept: application/json" \
  --data-urlencode 'token={{DIFFBOT_API_KEY}}' \
  --data-urlencode 'name={{$crawl_name}}' \
  --data-urlencode 'seeds={{$seed_url}}' \
  --data-urlencode 'apiUrl={{$api_url}}'