Prospect demo — ScrapeNinja (unofficial)
POST/scrape-js

Scrape a page with JavaScript rendering

Scrapes a URL with a real Chrome browser engine to render JavaScript-dependent pages. Use this endpoint when /scrape does not retrieve the required page content; set waitForSelector or postWaitTime when you need to control when page loading is considered complete. The response includes the rendered HTML and browser metadata.

  • IdempotentThe SDK sends Idempotency-Key, so a retried request is only applied once.

19 body fields

JSON request containing the URL to scrape and optional browser rendering, request, proxy, and retry settings.

urlstringrequired
URL to scrape
waitForSelectorstringoptional
CSS selector to wait to appear in DOM tree before page is considered as loaded.
postWaitTimeintegeroptional
Wait for specified amount of seconds after page load (from 1 to 12s). Use this only if ScrapeNinja failed to wait for required page elements automatically.
dumpIframestringoptional
If some particular iframe needs to be dumped, specify its `name` HTML value in this argument. The ScrapeNinja JS renderer will wait for <iframe name="icims_content_iframe"> to appear in DOM, then use `waitForSelectorIframe` CSS selector to wait for iframe DOM elements to appear inside.
waitForSelectorIframestringoptional
If `dumpIframe` is activated, this property allows to wait for CSS selector inside this iframe.
extractorTargetIframebooleanoptional
If `dumpIframe` is activated, this property allows to run JS extractor function against iframe HTML instead of running it against base body. This is only useful if `dumpIframe` is activated.
headersarray<string>optional
Custom headers to send with the request. By default, regular Chrome browser headers are sent to the target URL.
retryNumintegeroptional
Amount of attempts.
geostringoptional
Geo location for basic proxy pools (you can purchase premium ScrapeNinja proxies for wider country selection and higher proxy quality). [Read more about ScrapeNinja proxy setup](https://scrapeninja.net/docs/proxy-setup/)
Default:us
proxystringoptional
Premium or your own proxy URL (overrides `geo` field). [Read more about ScrapeNinja proxy setup](https://scrapeninja.net/docs/proxy-setup/)
timeoutintegeroptional
Timeout per attempt, in seconds. Each retry will take [timeout] number of seconds.
Default:16
textNotExpectedarray<string>optional
Text which will trigger a retry from another proxy address.
statusNotExpectedarray<integer>optional
HTTP response statuses which will trigger a retry from another proxy address.
Default:[403,502]
blockImagesbooleanoptional
Block images from loading. This will speed up page loading and reduce bandwidth usage.
blockMediabooleanoptional
Block (CSS, fonts) from loading. This will speed up page loading and reduce bandwidth usage.
screenshotbooleanoptional
Take a screenshot of the page. Pass "false" to increase the speed of the request.
catchAjaxHeadersUrlMaskstringoptional
Useful to dump some XHR response. Pass URL mask here. For example, if you need to catch all requests to https://example.com/api/data.json, pass "api/data.json" here. In response, you will get new property `.info.catchedAjax` with the XHR response data - { url, method, headers[], body , status, responseHeaders{} }
viewportobjectoptional
Advanced. Set custom viewport size. By default, viewport size is 1920x1080.
extractorstringoptional
Custom JS function to extract JSON values from scraped HTML. Write&test your own extractor on https://scrapeninja.net/cheerio-sandbox/

1 status code
200Returns the rendered HTML body and an `info` object containing response metadata, including the status code, final URL, headers, screenshot, and page cookies.
infoobjectoptional
bodystringoptional
HTML body of the rendered page.

Error handling

url is required and must identify the page to scrape. postWaitTime, when supplied, must be between 1 and 12 seconds; waitForSelectorIframe and extractorTargetIframe are useful only when dumpIframe is activated.