This is the 6th launch from Data Scraping Studio. View more
Puppeteer API
Convert your website into an API
Introducing our new integration of Puppeteer with scraping agent to allow users to write custom puppeteer scripts in Node.js backed by Agenty cloud for performance and scaling.
Hi Product Hunters!
I will be live today for discussions, please ask question or share your feedback. I will try to respond all questions and plan the improvements in next version, if you have any feedback.
@chrismessina Just write a puppeteer script and use that to send POST request to https://chrome.agenty.com/function API to turn the website into API.
For example, here is an example code to navigate to site > extract 4 fields > return JSON
const fetch = require('node-fetch');
const jsonBody = {"code" : "// Read the `url` from request, goto the page, extract products\n// capture screenshot and return the results\n\nmodule.exports = async ({ page, request }) => {\n const response = await page.goto(request.url);\n console.log(response.status());\n const result = await page.evaluate(() => {\n const data = [];\n var products = document.querySelectorAll('.product_pod');\n for (var product of products) {\n data.push({\n product_name: product.querySelector('h3').textContent,\n product_price: product.querySelector('.price_color').textContent,\n product_availability: product.querySelector('.availability').textContent,\n product_image: \"http://books.toscrape.com\" + product.querySelector('.thumbnail').getAttribute(\"src\"),\n product_link: \"http://books.toscrape.com\" + product.querySelector('h3 > a').getAttribute(\"href\")\n });\n }\n return data;\n });\n\n await page.screenshot({ path: 'books-toscrape.png', fullPage: true });\n\n return {\n data: result,\n type: 'application/json'\n };\n};" , "request" : {"url":"http://books.toscrape.com/"} };
fetch('https://chrome.agenty.com/functi...', {
method: 'post',
body: jsonBody,
headers: { 'Content-Type': 'application/json' },
})
.then(res => res.json())
.then(json => console.log(json));
See the postman collection here - https://github.com/Agenty/agenty... or example scripts in sidebar.
idm.in
idm.in
idm.in
Lifyzer
idm.in