Data Scraping Studio (v 1.7)

Crawl password protected websites using CSS Selectors

3 followers

Crawl password protected websites using CSS Selectors

3 followers

Visit website

Password managers

This is the 2nd launch from Data Scraping Studio (v 1.7). View more

Online Web Scraper

Cloud hosted data extraction tool

Launch tags:

Web App•Developer Tools•Tech

Launch Team

Vikash Rathee

idm.in

Maker

Hello Product Hunters, I started DataScraping.co around a year back by launching the desktop app "Data Scraping Studio" on product hunt (https://www.producthunt.com/tech...) as a point and click CSS selector scraping app with amazing capability to run multiple agents simultaneously and over 5000 users downloaded the application and uses now in their business. Over the time we noticed, one of the major ask from community was the online solution and REST API. To integrate and use the scraped data in existing system without installing the desktop app or to use across operating systems. And we keep recommending our user to try (YC Backed) Kimonolabs, but then suddenly they shutdown their platform on March, 16 leaving 100k+ users leaving behind on short notice. Today, I'm very excited to launch the online web scraper with the same capabilities and power of cloud along with REST API to get the same Kimono(or even better) experience back. Go to our website and signup to get 1000 pages credit free every month to test the features and you can go PRO anytime if you like it! I’ll be available all the day to answer any comments, feedback you have, really looking forward to it!.

Report

9yr ago

O O

@vikashrathee IMO scheduled, cloud scrapers are the way to go! Was a BIG fan of Kimono, but what did it for me with them was the ability to get an RSS feed from my extraction and a super reliable Google Sheets Add-on that would update when the scraper would run. Still can't find anything similar! Do you/ will you implement something similar? 😁

Report

9yr ago

Vikash Rathee

idm.in

Maker

@sandrojazzar Hey Sandro, thank you for the comment. Yes we do have RSS feed in plan and will be adding that in coming weeks. Google spreadsheet is not the priority right now but yes we will see this if more user requests the same. For now there is an alternative using the Zapier can post the data to Google Spreadsheet from webhook (we do support webhook). You can give a try. https://zapier.com/zapbook/zaps/...

Report

9yr ago

mp8

@sandrojazzar have you tried Scrapinghub.com they are they have a lot of the same features as kimono including scheduled runs.

Report

9yr ago

Vikash Rathee

idm.in

Maker

@marcelp81 @sandrojazzar No they don't have that to automate I guess. What they explained is manual import each time in this article. https://support.scrapinghub.com/... So better would be to use Zapier to post the data automatically in spreadsheet each time the scraping job completes using any tool of your choice.

Report

9yr ago

Kieran Daniels

@vikashrathee Do you have anyone or and example of someone scraping Zillow / Realtor.com for buyer/seller leads? Thanks!

Report

9yr ago

Nicola Mastrandrea

Why should I prefer OWS to Import.io?

Report

9yr ago

Vikash Rathee

idm.in

Maker

@nick88msn - Save money : Import.io costs $299 monthly for 50k pages while datascraping.co charges $130 only. - Unlimited agents : Import.io support only 30 runs storage, while in datascraping.co you can create any number of agents and upto 30 days historical data for any number of jobs run during 30 days. - Sub accounts : Create sub account for your team to share the same account with role based security. - Crawl with Limit : Advance limit option to crawl data in parts instead all in one go without modifying the input lists. - Modify data using JavaScript - Currently available for desktop app, will be added for hosted app soon (https://www.datascraping.co/doc/...) - Dynamic input : Extract many website using a single agent. (https://www.datascraping.co/doc/...) - Password protect site crawling : Easy form submission to crawl password protected websites (https://www.datascraping.co/doc/...) And lots more...

Report

9yr ago

Nicola Mastrandrea

@vikashrathee thank you

Report

9yr ago

Hadi Farnoud

Retime

unfortunately it doesn't work for me. I've tried to use it on https://web.archive.org/web/2016... to extract data from a lost site. data is inside a div that appears on hover over images. my agent is 20fce2dc52

Report

9yr ago

Vikash Rathee

idm.in

Maker

@hadifarnoud Hi Hadi, I just copied your agent in admin account to test and found that, the data was extracted but was not visible in table due to space and line breaks. I just enabled the clean, trim function in your account and you are good to re-run your agent now. Here is the screenshot I tested this in my account. And your question is posted here to help others to understand what went wrong. (https://www.datascraping.co/doc/...)