Spinneret: Record and automate anything on the web

Modash Influencer Search

Super interesting project. What do you think for you, would be the most interesting way people could use this?

5yr ago

@avery_schrader I think there is a lot of use-cases when we want to automate a web flow or process no? Especially when you need to have access to external sets of data without being connected.

Report

5yr ago

Zac Denham

RipDB

Maker

Hi @avery_schrader, thanks for asking! There are a bunch of use cases, three ones I think are particularly interesting are: 1. Bot Services - Spinneret lets you build an API on top of other websites without having to code, so you can easily spin up some pretty funky bots to automate repetitive web tasks with dynamic inputs Some fun bot ideas I had off the top of my head: - DMV Appointment scheduler -> automatically fills out appointment booking on DMV website based off your schedule. Checks every X minutes for new availability. - Resume Blast Bot - Fill in your resume/LinkedIn once, Spinneret simultaneously fills it out on 100 company sites you are interested in applying to. Excited to see what else the maker community comes up with! 2. Advanced Web Scraping - a lot of useful data is behind site login, pagination, button clicks and form fills. Especially for sites that are server side rendered, and API reversing isn't trivial. You can use spinneret to quickly spin up web scraping tracks (right click to extract & label text!), and get structured data from web pages at scale. This can be used for e-commerce price comparisons, growth hacking, sales insights. 3. QA Testing - spinneret could be used as a no-code alternative to automated UI testing software like selenium or puppeteer. Create regression tests for your website just by recording tracks in the browser. I include this one last because I think it would need a couple more features (assertions, editing tracks) before being a truly robust QA solution, but interesting nonetheless!

Report

5yr ago

Irina M

@avery_schrader @zachary_denham OMG someone finally read my mind!! Ive been begging for someone to come out with exactly this! Having profound hearing loss all my life, communication is a completely different survival skill than what it might mean for regular hearing people. For example. an I-phone for me, or a smart phone is a durable medical device - my heаring aids stream via bluetooth and are ios/andriod compatible. I was also recently implanted with a cochlear implant that is also iOs/andriod compatible. So yes I even went under the knife to get an iOs/android compatible latest and greatest technology in medicine so far... For me getting a new iphone every year is not exciting, it doesn;t appeal to me that I have to not only condigure all the devices, accessories microphones and lovely gadgets to the latest generation, but I am also constantly at risk with skipping an iOS system upgrade or if I dont keep up with the constant debiliting fear of something going wrong during an update, or a device phasing out without any forewarning causing me to sit in all day troubleshooting things, I never knew could shot and cause trouble beyond belief. Also don't forget everything needs to be documented, and try to keep up with it all technology moves fast, imagine what it does to me... Im still syncing my last iOS update to my 22 electrodes implanted in my head....for example I would absolutely love it if my phone rang directly to my MacBook when I am near it, once I answer a call it must go through a series of steps depending on what "TTY" option I use depending if I am answering my phone, speaking handsfree using my HAs or doing a conference call and which prompts the captioner immediately or uses AI closed captioning. Theres so many ways I could go with this, my ideas are endless. Except they aren't ideas, i need this.... my bionic ears Beyond grief, if this is loosing it at any minute. Every single behavior

Report

5yr ago

Irina M

@zachary_denham I have to try this out someday, i have a very specific sequence that I have sorta wrote on paper, but I need someone to help me build it as a workflow. Is this self explanatory or does it require more trainijng?

Report

5yr ago

Avery Schrader

Modash Influencer Search

@phb For sure, I like poking at the makers inner thought process though, sometimes that's even more interesting than the tool itself. Also, I'm not all that clever, so having a couple use cases helps spur my imagine.

Report

5yr ago

Zac Denham

RipDB

Maker

Hello! Stoked to be launching this project on PH!! Some details on spinneret: - It works by recording & saving DOM events as you interact with web pages in the browser - Each track has an API endpoint you can invoke to re-run the track programmatically - Tracks spin up a headless browser and recreate events with dynamic inputs This is the first version of the app and is pretty simplistic, future releases will include: - Automatic IP proxying - AI-powered site-change resistance - CSV Upload inputs to create a track - Chrome extension?

Report

5yr ago

Hans Dekker

Lyne

@zachary_denham would it work behind a login such as LinkedIn Sales Nav?

Report

5yr ago

Zac Denham

RipDB

Maker

@hans_dekker The goal is to support all websites, and simulate realistic human behavior on the web, so the algorithm to locate the right page elements and perform actions on them is always under improvement. I haven't tried running any tracks behind LinkedIn login, and I know they put quite a bit of effort into bot prevention. That being said, its definitely worth a shot!

Report

5yr ago

Hans Dekker

Lyne

@zachary_denham thanks! Final question: what about scraping sites such as Clutch? In order to get the right URL of companies I have to click the button linked as clutch.co?redirect_xyz and then copy the actual company url from my Chrome bar. Would that work? If so, I'm signing up for life ;)

Report

5yr ago

Francoo

@zachary_denham @hans_dekker also interested in behing login

Report

5yr ago

Zac Denham

RipDB

Maker

@hans_dekker this feature is not currently implemented, but is an easy add! If I’m understanding correctly, you are looking for an output of the site URL after performing some click actions, correct?

Report

5yr ago

Alex Pareto

MealsAI

Is there any API documentation btw? I created a track to save a Spotify track to a playlist - but I don’t see how to run it from the API

Report

5yr ago

Zac Denham

RipDB

Maker

@alex_pareto1 excellent question! There is no API documentation currently, but if you navigate in the spinneret browser to: tracks > run in cloud > snippets tab You can copy a code snippet with an example network request to the spinneret API in your language of choice (node, go & python are supported out of the box) and invoke the track programmatically. Also, don't forget to label your inputs if your track has text inputs, then you can post dynamic input in the request body!

Report

5yr ago

Spinneret

Record and automate anything on the web

Do you use Spinneret?

Review Spinneret?

Reviews of Spinneret