If you needed to scrape +100,000 LinkedIn profiles, where would you start? 

Good old Python has gotten me a list of URLs — but I need some actual profile data.

@day---break, day---break.com // nf.td/iam

DX Engineer | Head of Developer Relations at /pinata | Building orbiter.host | stevedylan.dev/links | opinions are my own

probs LinkedIn's servers 

ur welcome 🫡

Lover of random gems, greenery, nature, pets & flowers. Fueled by good music, clean eats, strong health & Web3 dreams

Essentially just the CV. Work history + education. 

Not entirely sure what the best path will be. Phantom Buster, Full Contact, etc have been floated as solutions. Trying avoid raw scraping so we don't burn anyones LN credentials on the team.

Nice work getting the URLs! Are you thinking of going headless browser with Selenium or stealthier with something like Puppeteer + proxies? What kind of profile data are you after?

CEO at Google • co-founder & rectangle mover at /icebreaker • icebreaker.xyz/j4ck • /j4ck • DJ sixsix

Identity and math

Co-founder at @icebreaker

icbk.me/web3pm.fc

this works pretty well but slow https://scrapingant.com/