Greetings everyone,

Well Im getting sucked back into the WebDev world for work and been tasked with a few things and was hit with a recent questions on a recommended company that scrapes common websites like craigslist and other common sites. From my understanding the companies that do this type of work, I would expect that they would already have this data on hand like a warehouse of sort. Any pointers would be greatly appreciated. I wouldn't mind working with a US or India company that handle these types of request but any recommendations would be very helpful.

Thank you!!

Web scraping is treading in murky legal waters - yes you can technically "scrape" content from other web sites but make sure you have permission from the site owners to do so. A better approach would be to use API's from whatever company/site you want to use data from.

So I wanted to come back and post here. It seems there is a legal check you need to do, then collect the data. Doing scraping may be an issue so you may have to get good with Proxychains as a website may limit how many post you can read over time to prevent getting scraped.