Who knows how to set up this web scraping tool properly? | C2C Community

Who knows how to set up this web scraping tool properly?

  • 11 August 2021
  • 1 reply

Userlevel 7
Badge +26

Hi everyone! I came across this free API tool to scrape the internet for data that is of interest to me...and I am completely failing at setting it up and getting the data to roll in.

Does anyone know which parts of this form are necessary to fill out to receive good data?

Thanks a ton!

1 reply

Userlevel 7
Badge +24

Hi Marissa,

When someone wants to scrap the web, it usually faces some common problems, like web pages that do not want to be scraped and blocks you, CAPTCHAS systems that do not let you pass, anti-scraping systems, and other things. I think this program, "WebscrapingAPI", lets you avoid these types of problems and always get to the web page regarding the anti-scraping system. Also helping if you want to scrap thousands of pages simultaneously.

But, once you got the data, you still need to use a scraping tool to get just the data you need, which is not an easy task. You need a developer to do that. I think this tool will help you get the raw data of a web page but will not present it nicely ordered for you.

If the web page you are trying to scrap is straightforward in terms of HTML composition (i.e., just a table), I could help, but you will need someone with scrapping experience if it is more complicated.

That's what I think after looking at the webpage and docs at https://www.webscrapingapi.com/, but if anyone has a better understanding, please let us know.