web scraping with python

I’ve recently run into problems scraping the new google site. Anyone else?

Comments

[Akash Jeez]

Scrapping giant websites like Google, Amazon, AWS, Apple etc will work for short time but not for long run because if admin added more security into the site.
Better to look for API which can get the same data as you trying to scrape from the site.
for Google Search -> pypi.org/project/googlesearch-python/

[Jonas Johnson]

Without checking I can’t be completely certain but I believe I was using this and ran into some issues where I had to rewrite this to get the specifics that I needed. I will look into it again though, thanks!

[nfkt]

Why scrape google

[Jonas Johnson]

I’m scraping the results to get more info about the websites based on a specific query.