Issue with Bing Search API Key and Web Scraping in Python
Hi everyone,
I hope you're doing well. I am currently working on a Python script that involves web scraping and sentiment analysis. However, I've encountered an issue that I can't seem to resolve. I would greatly appreciate your help in troubleshooting this problem.
Here's a snippet of the Python code I'm using:
# ... (code before)
class WebScraper:
def __init__(self):
self.api_key = os.environ.get("Key 1")
def scrape(self, source, word):
query = quote_plus(f"{word} site:{source}")
url = f"https://api.bing.microsoft.com/v7.0/search?q={query}"
headers = {"Key 2": self.api_key}
response = requests.get(url, headers=headers)
response.raise_for_status()
return response.json()
# ... (other methods)
if __name__ == "__main__":
words = ["A", "B", "C"]
sources = ["twitter.com", "reddit.com"]
main(words, sources)
The problem I'm facing is that when I run the script, I receive the following error message:
Error scraping data: 401 Client Error: PermissionDenied for url: https://api.bing.microsoft.com
I have a suspicion that the issue might be related to the API Key I'm using. I obtained the API Key from the "Keys and Endpoint" section of the "Bing Search" service. Is this the correct method to obtain the API Key? Or am I missing something?
Just to provide some context, I'm running this script on Debian 11, and there don't seem to be any network issues.
Any insights or suggestions you can provide would be greatly appreciated. Thank you in advance for your help!
Best regards,
Steven