Issue with Bing Search API Key and Web Scraping in Python

Smith Jack 0 Reputation points
2023-08-23T11:56:02.2066667+00:00

Hi everyone,

I hope you're doing well. I am currently working on a Python script that involves web scraping and sentiment analysis. However, I've encountered an issue that I can't seem to resolve. I would greatly appreciate your help in troubleshooting this problem.

Here's a snippet of the Python code I'm using:


# ... (code before)

class WebScraper:
    def __init__(self):
        self.api_key = os.environ.get("Key 1") 
def scrape(self, source, word):         
    query = quote_plus(f"{word} site:{source}")          
    url = f"https://api.bing.microsoft.com/v7.0/search?q={query}"          
    headers = {"Key 2": self.api_key}          
    response = requests.get(url, headers=headers)          
    response.raise_for_status()         
    return response.json()

    # ... (other methods)

if __name__ == "__main__":
    words = ["A", "B", "C"]
    sources = ["twitter.com", "reddit.com"]
    main(words, sources)

The problem I'm facing is that when I run the script, I receive the following error message:

Error scraping data: 401 Client Error: PermissionDenied for url: https://api.bing.microsoft.com

I have a suspicion that the issue might be related to the API Key I'm using. I obtained the API Key from the "Keys and Endpoint" section of the "Bing Search" service. Is this the correct method to obtain the API Key? Or am I missing something?

Just to provide some context, I'm running this script on Debian 11, and there don't seem to be any network issues.

Any insights or suggestions you can provide would be greatly appreciated. Thank you in advance for your help!

Best regards,

Steven

Bing Bing Search APIs Bing Web Search API
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.