Microsoft Speech Recognition

Question

Microsoft Speech Recognition

Starman 0

When I run my code, the Azure speech recognition API is canceled abruptly before any text is recognized. Another user has posted this problem before. I'm not sure if this is a firewall thing or not; I'm very new to using Azure and its speech technology. The API through the system environment variable should work and I'm on a stable connection. Maybe I have to set something up through Speech Azure? I have a school and personal account so does the API have to match up with whatever account I use on VS? I'm lost. There is a lot of new for me here so sorry if I am misinformed when answering back questions. [system is windows 10 / azure-cognitiveservices-speech is installed / i have not exceeded any audio tiers thru azure]

CLOSING speech recognition on SpeechRecognitionCanceledEventArgs(session_id=5cfec5e54e3f4778a7fff1ebc3fc9107, result=SpeechRecognitionResult(result_id=53718a3138524df494f543c3e4f2ca14, text="", reason=ResultReason.Canceled)) CLOSING speech recognition on SessionEventArgs(session_id=5cfec5e54e3f4778a7fff1ebc3fc9107)

navba-MSFT 27,605 Reputation points Microsoft Employee Moderator

2024-02-14T03:04:24.4666667+00:00

@Starman Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

The cancellation could be for multiple reasons platform, file format or incorrect speech resource configuration.

Plan 1:

Before testing your application from Visual Studio, Could you please test from the Speech Studio and check if you face the same issue ?

You can leverage both speech-to-text and text-to-speech functionalities depending on your scenario:

Plan 2:

If this works fine, you can test using the sample code provided here:

Quickstart: Recognize and convert speech to text

Quickstart: Convert text to speech

Plan 3:

Also ensure that you are using the most recent version of the Cognitive Speech SDK.

Plan 4:
Please share your sample code here and I can test the same at my end and debug this. Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

Starman 0

Thank you for responding so fast--I really do appreciate it!

Plan 1: Works fine in here, has previously too.

Plan 2: I will test this and see if it works--but it should.

Plan 3: Sure am! Plan 4:

It's actually based on code written here (w/o websockets). The code has been shown to work before online, which is why I think it's a firewall thing(?). https://github.com/DougDougGithub/Babagaboosh Presuming you used your own APIs and set ALL that up entirely the "problem" would be in chatgpt_character.py:

print("[green]Starting the loop, press F4 to begin")
while True:
    # Wait until user presses "f4" key
    if keyboard.read_key() != "f4":
        time.sleep(0.1)
        continue

    print("[green]User pressed F4 key! Now listening to your microphone:")

    # Get question from mic
    mic_result = speechtotext_manager.speechtotext_from_mic_continuous()
    
    if mic_result == '':
        print("[red]Did not receive any input from your microphone!")
        continue

    # Send question to OpenAi
    openai_result = openai_manager.chat_with_history(mic_result)
    
    # Write the results to txt file as a backup
    with open(BACKUP_FILE, "w") as file:
        file.write(str(openai_manager.chat_history))

    # Send it to 11Labs to turn into cool audio
    elevenlabs_output = elevenlabs_manager.text_to_audio(openai_result, ELEVENLABS_VOICE, False)

    # Play the mp3 file
    audio_manager.play_audio(elevenlabs_output, True, True, True)


    print("[green]\n!!!!!!!\nFINISHED PROCESSING DIALOGUE.\nREADY FOR NEXT INPUT\n!!!!!!!\n")

Starman 0 Reputation points

2024-02-14T06:14:32.1666667+00:00

(Unfortunately I do not believe my questions will be answered there, which is why I went here.) Firewall didn't seem to work--I used the IP for US EAST Azure and it didn't work, but I could have done it wrong! I'm not sure what would be causing it period as the code (which I can modify with guidance) should work and it definitely seems to be an Azure end thing. Do I need to change something their possibly? As soon as I hit F4 it hits me with that without my input but the code itself functions or otherwise I wouldn't be at that point even. I'm not sure where to jump from here is what I'm saying. Lotta great help I will be checking on those links. (I also don't know much about proxy in regards to what I'm doing)
navba-MSFT 27,605 Reputation points Microsoft Employee Moderator

2024-02-14T06:58:23.2533333+00:00

@Starman Thanks for your reply. Please keep me posted about the updates on my above suggestions.

1 answer

Your answer

navba-MSFT 27,605 Reputation points Microsoft Employee Moderator

2024-02-14T03:04:24.4666667+00:00

@Starman Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

The cancellation could be for multiple reasons platform, file format or incorrect speech resource configuration.

Plan 1:

Before testing your application from Visual Studio, Could you please test from the Speech Studio and check if you face the same issue ?

You can leverage both speech-to-text and text-to-speech functionalities depending on your scenario:

Plan 2:

If this works fine, you can test using the sample code provided here:

Quickstart: Recognize and convert speech to text

Quickstart: Convert text to speech

Plan 3:

Also ensure that you are using the most recent version of the Cognitive Speech SDK.

Plan 4:
Please share your sample code here and I can test the same at my end and debug this. Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.
Starman 0 Reputation points

2024-02-14T05:47:29.85+00:00

Thank you for responding so fast--I really do appreciate it!

Plan 1: Works fine in here, has previously too.

Plan 2: I will test this and see if it works--but it should.

Plan 3: Sure am! Plan 4:

It's actually based on code written here (w/o websockets). The code has been shown to work before online, which is why I think it's a firewall thing(?). https://github.com/DougDougGithub/Babagaboosh Presuming you used your own APIs and set ALL that up entirely the "problem" would be in chatgpt_character.py:

print("[green]Starting the loop, press F4 to begin") while True: # Wait until user presses "f4" key if keyboard.read_key() != "f4": time.sleep(0.1) continue print("[green]User pressed F4 key! Now listening to your microphone:") # Get question from mic mic_result = speechtotext_manager.speechtotext_from_mic_continuous() if mic_result == '': print("[red]Did not receive any input from your microphone!") continue # Send question to OpenAi openai_result = openai_manager.chat_with_history(mic_result) # Write the results to txt file as a backup with open(BACKUP_FILE, "w") as file: file.write(str(openai_manager.chat_history)) # Send it to 11Labs to turn into cool audio elevenlabs_output = elevenlabs_manager.text_to_audio(openai_result, ELEVENLABS_VOICE, False) # Play the mp3 file audio_manager.play_audio(elevenlabs_output, True, True, True) print("[green]\n!!!!!!!\nFINISHED PROCESSING DIALOGUE.\nREADY FOR NEXT INPUT\n!!!!!!!\n")
Starman 0 Reputation points

2024-02-14T06:14:32.1666667+00:00

(Unfortunately I do not believe my questions will be answered there, which is why I went here.) Firewall didn't seem to work--I used the IP for US EAST Azure and it didn't work, but I could have done it wrong! I'm not sure what would be causing it period as the code (which I can modify with guidance) should work and it definitely seems to be an Azure end thing. Do I need to change something their possibly? As soon as I hit F4 it hits me with that without my input but the code itself functions or otherwise I wouldn't be at that point even. I'm not sure where to jump from here is what I'm saying. Lotta great help I will be checking on those links. (I also don't know much about proxy in regards to what I'm doing)
navba-MSFT 27,605 Reputation points Microsoft Employee Moderator

2024-02-14T06:58:23.2533333+00:00

@Starman Thanks for your reply. Please keep me posted about the updates on my above suggestions.

Answer 1

@Starman Thanks for getting back.

Since the requirement here is to convert the speech to text while taking the input from microphone, You can leverage the sample provided here:

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/get-started-speech-to-text?tabs=linux%2Cterminal&pivots=programming-language-python#recognize-speech-from-a-microphone

Please refer the function recognize_from_microphone().

Also note that this github code which you shared https://github.com/DougDougGithub/Babagaboosh is open source, so if you are facing any issues, you should report it under its issue section.

If you feel its related to the proxy / firewall, as an isolation step, try to bypass it or test your application from a different network and see if that works.

If you cannot bypass your proxy, then you can use the SpeechConfig SetProxy() call and add the proxy details in your application:
https://learn.microsoft.com/en-us/cpp/cognitive-services/speech/speechconfig#setproxy

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

Share via

Microsoft Speech Recognition

1 answer

Your answer