this.connect error from microsoft pronunciation assessment sdk

Question

this.connect error from microsoft pronunciation assessment sdk

Hyojae Kim 5

I am using Azure's pronunciation assessment sdk for a service I am developing. I am currently saving audio files in lambda's tmp folder for ephemeral access. My code works fine on my local environment (microsoft can access my local tmp folder just fine), but I receive this error when I deploy it to the dev environment. This is my code and error:
My logs look like this:

    let audioConfig
    try {
      if (fs.existsSync(filePath)) {
        console.log('File Path Found for audio config: ', filePath)
        fs.readdir('/tmp', (err, files) => {
          console.log('TMP DIR FILES:', files)
        })
        audioConfig = sdk.AudioConfig.fromWavFileInput(fs.readFileSync(filePath))

        console.log('audioConfig: ', audioConfig)
      } else {
        return reject(new Error('File not found: ' + filePath))
      }
    } catch (err) {
      return reject(new Error('Error reading file: ' + err))
    }

{
    "message": "Recognition not successful. Reason: Error Code: 4, Error Details: Unable to contact server. StatusCode: 1006, undefined Reason:  this.connect is not a function"
}

Rohit Mungi 49,126 Reputation points Microsoft Employee Moderator

2023-12-20T11:43:41.4566667+00:00

Hyojae Kim How is the recognizer initialized in this case? I think the message or reason "this.connect is not a function" does not indicate an issue with path of the file. I think this could be with the connection endpoint used to initialize the recognizer. Please see a similar thread with the error. This comment from the user on the issue fixed the error when speechSdk.SpeechRecognizer.FromConfig() was used instead of speechSdk.SpeechConfig.fromEndpoint()

Hyojae Kim 5

Hi @Rohit Mungi . Thanks for the reply. This is how I am initializing my reco:

I've tried applying your suggestion, but I received a different error provided below.
I'm not sure if this has to do with the way I am setting up my recognizer, because it works fine on my local.

{
    "message": "Cannot read properties of undefined (reading 'mergeTo')"
}


    const pronunciationAssessmentConfig = sdk.PronunciationAssessmentConfig.fromJSON(
      '{"GradingSystem": "HundredMark", "Granularity": "Phoneme", "EnableMiscue": true}',
    )
    console.log('pronunciationAssessmentConfig: ', pronunciationAssessmentConfig)

    pronunciationAssessmentConfig.referenceText = sentence
    pronunciationAssessmentConfig.enableProsodyAssessment = true

    const reco = new sdk.SpeechRecognizer(speechConfig, audioConfig)
    console.log('reco', reco)
    pronunciationAssessmentConfig.applyTo(reco)
    console.log('recognizer applied to pronunciation assessment config!')

    reco.recognizeOnceAsync(
      (result) => {
        console.log('test log')
        console.log('result: ', result)
        if (result.reason === sdk.ResultReason.RecognizedSpeech) {
          console.log('test log 2')
          const pronunciationResult = sdk.PronunciationAssessmentResult.fromResult(result)
          const detailedResults = pronunciationResult.detailResult.Words.map((word, index) => ({
            word: word.Word,
            accuracyScore: word.PronunciationAssessment.AccuracyScore,
            errorType: word.PronunciationAssessment.ErrorType,
          }))

Hyojae Kim 5 Reputation points

2023-12-21T08:44:18.5433333+00:00

@Rohit Mungi Thanks a lot for your help! I actually think it has something to do with ffmpeg and lambda.

1 answer

Your answer

Rohit Mungi 49,126 Reputation points Microsoft Employee Moderator

2023-12-20T11:43:41.4566667+00:00

Hyojae Kim How is the recognizer initialized in this case? I think the message or reason "this.connect is not a function" does not indicate an issue with path of the file. I think this could be with the connection endpoint used to initialize the recognizer. Please see a similar thread with the error. This comment from the user on the issue fixed the error when speechSdk.SpeechRecognizer.FromConfig() was used instead of speechSdk.SpeechConfig.fromEndpoint()
Hyojae Kim 5 Reputation points

2023-12-21T08:06:58.0733333+00:00

Hi @Rohit Mungi . Thanks for the reply. This is how I am initializing my reco:

I've tried applying your suggestion, but I received a different error provided below.
I'm not sure if this has to do with the way I am setting up my recognizer, because it works fine on my local.

{ "message": "Cannot read properties of undefined (reading 'mergeTo')" }

const pronunciationAssessmentConfig = sdk.PronunciationAssessmentConfig.fromJSON( '{"GradingSystem": "HundredMark", "Granularity": "Phoneme", "EnableMiscue": true}', ) console.log('pronunciationAssessmentConfig: ', pronunciationAssessmentConfig) pronunciationAssessmentConfig.referenceText = sentence pronunciationAssessmentConfig.enableProsodyAssessment = true const reco = new sdk.SpeechRecognizer(speechConfig, audioConfig) console.log('reco', reco) pronunciationAssessmentConfig.applyTo(reco) console.log('recognizer applied to pronunciation assessment config!') reco.recognizeOnceAsync( (result) => { console.log('test log') console.log('result: ', result) if (result.reason === sdk.ResultReason.RecognizedSpeech) { console.log('test log 2') const pronunciationResult = sdk.PronunciationAssessmentResult.fromResult(result) const detailedResults = pronunciationResult.detailResult.Words.map((word, index) => ({ word: word.Word, accuracyScore: word.PronunciationAssessment.AccuracyScore, errorType: word.PronunciationAssessment.ErrorType, }))
Hyojae Kim 5 Reputation points

2023-12-21T08:44:18.5433333+00:00

@Rohit Mungi Thanks a lot for your help! I actually think it has something to do with ffmpeg and lambda.

Answer 1

@Hyojae Kim I think with lamda environment ( I am assuming it is AWS Lamda ), I am not sure if any additional setup is required with the SDK as compared to your local setup. But, in this case I think the first error code is now corrected as it was previously unable to connect to the endpoint. The new error seems to something that fails while processing your file input. A quick lookup of previous SDK issues indicates that a file object needs to be passed rather than file name. Please see the issue reference here.

However, I would recommend raising an issue on the SDK repo for clarification of the error with details of the setup i.e environment and SDK used. Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Share via

this.connect error from microsoft pronunciation assessment sdk

1 answer

Your answer