How can I successfully stream Azure TTS audio from my server to the client using Fetch API and PassThrough push stream?
I am attempting to stream Azure TTS from my server to the client using the Fetch API and a PassThrough push stream. The expected outcome is to receive the stream in chunks. The actual output is a response object with no information. I have tried creating a ReadableStream using Fetch, but when I try to log the response, I get an error message that my response object is size 0. I have also tried to see if anything is getting sent through in chunks, but everything is size 0. I have tried to debug my backend but from everything I can tell, it is working properly. If anyone has solved this issue or has demo code for streaming TTS in JavaScript, please let me know. This is my actual function code. I believe it works.:
const generateSpeechFromText = async (text) => {
const speechConfig = sdk.SpeechConfig.fromSubscription(
process.env.SPEECH_KEY,
process.env.SPEECH_REGION
);
speechConfig.speechSynthesisVoiceName = "en-US-JennyNeural";
speechConfig.speechSynthesisOutputFormat =
sdk.SpeechSynthesisOutputFormat.Audio16Khz32KBitRateMonoMp3;
const synthesizer = new sdk.SpeechSynthesizer(speechConfig);
return new Promise((resolve, reject) => {
synthesizer.speakTextAsync(
text,
(result) => {
if (result.reason === sdk.ResultReason.SynthesizingAudioCompleted) {
const bufferStream = new PassThrough();
bufferStream.end(Buffer.from(result.audioData));
resolve(bufferStream);
} else {
console.error("Speech synthesis canceled: " + result.errorDetails);
reject(new Error("Speech synthesis failed"));
}
synthesizer.close();
},
(error) => {
console.error("Error in speech synthesis: " + error);
synthesizer.close();
reject(error);
}
);
});
This is my index.js route code to send to the frontend. I believe it works but there could be an error.:
app.get("/textToSpeech", async (request, reply) => {
if (textWorks) {
try {
const stream = await generateSpeechFromText(
textWorks
);
console.log("Stream created, sending to client: ", stream);
reply.type("audio/mpeg").send(stream);
} catch (err) {
console.error(err);
reply.status(500).send("Error in text-to-speech synthesis");
}
} else {
reply.status(404).send("OpenAI response not found");
}
});
This is my frontend client code. I think the error has to do with the response object, but I am not sure.:
// Fetch TTS from Backend
export const fetchTTS = async (): Promise