SpeechToText doesn't work properly after Update

[LanzelotAsp]

Hi,

Recently I've updated from 8.0.3 to 8.6.3. After that the transcriptions endpoint didn't work any more. The spoken text wasn't recognized at all. Instead some random rubbish text came out.
After several tests I found out that reducing the bitrate to 16000 was the problem. While this works fine with 8.0.3 it doesn't work any more in 8.6.3. I also tried different frequencies but finally it had to be 44100 to make it work again.

Any idea why this is?

Regards,
Alex

[StephenHodgson]

Again, I'll need to see some sample code to get to the bottom of this.

I had to re tool all the audio to make sure it is being generated at the correct frequency for all runtime platforms. Sounds like you were doing the same manually? You should be able to play the clip directly or use the StreamAudioSouce implementation like in the latest versions of the samples.

The reason it forced to 44100 is a unity problem. The AudioClip sample buffer is always this length for whatever reason regardless of the AudioSettings.OutputFrequency. Just another quirky thing about the Engine you just have to learn through tribal knowledge.

The latest version of the plugin is correctly decoding and building AudioClips correctly so please take a look at the samples and adjust your implementation accordingly.

[StephenHodgson]

Likely not. It only fixes memory leak from memory stream copying.

[StephenHodgson]

How are you recording your audio?

[LanzelotAsp]

Basically with Microphone.Start() & End(). I record in a loop and then the part where the user speaks is taken out.
Sample Rate at recording is 44100.
maxRecordingTimeSeconds is 30.

		clipSizeSeconds = maxRecordingTimeSeconds + 2;
		AudioClip clip = Microphone.Start(micDeviceName, true, clipSizeSeconds, sampleRate);

This is how the final AudioClip is created:

	private void SaveAudioSegment(float[] audioData, int clipSampleRate)
	{
		// Debug.Log($"Saving audio segment, samplerate={clipSampleRate}");
		
		// Here you can write the audio data to a file or save it as an AudioClip
		// For example, creating a new AudioClip from the data and saving it
		AudioClip clip = AudioClip.Create("RecordedSegment", audioData.Length, 1, clipSampleRate, false);
		clip.SetData(audioData, 0);

		// Optionally, you can save this to a file using Unity's File API (e.g., Wav format)
		// This step requires additional code to encode the data to a file format
		// Debug.Log("Audio segment saved.");

		OnRecordingFinished.Invoke(clip);
	}

[StephenHodgson]

Yeah if you're handling the recording on your own then you're going to have to make sure that it's at 44100. It's just a quirk with the microphone in Unity.

I would encourage you to try out my recording manager that is used in the samples. It's probably much more reliable and handles more edge cases with cross platform support.

[LanzelotAsp]

Unfortunately I cannot use your Recording Manager because we need different functionality. One difference is that we have the mic always on. Starting the mic when the user wants to speak adds an unwanted delay until the microphone is ready. That is why we are recording continuously.

[StephenHodgson]

Check out the real-time samples because it effectively does the same thing.

Continuously records

[LanzelotAsp]

Even the current Recording Manager from the latest samples does not work with 16000 Hz so the Recording Manager is not the cause of the issue.

I can show you code with original Recording Manager which does not work with downsampled AudioClip.

[StephenHodgson]

It does work, like I said the audio clip buffer has to be 44100 samples and then you have to resample it down to 16000. I already handle all these edge cases in the recording manager. It's up to you to half the edge cases of you're doing this yourself.