Flutter App, Speech to Text and Text to Speech 🐣

What to expect from this article?

We initiated our Flutter project with the main libraries, and finished the login screen in the previous articles.

This article will cover the implementation of Speech to Text (STT) and Text to Speech (TTS) with bloc state management.

Before going on, I want to point out this amazing article series by AbdulMuaz Aqeel, his articles detail the project structure, clean architecture concepts, VS code extensions, and many other great details. feel free to use it as a more detailed reference for this article 🤓.

I'll try to cover as many details as possible without boring you, but I still expect you to be familiar with some aspects of Dart and Flutter.

the final version of the source code can be found at https://github.com/saad4software/alive-diary-app

Speech to text

We will use speech_to_text library, it is kinda simple to implement and totally free. If you haven't add add the library yet, you may add it using

flutter pub add speech_to_text 

STT requires certain permissions to be used in android, so let's add them in the manifest file

<manifest xmlns:android="http://schemas.android.com/apk/res/android">

    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.INTERNET"/>

    <queries>
        <intent>
            <action android:name="android.speech.RecognitionService" />
        </intent>
    </queries>

    <application
        android:label="alive_diary_app"
...

android/app/src/main/AndroidManifest.xml

with the manifest updated, let's begin by creating bloc logic for STT screen, any bloc logic consists of three parts, event, state and bloc, we always start with the event, we need an event that get triggered when start listening, let's call it HomeSTTEvent

part of 'home_bloc.dart';

@immutable
sealed class HomeEvent {}

class HomeSTTEvent extends HomeEvent {}

lib/presentation/screens/home/home_event.dart

After clicking the button, the app should listen to the user speech. after listening, we need to get the text from the state, so we need a listening state, and a success state with text. Error state is also a must in case we got any errors.

part of 'home_bloc.dart';

@immutable
sealed class HomeState {
  final String? text;
  final String? error;

  const HomeState({
    this.text,
    this.error,
  });
}

final class HomeInitial extends HomeState {}

final class HomeListeningState extends HomeState {}

final class HomeSTTState extends HomeState {
  const HomeSTTState({super.text});
}

final class HomeErrorState extends HomeState {
  const HomeErrorState({super.error});
}

lib/presentation/screens/home/home_state.dart

So, when we get the STT event, we start by emitting listening state, and when done listening, we return the text to the UI in a STT state.

import 'dart:async';

import 'package:bloc/bloc.dart';
import 'package:meta/meta.dart';
import 'package:speech_to_text/speech_to_text.dart';

part 'home_event.dart';
part 'home_state.dart';

class HomeBloc extends Bloc<HomeEvent, HomeState> {

  final SpeechToText stt;
  String? recognizedText = "";

  HomeBloc(this.stt) : super(HomeInitial()) {
    on<HomeSTTEvent>(handleSTTEvent);
  }

  FutureOr<void> handleSTTEvent(
      HomeSTTEvent event,
      Emitter<HomeState> emit,
  ) async {
    emit(HomeListeningState());

    bool available = await stt.initialize();

    if (available) {
      await listen();
      emit(HomeSTTState(text: recognizedText));

    } else {
      emit(HomeErrorState(
        error: "Speech recognition permission denied",
      ));
    }
  }

  // convert listen callback to Future
  Future listen() async {

    final completer = Completer();
    stt.listen(
        onResult: (result) {
          recognizedText = "${recognizedText?.trim()} ${result.recognizedWords}";
          completer.complete(recognizedText);
        },
        listenFor: const Duration(minutes: 2),
        localeId: "language".tr(),
        listenOptions: SpeechListenOptions(
          partialResults: false,
        )
    );
    return completer.future;
  }
}

lib/presentation/screens/home/home_bloc.dart

stt uses callback style, so we converted the callback to future using Completer in the listen function. the recognized speech is stored in recognizedText, it accumulate text for the user. we need to add another event to clear it later.
Notice that we are passing translated string "language" as a locale id, it has the value "en" and "ar" in the languages JSON, this is an easy workaround to get the current locale without a context instant.
when user trigger STTEvent event, we emit HomeListeningState to the UI, then we make sure the stt instant is initialized. after listening we emit the recognized text in STTState, and in case of error, we emit an error message in Error state.

After setting up the bloc, we need to make sure it is provided throughout the app, so in the main app file

...
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: MultiBlocProvider(
        providers: [
          BlocProvider(create: (context)=>HomeBloc(locator())),
          BlocProvider(create: (context)=>LoginBloc(locator(), locator())),
        ],
...

lib/main.dart

We are passing the STT from DI, but our dependency injection locator does not offer SST instance yet, so let's add it to our DI config file

import 'package:speech_to_text/speech_to_text.dart';

final locator = GetIt.instance;

Future<void> initializeDependencies() async {
  final dio = Dio();

  locator.registerSingleton<Dio>(dio);

  locator.registerSingleton<SpeechToText> (SpeechToText());
...

lib/config/dependencies.dart

great, now we can get the bloc from the provider in the home screen. let's make the simplest home screen design to test our STT

import 'package:auto_route/auto_route.dart';
import 'package:flutter/material.dart';
import 'package:flutter_bloc/flutter_bloc.dart';
import 'package:flutter_hooks/flutter_hooks.dart';
import 'home_bloc.dart';

@RoutePage()
class HomeScreen extends HookWidget {
  const HomeScreen({super.key, this.title});

  final String? title;

  @override
  Widget build(BuildContext context) {
    final bloc = BlocProvider.of<HomeBloc>(context);

    final isLoading = useState(false);
    final speechText = useState("");

    return BlocListener<HomeBloc, HomeState>(
      listener: (context, state) {
        isLoading.value = state is HomeListeningState;

        if (state is HomeSTTState) {
          speechText.value = state.text ?? "";
        }

      },
      child: Scaffold(
        appBar: AppBar(
          backgroundColor: Theme
              .of(context)
              .colorScheme
              .inversePrimary,
          title: Text(title ?? "home"),
        ),
        body: Center(
          child: Column(
            mainAxisAlignment: MainAxisAlignment.center,
            children: <Widget>[
              Text(speechText.value),
            ],
          ),
        ),
        floatingActionButton: FloatingActionButton(
          tooltip: 'Listen',
          child: Icon(
            Icons.keyboard_voice_outlined,
            color: isLoading.value ? Colors.green : Colors.blue,
          ),
          onPressed: () => bloc.add(HomeSTTEvent()),
        ),
      ),
    );
  }
}

lib/presentation/screens/home/home_screen.dart

Notice that we are not using bloc builder! actually flutter hook simplify the UI update by using notifier objects. since bloc builders complicate the widget tree, we are only using bloc listener, and pass the data to hook states to update the UI.

This screen should work properly, when clicking on the button, it triggers the STT event, when getting the state to a listening state, the icon color changes to green, it should turn back to blue if on any other state, and the text should appear in the text field.

We are done with the speech to text feature, let's start working on text to speech

Text to speech

We are going to use flutter tts, it is also free and easy to use. if you haven't add the library to your project, you can add it using

flutter pub add flutter_tts 

similar to STT, we need to add a query in the android app manifest file

...
    <queries>
        <intent>
            <action android:name="android.speech.RecognitionService" />
        </intent>
    </queries>

    <queries>
        <intent>
            <action android:name="android.intent.action.TTS_SERVICE" />
        </intent>
    </queries>
...

android/app/src/main/AndroidManifest.xml

Just add it below the STT query, let's update the bloc events to add a TTS event. TTS event should pass the text, then read it in the bloc logic, and return a reading status, and when done, return the initial state, or an idle state maybe. let's start with the event

part of 'home_bloc.dart';

@immutable
sealed class HomeEvent {
  final String? text;
  const HomeEvent({this.text});
}

class HomeSTTEvent extends HomeEvent {}

class HomeTTSEvent extends HomeEvent {
  const HomeTTSEvent({super.text});
}

lib/presentation/screens/home/home_event.dart

we are passing the text to the event, now for the state

...
final class HomeReadingState extends HomeState {}
...

lib/presentation/home/home_state.dart

We are just adding a simple reading state. the bloc code should be somewhat like

...
class HomeBloc extends Bloc<HomeEvent, HomeState> {

  final SpeechToText stt;
  final FlutterTts tts;
  String? recognizedText = "";

  HomeBloc(this.stt, this.tts) : super(HomeInitial()) {
    on<HomeSTTEvent>(handleSTTEvent);
    on<HomeTTSEvent>(handleTTSEvent);
  }
...
  FutureOr<void> handleTTSEvent(
      HomeTTSEvent event,
      Emitter<HomeState> emit,
  ) async {
    emit(HomeReadingState());

    await tts.setLanguage("language".tr());
    await tts.setSpeechRate(0.5);
    await tts.setVolume(1.0);
    await tts.setPitch(1.0);

    await tts.speak(event.text ?? "");
    recognizedText = ""; // clean the accumulated text

    emit(HomeInitial());
  }
}

lib/presentation/screens/home/home_bloc.dart

now where to get the TTS instance from? we need to add it to the DI config file, similar to STT

import 'package:flutter_tts/flutter_tts.dart';

...
  locator.registerSingleton<SpeechToText> (SpeechToText());
  locator.registerSingleton<FlutterTts> (FlutterTts());
...

lib/config/dependencies.dart

We are just adding the TTS instance similar to STT, now we can get it from the locator object and use it in the main app file

...
          BlocProvider(create: (context)=>HomeBloc(locator(), locator())),
...

lib/main.dart

We are just editing the HomeBloc provider to provide the TTS instance, I believe it is time to update the home screen to test our TTS, let's just add another button to trigger the TTS event

...
        floatingActionButton: Column(
          mainAxisAlignment: MainAxisAlignment.end,
          children: [
            FloatingActionButton(
              tooltip: 'Read',
              child: Icon(
                Icons.record_voice_over_outlined,
              ),
              onPressed: () => bloc.add(HomeTTSEvent(text: speechText.value)),
            ),
            const SizedBox(height: 10,),
            FloatingActionButton(
              tooltip: 'Listen',
              child: Icon(
                Icons.keyboard_voice_outlined,
                color: isLoading.value ? Colors.green : Colors.blue,
              ),
              onPressed: () => bloc.add(HomeSTTEvent()),
            ),
          ],
        ),
...

lib/presentation/screens/home/home_screen.dart

and it is done! the home screen should look like

We have integrated STT and TTS in the app, the current app allows users to recognize speech, show it in a text field, and read it again with a button. Great! We will start with the AI API implementation next. so

Stay tuned 😎