ElevenLabs API Integration into FlutterFlow

Hello,

I offer you a tutorial to use the ElevenLabs API. I've seen several topics where integrating this API is problematic. I've found a solution and I would like to share it to help as many people as possible :)

I will focus on the "Text to Speech" functionality of ElevenLabs, which I assume is the most requested.

Context :

I think everyone trying to integrate the ElevenLabs API encounters the same issue: it returns a "200 success" code but with "null" content, and when you look at the "Raw Text" part, you encounter strange encoding.

It's important to understand that the "Text to Speech" API call from ElevenLabs returns a file type value "audio/mpeg". However, when integrating this API through FlutterFlow's "API Call" menu/feature, there's a "filter" performed to only retrieve JSON type results, which therefore poses a problem in cases where the returned result is a "file".

Solution:

To work around the problem, create a Custom Action, which will make an API call, where you can retrieve ElevenLabs' binary response, decode it properly, and save it in FirebaseStorage as an mp3. You can then return the mp3 URL to integrate it into an Audio Player, for example, or save it in a Firebase collection or elsewhere.

Step 1:

Create a Custom Action and copy/paste the code below.

// Automatic FlutterFlow imports
import '/backend/backend.dart';
import '/backend/schema/structs/index.dart';
import '/flutter_flow/flutter_flow_theme.dart';
import '/flutter_flow/flutter_flow_util.dart';
import '/custom_code/actions/index.dart'; // Imports other custom actions
import '/flutter_flow/custom_functions.dart'; // Imports custom functions
import 'package:flutter/material.dart';
// Begin custom action code
// DO NOT REMOVE OR MODIFY THE CODE ABOVE!

// Set your action name, define your arguments and return parameter,
// and then add the boilerplate code using the green button on the right!
import 'package:http/http.dart' as http;
import 'dart:convert';
import 'package:firebase_storage/firebase_storage.dart';
import 'dart:typed_data'; // Pour Uint8List

Future<String> makeApiCallAndUploadAudio(
    String text, String filename, String voiceid, String userid) async {
  try {
    // Corps JSON spécifié
    Map<String, dynamic> jsonData = {
      "voice_settings": {
        "stability": 0.76,
        "similarity_boost": 0.75,
        "style": 0.26,
        "use_speaker_boost": true
      },
      "model_id": "eleven_multilingual_v2",
      "text": text
    };

    // Convertir les données JSON en chaîne
    String body = json.encode(jsonData);

    // Configurer les headers pour la requête
    Map<String, String> headers = {
      'Content-Type': 'application/json',
      'xi-api-key': 'SECRET KEY API',
    };

    // Faire la requête POST
    http.Response response = await http.post(
      Uri.parse(
          'https://api.elevenlabs.io/v1/text-to-speech/$voiceid?output_format=mp3_22050_32'),
      headers: headers,
      body: body,
    );

    // Vérifier le code de statut de la réponse
    if (response.statusCode == 200) {
      // Supposons que le body de la réponse est les données binaires du fichier audio
      Uint8List audioData = response.bodyBytes;

      // Télécharger les données audio dans Firebase Storage
      return await uploadAudioToFirebaseStorage(audioData, filename, userid);
    } else {
      // Extraire les détails de l'erreur du corps de la réponse
      var errorData = json.decode(response.body);
      String errorMessage = errorData['message'] ?? 'Unknown error occurred';
      return 'Failed to load audio data: ${response.statusCode}, Error: $errorMessage';
    }
  } catch (e) {
    // Gérer les erreurs de réseau ou de parsing
    return 'Error occurred during API call or file upload: $e';
  }
}

// Fonction pour télécharger des données audio dans Firebase Storage
Future<String> uploadAudioToFirebaseStorage(
    Uint8List audioData, String fileName, String userId) async {
  FirebaseStorage storage = FirebaseStorage.instance;
  Reference ref = storage.ref('users/$userId/uploads/$fileName.mp3');

  try {
    UploadTask uploadTask =
        ref.putData(audioData, SettableMetadata(contentType: 'audio/mpeg'));

    // Attendre la fin du téléchargement
    final TaskSnapshot snapshot = await uploadTask.whenComplete(() {});
    // Récupérer l'URL de téléchargement
    final downloadUrl = await snapshot.ref.getDownloadURL();
    return downloadUrl; // Retourne l'URL où le fichier peut être téléchargé
  } catch (e) {
    return 'Error during file upload: $e';
  }
}

- Add your ElevenLabs API secret key in place of "SECRET KEY".

Step 2 :

Create a PageState of type AudioPath, with the name "sound" for this example.

Default value to avoid errors, for example:
https://filesamples.com/samples/audio/mp3/sample3.mp3

Step 3 :

Add an AudioPlayer with the previously created PageState "sound" as the value.

Step 4 :

Add a button and create a Custom Action and call the action created above.

Complete the parameters for the proper functioning of the API call.
- text : Message you want to “vocalize” with AI
- filename : Name of the file that will be saved in FirebaseStorage
- voiceid : ID of the voice you want to use which is found in ElevenLabs
- userid : Reference of the connected user to save it in their FirebaseStorage folder
Add a name to the Output action which will be of type AudioPath (.mp3), for example "vocalai".

Step 5 :

Add an "Update Page State" action of the Page State "sound" created in step 2 and give as value the result of the Output action namely "vocalai".

You can launch your application in web testing, click on the button that launches the action, launch the AudioPlayer and enjoy the result 😊

I hope my explanations are clear. Please let me know if this was helpful to you or if I can help you if you ever encounter any difficulties in the implementation 🙂