Gemini Flash 1.5 API requires you to send image using base64 encoding. But there is no direct way in flutterflow to do so.
Gemini API call, sending image over network
I initially used local upload for image, but was having difficulty in converting the file directly because it was of type FFUploadByte or something like that. Then I uploaded images directly to firebase such that I could simply fetch the URL and send it over network. But I was getting this response:
{
"error": {
"code": 400,
"message": "Invalid value at 'contents[0].parts[1].inline_data.data' (TYPE_BYTES), Base64 decoding failed for \"'$(base64 -w0 null)'\"",
"status": "INVALID_ARGUMENT",
"details": [
{
"@type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "contents[0].parts[1].inline_data.data",
"description": "Invalid value at 'contents[0].parts[1].inline_data.data' (TYPE_BYTES), Base64 decoding failed for \"'$(base64 -w0 null)'\""
}
]
}
]
}
}
Finally I created a custom action which could fetch the image over URL and then convert it to base64. But now I am not getting any response from Gemini (possibly due to this faulty code that I might have written):
// Automatic FlutterFlow imports
import '/backend/backend.dart';
import '/flutter_flow/flutter_flow_theme.dart';
import '/flutter_flow/flutter_flow_util.dart';
import '/custom_code/actions/index.dart'; // Imports other custom actions
import '/flutter_flow/custom_functions.dart'; // Imports custom functions
import 'package:flutter/material.dart';
// Begin custom action code
// DO NOT REMOVE OR MODIFY THE CODE ABOVE!
import 'package:http/http.dart' as http;
import 'dart:convert';
Future<String> getImageBase64(String url) async {
http.Response response = await http.get(Uri.parse('$url'));
print(response.bodyBytes);
print("\n");
print(response);
String base64 = base64Encode(response.bodyBytes);
return base64;
}
// Set your action name, define your arguments and return parameter,
// and then add the boilerplate code using the green button on the right!
I request you to help me on urgent basis. Thanks and regards.