Custom Code Showcase
IM
Inactive Member

[Custom Function] Simple Full-Text Search

Hello, As we all know FireStore come with some limitations, especially for querying collection or/and documents, no built-in Full-text searching capability. We have to rely on expensive third party solution : Algolia / Typesense / ElasticSearch / Meilisearch, they all comes with theirs pros and cons, but they all share the sam main drawback, the cost. ( time or/and money ) Since   recently add the "Update Page On Change" for TextField we can now add a filter query that will be updated at any Keystroke input of the user. Firestore don't have a "contains" relation for filtering but we have the the array-contains to workaround. What we need is to create a Field Data Type List containing a list of keywords that the user will type in a TextField. Remember FireStore is 1MB limitation per Document, this is the equivalent of nearly an 500 pages Book. Plenty of space to play around and add thousands of keywords for creating an effective searching feature !! This tutorial is made for people with a good knowledge of queries, filters and FF in general.  Below the implementation : First we will need to create a custom Function that will generate our Keywords Tree : My needs was to sparse 2 fields ( Product Name  Product Reference ) [ Warehouse management Application ] But you can implement it with as many document's fields you want. Good Advice is using short inputs as : tittle, name, category etc..... ( using it on a long text, exemple a movie description will end-up with thousands or thousand keywords, it will still work but will generate massive data, longuer query etc... so at the end poor performance )

Keywords Tree Exemple : Product Name : Plasma TV Product Reference : 1234

 This will create : 
[
  ,
 plasma tv,
 1234,
 p,
 pl,
 pla,
 plas,
 plasm,
 plasma,
 t,
 tv,
 1,
 12,
 123,
 1234
]

Note that we will add on every document a keyword containing only a space. This will be our unique and commun keyword an all our document. This way by setting an empty space as default value on our TextField will end up having all our collection. Ok, Ok too much talk y'all get it ! Below the keywords array generator as a custom function : [image.png] The code is ugly, i'm sure there is a better way of doing this, i let a real developper optimize it

import 'dart:math' as math;

List keyWordsMultipleFields(
  String field1,
  String field2,
) {
  var string1 = field1.toLowerCase();
  var string2 = field2.toLowerCase();
  var inputString = string1 + " " + string2;
  final list = inputString.split(" ");
  var keywords = [
    " ",
    string1,
    string2,
  ];

  var listIterator = list.iterator;

  while (listIterator.moveNext()) {
    var appendString = "";

    listIterator.current.runes.forEach((int rune) {
      appendString += String.fromCharCode(rune);
      keywords.add(appendString);
    });

    inputString = inputString.replaceAll(listIterator.current.toString(), " ");
  }

  return keywords;
 }Result :[image.png]

Next step create a new field in the collection you want the searching feature:[image.png] Now we can setup the keyword field on the action "Create Record" or "Update Record" with our custom function as variable input. For Create Record[image.png] For Update Record this is the same but your select "Add to List" for the List update Type. Voila, now every time a document is created or updated all the searching keywords will be add. [ note that this will generate an error in FF : the Fixing at the end of the Tutorial ] Time for setting up the Query and the searching TextField : On your page add a TextField with the TextField Properties "Update Page On Change" enable with a 100ms Update Delay. [ the bigger your average size document the longuer the delay need to be ] You can add a leading Icon if you want too. [image.png][image.png]

Go to your Query and add a filter after all your existing one. Field : Choose your collection keywords field Relation : Array Contains Value Source : From Variable Source : Widget State Available Options : Select the searching input TextField [image.png]🍾 Everything is now set and ready, congrats 🍾 Unfortunately this feature can't be tested inside FF, a "bug" is present that will generate an error when you will set the Create or Update Action part. The issue come from the fact we can't direclty pass at once the generated keyword List items from the custom function to the keyword field. This will not allow you to make a Build. Below the link to the thread : https://community.flutterflow.io/c/suggestions-and-feedback/bug-feature-request-passing-list-values-to-list-any-field#comment_wrapper_4643811 To fix this issue, you will need to apply some modifications to the downloaded FF Code : The fix consist of : Replacing the [] by an {} in your exported code for Create Record Action. Or Deleting the [] in your exported code for Update Record Action. The custom function already output an [] , wihout this first fix it will endup in a Direct Nested Array [[]]. ( not supported by the firebase backend limitation ) Cool, now we pass an indirect nested Array..... but the backend Firestore requires you to pass those items individually !!! To fix this second issue you have to add the spread operator ... before the [Array] ( not for Update Record Action ) The code to edit and fix will be present in the dart file of your page or component. ( where is your create or update action ) Fix Create Record action : For fixing you have to replace the [] by an {} in your exported code  and adding the spread operator ... before the [Array] 

 
Before :
'Your_DataTypeList_Field': [                                  functions.keyWordsMultipleFields(                                      designationController.text,                                      referenceController.text)],After Fixing :
'Your_DataTypeList_Field': {                                  ...functions.keyWordsMultipleFields(                                      designationController.text,                                      referenceController.text)},Fix Update Record action :
For fixing you have to delete the [] in your exported code
Before :
'Your_DataTypeList_Field': FieldValue.arrayUnion([                                 functions.keyWordsMultipleFields(                                      designationController.text,                                      referenceController.text)]),After Fixing :
'Your_DataTypeList_Field': FieldValue.arrayUnion(                                 functions.keyWordsMultipleFields(                                      designationController.text,                                      referenceController.text)),

Build the app for your desired platform and Voila !!! You have a Full-Text searching capability to your application !!! This will cost you the number of document queried at EVERY keystroke entered by your users, so whach out !! You should probably setup a limit on the query to make sure you will not end up in bankruptcy 🤪 ( this will still be cheaper than most third party solution ) This solution isn't perfect but it fulfill my needs. I hope it will be usefull for others. Below the working result :[Full_Text_search.gif] Let's Hope, will find a solution to fix the "bug". This will avoid the edit fixing part outside FF UI environment.  Next evolution to V2 : implementing the keywords tree on the backend level with Firebase function, a must need when you populate your collection with thousand records using the json admin import tool.  ( i'm looking for some help 👀 ) PS: Pardon my English 🇫🇷

2
4 replies