How to eliminate listview flicker on add and delete from Firebase?

I get a weird flicker when I add or delete data from Firestore and show it in a listview. I'm doing a backend query and filtering by owner and a category. It does it for the third and fourth addition and on the last deletion. Any thoughts on how to remove this flickering effect?