Integrating Perspective API for harassment filtering

Database & APIs

I'm currently building an app that is a daily prompt journal but upon answering, users are shown other users' answers anonymously. That being said, I don't want trolls and hate speech to take over the platform, which I fear could happen due to the anonymity aspect.

I found a great API made by Google called Perspective AI that analyzes responses and gives each of them a toxicity score, which you can then filter out or delete responses by said score. Ideally the workflow would be: send the response to Perspective API upon clicking 'Submit' -> API analyzes -> delete responses from the database over set threshold (0.80) so the users will never have the chance to see them.

That being said, I can't figure out how to add it. I don't have a history in developing apps and I've used ChatGPT for a large portion of this app (meaning I don't know how to do this myself).

Any help would be greatly appreciated. Thank you!

What have you tried so far?

ChatGPT told me this:

To implement server-side filtering using the Perspective API, ensuring that inappropriate content is filtered out before it gets pulled into your FlutterFlow app, follow these steps:

1. Get an API Key from Perspective API:

- Sign up for the Perspective API at [Perspective API](https://perspectiveapi.com/).

- Obtain your API key from the Google Cloud Console.

2. Add API Integration:

Since FlutterFlow doesn’t directly support integrating third-party APIs through its UI, you’ll need to use custom functions or a backend service. You can set this up using Firebase Cloud Functions or a custom backend server to interact with the Perspective API.

3. Create a Firebase Cloud Function (if using Firebase):

- In your Firebase project, go to the Functions section.

- Write a Cloud Function to call the Perspective API and check the content of the responses.

- Example of a basic Cloud Function in Node.js:

```javascript

const functions = require('firebase-functions');

const axios = require('axios');

const admin = require('firebase-admin');

admin.initializeApp();

const PERSPECTIVE_API_KEY = 'YOUR_PERSPECTIVE_API_KEY'; // Replace with your API key

exports.filterContent = functions.firestore

.document('userResponses/{docId}') // I updated it to the correct Collection

.onCreate(async (snap, context) => {

const data = snap.data();

const userText = data.content; // Adjust if your field name is different

const url = https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=${PERSPECTIVE_API_KEY};

const body = {

comment: { text: userText },

languages: ["en"],

requestedAttributes: { TOXICITY: {} }

};

try {

const response = await axios.post(url, body);

const toxicityScore = response.data.attributeScores.TOXICITY.summaryScore.value;

// If toxicity score is above threshold (e.g., 0.7), reject the content

if (toxicityScore > 0.7) {

console.log('Content flagged as inappropriate:', userText);

return snap.ref.delete(); // Delete the document

} else {

return null; // Content is fine

}

} catch (error) {

console.error("Error checking content with Perspective API:", error);

return null;

}

});

```

4. Call the Cloud Function from FlutterFlow:

- In FlutterFlow, set up a custom action to call your Cloud Function.

- Pass the user input (or content) to the Cloud Function for analysis.

5. Handle the API Response:

- Use the response from the Perspective API to determine if the content is offensive.

- For example, if the toxicity score exceeds a certain threshold, you can filter out the content before displaying it to users.

6. Test and Deploy:

- Test the integration thoroughly to ensure it works as expected.

- Deploy the Cloud Function and ensure your app is set up to handle the API responses and update the UI accordingly.

By integrating the Perspective API this way, you can automatically filter out potentially offensive content and maintain a positive user experience in your app.


Upon following these steps (including providing my key correctly), I get this error in Google Cloud when creating a cloud function:


Cloud Run service projects/project-01-wjnkmu/locations/us-central1/services/function-2 for the function was not found. The function will not work correctly. Please redeploy.

Build failed with status: FAILURE and message: /workspace/index.js:21 const url = https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=${PERSPECTIVE_API_KEY}; ^ SyntaxError: Unexpected token ':' at wrapSafe (node:internal/modules/cjs/loader:1281:20) at checkSyntax (node:internal/main/check_syntax:78:3) Node.js v20.16.0. For more details see the logs at https://console.cloud.google.com/cloud-build/builds;region=us-central1/71115aa2-845a-4f81-af3a-e7fb87813aa5?project=152198385022.

Any ideas on how to do this differently or get this to work?

Did you check FlutterFlow's Documentation for this topic?
Yes
1
2 replies