Text Moderation - ML models BETA

Available classes

Class availability depends on the language of the submitted text.

One given text may obtain scores returned by the API indicating a match for multiple classes: a text can be toxic and insulting at the same time for instance.

Examples

Sexual class

The sexual class will also detect non-consensual sexual content related to topics such as rape, prostitution or pedophilia. For example:

Discriminatory class

Insulting class

Violent class

Toxic class

Toxic content might also match the description of one of the other classes.

Language support

The following languages are currently supported for Text Classification:

You are expected to specify the language or list of languages you want to support by setting their ISO 639-1 codes in the lang parameter.

If you are unsure about the language used by a user, or if users mix different languages in their messages you can specify multiple languages as a comma-separated list.

For instance set lang=en,fr,es for users that might write in english, french or spanish. We strongly recommend specifying the shortest possible list as this will yield better results both in speed and accuracy. If your users mostly speak english, specifying en will yield better results.

Other languages are available upon request. Please get in touch regarding your language needs.

Use the models

Let's say you want to moderate the following text item:

Simply send a POST request containing the UTF-8 formatted text along with the ISO 639-1 language code (such as en for english). Here is an example:

curl -X POST 'https://api.sightengine.com/1.0/text/check.json' \
  -F 'text=I love you so much' \
  -F 'lang=en' \
  -F 'mode=ml' \
  -F 'api_user={api_user}' \
  -F 'api_secret={api_secret}'

# this example uses requests
import requests
import json

data = {
  'text': 'I love you so much',
  'mode': 'ml',
  'lang': 'en',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
r = requests.post('https://api.sightengine.com/1.0/text/check.json', data=data)

output = json.loads(r.text)

$params = array(
  'text' => 'I love you so much',
  'lang' => 'en',
  'mode' => 'ml',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/text/check.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);

// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');

data = new FormData();
data.append('text', 'I love you so much');
data.append('lang', 'en');
data.append('mode', 'ml');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  url: 'https://api.sightengine.com/1.0/text/check.json',
  method:'post',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

As an example, here is the JSON request that you would receive for the above request:

{
  "status": "success",
  "request": {
    "id": "req_6cujQglQPgGApjI5odv0P",
    "timestamp": 1471947033.92,
    "operations": 1
  },
  "moderation_classes": {
    "available": [
      "sexual",
      "discriminatory",
      "insulting",
      "violent",
      "toxic"
    ],
    "sexual": 0.01,
    "discriminatory": 0.01,
    "insulting": 0.01,
    "violent": 0.01,
    "toxic": 0.01
  }
}