Receive Video Analysis Callbacks

Receiving webhook callbacks

Once you have submitted a video stream and received a 200 HTTP status code, the engine will start consuming the stream and will assign scores to relevant frames and send detailed results to your callback URL.

To avoid sending too many callback events, the engine will send callbacks only when one of the following happens:

• the video has been completely read and the analysis is done
• an "interesting event" has been detected. One such event would for instance be the presence of nudity, or an AI-generated frame. This is defined by setting threshold values on the relevant probabilities.
• an error was encountered after the initial request was closed

Server-side code for callback receipt

Sightengine will submit a POST request to your configured callback URL. You can listen for this callback and process it like this:

# callbacks cannot be received through the command line
# please choose a server-side language to receive and process callbacks
# or use an approach that does not require callbacks (such as sequential video moderation)

import json
from django.http import HttpResponse

# Using Django
@csrf_exempt
def my_callback_view(request):
  payload = request.body
  content = json.loads(payload)

  # You now have access to the following
  # the media id => content['media']['id']
  # the moderation data => content['data']
  # the moderation status => content['data']['status']

  # Acknowledge receipt
  return HttpResponse(status=200)

$payload = @file_get_contents('php://input');
$content = json_decode($payload, true)

// You now have access to the following:
// the media id => $content['media']['id']
// the moderation data => $content['data']
// the moderation status => $content['data']['status']

// Acknowledge receipt
http_response_code(200);

// This example uses Express to receive callbacks
const app = require('express')();

// Use body-parser to retrieve the raw body as a buffer
const bodyParser = require('body-parser');

// Match the raw body to content type application/json
app.post('/webhook', bodyParser.raw({type: 'application/json'}), (request, response) => {
  let event;

  content = JSON.parse(request.body);

  // You now have access to the following:
  // the media id => content.media.id
  // the moderation data => content.data
  // the moderation status => content.data.status

  // Acknowledge receipt
  response.json({received: true});
});

app.listen(8000, () => console.log('Running on port 8000'));

Callback content

The callback is a JSON object containing information on the job along with the analysis results.

Job status

Results

Results are accessible on a per-frame basis under the data key. The structure of the results depends on your choice of models. Please head to our Model reference for more details.

Job progress

If the duration of the video is known beforehand, the progress of the job will be provided in the progress field. The value will be between 0 and 1, with a value of 1 corresponding to completion of the job. As an example, a value of 0.5 means that 50% of the job has been completed.

In many cases such as live-streams, the duration of the video is not known in advance. In such cases, the progress cannot be provided and a null value is returned.

Example

{
  "media": {
    "id": "med_9urgHE38fdLs0R85qa7MX",
    "uri": "https://example.domain.com/path/to/video.mp4"
  },
  "request": "req_9urg32dJhsENyFeehczYi",
  "data": {
    "status": "finished",
    "started": 1618946523.492469,
    "last_update": 1618946527.68613,
    "progress": 1.0,
    "operations": 6,
    "frames": [
      {
        "info": {
          "id": "med_9urgHE38fdLs0R85qa7MX_1",
          "position": 0
        },
        "nudity": {
          "sexual_activity": 0.01,
          "sexual_display": 0.01,
          "erotica": 0.01,
          "sextoy": 0.01,
          "suggestive": 0.01,
          "suggestive_classes": {
            "bikini": 0.01,
            "cleavage": 0.01,
            "cleavage_categories": {
              "very_revealing": 0.01,
              "revealing": 0.01,
              "none": 0.99
            },
            "male_chest": 0.01,
            "male_chest_categories": {
              "very_revealing": 0.01,
              "revealing": 0.01,
              "slightly_revealing": 0.01,
              "none": 0.99
            },
            "male_underwear": 0.01,
            "lingerie": 0.01,
            "miniskirt": 0.01,
            "other": 0.01
          }, 
          "none": 0.99,
          "context": {
            "sea_lake_pool": 0.01,
            "outdoor_other": 0.99,
            "indoor_other": 0.01
          }
        }
      },
      {
        "info": {
          "id": "med_9urgHE38fdLs0R85qa7MX_2",
          "position": 1000
        },
        "nudity": {
          "sexual_activity": 0.01,
          "sexual_display": 0.01,
          "erotica": 0.01,
          "sextoy": 0.01,
          "suggestive": 0.01,
          "suggestive_classes": {
            "bikini": 0.01,
            "cleavage": 0.01,
            "cleavage_categories": {
              "very_revealing": 0.01,
              "revealing": 0.01,
              "none": 0.99
            },
            "male_chest": 0.01,
            "male_chest_categories": {
              "very_revealing": 0.01,
              "revealing": 0.01,
              "slightly_revealing": 0.01,
              "none": 0.99
            },
            "male_underwear": 0.01,
            "lingerie": 0.01,
            "miniskirt": 0.01,
            "other": 0.01
          }, 
          "none": 0.99,
          "context": {
            "sea_lake_pool": 0.01,
            "outdoor_other": 0.99,
            "indoor_other": 0.01
          }
        }
      },
      ...
    ]
  }
}

Callback recommendations

Acknowledging receipt

To acknowledge receipt of a callback, your endpoint should return a 2xx HTTP status code. Any other information returned in the request headers or request body is ignored. All response codes outside this range, including 3xx codes, will indicate to Sightengine that you did not receive the callback. This does mean that a URL redirection or a "Not Modified" response will be treated as a failure.

If your callback script performs complex logic, or makes network calls, it's possible the script would timeout before Sightengine sees its complete execution. For that reason, you may want to have your callback endpoint immediately acknowledge receipt by returning a 2xx HTTP status code, and then perform the rest of its duties.

CSRF

If you are using a web framework such as Rails or Django, your site might be expecting a CSRF token to be set for all incoming POST requests. CSRFs are important to prevent cross site request forgery, but they will prevent you from receiving legitimate callbacks from Sightengine. You should therefore exempt the callback route from CSRF protection.

Verifying callbacks

As an extra security measure, you should check that the media id value returned in the callback matches the media id returned in the initial API response. This is can be helpful if you have several streams analyzed at the same time and need to verify what media each callback refers to.

HTTPS

If you use an HTTPS URL for your callback endopint, please make sure that your server is correctly configured to support HTTPS with a valid server certificate. Sightengine will valide that the connection to your server is secure before sending the data.

Idempotency

Keep in mind that your callback may occasionally receive the same event more than once. We advise you to guard against duplicated event receipts by making your event processing idempotent.