Skip to content

Getting started with AI closed captions in live streams

This tutorial explains how to configure and manage a live stream with multilingual AI generated closed captions (subtitles, see also our introduction to closed captions in Clevercast.

These can be combined with AI-generated speech translations, as well as Simultaneous Interpretation and human captioning. Although these workflows are largely similar, we recommend reading the respective tutorials.

By default, the closed captions are generated automatically, without human intervention. To increase accuracy, you can have the initial captions (which are the result of speech-to-text conversion) edited in real-time by a human corrector (see the speech-to-text correction manual).

This tutorial applies to enterprise and webinar plans. Skip step 1 for webinars, since an event is automatically created when you create a webinar.

1 Creating an event

Select the Live > Events menu. On the events page, press the Create Multilingual Event button in the sidebar. In the popup dialog, enter the following info and press Create:

  • an event name (for your own information only)
  • select one of the broadcast protocols with support for AI.
  • select the origin of your broadcast
  • select the Default Language. This is the language being spoken in the live stream, which will also be the first closed caption language. If the floor audio contains multiple languages, you can select 'Original' instead of real language (this is currently not possible if you also use AI speech translations).
  • select the Latency. Choose 'Default Latency' for (slightly) more AI accuracy and more time for human correctors (if you use them), or 'Low Latency' for a shorter delay.
  • Optionally, select the Streaming Resolutions. Clevercast does cloud transcoding for adaptive bitrate streaming. Make sure the highest resolution doesn’t exceed the resolution in which you are broadcasting.

2 Configuring the event

When you press the Create button, Clevercast creates the event and redirects you to its detail page. Among other things, the event page allows you to do the following:

  • Copy the event’s embed code from the Management tab to your website or 3rd party platform.
  • On the Appearance tab, you can upload a poster image and show a countdown timer in the embedded player (among others)

To add closed captions to the event, go to the Caption Languages tab. Add the first caption language, which must be the result of speech-to-text conversion.

Next, you can add captions that are the result of automatic translation.

Notes:

  • When using a human corrector to edit the initial captions (which are the result of speech-to-text conversion), make sure that 'Allow captions to be corrected' on the Caption Languages tab is checked (= default). Note: this will add a slightly higher latency to the live stream.
  • Consider adding an AI Vocabulary containing terms (names, acronyms, industry jargon, technical phrases ...) that may appear in the live stream, so they will be correctly translated.

3 Test broadcast

Configure your encoder by using the settings on the Broadcast tab and our broadcast guidelines.

When you are ready to test, go to the Management tab and set the event status to Preview by pressing the Set to preview button. This is required for your broadcast to be processed, and for correctors to connect to their rooms.

Start your broadcast.

You can use the Preview player to see the closed captions. The live stream in the preview player has a delay of about 2 minutes. As soon as you start broadcasting, the preview player will indicate the number of seconds before the live stream is available.

After the event is set to Preview, correctors are already able to connect to the Correction room. After about one and a half minute, they can see the live stream and the first closed captions, and are able to correct them. If you are the event manager, you can use the Realtime Management page to communicate with correctors through text chat. Unfortunately, it is not yet possible to see closed captions in the realtime player on this page.

4 Starting the live stream

When the event is about to start, go the Management tab and press the Start event button. We recommend to start the event at least four minutes before the live action begins. This allows the player to start buffering and ensures that nobody misses the start.

When you start the event, Clevercast also starts recording the live stream. So, starting the event in time also ensures that the cloud recording is complete.

The embedded player automatically detects the status change and makes the live stream visible to your participants. You can also see the live stream in the Public Player on the Management tab. If you want to display a message in the embedded player, use the Service Message on the Management tab.

5 Stopping the live stream

After the live action has ended, wait at least 4 minutes before setting the event status to Ended, by pressing the Change Event Status button and selecting End event. The embedded player automatically detects this and shows a poster image or message to your viewers (see the Appearance tab) instead of the live stream.

Clevercast automatically completes the cloud recording and converts it to an MP4 file. You can download it via the Events > Recording menu. If your plan includes VoD, you can publish the recording to Video on-Demand.

Watch the video tutorial