​
Introduction

You can upload videos and generate captions automatically during the process of uploading new videos to Gcore Video Streaming or do it later for any earlier uploaded video.

​
AI captions for a new video

When uploading a video use new parameters in the end-point. The system will automatically create a task for transcribing the video and translating subtitles, and the finished result will be added to the video as a subtitle. AI caption parameters:
  • auto_transcribe_audio_language: auto|<language_code> – Immediately after successful transcoding, an AI task will be automatically created for transcription the video to original language. Output by default is one subtitle channel with original languages inside.
  • auto_translate_subtitles_language: default|<language_codes,> – In additional to transcription on original language it’s possible to translate to other languagaes, so AI-task of subtitles translation can be applied. Use that parameter for that. Here you can specify multiple set of languages to translate to, then a separate subtitle will be generated for each specified language.
Example to transcribe: 
POST https://api.gcore.com/streaming/videos
{
    "name": "Spritefright Blender",
    "description": "Video copied from an external S3 Storage",
    "origin_url": "https://demo-files.gvideo.io/apidocs/spritefright-blender-cut30sec.mp4",
    "auto_transcribe_audio_language": "auto"
}
Example to transcribe from “eng” and translate to “fre,ger,ita,spa”: 
POST https://api.gcore.com/streaming/videos
{
    "name": "Spritefright Blender",
    "description": "Video copied from an external S3 Storage",
    "origin_url": "https://demo-files.gvideo.io/apidocs/spritefright-blender-cut30sec.mp4",
    "auto_transcribe_audio_language": "eng"
    "auto_translate_subtitles_language": "fre,ger,ita,spa"
}
Read more about it in the API POST /streaming/videos.

​
AI captions for an exist video

It’s possible to use post-processing for generating AI captions. Using new attributes for subtitles API you will be able to create subtitles automatically. AI caption parameters are the same for uploading:
  • auto_transcribe_audio_language: auto|<language_code> – for transcribing.
  • auto_translate_subtitles_language: default|<language_codes,> – for translating.
Example to transcribe: 
POST https://api.gcore.com/streaming/videos/{video_id}/subtitles
{
    "auto_transcribe_audio_language": "auto"
}
Example to transcribe from a detected language and translate to “fre,ger,ita,spa”: 
POST https://api.gcore.com/streaming/videos/{video_id}/subtitles
{
    "auto_transcribe_audio_language": "auto"
    "auto_translate_subtitles_language": "fre,ger,ita,spa"
}
Read more about it in the API POST /streaming/videos/{video_id}/subtitles.

​
Advanced use of native AI API for any video inside or outside the video streaming platform

Methods above are for videos stored in the Gcore Video Platform only, because they were designed for simplicity. Using AI API directly you can process any videos stored inside or outside Gcore Video Streaming Platform. AI ​​system can process any video stored in an MP4 container and available via a direct download link. Please note that links to YouTube, Vimeo, etc., are not links to a video file, so they cannot be used for processing. If the video is from an external source, get a link to it. The link must be in HTTPS format and the file must be available for downloading without authorization. Let’s look at an example of getting a link from our video storage. To obtain an MP4 link to your video with 240p, 360p, or 468p quality. That quality is sufficient because it contains an 64Kbps audio track suitable for AI automatic speech recognition and transcription. To get your MP4 link, execute the following API request /docs/api-reference/streaming/videos/get-video : 
GET https://api.gcore.com/streaming/videos/{id}
From the response, copy the value of the mp4_url field.
WarningIf the mp4_url field isn’t shown in the response, contact technical support. MP4 support may be disabled for your account. Technical support will enable it for you.

​
Step 2. Assign a transcription task to AI

To assign a transcription task to Gcore AI ASR, execute the following API request: 
curl -L 'https://api.gcore.com/streaming/ai/transcribe' \
-H 'Content-Type: application/json' \
-H 'Authorization: APIKey {your_api_key}' \
-d '{
    "url": "https://demo-files.gvideo.io/{your_video_url/{quality}.mp4" 
}'
Ensure you replace the example values {your_api_key} and {your_video_url}/{quality}.mp4 with your actual API token and MP4 video URL. In the response, you will get a task ID. It’ll come in handy in the next step. For example: "task_id": "abc12345-6def…"

​
If you need translation

As well as original-language transcription, we support subtitle translation in 100+ languages. The full language list is available in the API documentation. Specify the language explicitly in the audio_language attribute in the body of the API request, and Gcore AI ASR will use this info to create subtitles. If this attribute is not set, AI will run auto language identification, and the subtitles will be in the original language. By default, the transcription language is the same as the audio language. In the example above, the video was in English and the desired subtitle language was also English, so there was no need to set the subtitle language separately. However, if the language of the video differs from the desired subtitle language, you need to add to the request body the "subtitles_language" parameter. So, this is how the request body will look like if the original language is not English, and subtitles are desired in English: 
curl -L 'https://api.gcore.com/streaming/ai/tasks' \
-H 'Content-Type: application/json' \
-H 'Authorization: APIKey {your_api_key}' \
-d '{
    "url": "https://demo-files.gvideo.io/{your_video_url.mp4}",
    "task_name": "transcription",
    "subtitles_language": "ger"
}'
You can skip the "audio_language" field, and Gcore AI ASR will detect it automatically. For the “code-switching” feature mentioned above, audio_language must be skipped.

​
Step 3. Get the task result

Wait a few minutes for the AI to complete the transcription and translation. To get the task result, execute the following API request: 
GET https://api.gcore.com/streaming/ai/tasks/{your_task_id}
You will receive a response containing the transcribed and translated (if set) text. The following response elements may be useful for further video processing:
  • “subtitles” array contains transcription divided into individual timecoded sentences. It includes "start_time", "end_time" and "text" fields.
  • “concatenated_text” field contains the full audio text. It can be used for robot scanning, Google indexing, or summarization using a service like ChatGPT.
  • “vttContent” field contains marked-up text that can be saved as a .vtt file and added to any system as ready-to-use subtitles.

​
Step 4. Add AI ASR subtitles to videos

1. Copy the value of the "vttContent" field from the previous step. This contains the subtitles for your video. 2. Add the subtitles to the video by executing the following API request: 
curl -L 'https://api.gcore.com/streaming/videos/{video_id}/subtitles' \
-H 'Content-Type: application/json' \
-H 'Authorization: APIKey 1234$abcd...' \
-d '{
     "name": "English (AI-translate)",
     "language": "eng",
     "vtt": "{your_vtt_field_content}"
}'
That’s it! You can now generate subtitles using AI, add translations (if necessary), and upload subtitles to your video.
InfoWe are working on fully integrating subtitle generation into the video upload process. Soon, you’ll be able to generate subtitles using unique attributes in the POST /videos method. Stay tuned!