A New Comprehensive Solution for Video Subtitles Translation

Igor Afanasyev 00:01 All right. So today we'll be talking about translating and subtitles challenges around that and how SmartCAT platform can help you with. With those challenges. The way we want to structure this webinar is the follows, I will give just a quick short overview of what smartcard is, from our perspective, I know that many of you already know and love smart cat, as you already tell us in the chat. Just to give you a quick overview of what smart get is, then we'll dive into the main subject to saving media, what's market does and what market does not do get. For that, we'll have a short demo of how subtitles and subtitle previews are currently working in this market. And then after that, we'll dive deeper into specific aspects of SmartCAT and how they will help your work with subtitles efficiently. The last slide would be before q&a session, the last slide would be what the future holds for media translation smartguard, what we are up to what we are planning to deliver in the near and not so near future. So let's start. First of all, what is market? smartcard is an all in one localization platform, and many of you are using it in a different way and only some aspects of SmartCAT. But what we want to have and how we want smartcard to be perceived by our clients is that it is a technology platform that also has a strong human marketplace component. So we provide we give you an easy way to do manual and automated translation, and we're focusing on automated translation. Mostly this time, because we believe this is the future. This allows you to really reduce your turnaround time and scale and increase scalability. We have a strong marketplace component where you can invite anyone from our marketplace from freelancers to language companies, where you can also invite colleagues and other suppliers to the platform and freely freely work on your content. And smartcard also allows you to really streamline the all the burden of managing multiple people and companies on the platform by unifying the payments components, so you don't have any paperwork and you have a single binder and single agreement with market. We also are pushing towards having much more integrations and automation, we believe that this is the growth factors for our clients and something that gives them a lot of value. Now let's dive into media translation. What we typically talk about like media translation is translating videos making your videos multilingual, and what we do with SmartCAT. And what you can do with SmartCAT in terms of localizing media, efficiently, localize and translate subtitles, how this usually works. So first, you have your video file, it can be your file, it can be something that's sent to your client. Now, what you need to do in order to translate the textual content is to have or create subtitles. So creating Subtitles can be done automatically is something that is also the transcription. But you can also start with the initial script and then use external to, to prepare those subtitles for the video. So once you have your video file, and then you have your subtitles file, you can upload both to SmartCAT platform. And we're going to the right hand side here. This is something that I will be showing you today. So you're uploading your subtitle file, you can upload your video file as a reference file, and then you can essentially use them together to get a rich preview of your content and then deliver multilingual subtitles is now some of us in some of our clients even don't know what the titles are. This was kind of a revelation for me, but people are usually thinking in terms of like we have videos, we want to have them speak multiple languages like what exactly are subtitles, and I will be talking about this just to just to present some terminology that I'll be using during the demo. So you have your video file, and you want it to be available in multiple languages. The easiest way to do that is to overlay some text that will represent what people are talking about in this video. So you have the video and you have your subtitles overlay. Sorry about that. Ah, this is typically rendered automatically by, by the video player. For example, if you go to YouTube, you can enable or disable subtitles and switch between subtitles. So it is a layer on top of failure of video that synchronizes some text with certain timeframes of the video. The subtitle file is essentially a textual file that is really simple in the structure, it has a list of so called queues. And those queues just have the, the number of that queue, the start time and the end time. This is where the subtitle text will appear in the video down to milliseconds. And this is where it will go off the screen. And then just the plain version of the cue text. So something that you see on top of the video. Now, it's demo time. Kate Vostokova 05:50 I'm sorry, sorry, for a sec, because I remember that we agreed if we receive any comments or questions during the presentation, we will just answer and command them back. Straightforward, like not leaving them to the end. And I have a command here from Elena, she's asking, is it only mp4 format or AVI format as well, it's referring to the slide that you showed like before? Yeah, so Igor Afanasyev 06:15 the original video can be in any file format, there are multiple file formats supported. For SmartCAT. Specifically, to preview your video insights market, you need to use a video that is supported by your browser. So mp4 format, is this universal file format that is compatible with the web technologies. And our current restriction is that you're uploading the file in mp4 format. But the original source one can be anything. Kate Vostokova 06:46 Okay, thanks. I hope that answered your question. So let's move to Igor Afanasyev 06:50 Yeah, by default, if you use, for example, some external tools like like YouTube to create subtitles in the first place, then you can upload files, I believe in multiple formats, and then download is simply fine, you know, like, lower resolution version of your, your video in mp4 format, so it's suitable for uploading into this market. All right, let's do a quick demo. So here I have my SmartCAT interface, I will create a new project and then I'll upload subtitles into it. So let's do some very quick things, I want to create a project. I will name it the demo. Webinars, I can find that, and then I will be translating this from English into Spanish and German. I'm selecting machine translation. Again, it's the easiest way to get the initial result. And then on top of that, you can alter the translations, you can start with just human translation. If you're translating movies like highly important content, you may want to do manual translation. But we'll be doing machine translation, just for the sake of the demo because I don't speak Spanish or German. Now, that's pretty much that I need to set up a new project. And once I'm done, I am presented with a screen where I am uploading the files. Now I'll pick the file here. And this is a pretty short, short subtitles with our promotional video for SmartCAT. So I'm uploading that click finish and this file is going to be processed with our automatic translation. So it will be translated into two languages. Now what I will do now is I will go into the file section and then open that file in the editor so you can see the preview and what are the next steps. Here's my file. I am translating that into Spanish and German. I am opening Spanish as an example here. And we open the editor and we immediately show you the preview of of subtitles. Now you don't have any video yet. So what you're seeing here is a subtitles track that you can scroll that you can playback, it will respect the timing so you can at least see how fast you need to read any portion. Any cue that appears on top of the video. Well, the problem is that there's no video no audio so you don't have additional context here. But here's the interesting thing. We suggest you to upload or select the video for preview. What we're doing is here is we are trying to associate this specific subtitle file with the reference video file in mp4 format that is stored in this specific project. Currently, it's a blank project I just have this battle file there are no reference files, so I can go to files. We'll open this in a new tab and we suggest you to add some reference files I I'll be uploading my mp4 video format, paedophile in mp4 format. And I'll click finish, it will take a few seconds to upload unfortunately, and this is something that we're working on, we don't show you currently the progress of uploading reference files, which is unfortunate for mp4 files, which are quite big. But after a few seconds, what we'll be doing is, we will have this mp4 file available for you as a reference one. And here you see it being uploaded, well, it's already uploaded in the project, you'll see this as a reference file. In the list of files, by the way, you can hide them, if you don't want them to be displayed all the time, you can hide those reference file and see just the content that needs to be translated. But you know that it's there. Now I can close this stuff that I opened and then go back to the, to the editor. And we immediately detect that there is another there's a reference file suitable to be linked with this preview. Now I click on that, and this creates this association between between the SRT file and mp4 file. By the way, we have two modes of presenting presenting subtitles, we have two tabs here. Here's the source preview. And here's the target for you. Let me reload the screen just quick. So we don't see those two tracks at once. So source preview, what it does is basically shows you the source text. So no matter what you're typing in the editor, you will be always seeing the original file the original video with original text so you can fully understand what people are speaking about in case. Subtitles do not exactly match the spoken language. This actually happens if somebody is talking pretty quickly and subtitles are being edited just for people to be able to read the context, you can play back, you can play back the file. You won't hear that in the presentation. But I'm hearing the audio here from from this file as well. This gives me a full context of what my source, what my source is, and I can understand what I need to talk about. And also by the way, if you're scrolling, and you're moving your mouse around this list of cues, were also showing you the timing and also the speed at which you need to read this text. So characters per second is the labels that you see, they range from from blue to green to yellow to red. Depending on how fast you need to read a specific cue. And this also gives you additional context of whether specific units to be edited, or like will need to become shorter in the translation so that people can at least at least read what's on the screen. So this is the source preview. And this is not the funnest part. The first part is the target preview. What we show here is the final translation that you currently have have here in the editor. And now if I edit something here, you can see that this change immediately is reflected in this queue, over here in the list and also on top of the video. So even before downloading the file, and then looking at subtitles and videos and uploading them somewhere to see the final product, you basically see this final result, right as you type in the translation. So this gives you an we are also updating this, this label. So you can see that as you type in more text, you will see how fast a end user needs to read that text on the screen. In interesting observation that sometimes you do not know about and don't see that is that basically what we're doing with cues is we are combining them into full sentences like here, for example. We have two cues here, because we want to present text gradually on the screen. But in SmartCAT, you have this as a single as a single segment. And why we're doing that is to help with automatic translation and help you to reuse those translations with translation memories, what I mean here is whenever you are trying to translate a specific cue. So So you are looking for a solution this is not a complete sentence. You cannot efficiently translate this with machine translation. And if you provide if you need to provide translation into other language the ordering of texts can be different. And what this means is that your translation memories will be having source text and target text which do not correspond to each other. So this is really hard to, to reuse those kinds of translation memories in the future. And they just don't make sense. They can like pollute your translation memories with low quality, low quality entries. When we are combining those cues into full sentences, we are able to pass them through machine translation. And also, it's easier for you to kind of review the whole sentence provide the proper translation, just leave the markers, where you believe the cues need to be split apart or where you need line breaks. And then we'll be rendering them properly as cues here in the preview and in the output file. So this, we believe, allows you to really easily translate any kind of content. And the quality of translation will be much higher just because you immediately see the final output. Now, this is all about the demo that I currently have. I will begin by going back to slides. But before we continue with the slides, I will take some questions. And I know that there's something in the chat already. So Olga asks, How can I create a source SRT file if the file is not uploaded on YouTube? That's a great question. There are multiple ways and multiple tools available that allow you to author subtitles. So you upload the video and there are online tools, like professional offline desktop tools available for editing. For online editing, you can, if you don't want to use YouTube, by the way, it's the easiest option and the best option. Because it's free. You can also use amara.org. This is something that I personally use in the past, they have a friendly Title Editor where you can put your your script and then split that into queues and provide all the timings I bet people from this audience can put in the chat some of the names of the tools by the way, it will be interesting to know which tools you are using to author subtitles, and then we can read through them and then tell you all about that as well. We have something in the chat. Okay, so I guess we went through all of the things, all of the questions in the chat. Right. Any other questions so far? No. Okay. Let's continue with the slides. And I will explain more about the features of SmartCAT and how they translate to creating efficiently creating and managing subtitles on the platform. So this is the slide that you already saw. These are kind of the main pillars of our platform, you have automated translation, you have a strong marketplace component and manual ating component. We can automate payments, and we can provide regular use integrations and ability to do additional automation on our platform. So let's dive into those topics. So for automated translation, you have access to all the major professional machine translation engines. So it's Amazon, VEDA, Gmail, Google, modern Mt. Yandex. We work with intento as a MT aggregator as well. We also have an ability to plug in your custom trained machine translation engines. So if you have huge volume of translation, and you have budget to train your your Mt. In the corpus of text that's needed to be trained, that's something that we can plug into SmartCAT as well. By the way we do machine translation engine training by ourselves. If you have those steps, we can help set up like a custom Google or ML model. Plug this into your account so you can use that with your with your day to day translation work. beyond just being able to select any machine translation engine that you want. By default, what you will be getting is what we call an intelligent routing. This means that when you're creating a project and say you want to translate this from English to Finnish, Spanish, Russian etc, what we'll be doing will be automatically associating those language pairs with a specific translation engine that that we believe will work best for this language square. So for English to Spanish will for example, use deep L or or Google for English to finish will use mortar and T for English to Russian will use Yandex. But he can change this in project settings. So we will provide you the defaults that we believe will work best. But you can say that you want to translate everything through Google, for example. And also translation memories. So the way it works is, whenever you're translating your repetitive content, and again, I explained why we are taking this extra step of recombining all the cues in subtitles into full sentences. So this allows you to really have a quality translation memories accumulated as you're translating your content. Next time you're translating your repetitive content, the similar content, we will be pulling those those translations from translation memories. So essentially, the platform kind of like improves for you and learns from your edits over time. Not on the marketplace and human editing component. So SmartCAT, unlike other translation management platforms, doesn't have any limit of seats, or the the the overall volume of translation, or the overall volume of text that can be stored in your account. You only pay for automated translation. So whenever you're using translation memories, reusing translation memories, or using machine translation, this is the amount of words that are counted towards your order. And even free users can get 10,000 words for free each month. So they can start using the platform, understand how machine translation works for them, and then decide if they want to go to to pay subscription on top of that. And by the way, unlimited users means that you can invite anyone from your team to do some post editing to do some manual editing. And you can invite also any language companies that you work with, freelancers, etc. But we also have freelancers and language companies registered in SmartCAT. And we have more than half a million freelancers registered and more than 5000 language companies on our platform. Now, how can you navigate this sheer amount of you know, of workforce? Well, we have algorithms that allow us to analyze the content that you're uploading, and also compare that to the previous content that was translated by specific freelancers. And what we're doing when you have this, when you have your content uploaded into a project, we will be picking from the entire marketplace, we'll be picking freelancers and highlighting them for you, which we believe like that have a best match with your content and also have good reviews and scores. So we will be surfacing up the best, the best freelancers form of the marketplace, and you can easily invite them to work at your products. What this means for you, again, if you have large volumes of translation, you can easily scale up your operations and SmartCAT just because you can hire as many people as needed to deliver regular translations on time. On top of that, you can also say that you want SmartCAT to be responsible for the for finding and inviting freelancers to your projects, and then be responsible for the quality of the content. And this is an option that we offer to our base subscribers. Basically, we can deliver projects on time with with good quality. And essentially manage those projects by ourselves if you don't have to, if you don't want to go and invite your freelancers and find find people and lead them to the platform automation integrations what this means. So what I demoed today is an ability to manually upload the file, upload the reference file and then start the translation. Once the translation is done, you're manually downloading the resulting subtitle files, multiple languages. And then you are doing whatever you need to do with those subtitles. Now, this is a manual process, if you have consistent volume of translation is going on and you want to upload files and download files multiple times a day, for example, you may want to think about automating this process. So we offer some pre built integrations for content management systems and repositories from where we can get files and upload them automatically to SmartCAT. Or you can use our API, our public API to do any custom automation. So the ability to create pro projects automatically upload files, then wait till the project is completed, download the files back and then stored them somewhere else. This is something that you can automate. You can also automate workflows. So in addition to machine translation, you can have a editing like human editing step and then more steps if you need to do like multistage review for some sensitive content. So you can set them up on a per project basis. And this allows for multiple people to be working essentially, in parallel or in like staggered fashion. And whenever the first segments are being have passed through for the first month later, they will be automatically appearing as a part of the job for the second one. So in that case, you're not wasting your time and your do not have to send your files like download files, send them to the another person to do another round of review. So our platform can automate that. And also, the last but not least, on this slide is that we can always help you to create some specific workflows, automated workflows, so we have a service of localization engineer. So they will, they will understand your needs, they can support, they can create some certain integrations or automation configuration files, to work with automatic content delivery from your repositories, but it will also have an option to to create custom integrations upon our clients requests. And the final thing about about SmartCAT, and why it's great to translate subtitles and any other type of content. And we support lots of different content from from Office file formats to, to, to developer friendly file formats. Regardless, it doesn't matter how many suppliers you have on the platform, how many people are working in this specific project. And if on this project, you work with one set of suppliers and the other project has a different set of suppliers, you can invite as many as you want. You don't have to actively manage, like payments for those suppliers. You don't have to sign individual contracts with language companies who work on our platform, what you have is a single, single contract with SmartCAT. And a single bill from SmartCAT. Like every month, we're billing you for the amount of money that you need to pay to your suppliers based on the work that they have done. And we will be disburse the payments in local currencies with all local regulations. So you don't have to worry and understand all the intricacies of you know, accounting with those individuals. So yeah, this allows you basically to reduce your costs, and reduce your management time. And really, really speed up the process and really tap into this ability to work to use freelancers at competitive prices, competitive prices, and with like best scalability. Now, the last slide will be what we want to do with this ability to work with media file formats, and SmartCAT. So what you saw right now is an ability to just upload mp4 files, subtitles, and then translate them on the platform. Then you get the translated SRT files back and then all the next steps happen outside of the platform. The creation of subtitles currently also happens outside of the platform, but we want to bring that into SmartCAT. What this means is, whenever you're uploading an mp4 file, we will be able to automatically run the transcription. So extract the text and timing from your video and create the first version of your subtitles. Then you will be able to edit them on a platform provide, like tweak the timing, tweak the text, make sure that it looks good in the source language, so that then you can start the actual translation process when the translation process happens, in addition to being able to download the SRT files, if that's like the target format that you're expecting, you will be able also to burn your subtitles into videos by burning. If you're not familiar with the term it means that those subtitles the text will be actually rendered into video frames. So then you when you have the source file, for example in English, the source video file In the end, you can download the file in Spanish and German, the video file in those languages. And when you play that file, you'll already see the subtitles on top of the video. So you don't have to use any specific like video player or you don't have to upload the files on to YouTube to preview those videos, you can just simply send the video out to your, to your users. So additional thing that we will be exploring in the future, and they were like looking into that right now, as well, as far as technology goes, is not only an ability to burn in subtitles, but also render the text using text to speech engines that are currently available, and are getting better and better render them as an audio track in those in those videos. And also add them to to videos and get the dubbed video back. This for instructional videos. For example, it will allow you to allow your users to just hear what what people are saying in the video in their language instead of having to read through all of the text. Sometimes when there is a highly like intensive visual information in the instructional video where you're displaying something on the screen, or how it certainly works, you want to draw attention to the actual picture not to the text below. So this is some of the options that we are exploring as well. And with that, I think we're done with the presentation. So I'll be happy to do to answer some questions from the Kate Vostokova 31:44 we do have a couple questions in the chat. First one is from Anna, she's asking is it possible to add subtitles if we need an F and subtitle and sorry, I'm not familiar with the terminology, probably, you know, what does it mean? Igor Afanasyev 31:59 Yeah, maybe if you can explain what that means. It's also doesn't ring a bell for me. Kate Vostokova 32:04 Okay, and I just raise your hand and I can actually you can write it. Or if you want, you can just join our life and we will talk on this. Igor Afanasyev 32:13 I just I just realized that I am not that expert as I thought I was the webinar. So I don't know what Fn is sorry. Kate Vostokova 32:23 Okay. Christina is commenting that she's tried to upload an mp4 video was referenced while but apparently it was too large. And this could be a problem with mutagens, like movies, titles? And do you have any suggestion to upload larger videos. Igor Afanasyev 32:40 So yeah, you have options. If you are uploading a video file, and let me probably show that to you real quick. When you are uploading your documents in SRT file here, hold on, you also have an option to provide. Show. So providing a URL to a hosted video online. What this means is basically you can use like if you already store large movie files elsewhere, for example, like Amazon, Amazon storage, Amazon s3. So you can provide a link to the direct view link to the mp4 video. And we'll be using that instead of you having to upload them to the platform. This should really help with with large files. We are also working on making the process of low uploading large files easier. So you at least see what's going on what's the progress of uploading. And that will really help with larger files. For you. Specifically, if you can just send us the link to the file to download we can set up the project for you just for Google to to evaluate how this will look like on the playground Kate Vostokova 34:05 we have another question from Vladimir, can we change the color of subtitles when using smart kid? I don't know why. Why it's needed to the colors to be changed, but probably there isn't. Igor Afanasyev 34:17 Yeah, usually usually it's used to denote like different different people talking, you know, like colors are able to separate. Separate different like lines of, of story on top of the video. We don't have a way to author anything right now beyond just providing the translations. So that's something that can be done in post editing post translation step as you download the subtitles back. You will be able to do that. We're also supporting VTG as of like pretty much last week, so VTT is WebVTT web video subtitle A format that allows you to have some extra information, some extra formatting for your subtitles. Now, we're not rendering those for translation purposes. But if you're uploading a video if you're uploading subtitles and vtt format that has some formatting, you will be getting the formatted result back. And then ultimately, you can you can tweak that in a an editor that supports Conditional Formatting. Kate Vostokova 35:32 We have some context for for my now regarding upon subtitles. So either these are subtitles that are added as translation of some narratives on screen like signs or descriptions, so forth narrative subtitles. This is what yeah. Is it possible to add them? So the question was, Igor Afanasyev 35:53 yeah, this is about, okay. This is about essentially about what we call source editing, in the ability to add something extra, as a source, or as a as something that was not there in the source subtitle file correctly, it is not possible. But this is something that we're actually working on right now and ability to eight source. So with that, you will be able to add a new cue and provide timing for that. And you will be seeing that on the screen in the source language, and then you will be able to provide translations for that in your target language. Kate Vostokova 36:30 Okay, great. Okay, any more questions? By the way, if maybe it's too hard to put your question in return, because it requires a lot of maybe context or whatever, feel free to raise your hand and I can turn on your microphone. And you can ask, tell us your story. Tell us your experience, your challenges, and I think Google can help and answer. But if we don't have any questions or any hands. Thank you, Anna. Thank you for your question. Yeah, here's the Eagles emails. So try this, translate with the subtitles, give please your feedback, you can send it directly to Edgar to his team. And I think any additions that we need to make will be done. I will have one more question. Is it possible to edit timings for the SLT to meet any special requirements like gaps, etc. Igor Afanasyev 37:31 Great question about the same thing of editing ability to edit something at a timings. In addition to providing translation, this is something that we're looking to add in the future. Right now, it's, you will need to do this. When you're done after you have downloaded the template files back. Or if you want to edit the timings in the source, that's something that you will need to prepare before uploading to SmartCAT. But really interesting thing, like if you need if there are some specific requirements about gaps, etc, we should be able to at least display them in the preview. So you can see that subtitles are meeting your requirements. This is an interesting thing to explore. And feel free to reach out to me by email and provide your insight and how you're working with suppliers and what you'd like to see in the preview etc. So again, we can incorporate that in the future iterations of this functionality. Kate Vostokova 38:34 Okay, so I don't know should we robot because actually, we are slightly timeout already. And I hope we answered all the questions. But if you have more, feel free to reach Eva. Let me also send this email to the chat so it will be much easier for you to write him. Okay. Um, okay, thank you. Thank you for this presentation. The recording will be available soon, guys. I will send it out to you and maybe 20 minutes. Thank you there. Thank you for attending this webinar. I hope it was useful for you and try, please try our translation of subtitles and share your feedback. Igor Afanasyev 39:19 Yeah, thank you for being listening, and have Kate Vostokova 39:21 a great weekend. Bye, everyone. Bye