-
Generate subtitles from script instead of hoping for “AI” to “get it” from audio
Salut !
As professional film makers, I’m sure many of us start off the subtitling or captions process WITH the actual official proper clean script.
As of now, all options I find involve some kind of approximative “AI” analysis of the audio, resulting in very approximative text, with wrong pronunciations, names, punctuations etc…
I would have thought that by order of priority, before addressing this more complex method, the more basic method would be covered : instead of asking the robot to understand the audio, we give the text to the robot. His only job is to now align it with the audio, instead of having to figure it out.
Has anyone been able to develop a workflow, or even better a software that does just that ?
I am aware that Youtube, vs all other paid editing software, the irony, does offer that feature, but it will auto-align a text only in certain languages. It is also not a super convenient workflow having to go back and forth from editing software to youtube to do it.I’m on Mac, using Resolve, but I’m open to other softwares if it works !
Thanks for your insights ! 🙏🏼
—
Michael
Current work : ANINAGIN, the Aliens from Earth (self produced)
https://www.youtube.com/channel/UCC71RypUFAfLnXEVIzw-llA -
3 Replies
-
PS : The proper technical term seems to be called ” Forced Alignment” 🫨
-
Hey Jacob,
Sorry, I stored this away in a tab amongst 1000s on FireFox, and today as destiny would have it, it was presented to me.
You make some interesting points, and raise good questions.
It is only in scripted productions (mainly drama) that you get a script. Some larger organisations create scripts from their unscripted shows at the end of the Post production. But to match it to the spoken word might be difficult – although, I’m sure someone could write a plug-in for that.
Although I have not worked with it, yet (have got the licence), Avid Scriptsync might do what you are looking for:
https://www.avid.com/products/media-composer-scriptsync-optionOn a wider note about speech to subtitles, I am not sure that A.I. is the right wording to use in the context of what is effectively “Machine Learning”?
However, if you want to “reverse” the process. Where the SOT (Sound On Tape) is transcribed, then fixed for errors, and turned into a “new” voice that is inserted into the right places on the edit. And, spaced out to match lips. Then, you’ll get rid of the original speaker, but use A.I. to mimic their voice. Then you are looking at something very different, that might not be received well by the person with the actual voice.
Only a week ago Epic games, with their new A.I. voice bots, walked into controversy on the subject of replacing the original voice actor with A.I. (Although some might argue that it will take a shovel and some amount of raising the ead from the Grave, to get Darth Vader’s original voice.
https://www.bbc.co.uk/news/articles/cn844l3eznqo
Just an opinion.
Atb
MadsPS: Love your AninagiN trailer, will watch the movie!
avid.com
Script Based Editing - Media Composer | ScriptSync Option
Speed up script based editing, eliminating time-consuming manual media syncing & quickly find the best take fast.
Reply to this Discussion! Login or Sign Up