Chicago World's Fair Science Finds, Industry Applies, Man Conforms

AI and Video Editing: A Very Sharp Two-Edged Sword

Science Finds, Industry Applies, Man Conforms. Those words, the celebratory motto of the 1933 World’s Fair in Chicago, expressed a revealing truth that holds true with today’s AI technology.

As with most new technologies, artificial intelligence (AI) is a two-edged sword that cuts both ways. It has both a positive and negative side. Because it is so new and widely misunderstood, the negatives often scare us into thinking it’s going to take our jobs.

It’s a valid fear. Every new technology has a bias and carries an inherent message. None is value-free. All have social and political implications inherent in their introduction. After a new technology is widely adopted, there are always winners and losers.

For all creative people, the ability to question and evaluate new technology is an important survival skill.  Only by moving beyond the marketing speak associated with technology introductions can one begin to cut through the distortions and define the tradeoffs in accepting it.

AI is especially difficult to assess because it is ever-changing. What’s said about it today may not hold true in the future. The unknown future is a big reason it frightens us. But, as of now, AI hasn’t come close to replacing human creativity. When used wisely, it actually can help.

Although AI is impacting virtually every profession, it’s making a thorough presence in the video and audio post-production industry. Some television equipment vendors use the term “AI” as a kind of vague, magical, solve-anything term to describe their products. But that’s marketing talk.

In essence, AI is the ability of computers to perform tasks that are typically associated with human intelligence, such as learning and problem-solving. It can amaze with what it can do, but it DOES NOT replace human intelligence. In today’s post-production environment, it is smart to use the best parts of AI to ease our work load, but at the same time keep a wary eye on its development. AI offers real aids in reducing the mundane labor normally needed in post chores. It is offers time-saving tools in both video and audio editing.

On the video side, AI uses algorithms that can easily identify and extract the most relevant parts of raw footage. This can be people, objects, background scenes or any desired visual element. Once these relevant components are identified, the AI automatically assembles the footage into a coherent video program.

AI object and scene recognition technology can edit action or sports footage, and can highlight key moments and create highlight reels.

In filmed car chases, an AI algorithm can identify when the cars are accelerating, braking or colliding.

Some contents or functionalities here are not available due to your cookie preferences!

This happens because the functionality/content marked as “Vimeo framework” uses cookies that you choosed to keep disabled. In order to view this content or use this functionality, please enable cookies: click here to open your cookie preferences.

Blackmagic’s DaVinci Resolve and Apple’s Final Cut Pro use AI to do object detection, along with many other features “smart” features.

For certain kinds of basic productions, this can save the editor from manually having to navigate through hours of footage to do a rough edit.

Facial recognition is also an AI-driven feature. It can identify individual faces in footage and track the facial movements throughout a video. This can be useful when editing interviews in documentary or news footage. For example, Nova AI automatically detects and tracks faces in a clip, making it easier to edit and add effects.

Nova AI’s Smart Search can detect faces of specific personalities, catologue them and retrieve them in a search

AI can identify when each participant is speaking in a video, then automatically switch to a camera angle that provides the clearest view of that person. This can be a major time saver. Crestron makes automatic intelligent video and audio switching systems.

Another place AI excels is color correction. AI can automatically adjust colors and exposure levels based on the content of the footage. It can correct white balance, skin tones and saturation, and help keep the color and tone consistent. Even Apple’s free iMovie editing software within Magic Movie does this at the click of a single button. also offers an AI program that can automatically match the rest of the scene to a key image or shot and do it much more quickly than a colorist could do on their own — i.e., in seconds rather than a half hour.

Video stabilization is another function of AI. It works by analyzing the video frame-by-frame and then detecting and tracking the movement of the camera. Complex calculations determine the movement patterns and identify areas that need correction. It then does the necessary cropping, scaling and warping of the video to compensate for the movements. 

AI video stabilization can handle rotational movement, linear movement or complex multi-directional movement with great precision. Virtually all major pro video editing software now includes video stabilization.

Boris FX CrumplePop plugin eliminates unwanted audio with EchoRemover’s AI technology

On the audio side, AI can automatically remove background music or noise and enhance voice clarity. Plugins like Boris FX CrumplePop can analyze an audio signal and identify the unwanted sounds, such as hum, hiss or clicks. It can remove these sounds while leaving the desired content intact. It uses advanced filters and equalizers to boost the voice while reducing distortion.

AI can convert spoken words in a video into text. These tools transcribe speech accurately, regardless of the accent or style of the speaker. For example, Adobe’s Sensei AI technology in Premiere Pro can generate a transcript and edit the video by creating a rough cut. All the editor has to do is copy and paste.  

Adobe Premiere Pro’s text-based editing eliminates the need to watch hours of footage just to find the right sound bites

Text can also be used to create closed captions, transcripts or subtitles for the video within apps like Nova AI. Also, having a transcript of the spoken words in a video can also make it easier to find specific keywords or phrases within hours of footage.

Composers can use an Avid’s tool for Sibelius, Chord Auto-Complete, to gather a wide array of potential sounds to integrate into their scores, creating innovative soundscapes. Editors can use AI-generated scores through Dynascore to aid the creative process along, rather than inserting blocks of stock music. AI-generated scores dynamically adapt to what the editor does.

In Dynascore, creators can customize the track to use in their videos, completely royalty-free.

All of this AI technology is now available in editing tools. None replace the editor’s unique creativity. But it does take a lot of the drudgery out of the time-consuming parts of the video post process.

As for the future, no one knows where AI is heading. But we can assume it will progressively get better and better. That’s why smart creative people are keeping a close eye on AI.

The writers and actors currently in the entertainment strike have rightfully made AI a major issue and are attempting to create job protections for the future. Predicting the future of any technology has always been a tricky business. But understanding it’s dual-sided nature is a must. That has been proven over and over throughout history.

Today, AI is taking much of the repetition and rote work out of post. It is now easier than ever to make professional video productions. That’s the good news. The bad news is we cannot predict how long it will stay that way. Every new technology can have a negative effect and it is smart to be aware of the dangers lurking ahead. There will be winners and losers — the trick is not to be a loser.

Sony’s Creators’ Cloud now enables Cloud upload of photos and videos shot with FX3 and FX30
Sony Electronic's cloud production platform Creators' Cloud now enables Cloud upload, which …
Maxon Release Offers Apple M3 Optimization, Streamlined Workflows and More
Redshift optimization for Apple’s M3 family of chips, improved C4D workflows for …

Enjoying the news? Sign up for the Creative COW Newsletter!

Sign up for the Creative COW newsletter and get weekly updates on industry news, forum highlights, jobs, inspirational tutorials, tips, burning questions, and more! Receive bulletins from the largest, longest-running community dedicated to supporting professionals working in film, video, and audio.

Enter your email address, and your first and last name below!

Sign up:

* indicates required


We use anonymous cookies to give you the best experience we can.
Our Privacy policy | GDPR Policy