Generative AI and professional post-production: Future fantasy or imminent reality?

Shared by Jigsaw14 Media

ChatGPT is not the first generative AI application. It’s not even the first generative AI application to be released by its creators at OpenAI, or the first natural language processing tool driven by AI technology.  But it is acknowledged to be the most advanced AI chatbot to date, and it’s release to the general public in November 2022 has escalated the conversation around the use of AI across all industries.  Other generative tools like DALL-E, Midjourney and Stable Diffusion garnered their fair share of attention over the last year, but ChatGPT’s ability to create pretty convincing written content, on almost any topic, based on natural language prompts, has captured global interest to an extent seldom seen before. 

So, what are the potential implications of this new breed of AI application for the professional post-production industry? We enlisted resident head of technical architecture at Jigsaw24 Media, Graham McGuinness, and indisputable expert in generative AI, ChatGPT itself, to help us figure this out by answering five (seemingly) simple questions.  

Question 1 : How is generative AI being used in professional post-production workflows today?

“What is being very effectively leveraged right now are AI capabilities like speech-to-text, automated tagging and facial recognition,” says McGuinness, adding that object recognition or machine vision is also developing at a rapid rate and becoming phenomenally powerful. He attributes ever growing training datasets and ever improving object detection algorithms with driving these impressive results.

What makes generative AI a hot topic now, according to McGuinness, is that “In 2023, we’re at the unintended intersection of near unlimited computing power, massive data sets and open-source ‘tooling’ that allows these capabilities to become accessible to wider sectors of the population.”

To this end, his team has already started experimenting with using ChatGPT to write code (as an intelligent assistant), debug webserver setups and help write comprehensive knowledgebase answers to support questions. According to ChatGPT, generative AI is currently being used to up-scale and restore low-resolution footage, for colour grading, to generate realistic visual effects and even to automate editing.  In professional audio post-production ChatGPT lists noise reduction, automatic dialogue replacement, music composition and speech-to-text as generative AI capabilities.

However, when asked to provide specific examples of generative AI tools used for video editing and colour grading, ChatGPT’s answer included products like DaVinci Resolve’s neural Engine, FilmConvert Nitrate, Adobe’s Autoframe and Avid’s MediaCentral Editorial Management tool – which prompted our next question…

Question 2 : What is the difference between AI and generative AI? 

ChatGPT answered this question with “Generative AI is a type of artificial intelligence that uses algorithms to generate new content, such as images, videos, and audio, that resemble existing examples. Unlike traditional AI, which is programmed to perform specific tasks based on predefined rules, generative AI has the ability to learn and create new content by itself.” 

This is an important distinction – because while most of the professional post-production tools that ChatGPT listed in the previous questions have powerful AI ‘trained’ capabilities, they don’t create new content in the same way that an AI-based text-to-image program does, and so don’t necessarily qualify as ‘generative’ AI applications. It’s a point that we felt was worth clarifying with one more follow-up question. 

Question 3 : Is it fair to say that most professional video and audio post-production tools don’t use generative AI capabilities at the moment, but that certain products use machine learning algorithms to improve functionality?

“Yes, that is a fair statement,” ChatGPT confirmed. “While there are some professional video and audio post-production tools that incorporate generative AI capabilities, most professional tools focus on using machine learning algorithms to improve functionality and assist with various tasks in post-production.  For example, tools such as DaVinci Resolve and Adobe’s Creative Cloud utilize machine learning algorithms to analyse footage and audio recordings and make intelligent adjustments to improve the overall quality of the finished product.” 

This is not to say that impressive generative AI video tools aren’t already available. Citing products like fylm.ai and ColourLab, McGuinness says “Some of their image correction capabilities are staggering, but the real divider is what they can do in the professional realm. The results are not always consistent.” Synthesized speech is another area that’s made massive strides in recent times, but McGuinness believes that we’re still ‘a way away’ from tools like these having a real impact on mid-to-high-end post-production workflows. This is, in part, because the process can be painfully slow.  But that’s not the only issue to consider…  

Question 4 : What are some of the potential issues with using generative AI tools in post-production?

Even though ChatGPT was trained using a database of around 300 billion words, OpenAI has been up-front about the fact that it may occasionally generate incorrect information. And this is true of any generative AI application. As McGuinness comments, “Any AI-driven process still needs to be assessed by humans to find errors. The net result is that tasks that deploy these tools will either be non-critical, or able to be corrected ahead of wider consumption. Many non-trivial tasks like translation and subtitling currently make use of AI tools and are only getting better.”  

ChatGPT weighs in with, “Of course, like any tool, generative AI is not fool proof, and it is important for post-production teams to use it in conjunction with human expertise and judgment. Moreover, generative AI tools are not a substitute for creativity and artistic judgment. While they can automate certain tasks and provide new ways to enhance content, they cannot replace the skills and intuition of experienced editors or sound engineers.”

McGuinness adds that there are also potential issues around security and copyright protection. With most generative AI applications hosted in the cloud, professional post-producers will no doubt have some reservations about the security of their media and the risk of rushes being leaked or copyrights infringed.

Perhaps more concerning, though, are some of the moral quandaries that generative AI tools present. As McGuinness points out, deepfake videos that are almost impossible to discern from reality have already been created using generative AI video and audio tools and this is only likely to increase as the technology improves and becomes more widely available.

Another issue he raises is around bias saying, “If you asked an image-based app to produce a picture of a football team then most of the results will feature men.  By using a database of historic sources to inform future content, we risk simply replicating or exacerbating current societal predispositions. That said, the broader social and societal aspects of technology are top of mind for many in this field aware of this challenge.”

Question 5 : What does the future hold?

Despite his aforementioned reservations, McGuinness still asserts that generative AI is more than ‘a cool piece of tech’ adding, “It promises to profoundly change the way we interact with the world because of its ability to (relatively) intelligently change or affect everything.” While he’s not concerned about losing his job just yet, he warns against complacency saying, “Five years ago people were quick to point out AI’s shortcomings but look at how quickly it’s progressed.”

ChatGPT is the first to admit that generative AI isn’t ready for the post-production big leagues just yet. When questioned about the future of generative AI in professional post-production it answered: “While generative AI has the potential to be a powerful tool in post-production, particularly for tasks such as automated scriptwriting or creating realistic CGI, its use in professional tools is still relatively limited. This is partly because the technology is still in the early stages of development and has yet to be fully integrated into professional workflows. However, as the technology continues to advance, it is likely that we will see more professional tools incorporating generative AI capabilities in the future.”

---

Related COW News

---

The final verdict

Our five questions have barely scratched the surface of this topic and, as it turns out, there are no simple answers when it comes to generative AI. As McGuinness observes, “The real challenge is that the more you find out, the less you know and it’s almost impossible to cover all the aspects.” 

What is relatively clear at this point is that, while the machine learning ‘garden variety’ of AI is being used quite extensively to create efficiencies and aid the creative process, there are still very limited instances of generative AI being used in professional post-production workflows.  All the evidence points towards this changing in the future, but the questions is, just how far away is that future? Judging by the rate of change we’ve witnessed while putting this piece together, it may be a lot sooner than some might believe.

As McGuinness puts it, “Is it interesting? Absolutely. Can you ignore it? Definitely not. Is it going to impact on your offering as a post-production facility? Undoubtedly.” Look out for more content on this topic from Jigsaw24 Media in the coming months.

Join the Conversation

Weigh in on the generative AI in post-production possibilities in the COW’s AI forum conversation!

###

Enjoying the news? Sign up for the Creative COW Newsletter!

Sign up for the Creative COW newsletter and get weekly updates on industry news, forum highlights, jobs, inspirational tutorials, tips, burning questions, and more! Receive bulletins from the largest, longest-running community dedicated to supporting professionals working in film, video, and audio.

Enter your email address, and your first and last name below!