AI and video editing

[Devrim Akteke]

Hi,

As with every new technological advancement we have to keep ourselves up to date. With the invention of video cameras, non-linear editing systems, smaller and cheaper computer systems; and developments with the software are getting faster every day. Lots of motion VFX work will be made easier than ever. We cannot be only editors and say I just edit offline and don’t do anything more. When I was 20ish years there were these older editors, mainly drama editors I know, they wouldn’t even put a dummy text on the screen. They would say this is not my job I only edit. We have to learn and adapt to these new software systems – AI. Otherwise, each member of the new generation will quickly pass us.

[Bob Cole]

I take your point, and if I were talking about something less fundamental, I’d leave it at that.

But this exposure to AI feels like something very different. We’re talking about a radical transformation of editing as we know it. We’re the taxicab drivers looking at driverless cars.

[Devrim Akteke]

I don’t feel like it is driverless cars yet. On the contrary, it needs a good driver to show the way. Roles are changing, those once who became motion graphics artists now will be AI artists and it will be easier to create. The human factor is still on the table as every creation process requires creative thinking and planning. AI still cannot create without humans showing the way. But who knows what will happen 20 years later?

[Bob Cole]

Devrim, yes, I posted this in hopes that I could be persuaded I was agree. And I agree with you about the importance of creativity. But you have to see this AI stuff to believe it. The AI software “read” the script, provided audio for the voice-over, looked over the images, and edited the images to the script. Then it said, in effect, “I can generate animated characters that will be more dynamic than the still images you gave me,” and produced a fully animated video, in seconds. It even figured out that there was a quotation from Richard Nixon and provided a (not great) rendering of Nixon giving his “two vee’s” salute.

The script was written by a human. But I bet AI could have generated the script from a set of bullet points as well.

The point is: It was GOOD ENOUGH. And for the price (zero, basically) it was amazing.

This reminds me of the endless film-vs-video debates. The privileged few can still shoot film, but for most of us, digital has won. I think this debate will be resolved a lot more quickly than that one. With this difference: Now it’s not just the recording medium we’re talking about; it’s the entire process, from concept to output.

To extend the film-vs-video analogy: Those who are talented enough to demand film will get to use film. The vast majority (including many very talented people) will use video. And with AI, the few who are supremely talented will be in more demand than ever. But the vast majority will find their roles severely eroded by AI.

Not a great commencement speech at an arts college, but it’s reality.

Bright spot: AI still can’t do humor. Maybe we should all go to improv school.

bob

[Hanno Hart]

AI can’t be ironic nor inventive. Its just an incredible huge plagiarism machine. It can imitate handwriting, but, as video artist Nam June Peik says: When too perfect, god is angry.

[Mary Walsh]

I think we’re going to have to come up with better vocabulary for this than just “AI” – AI covers everything from magic masks and noise reduction to the kind of stuff Bob is talking about, completely generated products.

I love AI when it takes the tedium out of sound editing or motion graphics. I despise it when it takes over search engines and provides erroneous answers. And I despair that clients won’t understand the difference between art and craft made by people and the same stuff copied by machines trained on it.

[Michael Gissing]

I prefer machine learning. It’s a much better descriptor and we will need to have the right words when machine learning becomes actually intelligent and genuinely creatively generative.

[Simon Morice]

This thread reminds me how often we’ve been here before. Whether it was the shift from film to video, or the move to digital that has all but erased the role of the assistant. I started on 16 and 35mm film back when syncing rushes, logging and managing trim bins was half the job. We’ve lost something, (some brilliant visualisers who thrived without WYSIWYG), but we gained a lot in speed, flexibility and access.

AI may push us to rethink the steps of editing once again. Can it do some grunt work – surfacing quotes that match narrative beats, sorting transcripts by emotion or theme? If so then maybe the editor’s focus shifts back to where it’s most useful: meaning, not mechanics.

AI is reiterative and not inventive. It won’t replace great storytelling. But it might clear clutter around it. And much like NLEs did, it could open the craft even further. YouTube already shows us how accessible video has become.

Not an end. A reframe. You are the artist – AI is the brush.

[Michael Gissing]

I’ve lived through the big tech changes of the past 45 years. I feel it only a few more years before we reach a level of machine learning that transcends into actual AI and at that point, we as artists will not be able to compete. I’ve been a champion of digital tech, being one of the first to do film sound recording using digital recorders in 1984. I’ve been a beta and sometimes alpha tester for Yamaha, Sony, dSP and Fairlight with the developments of the first DAWs and non linear video. This change is different and I don’t agree it’s going to be the same as before where technology replaced jobs through efficiency and raw processing power. This time the creative process is directly threatened. Ironically we have had a couple of years where machine learning has vastly helped us with masking, audio noise reduction, denoising, upscaling etc.

I’m not interesting in just writing a prompt for generative AI to do the actual work. After nearly five decades in the industry, I know humans will not cost compete and the end result, whilst inferior to a human for a while, will be perfectly acceptable for the vast majority. There will still be niche work, just like vinyl, film and live performance has but I don’t see how this time it will just be a transition with business as usual.

[Bob Cole]

Thanks for the thoughts. Some reactions:

1. Agree that there are many wonderful AI-aided innovations which automate tedious manual tasks; Resolve 20 has several of them.

2. Mediocre human-created edits will definitely lose out to mediocre AI, if only because AI will be faster and FAR less expensive.

3. For field production, I am looking forward to AI-driven lenses which improve AF tracking, even when the desired face target temporarily is hidden from view.

Outside of video production, I am all for advances in AI. For my personal life, I am looking forward to UAI: Utensil Artificial Intelligence. This will feature an AI-aided fork, with haptic technology which guides my hand to the vegetables and away from that extra pat of butter. Why take expensive weight-loss products when you can solve your weight issues at the source?