-
Spoken Word Search in Video Editing Applications
My name is Kip Watters – I am the Director or Product Management for Media at Nexidia. Nexidia is the company whose technology powers ScriptSync in Avid’s Media Composer. ScriptSync is one way to utilize Nexidia’s “phonetic search” capability – jumping to specific lines in scenes based on text selected from the shooting script.
I am very interested in gauging the interest in and ultimately bringing these types of features to the Apple community as well.
First, a couple notes on how Nexidia’s technology is different from Speech-to-text:
1) Rather than producing a text transcript, Nexidia’s indexing process (which produces a small index file for each media file) is several hundred times faster than speech-to-text. As an example, an hour long video can be indexed in several seconds with Nexidia. The index file is not human readable, but can be searched for any number of terms/phrases once it is created.
2) Phonetic search, unlike speech to text, does not require every word to be in the “dictionary” of possible terms for it to be found. This is especially important when searching regional or topical content with unique proper name and colloquialisms. Nexidia is also more tolerant of background noise and does not require a perfect recording to be useful.
I am also interested in hearing your thoughts on other ways that search of spoken word content in audio and video would benefit the professional video editing community.
For example, would having access to a search box through Avid client applications that located all occurrences of a word or phrase (anything you typed in) within a local and or network-based content archive – similar to a Google search for untranscribed audio and video – be valuable? Search results would jump you directly to each place in the audio where the search term was spoken.
This search would not rely on manual tagging, descriptions, captions, etc. of content, and is independent of any existing metadata (title, tags, descriptions, character names, etc.) that may already be searchable. In fact, dialogue/spoken-word and metadata-based search could be combined to ensure that all information, including what the actors say, is searched together.
Would an audio search be more useful in a particular client application that you use, and if so, which applications? Does a server-based search solution, able to access content on networked drives and from multiple integrated Avid tools on the network be more useful?
Do you have examples of jobs you’re working on in which fast, on-demand access to target content via keyword/phrase search would make it easier or possible to do things that you cannot do today?
-
18 Replies
-
That would be…pardon my LA vernacular… FREAKING AWESOME! SO many times I work on documentaries that have transcripts, but without timecode. So while I know what they say, finding that is, difficult, to say the least. If I could have a searchable way to find a word or phrase a person says, that would save TONS of time watching clips.
And if you can convince Apple to have a Script Sync feature too… I won’t look a gift horse in the mouth.
Shane
GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def -
Kip,
There is no doubt that, as Shane mentioned, having spoken text search capabilities in Final Cut Pro would be a real bonus for editors working on documentaries, and I wholeheartedly support any effort you and your company can do to bring that about.
However, while the advantages that your app has over speech-to-text apps seem very nice, your solution ultimately offers only about 50% of total equation that documentary filmmakers really need. Until there is a way to fully integrate searchable transcripts and spoken text search capabilities into a cohesive and comprehensive solution, documentary post-production will unfortunately continue to be both inefficient and antiquated.
Meanwhile, if there is anything I can do to help and/or beta test, please feel free to call on me anytime.
THNX,
DavidDavid Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los AngelesPOST-PRODUCTION WITHOUT THE USUAL INSANITY ™
A forum host of Creative COW’s Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.
-
Yeah, while I can’t imagine any sane editor that wouldn’t love to have something like this, its usefulness is somewhat crippled unless it also includes the ability to generate transcripts.
That said, even without speech-to-text capability I’d love to have something that was integrated with Final Cut Pro to allow me to speech search all clips in a project, or to restrict the search based on other metadata criteria like reel name, or to only search within a specific bin, etc. It would also be good to have the option to point it at a folder that contains media that isn’t necessarily imported into your project and search that way too. But the holy grail would be something that generates timcoded transcripts and then lets you search the transcript to locate bits of speech within the source media.
-
[Atticus Culver-Rease] ” its usefulness is somewhat crippled unless it also includes the ability to generate transcripts. “
I disagree. Now you are getting into something that Premiere does…which is FAR different here. Avid editors who load scripts into Script Sync will disagree with this feature being crippled. Heck, even doc editors who have transcripts (done the standard way, sent to a transcription company) find this VERY useful when on an Avid and looking for sound bytes.
And many time the “transcript” made by Premiere is nearly useless, due to the way the speaker talks. PLUS, this technology that he is pitching works one way, not the reverse. You want both ways. I can see wanting that…wanting to save a few bucks on transcripts. Still…”crippled?” I don’t see that.
Shane
GETTING ORGANIZED WITH FINAL CUT PRO DVD…don’t miss it.
Read my blog, Little Frog in High Def -
[Atticus Culver-Rease] “the holy grail would be something that generates timcoded transcripts and then lets you search the transcript to locate bits of speech within the source media.”
Yes, but it also has to do so with at least a degree of accuracy that makes it a time saver rather a waste of time.
If you check out my Cow review of Adobe Speech Search at https://library.creativecow.net/articles/weiss_roth_david/speech_search.php you’ll see that Speech Search is a great idea, but one whose time has not yet arrived.
David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los AngelesPOST-PRODUCTION WITHOUT THE USUAL INSANITY ™
A forum host of Creative COW’s Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.
-
[Shane Ross] “And many time the “transcript” made by Premiere is nearly useless, due to the way the speaker talks. PLUS, this technology that he is pitching works one way, not the reverse. You want both ways. I can see wanting that…wanting to save a few bucks on transcripts. Still…”crippled?” I don’t see that. “
Shane,
I agree with you both, but I still maintain that until the solution is does the transcript from the spoken word as well as any search capabilities that we are still dealing with an antiquated workflow.
David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los AngelesPOST-PRODUCTION WITHOUT THE USUAL INSANITY ™
A forum host of Creative COW’s Apple Final Cut Pro, Business & Marketing, Indie Film & Documentary, and Film History & Appreciations forums.
-
Maybe crippled is too strong… what the original post is proposing would definitely be cool regardless. I guess I’m just saying that in a perfect world you would have this functionality coupled with the ability to generate a text transcript. I know that Premiere does this, and from what I’ve heard their implementation of it isn’t that great. I am hopeful that someone else could do it better, because if it worked well it would be a huge timesaver.
-
Kip, Where have you been? For years we’ve been waiting to see Apple adopt a “script sync” like
feature for FCP. You ask are we interested? Of course we are. This is the best news I’ve read so far this year!Our company spends thousands of dollars a year generating transcripts of interviews. If we could leverage that investment and somehow make searches more efficient within FCP I would be the first person buy it. I’d be happy to have a more in depth, detailed conversation with you regarding your technology and how it would benefit our workflow.
Feel free to give me a call.
Mark Raudonis
VP Post Production
Bunim-Murray Productions
818 756 5100 -
YES, Kip Yes!!! Do it! As everyone has pointed out so far, we could use it in the FCP world. If you are looking for votes, you have another one.
Mitch Jacobson
Producer/Editor: Avid & FCP
New York City -
Hi Kip,
I am the product manager for a SaaS based digital asset management system. I’m very interested in this topic. Our customers are large brand name companies with large amounts of digital assets,….images, brochures, audio and video files…
You ask: “Do you have examples of jobs you’re working on in which fast, on-demand access to target content via keyword/phrase search would make it easier or possible to do things that you cannot do today?”
Answer: YES!
I give you a call….!
Reply to this Discussion! Login or Sign Up