Tag Archives: Premiere Pro

How Doc Filmmakers Are using A.I. to Create Captions and Search Footage in Premiere Pro

Artificial Intelligence (A.I.) and machine learning are changing how video editors deal with some common problems. 1) how do you get accurate transcriptions for captions or subtitles? And 2) how do you find something in hours of footage if you don’t know exactly where it is?

Getting out of the Transcription Dungeon

Kelley Slagle, director, producer and editor for Cavegirl Productions, has been working on Eye of the Beholder, a documentary on the artists that created the illustrations for the Dungeons and Dragon game. With over 40 hours of interview footage to comb through searching through it all has been made much easier by Transcriptive, a new A.I. plugin for Adobe Premiere Pro.


eye-beholder 

Why Transcribe?

Imagine having Google for your video project. Turning all the dialog into text makes everything easily searchable (and it supports 28 languages). Not too mention making it easy to create captions and subtitles.

The Dragon of Time And Money

Using a traditional transcription service for 40 hours of footage, you’re looking at a minimum of $2400 and a few days to turn it all around. Not exactly cost or time effective. Especially if you’re on a doc budget. However, it’s a problem for all of us.

Transcriptive helps solve the transcription problem, and the problems of searching video and captions/subtitles. It uses A.I. and machine learning to automatically generate transcripts with up to 95% accuracy and bring them into Premiere Pro. And the cost? About $4/hour (or much less depending on the options you choose) So, 40 hours is $160 vs $2400. And you’ll get all of it back in a few hours.

Yeah, it’s hard to believe.

Read what these three filmmakers have to say and try the Transcriptive demo out on your own footage. It’ll make it much easier to believe.

 

“We are using Transcriptive to transcribe all of our interviews for EYE OF THE BEHOLDER. The idea of paying a premium for that much manual transcription was daunting. I am in the editing phase now and we are collaborating with a co-producer in New York. We need to share our ideas for edits and content with him, so he is reviewing transcripts generated by Transcriptive and sending us his feedback and vice versa. The ability to get a mostly accurate transcription is fine for us, as we did not expect the engine to know proper names of characters and places in Dungeons & Dragons.” – Kelley Slagle, Cavegirl Productions

Google Your Video Clips and Premiere Project?

 

Since everything lives right within Premiere, all the dialog is fully searchable. It’s basically a word processor designed for transcripts, where every word has time code. Yep, every word of dialog has time code. Click on the word and jump to that point on the timeline. This means you don’t have to scrub through footage to find something. Search and jump right to it. It’s an amazing way for an editor to find any quote or quip.

As Kelley says, “We are able to find what we need by searching the text or searching the metadata thanks to the feature of saving the markers in our timelines. As an editor, I am now able to find an exact quote that one of my co-producers refers to, or find something by subject matter, and this speeds up the editing process greatly.”

Joy E. Reed of Oh My! Productions, who’s directing the documentary, ‘Ren and Luca’ adds, “We use sequence markers to mark up our interviews, so when we’re searching for specific words/phrases, we can find them and access them nearly instantly. Our workflow is much smoother once we’ve incorporated the Transcriptive markers into our project. We now keep the Markers window open and can hop to our desired areas without having to flip back and forth between our transcript in a text document and Premiere.”

Workflow, Captions, and Subtitles

ren-luca-L

Captions and subtitles are one of the key uses of Transcriptive. You can use it with the Premiere’s captioning tool or export many different file formats (SRT, SMPTE, SCC, MCC, VTT, etc) for use in any captioning application.

“We’re using Transcriptive to transcribe both sit down and on-the-fly interviews with our subjects. We also use it to get transcripts of finished projects to create closed captions/subtitles.”, says Joy. “We can’t even begin to say how useful it has been on Ren and Luca and how much time it saves us. The turnaround time to receive the transcripts is SO much faster than when we sent it out to a service. We’ve had the best luck with Speechmatics. The transcripts are only as accurate as our speakers – we have a teenage boy who tends to mumble, and his stuff has needed more tweaking than some of our other subjects, but it has been great for very clearly recorded material. The time it saves vs the time you need to tweak for errors is significant.”

captions

Transcriptive is fully integrated into Premiere Pro, you never have to leave the application or pass metadata and files around. This makes creating captions much easier, allowing you to easily edit each line while playing back the footage. There are also tools and keyboard shortcuts to make the editing much faster than a normal text editor. You then export everything to Premiere’s caption tool and use that to put on the finishing touches and deliver them with your media.

Another company doing documentary work is Windy Films. They are focused on telling stories of social impact and innovation, and like most doc makers are usually on tight budgets and deadlines. Transcriptive has been critical in helping them tell real stories with real people (with lots of real dialog that needs transcribing).

They recently completed a project for Planned Parenthood. The deadline was incredibly tight. Harvey Burrell, filmmaker at Windy, says, “We were trying to beat the senate vote on the healthcare repeal bill. We were editing while driving back from Iowa to Boston. The fact that we could get transcripts back in a matter of hours instead of a matter of days allowed us to get it done on time. We use Transcriptive for everything. The integration into premiere has been incredible. We’ve been getting transcripts done for a long time. The workflow was always a clunky; particularly to have transcripts in a word document off to one side. Having the ability to click on a word and just have Transcriptive take you there in the timeline is one of our favorite features.”

Getting Accurate Transcripts using A.I.

 

Audio quality matters. So the better the recording and the more the talent enunciates correctly, the better the transcript. You can get excellent results, around 95% accuracy, with very well recorded audio. That means your talent is well mic’d, there’s not a lot of background noise and they speak clearly. Even if you don’t have that, you’ll still usually get very good results as long as the talent is mic’d. Even accents are ok as long as they speak clearly. Talent that’s off mic or if there’s crosstalk will cause it to be less accurate.

6-Full-Screen

Transcriptive lets you sign up with the speech services directly, allowing you to get the best pricing. Most transcription products hide the service they’re using (they’re all using one of the big A.I. services), marking up the cost per minute to as much as .50/min. When you sign up directly, you get Speechmatics for $0.07/min. And Watson gives you the first 1000 minutes free. (Speechmatics is much more accurate but Watson can be useful).

Transcriptive itself costs $299 when you check out of the Digital Anarchy store. A web version is coming soon as well. To try transcribing with Transcriptive you can download the trial version here. (remember, Speechmatics is the more accurate service and the only service available in the demo) Reach out to sales@digitalanarchy.com if you have questions or want an extended trial.

Transcriptive is a plugin that many didn’t know they were waiting for. It is changing the workflow of many editors in the industry. See for yourself how we’re transforming the art of transcription.

Speeding Up Flicker Free: The Order You Apply Plugins in Your Video Editing App

Like Digital Anarchy On Facebook

 

One key way of speeding up the Flicker Free plugin is putting it first in the order of effects. What does this mean? Let’s say you’re using the Lumetri Color Corrector in Premiere. You want to apply Flicker Free first, then apply Lumetri. You’ll see about a 300+% speed increase vs. doing it with Lumetri first. So it looks like this:

Apply Flicker Free first in your video editing application to increase the rendering speed.

Why the Speed Difference?

Flicker Free has to analyze multiple frames to de-flicker the footage you’re using. It looks at up to 21 frames. If you have the effect applied before Flicker Free it means Lumetri is being applied TWENTY ONE times for every frame Flicker Free renders. And especially with a slow effect like Lumetri that will definitely slow everything down.

It fact, on slower machines it can bring Premiere to a grinding halt. Premiere has to render the other effect on 21 frames in order to render just one frame for Flicker Free. In this case, Flicker Free takes up a lot of memory, the other effect can take up a lot of memory and things start getting ugly fast.

Renders with Happy Endings

So to avoid this problem, just apply Flicker Free before any other effects. This goes for pretty much every video editing app. The render penalty will vary depending on the host app and what effect(s) you have applied. For example, using the Fast Color Corrector in Premiere Pro resulted in a slow down of only about 10% (vs. Lumetri and a slow down of 320%). In After Effects the slow down was about 20% with just the Synthetic Aperture color corrector that ships with AE. However, if you add more filters it can get a lot worse.

Either way, you’ll have much happier render times if you put Flicker Free first.

Hopefully this makes some sense. I’ll go into a few technical details for those that are interested. (Feel free to stop reading if it’s clear you just need to put Flicker Free first) (oh, and here are some other ways of speeding up Flicker Free)

Technical Details

With all host applications, Flicker Free, like all plugins, has to request frames through the host application API. With most plugins, like the Beauty Box Video plugin, the plugin only needs to request the current frame. You want to render frame X: Premiere Pro (or Avid, FCP, etc) has to load the frame, render any plugins and then display it. Plugins get rendered in the order you apply them. Fairly straightforward.

The Flicker Free plugin is different. It’s not JUST looking at the current frame. In order to figure out the correct luminance for each pixel (thus removing flicker) it has to look at pixels both before and after the current frame. This means it has to ask the API for up to 21 frames, analyze them, return the result to Premiere, which then finishes rendering the current frame.

So the API says, “Yes, I will do your bidding and get those 21 frames. But first, I must render them!”. And so it does. If there are no plugins applied to them, this is easy. It just hands Flicker Free the 21 original frames and goes on its merry way. If there are plugins applied, the API has to render those on each frame it gives to Flicker Free. FF has to wait around for all 21 frames to be rendered before it can render the current frame. It waits, therefore that means YOU wait. If you need a long coffee break these renders can be great. If not, they are frustrating.

If you use After Effects you may be familiar with pre-comping a layer with effects so that you can use it within a plugin applied to a different layer. This goes through a different portion of the API than when a plugin requests frames programmatically from AE. In the case of a layer in the layer pop-up the plugin just gets the original image with no effects applied. If the plugin actually asks AE for the frame one frame before it, AE has to render it.

One other thing that affects speed behind the scenes… some apps are better at caching frames that plugins ask for than other apps. After Effects does this pretty well, Premiere Pro less so. So this helps AE have faster render times when using Flicker Free and rendering sequentially. If you’re jumping around the timeline then this matters less.

Hopefully this helps you get better render times from Flicker Free. The KEY thing to remember however, is ALWAYS APPLY FLICKER FREE FIRST!

Happy Rendering!

Easy Ways of Animating Masks for Use with Beauty Box in After Effects, Premiere, and Final Cut Pro

Like Digital Anarchy On Facebook

 

We have a new set of tutorials up that will show you how to easily create masks and animate them for Beauty Box. This is extremely useful if you want to limit the skin retouching to just certain areas like the cheeks or forehead.

Traditionally this type of work has been the province of feature films and other big budget productions that had the money and time to hire rotoscopers to create masks frame by frame. New tools built into After Effects and Premiere Pro or available from third parties for FCP make this technique accessible to video editors and compositors on a much more modest budget or time constraints.

Using Masks that track the video to animate them with Beauty Box for more precise retouching

How Does Retouching Work Traditionally?

In the past someone would have to create a mask on Frame 1 and  move forward frame by frame, adjusting the mask on EVERY frame as the actor moved. This was a laborious and time consuming way of retouching video/film. The idea for Beauty Box came from watching a visual effects artist explain his process for retouching a music video of a high profile band of 40-somethings. Frame by frame by tedious frame. I thought there had to be an easier way and a few years later we released Beauty Box.

However, Beauty Box affects the entire image by default. The mask it creates affects all skin areas. This works very well for many uses but if you wanted more subtle retouching… you still had to go frame by frame.

The New Tools!

After Effects and Premiere have some amazing new tools for tracking mask points. You can apply bezier masks that only masks the effect of a plugin, like Beauty Box. The bezier points are ‘tracking’ points. Meaning that as the actor moves, the points move with him. It usually works very well, especially for talking head type footage where the talent isn’t moving around a lot. It’s a really impressive feature. It’s  available in both AE and Premiere Pro. Here’s a tutorial detailing how it works in Premiere:

After Effects also ships with Mocha Pro, another great tool for doing this type of work. This tutorial shows how to use Mocha and After Effects to control Beauty Box and get some, uh, ‘creative’ skin retouching effects!

The power of Mocha is also available for Final Cut Pro X as well. It’s available as a plugin from CoreMelt and they were kind enough to do a tutorial explaining how Splice X works with Beauty Box within FCP. It’s another very cool plugin, here’s the tutorial: