Blog

SDE_Maker

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive
 

 Just did 50 SDE´s (Same day edits) in a couple hours

 

It´s always complicated to do an SDE on a wedding day. There is always a lot of pressure and expectations are always high. So, I decided to make a script to automate the editing process. The script acts in three phases. It estimates the beats of the chosen song, imports video files and calculates the transition moments between clips so that they always happen at important music events.. So if I rate the best images on the GH4´s favorites through the day and use them i can edit an SDE in about 10 seconds.

 

It´s also very useful if you are a photographer and want to make a slideshow edited to the beats in a few seconds

                                                                   

An example of a video edited to the beat with 867 transitions with SDE_Maker:

                                                                   

For the first part (estimate beat events), i used phyton with librosa library (https://librosa.github.io/librosa/) to make the script. The librosa beat tracking uses the following approach: Ellis, Daniel PW. “Beat tracking by dynamic programming.” Journal of New Music Research 36.1 (2007): 51-60. http://labrosa.ee.columbia.edu/projects/beattrack/

The code:

import librosa
import numpy as np
import tkinter as tk
from tkinter import filedialog
import numpy
import tkinter
import matplotlib.pyplot as plt
import csv
from tqdm import tqdm
import time


root = tk.Tk()
root.lift()
root.withdraw()

root.wm_attributes('-topmost', 1)
file_path = filedialog.askopenfilename()
y, sr = librosa.load(file_path)



with open('/Program Files (x86)/Common Files/Adobe/CEP/extensions/Alert Panel/path.csv', 'w', newline='') as csvfile:
    fieldnames = ['path']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)

    writer.writeheader()
    writer.writerow({'path': file_path})

tempo, beats = librosa.beat.beat_track(y, sr=sr,units='time')
#librosa.output.times_csv('beat_times_11.csv', beats)
librosa.output.times_csv('/Program Files (x86)/Common Files/Adobe/CEP/extensions/Alert Panel/beat_times_11.csv', beats)


onset_env = librosa.onset.onset_strength(y, sr=sr,aggregate=np.median)
tempo, beats = librosa.beat.beat_track(onset_envelope=onset_env,sr=sr)
tempo
64.599609375

beats[:20]
([ 320,  357,  397,  436,  480,  525,  569,  609,  658,
        698,  737,  777,  817,  857,  896,  936,  976, 1016,
       1055, 1095])
hop_length = 512
plt.figure(figsize=(8, 4))
times = librosa.times_like(onset_env, sr=sr, hop_length=hop_length)
plt.plot(times, librosa.util.normalize(onset_env),label='Onset strength')
plt.vlines(times[beats], 0, 1, alpha=0.5, color='r',linestyle='--', label='Beats')
plt.legend(frameon=True, framealpha=0.75)

plt.tight_layout()
plt.savefig('/Program Files (x86)/Common Files/Adobe/CEP/extensions/Alert Panel/beats.png')

for i in tqdm(range(101),
              desc="ESTIMATING BEATS"):
    time.sleep(0.01)
print("Complete.")

This script estimates the beat events of the song chosen by the user and saves the exact moment in seconds when they occur in a CSV file. Just out of curiosity here are the beat events of  Queen's song "we will rock you":

 

 

For the second and third part (imports video files and calculates the transition moments between clips accordling with the CSV file) i used adobe´s Toolkit extendscript to do the script. The code:

 

 $.runScript = {


alert: function() {

var proj = app.project;



dim_files = 0;
dim_infoArray = 0;
num_marker = 0;
order_pic = 3;
num_markers_remain = 0;
 

var filterString = "";
if (Folder.fs === 'Windows'){
filterString = "All files:*.*";
}

var importFolder = new Folder;
importFolder = File.openDialog ("Import Files", filterString,true);
var importAry = [];

if (importFolder ){
for (var i = 0; i < importFolder .length; i++) {
importAry[i] = importFolder[i].fsName;}}

dim_files = importFolder .length;

proj.rootItem.createBin("SDE Maker");

var sequenceName = "SDE 1";
proj.createNewSequence(sequenceName,"ID123");
var activeSeq = app.project.activeSequence;



var musica = File(['C:\\Program Files (x86)\\Common Files\\Adobe\\CEP\\extensions\\Alert Panel\\path.csv']);
var musica = musica.fsName;
if(musica){

var file_2 = File(musica) 
file_2.open("r");
var fullText_2 = file_2.read();
file_2.close();
infoArray_2 = fullText_2.split("\n"); 
alert(infoArray_2.length);

for(var i=0;i<infoArray_2.length;i++){ 
infoArray_2[i] = infoArray_2[i].split(",");
;}}
caminho = (infoArray_2[1][0]);
alert(caminho);

proj.importFiles([caminho],1,proj.rootItem,0);
proj.importFiles(importAry,1,proj.rootItem,0);
var clip = proj.rootItem.children[2];
activeSeq.audioTracks[2].insertClip(clip,0);

File(['C:\\Users\\User\\PycharmProjects\\librosa_teste\\beat_times_6.csv']);
var dataFile = File(['C:\\Program Files (x86)\\Common Files\\Adobe\\CEP\\extensions\\Alert Panel\\beat_times_11.csv']);


if(dataFile){

var file = File(dataFile) 
file.open("r");
var fullText = file.read();
file.close();

infoArray = fullText.split("\n\n"); 
alert(infoArray.length);
dim_infoArray = infoArray.length;
num_marker = Math.floor((dim_infoArray) / dim_files);
num_markers_int = Math.abs((num_marker * dim_files) - dim_infoArray);
num_markers_remain = dim_infoArray - num_markers_int;


for(var a=0;a<infoArray.length;a++){ 
infoArray[a] = infoArray[a].split(","); 
b = parseFloat(infoArray[a][0]);

if (a == 0 || a % num_marker == 0 || a >= num_markers_remain) {

order_pic = (order_pic+1);
clip2 = proj.rootItem.children[order_pic];
activeSeq.videoTracks[1].overwriteClip(clip2,b);}
}
}

} 

}

 

To wrap things up in a html panel i had to create two more files, a manifest.xml to create the panel and an index.html to call the  external python executable and the javascript function on extendscript (extendScript.jsx).

The code in manifest.xml:

<?xml version="1.0" encoding="UTF-8"?>
<ExtensionManifest Version="6.0" ExtensionBundleId="MakeSDE" ExtensionBundleVersion="1.0.0" ExtensionBundleName="MakeSDE" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<ExtensionList>
		<Extension Id="MakeSDE" Version="1.0" />
	</ExtensionList>
	<ExecutionEnvironment>
		<HostList>
			<Host Name="PPRO" Version="9.0" />
		</HostList>
		<LocaleList>
			<Locale Code="All" />
		</LocaleList>
		<RequiredRuntimeList>
			<RequiredRuntime Name="CSXS" Version="7.0" />
		</RequiredRuntimeList>
	</ExecutionEnvironment>

	<DispatchInfoList>
		<Extension Id="MakeSDE">
			<DispatchInfo >
				<Resources>
					<MainPath>./index.html</MainPath>
					<ScriptPath>./jsx/extendScript.jsx</ScriptPath>
					<CEFCommandLine>
						<Parameter>--enable-nodejs</Parameter>
						<Parameter>--allow-file-access</Parameter>
						<Parameter>--allow-file-access-from-files</Parameter>
					</CEFCommandLine>
				</Resources>
				<Lifecycle>
					<AutoVisible>true</AutoVisible>
				</Lifecycle>
				<UI>
					<Type>Panel</Type>
					<Menu>SDE Maker</Menu>
					<Geometry>
						<Size>
							<Height>250</Height>
							<Width>500</Width>
						</Size>
					</Geometry>
				</UI>
			</DispatchInfo>
		</Extension>
	</DispatchInfoList>
</ExtensionManifest>

The code in index.html:

<!DOCTYPE html>
<!-- This document forms the dockable panel. The button is the <input> tag below. The <style> formats the button. 

		The extendscript is called through the <script type="text/javascript"> tag, which then routes it to use the adobe_cep_.evalScript to interpret that Javascript into Extendscript -->

<html> 

<head> 
   <script language="javascript" type="text/javascript">
function runApp()
{
const { execFile } = require('child_process');
//const child = execFile('C:\\Users\\User\\PycharmProjects\\SDE_maker\\CSV_2.exe', [], () => {
	const child = execFile('C:\\Program Files (x86)\\Common Files\\Adobe\\CEP\\extensions\\Alert Panel\\CSV_7.exe', [],(error, stdout, stderr) => {
  if (error) {
    throw error;
  }
  console.log(stdout);

});
}


</script>

     </head> 
      <body> 
       <h1>MUSIC CHOICE</h1> 
       
      <button onclick="runApp()">Open</button> 
     </body> 
        </html> 


     

<html>
	<head>
		<meta charset="utf-8">
		<title>SDE Maker</title>
		<script src="/./lib/CSInterface.js"></script>		
		<script type="text/javascript">
			
			function runAll() {
                
				var cs = new CSInterface;
								
				cs.evalScript('$.runScript.alert()')
			}

			
		</script>

		<style>
			body {
				background-color: #262626;
				color: #f5f5f5;
				text-align: center;
				}
			input {
				display: block;
				margin: auto;
				max-width: 100%;
					}
			 

		</style>
	</head>
   
	<body>

		<h1>UPLOAD FILES</h1>
		<button id="runButton" type="run" onclick="runAll()">MAKE SDE!!</button> 
				
			  
			     
	</body>
	<!img src="/beats.png" width="500">
</html>

 

Limitations and improvements:

The biggest limitations are the fact that it only works for WAV files and with one music at a time. I will change the script to accept mp3 files and more than one music. I´m also considering adding the ability to analise all the clips and use only the portions of each one that are in focus in order to save time in fine-tuning.

 

Testing

To test the script i picked 91 files from a wedding i shot and 50 random musics (downloaded at artlist.io) and used them to make 50 SDE´s diferent versions. The algorithm also tries to fill most of the song choosen, so all 50 versions have diferent durations. It´s a lot of fun to just pick random songs and watch the final result.  Here are some of my favorites versions (you can see them all by clicking here):

 

                                                                   

 

                                                                   

 

                                                                   

Did another experiment just for fun:

Edited this one in 4 minutes with SDE_Maker!!

Needed: folder with video clips (200 of them in this case downloaded @pexels )

Music (continent_by_anbr_Artlist downloaded @artlist.io)

Procedure: click button to estimate beat events click another button to import and edit clips to the beat

 

                                                                   

 

 

 Feedbak

 I would love to hear from you. Please leave a comment if you have a sugestion, doubt, opinion or if you just wanna say hello. 

 

To make things easy you can copy/paste one of the following:

- I would love to have an executable so i could try the extension.

- I would love to have a software with this features but independent from Premiere Pro.

- I don´t thing this extension would be useful for me.

 

 

  

 

Comments powered by CComment