Fade to Black

Well, that's 1 for past me, and 0 for current me; Their skill and foresight is really throwing a wrench in my creative juices /s. 

Recently I acquired a complete season ( or two ) of the kids show "Octonauts", which my wife and I have grown to enjoy as one of the less annoying kid oriented shows for our daughter. As well as being tolerable (ok, maybe even enjoyable), it also has a neat musical number at the end of the episodes that recaps the animal that the episode talked about. 

The acquired episodes came in "packs" of 2 11 minute episodes per file, with a beginning introductions, and an ending credit sequence. This is a pretty standard layout for shows with 11 minute episodes, and why past me got a point. As I looked at the files a little bit closer, I noticed that the episodes were completely out of order. Unfortunately, it wasn't limited to the entire file (i.e. episodes 3-4 packaged together and labelled as 8-9) but to each episode as well, so episode 1 is packaged with episode 7 and labelled as episode 3-4. This was going to cause a lot of headache when trying to get Plex to match up correctly, and future library maintenance.  

That's when I thought, "But what if I separate the file into individual videos?". That would mean that I could correctly label the files, and not have the issues that arise with Plex matching non-uniform names of episodes, and also prevent future issues from arising due to non-uniform names of files. Luckily my past self has not only had this thought, but has implemented a script to do exactly that, within some restrictions. 

The script relies on a common practice that the video industry has of rarely fading to a true black picture, or a frame with absolutely no color info, and the practice of inserting a true black frame when stitching together different pieces of video. This means that "in general" the only true black frames that exist in a video file are where the original creators of the file stitched together different video files to create the file. 

Past me, with some help from StackOverflow and shared code on GitHub, was able to write up a script that uses ffmpeg to detect the true black screens, separate, and recombine the video files into single video episodes. This allowed me to correctly title the episodes, and Plex to correctly match the episodes. 

There are some issues that the script has, as I had to hard-code the number of expected splits ( 6 - opening black, intro, part 1, part 2, outro, extra credits ), and prevent the script from recombining if the number of splits was too many or too few. Along with that, the script is single threaded, as is ffmpeg, so it takes quite a bit longer then if I were to have a ffmpeg process per hardware core/thread. Other then those issues the script worked well for what I needed. 

if interested you can checkout the script below:

#!/bin/bash
file=`realpath "$1"`

dir=$(dirname "$file")

#timetosplitat=$2
echo "timetosplitat: $timetosplitat"

#argument check

# fix file name
filename=`basename "$file"`
echo "filename: $filename"
extension="${filename##*.}"
echo "extension: $extension"
#filename="${filename%.*}"

printf -v out "$dir/temp"
printf "Seperate %s into parts" "$file"
echo ""

dur=0.05
stripaudio=""
ratio=1.00
th=0.05
add=0.00
trim=0.00


############## Seperate the video into parts
cut_part () {
  duration_flag=""
  if [[ "$3" != "" ]]; then
    duration_flag="-t"
  fi
  echo "cutting from $1 duration $3"
  echo ""
  printf -v fileout "$out/%04d_%s" $2 "$filename"
  ffmpeg -y -loglevel error -hide_banner -i "$file" -ss $1 $stripaudio $duration_flag $3 "$fileout" < /dev/null
}

mkdir -p "$out"
timefrom=0
i=1

ffmpeg -i "$file" -vf blackdetect=d=$dur:pic_th=$ratio:pix_th=$th -f null - 2> ffout
black_start=( $(grep blackdetect ffout | grep black_start:[0-9.]* -o | grep "[0-9]*(\.[0-9]*)?" -oE) )
black_duration=( $(grep blackdetect ffout | grep black_duration:[0-9.]* -o | grep "[0-9]*(\.[0-9]*)?" -oE) )
> timestamps
for ii in "${!black_start[@]}"; do
  half=$(bc -l <<< "${black_duration[$ii]}/2")
  middletime=$(bc -l <<< "${black_start[$ii]} + $half")
  echo $middletime | LC_ALL=en_US.UTF-8 awk '{printf "%f", $0}' >> timestamps
  echo "" >> timestamps
done

while read -r timestamp; do
  duration=`bc -l <<< "$timestamp-$timefrom+$add-$trim" | LC_ALL=en_US.UTF-8 awk '{printf "%f", $0}'`
  cut_part $timefrom "$i" $duration
  timefrom=`bc -l <<< "$timestamp+$add-$trim" | LC_ALL=en_US.UTF-8 awk '{printf "%f", $0}'`
  i=`expr $i + 1`
done < timestamps

if [[ "$timefrom" != 0 ]]; then
  cut_part $timefrom $i
fi
########### End Seperation



#Check that we have an Expected amount of splits, otherwise we should do it manually
#first if we have too many
# if we have 7 files, then we have too many splits
printf -v fileoutseven "$out/%04d_%s" 7 "$filename"
if [[ -f "$fileoutseven" ]]
then
  echo "Too many splits, bailing"
  exit 1
fi

#then check if we have too few
# if we dont have 6 files then we have too few
printf -v fileoutsix "$out/%04d_%s" 6 "$filename"
if [[ ! -f "$fileoutsix" ]]
then
	echo "Too few Splits, bailing"
	exit 1
fi


printf -v fileoutintro "$out/%04d_%s" 2 "$filename"
printf -v fileoutending "$out/%04d_%s" 5 "$filename"

echo "BEGIN Concatenation of part 01"
########### BEGIN Concatenation of part 1

printf -v fileoutthree "$out/%04d_%s" 3 "$filename"
printf -v fileoutfinalone "$out/part_01_%s" "$filename"

if [[ -f mylist.txt ]]
then
    rm mylist.txt
fi

echo "file ${fileoutintro@Q}" >> mylist.txt
echo "file ${fileoutthree@Q}" >> mylist.txt
echo "file ${fileoutending@Q}" >> mylist.txt

ffmpeg -nostdin -f concat -safe 0 -i mylist.txt -c copy "$fileoutfinalone"
rm mylist.txt
########### END Concatenation of part 01

echo "BEGIN Concatenation of part 02"
########### BEGIN Concatenation of Part 02  
printf -v fileoutfour "$out/%04d_%s" 4 "$filename"
printf -v fileoutfinaltwo "$out/part_02_%s" "$filename"

echo "file ${fileoutintro@Q}" >> mylist.txt
echo "file ${fileoutfour@Q}" >> mylist.txt
echo "file ${fileoutending@Q}" >> mylist.txt

ffmpeg -nostdin -f concat -safe 0 -i mylist.txt -c copy "$fileoutfinaltwo"
rm mylist.txt
########### END Concatenation of 5-6  

echo "Clean up seperated files"

########### CLEAN UP parts 
find "$out" -regex ".*/[0-9]+_$filename" -delete

########## CLEAN UP timestamps and ffout files
rm ffout
rm timestamps
exit 0