Building notes: How much area can this man fill with his conducting of Wagner's (kinda bad) opera? (Part 1)

I wanted to make a cool animation entirely by code but was like video editing software is neat.

Introductory Rant

I want to be clear: I despise Richard Wagner and his operas. Yet I am obsessed with the finale of Götterdämmerung (The Twilight of the Gods), the fourth and final opera of his famous 15-hour Ring Cycle, and I have listened to it hundreds of times. The entire Ring Cycle revolves around a magical, cursed ring that grants its owner the power to rule the world…If they are willing to renounce love. The Nordic gods, here, have debased themselves to corruption for the sake of power, using humans as pawns to maintain their worldly authority. In the end, Brünnhilde, the fallen Valkyrie, finds herself in possession of the object that led many men into ruin:

Her heroic act is to self-immolate with the cursed ring by throwing herself onto the funeral pyre of her dead lover. There, she commands the flames to rise, and the fire reaches Valhalla, burning the gods alive and bringing about their long-prophesied twilight. The cursed ring finally falls back into the Rhine River, from which it originally emerged, and as the waters symbolize the primordial state of being, they carry the promise of a new world free from the mechanisms of the gods.

In the flurry of different melodies (or Wagner’s famous leitmotifs), one of the most beautiful soars above them all. Somewhat erroneously named Liebeserlösung (Redemption through Love), the melody plays in the final seconds of the finale and it’s just breathtaking. It sucks, man. I’m thinking: Wagner, you piece of shit, for making me love a piece of your music. I hated all the other 14 hours and 40 minutes of it—but yet you make me love the last 20 minutes.

Wagner is such a stupid, weird loser. And if I met him in real life, I would probably loathe him in real life.

And now we gotta talk about my favorite conducting of the Götterdämmerung finale. And that is Georg Solti:

The man. The genius.

Georg Solti was a Hungarian conductor, and many have the opinion that his interpretation of the Ring Cycle as the best. I love watching Solti to conduct. Watching videos of him conducting is like watching an MMA fighter: it’s so aggressive, as if he’s fighting an imaginary ghost. Solti even stabbed himself with his own baton!

Just look at Solti’s interpretation of the Götterdämmerung finale. Look at this man go:

Full video below (with the great Birgit Nilsson)

Okay, the dumb project is: with all of Solti’s movement, does his aggressive conducting style allow him to cover the entire video frame?

Attack plan

So I’ve never annotated videos, but I’m assuming that I need to get the centroids of Solti’s hands. I came across this video and learned about cvat.ai, which is commonly used for annotations in machine learning. So, I’m somewhat using a glass cannon for something that is honestly very simple, but I can imagine myself using this in the future for more ‘machine-learn-y’ projects, so it is good to learn. Important aside: do not use CVAT in Safari - the bounding box tool does not work!

So I go frame by frame and bound boxes around the hands.

Because I legitimately do not care much about accuracy (this is a silly project), I somewhat used my best estimate of where the hands are. Ultimately, I’m interested in the centroids. Interestingly, even though I’m not using a machine-learning algorithm like YOLO, CVAT let me save the annotations, and by default, YOLO gives you the X and Y centroid positions, with each frame saved as a text file. I export the annotations. You can see that each frame has its own text file and within each text file, I should have two lines for each hand:

The first element is the class. And the only values that I care about are the second and third element because they are the x and y centroid coordinates. So I made a little Python script to do this and I should ultimately get a Data Frame that look like this:

For each frame, I should have both a left and right hand with their corresponding X and Y coordinates. I realized I should have done this in R when I started making a ggplot animation (I know plotnine has an animation feature, but I don’t… like it). So load up gganimate:

anim <- ggplot(centroid_df, 
  aes(x = x_coord * 480, y = y_coord * 360, color = hand, group = hand)) +
  geom_path(size = 1.2, alpha = 1) +         # trajectory line
  geom_point(size = 3) +                     # moving point
  theme_minimal() +
  scale_color_manual(values = c(”#20fafa”,”#fa2044”))+
  transition_reveal(along = frame_num)   +
  ease_aes(’elastic-in-out’)  +
  theme_void() + 
  theme(
    panel.background = element_rect(fill=’transparent’),
    plot.background = element_rect(fill=’transparent’, color=NA),
    panel.grid.major = element_blank(),
    panel.grid.minor = element_blank(),
    legend.position = ‘none’) 

The only things that you need to realize is that I use geom_path and that transition_reveal is the best function to use for anything with trajectories, in my opinion.

So I wanted to overlay this on top of my original video of Solti conducting. I tried various tricks using {magick} and even {ggimage} where for each frame of the conducting video I would have it as a background- none of these worked. If you have any suggestions, DM me. I wasn’t so irate because I’m trying to learn video editing, might as well just use Davinci Resolve to composite these two videos together:

And here is the final result:

This took a lotttttt of effort. So I’m splitting this into a two part series where I actually figure out how much of the area does he actually cover.

(WIP repo here: https://github.com/pakdamie/solti_hands)