I’ve been feeling my way around the latest AI tool to try to understand what makes it tick. AnimateDiff is an extension in Stable Diffusion software that allows you to create gifs based on generative art. To test different parameters in the creation process, I decided to create an animation depicting a movie star driving in a convertible down sunset boulevard. I tried to keep the settings consistent and mainly focused on changing the style, but I did experiment with some numerical values such as the number of frames and frames per second.
In the first animation, I did not specify a viewing angle or other qualities such as colour scheme. In my mind, I was envisioning a side view of a dame driving a convertible and on her way to meet her lover in a secret rendezvous. That’s not what the software generated.
If you look closely at this bizarre video you actually see different characters.
This is not what I was looking for, but it had a bit of that vintage charm from Hollywood films, so I thought I was in the right direction.
I reworked the prompt to include mention of a side view angle and gave it a more comic book style look.
Fantastic! I thought when the animation was generated. That was more like it. The sideview profile was excellent, the hat was perfect, shades were très bonne, and the vampy look of the female was wicked.
I played around with different styles to see how it would alter the output.
The sense of motion in these two generations is superb. One gets the sense of motion parallax, which is a cue that our visual system uses in order to provide us with a sense of 3D movement.
I even added a steampunk style to the piece and this interesting animation was generated.
I really liked the movement of her head, but as you can see, the software had some problems handling the change in perspective, particularly with the hat. Surprisingly, her facial features remained consistent and the sky in the background conveyed an abstract sense of motion.
I changed to a Victorian style. The generated animation is pleasant because the woman's head motion is rather smooth. Unfortunately, there is no motion on the background, so it doesn't look like she's driving to meet her lover but more like stuck in traffic on the way there.
Next, I tried a pointillist approach, and it resulted in an intriguing creation. I’m excited about the motion of the woman's lips, which nicely foreshadows what’s waiting for us over the horizon in terms of animation technology. We'll likely be able to type a phrase and have the characters make the correct mouth movements.
This last one is nice (if you ignore the distortions) because it has a photorealistic look. The reflection on the glasses is spot on, and there is a sense of movement in the background.
I'm really excited about this development in AI. It won't be long before we start seeing feature length films being generated in a few seconds by simply adding a prompt. It will democratize movie making when you no longer need a multimillion dollar budget to tell your story. Thrilling!
Thank you for coming along on the ride!
Images generated by @litguru using Stable Diffusion software