Workflow elements:
- Claude Opus 5.5 (in Claude Code) as the central agent
- Environment: Github Repo
- Storage: Supabase
- Voices and images: ElevenLabs (https://try.elevenlabs.io/zwzvn4vdg058)
- Videos: OpenArt
- Lip sync videos: HeyGen
First, I built an environment containing the following:
- All card lore (compiled separately by pulling down all lore from the Splinterlands API)
- All world lore (pulled from Splinterlore)
- Character and visual references for Vinnie and Mags
Then, I described the workflow I wanted to use to create short shorts. Here is the exact workflow as I described it to Claude:
- Input: I will provide some information about a Splinterlands character, along with its card art, as references, the setting for Vinnie's "short shorts" intro segment, which Vinnie to use for the narration parts (for now only "green jacket" exists, but we'll add more styles in the future), and which Mags to use (if Mags is to be used in the script, which is not always the case). I will also provide the music file which will accompany the video. The duration of the music file is important, and the script should last a few seconds longer than the music file, such that the video doesn't end abruptly. You should also inspect the music file to see if it should inspire the script in any way (e.g. in terms of timing transitions or emphasizing anything in the script).
- You will then start by drafting a storyboard for that character. The target length of the video should be 45-60 seconds. It starts with a 6-8 second long intro (i.e. the one where we see Vinnie close-up, and then it zooms out when he says the "shorts shorts" part of "it's time for... short shorts"), and then fades into a view of Vinnie in his studio introducing the character. It should alternate between Vinnie talking and footage relevant to the character and story. When it alternates back to Vinnie, it should generally show a different angle of Vinnie to keep things fresh.
- The video will end with an animated view of the card art (provided in the reference inputs). The card should animate into the frame with a background relevant to the character. I've added a reference image of this as well.
- The script, importantly, needs to be contained within the 45-60 second ideal timeframe. During this stage, use ElevenLabs to record the suggested lines, and use that to ensure your suggested script roughly matches the desired timeframe.
- Once you have completed the suggested storyboard, you'll give it to me, describing what is shown and what is said at each timestamp, and also putting each voice file in there for my review.
- I will either approve or ask for edits on the storyboard or voice files. Once it's approved, we move into the visual production stage.
- In the first step of the visual production stage, we'll get our visual assets together:
- Make a reference image for Vinnie in his short shorts using ElevenLabs, or use an existing one (this will be specified as part of the inputs).
- Make reference images for the various "background footage" scenes using ElevenLabs.
- Make the outro card image using the card image given as reference, again in ElevenLabs
- Images in ElevenLabs will use the "GPT Image 2" model, 9:16 aspect ratio (portrait, for shorts), 1k resolution, "High" quality, and generate 2 images.
- OPTIONALLY, I may tell you that I have images already and simply pass them to you, in which case we'll skip the step of creating reference images.
- Once we have all the reference images, we move on to video creation.
- The intro segment will be done in HeyGen, which will require the image of him in short shorts along with the intro line by Vinnie. I'm not sure what settings to use for HeyGen lip sync with a reference image, so you'll need to look into that and once we figure it out we'll institutionalize it into the workflow.
- The videos where Vinnie is narrating will also be done in HeyGen.
- The "background footage" videos will be done in OpenArt, passing reference images creating in the visual production stage.
- The outro video (animating the card reference image) will be done in OpenArt.
OpenArt videos should use Kling 3.0 in 9:16 aspect ratio with no audio, and only output the duration required for the segment (not more, as that would cost credits for nothing).- Once videos are ready, you will output them all and give me links so I can inspect each one. I will tell you if any needs to be redone.
- Afterwards, we'll stitch it all together per the storyboard, and I will ask you to consider transitions between videos carefully (e.g. the outro likely is a fade-out / fade-in transition, while other clips may just transition directly without fade). In this stage we should also be adding the subtitles to match what's being spoken. Finally, this final video should include the music file added as part of the inputs.
One part of the workflow, which was suggested by Claude, was to actually build preview pages to help me easily see the storyboard, test the audio outputs, and view images, so I could more easily give it feedback.
Here are some screenshots of the pages Claude built to help me react to its progress:
For the image generation stage, here are some of screenshots of the workflow in action (I asked for 2 images for each part of the storyboard in order to be able to pick).
Video gen stage screenshots:
After a couple rounds of reviews/edits, here's the result:
Cost:
- 20% of weekly usage on my Claude Pro Plan (using Opus 5.5 on High effort)
- 10% of monthly credits on HeyGen account
- 20% of monthly credits on ElevenLabs account
- 6% of monthly credits on OpenArt account
I might consider switching image gen to OpenArt, but for now I have roll-over credits on EL to use so I'll keep it there. Would be great if all this was on ChatGPT which has built-in image generation, but Claude Opus 5.5 is just objectively better for all this right now I think. However, since this is in a Github repo with the workflow as the source of truth in the repo, I can easily switch from Claude to ChatGPT for a test later on.