Runway Aleph experiment: using one initial vid clip as a character 'video reference' to generate many more with consistent aesthetics/world
Whole video is made from aleph generations based on one initial clip (prompt and clip in the comments) https://t.co/M8mkHFmNGO
initial clip was this one I had laying around
prompt: "Extrapolate another scene from the same 60 minutes of this footage. Ensure the footage visually matches perfectly (color grading, film stock, visual qualities" (modifications/variations for different activities/settings) https://t.co/n58iMMu6Rc
why is this interesting? I mean you can use different images and references for the same thing right?
except there's more implicit information about intention within a video clip. I've been wondering for a while if we'll ever get a 'character reference' style functionality that allows using a video of a character in motion, rather than an image.
Because far more of the character's intended motion, personality etc. is contained within that footage than could be conveyed in a single or multiple images
I think to some extent in the longer run this may be something needed to more smoothly integrate characters into new scenes and match existing movies in immersive realism.
With longer footage/movies having a reference for the personality/acting style associated with a particular character played by a person for instance.
