So while we were all waiting for Sora to be available, Luma dropped. And it's really mind blowing 🤯
To put it through its paces, I made a quick comparison of Luma vs Runway Gen 2. No Sora comparison since I don't have access yet. But Luma is clearly way ahead of Runway.
That is to say Runway Gen 2. Because Runway just announced that their Gen 3 model, which is supposed to be a general world model, is coming in the next few days.
Let's get back to the comparison.
To keep things equal, I did not provide any text prompt.
I simply uploaded the same Midjourney image into both tools and let them convert that to video in the best way they thought.
This approach also reveals the image assessment qualities of both video generation tools, as the animation gets decided based on their assessment of different elements and the inferred motion within the source image.
Looking at these comparisons, Runway's animations tend to be very subtle. That is probably to limit morphing. Gen 2 is not a general world model, so it's animations are based on known movements of things like clouds, water, etc. This is also why it tends to produce a lot of morphing when human movement is requested like walking, running, etc.
Luma on the other hand favours BIG motions. Head turns and sweeping camera motions that really show off its general world model.
It's impressive that Luma kept object and landscape geometry intact during these large motions. The reflections on the muddy water in the jungle scene for instance is incredibly impressive.
Having said that, I haven't had much luck with Luma when it comes to controlling the motion with text prompts when an image is used in input. It tends to favour its own assessment over the text prompt when it comes to motion. If you have figured out how to control Luma better, drop a comment below! Would love to learn.
As soon as Runway Gen 3 is released, I will do another comparison, but until then, check out Luma. It's free for up to 10 videos per day.
See you next time!
#GenAI #Luma
