Strengths and limitations of AI-generated videos powered by Sora creators

OpenAI’s video generation tool, Sora, made waves in the AI community with its impressive capabilities, but a recent behind-the-scenes look reveals the tool’s limitations in the filmmaking process.

Shy Kids, a digital production team based in Toronto, was given early access to Sora to create a short film titled “air head.” In an interview with fxguide, post-production artist Patrick Cederberg shared insights into the challenges of using Sora for professional productions.

One major hurdle is the lack of control over the generated content. Cederberg explained that precise details like character wardrobe and movements require elaborate workarounds and checks due to the system’s independent shot creation process. Additionally, unwanted elements like faces on balloons or strings had to be removed in post-production, adding to the laborious process.

Timing and movements of characters or the camera are also imprecise in Sora, making it difficult to achieve specific actions like gestures or camera pans. The team had to render shots in portrait orientation and crop them in post to achieve desired effects.

Despite its impressive capabilities, Sora’s inconsistency in understanding filmmaking language and limitations in content generation highlight its current shortcomings in the filmmaking process. The tool’s refusal to generate copyrighted content like “Star Wars” clips raises questions about its training data and recognition capabilities.

While Sora is a powerful tool for certain applications, its current limitations suggest that it is not yet capable of creating films independently. As filmmakers continue to explore its potential, it remains to be seen how Sora will evolve in the future.