Cuebric generative AI: Now we are all image makers

October 17, 2023

By Adrian Pennington, IBC.ORG, 11 October 2023. Image by Seyhan Lee

The co-developer of a new AI tool for filmmaking insists that AI without human creativity is nothing, and therefore nothing to fear, writes Adrian Pennington.

In the future the director of a piece of entertainment might be able to shape their vision with holograms to create a realistic 3D simulation of anything they would like to see. Perhaps the ‘director’ is also the consumer, in the sense that content can be created, live, and experienced by an individual participant simply with a few voice prompts.

This where we are going with Generative AI, if you believe Pinar Seyhan Demirdag, Co-Founder (with Gary Lee Koepke) of Seyhan Lee, a Boston-based start-up which claims to have developed the first virtual production tool that runs on Gen-AI.

“I would like to imagine a world where no matter your background or talent, anyone can play a central role in the filmmaking process,” she told IBC365. “A world where their vision is facilitated and co-produced by AI tools in real time.”

Seyhan Lee’s Cuebric enables filmmakers to dream up camera-ready media in seconds, for instance to playback in an LED volume. Fifteen studios are experimenting with the technology and it was on track to return the founder’s initial investment earlier this year, in double-quick time.

The business case for Cuebric
It is one of many AI tools being introduced to the industry as a means of cutting costs and, the developers argue, freeing up time for creatives to be creative.

“Let’s start with notion of waiting on set,” Demirdag said. “We always feel we’re rushing so we can collectively wait. With a realtime environment generation tool like Cuebric the director can change their mind in realtime and make, I would argue, the world’s most collaborative artform even more collaborative. It energises people to be more creative on set.

A second business case for Cuebric is that it consigns manual rotoscoping (the process of tracing over live-action footage frame by frame) to history. “Rotoscoping is one of the most tedious and time-consuming parts of the filmmaking process. If it were to disappear from the face of the planet no one would argue.”

Currently, Cuebric enables 2D rotoscoping but future upgrades would, for example, allow users to select a 3D character and remove her from every frame in a scene.

Putting exact figures on the costs saved when using a new tool like this is tricky but Seyhan Lee have identified one area where producers can realise instant financial benefits.

“Five percent to 20% of budgets today goes into reshoots because even when you’re working with green screen or virtual production, reshoots are often the only way to achieve the director’s intent.”

Seyhan Lee calculate that for a medium size picture, reshoots cost a production $375,000 on average. For bigger budget shows that rises to an astonishing $24 million.

“If we were to save the industry even a fraction of that using AI it could help funnel those funds back into creativity and cut out unnecessary labour,” Demirdag said.

Cuebric is a website [cubebric.com] allowing uses to write prompts to generate synthetic media. All the data processing requires banks of GPUs in order to render photoreal imagery at speed and runs in the cloud. It bundles together five algorithms. One of them will turn existing photographic plates into camera ready LED screen definitions, up to 16K. You can add depth to scenes and also choose between Classic, Moody and Sci-Fi modes for cinematic looks. It’s main ‘trick’, though, is to output original pixel-ready environments, VFX or concept art trained on Stable Diffusion dataset LAION.

All of LAION’s image datasets are built off of Common Crawl, a nonprofit that scrapes billions of webpages monthly and releases them as massive datasets.

“This is a dataset trained on the total sum of humanity, any picture that ever appeared online and not the artwork of one person,” Demirdag explained.

In the US, developers of leading Gen-AI tools including Stable Diffusion and ChatGPT owner OpenAI are battling lawsuits from stock photo agencies and groups of artists alleging that their copyright has been infringed. Continue reading