Does a future with AI scare you? Excite you? Both of those things? Does it feel as if it’s opening up a world full of possibilities that are both brilliant and terrible? If it’s any of that, I get you.
I’m terrified of the 2024 elections. Between fake news and AI images becoming universally available and very good, we are facing a particularly ugly election.
If you’ve spent any time on Instagram, you will have seen photos edited using Photoleap and videos transformed with Videoleap. The company is showing off its next-gen video editing software that can copy one style of video to another. It isn’t flawless, but it is head and shoulders above some of the other technologies out there.
To get a better idea of what this future could look like, I spoke to Zeev Farbman, co-founder and CEO of Lightricks. The company has been at the cutting edge of adding AI functionality to customer-facing tech, both in the aforementioned Photoleap and Videoleap apps and in the company’s research lab.
Look, I’m not naïve about this. AI is a tool like, say, dynamite. It can be used to blast through a mountain to build a road, or you can use it to blow up things important to other people. Unlike dynamite, however, there seems to be little regulation around AI.
Right out the gate, Farbman stressed that a lot of what we can accomplish with AI video is a continuation of what we’re already doing with photo AI. He’s not wrong. Adobe added Content-Aware Fill to Photoshop in CS5 back in 2010. It used AI that’s several generations old, but it shows that we’re merely on a technology continuum here.
“If you’re looking at the world of image creation, you can almost claim by now that it’s a solved problem,” said Farbman. “It’s not solved yet in terms of user interfaces, because if you want to use the latest and greatest controls, you need to download all kinds of weird open source tools and combine them together. It’s a messy process. But at the end of the day, in terms of research, we’re getting to a point where, if you know what you’re doing, you can imagine any kind of picture and create it.”
For a long while, animations like the video below have been the cutting edge of consumer AI video. They are nifty, but there’s very little consistency between frames. You get an interesting, psychedelic effect — but nobody is going to mistake this type of video for the real thing.
That’s about to change very rapidly, however.
The first tool Farbman showed me was a restyling tool that lets you take a video of a space around you and give it an AI makeover. You can imagine this being used for, say, interior decoration, where you can easily change wall colors and soft furnishings to see what the space looks like. But you can also send yourself back in time and transform a room in 2023 to how it may have looked in, for example, 1250 BCE Egypt under the rule of Ramesses II. Or you could push forward into time, head into a different world, or jump into a Studio Ghibli film.
It serves us well to keep two points in mind when making such transformations with video AI. First, the process is hands-free for whoever is implementing it and happens with the touch of a button. This is easy for the user. And second, the transformation is consistent — you’re going from today to yesterday to tomorrow within the same space.
Farbman demonstrated another tool that lets you draw on a scene, then transforms and adds your scribble to the scene and makes it interact with the elements in the scene dynamically.
Not sure how to imagine that? Think of some video footage of a waterfall. Using the AI program, you can squiggle a lozenge shape onto the river above the waterfall. The program recognizes this as a boat, which it then creates and can integrate it into the scene and show it moving along the river and over the waterfall. Magic? Maybe!
I have some experience as an amateur compositor (Adobe After Effects and I are pretty good friends), and the technology Farbman was showing off is not on par with what I can do. But it isn’t far off, and it’s catching up very quickly indeed.
If ChatGPT brought text-based AI to anyone with a computer, what does the future for video-based AI look like? As Farbman explains it, you’ll be able to start with an image and then ask AI to step into it and imagine where it goes from that point. It might not produce Hollywood levels of special effects right now, but it’s going to look impressive.
“If kids are doing ‘Hamlet’ as the school play, for example,” said Farbman, “the level of visual effects they’re going to have available a year from now won’t rival a Hollywood production, but they could make a pretty serious production.”
Farbman is far from oblivious to the problems that AI can spark and recognizes that people will also use it for what he describes as “shady stuff.” There’s no putting the toothpaste back in the tube: What’s possible is possible, and it makes the most sense to educate people about AI’s capabilities.
“The genie is out of the bottle,” said Farbman. “A year ago, state agents were able to create any kind of deep fakes they wanted. What’s changed now is that we’re finally getting public awareness of the fact that you should be extremely critical of every visual tab that you’re seeing. Making the technology more widely available is the only way to tackle this issue.”
Perhaps it’s better this way. People might finally be inspired to think a bit critically if virtually anyone can create and post fake videos. When people understand just how much AI can change with minimal effort, their eyes will open to its possibilities, both good and bad.
And what does the far future look like for video AI? Could it stitch together entire sequences of AI-generated video content to create a complete film? That certainly something feels like the natural progression of what we have now.
Photo AI might be academically “done,” but video AI is still very exciting and seems to be the next frontier of the technology.
What a time to be alive.