Adobe’s Magic Fixup uses video for revolutionary AI image editing

Adobe researchers have unveiled Magic Fixup, a new model that uses the power of video to edit your photos. It’s made to automate complex image adjustments while preserving artistic intent. By learning from millions of video frame pairs, Magic Fixup is able to understand the ways objects and scenes change under varying conditions, enabling it to perform edits that were previously challenging for AI tools.

“We propose a generative model that, given a coarsely edited image, synthesizes a photorealistic output that follows the prescribed layout,” Adobe writes in its paper. “Our method transfers fine details from the original image and preserve the identity of its parts. Yet, it adapts it to the lighting and context defined by the new layout.”

Key features of Magic Fixup include:

• Video-based training: Magic Fixup learns from video data, allowing it to understand how objects and scenes change under different conditions.
• Coarse-to-fine editing: Users can make simple adjustments, and Magic Fixup refines them with remarkable sophistication.
• Preservation of detail: The model maintains image integrity while making significant changes.
• Natural reflections and lighting: Magic Fixup can intelligently edit images while preserving natural reflections and lighting.

How does Magic Fixup work?

Magic Fixup uses two diffusion models working in parallel: a detail extractor and a synthesizer. The detail extractor processes the reference image and a noisy version of it, producing features that guide the synthesis and preserve fine details from the original image. The synthesizer then generates the output conditioned on the user’s coarse edit and the extracted details.

Adobe released Magic Fixup!

local @Gradio demo: https://t.co/pCk07tfehy

enable users to edit images with simple a cut-and-paste like approach, and fixup those edits automatically. pic.twitter.com/xBC1mMAkIW

— AK (@_akhaliq) August 21, 2024

Surprisingly, Adobe has decided to release the research code for Magic Fixup on GitHub, which is a major shift in Adobe’s approach to AI development.

Implications and possible concerns

For photographers, using a tool like this could lead to a faster and more automated process. Perfect for those who don’t really enjoy the painstaking process of photo editing. However, the implications of Magic Fixup extend far beyond simple photo touch-ups. It has the potential to revolutionize and change advertising, film and television production, social media, forensics, and historical preservation.

But just like with any AI tool, ethical concerns arise as well. The ease with which realistic image manipulations can be created could lead to increased misinformation and fake news. Adobe will likely face pressure to implement safeguards and possibly some form of digital watermarking to mitigate potential misuse.

You can read more about Magic Fixup here.

[via Venture Beats]