Apple is a bit behind on the generative AI front, aside from a few minor features added to iOS 17. That said, 2024 is shaping up to be a substantial year for Apple on AI. All eyes are on iOS 18, which is expected to come with AI features including improved Siri.
Ahead of this publication, Apple researchers, in collaboration with the University of California, Santa Barbara, presented open source artificial intelligence model who understands instructions in natural language. In tiny, you tell the AI to change something in the photo, and it will.
What is Apple’s MGIE AI Image Editor?
Dubbed “MGIE” (MLLM Guided Image Editing), this novel AI model accepts standard commands from the user to achieve three different editing goals: “Photoshop-style modification, global photo optimization, and local editing.”
Photoshop-style modifications include actions such as cropping, rotating, and changing the background; global photo optimization includes adjusting effects for the entire image, including image brightness, contrast or sharpness; while local editing affects specific areas of the image such as its shape, size and color.
MGIE is mainly based on the MLLM (Multimodal Immense Language Model), which is a type of LLM capable of interpreting visual and audio elements in addition to text. In this case, MLLM is used to accept user commands and interpret them as the correct editing direction. MGIE research article explains that this is traditionally a arduous task because user commands can often be too vague for the system to properly understand without additional context. (What does the show say “make pizza look healthier” should mean?) But researchers say MLLM methods like MGIE are effective in this case.
Based on the research article, MGIE enables many different types of visual editing. You can ask him to add a lightning bolt to an image of a body of water and make the water reflect this lightning bolt; remove an object in the background of an image, such as a person who has accidentally photobombed; turn things into other things, like a plate of donuts into a pizza; escalate sharpness on a blurry object; remove text from an otherwise nice photo and take advantage of many other possibilities.
You can get an idea of how the technology will work by reading the full research paper, which includes examples of how the editor works; is available here.
Of course, this is not the first employ of artificial intelligence in photo editing. Photoshop has tons of AI editing tools for some time now, including those generated based on user suggestions. But MGIE may be the most realized vision of a command-based AI image editor yet.
How to try Apple’s MGIE image editor yourself
Because the model is open source, anyone can download it and integrate it with their own tools. However, if you’re like me and don’t know where to start, you can give it a try this is a demo conducted by one of the project researchers. You can upload the image you want to edit, enter a command, and then process it.
However, right now the demo is backing up a sizable queue of requests. I am currently one of 237 and I believe this number may continue to grow as more and more people want to try this model.
It is unclear whether or how Apple will integrate MGIE into its own platforms. But if the company had a year to do so, it would definitely be 2024.
