Apple's AI Photo Tools: A Double-Edged Sword

The latest iteration of iOS, version 27, introduces a groundbreaking suite of AI-powered photo editing features to the iPhone, widely recognized as the world's most popular camera. This marks a significant shift, and many may not yet fully grasp its implications.

While these new AI capabilities in iOS 27 might appear modest when compared to the advanced functionalities offered by devices like Google's Pixel phones, they nonetheless represent a pivotal moment for the iPhone. They fundamentally redefine what users can achieve within the native Photos app, challenging our very perception of what constitutes a "photo" or a "memory."

Currently available within the iOS 27 developer beta, these features are subject to further refinement before their public release. The update encompasses three distinct, or perhaps two and a half, new AI editing tools. The "Clean Up" tool, previously existing but largely ineffective, has received a substantial upgrade, now counting as a half-feature. This tool is designed to seamlessly remove unwanted elements, such as photobombers, from the background of images. Additionally, "Extend" allows users to intelligently expand the borders of a photograph, using AI to generate plausible filler content. The third, and arguably most ambitious and potentially problematic, feature is "Spatial Reframing," which simulates camera movement to enable users to recompose an existing photograph from a new perspective.

Let's begin with "Clean Up," which has been dramatically improved and is now genuinely effective. Unlike its predecessor, which relied solely on less powerful on-device models, the new version leverages more robust cloud-based AI. This approach mirrors Google's long-standing strategy, which has historically positioned its Magic Editor tools far ahead of Apple's earlier attempts. The previous on-device Clean Up often struggled to convincingly fill in removed details, frequently leaving behind noticeable artifacts and proving more cumbersome than beneficial. The enhanced "Clean Up 2.0," however, performs its task admirably.

From a user perspective, AI-driven object removal is arguably the least contentious form of generative editing. It offers practical utility, such as tidying minor imperfections or discreetly removing strangers from a background. The latest iOS implementation handles these tasks flawlessly, and it is anticipated to be a highly popular feature among iPhone users.

Moving to a higher plane of complexity and conceptual depth, we encounter "Extend." This feature can be conceptualized as the inverse of cropping, enabling users to expand the boundaries of their frame. This is particularly useful for compositions that might be too tight on a subject, allowing for the addition of "breathing room." Extend offers this capability, albeit with certain constraints. It appears to avoid altering human subjects and may sometimes restrict extensions to specific directions. Furthermore, it adds only a minimal amount of padding, which thoughtfully limits its potential for misuse—a design choice that is commendable. Similar to Clean Up, Extend delivers convincing results, often demonstrating a predisposition for symmetry. For instance, it successfully added a portion of a rally car that was originally out of frame, including a side mirror that mirrored one already present in the photo.

Notably, Extend seems less inclined to invent elaborate elements for photos compared to some earlier AI efforts, such as those from Samsung. However, during testing, it did generate a potted plant on a side table. While this addition appeared reasonably convincing, the author was aware of its artificial origin, raising a degree of discomfort about sharing such a modified image on platforms like Instagram.

While Extend operates within a two-dimensional photographic space, "Spatial Reframing" introduces a third dimension. Building upon an existing feature that lends a 3D-like quality to photos, Spatial Reframing allows users to reframe an image as if they had physically shifted the camera and altered their perspective within the scene. The extent of this "movement" is limited, roughly correlating to how far one's arm could have moved during the original capture. The core intent is to rectify minor framing imperfections that might have been missed during the initial shot.

For those with a penchant for precision, this feature holds significant appeal. It addresses those moments when a photo is otherwise perfect, save for a minor compositional flaw—for example, wishing one had stepped slightly to the left to eliminate a distracting element. Such nuances are often difficult to catch in real time, and Spatial Reframing is designed precisely for these subtle adjustments.

Despite its seemingly reasonable premise, Spatial Reframing harbors the potential for introducing significant conceptual discrepancies, even with minor adjustments. In one instance, while attempting to reframe a photo taken at a tech talk featuring Apple executives post-WWDC keynote, the AI fabricated an entirely new individual sitting next to Craig Federighi when the original framing had largely obscured another executive. This demonstrates the tool's capacity for creating non-existent elements.

Predicting and generating content in a two-dimensional space appears to be a less complex challenge for AI than doing so in three dimensions. The comparative results between Spatial Reframing and Extend underscore this difference, with the former often yielding "weirder" outcomes. When subjects are farther from the camera, the latitude for "re-composition" is reduced, leading to more realistic AI-generated elements. However, this often results in an image that is only subtly different from the original, prompting questions about the practical value of such minimal alterations.

Conversely, when subjects are closer to the camera, Spatial Reframing produces more dramatic, and often unsettling, effects. The increased scope for "re-composing" necessitates greater AI interpolation to fill in gaps. Attempting to alter the perspective in a selfie, for example, requires the AI to generate significant facial details, quickly leading to an "uncanny valley" effect. Even within its limited adjustment range, it can distort faces, making them appear skewed or "off," and is more prone to fabricating elements that were never present. While the concept of salvaging a poorly composed photo is appealing, the practical application often proves disappointing.

A minor reassurance comes from the fact that images edited with these AI tools are automatically assigned Synth ID labels, indicating AI modification. Instagram, for instance, recognized this information upon upload, although it only displays it if a user actively navigates to an "AI Info" menu for that specific image. Such labeling systems are currently far from foolproof. A greater concern, however, is the rapid erosion of the fundamental trust we typically place in photographs shared from personal devices. While Apple is not the most aggressive innovator in this space, even subtle doubts—whether about the authenticity of a seemingly innocuous houseplant or the precise vantage point from which a photo was taken—could cumulatively lead to significant issues over time.