PhotoKit helps developers call the shots
iOS 8 in context
This post is an excerpted chapter from the iOS 8 in context ebook.
In previous years, Apple’s keynote at its annual Worldwide Developers Conference has been the platform for the introduction of consumer-facing products and features, including the iPhone 4 in 2010 and iOS 7 in 2013. But this year’s conference was different—the final third of Apple’s presentation addressed almost zero consumer features. Instead, Cook and his team fulfilled thousands of developers’ wish lists to the sound of thunderous applause. While non-developers might have been itching for a glimpse of this fall’s anticipated new iPhones, little did they know that these additions to iOS will have a greater impact on their everyday lives than any flashy new hardware or buzz-worthy new feature.
If the redesign in iOS 7 was a fresh paint job, iOS 8 will be an entirely new engine. This is iOS 8 in context.
PhotoKit, iCloud Photo Library, and Camera APIs
In the early 2000s, as Steve Jobs began reinventing the company he’d founded, Apple prescribed a vision for the future of personal computing. The Mac was to become the digital hub of users’ lives, consolidating and managing all of their music, video, documents, and photos into one tidy fluorescent blue orb. At the time, these content types were dependent on external equipment, and Apple Stores showcased the latest digital cameras, camcorders, and even pre-iPod MP3 players alongside shiny new iMacs and PowerBooks. Jobs’s vision rang true with millions of customers, and Apple began developing some of the industry’s best consumer photo, video, and multimedia software in its iLife suite: including iPhoto, iMovie, iTunes, and iDVD. The universally popular iPods that followed began to flesh out Apple’s product family, serving as a digital “escape pod” for audio content sourced from CDs and locked up in iTunes. The Mac had earned its place as the center of users’ digital lives, but the iPod proved that center of that content nexus might not hold—and when Apple introduced the iPhone, it ushered in the era of internet connectivity in our pockets. The central hub of digital content began rapidly evacuating Mac and PC hard drives as the ubiquity and accessibility of cloud services became prevalent. iPhones, now, began serving as the center of Apple fans’ digital lifestyles. And not only could they store and manage all of that content, iPhones were equipped to begin creating it.
Digital cameras and camcorders began disappearing from Apple’s online stores and retail locations, and with good reason. As the iPhone evolved and its camera capabilities steadily improved, the smartphone could replace point-and-shoot cameras for most customers. The addition of video recording in the iPhone 3Gs, and its bump to 1080p HD video in the iPhone 4s, obviated the need for digital HD camcorders. iPhones even began to eclipse the iPod’s usefulness with easy music downloads and playback capabilities, and third-party apps even allowed iPhone customers to begin recording music of their own. With iOS 5, customers could purchase an iPhone or iPad as their primary and only digital device, eliminating the need to purchase a $1500 MacBook with an all-encompassing $500 tablet. The notion of a “digital hub” was well-founded, but the label was ill-placed: iOS had become master of our songs, our videos, and our photos—but the tools Apple provided to manage them weren’t yet up to the task.
Perhaps more than any other content, photos are the lifeblood of iOS. Among the most popular third-party applications center around ways to store, share, and interact with our devices’ photo and video content, and Apple itself has expanded its iLife suite to offer iPhoto and other apps on iOS. But there was no good solution for managing users’ expanding photo libraries, and thousands upon thousands of high-resolution images began eating up iPhones’ limited storage capacities. Apps like Dropbox’s Carousel or Google+ Photos aimed to supplement this storage with cloud synchronization, and even Apple’s own iPhoto duplicated photo libraries into new organizational schemes, but photos remained difficult for users to organize, sort, and reasonably understand. While iOS 7 revamped the stock Photos app with new “moments” and “collections” views, the platform’s treatment of images was sorely in need of an overhaul. Apple products have a legacy of photo management and integration, and iOS 8 gave the company an opportunity to retool and build a framework for the next chapter. With new camera tools and its PhotoKit API, Apple is solving photo management problems as old as iOS while setting the foundation for how third-party apps can interact with mobile photos in the future.
Framing the shot
Apple has always developed a number of photo APIs for third-party apps to access the Camera Roll and submit new photos. This allows users to take photos in camera apps of their choice, and to save any edits from dedicated photo editors directly into the Photos application, in order to share it or access it from other apps. The frameworks were limited to photos and videos, and severely limited in terms of content compatibility: when Apple introduced “burst” photo sets and slow-motion video with the iPhone 5s, certain apps weren’t able to access or modify that content. And as Apple began tweaking how photos are stored and organized within the Photos app, adding Shared Photo Streams and iCloud Photo Stream content in iOS 6, third-party editors had trouble making use of the wealth of off-board photo and video content, as well.
In iOS 8, Apple completely rethought how photos and videos could be stored and managed on iOS devices. Taking a cue from Dropbox and Google’s cloud-centric ideas in the space, Apple launched a beta version of a new service called iCloud Photo Library alongside iOS 8.1. Rather than organizing photos taken with a particular device into its “Camera Roll,” and rather than showing photos taken with iCloud-connected devices in a time-sensitive iCloud “Photo Stream,” iCloud Photo Library saves 100% of user-generated photos and videos from any of their devices in one cohesive location. Every photo taken since the beginning of time, from any device or any source, coexists in a single chronological list. With smart features in Settings to prevent full-resolution photos and videos from clogging up limited iOS device storage, iCloud Photo Library gives iOS users access to potential thousands of photos and videos right from their devices. But with all these dramatic changes to photo library structure and organization within the Photos app, how can third parties begin accessing the data and manipulating content?
Apple also released PhotoKit in iOS 8, an API that provides access to the users’ photo and video content in new ways. PhotoKit allows third-party apps to read content from the iCloud Photo Library, including burst mode photos, panoramas, and video, and provide users access to their content in the plethora of photo-editing applications in the App Store. PhotoKit enables third-party apps to edit photos directly in the Photos application, without the need to first import them. Further, it lets apps non-destructively make edits and enhancements to those photos, meaning third-party image editors can share universal revision histories between applications without compromising the original document or unnecessarily duplicating photos. PhotoKit modernizes how applications on iOS approach photos, and it will begin to allow for new types of photo editors as developers begin adopting modern iOS 8 frameworks. But it’s only the beginning of what iOS 8 brings to the world of mobile photography, and its additions only scratch the surface of what iOS devices will soon be capable of.
Shutter speak
The iPhone’s camera—and, especially in its latest iteration, the iPad’s camera—is among the device’s most popular and essential features. As the iPhone continues to dominate mobile photography, replacing point-and-shoot cameras for millions of people and remaining the most popular camera of any type on Flickr, Apple has invested heavily in improving the camera’s quality and speed. iPhone 6 represents a culmination of these efforts: boasting an 8-megapixel sensor, a five-element lens, and new “Focus Pixels” for smart autofocusing, the iSight rear shooter is considered best-in-class for mobile photography. Apple’s software, too, has continually evolved to better support the camera, adding face detection, processor-assisted video stabilization, and algorithmic low-light improvement over the years. The camera app enjoyed a slight overhaul in iOS 7, with new filter effects and photo modes that, for some, matched the utility of certain popular third-party apps. But since the beginning of the App Store, iOS users have demanded more from their cameras, and third-party camera applications have sprung up to fill that void.
In iOS 8, Apple introduced new Camera APIs to allow these apps greater control over the photos they shoot, and to prevent them from writing custom image analyzation code themselves. The new AVFoundation framework allows third-party applications to better leverage iOS devices’ advanced camera technologies and image processing capabilities, allowing apps to take direct control over camera focus, white balance, and settings for exposure. The capabilities will empower a new generation of secondary camera apps, and enable third-party apps like Instagram to add new sophistication to their in-app viewfinders. In short, AVFoundation will help users take higher-quality photos with a greater degree of customization no matter what app they choose, and will put the full power of iOS devices’ powerful iSight cameras and image processors directly into end users’ hands. Third party apps can now help take better, more tightly controlled photos—but Apple’s investments in iOS 8 photo capabilities don’t stop there.
Editing in post
For years, the stock iOS Photos application has allowed for basic editing of photos snapped on-device: cropping, reducing red eye, and even a handy “auto enhance” functionality that quickly balances exposure and saturation. For years, more advanced photo editing was available in the iPhoto app for iOS, which included powerful controls for highlights, shadows, color control, and even some social-friendly filters. iPhoto was a complex and overpowered animal, and one of the most frequently requested hands-on workshops at Apple’s retail locations focused on humanizing its convoluted interface. iPhoto matched its desktop counterpart in many ways, but added new organizational views like Journals and in-app Albums into the mix. The app was powerful almost to a fault: overcomplicated and confusing even to power users, Apple announced they were phasing out iPhoto development earlier this year.
But what comes next for Apple’s storied photo editors? Despite years of development investment into the iPhoto suite on both its platforms, Apple announced its intention to transition much of the app’s utility into the stock Photos app on iOS. In iOS 7, Apple began adding additional layers of photo editing enhancements, allowing the application of filters to bolster the stock Photos app’s image processing capabilities. This focus continued into iOS 8, but was amplified significantly—Apple added new controls for exposure, saturation, and black and white alteration directly into the Photos app. But Apple’s ambitions for its mobile image editors stretched beyond basic adjustment sliders—the company wanted iOS to match its professional-grade desktop editor, Aperture.
The answer came with the addition of Extensibility in iOS 8. Just as social networking apps could offer extension bundles to the action menu or news tickers could live within Notification Center, third-party photo editors, too, could offer their services directly to the Photos app. New versions of popular third-party photo editing apps like VSCO Cam or Afterlight can take advantage of a new ellipsis button within the Photos editor view, and can provide custom editing interfaces and effects directly to photos within the Photo Library. Because of additions with PhotoKit, these edits are non-destructive and happen directly within the Photo Library, eliminating tedious duplication and additional content groupings locked away within third-party apps. The approach empowers a new generation of near-headless photo editing applications, which can now be tucked away into a folder until their services are summoned by the Photos app’s editor tool.
iPhoto on iOS had been wholly supplanted, but Apple’s new approach to photo management applications isn’t limited to iOS devices. Apple demoed a new application for OS X Yosemite at WWDC, aptly called Photos, which allows Mac users to access their full iCloud Photo Library, as well. Leveraging many of the same extension bundles now available for the Mac platform, Photos for Mac can replicate many of the more advanced features of iPhoto—and even of Aperture. Herein lies the real reason for Apple’s abandonment of photo editing software: third parties can do it better, and Apple is now giving them the tools to start. With PhotoKit and Extensions on iOS and OS X, and with new access to the camera with the AVFoundation framework, apps are more empowered than ever to insert themselves into the mobile photographer’s workflow. Whereas Apple once saw its desktop computers and powerful media management software suite as the digital hub of connected users’ lifestyles, the company has started to adapt to the realities its breakthrough products have wrought. While cloud services have replaced the Mac hard drive for monolithic storage, there is no centralization to be had with photo editing tools—apps and users can create custom photography experiences for themselves. And for developers whose apps even remotely deal with users’ photo and video content, the canvas has never been wider.