The future of smartphone camera technology
Apple is facing stiff competition for the camera crown – is it still king of the smartphone shooters?
After years of incremental changes and slight improvements, Apple came out with a radical camera upgrade in the iPhone 7 Plus.
That was the first iPhone to feature two cameras on the back, giving it a big advantage over earlier models. With it came new features: a telephoto lens with 2x optical zoom and 10x digital zoom; Portrait mode, which blurs backgrounds in pictures for a depth-of-field effect; and a new Image Signal Processor to handle your photos.
The camera in the iPhone 7 Plus put Apple at the forefront of mobile phone camera technology. But since then, it’s arguable that Apple has not stretched that lead – rather, its competitors have made a serious attempt to catch up with – and even surpass – the Cupertino giant.
When cameras first made a splash on mobile phones, the immediate goal for all involved seemed to be increasing the megapixel (MP) count as much as possible. That’s not surprising – the rear-facing camera in the original iPhone was just 2MP – but since then the focus has shifted, and megapixels aren’t much of a selling point any more.
Beyond a point, having a higher megapixel count doesn’t mean you’re taking better pictures, just bigger pictures. For mobile phones and their limited storage, that’s not always a good thing. As well as eating up your storage space, larger megapixel counts also make it harder to upload pictures or to shoot quick bursts of shots. Furthermore, more megapixels means having to fit more information into the phone camera’s sensor, which itself is very limited in size. As a result, your pictures can end up with more noise.
These days most phone makers have settled on 12MP or so for their rear-facing cameras. Instead, these companies are looking for other ways to differentiate their phone cameras. So what are they working on in their latest phones? And what tech is Apple developing to give its cameras an edge once more?
Machine learning
One way that all phone makers can advance their devices is through artificial intelligence (AI) and machine learning. These technologies are hot among the world’s phone manufacturers, and integrated cameras have not escaped their touch.
Google Pixel 2, for example, only has one rear-facing camera but is still able to create the kind of portrait mode effect the iPhone achieves
with two. It does that with what it calls dual-pixel autofocus, which essentially divides each pixel into two so that the camera can ascertain the depth of objects in the frame. Once it has done that, it automatically applies a blur to the background elements. This also means you get portrait mode on the front camera, which iPhone 8 and earlier do not.
Of course, imitating a dual-lens camera system using only one lens will not be perfect every time. But it’s an interesting example of the way the tech may be moving – if you can achieve a convincing portrait mode effect using only one lens, you can potentially save space inside the phone for other components, which Apple has always been keen to do.
Google isn’t the only company making use of artificial intelligence to improve its phones’ cameras. Huawei has done something similar in its new P2 Pro phone, the camera of which has received excellent reviews. Its Master AI system can detect more than 500 different scenarios, using this analysis to automatically adjust the camera’s settings appropriately. For example, if it detects people in the photo, it can activate portrait mode for you.
Huawei is an example of a company trying to make its phone camera easier to use – to instantly adjust settings and presets to predict what you want without you having to lift a finger. It shows how the improvements to smartphone cameras we’ve seen recently haven’t just come in hardware, but in how that hardware interacts with the phone’s on-board software in new ways.
Lenses in focus
Of course, hardware is still very important to these high-end cameras. The P20 Plus, for example, is one of the first to make use of three lenses for photography – the Lenovo Phab 2 Pro and Asus ZenFone AR also have three lenses, but they are used for augmented reality (AR), not photos. That trio consists of a 40MP colour camera with an f/1.8 lens, a 20MP monochrome camera with an f/1.6 lens, and an 8MP telephoto camera with an f/2.4 lens (the monochrome camera is used to add extra light, depth and detail information). According to Taiwan’s Economic Daily
News, Apple could follow suit in 2019 with its own triple-lens iPhone, equipped with a trio of 12MP cameras (although what exactly the third lens would be used for remains unclear). On iPhone X, iPhone 8 and
iPhone X is currently the only model to take Portrait mode selfies
8 Plus, the rear-facing cameras are all 12MP.
Apple’s main rival Samsung, however, is doing things a little differently. In its flagship Galaxy S9 Plus phone, it’s stuck with two rear cameras, but has an interesting trick up its sleeve: the main camera has a dual-aperture lens. This means that in low light conditions the phone automatically switches to an f/1.5 aperture, letting more light hit the sensor in order to make up for the darkness. In more favourable lighting, the lens switches to an f/2.4 aperture, which enables it to focus on achieving high levels of sharpness in the images it produces. Consequently, reviewers have lauded the S9 Plus’s photographic performance, especially in low light. In tests by reputable imaging website DxOMark, the S9’s camera received a score of 99, the highest score of any phone camera the website has reviewed – in fact, it scored 104 for its photo capabilities and 91 for video.
Sony, meanwhile, is also making headway in low-light photography, making use of its expertise in the DSLR arena to improve its smartphone cameras. In its new Xperia XZ2 Premium phone, for example, you can shoot photos at up to ISO 51200, giving it a big boost in low light conditions.
The question now is whether Apple is going to replicate – or surpass – the features that have already made it into its competitors’ impressive cameras.
Apple’s patent plans
While these developments from its rivals appear to leave Apple in the dust in some regards, there are plenty of reasons to be hopeful for Tim Cook and co. A number of patents relating to Apple’s camera technology might provide us with some interesting hints as to what Apple is planning.
A patent published by the US Patent and Trademark Office (USPTO) in March 2017 suggests that Apple might be working on a system to create 3D images using your iPhone. The patent shows that the system would compile “a relatively small number of basis images” to create a 3D image, rather than the dozens of pictures that are normally required. The patent also notes that the new system might be able to simulate “shadows and parallax artifacts” in some instances.
That’s not the only interesting camera patent from Apple that has recently emerged. A patent published by the USPTO in October 2017 suggests that Apple is developing a camera lens made up of six layered lenses. This would allow it to greatly reduce – or even totally eliminate – lens flares in your photos. Further Apple patent applications for enhancing smartphone cameras include its own accessory system to augment a phone’s built-in lenses. That’s something third parties already cater to, but Apple’s proposed system would use a ‘Sensing Technology Interface’ to analyse the alignment of an accessory and adjust the phone’s built-in lens to match. Naturally, Apple continues to look at ways to improve existing features. An application from February 2017 talks about using shock absorption components as a physical method of assisting optical image stabilisation. With the first generation of Face ID’s depth-sensing camera only on iPhone X at present, expect front-facing camera tech to evolve rapidly too, especially as more models gain this hardware. Tim Cook has made his excitement clear for what AR might enable in the future – surely more than Apple is currently doing, with novelties such as Animoji and the Clips app.
DSLR or iPhone?
Apple has come a long way since 2007’s 2MP iPhone camera. These days its smartphone shooter is, in some ways, able to compete with entry-level DSLRs for casual use.
Of course, there are plenty of reasons to buy a dedicated camera instead of relying on a phone – high-quality, changeable lenses and manual controls, say – but the camera tech in smartphones continues to advance apace. Apple and its rivals are doing great work here, and that competition, and applications such as AR, are sure to keep it going for years.
Is Apple going to replicate – or surpass – the features of its competitors’ cameras?