Facial matching security
Facial matching is performed within the Secure Enclave using neural networks trained specifically for that purpose. Apple developed the facial matching neural networks using over a billion images, including infrared (IR) and depth images collected in studies conducted with the participants’ informed consent. Apple then worked with participants from around the world to include a representative group of people accounting for gender, age, ethnicity and other factors. The studies were augmented as needed to provide a high degree of accuracy for a diverse range of users. Face ID is designed to work with hats, scarves, eyeglasses, contact lenses and many types of sunglasses. Face ID also supports unlocking with a mask on iPhone devices starting with iPhone 12 and iOS 15.4 or later. Furthermore, it’s designed to work indoors, outdoors and even in total darkness. An additional neural network — that’s trained to spot and resist spoofing — defends against attempts to unlock the device with photos or masks. Face ID data, including mathematical representations of a user’s face, is encrypted and available only to the Secure Enclave. This data never leaves the device. It’s not sent to Apple, nor is it included in device backups. The following Face ID data is saved, encrypted only for use by the Secure Enclave, during normal operation:
The mathematical representations of a user’s face calculated during enrolment
The mathematical representations of a user’s face calculated during some unlock attempts if Face ID deems them useful to augment future matching
Face images captured during normal operation aren’t saved but are instead immediately discarded after the mathematical representation is calculated for either enrolment in Face ID or comparison with enrolled Face ID data.
Improving Face ID matches
To improve match performance and keep pace with the natural changes of a face and look, Face ID augments its stored mathematical representation over time. Upon a successful match, Face ID may use the newly calculated mathematical representation — if its quality is sufficient — for a finite number of additional matches before that data is discarded. Conversely, if Face ID fails to recognise a face but the match quality is higher than a certain threshold and a user immediately follows the failure by entering their passcode, Face ID takes another capture and augments its enrolled Face ID data with the newly calculated mathematical representation. This new Face ID data is discarded if the user stops matching against it or after a finite number of matches; the new data is also discarded when the option to reset Face ID is selected. These augmentation processes allow Face ID to keep up with dramatic changes in a user’s facial hair or make-up use while minimising false acceptance.