POSTER: Breaking the Android Pattern Lock Screen with

Neural Networks and Smudge Attacks

∗
Panagiotis Andriotis Theo Tryfonas Zhaoqian Yu
University of Bristol, MVB, University of Bristol, QB, University of Bristol, MVB,
Bristol, BS8 1UB, U.K. Bristol, BS8 1TR, U.K. Bristol, BS8 1UB, U.K.
p.andriotis@bristol.ac.uk theo.tryfonas@bristol.ac.uk zy13643.2013@my.bristol.ac.uk

ABSTRACT The Android Consortium introduced the pattern lock screen

The Android pattern lock screen is a popular mechanism of- when they released the second version of their operating sys-
fered for user authentication on smartphones and tablets. It tem and became very popular because of the usability it
is a graphical password scheme that provides usability and offers. However, studies have shown that the authentica-
memorability. Despite the wide password space of the mech- tion scheme is vulnerable to smudge attacks [3] and shoul-
anism, there exist well-known techniques (such as smudge der surfing. We have also seen some early indications that
attacks) questioning its security strengths. With this study the formation of a graphical password (such as the Android
we aim to demonstrate that if we use previous knowledge, pattern lock) might be affected by heuristic rules originated
which describes the results of biased password input, in ad- from the human nature [2, 4]. The users’ perception on the
dition to a method that extracts traces of residues from a pattern lock screen security has been studied in a survey [1]
mobile device screen, we will be able to develop a lightweight that investigates the outcome of the introduction of a pass-
automated tool capable to predict the user’s chosen pass- word meter on the authentication mechanism. The conclu-
word. sion of this work is that a password meter could urge users
to be more careful when they form their patterns and shield
their systems with more complex passwords increasing the
Categories and Subject Descriptors security of their Android mobile devices.
D.4.6 [Software]: Operating Systems—Security and Pro-
tection 2. MOTIVATION
The current work aims to deliver an Android application,
which will be used as a lightweight forensic tool able to pre-
General Terms dict the pattern that unlocks the mobile device. There are
Security, Human Factors numerous ways to bypass the lock screen [5] but most of
them need root privileges in order to work. Another draw-
Keywords back is that in most of the cases the mobile device must be
restarted and this procedure could limit forensic examina-
Graphical, password, heuristics, forensics, tool, smartphone, tions because the volatile memory loses all its data. Thus,
prediction our intentions are to present a framework that will use the
oily residues left on the screen and also the heuristic rules
1. INTRODUCTION that define the password construction. The combination of
these resources will provide the capability to produce a list
The proliferation of mobile devices in modern societies with the most possible patterns that unlock the device under
and the increasing hardware capabilities have made smart- examination.
phones and tablets affordable tools that support their users In [3] the authors demonstrated the best conditions un-
in various tasks. Most applications store data in the de- der which the retrieval of residues is possible. The study
vices’ internal memory in order to work properly. Thus, our was replicated in [2] and additionally, the research included
smartphones and tablets contain a lot of personal informa- the examination of various features of patterns obtained by
tion, which should be protected by adversaries. running a web survey. The analysis portrayed that there is
∗Corresponding Author an inclination the users to start their patterns from specific
nodes. Also, the study revealed popular passwords, sub-
patterns (bigrams, trigrams) and ending points. We intend
to feed all these information into a tool that will have the
Permission to make digital or hard copies of part or all of this work for ability to capture a photo of the screen, extract residues (and
personal or classroom use is granted without fee provided that copies are directionality) and propose possible passwords to bypass the
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for third- screen lock.
party components of this work must be honored. For all other uses, contact
the Owner/Author. 3. METHODOLOGY
Copyright is held by the owner/author(s).
WiSec’14 Jul 23-25 2014, Oxford, United Kingdom. We assume that we acquire an Android smartphone that
ACM 978-1-4503-2972-9/14/07. runs the application we developed. The goal is to bypass
the pattern lock screen that protects a seized smartphone. 3.4 Evaluate the proposed passwords
The smudges that exist on the screen suggest that the user The final part involves the evaluation of the proposed
has cleaned it recently. We propose four distinct stages of scheme, which will be done on the actual seized device, by
activity in order to come up with the set of the patterns trying to break its pattern lock screen, using the proposed
that match the criteria we stated at Section 2. These are list from our system.
described below.
3.1 Capture photo 4. CONCLUSION
The investigator captures a photo of the screen under ex- The current work aims to merge Image Processing meth-
amination, using properties described at [3]. ods and Machine Learning techniques to take advantage of
possible security issues that biased input can cause to the
3.2 Clear image from ‘noise’ Android’s graphical user authentication scheme. The pass-
The application will perform various Image Processing word space of the pattern lock screen shrinks dramatically
steps on the acquired image to wipe the unwanted ‘noise’ if we take into account the fact that more than 50% of
and extract as many nodes as possible from the pattern that the passwords provided by users in previous studies [1, 2]
was used on the seized phone. started from the top left node. Such knowledge, in addi-
tion to the existence of nodes (monograms) retrieved from
3.2.1 Decolourisation screens will provide a framework that will be able to pro-
Grayscaling is one of the most commonly used pre-processing pose lists of patterns that break the mobile devices’ secu-
techniques. It is the process of converting a colour image to rity scheme. The project extends the OpenCV function-
a grayscale image, each pixel of which has the same value ality proposing improvements of existing algorithms (Otsu
for all channels (i.e. RGB). It simplifies and reduces com- thresholding, Canny Edge Detector) and brings to the foren-
putational requirements, and is often used as a prerequisite sics community a lightweight, easy to use tool to bypass the
for other processes such as thresholding. Android pattern lock screen authentication.

3.2.2 Fingerprint detection 5. ACKNOWLEDGMENTS

Being able to extract the contour of the trace-fingerprint
This work has been supported by the European Union’s
is enough to perform node extraction, though a pattern lock
Prevention of and Fight against Crime Programme “Ille-
can go in either direction; hence, we are also interested in
gal Use of Internet” ISEC 2010 Action Grants, grant ref.
directionality of the fingerprints.
HOME/2010/ISEC/AG/INT-002 and the Systems Centre
3.2.3 Canny Edge detection of the University of Bristol.
The aim of this stage is to extract the contour of the
trace, which consists of edges, by the use of an edge detector. 6. REFERENCES
An edge can be defined as a discontinuity in pixel intensity [1] P. Andriotis, T. Tryfonas, and G. Oikonomou.
within the given image. Complexity metrics and user strength perceptions of
the pattern-lock graphical authentication method. In
3.2.4 Thresholding Lecture Notes in Computer Science, volume 8533.
Thresholding is a simple yet powerful image segmentation Springer, 2014.
technique that converts a grayscale image to a binary image, [2] P. Andriotis, T. Tryfonas, G. Oikonomou, and
so that objects can be separated from their background in C. Yildiz. A pilot study on the security of pattern
an easier way. screen-lock methods and soft side channel attacks. In
Proceedings of the sixth ACM conference on Security
3.3 Build a Neural Network and privacy in wireless and mobile networks, pages 1–6.
3.3.1 Node extraction ACM, 2013.
[3] A. J. Aviv, K. Gibson, E. Mossop, M. Blaze, and J. M.
The processes discussed so far will provide a set of traces-
Smith. Smudge attacks on smartphone touch screens. In
nodes that will be put on a grid that is defined physically
Proceedings of the 4th USENIX conference on Offensive
by the dimensions of the device.
technologies, pages 1–7. USENIX Association, 2010.
3.3.2 Pattern Lock suggestion [4] S. Uellenbeck, M. Dürmuth, C. Wolf, and T. Holz.
This is the stage where the app has to figure out the prob- Quantifying the security of graphical passwords: the
ability of each pattern to occur and rank them in order to case of android unlock patterns. In Proceedings of the
provide a sufficient password suggestion. For this task we 2013 ACM SIGSAC conference on Computer &
will use Neural Networks (N.N.) and especially fuzzy N.N. communications security, pages 161–172. ACM, 2013.
The reason for using a fuzzy N.N. is that a fuzzy system [5] xda developers. [android][guide]hacking and bypassing
has a set of IF-THEN rules incorporated into the system, android password/pattern/face/pi. http://forum.
which is expandable. In the context of this project, an ex- xda-developers.com/showthread.php?t=2620456.
ample rule would be: “IF a pattern consists of knight moves, Accessed May 05, 2014.
THEN the likelihood is 40%”. With the addition of N.N.,
the fuzzy system is able to learn. Therefore, this approach
would better suit here because we can use the results of pre-
vious studies [1, 2] to train our scheme and learn heuristic
rules that define the formation of patterns.