Smartphones store large amounts of sensitive data, such as SMS messages, photos, or email. In this paper, we report the results of a study investigating users' concerns about unauthorized data access on their smartphones (22 interviewed and 724 surveyed subjects). We found that users are generally concerned about insiders (e.g., friends) accessing their data on smartphones. Furthermore, we present the first evidence that the insider threat is a real problem impacting smartphone users. In particular, 12% of subjects reported a negative experience with unauthorized access. We also found that younger users are at higher risk of experiencing unauthorized access. Based on our results, we propose a stronger adversarial model that incorporates the insider threat. To better reflect users' concerns and risks, a stronger adversarial model must be considered during the design and evaluation of data protection systems and authentication methods for smartphones.