ACSAC 2020 Paper #193 Reviews and Comments =========================================================================== Paper #193 Towards Usable and Secure Location-based Smartphone Authentication Review #193A =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 2. Some familiarity Paper summary ------------- The authors explored a novel approach to location-based smartphone unlocking, through an interview study, and a field study with a working prototype. Strengths --------- Location based unlock is a novel area for study. Combination interview study and field study with working prototype. Good description of the field study application's design. Weaknesses ---------- Field study validity and security negatively impacted by restrictions on removing a place in the first 10 day Comments for author ------------------- 2.1 - you need to be more specific on how you did the selective portion of your recruitment. It's the norm to put things like survey instruments in an appendix for ease of recreation and better context. Is it an accurate assumption that your survey was 6 questions, each in italics in sections 2.2.2 - 2.2.7? It's the norm to list the codes identified in an appendix. I don't see the 23 codes anywhere. In addition it would be interesting to see which of the 8 reason for selecting codes seemed to align with usability, and which with security. Your script for the pre interview explanations in 2.1's (1) and (2) should be included in an appendix. The three simple questions you asked to check understanding should also be included. As should information on what happened if you explained it some more. The choice of all of these can introduce bias into the results, which you do not discuss as a potential limitation. 2.2.1 - A table would make. your demographics information easier to read. It should include information on which participants were selectively recruited as well. 2.2.3 - People aren't very good at this kind of very abstract thinking. There are better ways to get higher quality feedback (role playing, wizard of oz, or as P14 says, finding out what they put up with for other registrations). The same for 2.2.5, 2.2.6, and 2.2.7. This reads more like early market data for design characteristics than an exploratory set of interviews. I see that 2.4 claims the field study mitigates that. However that would be true only if it validated or changed those findings, and I don't recall that you addressed that. 2.3 - (5) I don't think the data support having the same tolerance for failures of both lock and unlock. The data you report seemed tighter about locking failures. 3.3 - that FAR seems high for both user perception and actual security. 4.1 - I like your hypotheses. 4.2 - I appreciate footnote 1. Same comments about selective invitations as above. Disabling the "remove" button was an odd and harsh way to ensure compliance; some sort of logging would be more the norm. It calls into question the security of the participants (since they could determine that a previously acceptable location was unacceptable at any time, not just after an hour). It could definitely impact the choices the participants initially make (since they can't experiment much). It could have caused them to avoid those locations (or at least the ones that were removed after day 10). 4.3.2 - at this point (participants removing locations after the 10th day) I wanted confirmation that participants also had instructions about how to exit the study at any time. I would have liked to see that in 4.2. 4.3.5 - it would have been useful to know if size adjustments were reactions to failed unlocks. 4.3.6 - couldn't false unlocks have been missed, because the user didn't look at the phone when it happened? Review #193B =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 3. Knowledgeable Paper summary ------------- In this work, the authors worked on a location-based implicit authentication system. They have claimed to improve GPS based location authentication by adding Wi-Fi based location authentication to achieve more precision. The authors have done a preliminary study to identify problems and user’s expectations and then designed a system according to that. Later they have done another study about the usage of their developed smartphone app. They have found better location precision and introduced better flexibility in selecting location range. They have also claimed a significant amount of improvement in the usability of the unlocking of the lock screen. Strengths --------- 1. The paper is well written and well structured. Content of the article and all figures are easy to understand. 2. Introduction of WIFI RSSI based location determination is a significant improvement comparing to current commercial solutions. It also gives the flexibility of choosing the range of their preferred location. 3. The authors have developed their authentication system in a well-structured way, first determining the user perspective, designing the system, and then finally again user study. Weaknesses ---------- 1. The proposed system has many limitations, including a pretty high FRR (False Rejection Rate) and FAR (False Acceptance Rate). Also, their application consumes a lot of battery. It needs lots of improvement before public deployment. Location precision also needs to be improved. 2. The requirement and the field study size is limited. The researchers should include more people to extract more insights. Due to a small number of responses (For Example, Trusted location number is 1 for Church), it is hard to decide what it implies. 3. Location authentication security should be discussed in this paper. No threat models are presented for which the proposed system may vulnerable. Comments for author ------------------- In the requirement study, only one participant has previous experience in using a location-based authentication system. Requirement insights should come better from people who have experience of using it. The access point can be removed, or the location may have been changed. Also, the user can select a place where no access point is present. A proper mitigation strategy should be designed for these scenarios. There may be a wall or some barrier in the trusted location, which can reduce the trusted location limit and cause more FRR. So, obstacle analysis should be included. Will the phone remain idle while sitting idle in the permitted area? We know when idle, smartphones get locked for security reasons. When has the default lock screen been replaced with “Loclock” what will be the behavior in this case? In a trusted location, if the phone remains unlock all the time or automatically unlock after a notification arrives, then it will be vulnerable to frequent attacks like phone-malware attack, shoulder-surfing attack, etc. Notably, the attacker may get a chance at an indoor place like an office where more people work in a limited space. In the field study, many users found the authentication system as inconvenient, which implies that authors should consider more factors in usability and security. Review #193C =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 2. Some familiarity Paper summary ------------- This paper proposes an implicit authentication scheme that unlocks smartphones automatically based on location information. To meet users' usability and security expectations, the authors conduct an interview study with 18 participants and a field study with 29 participants. Specifically, they first summarize the key design requirements of this scheme through investigation, and then design the application accordingly. Finally, they conduct the comprehensive evaluations and analysis. Strengths --------- Strength: 1) The problem is well articulated. 2) The process of the two studies in this paper is reasonable. 3) The interview and field study results are well discussed. Weaknesses ---------- 1) The motivation is a bit weak. 2) Spending lots of efforts and spaces on WiFi location applications seems not well deserved. 3) The findings provided are not unexpected. Comments for author ------------------- 1) Some existing IA unlocking schemes (based on on-body Detection and Trusted Devices provided by Google’s “Smart Lock” ) seems to be more practical (secure and/or convenient) than schemes based on location information. Why the authors focus on location information? In other words, the authors should further emphasize the superiority (e.g., convenience pr stronger security) of schemes based on location information. 2) WiFi location fingerprinting is a rather mature research area, see the following works. I am wondering why the authors design their own one (not compared with existing schemes) but not employ existing ones? The authors should discuss this. Kun Qian, Chenshu Wu, Yi Zhang, Guidong Zhang, Zheng Yang, Yunhao Liu, "Widar2.0: Passive Human Tracking with a Single Wi-Fi Link", ACM MobiSys, Munich, Germany, June 10-15, 2018. The tool site: http://tns.thss.tsinghua.edu.cn/wifiradar/index_chi.html Mok, E., & Retscher, G. (2007). Location determination using WiFi fingerprinting versus WiFi trilateration. Journal of Location Based Services, 1(2), 145-159. Farshad, Arsham, et al. "A microscopic look at WiFi fingerprinting for indoor mobile phone localization in diverse environments." International conference on indoor positioning and indoor navigation. IEEE, 2013. Lin, Peng, et al. "A real-time location-based services system using WiFi fingerprinting algorithm for safety risk assessment of workers in tunnels." Mathematical Problems in Engineering 2014 (2014). Radu, Valentin, and Mahesh K. Marina. "HiMLoc: Indoor smartphone localization via activity aware pedestrian dead reckoning with selective crowdsourced WiFi fingerprinting." International conference on indoor positioning and indoor navigation. IEEE, 2013. 3) The authors do not provide detailed information about the application, such as the performance of the application (e.g., memory requirements). Considering that 7 participants have encountered communication errors, I'm concerning the stability and compatibility of the application. 4) The findings from the interview and field study is rather expected. Are there any unexpected findings?