Karel Kubíček's site

Block Cookies, Not Websites: Analysing Mental Models and Usability of the Privacy-Preserving Browser Extension CookieBlock

Authors: Lorin Schöni lorin.schoeni@gess.ethz.ch, Karel Kubicek karel.kubicek@inf.ethz.ch, Verena Zimmermann verena.zimmermann@gess.ethz.ch

Abstract: In the modern web, users are confronted with a plethora of complex privacy-related decisions about cookies and consent, often compounded by misleading policies and deceptive patterns. Past efforts to enhance online privacy have failed due to their dependence on website compliance. A solution to this lies in privacy-enhancing tools that are directly controlled by the user. However, challenges related to the usability and flawed understanding of the tools’ functionality hinder their widespread adoption. To address this problem, we evaluated the browser extension CookieBlock as an example of a current tool, which supports users by blocking tracking cookies independent of website compliance.

We used a complementary approach consisting of an expert evaluation of CookieBlock and the related tools NoScript and Ghostery, and a laboratory user study focusing on the unique details of how users interact with CookieBlock specifically. The laboratory study with 42 participants investigated usage, mental models, and usability of CookieBlock based on eye tracking, interaction, and self-report data. While CookieBlock received good usability ratings, 18 participants were unable to solve a website breakage caused by cookie misclassification on their own. Overall, the results revealed flawed mental models of CookieBlock’s functionality and resulting challenges in making the connection between website breakage and cookie misclassification. Implications for CookieBlock and related applications include interface design recommendations supporting accurate mental models and the proposal of improved heuristics to better guide users and warn them about potential identified website breakage.

BibTeX

@article{shoni2024block,
  title={Block Cookies, Not Websites: Analysing Mental Models and Usability of the Privacy-Preserving Browser Extension {CookieBlock}},
  author={Lorin Schöni and Karel Kubicek and Verena Zimmermann},
  journal={Proceedings on Privacy Enhancing Technologies},
  volume={2024},
  number={1},
  year={2024},
  pages={192–216},
  doi={10.56553/popets-2024-0012}
}

Users seeking privacy turn to extensions

Majority of CS research shows that websites are collecting too much user data. A viable way of protecting the privacy are browser extensions that can prevent the data collection on the client side. However, these extensions are often designed by enthusiasts and they target other knowledgeable users.

This work evaluates the usability of privacy-enhancing browser extensions. In a expert evaluation, we compared extensions Ghostery (which is known for good usability), NoScript (which trades usability for complete control) with CookieBlock, extension developed in our prior work that represent an alternative of ML-driven privacy enforcement. We also evaluated CookieBlock in a laboratory study with 42 participants, evaluating their understanding of CookieBlock (mental models), and their interaction during installation and encountering a website where CookieBlock breaks the functionality due to ML-misclassification of a cookie purpose.

CookieBlock after our changes Based on results of our study, we proposed changes to COokieBlock, which we also implemented. Here is the popup after interface update.

User study evaluation

Design

The user study had four steps:

Observations

Eye trackingViolations histogram Heat maps of eye fixations on the drop-down menu for people who did (left) vs, did not (right) require help in making the connection between website breakage and CookieBlock.

Recommendations

We draw in the paper multiple design recommendations to not only CookieBlock but any privacy-enhancing browser extension.

Expert evaluation

Four usability experts evaluated the following aspects of the extensions. Here are the results:

Usability Heuristic CookieBlock NoScript Ghostery
Visibility of system status Installation: Feedback on successful installation but not on whether CookieBlock is already active and pinned. Feedback on setting selection and removing cookies is difficult to note in normal font size, not really clear what happened.
Settings: “Add Domain Exception” button switches to “Remove Domain Exception” when clicked and is greyed out when clicking not possible, greyed out button could be misunderstood as an error and difficult to understand why button is not always clickable -> indicate state via radio button? Difference between “Cookie Configuration” and “Settings” unclear, why is configuration not part of settings? Scale for Bias for Necessary Cookies not self-explainable. Unclear what happens when you choose “exempted”. Sometimes not clear that something is clickable, e.g., known cookie list or cookie statistics
Use: For lay users it might already be difficult to decide, in case of a problem, whether to pause cookie removal or to add domain exception. Feedback is only provided for pause cookie removal, then icon is greyed out. No feedback what happens on a specific website -> should at least be visible when clicking on icon.		 Installation: Feedback on successful installation but not on whether NoScript is already active and pinned.
Settings: Feedback that NoScript site cannot be configured because it’s privileged helpful, but looks like warning message -> should look more neutral. Many different symbols of which meaning only becomes clearer through mouseover. Greyed-out symbols indicate non-clickable state, but then no mouseover available to understand meaning. Unclear what lock symbol means and what happens when it is clicked and turns red and unlocked, appears like an insecure state. Click on domain leads to error message rather than forwarding to domain.
Use: Not clear what default setting is – trusted or untrusted? Feedback provided on which symbol is selected as it gets larger while the mouse hovers over one. Textual mouseover for each symbol available. Feedback on number of blocked things as compared to list that appears when clicking on symbol inconsistent, e.g., 6 things in list in pop-up and feedback that 7 blocked in symbol. Unclear why more blocked things appear when blocking temporarily deactivated. That symbol changes appearance when not active provides helpful feedback.		 Installation: Start in inactive state and first needs to be enabled, but provides visual information how and where to pin as well as feedback that setup was successful.
Settings: Visual feedback on selected settings, e.g., checkbox and red text next to selected type of blocked content
Use: Feedback on number of trackers blocked in little number below symbol and in detailed screen when clicking on symbol. Actions are visible and feedback on implications of choices is provided. Visual color-feedback which choice is activated. Meaning of “Requests modified” unclear -> Is only ad blocked on website or other content, too?
Match between system and the real world. Installation: /
Settings: General and advanced settings on one page -> should be better differentiated. Often technical language that might not be understandable, e.g., terms “classifier”, “domain”, “class”.
Use: /		 Installation: /
Settings: What is a privileged site? Naming and terms for options concerning default, trusted and untrusted is very technical and might be unclear to the user.
Use: /		 Installation: /
Settings: /
Use: The use of tracking categories like “Social Media” does not seem entirely clear and some trackers are simply “Unidentified.” The never-consent option claims that it also blocks tracking, but in reality it only rejects choices in consent notice popups and does not directly interact with tracking.
User control and freedom Installation: /
Settings: When closing, the icon disappears. It first has to be pinned, which might be unclear to user. Accidentally added domains or typos can easily be removed by clicking on a cross. Info that bias configuration is only for expert users is quite hidden -> could be a separate area.
Use: /		 Installation: /
Settings: When closing, the icon disappears. It first has to be pinned, which might be unclear to user. Accidentally added domains or typos can easily be removed by clicking on a cross. Info that bias configuration is only for expert users is quite hidden -> could be a separate area.
Use: /		 Installation: /
Settings: Information that Ghostery symbol first has to be pinned. Settings can be easily reversed, e.g., trust or block site.
Use: /
Consistency and standards Installation: Symbol on installation site and in browser consistent. The four cookie categories and the setting “keep track of cookie history” look very similar so that it looks as an additional category. Expectation that normally info on activating/pinning CookieBlock in installation process, but not given here.
Settings: CookieBlock symbol is in expected place.
Use: /		 Installation: Expectation that normally info on activating/pinning NoScript in installation process, but not given here. Symbol of NoScript in installation process looks different from the symbol in browser.
Settings: Symbols in pop-up are not intuitive (label in mouseover). No settings list or “general settings” as expected from platforms.
Use: /		 Installation: Symbol on installation site and in browser consistent.
Settings: standards are considered, e.g. green and red color, underlined text for links and further information.
Use: In the settings menu, information that does not belong together is visually grouped together so that it appears to be belonging together.
Error prevention Installation: While no errors seem apparent, unintended actions may be possible, e.g., by clicking on “Keep track of cookie history”, because it looks like an additional cookie category.
Settings: When click on cookie statistics, then all cookie data appears which looks like an error, no option to go back and no explanation but you can just close the tab. When you enter a domain exception you can remove it by clicking a cross which is an expected action.
Use: No easily visible error recovery solution in case of website breakage as option for cookie pause is available but not obvious as solution for problem (connection between CookieBlock and website breakage not visible).		 Installation: /
Settings: Click on a domain leads to an error message, unclear whether to click on proceed or cancel. Implications of selections in advanced settings are unclear.
Use: /		 Installation: /
Settings: /
Use: If site is restricted, no sub settings or improvements can be made, site first has to be activated again, hierarchy of settings thus seems not plausible.
Recognition rather than recall Installation: /
Settings: Symbol for CookieBlock with shield can be recognized and does not need to be recalled. In settings screen lack of symbols and structure -> would be beneficial for recognition.
Use: /		 Installation: Symbols for NoScript are not consistent.
Settings: Symbols are not recognizable without text.
Use: Symbols are not recognizable but require further explanation.		 Installation: /
Settings: Symbols are consistently used and explained (and info buttons available in many places).
Use: /
Flexibility and efficiency of use Installation: /
Settings: Configuration of cookies very detailed and complex for lay users -> better differentiation for lay and expert user would help to increase flexibility, all expert settings could be hidden or below lay user setting.
Use: /		 Installation: /
Settings: Settings use tabs for easy overview. Advanced settings are separated from general settings. Search function allows for shortcuts for experienced users.
Use: /		 Installation: /
Settings: /
Use: : It seems easy to block or trust overall, but difficult to set granular advanced settings. “Submit a tracker” seems to be an advanced feature, so unclear why positioned in main menu.
Aesthetic and minimalist design Installation: Information on how to provide feedback and suggestions on CookieBlock before actual cookie selection even though the latter is probably the user’s priority.
Settings: Settings and pop-up when clicking on icon is minimalist -> Setting screen could profit from structure, e.g., lay user and expert settings. Design of “cookie configuration” screen is not minimalist but has very detailed settings.
Use: /		 Installation: /
Settings: While minimalist in the sense that there is little text, there are many symbols that are not self-explanatory. Setting to be able to choose dark mode good in terms of accessibility. Change of symbol might not be necessary and is not consistently applied across screens.
Use: /		 Installation: Visual illustration of installation process. Aesthetic and modern design. Consistent use of symbols.
Settings: List and structure so that not all information is displayed at once.
Use: /
Help users recognize, diagnose, and recover from errors Installation: /
Settings: /
Use: Not clear that website breakage is related to CookieBlock from interface or feedback, no easily visible way out.		 Installation: /
Settings: /
Use: When content is blocked, NoScript symbol appears on blocked content (e.g., video), so connection between blocking and breakage can be made. However, no information what to do or how to recover.		 Installation: /
Settings: /
Use: Perhaps not clear that there is a connection between Ghostery’s blocking functionality and potential website breakage, but Ghostery does not seem disruptive in comparison.
Help and documentation Installation: Cookie types are explained on initial set-up page, but more info or different wording could be helpful in some cases, e.g., button “Categorize and remove stored cookies”.
Settings: Information text available for each setting, but some explanations require expert knowledge.
Use: No information on what CookieBlock is currently doing on website.		 Installation: /
Settings: No further help or information available on first glance.
Use: /		 Installation: Help module and information on why and how to pin extension available.
Settings: /
Use: In many places little information symbols and further information available.

Acknowledgement

We thank Yanis Isenring and Linda Fanconi for their contribution to data collection. We also thank Linda Fanconi for contributions to analysis.

Updates