Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Playing in the Sandbox: A Study on the Usability of Seccomp

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Even experienced Seccomp developers cannot reliably sandbox a simple program.

desk verdict First empirical Seccomp usability study with honest methods; the novelty claim is too broad and the minimal-syscall metric needs grounding, but the core finding — divergent sandbox designs from experienced developers — holds. read the letter →

arxiv 2506.10234 v1 pith:OHAALOEF submitted 2025-06-11 cs.OS cs.SE

Maysara Alhindi , Joseph Hallett This is my paper · ORCID
classification cs.OScs.SE
keywords seccompusabilityLinuxsandboxingsystemcallfilteringusablesecurityAPIsdeveloperstudyleastprivilegesandboxdesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a usability study of Seccomp, Linux's system-call sandboxing mechanism, and it tries to explain why so few applications are sandboxed despite Seccomp's power. Seven developers who had used Seccomp before were given one hour to sandbox a 115-line watchdog program with no dynamic code. Every participant produced a different sandbox design; six produced working sandboxes, one did not, and two over-privileged their sandbox by trusting strace output (and, in one case, ChatGPT) and allowing unnecessary syscalls. The central difficulty was mapping the program's functionality to the syscalls it makes, and participants repeatedly expressed confusion about where syscalls came from. The paper concludes that Seccomp's flexibility, built on the complex and architecture-dependent syscall interface, puts too much burden on developers and that abstraction, automation, and function-level sandboxing would help.

What carries the argument

The paper's central object is Seccomp's filter model: developers initialize a seccomp context, add rules that allow or deny individual syscalls (optionally checking argument values), and load the filter at some point in the program. The study uses a 115-line C watchdog as the common task and traces participants' rule-building against strace output, source reading, and documentation. The load-bearing mechanism is the mapping from program behavior to syscall names—participants had to figure out which syscalls the program made, where, and which were setup-only—and the paper shows this mapping, not the BPF syntax, is what makes Seccomp hard.

What would settle it

Run the same 115-line watchdog sandboxing task with a larger random sample of experienced Seccomp developers in a routine, unobserved work setting; if most of them independently produced similar, correctly-scoped sandboxes and could name where each allowed syscall is used, the paper's central claim would fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that Seccomp is flexible to a fault: for a simple, statically-behaving program, experienced developers do not converge on a sandbox, and the main source of difficulty is not writing BPF rules but deciding which syscalls the program actually needs and where they occur. The seven participants all stated the goal of least privilege, yet their allowed syscall sets, filter placement, use of argument checking, and code structure differed completely. P2 was the only participant to restrict syscall arguments. P5 outsourced filter generation to ChatGPT; P5 and P7 allowed unneeded syscalls such as execv and mmap because they trusted strace output. P4 could not finish. The paper takes this divergence and confusion as evidence that the syscall interface is a poor place to impose a sandbox policy for ordinary developers, and that a more abstract, automated, or function-local model would make sandboxing something developers can do reliably.

Load-bearing premise

The study assumes that the behavior of seven self-selected, all-male, compensated developers working for one observed hour on a contrived watchdog task tells us how developers in general would use Seccomp in real production settings.

Editorial extensions

If this is right

  • Even simple programs produce divergent sandbox designs, so without standard patterns the security of a sandboxed program depends heavily on the individual developer's approach.
  • Developers' difficulty mapping syscalls to program functionality leads to over-privileged sandboxes, which weakens the least-privilege guarantee that motivates sandboxing in the first place.
  • Abstraction layers, automation, and function-local sandbox rules align with the mental models participants described and could make sandboxing more reliable.
  • The system call interface itself is a questionable enforcement point for security policy, as participants could not always tell which syscalls were needed, which were setup-only, and which were architecture variations.
  • Without verification tools or standard design patterns, developers cannot easily tell whether their Seccomp policy is correct, and several participants reported low confidence even in working solutions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper, the finding that some developers blindly trusted strace output suggests that automated Seccomp filter generators should be evaluated not only for functional correctness but also for over-privilege relative to the program's actual needs.
  • One testable extension is to compare over-privilege rates in LLM-generated Seccomp policies against expert-written policies on the same program, using the watchdog task as a benchmark.
  • The function-local sandboxing idea proposed by one participant could be prototyped as a library or language extension that derives each function's required syscalls automatically, then tested with a larger and more diverse developer sample.
  • The paper's recruitment through security mailing lists and freelancing platforms may have selected for unusually security-conscious developers, which would make the observed struggles a lower bound for the general developer population.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper reports a qualitative usability study of the Linux Seccomp sandboxing API. Seven developers with prior Seccomp experience were asked to sandbox a 115-line C watchdog program under observation, using a think-aloud protocol, a System Usability Scale questionnaire, and a participatory design exercise. The paper finds that participants arrived at very different sandbox implementations, struggled to map syscalls to program functionality, commonly over-privileged their sandboxes (notably by trusting strace output), and one participant could not produce a working filter. The authors conclude that Seccomp's flexibility and low-level interface create usability challenges and argue for higher-level abstractions and design patterns.

Significance. The paper is a useful and timely contribution to the small but growing literature on usable security APIs. Its strengths include a transparent study protocol (Appendix B), direct participant quotes, a detailed description of participants' implementations (Table 2), and what appears to be the first empirical usability study of Seccomp. The qualitative findings are plausible and contribute new evidence about why sandboxing adoption is low. The paper is appropriately cautious about generalizability, but a key security-relevant claim—that two participants' sandboxes were over-privileged by allowing unnecessary syscalls—currently lacks a documented ground-truth analysis and needs strengthening before publication.

major comments (3)
  1. [Section 4.1.2, Table 2] The claim that P5 and P7's sandboxes were over-privileged and allowed '15 syscalls that were not needed by the watchdog program' is not supported by a reported minimal-syscall analysis. The paper does not state how the authors determined that ioctl, mmap, access, and execv are unnecessary, nor does it report the test conditions (libc version, compile flags, network reachability, signal handling, log-rotation paths) under which a ground-truth syscall set was obtained. Because the watchdog program uses glibc I/O and allocation functions whose syscall usage can vary at runtime (e.g., mmap for large malloc chunks or access for permission checks), a single strace run is insufficient to establish that these syscalls are never required in a correct execution. Without this analysis, the conclusion that strace-based workflows lead to 'somewhat ineffective' sandboxes is overstated; I recommend either adding a systematic enumeration of required syscalls under varied execution conditions or tempering the security-effectiveness claim.
  2. [Section 3.3 and Appendix B.2] The study did not record participants' screens, yet Table 2 reports counts such as 'Times documentation referred to' and 'Attempts running the code', and Figure 1 constructs a detailed timeline of participant steps. The threats-to-validity section does not explain how these data were captured reliably without screen recording, nor what information may have been missed or misattributed. Please describe the real-time note-taking and verification process used during the sessions and discuss the potential impact of this limitation on the process-level findings.
  3. [Section 4.1 (P4 case)] The interpretation of P4's failure as a Seccomp usability problem is somewhat confounded by P4's choice to work with raw BPF filters, which is a particularly low-level interface, rather than the libseccomp wrapper that other participants used. While raw BPF is part of the Seccomp ecosystem, the authors should clarify whether the task instructions encouraged any particular API and discuss how much of P4's struggle is attributable to Seccomp versus to the specific programming interface P4 selected.
minor comments (6)
  1. [Section 3.1 vs Appendix C] The session length and compensation are described inconsistently: Section 3.1 says 1.5 hours and 'approximately $250', while Appendix C says '2 hours' and '£200'. Align these details.
  2. [Section 4.2.3] The typo 'SMPT_ACT_ALLOW' should be 'SCMP_ACT_ALLOW'.
  3. [Section 4.4] The SUS responses are presented only as individual item ratings (Figure 2); reporting the aggregate SUS score would allow comparison with other usability benchmarks.
  4. [Section 2.2] The authors' prior work [2] is cited for background; please make the self-citation more explicit in the text.
  5. [Section 4.3.2] The discussion of P3's 'time' syscall and VDSO should clarify whether VDSO syscalls are subject to seccomp filtering and whether omitting them from the filter is safe.
  6. [Section 4.1.2] The list of essential syscalls includes 'clock_nanosleep' but the program calls sleep(); clarify whether this is the libc wrapper behavior and whether variants such as 'nanosleep' were also observed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the study's findings are direct empirical observations; the only self-citation is background and not load-bearing.

full rationale

This paper is an observational usability study, not a derivation or prediction chain. Its central findings—that seven experienced developers produced different Seccomp implementations, that they struggled to map syscalls to program functionality, and that some over-privileged their sandboxes—are grounded in recorded sessions, code artifacts, and think-aloud transcripts, not in a fitted model or prior theoretical framework. The only self-citation (Alhindi and Hallett 2024, ref. [2]) is used as background motivation to describe low sandboxing adoption in Linux packages; it does not supply the paper's conclusions or define its outcome measures. The authors' characterization of P5 and P7 as over-privileged relies on an implicit judgment about which syscalls the watchdog program actually needs; this judgment is contestable on evidentiary grounds and could weaken a specific empirical claim if the minimal syscall set is wrong, but it is not circular—the finding is not defined in terms of itself, nor is any claimed result equivalent to its own input by construction. The paper explicitly acknowledges validity limitations such as the tiny, non-generalizing sample and potential Hawthorne effects (Section 3.3), which further confirms that the authors treat their results as observed phenomena rather than as consequences of their assumptions. No equation, parameter, or construction reduces a claimed result to an input, so there is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This qualitative study does not fit parameters or introduce entities; it rests on domain assumptions about the representativeness of the sample and the validity of self-report, which are listed above.

assumptions (4)
  • domain assumption Think-aloud verbalizations reflect actual reasoning processes.
    Section 3 states participants were asked to think aloud and audio was transcribed; the analysis of struggles relies on these self-reports.
  • domain assumption The watchdog task is representative of real-world sandboxing tasks.
    Section 3.1.1 describes the task as simulating a common watchdog service; the transferability to other programs is assumed.
  • domain assumption The researchers' classification of sandboxes as functional or over-privileged is accurate.
    Section 4.1 and Table 2 assess which sandboxes work and which allow too many syscalls, based on the researchers' model of the program's syscall needs.
  • domain assumption The SUS adapted to Seccomp is a meaningful usability measure.
    Section 3.1.2 uses a modified SUS; its validity for Seccomp is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Playing in the Sandbox: A Study on the Usability of Seccomp." pith.science (2026). https://pith.science/paper/OHAALOEF

@misc{pith2026250610234,
  author       = {Pith},
  title        = {Pith review of: Playing in the Sandbox: A Study on the Usability of Seccomp},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OHAALOEF}},
  note         = {Machine review of arXiv:2506.10234}
}
read the original abstract

Sandboxing restricts what applications do, and prevents exploited processes being abused; yet relatively few applications get sandboxed: why? We report a usability trial with 7 experienced Seccomp developers exploring how they approached sandboxing an application and the difficulties they faced. The developers each approached sandboxing the application differently and each came to different solutions. We highlight many challenges of using Seccomp, the sandboxing designs by the participants, and what developers think would make it easier for them to sandbox applications effectively.

Figures

Figures reproduced from arXiv: 2506.10234 by the authors.

Figure 1
Figure 1. Timeline of participants steps when implementing the sandbox [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Answers to SUS questions. learn a lot of things before using Seccomp, mainly because it needs an understanding of the concept of syscalls, and of system internals. “If we really just talk about developers and various backgrounds. I would imagine that it’s very, very difficult.”—P2 “You have to learn the byte code. You have to learn how this library works underneath.”—P3 “I was able to sort of get going with it fairl… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Measurement Study on the Adoption of Pledges and Unveils in the OpenBSD Operating System

    cs.SE 2026-07 accept novelty 6.0 of 10

    Longitudinal counts over 19 OpenBSD releases show continuous growth in pledge and unveil use, weak size correlation, and heavy but not exclusive reliance on I/O promises.

Reference graph

Works this paper leans on

55 extracted references · 54 canonical work pages · cited by 1 Pith paper

  1. [1]

    Com- paring the usability of cryptographic APIs

    Yasemin Acar, Michael Backes, Sascha Fahl, Simson Garfinkel, Doowon Kim, Michelle L Mazurek, and Christian Stransky. Com- paring the usability of cryptographic APIs. In 2017 IEEE Symposium on Security and Privacy (SP), pages 154–171. IEEE, 2017

    work page 2017

  2. [2]

    Sandboxing adoption in open source ecosystems

    Maysara Alhindi and Joseph Hallett. Sandboxing adoption in open source ecosystems. In SESoS-ICSE 2024: 12th ACM/IEEE Interna- tional Workshop on Software Engineering for Systems-of-Systems and Software Ecosystems. Association for Computing Machinery (ACM), 2024

    work page 2024

  3. [3]

    A comparison of UNIX sandboxing techniques

    Jonathan Anderson. A comparison of UNIX sandboxing techniques. FreeBSD Journal, 2017

    work page 2017

  4. [4]

    Nigel Bevan, James Carter, and Susan Harker. ISO 9241-11 revised: What have we learnt about usability since 1998? In Human-Computer Interaction: Design and Evaluation: 17th International Conference, HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Pro- ceedings, Part I 17, pages 143–151. Springer, 2015

    work page 1998

  5. [5]

    Automating Seccomp filter generation for Linux applications

    Claudio Canella, Mario Werner, Daniel Gruss, and Michael Schwarz. Automating Seccomp filter generation for Linux applications. In Pro- ceedings of the 2021 on Cloud Computing Security Workshop, pages 139–151, 2021

    work page 2021

  6. [6]

    Sysfilter: Automated system call filtering for commodity software

    Nicholas DeMarinis, Kent Williams-King, Di Jin, Rodrigo Fonseca, and Vasileios P Kemerlis. Sysfilter: Automated system call filtering for commodity software. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), pages 459–474, 2020

    work page 2020

  7. [7]

    Aligning usability and security: a usability study of Polaris

    Alexander J DeWitt and Jasna Kuljis. Aligning usability and security: a usability study of Polaris. In Proceedings of the second symposium on Usable privacy and security, pages 1–7, 2006

    work page 2006

  8. [8]

    No good reason to remove features

    Steve Dodier-Lazaro, Ingolf Becker, Jens Krinke, and M Angela Sasse. “No good reason to remove features" expert users value useful apps over secure ones. In Human Aspects of Information Security, Pri- vacy and Trust: 5th International Conference, HAS 2017, Held as Part of HCI International 2017, Vancouver, BC, Canada, July 9-14, 2017, Proceedings 5, pages...

    work page 2017

Show all 55 references
  1. [9]

    Traps and pitfalls: Practical problems in system call interposition based security tools

    Tal Garfinkel. Traps and pitfalls: Practical problems in system call interposition based security tools. In In Proc. Network and Distributed Systems Security Symposium, 2003

    2003

  2. [10]

    Seccomp filters from fuzzing

    Marcus Gelderie, Valentin Barth, Maximilian Luff, and Julian Birami. Seccomp filters from fuzzing. In SECRYPT, pages 507–512, 2022

    2022

  3. [11]

    Developers are not the enemy!: The need for usable security APIs

    Matthew Green and Matthew Smith. Developers are not the enemy!: The need for usable security APIs. IEEE Security & Privacy, 14(5):40– 46, 2016

    2016

  4. [12]

    Social desirability bias

    Pamela Grimm. Social desirability bias. Wiley international encyclo- pedia of marketing, 2010

    2010

  5. [13]

    Why don’t software developers use static analysis tools to find bugs? In 2013 35th International Conference on Software Engineering (ICSE), pages 672–681

    Brittany Johnson, Yoonki Song, Emerson Murphy-Hill, and Robert Bowdidge. Why don’t software developers use static analysis tools to find bugs? In 2013 35th International Conference on Software Engineering (ICSE), pages 672–681. IEEE, 2013

    2013

  6. [14]

    Was there a Hawthorne effect? American Journal of sociology, 98(3):451–468, 1992

    Stephen RG Jones. Was there a Hawthorne effect? American Journal of sociology, 98(3):451–468, 1992

    1992

  7. [15]

    I have no idea what I’m doing

    Katharina Krombholz, Wilfried Mayer, Martin Schmiedecker, and Edgar Weippl. “I have no idea what I’m doing” — on the usabil- ity of deploying HTTPS. In Proceedings of the 26th USENIX Security Symposium. USENIX, 2017

    2017

  8. [16]

    thinking-aloud

    Clayton Lewis. Using the “thinking-aloud” method in cognitive inter- face design. Technical Report RC 9265, IBM, 1982

    1982

  9. [17]

    The system usability scale: past, present, and future

    James R Lewis. The system usability scale: past, present, and future. International Journal of Human–Computer Interaction, 34(7):577–590, 2018

    2018

  10. [18]

    On the critical path to implant backdoors and the effectiveness of potential mitigation techniques: Early learnings from xz

    Mario Lins, René Mayrhofer, Michael Roland, Daniel Hofer, and Mar- tin Schwaighofer. On the critical path to implant backdoors and the effectiveness of potential mitigation techniques: Early learnings from xz. arXiv preprint arXiv:2404.08987, 2024

    2024 arXiv

  11. [19]

    A theory and tools for applying sandboxes effectively

    Michael Maass. A theory and tools for applying sandboxes effectively. page 166, 2016

    2016

  12. [20]

    A systematic analysis of the science of sandboxing

    Michael Maass, Adam Sales, Benjamin Chung, and Joshua Sunshine. A systematic analysis of the science of sandboxing. PeerJ Computer Science, 2:e43, 2016

    2016

  13. [21]

    seccomp - secure computing mode, 2024

    Linux man pages. seccomp - secure computing mode, 2024. Accessed: 2024-07-14

    2024

  14. [22]

    pledge - restrict system operations, 2024

    OpenBSD man pages. pledge - restrict system operations, 2024. Ac- cessed: 2024-07-14

    2024

  15. [23]

    Why you only need to test with 5 users, 2000

    Jakob Nielsen. Why you only need to test with 5 users, 2000

    2000

  16. [24]

    Usability smells: An analysis of developers’ struggle with crypto libraries

    Nikhil Patnaik, Joseph Hallett, and Awais Rashid. Usability smells: An analysis of developers’ struggle with crypto libraries. In Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019) , pages 245–257, 2019

    2019

  17. [25]

    If you build it, will they come? developer recruitment for security studies

    Nikhil Patnaik, Joseph Hallett, Mohammad Tahaei, and Awais Rashid. If you build it, will they come? developer recruitment for security studies. ROPES-ICSE 202, 2022

    2022

  18. [26]

    A usability evalua- tion of afl and libfuzzer with cs students

    Stephan Plöger, Mischa Meier, and Matthew Smith. A usability evalua- tion of afl and libfuzzer with cs students. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023

    2023

  19. [27]

    Measuring, assessing and improving software quality based on object- oriented design principles

    Reinhold Plösch, Johannes Bräuer, Christian Körner, and Matthias Saft. Measuring, assessing and improving software quality based on object- oriented design principles. Open Computer Science , 6(1):187–207, 2016

    2016

  20. [28]

    Brief note on the origins, evolution, and meaning of the qualitative research concept thick description

    Joseph G Ponterotto. Brief note on the origins, evolution, and meaning of the qualitative research concept thick description. The qualitative report, 11(3):538–549, 2006

    2006

  21. [29]

    Saltzer and M.D

    J.H. Saltzer and M.D. Schroeder. The protection of information in computer systems. Proceedings of the IEEE, 63(9):1278–1308, Sep 1975

    1975

  22. [30]

    Towards usable application-oriented access controls: qualitative results from a usability study of SELinux, AppArmor and FBAC-LSM

    Z Cliffe Schreuders, Tanya McGill, and Christian Payne. Towards usable application-oriented access controls: qualitative results from a usability study of SELinux, AppArmor and FBAC-LSM. International Journal of Information Security and Privacy (IJISP), 6(1):57–76, 2012

    2012

  23. [31]

    The state of the art of application restrictions and sandboxes: A survey of application-oriented access controls and their shortfalls

    Z Cliffe Schreuders, Tanya McGill, and Christian Payne. The state of the art of application restrictions and sandboxes: A survey of application-oriented access controls and their shortfalls. Computers & Security, 32:219–241, 2013

    2013

  24. [32]

    The methodology of participatory design

    Clay Spinuzzi. The methodology of participatory design. Technical communication, 52(2):163–174, 2005

    2005

  25. [33]

    Open coding

    Anselm L Strauss and Juliet Corbin. Open coding. Social research methods: A reader, pages 303–306, 2004

    2004

  26. [34]

    A usability evaluation of Let’s Encrypt and Certbot: Usable security done right

    Christian Tiefenau, Emanuel von Zezschwitz, Maximilian Häring, Katharina Krombholz, and Matthew Smith. A usability evaluation of Let’s Encrypt and Certbot: Usable security done right. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, 2019

    2019

  27. [35]

    Enable ftrace-based system call filtering, 2011

    Linus Torvalds. Enable ftrace-based system call filtering, 2011. Kernel mailing list

    2011

  28. [36]

    The use of triangulation in qualitative research

    Data Source Triangulation. The use of triangulation in qualitative research. In Oncology Nursing forum, volume 41, pages 545–7, 2014

    2014

  29. [37]

    Refining the test phase of usability evaluation: How many subjects is enough? Human factors, 34(4):457–468, 1992

    Robert A Virzi. Refining the test phase of usability evaluation: How many subjects is enough? Human factors, 34(4):457–468, 1992

    1992

  30. [38]

    Martucci, and Stefan Lindskog

    Artem V oronkov, Leonardo A. Martucci, and Stefan Lindskog. Mea- suring the usability of firewall rule sets. IEEE Access, 2020

    2020

  31. [39]

    Capsicum: practical capabilities for UNIX

    Robert N M Watson, Jonathan Anderson, Kris Kennaway, and Ben Laurie. Capsicum: practical capabilities for UNIX. page 17, 2010

    2010

  32. [40]

    /tmp/watchdog.log

    Ka-Ping Yee. Aligning security and usability.IEEE Security & Privacy, 2(5):48–55, 2004. A Watchdog script code Listing 4: Interview task code 1 #include <stdio.h> 2 #include <stdlib.h> 3 #include <unistd.h> 4 #include <signal.h> 5 #include <sys/types.h> 6 #include <sys/socket....

    2004

  33. [41]

    I will start the audio recording now

    “I will start the audio recording now”

  34. [42]

    Read the task scenario

  35. [43]

    • Every time participants modify the Seccomp sandbox (initialize the sandbox, change rules/examine arguments)

    Take note of and keep a timeline of: • Every time participants refer to the documentation. • Every time participants modify the Seccomp sandbox (initialize the sandbox, change rules/examine arguments). • Every time participants run the code and how they run it. B.2.1 Scenario ...

  36. [44]

    Why did you choose the allow/deny listing approach in your solution? What is your reasoning behind this decision?

    When the participants reach a solution they are happy with, or at the end of the task time ask about: • Decision on allow/deny list approach. “Why did you choose the allow/deny listing approach in your solution? What is your reasoning behind this decision?” • Decision on exami...

  37. [45]

    When seems to be stuck for a while

  38. [46]

    When the participant exhibits an interesting behaviour, ask about the reasoning behind this behaviour and try to note their train of thoughts

  39. [47]

    Can you tell me what are you looking for and why

    When the participant refers to a documentation, ask about what they are looking for exactly and why? “Can you tell me what are you looking for and why”? B.3 Filling in the SUS form ( ∼15 minutes)

  40. [48]

    I will send you the form link in the meeting chat

    “The next stage of the interview will be filling a form. I will send you the form link in the meeting chat. Please let me know if you can access the form. The form has 10 statements, you can rank each statement with a number from 0-5, 0 means that you strongly disagree, while ...

  41. [49]

    B.4 Design exercise ( ∼15 min)

    Let the participants describe their answers fully and ask follow up questions if any interesting point came up or if answers seemed vague. B.4 Design exercise ( ∼15 min)

  42. [50]

    The last stage of the interview is a design exercise. I will share a white board with you now. Can you check if you can draw on the white board and access it without any problems?

    “The last stage of the interview is a design exercise. I will share a white board with you now. Can you check if you can draw on the white board and access it without any problems?”

  43. [51]

    The sandbox should be able to restrict what your code can do

    “I am going to ask you to design a sandboxing model that can be used within a codebase (API), a sandboxing model that you would use frequently, the perfect sandbox. The sandbox should be able to restrict what your code can do. It is up to you how to design the sandbox, you can...

  44. [52]

    Can you tell me why you chose this model of sandbox- ing?

    After the participants reach an initial design, ask about why they chose this model? “Can you tell me why you chose this model of sandbox- ing?”

  45. [53]

    Can you expand more on . . . please?

    Ask participants clarifying questions if their design seems unclear. “Can you expand more on . . . please?” “What do you mean by. . . ? B.5 End of the interview • Thank the participant and start the process of issuing the voucher. • Make sure the audio recording is saved. • Ma...

  46. [54]

    Tell us more about your C programming experience, what kind of projects have you worked on before that required C programming?

  47. [55]

    Have you used Seccomp before? What kind of projects and why you used Seccomp there? This study aims to evaluate the usability of Seccomp and is part of a PhD project

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.