Pith. sign in

REVIEW 3 major objections 5 minor 59 references

Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that pairing conversational natural-language instructions with GUI demonstrations enables non-programmers to author conditional task automations, because ambiguous concepts can be recursively grounded in values visible in…

desk verdict Solid HCI systems paper with a real open-source implementation and honest usability data, but the arXiv version is a recap of the UIST 2019 paper and the domain-independence claim outruns what the evaluation supports. read the letter →

arxiv 1909.00031 v2 pith:LTJ2WYG2 submitted 2019-08-30 cs.HC cs.AI

Toby Jia-Jun Li , Marissa Radensky , Justin Jia , Kirielle Singarajah , Tom M. Mitchell , Brad A. Myers This is my paper · ORCID
classification cs.HCcs.AI
keywords end-userdevelopmentnaturallanguageprogrammingbydemonstrationtaskautomationconceptlearningconditionalstatementsconversationalagentsmobileappGUIs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that non-programmers can author conditional task automations by speaking naturally and demonstrating inside existing mobile apps. Its key claim is that instead of forcing users to define everything up front, the agent can parse a high-level instruction, spot unknown concepts, and recursively ask for explanations or demonstrations until every condition is grounded in concrete GUI content. A formative study found that natural instructions often use vague concepts like "hot" or "heavy traffic," and that showing app screens reduced such vagueness; the paper argues the agent's design addresses this. The paper reports a lab study with 10 users, all of whom completed four automation tasks, as evidence the approach is usable by people with little or no programming experience.

What carries the argument

The central mechanism is the parser's typed resolve() function, a placeholder inserted into the semantic parse wherever an unknown or unclear concept appears; at runtime the agent executes these placeholders depth-first, asking the user to explain or demonstrate each one, and replaces them with definitions. The system also maintains a UI snapshot graph of GUI objects visible in third-party apps, so a user can ground a value concept by pointing at a screen element, and a persistent knowledge graph of procedures and concepts for reuse.

What would settle it

Give the system a task whose condition depends on information not displayed in any app, such as "if I am feeling tired" or "if the house is empty"; the recursive clarification has no GUI content to point to, so it cannot produce a grounded Boolean concept.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a workflow: a user says something like "if it's hot, order iced coffee," the agent parses the conditional structure, flags "hot" and "order iced coffee" as unresolved typed components, and then asks the user how to determine "hot" and how to perform the order. The user can answer with more language or with a demonstration inside an app; each new unknown recursively triggers another question until everything bottoms out in values visible on app screens. The paper further claims that the resulting knowledge is stored in a persistent graph that supports generalization and reuse across contexts, and it reports that in a lab study all 10 participants completed all 4 automation tasks, with subjective ratings around 6 on the 7-point scales for ease of use and naturalness.

Load-bearing premise

The approach assumes that every concept a user needs can be grounded in information visible on the screen of an existing third-party mobile app, and that the user is familiar enough with that app to demonstrate where to find it.

Editorial extensions

If this is right

  • Non-programmers can author task automations with conditionals without writing code, by explaining vague conditions in terms of concrete values and demonstrating where those values appear.
  • A concept taught once, such as "hot," can be reused in a new task domain with the same comparison but a different threshold or a different data source.
  • The agent's habit of asking about the "else" case addresses the frequent omission of alternative branches in natural instructions.
  • Because all grounding is done through existing third-party app interfaces, the approach works across task domains as long as a relevant app exists.
  • Learned concepts and procedures are stored persistently, so later instructions can refer to them and trigger the agent to ask whether the old definition still applies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit is that the same recursive grounding loop could work on desktop or web interfaces whenever GUI content can be programmatically read, so the approach is not inherently limited to mobile apps.
  • The persistent knowledge graph points toward cross-user sharing of taught concepts, but the paper's own limitation section notes that personal preferences and generalizable knowledge are not yet separated, so sharing would require a privacy boundary first.
  • A testable extension would be to support logical combinations like "hot and raining" or arithmetic like "at least $10 more expensive," which the paper lists as future work; usability with these richer expressions is unmeasured.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents PUMICE, an end-user programmable agent that combines natural language instructions with GUI demonstrations to let users specify conditional automation rules and teach new concepts on Android smartphones. The authors first report a formative Mechanical Turk study (n=58) that motivates the design, then describe the PUMICE architecture, including a SEMPRE-based semantic parser with typed resolve() functions, a SUGILITE-based demonstration recorder, and a persistent knowledge graph for learned procedures and concepts. The evaluation is a lab usability study with 10 participants who completed four author-designed tasks; all participants completed all tasks, and post-survey Likert ratings were positive. The paper is an empirical systems paper with no formal or mathematical derivation. The main claims are that the multi-modal, top-down recursive clarification approach is usable by non-programmers and that the approach is domain-independent by virtue of grounding concepts in third-party app GUI contents.

Significance. If the result holds, the contribution is valuable: the paper provides concrete evidence that end users without programming expertise can define abstract conditions such as 'hot' or 'heavy traffic' through a mix of conversational clarification and in-app demonstration, and the open-sourced implementation supports replication and extension. The formative study is a useful design contribution, and the candor of the Limitations section is a strength. The significance is moderated by three factors: the usability claim rests on a small, no-control study; the correctness of the produced scripts is not reported despite being checked; and the 'domain-independent' claim is narrower than stated, as the authors themselves acknowledge in the Limitations and Future Work section. The paper is best read as a feasibility demonstration of a specific interaction design rather than a comparative validation of the approach against alternatives.

major comments (3)
  1. [User Study (Tasks and Results)] The central usability claim, stated in the Abstract as 'A lab study with 10 users showed its usability,' is supported only by a 10-participant study with no control condition, in which the tasks were designed by the authors to exercise PUMICE's features and participants received a 5-minute tutorial with an example that pre-taught the target workflow (e.g., 'late' defined as 'current time is after 8pm'). Because there is no comparison against a baseline such as programming by demonstration alone, natural language alone, or an existing tool, the data cannot distinguish the contribution of the multi-modal conversational approach from the effect of the tutorial, the pre-selected tasks, or the pre-stored locations and concepts. I recommend either adding a comparison condition, or explicitly re-scoping the claim from 'usability' to 'feasibility' with the current evidence.
  2. [User Study (Results)] The paper states that the experimenter 'checked the scripts at the end of each study session to evaluate their correctness' (User Study, Procedure), but no correctness results are reported anywhere in the Results section. The statement that 'all participants were able to complete all 4 tasks' indicates self-reported or observed completion, not that the generated scripts correctly executed the intended automation. Since the authors also note that participants encountered speech recognition errors and parsing errors and used the undo feature to recover, completion alone does not establish that the final programs were correct. Please report the number of scripts that passed the correctness check and the number that required corrections, so that the reader can assess whether the interaction design actually led to correct end-user programs.
  3. [PUMICE and Limitations and Future Work] The claim that PUMICE is 'domain-independent' is load-bearing for the paper's contribution statement, but the paper's own Limitations section states that the system has 'no semantic understanding of information involved in tasks,' cannot handle implicit parameters, relations between concepts, references, synonyms, antonyms, or implicit conjunctions/disjunctions, and that the DSL only supports single basic Boolean operations. In addition, all concepts must be grounded in information visible in third-party app GUIs, which the stress-test note correctly identifies as a boundary of the approach. As written, the contribution is more accurately described as a domain-independent interaction mechanism (top-down conversational decomposition plus GUI demonstration) over app-specific grounding, rather than domain independence of the learned representations or of the parser. Please either soften the 'domain-independent' claim throughout the Abstract and contributions, or provide evidence that the mechanism transfers across unseen task domains without per-domain engineering beyond the four study tasks.
minor comments (5)
  1. [Formative Study] The comparison between 33% and 9% is reported as statistically significant with p < 0.05, but the test statistic, test type, and effect size are not given; please report these details for reproducibility.
  2. [System Implementation] The semantic parser is described as trained on n=905 examples, but no parser accuracy, per-utterance success rate, or analysis of failure modes is reported; a small evaluation table would strengthen the reproducibility of the system claims.
  3. [Figure 2] The caption says the red overlays 'highlight all durations' that PUMICE identified on the Google Maps GUI; the accompanying text in the PUMICE section describes highlighting 'possible items' and 'all values visible on the screen.' Please make the figure caption and body text consistent.
  4. [Abstract and Conclusion] The phrases 'showed its usability' and 'supported by our summative lab usability study' overstate the evidence given the small no-control design; consider replacing them with 'showed feasibility' or adding a sentence that acknowledges the lack of a control condition.
  5. [References] The paper cites [Li et al. 2019] as the earlier UIST version of PUMICE; the present version appears to be the workshop version of the same work. Please clarify the delta between this submission and the UIST paper to help readers understand the incremental contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical systems paper; reuse of prior frameworks is component reuse, not a derivation.

full rationale

This paper is an HCI systems paper rather than a formal derivation, so the standard circularity failure modes do not apply. The central claim is that PUMICE implements a multi-modal, domain-independent approach for task and concept learning, supported by a 10-participant lab usability study. The formative study motivates design choices such as GUI grounding and proactive else-condition prompting, but the subsequent user study independently tests whether participants can complete four concrete automation tasks; the usability result is not obtained by construction from the design goals. The semantic parser, typed resolve() functions, persistent knowledge graph, and UI snapshot graph are described as implementation mechanisms, not as results derived from fitted parameters. The paper's self-citations are normal component reuse: it states that PUMICE extends the authors' SUGILITE system and uses SUGILITE for demonstration recording and replaying, and APPINITE for UI snapshot graphs. These citations provide reusable infrastructure; the paper does not rely on them to prove the novelty of concept learning or conditionals, and the paper acknowledges limitations explicitly, including lack of semantic understanding of implicit parameters, support for only basic Boolean operations, foreground-only execution, and the lab-only evaluation. No equation is defined in terms of its target, no fitted input is renamed as a prediction, and no uniqueness theorem is imported to force the design. Therefore no circular step can be quoted, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or formal entities, no free numeric parameters, and no fitted constants. The only 'fitted' component is the semantic parser, trained on 905 utterances, but its weights are not reported and are not load-bearing for the qualitative usability claim. The listed axioms are the domain assumptions that the central approach depends on.

assumptions (3)
  • domain assumption The Android Accessibility API provides reliable access to GUI objects and actions across third-party native apps.
    PUMICE depends on the underlying SUGILITE framework for recording and replaying demonstrations through the Android Accessibility API. The paper notes in System Implementation that it 'operates well on most native Android apps, but may have problems working with web apps and apps with special graphic engines.'
  • domain assumption Users are sufficiently familiar with third-party apps to know how to demonstrate concepts and procedures.
    The entire demonstration-based interaction assumes users can navigate apps to find values and perform steps. The User Study procedure explicitly made sure participants understood the apps, stating that the design assumption is that users are familiar with the apps.
  • domain assumption Vague concepts can be grounded in visible app GUI content.
    The approach represents declarative concepts only through references to GUI contents. If the needed information is not present on an app screen, the recursive resolution process cannot succeed. This is acknowledged in the limitations: the system has no semantic understanding of information not visible in the GUI.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations." pith.science (2026). https://pith.science/paper/LTJ2WYG2

@misc{pith2026190900031,
  author       = {Pith},
  title        = {Pith review of: Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LTJ2WYG2}},
  note         = {Machine review of arXiv:1909.00031}
}
read the original abstract

Natural language programming is a promising approach to enable end users to instruct new tasks for intelligent agents. However, our formative study found that end users would often use unclear, ambiguous or vague concepts when naturally instructing tasks in natural language, especially when specifying conditionals. Existing systems have limited support for letting the user teach agents new concepts or explaining unclear concepts. In this paper, we describe a new multi-modal domain-independent approach that combines natural language programming and programming-by-demonstration to allow users to first naturally describe tasks and associated conditions at a high level, and then collaborate with the agent to recursively resolve any ambiguities or vagueness through conversations and demonstrations. Users can also define new procedures and concepts by demonstrating and referring to contents within GUIs of existing mobile apps. We demonstrate this approach in PUMICE, an end-user programmable agent that implements this approach. A lab study with 10 users showed its usability.

Figures

Figures reproduced from arXiv: 1909.00031 by the authors.

Figure 1
Figure 1. Example structure of how PUMICE learns the concepts and procedures in the command “If it’s hot, order a cup of Iced Cappuccino.” The numbers indicate the order of utterances. The screenshot on the right shows the conversational interface of PUMICE. In this interactive parsing process, the agent learns how to query the current temperature, how to order any kind of drink from Starbucks, and the generalized concept of … view at source ↗
Figure 2
Figure 2. The user teaches the value concept “commute [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. An example showing how PUMICE parses the user’s demonstrated action and verbal reference to an app’s GUI content into a SET VALUE statement with a query over the UI snapshot graph when resolving a new value concept “current temperature” for demonstration, and a persistent knowledge base for stor￾ing learned procedures and concepts. The purpose of the UI snapshot graph is to support under￾standing the user’s referenc… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The graphical prompt used for Task 1 – A possible [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The average task completion times for each task. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 56 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    Allen, J.; Chambers, N.; Ferguson, G.; Galescu, L.; Jung, H.; Swift, M.; and Taysom, W. 2007. PLOW : A Collaborative Task Learning Agent . In Proceedings of the 22Nd National Conference on Artificial Intelligence - Volume 2 , AAAI '07, 1514--1519. Vancouver, British Columbia, Canada: AAAI Press

    work page 2007

  3. [3]

    D.; Chernova, S.; Veloso, M.; and Browning, B

    Argall, B. D.; Chernova, S.; Veloso, M.; and Browning, B. 2009. A Survey of Robot Learning from Demonstration . Robot. Auton. Syst. 57(5):469--483

    work page 2009

  4. [4]

    Azaria, A.; Krishnamurthy, J.; and Mitchell, T. M. 2016. Instructable Intelligent Personal Agent . In Proc. The 30th AAAI Conference on Artificial Intelligence ( AAAI ) , volume 4

    work page 2016

  5. [5]

    W., and Biermann, A

    Ballard, B. W., and Biermann, A. W. 1979. Programming in Natural Language `` NLC '' As a Prototype . In Proceedings of the 1979 Annual Conference , ACM '79, 228--237. New York, NY, USA: ACM

    work page 1979

  6. [6]

    Berant, J.; Chou, A.; Frostig, R.; and Liang, P. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing , 1533--1544

    work page 2013

  7. [7]

    Biermann, A. W. 1983. Natural Language Programming . In Biermann, A. W., and Guiho, G., eds., Computer Program Synthesis Methodologies , NATO Advanced Study Institutes Series , 335--368. Springer Netherlands

    work page 1983

  8. [8]

    Bollacker, K.; Evans, C.; Paritosh, P.; Sturge, T.; and Taylor, J. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data , 1247--1250. ACM

    work page 2008

Show all 59 references
  1. [9]

    E.; Mueller, M.; and Bodik, R

    Chasins, S. E.; Mueller, M.; and Bodik, R. 2018. Rousillon: Scraping distributed hierarchical web data. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology , UIST '18, 963--975. New York, NY, USA: ACM

    2018

  2. [10]

    Chen, Y.; Martins, R.; and Feng, Y. 2019. Maximal multi-layer specification synthesis. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 602--612. ACM

    2019

  3. [11]

    Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20(1):37--46

    1960

  4. [12]

    Cypher, A., and Halbert, D. C. 1993. Watch what I do: programming by demonstration . MIT press

    1993

  5. [13]

    Fast, E.; Chen, B.; Mendelsohn, J.; Bassen, J.; and Bernstein, M. S. 2018. Iris: A Conversational Agent for Complex Tasks . In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems , CHI '18, 473:1--473:12. New York, NY, USA: ACM

    2018

  6. [14]

    Grabler, F.; Agrawala, M.; Li, W.; Dontcheva, M.; and Igarashi, T. 2009. Generating Photo Manipulation Tutorials by Demonstration . In ACM SIGGRAPH 2009 Papers , SIGGRAPH '09, 66:1--66:9. New York, NY, USA: ACM

    2009

  7. [15]

    Green, T. R. G., and Petre, M. 1996. Usability Analysis of Visual Programming Environments : A ' Cognitive Dimensions ' Framework . Journal of Visual Languages & Computing 7(2):131--174

    1996

  8. [16]

    P.; Cole, P.; Morgan, J.; et al

    Grice, H. P.; Cole, P.; Morgan, J.; et al. 1975. Logic and conversation. 1975 41--58

    1975

  9. [17]

    Hartmann, B.; Wu, L.; Collins, K.; and Klemmer, S. R. 2007. Programming by a sample: Rapidly creating web applications with d.mix. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology , UIST '07, 241--250. New York, NY, USA: ACM

    2007

  10. [18]

    Intharah, T.; Turmukhambetov, D.; and Brostow, G. J. 2019. Hilc: Domain-independent pbd system via computer vision and follow-up questions. ACM Trans. Interact. Intell. Syst. 9(2-3):16:1--16:27

    2019

  11. [19]

    J.; Wong, Y

    Kate, R. J.; Wong, Y. W.; and Mooney, R. J. 2005. Learning to Transform Natural to Formal Languages . In Proceedings of the 20th National Conference on Artificial Intelligence - Volume 3 , AAAI '05, 1062--1068. Pittsburgh, Pennsylvania: AAAI Press

    2005

  12. [20]

    E.; Gluck, K.; Anderson, J.; Forbus, K

    Laird, J. E.; Gluck, K.; Anderson, J.; Forbus, K. D.; Jenkins, O. C.; Lebiere, C.; Salvucci, D.; Scheutz, M.; Thomaz, A.; Trafton, G.; et al. 2017. Interactive task learning. IEEE Intelligent Systems 32(4):6--21

    2017

  13. [21]

    A.; Domingos, P.; and Weld, D

    Lau, T.; Wolfman, S. A.; Domingos, P.; and Weld, D. S. 2001. Your wish is my command. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. chapter Learning Repetitive Text-editing Procedures with SMARTedit, 209--226

    2001

  14. [22]

    M.; Matthews, T.; and Lau, T

    Leshed, G.; Haber, E. M.; Matthews, T.; and Lau, T. 2008. CoScripter : Automating & Sharing How -to Knowledge in the Enterprise . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , CHI '08, 1719--1728. New York, NY, USA: ACM

    2008

  15. [23]

    J.-J., and Riva, O

    Li, T. J.-J., and Riva, O. 2018. KITE : Building conversational bots from mobile apps. In Proceedings of the 16th ACM International Conference on Mobile Systems , Applications , and Services ( MobiSys 2018) . ACM

    2018

  16. [24]

    J.-J.; Azaria, A.; and Myers, B

    Li, T. J.-J.; Azaria, A.; and Myers, B. A. 2017. SUGILITE : Creating Multimodal Smartphone Automation by Demonstration . In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems , CHI '17, 6038--6049. New York, NY, USA: ACM

    2017

  17. [25]

    J.-J.; Li, Y.; Chen, F.; and Myers, B

    Li, T. J.-J.; Li, Y.; Chen, F.; and Myers, B. A. 2017. Programming IoT Devices by Demonstration Using Mobile Apps . In Barbosa, S.; Markopoulos, P.; Paterno, F.; Stumpf, S.; and Valtolina, S., eds., End- User Development , 3--17. Cham: Springer International Publishing

    2017

  18. [26]

    J.-J.; Labutov, I.; Li, X

    Li, T. J.-J.; Labutov, I.; Li, X. N.; Zhang, X.; Shi, W.; Mitchell, T. M.; and Myers, B. A. 2018a. APPINITE : A Multi - Modal Interface for Specifying Data Descriptions in Programming by Demonstration Using Verbal Instructions . In Proceedings of the 2018 IEEE Symposium on Vis...

    2018

  19. [27]

    J.-J.; Labutov, I.; Myers, B

    Li, T. J.-J.; Labutov, I.; Myers, B. A.; Azaria, A.; Rudnicky, A. I.; and Mitchell, T. M. 2018b. Teaching Agents When They Fail : End User Development in Goal -oriented Conversational Agents . In Studies in Conversational UX Design . Springer

  20. [28]

    J.-J.; Radensky, M.; Jia, J.; Singarajah, K.; Mitchell, T

    Li, T. J.-J.; Radensky, M.; Jia, J.; Singarajah, K.; Mitchell, T. M.; and Myers, B. A. 2019. PUMICE : A Multi - Modal Agent that Learns Concepts and Conditionals from Natural Language and Demonstrations . In Proceedings of the 32nd Annual ACM Symposium on User Interface Softwa...

    2019

  21. [29]

    Lieberman, H., and Liu, H. 2006. Feasibility studies for programming in natural language. In End User Development . Springer. 459--473

    2006

  22. [30]

    Lieberman, H., and Maulsby, D. 1996. Instructible agents: Software that just keeps getting better. IBM Systems Journal 35(3.4):539--556

    1996

  23. [31]

    Lieberman, H.; Liu, H.; Singh, P.; and Barry, B. 2004. Beating Common Sense into Interactive Applications . AI Magazine 25(4):63--63

    2004

  24. [32]

    Lieberman, H. 2001. Your wish is my command: Programming by example . Morgan Kaufmann

    2001

  25. [33]

    Lin, J.; Wong, J.; Nichols, J.; Cypher, A.; and Lau, T. A. 2009. End-user programming of mashups with vegemite. In Proceedings of the 14th International Conference on Intelligent User Interfaces , IUI '09, 97--106. New York, NY, USA: ACM

    2009

  26. [34]

    Liu, H., and Singh, P. 2004. Conceptnet --- a practical commonsense reasoning tool-kit. BT Technology Journal 22(4):211--226

    2004

  27. [35]

    J.; Harpstead, E.; Marinier III, R

    MacLellan, C. J.; Harpstead, E.; Marinier III, R. P.; and Koedinger, K. R. 2018. A Framework for Natural Cognitive System Training Interactions . Advances in Cognitive Systems

    2018

  28. [36]

    Maes, P. 1994. Agents That Reduce Work and Information Overload . Commun. ACM 37(7):30--40

    1994

  29. [37]

    D.; and Hudson, S

    Mankoff, J.; Abowd, G. D.; and Hudson, S. E. 2000. Oops: a toolkit supporting mediation techniques for resolving ambiguity in recognition-based interfaces. Computers & Graphics 24(6):819--834

    2000

  30. [38]

    Bethard, S.; and McClosky, D

    Manning, C.; Surdeanu, M.; Bauer, J.; Finkel, J.; J. Bethard, S.; and McClosky, D. 2014. The Stanford CoreNLP Natural Language Processing Toolkit . In Proceedings of 52Nd Annual Meeting of the Association for Computational Linguistics : System Demonstrations

    2014

  31. [39]

    Mihalcea, R.; Liu, H.; and Lieberman, H. 2006. NLP ( Natural Language Processing ) for NLP ( Natural Language Programming ). In Gelbukh, A., ed., Computational Linguistics and Intelligent Text Processing , Lecture Notes in Computer Science , 319--330. Springer Berlin Heidelberg

    2006

  32. [40]

    Mitchell, T.; Cohen, W.; Hruschka, E.; Talukdar, P.; Yang, B.; Betteridge, J.; Carlson, A.; Dalvi, B.; Gardner, M.; Kisiel, B.; et al. 2018. Never-ending learning. Communications of the ACM 61(5):103--115

    2018

  33. [41]

    A.; Ko, A

    Myers, B. A.; Ko, A. J.; LaToza, T. D.; and Yoon, Y. 2016. Programmers Are Users Too : Human - Centered Methods for Improving Programming Tools . Computer 49(7):44--52

    2016

  34. [42]

    A.; Ko, A

    Myers, B. A.; Ko, A. J.; Scaffidi, C.; Oney, S.; Yoon, Y.; Chang, K.; Kery, M. B.; and Li, T. J.-J. 2017. Making End User Development More Natural . In New Perspectives in End - User Development . Springer, Cham. 1--22

    2017

  35. [43]

    A.; Pane, J

    Myers, B. A.; Pane, J. F.; and Ko, A. J. 2004. Natural Programming Languages and Environments . Commun. ACM 47(9):47--52

    2004

  36. [44]

    Oviatt, S. 1999a. Mutual disambiguation of recognition errors in a multimodel architecture. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems , 576--583. ACM

  37. [45]

    Oviatt, S. 1999b. Ten Myths of Multimodal Interaction . Commun. ACM 42(11):74--81

  38. [46]

    F.; Myers, B

    Pane, J. F.; Myers, B. A.; and others . 2001. Studying the language and structure in non-programmers' solutions to programming problems. International Journal of Human-Computer Studies 54(2):237--264

    2001

  39. [47]

    Pasupat, P., and Liang, P. 2015. Compositional Semantic Parsing on Semi - Structured Tables . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing . arXiv: 1508.00305

    2015 arXiv

  40. [48]

    Paterno, F., and Wulf, V. 2017. New Perspectives in End-User Development . Springer, 1st edition

    2017

  41. [49]

    Price, D.; Rilofff, E.; Zachary, J.; and Harvey, B. 2000. NaturalJava : A Natural Language Interface for Programming in Java . In Proceedings of the 5th International Conference on Intelligent User Interfaces , IUI '00, 207--211. New York, NY, USA: ACM

    2000

  42. [50]

    J.-J.; and Myers, B

    Radensky, M.; Li, T. J.-J.; and Myers, B. A. 2018. How end users express conditionals in programming by demonstration for mobile apps. In 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) , 311--312. IEEE

    2018

  43. [51]

    J.-J.; Team, W.; and Hecht, B

    Sen, S.; Li, T. J.-J.; Team, W.; and Hecht, B. 2014. Wikibrain: democratizing computation on wikipedia. In Proceedings of The International Symposium on Open Collaboration , 27. ACM

    2014

  44. [52]

    Srivastava, S.; Labutov, I.; and Mitchell, T. 2017. Joint concept learning and semantic parsing from natural language explanations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , 1527--1536

    2017

  45. [53]

    Strauss, A., and Corbin, J. M. 1990. Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications, Inc

    1990

  46. [54]

    Vadas, D., and Curran, J. R. 2005. Programming with unrestricted natural language. In Proceedings of the Australasian Language Technology Workshop 2005 , 191--199

    2005

  47. [55]

    Wing, J. M. 2006. Computational Thinking . Commun. ACM 49(3):33--35

    2006

  48. [56]

    Yeh, T.; Chang, T.-H.; and Miller, R. C. 2009. Sikuli: Using GUI Screenshots for Search and Automation . In Proceedings of the 22Nd Annual ACM Symposium on User Interface Software and Technology , UIST '09, 183--192. New York, NY, USA: ACM

    2009

  49. [57]

    Yin, P., and Neubig, G. 2017. A syntactic neural model for general-purpose code generation. CoRR abs/1704.01696

    2017 arXiv

  50. [58]

    Yin , P.; Deng , B.; Chen , E.; Vasilescu , B.; and Neubig , G. 2018. Learning to mine aligned code and natural language pairs from stack overflow. In 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR) , 476--486

    2018

  51. [59]

    S.; Caspi, A.; Fogarty, J.; and Wobbrock, J

    Zhang, X.; Ross, A. S.; Caspi, A.; Fogarty, J.; and Wobbrock, J. O. 2017. Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility . In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems , CHI '17, 6024--6037. New Yor...

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.