Teaching Shortest Path Algorithms With a Robot and Overlaid Projections

Jordan Aiko Deja; Klen \v{C}opi\v{c} Pucihar; Matja\v{z} Kljun; Pavel Jolakoski

REVIEW 3 major objections 7 minor 17 references

Teaching Shortest Path Algorithms With a Robot and Overlaid Projections

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A robot that physically walks through projected graphs was preferred by all ten observers in one study and by four of six active users in a pilot over screen-only animation, though usability ratings favored the screen.

desk verdict A likeable small educational-robotics paper with an internal contradiction: the Discussion says pilot participants preferred the screen, while the results section says 4/6 preferred the robot. read the letter →

arxiv 2411.15535 v1 pith:HSMMZUVZ submitted 2024-11-23 cs.RO cs.HC

Pavel Jolakoski , Jordan Aiko Deja , Klen Čopič Pucihar , Matjaž Kljun This is my paper · ORCID

classification cs.ROcs.HC

keywords GoPishortestpathalgorithmsteachinggraphsrobotsprojections

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a physical robot can make abstract graph algorithms—Dijkstra's, A*, and Bellman-Ford—tangible enough that learners prefer it over a plain screen animation. It presents Timmy, a GoPiGo robot that moves across a projected graph while a JavaScript application colors edges and shows pseudocode alongside distance and predecessor updates. Two small within-subject studies compared the robot-synced mode with an on-screen-only mode: in the observation-only study (n=10) all participants preferred the robot, and in the interactive pilot (n=6) four of six did, with everyone saying they would recommend both. Self-reported cognitive load stayed manageable, around 20 percent on the NASA-TLX workload scale, and both modes received above-average System Usability Scale scores. The paper's conclusion is cautious: robots are engaging and hold attention, but preference depends on how much handling the robot requires, and teaching effectiveness still needs larger studies.

What carries the argument

The central object is Timmy, a GoPiGo robot paired with a top-down projector and a JavaScript graph-drawing application. The projector displays the graph, pseudocode, and distance and predecessor updates on the surface around the robot, and in robot-synced mode Timmy's physical movements replace the green animated edge traversal of the screen-only mode. Synchronization runs through text files containing vertex coordinates and adjacency lists, a watcher script that stores the robot's position and orientation in JSON, and timing calibrated from measured movement and turn rates. Timmy executes four primitives—select a vertex, poke a vertex, traverse the shortest path, and celebrate—so a learner watches a physical agent act out the algorithm's decisions. This mechanism is what translates an abstract computation into visible, embodied motion, the feature the paper hypothesizes will improve engagement.

What would settle it

A controlled experiment that removes the extra handling—an autonomous robot with pacing matched to the screen and no manual placement—and then measures learning with pre/post tests would settle the claim: if the robot-synced mode no longer wins preference or shows no learning improvement over screen-only, the central claim that the physical robot itself aids teaching would be refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a proof of concept: a synchronized robot-plus-projection environment can demonstrate shortest path algorithms in a way that learners find engaging and mostly prefer to screen-only animation. Concretely, all ten passive observers chose the robot as their preferred condition and believed it was more effective for demonstrating the algorithms, while four of six interactive pilot participants preferred it. The paper also reports that preference shifts when participants have to position the robot themselves, that the screen condition received a higher mean usability score in the pilot, and that all pilot participants recommended using both modes and would welcome robots in class. The authors frame this as evidence that robots offer an engaging tool for teaching advanced algorithmic concepts, while cautioning that they measured preference, not learning outcomes.

Load-bearing premise

The comparisons assume that differences in preference come from the robot's physical visualization, even though the robot condition also required extra manual positioning, ran more slowly, and had connection problems—so the preference could stem from novelty or added effort rather than from the value of the physical movement itself.

Editorial extensions

If this is right

If the findings hold, robot-synced demonstrations can serve as an engaging supplement to screen animations, but not as an automatic replacement: the screen was rated more usable in the pilot, where users had to handle the robot.
The unanimous preference for the robot in the passive observation study points to demonstration-style teaching, with the instructor operating the robot, as the most promising near-term classroom use.
All pilot participants said they would recommend both modes and would like robots in class, implying that offering both interfaces can accommodate different learning styles and preferences.
The pilot's unanimous view that the robot would be better for teaching primary-school children marks a natural target audience, although the authors note that this does not establish suitability for advanced concepts.
Because both studies captured preferences rather than learning gains, the direct corollary is that engagement must still be shown to translate into understanding before classroom adoption is justified.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, if manual handling is what lowered the robot's preference in the pilot, an autonomous robot that positions itself should restore the near-unanimous preference seen in the passive study; this could be tested by comparing self-moving and manually placed conditions.
Beyond the paper, combining the robot's physical traversal with on-screen edge coloring would address both pilot complaints about the robot being slow and hard to track, since the screen would keep a persistent visual record while the robot adds embodiment.
Beyond the paper, the robot's slower pace, which one participant saw as a drawback, might become a teaching advantage if future research shows that slower, embodied demonstrations improve retention; a pre/post test of learning would settle that.
Beyond the paper, the split between liking the robot and finding it harder to handle suggests that engagement and usability are separable dimensions, so educational-robot evaluations should measure both rather than treat preference as a single score.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look if you work on educational robotics, but read the pilot sections twice: Section 4.2.2 reports that four of six pilot participants preferred the robot-synced mode, while Section 5 states that in the pilot study 'they preferred the screen-based learning condition.' Those cannot both be true as written. The abstract leans on the pilot for its 'engaging tool' claim, so this is a load-bearing inconsistency, not a typo. If the Discussion is right, then the preference data are misreported; if the results section is right, then the Discussion misstates the outcome. Either way, the paper needs a correction before its findings can be trusted.

What is actually new: the Timmy system itself — a GoPiGo robot synced with projected graph visualizations and a graph editor, supporting Dijkstra, A*, and Bellman-Ford. That integration is not in the cited prior GoPiGo work. The authors also report original preference data from two within-subject studies. To their credit, they are explicit that they measured preferences, not learning, and they openly list the robot's practical problems (battery, VNC, slow Pi). The initial study's unanimous preference for the robot is a clean, if small, descriptive result.

Soft spots beyond the contradiction: the pilot protocol says '6 participants' and then immediately says '(n = 10, aged 18 to 25)' — a copy-paste error that does not inspire confidence. There are no inferential statistics, so the 4/6 preference is just a count. The robot condition required manual handling (positioning the robot before each run) that the screen condition did not, and the authors acknowledge this as a possible cause of the lower SUS score. No code or data are provided, which is common for a workshop paper but still limits reproducibility.

The system description is clear and the authors seem genuinely engaged with the limitations. The contradiction, though, means the paper is not coherent on its own terms right now. If the authors fix the reporting discrepancy, clarify what was actually asked in the pilot, and add the per-question numbers, this becomes a useful data point for the HCI/educational-robotics community. It deserves a serious referee, but not a clean accept as is.

Referee Report

3 major / 7 minor

Summary. This paper presents Timmy, a GoPiGo3 robot augmented with a ceiling-mounted projector that renders a graph-drawing application around the robot, with the robot physically executing the steps of Dijkstra, A*, and Bellman-Ford shortest path algorithms in synchrony with the visualization. The system is evaluated through two within-subject preference studies: an initial observation-only study (n = 10, Dijkstra only) and an interactive pilot (n = 6, all three algorithms), comparing a robot-synced mode against an on-screen-only mode using NASA-TLX (initial study) and SUS (pilot) questionnaires plus preference questions. The paper reports that all 10 initial participants preferred the robot-synced mode and perceived it as better for learning, while in the pilot 4/6 preferred the robot-synced mode as the liked condition, all 6 chose it for teaching primary school children, and 4/6 found the screen easier to handle. The authors explicitly state that both studies measured preferences, not learning outcomes, and they conclude with calls for larger studies and methodological refinements, also listing technical limitations of the GoPiGo platform.

Significance. Taken at face value, the preference data provide modest self-report evidence that embodied, physically traversing algorithm demonstrations can be engaging and attention-maintaining for university students, and that users perceive such demonstrations as suitable for teaching younger children. The paper's concrete strengths are the replicable system description (grid-normalized movement calibration, file-based command synchronization between the JavaScript application and the robot, and the select/poke/traverse/celebrate action vocabulary), the transparent per-participant workload reporting in Figure 3, and the honest, explicit acknowledgement that no learning outcome was measured and that sample sizes are small. Because no derivation, fitting, or parameter estimation is involved, circularity is not at issue; the self-report data directly support the descriptive claims. The significance is limited by the inconsistent reporting of the pilot result (Section 4.2.2 versus Section 5), the confound of extra manual handling in the robot condition, and the absence of any inferential statistics.

major comments (3)

[§4.2.2 and §5] The pilot preference finding is stated inconsistently. Section 4.2.2 reports that 'Four out of six (66%) participants preferred the robot-synced over the on-screen-only condition,' while Section 5 states that 'in the pilot study, where participants had to build their own graphs and interact with the robot, they preferred the screen-based learning condition.' These two sentences cannot describe the same preference question. If the Section 5 sentence refers to a question about which condition is better for learning (an analogue of the initial study's Q2), then the paper has omitted the result most relevant to its teaching claim while featuring the engagement preference in the abstract; if it refers to the preference question reported in Section 4.2.2, then one of the statements is wrong. The Discussion's summary that participants' preference 'shifted based on the task' presupposes that this contradiction has been resolved. The authors must report the pilot questionnaire item by item, give the counts for each item (including any 'better for learning' question), and rewrite Section 5 so it is consistent with Section 4.2.2. Because the abstract's 'initial findings' rest in part on the 4/6 pilot majority, this inconsistency is load-bearing and must be fixed before the central claim can be evaluated.
[§4.1, §4.2.1, §2.2] The robot-versus-screen comparison is confounded in two ways that the preference conclusions do not control for. First, in the robot-synced condition of the pilot, participants had to manually position the robot before each algorithm run ('users were first given additional instructions on where to place the robot... This process was repeated each time they chose an algorithm'), whereas the on-screen-only condition required only clicking a button, as participant P_P01's remark 'Just clicking a button' makes explicit. The SUS means (screen 80.42 versus robot 72.92) and the Q4 handling result (4/6 screen) are plausibly driven by this asymmetry, so the 4/6 Q1 preference for the robot cannot be unambiguously attributed to the delivery medium as opposed to the added handling burden. Second, the two conditions are not informationally equivalent: Section 2.2 states that in robot-synced mode 'edge colouring is omitted,' so the green edge-exploration animations shown on screen are absent from the robot condition; the robot's 'poke' actions are not argued to convey the same algorithmic detail. The authors should equalize handling effort across conditions, measure handling burden separately from visualization modality, and discuss the omitted edge coloring when interpreting the preference results.
[Title, §6, Abstract] The paper's framing claims more than the measurements support. The title is 'Teaching Shortest Path Algorithms With a Robot and Overlaid Projections,' the conclusion states that 'results suggest that robots can maintain users' attention and show promise as educational tools,' and Section 5's summary about participants' preference shifting 'based on the task' invokes learning-related preference. Yet the abstract itself states that 'In both studies we investigated the preferences towards the system and not the teaching outcome,' and Section 5 concedes that 'future research should explore the effectiveness of using such teaching methods by testing participants' understanding of the content being taught.' No learning measure, knowledge test, or retention data appear anywhere in the manuscript. The authors should either add a learning-related outcome or systematically rephrase the title, abstract, and conclusion so that they claim only engagement and preference evidence; the initial study's all-10 'better for learning' response must be explicitly labeled as a subjective perception, not a measured learning gain.

minor comments (7)

[§4.1] The protocol sentence reads 'We recruited 6 participants, all university students (n = 10, aged 18 to 25)'; the parenthetical n = 10 contradicts the stated n = 6 and should be corrected.
[§4.1] The phrase 'with n = 3 participants per condition, and the conditions were counterbalanced' is misleading for a within-subject design; it should say that three participants received each condition order.
[§4.2.2] The preference-question numbering appears shifted relative to the questionnaire in Section 3.1: in the pilot, 'Q3' is the primary-school teaching question and 'Q4' is ease of handling, whereas in the initial study Q3 was ease of handling and Q4 was recommendation to colleagues; clarify the pilot questionnaire numbering explicitly.
[§4.2.2] The closing sentence 'user preference for educational methods depends significantly on user preference' is tautological; rephrase to say that preferences varied across participants and depended on individual priorities such as speed versus engagement.
[§4.2.2] No inferential statistics accompany any reported count or SUS mean; given n = 6, the '66%' framing is fragile, and reporting exact binomial confidence intervals or effect sizes would prevent over-reading; also, the NASA-TLX and SUS instruments are used without citations, which should be added.
[§4.1] All pilot participants experienced the three algorithms in a fixed order (Dijkstra, then A*, then Bellman-Ford), so order effects cannot be excluded and should be acknowledged.
[§1 and header] Several small textual errors should be fixed: 'educators educators' and 'One such options' in Section 1, the corrupted token '/envel⌢pe-⌢pen' in the author-address line, and the stray 'Both-' in the initial study's Q5 option list.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper reports empirical preference and usability measurements with no fitted parameters or self-referential derivation.

full rationale

This paper contains no derivation chain whose outputs are equivalent to its inputs. The central findings are self-reported preference counts (10/10 in the initial study, 4/6 in the pilot) and NASA-TLX and SUS scores, all measured directly from participants rather than computed from assumptions or fitted to the claims. The system description (Sections 2) is an engineering implementation, and the evaluation (Sections 3-4) compares conditions empirically. The Discussion's statement that pilot participants preferred the screen-based learning condition (Section 5) appears inconsistent with the 4/6 robot preference reported in Section 4.2.2, but this is a data-consistency or reporting issue, not circularity: it does not involve a prediction reducing to a fit, a definition in terms of the target, or a self-citation carrying the argument. The paper also explicitly disclaims measuring teaching outcomes and acknowledges small sample sizes, which are limitations rather than circular steps. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

This is an empirical HCI paper, not a derivation, so there are no fitted parameters or invented theoretical entities. The relevant assumptions are methodological: the validity of self-report instruments, the technical fidelity of the robot synchronization, and the representativeness of the convenience sample.

assumptions (3)

domain assumption NASA-TLX and SUS scores are valid, interpretable measures of cognitive load and usability in this context.
Used to draw conclusions about cognitive load and usability in Sections 3.2 and 4.2.1 without validating against learning outcomes.
domain assumption The GoPiGo robot's movement and the projection system can be synchronized accurately enough that the robot faithfully represents the algorithm steps.
System relies on file-based coordinate transfer, JSON position tracking, and timing based on measured movement rates (Section 2.3); if synchronization is inaccurate, participants' preferences may reflect errors rather than the demonstration.
domain assumption University students with prior knowledge of graph algorithms are an appropriate sample for assessing an educational tool intended for broader audiences, including children.
Recruitment described in Sections 3.1 and 4.1; authors acknowledge this may not represent the target audience in Section 5.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Teaching Shortest Path Algorithms With a Robot and Overlaid Projections." pith.science (2026). https://pith.science/paper/HSMMZUVZ

@misc{pith2026241115535,
  author       = {Pith},
  title        = {Pith review of: Teaching Shortest Path Algorithms With a Robot and Overlaid Projections},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSMMZUVZ}},
  note         = {Machine review of arXiv:2411.15535}
}

read the original abstract

Robots have the potential to enhance teaching of advanced computer science topics, making abstract concepts more tangible and interactive. In this paper, we present Timmy-a GoPiGo robot augmented with projections to demonstrate shortest path algorithms in an interactive learning environment. We integrated a JavaScript-based application that is projected around the robot, which allows users to construct graphs and visualise three different shortest path algorithms with colour-coded edges and vertices. Animated graph exploration and traversal are augmented by robot movements. To evaluate Timmy, we conducted two user studies. An initial study (n=10) to explore the feasibility of this type of teaching where participants were just observing both robot-synced and the on-screen-only visualisations. And a pilot study (n=6) where participants actively interacted with the system, constructed graphs and selected desired algorithms. In both studies we investigated the preferences towards the system and not the teaching outcome. Initial findings suggest that robots offer an engaging tool for teaching advanced algorithmic concepts, but highlight the need for further methodological refinements and larger-scale studies to fully evaluate their effectiveness.

Figures

Figures reproduced from arXiv: 2411.15535 by the authors.

**Figure 1.** (a) Full setup of the project. (b) The robot in the projected environment. (c) Participant observing the robot in the robot condition. (d) Participant interacting with the application [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** The view of the application interface. On the left panel is the pseudocode. In the middle is the main canvas with the graph showing the vertices, edges and shortest path. On the right panel we see the distance and predecessor updates. 2. Timmy and the Learning environment: Design and Features 2.1. Overview The learning environment consists of two primary components (i) the robot Timmy and the (ii) synchronised appli… view at source ↗

**Figure 3.** Cognitive loads of both conditions. The values are shown per participant and are not aggregated to show the varying levels across each dimension. the screen. Participants passively-observed the execution of the algorithm on the pre-drawn graph and were given time to review before continuing. After each condition, participants’ cognitive load was assessed using the NASA-TLX questionnaire. They were then given a simi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 15 canonical work pages

[1]

De Freitas, G

S. De Freitas, G. Rebolledo-Mendez, F. Liarokapis, G. Magoulas, A. Poulovassilis, Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world, British journal of educational technology 41 (2010) 69–85

work page 2010
[2]

Wagner, L

C. Wagner, L. Liu, Creating immersive learning experiences: A pedagogical design per- spective, in: Creative and collaborative learning through immersion: Interdisciplinary and international perspectives, Springer, 2021, pp. 71–87

work page 2021
[3]

del Campo, V

J. del Campo, V. Negro, M. Núñez, Traditional education vs modern education. What is the impact of teaching techniques’ evolution on students’ learning process?, in: INTED 2012 Proceedings, IATED, 2012, pp. 5762–5766

work page 2012
[4]

Reich-Stiebert, F

N. Reich-Stiebert, F. Eyssel, Robots in the classroom: What teachers think about teaching and learning with education robots, in: Social Robotics: 8th International Conference, ICSR 2016, Kansas City, MO, USA, November 1-3, 2016 Proceedings 8, Springer, 2016, pp. 671–680

work page 2016
[5]

Industries, Gopigo: An educational robot for learning programming and robotics, https://www.dexterindustries.com/gopigo, 2024

D. Industries, Gopigo: An educational robot for learning programming and robotics, https://www.dexterindustries.com/gopigo, 2024

work page 2024
[6]

Persson, H

M. Persson, H. Steenari, M. Thelin, F. Thune, The pedagogo. teaching children computer programming (2016)

work page 2016
[7]

B. I. Edwards, A. D. Cheok, Why not robot teachers: artificial intelligence for addressing teacher shortage, Applied Artificial Intelligence 32 (2018) 345–360

work page 2018
[8]

Y. Cai, D. Youngstrom, W. Zhang, Exploring approaches for teaching cybersecurity and ai for k-12, in: 2023 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, 2023, pp. 1559–1564

work page 2023

Show all 17 references

[9]

Gallo, S

G. Gallo, S. Pallottino, Shortest path algorithms, Annals of operations research 13 (1988) 1–79

1988
[10]

E. W. Dijkstra, A note on two problems in connexion with graphs, in: Edsger Wybe Dijkstra: his life, work, and legacy, 2022, pp. 287–290

2022
[11]

P. E. Hart, N. J. Nilsson, B. Raphael, A formal basis for the heuristic determination of minimum cost paths, IEEE transactions on Systems Science and Cybernetics 4 (1968) 100–107

1968
[12]

Bellman, On a routing problem, Quarterly of applied mathematics 16 (1958) 87–90

R. Bellman, On a routing problem, Quarterly of applied mathematics 16 (1958) 87–90

1958
[13]

Baxter, E

P. Baxter, E. Ashurst, R. Read, J. Kennedy, T. Belpaeme, Robot education peers in a situated primary school study: Personalisation promotes child learning, PloS one 12 (2017) e0178126

2017
[14]

Fagin, L

B. Fagin, L. Merkle, Measuring the effectiveness of robots in teaching computer science, Acm sigcse bulletin 35 (2003) 307–311

2003
[15]

J. S. Kay, J. G. Moss, Using robots to teach programming to k-12 teachers, in: 2012 Frontiers in Education Conference Proceedings, IEEE, 2012, pp. 1–6

2012
[16]

Burbaiṫe, R

R. Burbaiṫe, R. Damaševičius, V. Štuikys, Using robots as learning objects for teaching computer science, in: X world conference on computers in education, 2013, pp. 101–110

2013
[17]

R. D. Beer, H. J. Chiel, R. F. Drushel, Using autonomous robotics to teach science and engineering, Communications of the ACM 42 (1999) 85–92

1999

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

De Freitas, G

S. De Freitas, G. Rebolledo-Mendez, F. Liarokapis, G. Magoulas, A. Poulovassilis, Learning as immersive experiences: Using the four-dimensional framework for designing and evaluating immersive learning experiences in a virtual world, British journal of educational technology 41 (2010) 69–85

work page 2010

[2] [2]

Wagner, L

C. Wagner, L. Liu, Creating immersive learning experiences: A pedagogical design per- spective, in: Creative and collaborative learning through immersion: Interdisciplinary and international perspectives, Springer, 2021, pp. 71–87

work page 2021

[3] [3]

del Campo, V

J. del Campo, V. Negro, M. Núñez, Traditional education vs modern education. What is the impact of teaching techniques’ evolution on students’ learning process?, in: INTED 2012 Proceedings, IATED, 2012, pp. 5762–5766

work page 2012

[4] [4]

Reich-Stiebert, F

N. Reich-Stiebert, F. Eyssel, Robots in the classroom: What teachers think about teaching and learning with education robots, in: Social Robotics: 8th International Conference, ICSR 2016, Kansas City, MO, USA, November 1-3, 2016 Proceedings 8, Springer, 2016, pp. 671–680

work page 2016

[5] [5]

Industries, Gopigo: An educational robot for learning programming and robotics, https://www.dexterindustries.com/gopigo, 2024

D. Industries, Gopigo: An educational robot for learning programming and robotics, https://www.dexterindustries.com/gopigo, 2024

work page 2024

[6] [6]

Persson, H

M. Persson, H. Steenari, M. Thelin, F. Thune, The pedagogo. teaching children computer programming (2016)

work page 2016

[7] [7]

B. I. Edwards, A. D. Cheok, Why not robot teachers: artificial intelligence for addressing teacher shortage, Applied Artificial Intelligence 32 (2018) 345–360

work page 2018

[8] [8]

Y. Cai, D. Youngstrom, W. Zhang, Exploring approaches for teaching cybersecurity and ai for k-12, in: 2023 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, 2023, pp. 1559–1564

work page 2023