{"id":"f4c20432-230f-4ad4-a3e9-49c6abd4fecd","arxiv_id":"2412.12383","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A 17-response 2021 follow-up survey shows Australian astronomers still rarely use advanced displays for research, and the authors argue VR/MR headsets could now replace most of those displays.","lead":"A 17-response survey of Australian astronomers found that nearly all use ordinary laptop and monitor screens, and almost none use advanced displays such as VR headsets for research. The authors argue VR/MR headsets have matured enough to replace most specialized astronomy display facilities, but the tiny response rate makes the population-wide claims fragile.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The inferential step from 17 self-selected respondents to ~750 ASA members assumes non-response is ignorable; the paper's own bias admission contradicts this, so the abstract's 'we infer' claims and the CIs in Table 14 lack support.","rationale":"I agree with the Reader's weakest_assumption. The concern is not small sample size per se, but the causal link between self-selection and outcomes. The paper's confidence intervals treat the sample as though it were randomly drawn from a finite population, while the recruitment mechanism makes that assumption untenable. The AIDA 2005 survey is not a control, because it suffers from the same opt-in issue. The VR/MR replacement argument is separate and largely qualitative, supported by existing software and published applications; I do not attack that part. The paper's limitation paragraphs are honest and should be credited, but the abstract's inference claims are stronger than the design supports. The Reader's CONDITIONAL verdict already captures this tension, so I recommend no change. If the authors revised the abstract to describe the 17 respondents' experiences without population inference, or added the sensitivity analysis, the paper would be acceptable as a scoping study.","tokens_in":26570,"tokens_out":4726,"duration_ms":45850,"concrete_test":"Perform a non-ignorable missing-data sensitivity analysis on Table 14. For each headline question, recompute the ASA population proportion under a plausible missing-not-at-random scenario: assume nonrespondents are only half as likely as respondents to have seen/used the advanced display in question. For Q5 VR/MR HMD (13/17 seen), take p_nonrespondent = 0.5 * (13/17) = 0.382; then the ASA proportion becomes (13 + 733 * 0.382) / 750 = 0.391, versus the reported sample proportion 0.76. Compare this with the reported CI (59-93%): the adjusted estimate falls below the lower bound, so the interval is not robust to the bias the authors themselves identify. A definitive version would also compute the extreme bounds (all nonrespondents at 0 and all at 1 for each outcome) and state whether the abstract's qualitative labels survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 14 and Eq. A.3 present 90% population-proportion confidence intervals for the ~750 ASA members based on Nsample = 17. This inferential step is valid only if the respondents are a probability sample, or if non-response is independent of the outcomes of interest. The actual mechanism is two email-list advertisements and voluntary opt-in during a pandemic (Section 3.2), and the authors themselves concede: 'the most likely participation bias is an underlying interest in the use of advanced image displays' (Section 4.1). If interest in advanced displays drives participation, the outcome variable is correlated with the missingness indicator; the data are missing-not-at-random. The finite population correction in Eq. A.2 and the normal-approximation CI in Eq. A.3 adjust for sampling without replacement from a known frame; they do not adjust for selection bias. The direction of the bias is known for awareness and usage estimates: it inflates them. Quantitatively, Q5 shows 13/17 respondents had seen a VR/MR HMD, yielding CI 59-93%. If the 733 nonrespondents mostly had not seen one, the population proportion would be far lower: 13/750 = 1.7% in the extreme. The abstract's claims that a 'moderate proportion' have seen an HMD and that a 'high proportion' do not use advanced displays therefore rest on an unverified assumption. The paper is transparent about small numbers, but transparency does not make the inductive leap secure. The load-bearing premise is the ignorability of non-response, and it is contradicted by the authors' own stated interpretation of who responded.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports the results of the second Advanced Image Displays for Astronomy (AIDA) survey, advertised to members of the Astronomical Society of Australia in 2021. Only 17 complete responses were obtained (~2.3% of the ~750 membership), and the authors compute 90% population-proportion confidence intervals (Table 14, Eq. A.3) to infer usage and awareness levels for the ASA membership. They find that most respondents use standard displays and few use advanced displays, that VR/MR head-mounted displays are the most familiar advanced display category, and that lack of knowledge and suitable software are the main barriers. Building on these results and on a review of recent VR applications in astronomy, the authors argue that VR/MR HMDs have reached sufficient maturity to replicate or replace most other advanced display categories.","tokens_in":26854,"tokens_out":2668,"duration_ms":25077,"significance":"If the survey inferences were valid, the paper would provide a useful snapshot of display adoption in a national astronomical community and would strengthen the case for prioritizing VR/MR investment. The paper's strength lies in its transparent presentation of raw response data, its explicit acknowledgment of the small sample, and its detailed technical argument that VR/MR HMDs can functionally emulate stereoscopic displays, digital domes, tiled display walls, and CAVE/CAVE2 environments, supported by a substantial review of recent VR applications in astronomy. However, the central statistical claim that these 17 self-selected responses support inferences about the ASA membership (or, by extension, international astronomers) is not presently justified, and the paper's own admitted participation bias undermines the abstract's 'we infer' statements.","major_comments":[{"comment":"The population-proportion confidence intervals are computed under sampling assumptions that do not hold for this survey. Equation A.3 applies the finite-population correction and normal approximation as if the 17 respondents were a probability sample from the ASA frame; in fact, participation was via two self-selected email responses during a pandemic, and Section 4.1 states that 'the most likely participation bias is an underlying interest in the use of advanced image displays.' When the outcome (interest/awareness) is correlated with the missingness indicator, the data are missing not at random, and the CIs in Table 14 are not valid population estimates. The direction of bias is known: for awareness and usage questions, the estimates are inflated. For example, Q5 shows 13/17 respondents had seen a VR/MR HMD (CI 59-93%), but if most of the 733 nonrespondents had not seen one, the population proportion could be as low as 13/750 ≈ 1.7%. The abstract's claims (1), (2), and (3) all rest on this unvalidated inductive step. I request either a sensitivity analysis under extreme-case nonresponse assumptions, or a reframing of all conclusions as descriptive of the respondent cohort only.","section":"Sec. 4.1, Eq. A.3, Table 14"},{"comment":"The abstract states 'From 17 complete survey responses... we infer that: (1) a high proportion of ASA members use standard displays but do not use advanced displays; (2) a moderate proportion have seen a VR/MR HMD...' and the Conclusions repeat these as 90% confidence statements. This is in tension with the paper's own earlier caveats in Section 3.4 ('17 responses is insufficient to draw strong conclusions') and Section 5 ('does not allow us to draw far-reaching conclusions'). The self-acknowledged limitation should be reflected in the abstract and conclusions; as written, the summary overstates the inferential strength. At minimum, the abstract should be revised to say that the results describe the respondents and are indicative only, or the authors should provide a defensible statistical justification for the population inference.","section":"Abstract and Sec. 5"},{"comment":"The response-rate description (Section 3.2) reports that two email blasts elicited only 20 responses, 17 of which are analyzed. The paper attributes the low response to 'survey fatigue' during COVID-19, citing external literature, but this does not address the more fundamental threat of non-ignorable nonresponse. The external-response citations are about response rates generally, not about the specific self-selection mechanism here. I recommend the authors explicitly discuss why nonresponse is ignorable with respect to the key outcomes, or present a bound analysis showing the conclusions are robust to plausible nonresponse scenarios. Without this, the 90% CIs in Table 14 should not be presented as if they capture the population uncertainty.","section":"Sec. 3.2, Fig. 2"}],"minor_comments":[{"comment":"The text states that the upper confidence level for Q1 is 'clipped at 100, with a value of 103.4 calculated.' This is reasonable, but it would be clearer to report the unclipped value in the table note rather than only in the text, since Table 14 displays 100.0.","section":"Sec. 4.1"},{"comment":"Table 16 reports responses from 82 participants, not the 17 AIDA 2021 respondents. It should be explicitly stated, both in the main text and in the table caption, that this question came from the larger survey described in Walsh et al. (submitted) and not from the AIDA 2021 subset. Currently, a reader could wrongly assume these are the same respondents.","section":"Sec. 4.5 and Table 16"},{"comment":"The comparison with AIDA 2005 uses different cohort definitions and different sample sizes (41 vs. 17). The paper acknowledges this partly, but Table 15 would benefit from showing the raw N values for each cell so the percentages are not over-interpreted.","section":"Sec. 4.2, Table 15"},{"comment":"The sentence 'As none of the respondents selected smartphone, advanced display or used the free text option' is slightly ambiguous: Table 7 shows 15/17 use a smartphone for non-research purposes, so 'smartphone' in Q3 may mean as a work display component. Please clarify the Q3 options.","section":"Sec. 3.3, Table 5"},{"comment":"There are occasional typographical issues (e.g., 'occured' in Sec. 3.4, 'and' in 'Visualisation of the local Milky Way projected onto the dome' in the Figure 1 caption). A careful copyedit is recommended.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is borderline for a methods-focused journal because its central inferential claim is not statistically supported. However, the empirical material (response tables, honest caveats) and the technical review of VR/MR as a multi-purpose display are valuable and can likely be rehabilitated with a modest revision that reframes the claims or adds a sensitivity analysis. The self-citation pattern is noticeable but acceptable given the authors' prior work in this niche. I would not recommend rejection without giving the authors the chance to correct the inference framing, since the underlying survey data are presented transparently."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the genuinely new material here is the 2021 AIDA survey data and its comparison to the 2005 baseline. The paper is clearly written and honestly acknowledges its tiny N and likely participation bias. The soft spot is the inferential step from 17 self-selected respondents to ~750 ASA members: the normal-approximation confidence intervals in Table 14 and Eq. A.3 adjust for sampling without replacement but not for non-response bias. The authors themselves say the most likely participation bias is underlying interest in advanced displays, which means the missingness is probably correlated with the outcome. If the 733 nonrespondents mostly had no interest, the population proportions could be far lower than the CIs suggest—for example, 13/750 = 1.7% having seen an HMD. So the abstract's 'we infer' claims about high and moderate proportions go beyond what the data can support. This is fixable by framing the results as descriptive of the self-selected sample and adding a sensitivity analysis, but as written it is the load-bearing weakness.\n\nWhat the paper does well: the survey design is sensible, the tables are clear, the barrier analysis is useful, and the comparison to 2005 showing little change in advanced display use is a legitimate longitudinal data point. The VR/MR replacement argument is explicitly qualitative and well-grounded in the literature, so I don't treat it as a measured result. The authors are transparent about limitations, which earns credit. Minor gripes: raw data and the full questionnaire are not provided, which limits reproducibility, and the self-citation is heavy, though the AIDA 2005 baseline is a legitimate comparison.\n\nWho this is for: people working in astro visualization and anyone making decisions about institutional investment in display facilities. It is a scoping study, not a definitive population estimate. A serious referee could help the authors align the abstract and conclusions with the actual evidential weight, so I would send it out rather than desk reject—but I would ask for major revision on the inference framing. If the response-rate problem is left unaddressed, the paper remains a useful anecdote with a strong VR/MR opinion attached.","headline":"A transparent, useful scoping survey whose new 2021 data point is real, but whose abstract overstates what 17 self-selected respondents can support about the whole ASA membership.","tokens_in":27451,"tokens_out":1855,"would_cite":false,"duration_ms":20187,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that VR/MR head-mounted displays have reached the technical maturity to replicate or replace the functionality of most other advanced image displays in astronomy, and its survey evidence points to VR/MR as the one…","keywords":["advanced image displays","virtual reality","head-mounted displays","astronomy data visualisation","AIDA survey","tiled display walls","digital domes","CAVE immersive environments"],"falsifier":"Run the same ten-question instrument on a randomized sample of several hundred ASA members: if the response pattern shows VR/MR awareness no higher than other advanced display categories, or self-reported research use of advanced displays above a few percent, the paper's three inferences would be contradicted. A smaller observational check: if teams that adopt HMDs for collaborative analysis report that the single-user headset fails the shared-viewing tasks that walls and domes handle, the claim that VR/MR can replicate or replace most other advanced displays is only partly true.","tokens_in":26353,"feed_emoji":"🥽","tokens_out":6493,"duration_ms":55510,"temperature":0.7,"pith_summary":"This paper tries to establish that astronomers still do almost all of their data visualisation on standard displays, that virtual reality and mixed reality head-mounted displays (VR/MR HMDs) are the only advanced display category whose awareness is growing, and that these headsets have now matured enough to stand in for stereoscopic screens, tiled display walls, digital domes, and CAVE-style immersive rooms. The stakes are practical: if true, departments and facilities can avoid investing in expensive, room-fixed display infrastructure and concentrate on one portable, comparatively cheap device. The evidence is a 2021 survey of ASA members that drew 17 complete responses, from which the authors compute 90% confidence intervals and infer high standard-display use, low advanced-display use, moderate hands-on familiarity with VR/MR headsets, and a request for better software and knowledge rather than for more hardware.","feed_headline":"One VR headset could replace most advanced astronomy displays","feed_subtitle":"A 17-respondent survey finds astronomers still work on laptops and monitors, with VR headsets the one advanced display gaining ground.","key_machinery":"The carrying mechanism is the VR/MR head-mounted display treated as a multi-purpose advanced display: an HMD is a display, not just a peripheral, because it supplies the left-eye/right-eye image pair of a stereoscopic display, the immersive all-sky field of a dome, the high-pixel navigable canvas of a tiled wall, and the head-tracked enclosure of a CAVE, all within a headset. The paper's measuring instrument is the AIDA 2021 survey, a ten-question instrument whose responses are turned into population-proportion confidence intervals with a finite-population correction for a sample of 17 drawn from roughly 750 ASA members, which is what lets the authors convert counts into claims about the wider membership.","core_discovery":"On the paper's own terms, the central discovery is that a single class of device, the VR/MR head-mounted display, can now perform the jobs of the four most significant advanced displays: it is inherently stereoscopic because it feeds separate images to the two eyes; it offers a 4π-steradian field of view, covering the all-sky role of a digital dome without a physical surface or projectors; it can host a virtual tiled display wall that the user walks around and zooms into; and it delivers the head-tracked, immersive experience that CAVE and CAVE2 rooms were built to provide, at consumer prices and without a dedicated facility. The 2021 AIDA survey responses are used to argue that awareness of VR/MR headsets has risen since the 2005 survey while usage of every other advanced display category remains essentially flat, and that the binding constraints have shifted from hardware cost and access to missing software and missing knowledge.","pith_inferences":["The 90% confidence intervals rest on 17 self-selected respondents, and the authors concede the likely participation bias is an existing interest in advanced displays; if uninterested astronomers stayed silent, the true usage and awareness rates are lower than stated, so the VR awareness advantage is the least secure of the three headline inferences.","The replacement claim is strongest for single-user exploration and weakest for the collaborative and audience-facing roles of domes and large walls, since an HMD isolates one viewer; a dome or wall remains better for a room of people or a public show.","A testable extension would repeat the survey with a larger, randomized sample of the same membership: the paper's prediction is that VR/MR awareness and non-research use will again outrank all other advanced display categories while research use stays near zero.","If headset resolution keeps improving toward 8K-class per-eye panels, the pixel-count objection to replacing tiled walls weakens further, so the paper's practical conclusion becomes more likely over time rather than less."],"forward_implications":["If the central claim is right, institutions can treat a small stock of VR/MR headsets as a substitute for maintaining or building tiled display walls, digital domes, stereoscopic projection setups, and CAVE rooms.","The main remaining barrier is software: survey respondents name missing applications and missing knowledge as the top obstacles, so the payoff for the community is a Python-compatible layer that connects astronomers' standard workflows to VR environments.","Awareness campaigns matter more than hardware subsidies: improved knowledge was the intervention respondents most often selected as likely to change their use.","Because 88% of respondents work in Unix/Linux or macOS environments, VR software that is tied to a single vendor's operating system will stall adoption; multiplatform toolchains are the direct corollary.","The result implies the international astronomy community, assumed to resemble the ASA on the authors' reasoning, is closer to a single-display future than expenditure on room-scale facilities would suggest."],"supporting_citations":[{"why":"Supplies the AIDA 2005 baseline survey whose 16-year comparison shows VR/MR awareness rising while other advanced-display usage stays flat.","marker":"Fluke et al. (2006)"},{"why":"Defines the original CAVE automatic virtual environment that the paper argues a VR/MR HMD can now emulate.","marker":"Cruz-Neira et al. (1992)"},{"why":"Documents CAVE2, the screen-based immersive room the paper cites as the high-cost facility a headset can replace.","marker":"Febretti et al. (2013)"},{"why":"Presents a VR tool for spectral data cubes, used as evidence that astronomy-specific VR software already exists.","marker":"Jarrett et al. (2021)"},{"why":"Reports circumstellar disk candidates found in Gaia data using VR, cited as proof that VR-based analysis yields scientific discoveries.","marker":"Higashio et al. (2022)"},{"why":"Provides the survey-fatigue explanation that contextualizes the low 17-response participation during the COVID-19 pandemic.","marker":"de Koning et al. (2021)"},{"why":"Documents the Python ecosystem whose dominant use among respondents motivates the paper's call for a Python-to-VR development bridge.","marker":"Astropy Collaboration et al. (2013)"},{"why":"Describes an earlier display abstraction layer cited as the precedent for today's Python-to-VR translation tools.","marker":"Barnes et al. (2006)"}],"fun_headline_variants":["VR headsets replace domes, walls, and immersive rooms","One VR headset could handle all advanced astronomy displays","Astronomers might only need one advanced display: a VR headset","Survey suggests VR headsets are the sole advanced display to keep","VR headsets now cover all-sky, 3D, and immersive astronomy views"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on 17 self-selected respondents, recruited through two email blasts during a pandemic, standing in for the roughly 750 members of the surveyed society and, by the authors' extension, for astronomers worldwide; if the astronomers who ignored the survey differ from those who answered, every usage and awareness rate in the conclusions is an overestimate.","fun_headline_variants_meta":{"raw":{"variants":["VR headsets replace domes, walls, and immersive rooms","One VR headset could handle all advanced astronomy displays","Astronomers might only need one advanced display: a VR headset","Survey suggests VR headsets are the sole advanced display to keep","VR headsets now cover all-sky, 3D, and immersive astronomy views"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000271,"raw_usage":{"total_tokens":1680,"prompt_tokens":1049,"completion_tokens":631,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":665,"completion_tokens_details":{"reasoning_tokens":540}},"tokens_in":665,"tokens_out":631,"duration_ms":6449,"temperature":1.0,"reasoning_tokens":540,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:07:45.801119+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same ten-question instrument on a randomized sample of several hundred ASA members: if the response pattern shows VR/MR awareness no higher than other advanced display categories, or self-reported research use of advanced displays above a few percent, the paper's three inferences would be contradicted. A smaller observational check: if teams that adopt HMDs for collaborative analysis report that the single-user headset fails the shared-viewing tasks that walls and domes handle, the claim that VR/MR can replicate or replace most other advanced displays is only partly true.","supporting_citations":[{"cited_title":", author Bourke , P.D","cited_arxiv_id":null,"evidence_quote":"Supplies the AIDA 2005 baseline survey whose 16-year comparison shows VR/MR awareness rising while other advanced-display usage stays flat."},{"cited_title":", author Sandin, D.J","cited_arxiv_id":null,"evidence_quote":"Defines the original CAVE automatic virtual environment that the paper argues a VR/MR HMD can now emulate."},{"cited_title":", author Nishimoto , A","cited_arxiv_id":null,"evidence_quote":"Documents CAVE2, the screen-based immersive room the paper cites as the high-cost facility a headset can replace."},{"cited_title":", author Comrie , A","cited_arxiv_id":null,"evidence_quote":"Presents a VR tool for spectral data cubes, used as evidence that astronomy-specific VR software already exists."},{"cited_title":", author Kuchner , M.J","cited_arxiv_id":null,"evidence_quote":"Reports circumstellar disk candidates found in Gaia data using VR, cited as proof that VR-based analysis yields scientific discoveries."},{"cited_title":", author Egiz , A","cited_arxiv_id":null,"evidence_quote":"Provides the survey-fatigue explanation that contextualizes the low 17-response participation during the COVID-19 pandemic."}],"review_version":1}