Investigating the Use of LLMs for Evidence Briefings Generation in Software Engineering

Bianca Trinkenreich; Bruno Cartaxo; Marcos Alves; Marcos Kalinowski; Mauro Marcelino; S\'ergio Soares; Simone D.J. Barbosa

REVIEW 3 major objections 4 minor 28 references

Investigating the Use of LLMs for Evidence Briefings Generation in Software Engineering

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This registered report proposes a controlled experiment to determine whether LLM-generated evidence briefings can match human-made briefings in perceived content fidelity, ease of understanding, and usefulness; the experiment has not yet…

desk verdict A careful registered protocol for comparing LLM vs human evidence briefings, held back by a content-fidelity instrument that cannot detect omitted findings. read the letter →

arxiv 2507.15828 v1 pith:IFWERXI4 submitted 2025-07-21 cs.SE

Mauro Marcelino , Marcos Alves , Bianca Trinkenreich , Bruno Cartaxo , Sérgio Soares , Simone D.J. Barbosa , Marcos Kalinowski This is my paper · ORCID

classification cs.SE

keywords evidencebriefingslargelanguagemodelsretrieval-augmentedgenerationcontrolledexperimentregisteredreportevidence-basedsoftwareengineeringcontentfidelityresearchsynthesis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is a registered report: it lays out a protocol for testing whether automatically generated evidence briefings can stand in for manually produced ones. Evidence briefings are one-page summaries of systematic-review findings designed to move research into practice, but writing them by hand does not scale. The authors built a retrieval-augmented LLM tool, used it to generate briefings for two existing secondary studies, and designed two blinded controlled experiments that compare those outputs with the human-made versions. The decisive claim is that the planned null hypotheses—no perceived difference in content fidelity, ease of understanding, or usefulness—are the right test of suitability for broader adoption. Results are to come after the trials.

What carries the argument

The experimental factor is generated by a pipeline that combines a commercial LLM with retrieval-augmented generation: for each briefing section, examples are retrieved from a store of 54 human-made evidence briefings and fed to the model as grounded context, with fixed generation parameters and instruction-based prompting. The measurement machinery is a 7-point Likert survey whose items operationalize the three constructs: content fidelity via contradiction, certainty illusion, and fabricated content; ease of understanding via clarity, structure, and conciseness; and usefulness via relevance and actionability. The design machinery is a completely randomized one-factor, two-treatment crossover in which each participant rates two different topics, pair programming and definition of done, so order and carryover effects are controlled.

What would settle it

Collect the two generated briefings and the two source papers, have coders independently count contradictions, certainty mismatches, and unsupported claims, then compare those counts with the participants' perceived-fidelity Likert ratings; a weak or absent correlation would undercut the construct-validity assumption on which the experiment's conclusions depend.

Watch

Extended reading notes

Core claim

The central claim is not yet an empirical result; it is that the suitability of LLM-generated evidence briefings can be settled by a crossover experiment in which researchers rate content fidelity against the full source paper and practitioners rate ease of understanding and usefulness, with everyone blinded to which briefing is automatic and which is human-made. The paper asserts that this design, with its three null hypotheses, provides an appropriate test. It also demonstrates that the material conditions for the test exist: an automated pipeline can produce briefings for two previously hand-made cases, and the prompts, tool configuration, outputs, and surveys are archived for reproduction.

Load-bearing premise

The load-bearing premise is that participants' ratings measure the constructs they are asked about: that Likert answers about contradiction, certainty, and fabrication track real fidelity to the source text, and that self-reported understanding and usefulness track how well practitioners can actually use the briefing.

Editorial extensions

If this is right

If the null hypotheses are not rejected, automation could produce briefings perceived as equivalent to human ones, removing the main scalability barrier to evidence briefings.
If practitioners rate the LLM briefings lower on understanding or usefulness, the result would show that current generation settings are not yet ready to replace manual production.
The open-text responses embedded in the questionnaire let the authors explain quantitative differences with participant reasoning, strengthening the interpretation.
Because all generation parameters, prompts, outputs, and instruments are archived, other teams can replicate or extend the comparison on different secondary studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The comparison may partly measure briefing-quality differences rather than generation method alone, since one human-made briefing was previously validated and the other was not; a null result could reflect that asymmetry as much as LLM competence.
The experiment as designed tests a hybrid pipeline, human-written briefings as retrieval examples plus LLM rewriting, rather than pure LLM generation; varying the retrieval corpus would reveal how much of any measured quality depends on those human examples.
The Likert self-reports leave room for an objective companion check: counting contradictions, certainty mismatches, or fabricated claims in the generated briefings and comparing those counts with perceived-fidelity ratings.
A cost-adjusted reading would be that LLM briefings do not need to be better, only comparable, since the motivation is labor savings; the current protocol is well suited to establish equivalence, not superiority.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a protocol, not a result, and should be judged as one. The design is thoughtful and mostly well reported; the main weakness is that the content-fidelity measure misses a central failure mode.

What they do right: a crossover design with counterbalancing, separate researcher and practitioner experiments, random assignment, blinding, adherence to LLM reporting guidelines, and an immutable open-science repository with prompts, code, briefings, and instruments. The RAG pipeline is described concretely enough to reproduce. They also honestly acknowledge the uneven validation of the two human-made briefings used as comparators.

The soft spots, in order of seriousness. First, the content-fidelity instrument in Section III-D is built from three Likert dimensions adapted from Tang et al.: contradiction, certainty illusion, and fabricated content. All three detect the presence of incorrect or unsupported material; none detects the absence of important material. A briefing that quietly drops a key finding from the source SLR can still receive high ratings on these items, even though the paper itself defines a briefing as needing to present the main findings. The optional open-text field might catch this qualitatively, but the quantitative test of H0_1 will be blind to it. The pilot of two researchers and two practitioners does not establish construct coverage. The stated mitigation in III-H—adapting an established taxonomy—addresses reliability of error detection, not under-coverage. If the instrument cannot register omission, a finding of "no difference" becomes hard to interpret: it could reflect the measure's blindness rather than LLM-human equivalence.

Second, the model identifier in Table I is inconsistent: "GPT-4-o-mini (gpt-4-0125-preview)" mixes two different model names. That is easy to fix but matters for reproducibility. Third, although the paper calls itself a registered report, it gives no registration ID and no pre-specified statistical analysis plan or sample-size justification, even though conclusion validity flags power as a threat. Fourth, the "definition of done" human briefing did not go through the same validation as the pair programming one; they acknowledge it, but it remains an extra confound.

None of this is fatal. The protocol is a solid foundation for a useful empirical study, and the blind spot in the instrument is fixable before trials or can be handled in interpretation. This paper is for researchers working on evidence transfer in SE and on evaluating LLM summarization. It deserves a serious referee, with the expectation that the instrument and the missing methodological details get addressed.

Referee Report

3 major / 4 minor

Summary. This registered report describes a protocol for two controlled experiments comparing LLM-generated evidence briefings against human-made ones. In the first experiment, researchers rate content fidelity after reading full research papers and briefings; in the second, practitioners rate ease of understanding and usefulness from the briefings alone. The briefings cover pair programming and definition of done, each with both a human and an LLM-generated version produced by an RAG-based tool. The paper defines null hypotheses for each research question, a counterbalanced crossover design, survey instruments with Likert items, and a threat analysis. No results are reported; the central claim is that the described protocol can test the hypotheses.

Significance. The paper is a protocol paper, so its contribution is a falsifiable experimental design. If executed, it would provide empirical evidence on whether LLM-generated evidence briefings can match human-produced ones in perceived fidelity, understandability, and usefulness, which is relevant to evidence-based software engineering and the adoption of LLM synthesis tools. The authors openly provide their materials, follow LLM reporting guidelines, and sensibly separate researcher and practitioner evaluation roles. The main weaknesses are the incomplete operationalization of content fidelity, the absence of a pre-specified analysis plan and sample-size justification, and the use of an unvalidated human comparator for one of the two briefings; these are fixable in revision.

major comments (3)

[III-D / III-G] The content fidelity instrument in Section III-D uses three Likert dimensions adapted from Tang et al. (contradiction, certainty illusion, fabricated content). All three detect the presence of incorrect or unsupported content; none detects the absence of important content. Since the paper defines content fidelity as faithfully representing the content of the original research article and the evidence briefing template includes a 'Main Findings' section, an LLM briefing that omits a key finding could still receive maximal fidelity ratings, making the test for H0_1 insensitive to omission, a central failure mode in automatic summarization. The pilot described in Section III-G (two researchers, two practitioners) does not establish construct coverage. I recommend adding a completeness dimension (e.g., items on whether all key findings are present) or a verification task against a predefined list of key findings, and reporting pilot evidence on content coverage.
[III-H / overall] The protocol does not pre-specify an analysis plan or sample-size justification. Section III-H states that sample size will be 'carefully considered throughout the experiment,' which is insufficient for a confirmatory registered report. The null hypotheses H0_1–H0_3 can only be tested if the statistical procedures, effect size of interest, significance level, power, and handling of the repeated-measures crossover structure are fixed before data collection. Please specify, for example, the planned mixed-effects model (with random effects for participants and period/sequence), the treatment of ordinal Likert data, and a power analysis based on a planned effect size.
[III-G / III-H] The human-made comparator for the definition-of-done briefing was not validated to the same degree as the pair-programming briefing (Section III-G, External Validity). Since this briefing is one of only two baselines in the crossover, any result concerning equivalence or difference could be confounded by the quality of the human comparator. The acknowledgment in the threats section is appropriate, but the protocol should either include two validated human briefings or add a preliminary validation step for the unvalidated briefing (e.g., review by SLR authors) to ensure the baseline represents the intended condition.

minor comments (4)

[Table I] The model identifier 'GPT-4-o-mini (gpt-4-0125-preview)' is inconsistent; gpt-4-0125-preview is a GPT-4 Turbo model, not GPT-4o mini. Specify the exact model string used in the API calls to ensure reproducibility.
[III-G] The pilot study is described as having two researchers and two practitioners with 'no further adjustments deemed necessary.' Please report what was assessed (e.g., item clarity, instrument length) and any quantitative pilot results, to support the claim that the instruments are ready for deployment.
[III-D] In the Likert scale description, '3- Slightly Disagree' should be formatted as '3 - Slightly Disagree' to match the other anchors; also add a space after '2 - Disagree' for consistency.
[III-D] The description of the RAG mechanism does not specify retrieval details (e.g., number of snippets per section, similarity threshold, chunk size). Providing these parameters would improve reproducibility, consistent with the open-science claim.

Circularity Check

0 steps flagged · score 0.0 of 10

Registered protocol for comparing LLM- and human-generated evidence briefings; no fitted parameters, no derivation by construction, and no load-bearing self-citation.

full rationale

This is a registered report describing a planned controlled experiment, not a paper claiming derived predictions. No parameter is fitted to data and then renamed as a prediction; the LLM tool is configured with fixed, documented parameters and its outputs are archived rather than tuned against the dependent variables. The hypotheses H0_1 through H0_3 (Section III-C) are testable comparisons of perceived content fidelity, ease of understanding, and usefulness, and the measurement instruments are adapted from an external taxonomy (Tang et al. [21]) and external survey guidelines, not defined circularly in terms of the hypotheses. The human-made briefings used as comparators come from prior work [7], [8] with some author overlap, but this is a baseline artifact rather than a load-bearing theorem: the experiment does not assume human briefings are correct, and the outcome is an empirical comparison by third-party participants. The RAG database contains 54 human-made briefings as style exemplars, but it explicitly excludes the two briefings under evaluation, so the generated outputs are not copied from the objects of comparison. The skeptic concern about content-fidelity items omitting completeness is a construct-validity threat, not a circular step, because the paper does not define fidelity solely in terms of those items; it operationalizes fidelity through a subset of error types while also collecting open-text responses. No equation, derivation, or cited uniqueness theorem reduces the protocol's claims to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The experiment rests on no fitted scientific parameters; the only free numbers are hand-chosen LLM hyperparameters. The central assumptions are construct-validity premises and sampling premises. The RAG corpus of 54 human briefings is an input artifact, not an invented entity.

free parameters (3)

LLM temperature = 0.5
Hand-chosen generation hyperparameter; no sensitivity analysis is reported, so the comparison is conditional on this value.
Top-p = 1.0
Hand-chosen nucleus sampling threshold; fixed without reported rationale or sensitivity check.
Max tokens = 1024
Output limit set manually; constrains briefing length and could bias perceived conciseness.

assumptions (4)

domain assumption Evidence briefings, as defined by Cartaxo et al., are a valid and useful knowledge-transfer medium for SE practice.
Section II-A adopts this premise; if false, optimizing briefing generation would not improve research-practice transfer.
domain assumption The Tang et al. error taxonomy can be operationalized as Likert items to measure content fidelity.
Section III-D adapts qualitative coding categories into rating items without reported psychometric validation.
domain assumption Convenience-sampled volunteers, randomly assigned, yield unbiased estimates of treatment differences.
Section III-E acknowledges convenience sampling but relies on randomization and blocking to control bias.
domain assumption Researchers and practitioners are distinct populations with appropriate evaluation roles.
Section III-B assigns fidelity to researchers and ease and usefulness to practitioners; if practitioners could not judge usefulness without the source text, the design would be weaker.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Investigating the Use of LLMs for Evidence Briefings Generation in Software Engineering." pith.science (2026). https://pith.science/paper/IFWERXI4

@misc{pith2026250715828,
  author       = {Pith},
  title        = {Pith review of: Investigating the Use of LLMs for Evidence Briefings Generation in Software Engineering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IFWERXI4}},
  note         = {Machine review of arXiv:2507.15828}
}

read the original abstract

[Context] An evidence briefing is a concise and objective transfer medium that can present the main findings of a study to software engineers in the industry. Although practitioners and researchers have deemed Evidence Briefings useful, their production requires manual labor, which may be a significant challenge to their broad adoption. [Goal] The goal of this registered report is to describe an experimental protocol for evaluating LLM-generated evidence briefings for secondary studies in terms of content fidelity, ease of understanding, and usefulness, as perceived by researchers and practitioners, compared to human-made briefings. [Method] We developed an RAG-based LLM tool to generate evidence briefings. We used the tool to automatically generate two evidence briefings that had been manually generated in previous research efforts. We designed a controlled experiment to evaluate how the LLM-generated briefings compare to the human-made ones regarding perceived content fidelity, ease of understanding, and usefulness. [Results] To be reported after the experimental trials. [Conclusion] Depending on the experiment results.

Figures

Figures reproduced from arXiv: 2507.15828 by the authors.

**Figure 1.** Experiment planning steps [25]. A. Goal Definition The main goal is to investigate whether automatically generated evidence briefings can be suitable for broader adoption. This goal was tailored into two specific sub-goals, defined using the Goal-Question-Metric (GQM) goal definition template proposed by Basili et al. [4] as follows. GOAL 1: Analyze LLM-based evidence briefing generation for the purpose of charact… view at source ↗

**Figure 2.** Crossover design [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 24 canonical work pages

[1]

The design of a survey on bridging the gap between software industry expectations and academia

Deniz Akdur. The design of a survey on bridging the gap between software industry expectations and academia. In2019 8th Mediterranean Conference on Embedded Computing (MECO), pages 1–5. IEEE, 2019

work page 2019
[2]

Deepika Badampudi, Farnaz Fotrousi, Bruno Cartaxo, and Muhammad Usman. Reporting consent, anonymity and confidentiality procedures adopted in empirical studies using human participants.e-Informatica Software Engineering Journal, 16(1):220109, July 2022. Available online: 22 Jul. 2022

work page 2022
[3]

Rouge metric evaluation for text summarization techniques.Available at SSRN 4120317, 2022

Marcello Barbella and Genoveffa Tortora. Rouge metric evaluation for text summarization techniques.Available at SSRN 4120317, 2022

work page 2022
[4]

Basili and H.D

V .R. Basili and H.D. Rombach. The tame project: towards improvement- oriented software environments.IEEE Transactions on Software Engi- neering, 14(6):758–773, 1988

work page 1988
[5]

Making software engineering research relevant.Computer, 47(4):80–83, 2014

Sarah Beecham, P ´adraig O’Leary, Sean Baker, Ita Richardson, and John Noll. Making software engineering research relevant.Computer, 47(4):80–83, 2014

work page 2014
[6]

What support do systematic reviews provide for evidence-informed teaching about software engineering practice?e-informatica software engineering journal, 14(1):7–60, 2020

David Budgen, Pearl Brereton, Nikki Williams, and Sarah Drummond. What support do systematic reviews provide for evidence-informed teaching about software engineering practice?e-informatica software engineering journal, 14(1):7–60, 2020

work page 2020
[7]

Towards a model to transfer knowledge from software engineering research to practice

Bruno Cartaxo, Gustavo Pinto, and Sergio Soares. Towards a model to transfer knowledge from software engineering research to practice. Information and Software Technology, 97:80–82, 2018

work page 2018
[8]

Ev- idence briefings: Towards a medium to transfer knowledge from sys- tematic reviews to practitioners

Bruno Cartaxo, Gustavo Pinto, Elton Vieira, and S ´ergio Soares. Ev- idence briefings: Towards a medium to transfer knowledge from sys- tematic reviews to practitioners. InProceedings of the 10th ACM/IEEE international symposium on empirical software engineering and mea- surement, ESEM ’16, pages 1–10, New York, NY , USA, 2016. Associ- ation for Computing...

work page 2016

Show all 28 references

[9]

Sage publications, 2017

John W Creswell and J David Creswell.Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications, 2017

2017
[10]

Fabbri, Wojciech Kry ´sci´nski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev

Alexander R. Fabbri, Wojciech Kry ´sci´nski, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. Summeval: Re-evaluating summarization evaluation, 2021

2021
[11]

Practical relevance of software engineering research: synthesizing the community’s voice

Vahid Garousi, Markus Borg, and Markku Oivo. Practical relevance of software engineering research: synthesizing the community’s voice. Empirical Software Engineering, 25:1687–1754, 2020

2020
[12]

Aligning software engineering education with industrial needs: A meta-analysis.Journal of Systems and Software, 156:65–83, 2019

Vahid Garousi, G ¨orkem Giray, Eray T ¨uz¨un, Cagatay Catal, and Michael Felderer. Aligning software engineering education with industrial needs: A meta-analysis.Journal of Systems and Software, 156:65–83, 2019

2019
[13]

Key challenges in prompt engineering

Vladimir Geroimenko. Key challenges in prompt engineering. InThe Essential Guide to Prompt Engineering: Key Principles, Techniques, Challenges, and Security Risks, pages 85–102. Springer, 2025

2025
[14]

Hannay, Tore Dyb ˚a, Erik Arisholm, and Dag I.K

Jo E. Hannay, Tore Dyb ˚a, Erik Arisholm, and Dag I.K. Sjøberg. The effectiveness of pair programming: A meta-analysis.Information and Software Technology, 51(7):1110–1122, 2009. Special Section: Software Engineering for Secure Systems

2009
[15]

Springer Nature Switzerland, Cham, 2024

Marcos Kalinowski, Allysson Allex Ara ´ujo, and Daniel Mendez.Teach- ing Survey Research in Software Engineering, pages 501–527. Springer Nature Switzerland, Cham, 2024

2024
[16]

Jorgensen

Barbara Kitchenham, Tore Dyb ˚a, and M. Jorgensen. Evidence-based software engineering. InProceedings. 26th International Conference on Software Engineering, pages 273– 281, 06 2004

2004
[17]

What makes agile software development agile?IEEE Transactions on Software Engineering, 48(9):3523–3539, 2022

Marco Kuhrmann, Paolo Tell, Regina Hebig, et al. What makes agile software development agile?IEEE Transactions on Software Engineering, 48(9):3523–3539, 2022

2022
[18]

A systematic review on the use of definition of done on agile software development projects

Mirko Perkusich, Ana Silva, Thalles Ara ´ujo, Ednaldo Dilorenzo, Jo ˜ao Nunes, Hyggo Almeida, and Angelo Perkusich. A systematic review on the use of definition of done on agile software development projects. InProceedings of the 21st international conference on evaluation and...

2017
[19]

Building understandable messaging for policy and evidence review (bumper) with ai.arXiv preprint arXiv:2407.12812, 2024

Katherine A Rosenfeld, Maike Sonnewald, Sonia J Jindal, Kevin A McCarthy, and Joshua L Proctor. Building understandable messaging for policy and evidence review (bumper) with ai.arXiv preprint arXiv:2407.12812, 2024

2024 arXiv
[20]

Motivation to perform systematic reviews and their impact on software engineering practice

Ronnie ES Santos and Fabio QB Da Silva. Motivation to perform systematic reviews and their impact on software engineering practice. In2013 ACM/IEEE international symposium on empirical software engineering and measurement, pages 292–295. IEEE, 2013

2013
[21]

Evaluating large language models on medical evidence summa- rization.NPJ digital medicine, 6(1):158, 2023

Liyan Tang, Zhaoyi Sun, Betina Idnay, Jordan G Nestor, Ali Soroush, Pierre A Elias, Ziyang Xu, Ying Ding, Greg Durrett, Justin F Rousseau, et al. Evaluating large language models on medical evidence summa- rization.NPJ digital medicine, 6(1):158, 2023

2023
[22]

Gomez, Łukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InProceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Ho...

2017
[23]

Towards evaluation guidelines for empirical studies involving llms

Stefan Wagner, Marvin Mu ˜noz Bar ´on, Davide Falessi, and Sebastian Baltes. Towards evaluation guidelines for empirical studies involving llms. InProceedings of the 2nd International Workshop on Method- ological Issues with Empirical Studies in Software Engineering (WSESE 202...

2025 arXiv
[24]

Thoughts on applicability.Journal of Systems and Software, 215:112086, 2024

Titus Winters. Thoughts on applicability.Journal of Systems and Software, 215:112086, 2024

2024
[25]

Springer Nature, 2024

Claes Wohlin, Per Runeson, Martin H ¨ost, Magnus C Ohlsson, Bj ¨o Regnell, and Anders Wessl´en.Experimentation in Software Engineering. Springer Nature, 2024

2024
[26]

Closing the gap between open source and commercial large language models for medical evidence summarization.NPJ digital medicine, 7(1):239, 2024

Gongbo Zhang, Qiao Jin, Yiliang Zhou, Song Wang, Betina Idnay, Yiming Luo, Elizabeth Park, Jordan G Nestor, Matthew E Spotnitz, Ali Soroush, et al. Closing the gap between open source and commercial large language models for medical evidence summarization.NPJ digital medicine,...

2024
[27]

Yu, and Jiawei Zhang

Haopeng Zhang, Philip S. Yu, and Jiawei Zhang. A systematic survey of text summarization: From statistical methods to large language models, 2024

2024
[28]

Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675, 2019

1904 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

The design of a survey on bridging the gap between software industry expectations and academia

Deniz Akdur. The design of a survey on bridging the gap between software industry expectations and academia. In2019 8th Mediterranean Conference on Embedded Computing (MECO), pages 1–5. IEEE, 2019

work page 2019

[2] [2]

Deepika Badampudi, Farnaz Fotrousi, Bruno Cartaxo, and Muhammad Usman. Reporting consent, anonymity and confidentiality procedures adopted in empirical studies using human participants.e-Informatica Software Engineering Journal, 16(1):220109, July 2022. Available online: 22 Jul. 2022

work page 2022

[3] [3]

Rouge metric evaluation for text summarization techniques.Available at SSRN 4120317, 2022

Marcello Barbella and Genoveffa Tortora. Rouge metric evaluation for text summarization techniques.Available at SSRN 4120317, 2022

work page 2022

[4] [4]

Basili and H.D

V .R. Basili and H.D. Rombach. The tame project: towards improvement- oriented software environments.IEEE Transactions on Software Engi- neering, 14(6):758–773, 1988

work page 1988

[5] [5]

Making software engineering research relevant.Computer, 47(4):80–83, 2014

Sarah Beecham, P ´adraig O’Leary, Sean Baker, Ita Richardson, and John Noll. Making software engineering research relevant.Computer, 47(4):80–83, 2014

work page 2014

[6] [6]

What support do systematic reviews provide for evidence-informed teaching about software engineering practice?e-informatica software engineering journal, 14(1):7–60, 2020

David Budgen, Pearl Brereton, Nikki Williams, and Sarah Drummond. What support do systematic reviews provide for evidence-informed teaching about software engineering practice?e-informatica software engineering journal, 14(1):7–60, 2020

work page 2020

[7] [7]

Towards a model to transfer knowledge from software engineering research to practice

Bruno Cartaxo, Gustavo Pinto, and Sergio Soares. Towards a model to transfer knowledge from software engineering research to practice. Information and Software Technology, 97:80–82, 2018

work page 2018

[8] [8]

Ev- idence briefings: Towards a medium to transfer knowledge from sys- tematic reviews to practitioners

Bruno Cartaxo, Gustavo Pinto, Elton Vieira, and S ´ergio Soares. Ev- idence briefings: Towards a medium to transfer knowledge from sys- tematic reviews to practitioners. InProceedings of the 10th ACM/IEEE international symposium on empirical software engineering and mea- surement, ESEM ’16, pages 1–10, New York, NY , USA, 2016. Associ- ation for Computing...

work page 2016