ESG working group in automation

ESG working group in automation and technological advancement

The ESG Best Practice Working Group in Automation and Technological Advancement is chaired jointly by James Thomas (LACES) and Alex Sutton (@CRSU).

The group’s activities fall into three main areas

Meeting on on a monthly basis to share tools, experiences and ideas and to further understanding across ESGs
Sharing tools and resources on this web page
Developing and sharing templates for studies within a review (SWARs) for ESGs (and others) to use

(See links where available for slides)

2026 upcoming meetings

May 21: Presentation and discussion: agentic mapping (Kaitlyn Hair, Nadia Soliman, Jonah Varion)
June 18: Strategic issues for the group:
- Developing a position statement on the safe and responsible use of AI tools
- Developing guidance on within-review validation studies
July 16: Evaluating the Elicit LLM-based tool: results from four studies

* individual tool developers invited to present and discuss their tools

2026 meeting history

January 15: Further discussion and development of our survey outcomes
February 19: Continued discussion on ESG tools survey: which tools are safe to use, and when (and how do we know)
March 19: Living mode parameters and automation in living Evidence and Gap Maps (L-EGMs): a scoping review: Tomasz Kozakiewicz | Newcastle University
April 16: Andrew Booth: Agentic search tools for exploratory and supplementary searching

2025 meeting history

January 16: Tool demo and discussion: Nested Knowledge*
February 20: How do we validate tools for use?
March 20: Tool demo and discussion: TERA
April 17: (cancelled - university closure day)
May 15: Overview and Demo of Apps for Evidence Synthesis Developed by the Complex Review Synthesis Unit (CRSU): Alex Sutton
June 19: Living SWAR of GenAI tools for eligibility screening: Ian Shemilt
July 17: Guidance for the responsible use of AI in evidence synthesis (RAISE: Link to Open Science Framework Page)
September 20: Critiquing and evaluating this new generation of LLM-based evidence synthesis tools
October 16: Taking stock: Alex Sutton
November 20: Alex Sutton: results of survey on which tools ESGs are using
December 18: continued discussion on ESG tools survey: which tools are safe to use, and when (and how do we know)?

2024 meeting history

April: Alex Sutton on using R for meta-analysis and systematic review processes
May 16: James Thomas on collaborative work to develop guidelines for the responsible use of AI in systematic reviews
June 20: Andrew Booth on automation methods to assist in study selection
July 18: 'Lightning presentations' from group members with the aim of taking stock: where are we in terms of automation? What do we need? What do we need to know?
August 15: Open discussion
September 19: Large language models for data extraction (can we? should we?)
October 17: Gerald Gartlehner: Opportunities and challenges for data extraction with a large language model
November 21: Joint meeting with the ESG working group on Studies Within A Review (SWAR)
December 19:Evidence Synthesis Infrastructure Collaborative slides can be downloaded here

Meetings are open to anyone in an NIHR ESG. Please contact laces@ucl.ac.uk if you would like to be added to the calendar invite list.