The ESG Best Practice Working Group in Automation and Technological Advancement is chaired jointly by James Thomas (LACES) and Alex Sutton (@CRSU).

The group’s activities fall into three main areas

• Meeting on on a monthly basis to share tools, experiences and ideas and to further understanding across ESGs
• Sharing tools and resources on this web page
• Developing and sharing templates for studies within a review (SWARs) for ESGs (and others) to use

Agenda for 2024-5 meetings (1pm on third Thursday of the month)

(See links where available for slides)

2026 upcoming meetings

• February 19: Continued discussion on ESG tools survey: which tools are safe to use, and when (and how do we know)?
• March 19: Tool demo*
• April 16: Agentic search tools for exploratory and supplementary searching
• May 14: TBA
• June 18
• July 16

* individual tool developers invited to present and discuss their tools

2026 meeting history

• January 15: Further discussion and development of our survey outcomes

2025 meeting history

• January 16: Tool demo and discussion: Nested Knowledge*
• February 20: How do we validate tools for use?
• March 20: Tool demo and discussion: TERA
• April 17: (cancelled - university closure day)
• May 15: Overview and Demo of Apps for Evidence Synthesis Developed by the Complex Review Synthesis Unit (CRSU): Alex Sutton
• June 19: Living SWAR of GenAI tools for eligibility screening: Ian Shemilt
• July 17: Guidance for the responsible use of AI in evidence synthesis (RAISE: Link to Open Science Framework Page)
• September 20: Critiquing and evaluating this new generation of LLM-based evidence synthesis tools
• October 16: Taking stock: Alex Sutton
• November 20: Alex Sutton: results of survey on which tools ESGs are using
• December 18: continued discussion on ESG tools survey: which tools are safe to use, and when (and how do we know)?

2024 meeting history

• April: Alex Sutton on using R for meta-analysis and systematic review processes
• May 16: James Thomas on collaborative work to develop guidelines for the responsible use of AI in systematic reviews
• June 20: Andrew Booth on automation methods to assist in study selection
• July 18: 'Lightning presentations' from group members with the aim of taking stock: where are we in terms of automation? What do we need? What do we need to know?
• August 15: Open discussion
• September 19: Large language models for data extraction (can we? should we?)
• October 17: Gerald Gartlehner: Opportunities and challenges for data extraction with a large language model
• November 21: Joint meeting with the ESG working group on Studies Within A Review (SWAR)
• December 19:Evidence Synthesis Infrastructure Collaborative slides can be downloaded here

Meetings are open to anyone in an NIHR ESG. Please contact laces@ucl.ac.uk if you would like to be added to the calendar invite list. 

Resources

• Database of digital evidence synthesis tools
• A list of useful tools to try out
• Screening tools to try / evaluate / use (from Andrew Booth's talk in June 2024)