New Guidelines to make AI in Clinical Trials Trustworthy

It’s perhaps one of the riskiest business along with oil exploration. It takes over a decade, costs an average of US$2.5 billion per experiment and the success rate is an abysmal 14%. Yes, we are talking about clinical research, the critical element in the workflow in bringing a drug from the laboratory to the market.

Nearly 50% of the total time, and capital expenditure during the drug development process, is on conducting clinical research; but it promises incredibly high returns if the drug is successful.

Organizations have been trying to ensure accuracy in research, increase the success rate of drug discovery, and cutting down the time from lab-to-market. Major IT companies like Microsoft and IBM have been teaming up with big pharma corporations to introduce artificial intelligence (AI) in search of the holy grail of laser-focused clinical research bringing successful drugs to the market in just a couple of years or even months.

A quantum leap from the decades it now takes through the traditional route. The inordinate research time and dreadful success rate are the chief reasons why new drugs are so expensive.
Pfizer has partnered with IBM Watson to identify more robust targets during the discovery phase.

They are processing thousands of scientific publications to determine novel combinations of drugs for improved efficacy and optimise the patient selection for clinical trials. Insilico Medicine and Exscientia are attempting to utilise genomics and artificial intelligence tools for computational design of new drug candidates.

If the integration of AI into the drug discovery and design process works, it can have incredible disruptive effects. It could revolutionise the entire drug discovery procedure. Cutting down drug discovery from years to months, especially in the pandemic scenario, would mean an incalculable effect on the larger pharma environment.

For an industry which justifies high drug prices by the time and cost invested at the development phase, it will be a tectonic shift that seems like a gamble worth taking. AI in drug discovery market is estimated to grow at a CAGR of 40.5% to an estimated value of US$4 million by 2027. However, factors such as lack of data sets and dearth of skilled labour will act as restraints for the market in the 2020–2027 forecast period according to a report from Data Bridge Market Research.

It is now widely accepted that AI will revolutionise many aspects of healthcare, but there is currently a trust issue. Despite a growing number of headlines reporting that AI systems outperform doctors in making medical diagnoses, research studies have highlighted serious concerns about their quality.

There is a mounting demand for guidelines to ensure that medical AI research is subject to the same scrutiny as drug development and diagnostic tests. An international consortium of medical experts has now introduced the first official standards for clinical trials that involve artificial intelligence. The move comes at a time when hype around medical AI is peaking, with inflated and unverified claims about the effectiveness of certain tools threatening to undermine people’s trust in AI overall.

Announced in Nature Medicine, the British Medical Journal, and the Lancet, the new standards offer two sets of guidelines on how clinical trials are to be conducted and reported. AI researchers will now have to describe the skills needed to use an AI tool, the setting in which the AI is evaluated, details about how humans interact with the AI, the analysis of error cases, and more. The CONSORT 2010 (Consolidated Standards of Reporting Trials) has set minimum guidelines for reporting randomised trials.

More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI extension is a new reporting guideline for clinical trials evaluating interventions with an AI component.

It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI. Both guidelines were developed through a staged consensus process, involving a literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed on in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants).
CONSORT-AI includes 14 new items, which were considered sufficiently important for AI interventions, and should be routinely reported in addition to the core CONSORT 2010 items.

CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and providing analysis of error cases.

Apart from promoting transparency and completeness in reporting clinical trials, this will assist editors and peer-reviewers, as well as general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.