AI in healthcare: Balancing progress with patient-led care

Our main concerns

The critical role of human connection

The immediate impact of AI implementation on patient experience raises significant concerns about the fundamental relationship between clinicians and patients. Whilst AI promises enhanced efficiency in diagnosis and treatment, there’s a real risk of diminishing the human element that many patients rely upon. 

The latest GP Patient Survey findings are particularly revealing; patients with long-term conditions consistently report that meaningful conversations about managing their conditions are more valuable than formal care plans. This highlights an essential truth – healthcare is not just about clinical outcomes, but about human connection and understanding. 

Digital first and health inequalities

The Government’s planned shift ‘from analogue to digital’ embraces new technologies, such as AI, however taking a ‘digital first’ approach as standard risks excluding certain segments of the population, creating new, or further entrenching existing health inequalities. 

The implementation of AI-driven healthcare systems that rely on patients accessing services digitally presents a complex challenge in terms of accessibility. Several vulnerable groups face potential exclusion: 

• People with learning disabilities and/or autistic people who may struggle with digital interfaces.
• Individuals with low literacy levels or people who are digitally excluded.
• People with sight loss who may find using digital tools challenging.

More concerning is the potential impact on patients with complex needs. Older adults and those with multiple health conditions often require an integrated, holistic approach to care that may be compromised by AI systems designed and trained to address individual symptoms rather than considering the whole person and how their complex needs interact with each other. 

Geographic and socioeconomic disparities

A troubling pattern is emerging at Integrated Care System (ICS) level, where AI implementation appears to be creating geographical disparities when it comes to healthcare access. The seemingly random distribution of sites piloting AI interventions, often determined by existing relationships between AI service providers and specific NHS Trusts, risks deepening existing healthcare inequalities. 

Research shows that patients in more deprived areas already benefit less from healthcare improvements and are more quickly affected by service restrictions. The introduction of AI services without strategic planning could exacerbate these disparities. 

Representation and bias in AI development

Another way in which AI could potentially exacerbate inequalities stems from the lack of representation of diverse communities both in the data used to train models and in the design of health and care services that harness the power of AI. 

A critical concern lies in the development and training of AI systems. Our research, which explores barriers to participation in clinical trials demonstrates that women, and people belonging to ethnically diverse communities are currently underrepresented in clinical trials at present.  

If this lack of representation is replicated in the data feeding AI models it could lead to AI tools that fail to address the needs of large segments of the population. 

Furthermore, co-production is a key guiding principle at National Voices, and we advocate for designing and delivering services with patients who are most likely to experience health inequalities to ensure that they meet the needs of everyone – not just the ‘average’ person. Currently, there’s limited evidence of meaningful patient engagement in AI healthcare development. 

Promising opportunities

Despite these challenges, we acknowledge that AI presents several exciting opportunities for healthcare improvement: 

Real-time diagnostic validation

We’re seeing exciting research into developing AI programmes that can immediately validate medical scans, eliminating the need for return visits. This stands to particularly benefit people who face barriers to healthcare such as unpaid time off work, travel costs, or childcare arrangements. 

Administrative efficiency

AI-driven streamlining of administrative tasks has the potential to increase the time clinicians and other professionals can spend with patients, improving both the quality of care and patient satisfaction. This increase in capacity also brings with it the opportunity to spend time resting, or on training and development.

Our recent research with eight different community groups highlighted persistent challenges with interpreter services. AI could revolutionise how we manage patient communication preferences and allocate human interpreters where they are needed, ensuring better informed consent and patient understanding, as well as that extra layer of reassurance for people before receiving care. This research also underlined the importance of highly trained and culturally sensitive human interpreters, who cannot and should not be replaced by technological solutions. 

The way forward

To conclude, our concerns about the increased use of AI in healthcare are about exacerbating the inequalities our members see, and the people and communities they serve experience on a daily basis. The opportunities AI has to offer are around getting the basics right in the medium-term, and in the longer-term shifting the power back to patients and communities. To maximise the potential of these opportunities, the incoming NHS 10 Year Plan must develop and maintain public trust in the use of AI as part of the shift from analogue to digital. Building patient trust in technologically-enabled solutions will be challenging when the most basic expectations of an effective health and care system are currently not being met.

Successfully implementing AI in healthcare requires a balanced approach that: 

• Proactively prioritises reducing health inequalities.
• Maintains and facilitates increased essential human connections in healthcare delivery.
• Ensures diverse representation of patient voice in AI development and testing.
• Implements systematic needs-based approaches to AI adoption within the NHS.
• Involves patients, communities and voluntary and community sector organisations and design and implementation decisions.

As we navigate this exciting technological transformation, our focus must remain on empowering patients whilst ensuring that new innovations serve all members of society, not just those who are easiest to reach. 

Get in touch

I am really keen to speak to anyone working in this space, whether you are considering the impact of AI on patient groups; involved in the design and delivery of AI interventions; or considering how best to engage with patients around this topic – please do get in touch! My email address is louis.horsley@nationalvoices.org.uk.