Research from Michael Lones, a professor at Heriot-Watt University, warns that incorporating generative AI into machine learning systems could significantly heighten risks such as cyber-attacks and data breaches, while also perpetuating bias against underrepresented groups.
The study, published in the journal Patterns, outlines the unintended harms associated with generative AI despite its potential cost and efficiency benefits. Lones emphasizes the necessity for developers to balance enhancements in capability with the possible dangers involved.
Lones stated, “Machine learning developers need to be aware of the risks of using Gen AI in machine learning and find a sensible balance between improvements in capability and the risks that might come with that.” He cautioned that just because a capability exists does not imply it should be used.
Generative AI is being increasingly utilized across sectors to design, build, and operate machine learning systems. Lones identified four primary applications of generative AI in this context: as components in machine learning pipelines, for designing and coding those pipelines, synthesizing training data, and analyzing outputs. However, each application carries inherent risks.
He noted that the use of large language models (LLMs) for multiple tasks or in autonomous roles increases these risks. Inaccuracies and fabricated information may arise from LLMs, complicating the evaluation of their performance. Lones pointed out the challenges posed by LLMs’ non-transparent operations, especially in sectors like medicine and finance that require reliable and explainable decision-making.
As Lones explained, “In areas like medicine or finance, there are laws about being able to show that the machine learning system is reliable, and that you can explain how it reaches decisions. As soon as you start using LLMs, that gets really hard, because they’re so opaque.” He warned that while companies may utilize generative AI to enhance user experiences and cut costs, such deployments may lead to negative outcomes, including bias and unfairness.
