Towards Safer Large Language Models

The astonishing performance of large language models (LLMs) for many tasks has led to an exponential increase in real-world applications. They are reshaping the way society and industry interact with computer systems and textual information, as well as expanding our creativity. These characteristics, along with their flexibility in processing and interpreting natural language, let them accomplish a diverse range of tasks, such as generating textual content, powering virtual assistants and providing insights from large-scale data analysis.

However, as we explore the many benefits that LLMs offer, we must also tread cautiously, acknowledging their limitations and considering possible mitigation strategies. We need to be able to identify suitable LLM use cases and deploy strategies to reduce risk, such as prompting an LLM to explain its reasoning path in natural language and designing systems that keep people in control. Managed carefully, we can benefit from the great potential of LLMs while minimizing their risks.

LLMs are a family of neural network models for text processing generally based on networks that implement the Transformer architecture.¹ Unlike past language models trained on task-specific, labeled datasets, LLMs are trained using unsupervised learning on massive amounts of data.

The training objective of an LLM is to predict correct output based on an input prompt. The simplicity of the objective and the ability to learn from unlabeled data means these models can be scaled to ingest massive amounts of data. This training regime unlocks the model’s ability to solve previously unknown tasks and acquire emergent behaviors.^2,3 For instance, LLMs can answer questions, generate stories, extract information and summarize text. They can also reason, solve problems, generate creative text and be adapted to different domains.⁴

LLMs show significant capabilities in many tasks, often outperforming more narrowly focused natural language processing (NLP) or machine learning models. LLMs can also be used as-is for target tasks with no need for modification, further training or domain-specific datasets, which is unleashing an unprecedented pace of development and innovation in machine learning applications. LLM capabilities include:

Responding fluently – LLMs demonstrate great versatility in generating fluent text in different styles and contexts and are able to switch from colloquial prose to poetry to formal domain-specific writing, such as text for legal documents. Combined with a seamless ability to handle text in multiple languages, this capability of LLMs unlocks a universal text interface for any system’s input and output.

Inferring tasks – Following instructions, LLMs can often perform a task without referring to examples, sometimes referred to as zero-shot learning. If more instruction is required to teach an LLM how to perform a task, examples can be provided in the LLM text prompt field and the LLM can extract the target task from this. This capability is called few-shot learning and is particularly useful when a verbal description of the task might be ambiguous or otherwise difficult for the user.

Producing code – Because LLMs are able to process programming languages as well as natural languages, they demonstrate other interesting capabilities: generating working programs and programming code, understanding and explaining source code, fixing bugs in code snippets and translating across programming languages and software libraries.

While LLMs have great potential, they also have significant limitations. For instance, LLMs might generate incorrect output, a phenomenon sometimes called hallucinations.^5,6 In these cases, answers are imprecise, incorrect or even completely made up. 

Hallucinations are particularly dangerous when complex answers mix facts and false information. The user might also trust the response, falling victim to the argument from authority fallacy. Other key limitations include:

Lack of common sense – Although they excel at generating human-like text, LLMs often lack common-sense understanding. Given an input, they are trained to predict the next words based on statistical patterns in their training data. This can lead to factual inaccuracies and illogical responses. For this reason, LLMs have also been called stochastic parrots.⁷ Complex reasoning tasks where LLMs might fail include topics such as multistep, arithmetic, social, temporal or multimodal reasoning (due to a lack of understanding of the physical world).⁸

Biased training data – Prone to inheriting biases from their training data, LLMs may perpetuate or even amplify societal biases and stereotypes. This can affect the way LLMs generate text and make decisions. For example, many LLMs are predominantly trained on English data and, therefore, likely to produce outputs conforming to the culture of English-speaking countries.

Similarly, if an LLM is trained on social media data, it may exhibit bias or discriminatory views present in that training data. Addressing LLM bias is a significant challenge.⁹ For example, it requires careful curation of training data and ongoing monitoring to mitigate unintended consequences.

Harmful information – One under investigated issue is the hidden and harmful capabilities LLMs might have. For a start, it’s not fully known how safe the documents are on which these models have been trained. While LLMs often refuse to reveal what harmful information they know when prompted, adversarial prompting has been shown to succeed in causing them to reveal harmful information, such as instructions about how to steal someone’s identity or build an explosive device.¹⁰

Given the advantages and limitations of current LLMs, two key questions are, “What are good application areas?” and “What can we do to facilitate the safer use of LLMs?” To answer these questions, several approaches are available:

Classifying risk – Choosing the right strategy to reduce the risk of using LLMs inappropriately should begin with a use-case risk assessment. Ask yourself the question, “How much risk is there if I use this LLM for that task?” For example, there are no serious consequences if you use an LLM to recommend a book that doesn’t exist. In contrast, using an LLM to generate a medical report for a patient is high risk. If the medical report contains a hallucination, the doctor might choose the wrong treatment or even a dangerous one.

To assess risk, we can employ the risk definition outlined by the European Union’s Artificial Intelligence Act¹¹ (see Figure 1). The Act identifies four risk categories: Minimal Risk – users must be informed about AI usage and have the option to opt out; Limited Risk – transparency is required; High Risk – a conformity assessment should be run before AI can be deployed within the EU; Unacceptable Risk – AI may not be used.

Assessing accuracy – While LLMs generate text, the information they produce may be incorrect, making them useful in applications where strict correctness is not an issue, for example, when writing fiction. They can also be of value when correctness is important, if their responses are considered carefully by humans.

It is not uncommon for mistakes to occur during tasks being carried out by human experts. To avoid these, intermediate and time-consuming human reviews are often carried out. To reduce the time needed for these, LLM-based machine learning applications can undertake initial text generation tasks and seamlessly coordinate these with human experts as part of task workflows.

Identifying errors – One way to reduce the risk of overlooking incorrect LLM output is to explicitly ask the LLM to provide a natural language explanation behind its reasoning. For example, if we ask an LLM to solve a text-based math problem with multiple steps, it might become confused and provide an incorrect answer. As humans, we have no way of ascertaining why the LLM provided its answer without running the calculation ourselves, which defeats the purpose of asking the LLM!

This type of prompting aids the LLM in producing the correct answer as well as providing the reasoning path that led to it. However, there is one potential drawback with this approach: The engineering technique used to prompt the model is currently very brittle; adding a certain phrase cannot guarantee a correct answer and changing the phrase slightly could revert to an incorrect result.

Implementing controls – As it is impossible to ensure that LLMs work 100% correctly, what can be done to increase their safe use? We can facilitate this by designing applications that include human-centric LLMs. This increases the control humans have when using these applications.¹²

Typically, the amount of automation a computer application provides is seen on a single axis ranging from low to high. But this can be extended with an additional axis that embodies the amount of control we give to a user¹³ (see Figure 2).

Applying this paradigm to LLMs¹⁴ leads to three possible scenarios. First, without LLMs, humans are in full control of writing texts. Second, when LLMs generate text that is blindly trusted by the human user, there is a loss of control and the user is exposed to the dangers of LLMs. We can mitigate this with the third scenario: offering humans the right tools to restore control through innovative technologies.

To help ensure that correct LLM output is provided, NEC Laboratories Europe has created the Safe LLM Framework that consists of two technologies – LLM Explainer and Fact Checker. Developed by NEC Laboratories Europe, these machine learning models verify LLM output and enable the safer use of LLMs, minimizing the risk of hallucinations, falsehoods, biases and other problems that can affect output quality.