In May 2024, at the International Conference on Learning Representations, a study will be officially introduced, contributing a new piece to the extensive puzzle of understanding the workings of large-scale linguistic models.

Large Language Models (LLMs), or simply “large-scale linguistic models” belong to a domain within artificial intelligence research focused on developing systems capable of generating written texts from a precise linguistic input. These texts find their applications in automatic translations, responses to customer support questions, or user queries in online searches, to name the most prominent uses. A recurrent example when discussing LLM is OpenAI’s ChatGPT chatbot, now in its fourth iteration.

At the heart of Large Language Models lies a neural network architecture called “transformer.” This was first introduced by a research group at Google Brain and Google Research in a paper, “Attention Is All You Need“, presented during the 2017 Neural Information Processing Systems (NeurIPS) international conference. In describing it, the team referred to it as «a new, simpler network architecture based solely on attention mechanisms, completely eliminating recurrence and convolutions» typical of complex neural networks, speculating seven years ago that «some automatic translation experiments demonstrate that transformer models are of superior quality, while being more parallelizable and requiring much less time for training».

In particular, when analyzing a linguistic input, the attention mechanism of the transformer model discussed by researchers does not focus on individual words in the text but rather on the structure of the sentence in which they are embedded, capturing their relationships and context [source: “Transformers and Large Language Models” – Stanford University].

---

TAKEAWAYS

---

Transformer models and the concept of “entity relationships”

Revisiting the subject to delve into its key meanings, a collaborative study conducted by the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology, Northeastern University, the Israel Institute of Technology, and Harvard University – “Linearity of Relation Decoding in Transformer Language Models” – has been published among the papers of the International Conference on Learning Representations (ICLR), taking place in Vienna from the 7th to the 11th of May 2024.

At the heart of the research is the analysis of the relationships between various elements that constitute the linguistic information decoded by transformer models, namely – as previously mentioned – the network architecture of Large Language Models.

The explicit question initiating the proceedings concerns how transformer models represent “entity relationships” and how they retrieve them when needed, that is, when called upon to decode new incoming texts.

The information contained in the input texts, which is progressively decoded and, over time, stored by the models – the authors explain – is of various natures. For instance, it may pertain to “world facts” (the election of a president, the moon landing, the story of a figure) or knowledge stemming from «common sense associations» like “lifeguards work on beaches” or “surgeons operate in operating theatres”.

Both facts and knowledge encompass a series of relationships between different elements (the “entities“), which can be «properties or mere lexical elements».

In focusing on entity relationships, transformer models analyse a fact expressed within a written text (“Louis Armstrong was a trumpeter”) as a relationship that connects a subject entity (Louis Armstrong) with an object entity (the trumpet). Similar facts (“Jimi Hendrix was a guitarist”) can be represented through the same type of relationship, namely “Jimi Hendrix (subject entity) plays the guitar (object entity)”.

Large Language Models: what mechanism for retrieving stored facts?

Regarding Large Language Models and the retrieval of already archived information, prior research – the team notes – has shown that questions within incoming texts (“Who was Louis Armstrong?”) «serve as their own keys for accessing facts stored in memory: once an input text references a particular subject, transformer models build representations of the relationships among entities related to that subject from earlier data». It is through this inferential process that the neural network of LLMs acquires the ability to respondappropriately to each inquiry.

To clarify, if the model is queried about the instrument played by Jimi Hendrix, it should identify the instrument as “guitar” rather than “piano,” by retrieving the specific relationship between subject entity and object entity present in information on Hendrix within texts previously decoded by the neural network.

However, “Inspecting and Editing Knowledge Representations in Language Models” – a study from the MIT Artificial Intelligence Laboratory dating back to May 2023 – to better frame the process aimed at retrieving specific previously decoded information, cautions against confusing the facts that simply come from the dataset used for training the neural network (where, for example, a representation of the word “trumpet” encodes the fact that trumpets are musical instruments) with those meaning-laden facts that actually derive from incoming texts, where a sentence like “I poured all the milk from the jug” is linked to already decoded information, in which the representation of the relationships between entities suggests that “the jug has been emptied”.

The study from October 2023, “Dissecting Recall of Factual Associations in Auto-Regressive Language Models“, involving Google DeepMind, Google Research, and Tel Aviv University, revisits the thesis on attention mechanisms developed by the pioneers of transformer models and contends that the retrieval of specific facts already decoded by the models occurs through «a distribution over multiple levels of attention». But through which calculations? This is the question that, to this day, remains open.

Specific linear functions for each type of information to be retrieved

Retrieving the precise information that “Louis Armstrong played the trumpet”, extracting it from all texts containing other data – long stored – related to the same subject, in order to generate a written text in response to an incoming linguistic stimulus, underlies a complex mechanism, still not thoroughly explored, which the authors of the aforementioned paper, to be presented in May 2024 at the International Conference on Learning Representations, aimed to elucidate. They developed an LLM with an artificial neural network featuring transformer architecture and then tested it through a series of experiments.

In detail, they discovered that Large Language Models, through transformer models and by utilising relationships between entities, «decode information using a simple linear function, and each of these functions is specific to the type of fact or knowledge to be retrieved».

This means the neural network will use a different decoding function depending on whether it needs to retrieve information about, for example, the instrument played by a famous musician or to answer a question about the birthplace of that same person.

With the goal of investigating the number of these different functions, the research team developed a bespoke method that led to the calculation of decoding functions for no fewer than forty-seven types of entity relationships, in response to incoming stimuli like “American football championship winner”, “capital of a state”, “current president of a given country”, and so on.

As for the composition of the dataset used to train the neural network, it encompassed forty-seven types of entity relationships, which were further categorized into four distinct groups of information that had already been decoded: these groups pertained to factual knowledge about the world, insights derived from common-sense connections, linguistic elements, and biases.

Each function was tested by altering the object each time, to verify if it could always retrieve accurate information about that specific object: for instance, the function for “capital of a state” should retrieve Athens if the subject is Greece and Rome if the subject is Italy, and not vice versa. Indeed, during the experiments – the researchers report – the functions retrieved accurate information in more than 60 percent of the cases.

Glimpses of Futures

In the future, the authors of the study announce, another intriguing avenue of research could be the examination of textual information not correctly archived by Large Language Models, which are guilty of compromising the entire dynamic upon which the retrieval of already decoded and stored facts rests, and thus, the full understanding of incoming linguistic inputs, from which to generate coherent written texts. The aim is singular: to prevent transformer models from producing texts with erroneous content and – an even more serious risk – falsehoods.

Now, employing the STEPS matrix, they aim to anticipate possible future scenarios, analysing the impacts that the evolution of research into the processes guiding the retrieval of information already decoded by large-scale linguistic models might have from social, technological, economic, political, and sustainability perspectives.

S – SOCIAL: refining the investigation into the mechanisms of retrieving already decoded textual information – upon which to decode incoming data more accurately and create increasingly precise outputs – by artificial intelligence systems like chatbots, tasked with holding complex written conversations and responding to all manner of user queries, will mean, in the future, being able to rely on machines that are even more reliable in terms of performance and safer in terms of interaction with humans. This contributes to lowering the risk level posed by the potential generation of texts with content that is not true to reality – and, therefore, potentially manipulative – extending to those that are discriminatory, offensive, and harmful to personal dignity.

T – TECHNOLOGICAL: in the future, the designated research trajectory must inevitably delve deeper into the structure characterising the unique network architecture of Large Language Models and their attention mechanisms. More precisely, the development of methods and techniques for information retrieval will need to address the challenges posed by ever-larger models and, hence, an increased influx of linguistic inputs. It will be intriguing to investigate the operation of specific linear decoding functions for each type of fact and knowledge to be recovered and, importantly, to ascertain their number in the context of highly expansive linguistic models, thereby addressing a broader array of entity relationships.

E – ECONOMIC: beyond exploring the retrieval processes of certain textual data already stored by LLMs, the research team has also developed a tool they’ve termed the “attribute lens.” This tool will allow developers and producers of AI systems that generate texts from linguistic inputs to visualise, within the multiple layers of the transformer neural network, where and when past facts and knowledge are stored. The aim is corrective: with the attribute lens, those working in the Large Language Model sector will be able to manage and correct the vast amount of information archived by the model during experimentation, ensuring that text generators with inaccurate, incorrect, misleading, or unethical content are not released onto the market. This aligns with the foundational principles of the EU AI Act, approved on 13 March 2024 by the European Council, ensuring reliable, safe artificial intelligence that respects fundamental rights and ethical principles.

P – POLITICAL: chatbots are among the AI systems classified as “limited risk” by the aforementioned EU AI Act (or Artificial Intelligence Act), as their use does not pose a danger requiring them to be subject to stricter regulatory constraints, unlike, for example, tools used in personnel selection or in assessing candidates’ eligibility for certain benefits or services. However, a steadfast requirement of AI Act is “transparency” towards users of written text generators, who must always be informed by the producing companies about the characteristics, functions, and applications of such systems.

S – SUSTAINABILITY: from the standpoint of environmental sustainability, all processes underpinning the functionalities of Large Language Models, encompassing those tied to the retrieval of previously decoded information, as well as those pertaining to the implementation, scalability, and maintenance of the models, necessitate millions of processing hours. These processes result in the emission of significant amounts of CO2. This challenge is prevalent across all artificial intelligence methodologies and, more broadly, affects the entire digital realm, where the carbon footprint constitutes an often overlooked and underreported issue. Moving forward, it is imperative for various industry segments to undertake more empirical research into the energy consumption of different AI systems, thereby enabling more precise comparisons of their carbon emissions.