Verbalizable Representations Form a Global Workspace in Language Models

Verbalizable Representations Form a Global Workspace in Language Models

Introduction

If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of processing takes place without our knowledge: our visual systems parsing the contours of a face, our motor circuits maintaining our posture. At any given moment, only a small fraction of this neural activity is accessible to us. Yet it is this privileged sliver of activity that we rely on to reason deliberately: to plan what ingredients to buy for a recipe, or to puzzle out why an engine won’t start. Such thoughts can be articulated out loud, deliberately held in mind, and brought to bear on whatever task the moment demands. This distinction, between our accessible thoughts and our unconscious processing, is perhaps the most striking feature of human cognition.

In this paper, we present evidence that an analogous functional distinction has emerged in modern AI models. Specifically, we observe that language models maintain a privileged set of internal representations, available for report, modulation, and flexible internal reasoning, atop a much larger volume of automatic processing. We identify these representations using a new interpretability technique, which surfaces the concepts a model is poised to verbalize at any point in its processing. Measuring and intervening on these representations provides us a window into a model’s thought processes, uncovering internal reasoning and reactions that do not appear in its output.

Motivation: conscious access and the global workspace

The phenomenon described above is sometimes referred to as access consciousness: out of everything the brain processes, only a subset is consciously accessible, in the sense of being poised for use in reasoning and in the direct control of action and speech . Note that access consciousness is a purely functional notion; the relationship that it has with subjective experience (sometimes called phenomenal consciousness) is widely debated. In this paper, we take no position on this issue, and instead focus on the functional role played by consciously accessible information. How is it represented or processed differently from other information? Which mental faculties rely on it, and which do not?

Several functional properties are commonly held to distinguish consciously accessible information from unconscious processing. This information is typically reportable, in the sense that it can be put into words on request; indeed, verbal report has often served as a primary empirical signature of conscious access . It is subject to top-down control: a concept can be deliberately summoned, held in mind, and dismissed . It is the medium of deliberate reasoning: the effortful, step-by-step chaining of one thought to the next . It permits flexible generalization: the same content can be routed to whatever operation the current task demands and recombined with other accessible contents in novel configurations . And it is selective: only a small fraction of the brain’s ongoing processing is accessible in this way at any moment, with the bulk of perceptual, motor, and linguistic computation proceeding automatically, without the involvement of conscious access .

One influential proposal in neuroscience, the global workspace theory, grounds these functional properties in architectural and computational features of the brain . In this account, the brain is composed of many specialized processors operating largely in parallel and in isolation, whose activity proceeds outside of conscious access. A representation becomes consciously accessible when it is posted to a shared “global workspace” from which many downstream processes can read . Under the theory, the workspace is a processing hub that integrates and broadcasts information, allowing it to be used for flexible internal reasoning and report . Notably, the workspace is held to be limited in capacity, so entry is competitive and subject to attentional modulation, and the contents of the workspace at any moment are a small selection from the brain’s ongoing activity . While the global workspace model is not universally accepted, and there exist other theories that explain conscious access in different ways (??), we find it a useful comparison point to ground our investigations in language models.

A global workspace in language models

Modern large language models (LLMs) are known to perform sophisticated, multi-step internal computations in order to select their actions . As part of their internal processing, might LLMs have developed a global workspace of their own, to serve a functional role analogous to conscious access? It is not obvious that they should; in the brain, the workspace is closely associated with recurrent dynamics and brain region interactions that have no direct analog in the transformer architecture on which LLMs are based. On the other hand, maintaining a global workspace is likely computationally useful: a common representational format allows intermediate results to be written once and read by many neural processes. A language model that must chain reasoning steps, apply general operations in arbitrary contexts, and answer questions about its own processing also stands to benefit from this organization. Even if the implementations differ, it is natural to ask whether the functional properties associated with the global workspace have emerged in LLMs.

What would it mean for an LLM to have a global workspace? LLMs represent internal states as high-dimensional vectors, which are composed of more primitive vector representations of specific concepts. These representations encode diverse kinds of information, ranging from low-level bookkeeping—the part of speech of the present word , or the length in characters of a line of text —to higher-level abstractions like entities (e.g. the Golden Gate Bridge ), psychological states (e.g. desperation ), and situational knowledge (e.g. the awareness of being in an evaluation ). If language models possess anything like a global workspace, we…