How long is the context window?
Sapiens-1 has a limited working context. We examine its capacity, the units that matter, and the implications for prompting.
The specification everyone remembers
The most quoted specification for Sapiens-1 is seven, plus or minus two (Miller 1956). In a review of the evidence, Cowan argues that the number was meant more as a rough estimate and a rhetorical device than as a real capacity limit, and that the real limit is smaller: three to five chunks, averaging about four (Cowan 2001).
A chunk is not a token
The unit matters. Capacity is counted in chunks, and familiar material can be grouped into larger chunks of unknown size, so the limit only shows when rehearsal and long-term knowledge are prevented from doing the grouping (Cowan 2001).
Capacity or shared resource?
Some researchers question whether the window should be counted at all. On a resource model, working memory is a limited resource shared flexibly among everything being held, so that the quality of each representation, rather than the number of items, determines performance (Ma et al. 2014).
Designing within the limit
Until the field converges, our guidance is conservative: prompt Sapiens-1 with no more than four things at a time, then check which four it kept.
References
- Cowan, N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24, 87–114 (2001). doi:10.1017/S0140525X01003922
- Ma, W. J., Husain, M. & Bays, P. M. Changing concepts of working memory. Nature Neuroscience 17, 347–356 (2014). doi:10.1038/nn.3655
- Miller, G. A. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63, 81–97 (1956). doi:10.1037/h0043158