Recovering Temporal and Geographic Signals from Language Model Embeddings

Summary

The paper investigates whether language-model embeddings contain structured information about time and geography. It introduces a simple projection-based method that uses a small set of seed examples to define an axis in embedding space, then ranks texts or entities by their position along that axis. The method is fully black-box and model-agnostic: it needs only output embeddings and does not require model weights, internal activations, auxiliary probes, or additional training. This allows analysis of embedding models that are accessible only through APIs. Applied to temporal and geographic datasets, the method recovers meaningful chronological and spatial structure. The findings provide evidence that output embeddings encode signals relevant to time and space, while also suggesting a lightweight interpretability tool. The same projections could support downstream tasks such as temporal ordering, geographic ranking, and tagging, although the abstract does not report broader implementation details or error analysis.