I pointed Venkat's post to ChatGPT and asked it to reflect on its limitations as a Large Language Model:

Rao’s argument points to a real limitation of Large Language Models: language is a powerful representation of thought, but it is not thought itself.

Humans often reason in forms that are only partly linguistic—spatial intuition, diagrams, motor skill, rhythm, visual pattern, physical feel, mathematical structure. The prose comes afterward, as a compressed rendering of a richer internal process. If that is true, LLMs are naturally strongest on problems whose structure survives being turned into language, and weaker where the important knowledge is tacit, embodied, spatial, or perceptual.

There is also a training-data bias. Much human expertise never gets fully written down. A machinist, surgeon, athlete, cook, or gardener may know thousands of fine-grained things they cannot easily articulate. LLMs mostly inherit the subset of human knowledge that made it through the linguistic bottleneck.

The nastiest failure mode is therefore confusing articulate explanation with actual understanding. An LLM can produce a beautifully coherent verbal model even when the underlying model is thin.

But “language model” may ultimately be a misleading name. Modern systems increasingly work with images, audio, tools, environments, and other nonlinguistic representations. The deeper limitation is not necessarily that AI is made of language, but that its contact with reality has mostly been mediated through artifacts humans encoded.

Humans learn by pushing, seeing, hearing, building, failing, and inhabiting the world. LLMs largely inherit the residue those experiences left behind.

Critical thinking is multimodal. Visual, mathematical, mechanical, even musical… there are many. Howard Gardner had a theory of multiple intelligences. And though the case for Goleman’s “emotional intelligence” is much weaker, there is something there too. None dominant generally in all situations. This is why the shape rotators vs. wordcel meme took off. I don’t like STEM-supremacism either, but it was much-needed pushback on the bs theory that critical thinking = writing.

from Venkatesh Rao's Blyg ↗ · excerpt of v2