FILE ON LLMSEntity
LLMs
introspective behavior is narrow and task-dependent1. LLM introspection findings are very recent and methodologically immature2.
DEVELOPING FILE
This file has 3 sources on record so far more may surface as the record grows.
On the record see profile →
introspective behavior is narrow and task-dependent
The Examined Cognition: Substrates of Metacognition and Meth
“finds that introspective behavior is "narrow and task-dependent, with models failing at naive self-report while succeeding on" structured perturbation tasks”
VERIFIED — a word-for-word match in the stored source
LLM introspection findings are very recent and methodologically immature
The Examined Cognition: Substrates of Metacognition and Meth
“The LLM introspection findings are very recent and methodologically immature:”
VERIFIED — a word-for-word match in the stored source
Disputed — who says what, against what see standalone →
These are things people say about LLMs that aren't settled fact — each one attributed to who's actually claiming it, so a claim never quietly passes as established just because it showed up in a document. Weigh it yourself.
Disputed
"the temptation to declare them entirely disanalogous should also be resisted"
Unattributed assertion · stance: pro
The evidence on file mostly supports this, but it's still someone's claim, not a settled fact.
The Examined Cognition: Substrates of Metacognition and Meth ↗
Disputed
"the temptation to map LLM introspection one-to-one onto human introspection should be resisted"
Unattributed assertion · stance: pro
The evidence on file mostly supports this, but it's still someone's claim, not a settled fact.
The Examined Cognition: Substrates of Metacognition and Meth ↗
People
Sartreother
Thomas Metzingerother
McLuhanother
Jack Lindseyother