If it looks like a duck, swims like a duck, and quacks like a duck, then your LLM’s priors are likely to predict the next tokens to amount to the word “duck” based on its learned data distribution—but has it reasoned about its data to deduce the duck? Large language models (LLMs) produce remarkably fluent text, enough to result in widespread claims of their ability to “understand” and “reason”. However, a dissection of the key architectural features of LLMs, and more generally ANNs, in particular their reliance on probabilistic pattern-matching, exposes their absence of critical structures analogous to the neuro-biological substrates known to be involved in human reasoning, goal-directed behavior, and cumulative learning. Furthermore, LLMs lack mechanisms to perform explicit goal-driven cause-effect guided use of deduction, induction, abduction and analogy; if a context requires an unseen and unlikely output \((x^*)\) not supported in the training-data manifold  \(\mathcal {M}\) (i.e. outside the convex hull of what was seen during training), the model has no basis for producing an answer corresponding to the physical world, being instead limited to interpolate on \(\mathcal {M}\) , from which next-token predictions are drawn via weighted sums over attention heads. Our formalism suggests that token-level statistical interpolation already suffices for the observed behavior; explicit internal reasoning modules are therefore not required to explain output. Consequently, we argue that claims attributing human-like cognition to contemporary LLMs are empirically unsupported, confusing surface fluency with cognitive processes in what essentially are two levels of the same misattribution: (i) Artificial Dualism: researchers project hidden reasoning modules into purely statistical models; (ii) Turing Trap: observers project agency from fluent dialogue alone.