Oldest Papers
On Evaluating Cognitive Capabilities in Machines (and Other "Alien" Intelligences)
Melanie Mitchell
Substack·2026
At NeurIPS 2025
I had the great honor of being invited to give a keynote lecture at NeurIPS 20251, one of the most important annual AI/machine-learning conferences. The conference took place in San Diego, in the largest conference center I have ever seen. There were close to 30,000 people registered, and walking through the halls reminded me of walking around New York City, where on the crowded sidewalks you pass by what seems to be an unending stream of humanity. But imagine if everyone you pass on the New York sidewalk were wearing a name tag, about half of them sporting affiliations of AI startup companies all named some-random-word.ai. That was NeurIPS. It was quite overwhelming.
No ratings yet
View paper →A new digital divide? Coder worldviews, the ‘Slop economy,’ and democracy in the age of AI
Jason Miklian, Kristian Hoelscher
Information, Communication & Society·2026
Digital technologies are transforming democratic life in conflicting ways. This article bridges two perspectives to unpack these tensions. First, we present an original survey of software developers in Silicon Valley, interrogating how coders’ worldviews, ethics, and workplace cultures shape the democratic potential and social impact of the technologies they build. Results indicate that while most developers recognize the power of their products to influence civil liberties and political discourse, they often face ethical dilemmas and top-down pressures that can lead to design choices undermining democratic ideals. Second, we critically investigate these findings in the context of an emerging ‘new digital divide’, not of internet access but of information quality. We interrogate the survey findings in the context of the ‘slop economy’, in which billions of users unable to pay for high-quality content experience an internet dominated by low-quality, AI-generated ad-driven content. We find a reinforcing cycle between tech creator beliefs and the digital ecosystems they spawn. We discuss implications for democratic governance, arguing for more ethically informed design and policy interventions to help bridge the digital divide to ensure that technological innovation supports rather than subverts democratic values in the next chapter of the digital age.
No ratings yet
View paper →Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models
Ni, Jingwei, Fadeeva, Ekaterina, Wu, Tianyi, Akhtar, Mubashara, Zhang, Jiaheng, Ash, Elliott, Leippold, Markus, Baldwin, Timothy, Ng, See-Kiong, Shelmanov, Artem, Sachan, Mrinmaya
arXiv·2026
LLMs can solve complex tasks by generating long, multi-step reasoning chains. Test-time scaling (TTS) can further improve LLM performance by sampling multiple variants of intermediate reasoning steps, verifying their correctness, and strategically choosing the best steps for continuation. However, existing verification approaches, such as Process Reward Models (PRMs), are computationally expensive, limited to specific domains, and require large-scale human or model-generated annotations. We propose a lightweight alternative for step-level reasoning verification based on probing the internal states of LLMs. We train a transformer-based probe that uses the internal states of the frozen LLM to estimate the credibility of its reasoning steps during generation. Annotation can be generated either by another larger LLM (e.g., DeepSeek-R1) or in a self-supervised manner by the original model itself. The probes are both effective and lightweight, containing fewer than 10M parameters. Across multiple domains, including mathematics, planning, and general knowledge question answering, our probes match or even exceed the performance of PRMs that are up to 810x larger. Our findings suggest that the internal states of LLMs encode their confidence in reasoning processes and can serve as reliable signals for reasoning step verification, offering a promising direction towards scalable and generalizable TTS and introspective LLMs.
No ratings yet
View paper →