Newest Papers
How to Set the Batch Size for Large-Scale Pre-training?
Yunhua Zhou, Junhao Huang, Shuhao Xin, Yechen Zhang, Runyu Peng, Qiping Guo, Xipeng Qiu
arXiv·2026
The concept of Critical Batch Size, as pioneered by OpenAI, has long served as a foundational principle for large-scale pre-training. However, with the paradigm shift towards the Warmup-Stable-Decay (WSD) learning rate scheduler, we observe that the original theoretical framework and its underlying mechanisms fail to align with new pre-training dynamics. To bridge this gap between theory and practice, this paper derives a revised E(S) relationship tailored for WSD scheduler, characterizing the trade-off between training data consumption E and steps S during pre-training. Our theoretical analysis reveals two fundamental properties of WSD-based pre-training: 1) B_min, the minimum batch size threshold required to achieve a target loss, and 2) B_opt, the optimal batch size that maximizes data efficiency by minimizing total tokens. Building upon these properties, we propose a dynamic Batch Size Scheduler. Extensive experiments demonstrate that our revised formula precisely captures the dynamics of large-scale pre-training, and the resulting scheduling strategy significantly enhances both training efficiency and final model quality.
No ratings yet
View paper →The Squirrel Parser: A Linear-Time PEG Packrat Parser Capable of Left Recursion and Optimal Error Recovery
Luke A. D. Hutchison
arXiv·2026
We present the squirrel parser, a PEG packrat parser that directly handles all forms of left recursion with optimal error recovery, while maintaining linear time complexity in the length of the input even in the presence of an arbitrary number of errors. Traditional approaches to handling left recursion in a recursive descent parser require grammar rewriting or complex algorithmic extensions. We derive a minimal algorithm from first principles: cycle detection via per-position state tracking and $O(1)$-per-LR-cycle communication from descendant to ancestor recursion frames, and fixed-point search via iterative expansion. For error recovery, we derived a set of four axioms and twelve constraints that must be imposed upon an optimal error recovery design to ensure completeness, correctness, optimality of performance, and intuitiveness of behavior. We utilized a constraint satisfaction mechanism to search the space of all possibilities, arriving at a provably optimal and robust error recovery strategy that maintains perfect performance linearity.
No ratings yet
View paper →The tragic life and principled politics of Trần Đức Thảo | Aeon Essays
Rory O’Sullivan
aeon.co·2026
How the persecuted Vietnamese philosopher became one of the first theorists of the divide between colonised and coloniser
No ratings yet
View paper →QCD Crossover at Low Temperatures from Lee-Yang Edge Singularity
D. A. Clarke, H. -T. Ding, J. -B. Gu, S. -T. Li, Swagato Mukherjee, P. Petreczky, C. Schmidt, H. -T. Shu, K. -F. Ye
arXiv·2026
We provide the first lattice-QCD estimate of the crossover line down to $T\simeq108$~MeV. We introduce a new method that combines the Lee-Yang edge in the complex plane of baryon chemical potential $μ_B$ with universal chiral scaling to determine the $μ_B$ dependence of the QCD chiral critical and pseudo-critical temperatures. By performing $(2\!+\!1)$-flavor lattice QCD simulations at $T\simeq108$~MeV and purely imaginary $μ_B$ with a single lattice spacing and two volumes, we compute $μ_B$-dependent baryon-number susceptibilities and extract the location of the Lee-Yang edge. Together with universal scaling near the QCD chiral transition, it constrains the mapping function between $\{T,μ_B\}$ and the scaling variable (\textit{i.e.}\ the argument of the universal scaling functions). This mapping function then yields the $μ_B$ dependence of the critical and pseudo-critical temperatures for $T\gtrsim108$~MeV. While our calculation is performed only at a single value of low temperature without explicit input from small-$μ_B$ expansion, the resulting $μ_B$ dependence of the pseudo-critical temperature is consistent with established lattice-QCD determinations at small $μ_B$ and compatible with chemical freeze-out parameters of heavy-ion collisions down to low temperatures, demonstrating the validity and robustness of the method. Application of this method can be systematically extended to additional temperatures and finer discretizations, opening a pathway to charting the QCD phase diagram in the low-$T$, high-$μ_B$ regime.
No ratings yet
View paper →Lenses for Partially-Specified States (Extended Version)
Kazutaka Matsuda, Minh Nguyen, Meng Wang
arXiv·2026
A bidirectional transformation is a pair of transformations satisfying certain well-behavedness properties: one maps source data into view data, and the other translates changes on the view back to the source. However, when multiple views share a source, an update on one view may affect the others, making it hard to maintain correspondence while preserving the user's update, especially when multiple views are changed at once. Ensuring these properties within a compositional framework is even more challenging. In this paper, we propose partial-state lenses, which allow source and view states to be partially specified to precisely represent the user's update intentions. These intentions are partially ordered, providing clear semantics for merging intentions of updates coming from multiple views and a refined notion of update preservation compatible with this merging. We formalize partial-state lenses, together with partial-specifiedness-aware well-behavedness that supports compositional reasoning and ensures update preservation. In addition, we demonstrate the utility of the proposed system through examples.
No ratings yet
View paper →Sharded Elimination and Combining for Highly-Efficient Concurrent Stacks
Ajay Singh, Nikos Metaxakis, Panagiota Fatourou
arXiv·2026
We present a new blocking linearizable stack implementation which utilizes sharding and fetch&increment to achieve significantly better performance than all existing concurrent stacks. The proposed implementation is based on a novel elimination mechanism and a new combining approach that are efficiently blended to gain high performance. Our implementation results in enhanced parallelism and low contention when accessing the shared stack. Experiments show that the proposed stack implementation outperforms all existing concurrent stacks by up to 2X in most workloads. It is particularly efficient in systems supporting a large number of threads and in high contention scenarios.
No ratings yet
View paper →Direct and Indirect Loop Equations in Lattice Yang-Mills Theory
Xizhe Liu, Gang Yang
arXiv·2026
The dynamics of Wilson loops is governed by an infinite set of Schwinger-Dyson equations and trace relations. In the context of the lattice positivity bootstrap, a central challenge is determining a dynamically independent basis of these operators within a truncated space. We present a systematic framework to solve this problem, utilizing a geometric plaquette-cut and subloop-cut strategy to efficiently generate all (local) direct equations. Furthermore, we identify and analyze ``indirect equations", which arise from the elimination of higher-length intermediate loops. We elucidate the origin of these subtle relations and propose a vertex-filtering strategy to construct them. Applying the above framework to SU(2) lattice Yang-Mills theory, we provide explicit counting of independent canonical loops and equations in 2, 3, and 4 dimensions, along with a statistical analysis of their asymptotic growth.
No ratings yet
View paper →Justin McDaniel Found a Way to Make Students Read Again
Lila Shapiro
Vulture·2026
Professor Justin McDaniel, the tenured chair of the religious-studies department at the University of Pennsylvania, has developed a cult following for getting his students to read novels — as long as they follow his rules.
No ratings yet
View paper →Mimicking opioid analgesia in cortical pain circuits
Corinna S. Oswell, Sophie A. Rogers, Justin G. James, Nora M. McCall, Alex I. Hsu, Gregory J. Salimando, Malaika Mahmood, Lisa M. Wooldridge, Meghan Wachira, Adrienne Y. Jo, Raquel Adaia Sandoval Ortega, Jessica A. Wojick, Katherine Beattie, Sofia A. Farinas, Samar N. Chehimi, Amrith Rodrigues, Jacqueline W. K. Wu, Lindsay L. Ejoh, Blake A. Kimmey, Emily Lo, Ghalia Azouz, Jose J. Vasquez, Matthew R. Banghart, Kevin T. Beier, Kate Townsend Creasy, Richard C. Crist, Charu Ramakrishnan, Benjamin C. Reiner, Karl Deisseroth, Eric A. Yttri, Gregory Corder
Nature·2026
No abstract available
No ratings yet
View paper →Examining the Link Between Critical Thinking, In-School Racial Discrimination, and Perceptions of Future College-Going Beliefs
Adrian Gale, Danielle R. Harrell, Edward D. Scott, James M. Ellis
Social Sciences·2026
<jats:p>Youth from racial minority backgrounds frequently report encountering racial discrimination within their school. These experiences can create additional barriers to their pursuit of higher education. This study explored whether critical thinking can protect and enhance youth college aspirations despite discrimination. We examined whether critical thinking protects against the impact of racial discrimination on beliefs about future college plans in a racially/ethnically diverse sample of 189 adolescents (Mage = 14.47, SD = 1.402, 66.8% female) using a cross-sectional approach. Results indicated that among adolescents with low and moderate critical thinking skills, higher levels of in-school racial discrimination were linked to lower beliefs in their future college attendance. This suggests that in environments where racial discrimination is prevalent, individuals with lower critical thinking skills are more susceptible to negative effects on their college expectations. Conversely, for adolescents with high critical thinking skills, the relationship between in-school racial discrimination and beliefs about college plans was not significant. These findings emphasize the importance of developing critical thinking skills through interventions and policies to mitigate the adverse effects of in-school racial discrimination and to promote college access and success among underrepresented racial minority youth.</jats:p>
No ratings yet
View paper →From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligence
Marc Finzi, Shikai Qiu, Yiding Jiang, Pavel Izmailov, J. Zico Kolter, Andrew Gordon Wilson
arXiv·2026
Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated without considering a downstream task? On these questions, Shannon information and Kolmogorov complexity come up nearly empty-handed, in part because they assume observers with unlimited computational capacity and fail to target the useful information content. In this work, we identify and exemplify three seeming paradoxes in information theory: (1) information cannot be increased by deterministic transformations; (2) information is independent of the order of data; (3) likelihood modeling is merely distribution matching. To shed light on the tension between these results and modern practice, and to quantify the value of data, we introduce epiplexity, a formalization of information capturing what computationally bounded observers can learn from data. Epiplexity captures the structural content in data while excluding time-bounded entropy, the random unpredictable content exemplified by pseudorandom number generators and chaotic dynamical systems. With these concepts, we demonstrate how information can be created with computation, how it depends on the ordering of the data, and how likelihood modeling can produce more complex programs than present in the data generating process itself. We also present practical procedures to estimate epiplexity which we show capture differences across data sources, track with downstream performance, and highlight dataset interventions that improve out-of-distribution generalization. In contrast to principles of model selection, epiplexity provides a theoretical foundation for data selection, guiding how to select, generate, or transform data for learning systems.
No ratings yet
View paper →When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability
Raphael Ronge, Markus Maier, Frederick Eberhardt
arXiv·2026
Recent work by Anthropic on Mechanistic interpretability claims to understand and control Large Language Models by extracting human-interpretable features from their neural activation patterns using sparse autoencoders (SAEs). If successful, this approach offers one of the most promising routes for human oversight in AI safety. We conduct an initial stress-test of these claims by replicating their main results with open-source SAEs for Llama 3.1. While we successfully reproduce basic feature extraction and steering capabilities, our investigation suggests that major caution is warranted regarding the generalizability of these claims. We find that feature steering exhibits substantial fragility, with sensitivity to layer selection, steering magnitude, and context. We observe non-standard activation behavior and demonstrate the difficulty to distinguish thematically similar features from one another. While SAE-based interpretability produces compelling demonstrations in selected cases, current methods often fall short of the systematic reliability required for safety-critical applications. This suggests a necessary shift in focus from prioritizing interpretability of internal representations toward reliable prediction and control of model output. Our work contributes to a more nuanced understanding of what mechanistic interpretability has achieved and highlights fundamental challenges for AI safety that remain unresolved.
No ratings yet
View paper →