Newest Papers
Understanding gas mixing in the circumgalactic medium
Hilay Shah, Freeke van de Voort, Amit Seta, Christoph Federrath
arXiv·2025
We study gas mixing in a simulated Milky Way-mass galaxy's circumgalactic medium (CGM) using cosmological `zoom-in' simulations. We insert tracer dyes in the CGM with different gas flows (shearing, coherent, and static) and diverse physical properties to track gas mixing. We correlate the extent and shape of the dye spread with the local gas properties to understand gas mixing. Velocity dispersion and traceless symmetric shear tensors (pure shear deformation) in small regions (<= 5 kpc) around the dye injection locations best predict the dye spread extent after 200 Myr. We use this to determine diffusion calibration constants for subgrid-scale mixing models. While the dye shape after 200 Myr aligns well with the velocity dispersion and magnetic field dispersion, the best alignment occurs with the dispersion of stretching eigenvectors (traceless symmetric shear tensor) and plane-of-rotation (antisymmetric shear or vorticity tensor) in large regions (10 kpc) around the dye injection locations. Therefore, shear statistics and velocity dispersion best predict the extent and shape of mixed gas. The linear temporal dependence of the dye spread suggests superdiffusion in the CGM, potentially due to turbulent and large-scale coherent flows or numerical diffusion. Despite significant numerical mixing from our 1 kpc resolution (insufficient to resolve Reynolds numbers ~10^2-10^3, which require a few hundred pc resolution), our correlation results are robust thanks to fixed spatial resolution throughout the CGM. These results can be used to predict diffusion coefficients to model magnetic field diffusion, heat transport, and metal mixing.
No ratings yet
View paper →Mathematical Modelling and Optimisation of Athletic Performance: Tapering and Periodisation
David Ceddia, Howard Bondell, Peter Taylor
arXiv·2025
We conduct a mathematical optimisation of the training load to maximise performance for two seminal athletic performance models: the Banister et al. 1975 Fitness-Fatigue Impulse Response Model and the Busso 2003 Variable Dose-Response Model. We discuss discrepancies in the general trends of the optimised training loads compared to common training practices recommended in the sports science literature, such as tapering and periodisation. We then propose a set of interpretable nonlinear modifications in the magnitude and time response to training in the fitness-fatigue model such that the optimised training load demonstrates these trends.
No ratings yet
View paper →Code Researcher: Deep Research Agent for Large Systems Code and Commit History
Ramneet Singh, Sathvik Joel, Abhav Mehrotra, Nalin Wadhwa, Ramakrishna B Bairi, Aditya Kanade, Nagarajan Natarajan
arXiv·2025
Large Language Model (LLM)-based coding agents have shown promising results on coding benchmarks, but their effectiveness on systems code remains underexplored. Due to the size and complexities of systems code, making changes to a systems codebase is a daunting task, even for humans. It requires researching about many pieces of context, derived from the large codebase and its massive commit history, before making changes. Inspired by the recent progress on deep research agents, we design the first deep research agent for code, called Code Researcher, and apply it to the problem of generating patches for mitigating crashes reported in systems code. Code Researcher performs multi-step reasoning about semantics, patterns, and commit history of code to gather sufficient context. The context is stored in a structured memory which is used for synthesizing a patch. We evaluate Code Researcher on kBenchSyz, a benchmark of Linux kernel crashes, and show that it significantly outperforms strong baselines, achieving a crash-resolution rate of 58%, compared to 37.5% by SWE-agent. On an average, Code Researcher explores 10 files in each trajectory whereas SWE-agent explores only 1.33 files, highlighting Code Researcher's ability to deeply explore the codebase. Through another experiment on an open-source multimedia software, we show the generalizability of Code Researcher. Our experiments highlight the importance of global context gathering and multi-faceted reasoning for large codebases.
No ratings yet
View paper →MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoning
Thang Nguyen, Peter Chin, Yu-Wing Tai
arXiv·2025
We present MA-RAG, a Multi-Agent framework for Retrieval-Augmented Generation (RAG) that addresses the inherent ambiguities and reasoning challenges in complex information-seeking tasks. Unlike conventional RAG methods that rely on end-to-end fine-tuning or isolated component enhancements, MA-RAG orchestrates a collaborative set of specialized AI agents: Planner, Step Definer, Extractor, and QA Agents, each responsible for a distinct stage of the RAG pipeline. By decomposing tasks into subtasks such as query disambiguation, evidence extraction, and answer synthesis, and enabling agents to communicate intermediate reasoning via chain-of-thought prompting, MA-RAG progressively refines retrieval and synthesis while maintaining modular interpretability. Extensive experiments on multi-hop and ambiguous QA benchmarks, including NQ, HotpotQA, 2WikimQA, and TriviaQA, demonstrate that MA-RAG significantly outperforms standalone LLMs and existing RAG methods across all model scales. Notably, even a small LLaMA3-8B model equipped with MA-RAG surpasses larger standalone LLMs, while larger variants (LLaMA3-70B and GPT-4o-mini) set new state-of-the-art results on challenging multi-hop datasets. Ablation studies reveal that both the planner and extractor agents are critical for multi-hop reasoning, and that high-capacity models are especially important for the QA agent to synthesize answers effectively. Beyond general-domain QA, MA-RAG generalizes to specialized domains such as medical QA, achieving competitive performance against domain-specific models without any domain-specific fine-tuning. Our results highlight the effectiveness of collaborative, modular reasoning in retrieval-augmented systems: MA-RAG not only improves answer accuracy and robustness but also provides interpretable intermediate reasoning steps, establishing a new paradigm for efficient and reliable multi-agent RAG.
No ratings yet
View paper →How to structure an effective campaign plan to win more customers
Dave Chaffey
Smart Insights·2025
6 essential parts of a campaign planning template using the RACE and SOSTAC® frameworks for campaign planning
★ 5.0 (1)
View paper →why some conversations rearrange your brain
maja
Substack·2025
how the right questions unlock hidden selves
No ratings yet
View paper →Next-token pretraining implies in-context learning
Paul M. Riechers, Henry R. Bigelow, Eric A. Alt, Adam Shai
arXiv·2025
We argue that in-context learning (ICL) predictably arises from standard self-supervised next-token pretraining, rather than being an exotic emergent property. This work establishes the foundational principles of this emergence by focusing on in-distribution ICL, demonstrating how models necessarily adapt to context when trained on token sequences, especially from non-ergodic sources. Our information-theoretic framework precisely predicts these in-distribution ICL dynamics (i.e., context-dependent loss reduction). We verify this with experiments using synthetic datasets of differing types of correlational structure, reproducing characteristic phenomena like phase transitions in training loss for induction head formation and power-law scaling of in-context loss. We further show that a model's in-context performance on any task is mathematically coupled to the ensemble of tasks seen in pretraining, offering a fundamental explanation, grounded in architecture- and modality-independent principles, for such inference-time learning.
No ratings yet
View paper →Data Mixing Can Induce Phase Transitions in Knowledge Acquisition
Xinran Gu, Kaifeng Lyu, Jiazheng Li, Jingzhao Zhang
arXiv·2025
Large Language Models (LLMs) are typically trained on data mixtures: most data come from web scrapes, while a small portion is curated from high-quality sources with dense domain-specific knowledge. In this paper, we show that when training LLMs on such data mixtures, knowledge acquisition from knowledge-dense datasets, unlike training exclusively on knowledge-dense data (arXiv:2404.05405), does not always follow a smooth scaling law but can exhibit phase transitions with respect to the mixing ratio and model size. Through controlled experiments on a synthetic biography dataset mixed with web-scraped data, we demonstrate that: (1) as we increase the model size to a critical value, the model suddenly transitions from memorizing very few to most of the biographies; (2) below a critical mixing ratio, the model memorizes almost nothing even with extensive training, but beyond this threshold, it rapidly memorizes more biographies. We attribute these phase transitions to a capacity allocation phenomenon: a model with bounded capacity must act like a knapsack problem solver to minimize the overall test loss, and the optimal allocation across datasets can change discontinuously as the model size or mixing ratio varies. We formalize this intuition in an information-theoretic framework and reveal that these phase transitions are predictable, with the critical mixing ratio following a power-law relationship with the model size. Our findings highlight a concrete case where a good mixing recipe for large models may not be optimal for small models, and vice versa.
No ratings yet
View paper →Outcome-based Reinforcement Learning to Predict the Future
Benjamin Turtel, Danny Franklin, Kris Skotheim, Luke Hewitt, Philipp Schoenegger
arXiv·2025
Reinforcement learning with verifiable rewards (RLVR) has boosted math and
coding in large language models, yet there has been little effort to extend
RLVR into messier, real-world domains like forecasting. One sticking point is
that outcome-based reinforcement learning for forecasting must learn from
binary, delayed, and noisy rewards, a regime where standard fine-tuning is
brittle. We show that outcome-only online RL on a 14B model can match
frontier-scale accuracy and surpass it in calibration and hypothetical
prediction market betting by adapting two leading algorithms, Group-Relative
Policy Optimisation (GRPO) and ReMax, to the forecasting setting. Our
adaptations remove per-question variance scaling in GRPO, apply
baseline-subtracted advantages in ReMax, hydrate training with 100k temporally
consistent synthetic questions, and introduce lightweight guard-rails that
penalise gibberish, non-English responses and missing rationales, enabling a
single stable pass over 110k events. Scaling ReMax to 110k questions and
ensembling seven predictions yields a 14B model that matches frontier baseline
o1 on accuracy on our holdout set (Brier = 0.193, p = 0.23) while beating it in
calibration (ECE = 0.042, p < 0.001). A simple trading rule turns this
calibration edge into \$127 of hypothetical profit versus \$92 for o1 (p =
0.037). This demonstrates that refined RLVR methods can convert small-scale
LLMs into potentially economically valuable forecasting tools, with
implications for scaling this to larger models.
No ratings yet
View paper →Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training
Tony Bonnaire, Raphaël Urfin, Giulio Biroli, Marc Mézard
arXiv·2025
Diffusion models have achieved remarkable success across a wide range of generative tasks. A key challenge is understanding the mechanisms that prevent their memorization of training data and allow generalization. In this work, we investigate the role of the training dynamics in the transition from generalization to memorization. Through extensive experiments and theoretical analysis, we identify two distinct timescales: an early time $τ_\mathrm{gen}$ at which models begin to generate high-quality samples, and a later time $τ_\mathrm{mem}$ beyond which memorization emerges. Crucially, we find that $τ_\mathrm{mem}$ increases linearly with the training set size $n$, while $τ_\mathrm{gen}$ remains constant. This creates a growing window of training times with $n$ where models generalize effectively, despite showing strong memorization if training continues beyond it. It is only when $n$ becomes larger than a model-dependent threshold that overfitting disappears at infinite training times. These findings reveal a form of implicit dynamical regularization in the training dynamics, which allow to avoid memorization even in highly overparameterized settings. Our results are supported by numerical experiments with standard U-Net architectures on realistic and synthetic datasets, and by a theoretical analysis using a tractable random features model studied in the high-dimensional limit.
★ 4.0 (1)
View paper →The Spacetime of Diffusion Models: An Information Geometry Perspective
Rafał Karczewski, Markus Heinonen, Alison Pouplin, Søren Hauberg, Vikas Garg
arXiv·2025
We present a novel geometric perspective on the latent space of diffusion models. We first show that the standard pullback approach, utilizing the deterministic probability flow ODE decoder, is fundamentally flawed. It provably forces geodesics to decode as straight segments in data space, effectively ignoring any intrinsic data geometry beyond the ambient Euclidean space. Complementing this view, diffusion also admits a stochastic decoder via the reverse SDE, which enables an information geometric treatment with the Fisher-Rao metric. However, a choice of $x_T$ as the latent representation collapses this metric due to memorylessness. We address this by introducing a latent spacetime $z=(x_t,t)$ that indexes the family of denoising distributions $p(x_0 | x_t)$ across all noise scales, yielding a nontrivial geometric structure. We prove these distributions form an exponential family and derive simulation-free estimators for curve lengths, enabling efficient geodesic computation. The resulting structure induces a principled Diffusion Edit Distance, where geodesics trace minimal sequences of noise and denoise edits between data. We also demonstrate benefits for transition path sampling in molecular systems, including constrained variants such as low-variance transitions and region avoidance. Code is available at: https://github.com/rafalkarczewski/spacetime-geometry
No ratings yet
View paper →‘I am treated unfairly because … ’: exploring the evolution of ethnic discrimination perceptions among primary school children in Flanders, Belgium
Fien Geenen, Lies Saelens, Alain Van Hiel, Bart Van de Putte, Barbara Valcke, Peter Stevens
Education 3-13·2025
This study examines how primary school pupils expect and perceive ethnic discrimination in teachers’ allocation of educational resources using a mixed-methods design. Participants were 265 children from the first to the sixth grade across five primary schools in Flanders, Belgium. Quantitative data reveal a nuanced understanding of discrimination, with an age-related trend indicating that younger pupils exhibit more explicit ethnicity bias in their expectations of teacher behaviour. As pupils grow older, this bias diminishes and even reverses, suggesting an evolving understanding of and negative attitude towards discrimination. Qualitative data highlight children’s ability to discern both overt and subtle forms of discrimination, influenced by personal experiences and socialisation processes in school. The results discuss implications for social policy, theory, and practice.
No ratings yet
View paper →