Oldest and Lowest Rated Papers
Competitive Programming with Large Reasoning Models
OpenAI, :, Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, Jerry Tworek, Lorenz Kuhn, Lukasz Kaiser, Mark Chen, Max Schwarzer, Mostafa Rohaninejad, Nat McAleese, o3 contributors, Oleg Mürk, Rhythm Garg, Rui Shu, Szymon Sidor, Vineet Kosaraju, Wenda Zhou
arXiv·2025
We show that reinforcement learning applied to large language models (LLMs)
significantly boosts performance on complex coding and reasoning tasks.
Additionally, we compare two general-purpose reasoning models - OpenAI o1 and
an early checkpoint of o3 - with a domain-specific system, o1-ioi, which uses
hand-engineered inference strategies designed for competing in the 2024
International Olympiad in Informatics (IOI). We competed live at IOI 2024 with
o1-ioi and, using hand-crafted test-time strategies, placed in the 49th
percentile. Under relaxed competition constraints, o1-ioi achieved a gold
medal. However, when evaluating later models such as o3, we find that o3
achieves gold without hand-crafted domain-specific strategies or relaxed
constraints. Our findings show that although specialized pipelines such as
o1-ioi yield solid improvements, the scaled-up, general-purpose o3 model
surpasses those results without relying on hand-crafted inference heuristics.
Notably, o3 achieves a gold medal at the 2024 IOI and obtains a Codeforces
rating on par with elite human competitors. Overall, these results indicate
that scaling general-purpose reinforcement learning, rather than relying on
domain-specific techniques, offers a robust path toward state-of-the-art AI in
reasoning domains, such as competitive programming.
No ratings yet
View paper →Mantodea phylogenomics provides new insights into X-chromosome progression and evolutionary radiation
Hangwei Liu, Lihong Lei, Fan Jiang, Bo Zhang, Hengchao Wang, Yutong Zhang, Anqi Wang, Hanbo Zhao, Guirong Wang, Wei Fan
arXiv·2025
Background: Praying mantises, members of the order Mantodea, play important
roles in agriculture, medicine, bionics, and entertainment. However, the
scarcity of genomic resources has hindered extensive studies on mantis
evolution and behaviour. Results: Here, we present the chromosome-scale
reference genomes of five mantis species: the European mantis (Mantis
religiosa), Chinese mantis (Tenodera sinensis), triangle dead leaf mantis
(Deroplatys truncata), orchid mantis (Hymenopus coronatus), and metallic mantis
(Metallyticus violaceus). We found that transposable element expansion is the
major force governing genome size in Mantodea. Based on whole-alignments, we
deduced that the Mantodea ancestor may have had only one X chromosome and that
translocations between the X chromosome and an autosome may have occurred in
the lineage of the superfamily Mantoidea. Furthermore, we found a lower
evolutionary rate for the metallic mantis than for the other mantises. We also
found that Mantodea underwent rapid radiation after the K-Pg mass extinction
event, which could have contributed to the confusion in species classification.
Conclusions: We present the chromosome-scale reference genomes of five mantis
species to reveal the X-chromosome evolution, clarify the phylogeny
relationship, and transposable element expansion.
No ratings yet
View paper →Diffusion Models Beat GANs on Image Synthesis
Prafulla Dhariwal, Alex Nichol
arXiv·2021
We show that diffusion models can achieve image sample quality superior to
the current state-of-the-art generative models. We achieve this on
unconditional image synthesis by finding a better architecture through a series
of ablations. For conditional image synthesis, we further improve sample
quality with classifier guidance: a simple, compute-efficient method for
trading off diversity for fidelity using gradients from a classifier. We
achieve an FID of 2.97 on ImageNet 128$\times$128, 4.59 on ImageNet
256$\times$256, and 7.72 on ImageNet 512$\times$512, and we match BigGAN-deep
even with as few as 25 forward passes per sample, all while maintaining better
coverage of the distribution. Finally, we find that classifier guidance
combines well with upsampling diffusion models, further improving FID to 3.94
on ImageNet 256$\times$256 and 3.85 on ImageNet 512$\times$512. We release our
code at https://github.com/openai/guided-diffusion
No ratings yet
View paper →Paired Completion: Flexible Quantification of Issue-framing at Scale with LLMs
Simon D Angus, Lachlan O'Neill
arXiv·2024
Detecting issue framing in text - how different perspectives approach the
same topic - is valuable for social science and policy analysis, yet
challenging for automated methods due to subtle linguistic differences. We
introduce `paired completion', a novel approach using LLM next-token log
probabilities to detect contrasting frames using minimal examples. Through
extensive evaluation across synthetic datasets and a human-labeled corpus, we
demonstrate that paired completion is a cost-efficient, low-bias alternative to
both prompt-based and embedding-based methods, offering a scalable solution for
analyzing issue framing in large text collections, especially suited to
low-resource settings.
No ratings yet
View paper →Weighted Tensor Decompositions for Context-aware Collaborative Filtering
Joey De Pauw, Bart Goethals
arXiv·2025
Over recent years it has become well accepted that user interest is not
static or immutable. There are a variety of contextual factors, such as time of
day, the weather or the user's mood, that influence the current interests of
the user. Modelling approaches need to take these factors into account if they
want to succeed at finding the most relevant content to recommend given the
situation.
A popular method for context-aware recommendation is to encode context
attributes as extra dimensions of the classic user-item interaction matrix,
effectively turning it into a tensor, followed by applying the appropriate
tensor decomposition methods to learn missing values. However, unlike with
matrix factorization, where all decompositions are essentially a product of
matrices, there exist many more options for decomposing tensors by combining
vector, matrix and tensor products. We study the most successful decomposition
methods that use weighted square loss and categorize them based on their tensor
structure and regularization strategy. Additionally, we further extend the pool
of methods by filling in the missing combinations.
In this paper we provide an overview of the properties of the different
decomposition methods, such as their complexity, scalability, and modelling
capacity. These benefits are then contrasted with the performances achieved in
offline experiments to gain more insight into which method to choose depending
on a specific situation and constraints.
No ratings yet
View paper →Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments
Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, Ilya Sutskever, Igor Mordatch, Pieter Abbeel
arXiv·2017
Ability to continuously learn and adapt from limited experience in
nonstationary environments is an important milestone on the path towards
general intelligence. In this paper, we cast the problem of continuous
adaptation into the learning-to-learn framework. We develop a simple
gradient-based meta-learning algorithm suitable for adaptation in dynamically
changing and adversarial scenarios. Additionally, we design a new multi-agent
competitive environment, RoboSumo, and define iterated adaptation games for
testing various aspects of continuous adaptation strategies. We demonstrate
that meta-learning enables significantly more efficient adaptation than
reactive baselines in the few-shot regime. Our experiments with a population of
agents that learn and compete suggest that meta-learners are the fittest.
No ratings yet
View paper →Learning Iterative Reasoning through Energy Minimization
Yilun Du, Shuang Li, Joshua B. Tenenbaum, Igor Mordatch
arXiv·2022
Deep learning has excelled on complex pattern recognition tasks such as image
classification and object recognition. However, it struggles with tasks
requiring nontrivial reasoning, such as algorithmic computation. Humans are
able to solve such tasks through iterative reasoning -- spending more time
thinking about harder tasks. Most existing neural networks, however, exhibit a
fixed computational budget controlled by the neural network architecture,
preventing additional computational processing on harder tasks. In this work,
we present a new framework for iterative reasoning with neural networks. We
train a neural network to parameterize an energy landscape over all outputs,
and implement each step of the iterative reasoning as an energy minimization
step to find a minimal energy solution. By formulating reasoning as an energy
minimization problem, for harder problems that lead to more complex energy
landscapes, we may then adjust our underlying computational budget by running a
more complex optimization procedure. We empirically illustrate that our
iterative reasoning approach can solve more accurate and generalizable
algorithmic reasoning tasks in both graph and continuous domains. Finally, we
illustrate that our approach can recursively solve algorithmic problems
requiring nested reasoning
No ratings yet
View paper →A General Language Assistant as a Laboratory for Alignment
Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Jared Kaplan
arXiv·2021
Given the broad capabilities of large language models, it should be possible
to work towards a general-purpose, text-based assistant that is aligned with
human values, meaning that it is helpful, honest, and harmless. As an initial
foray in this direction we study simple baseline techniques and evaluations,
such as prompting. We find that the benefits from modest interventions increase
with model size, generalize to a variety of alignment evaluations, and do not
compromise the performance of large models. Next we investigate scaling trends
for several training objectives relevant to alignment, comparing imitation
learning, binary discrimination, and ranked preference modeling. We find that
ranked preference modeling performs much better than imitation learning, and
often scales more favorably with model size. In contrast, binary discrimination
typically performs and scales very similarly to imitation learning. Finally we
study a `preference model pre-training' stage of training, with the goal of
improving sample efficiency when finetuning on human preferences.
No ratings yet
View paper →Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback
Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, Scott A. Hale
arXiv·2023
Large language models (LLMs) are used to generate content for a wide range of
tasks, and are set to reach a growing audience in coming years due to
integration in product interfaces like ChatGPT or search engines like Bing.
This intensifies the need to ensure that models are aligned with human
preferences and do not produce unsafe, inaccurate or toxic outputs. While
alignment techniques like reinforcement learning with human feedback (RLHF) and
red-teaming can mitigate some safety concerns and improve model capabilities,
it is unlikely that an aggregate fine-tuning process can adequately represent
the full range of users' preferences and values. Different people may
legitimately disagree on their preferences for language and conversational
norms, as well as on values or ideologies which guide their communication.
Personalising LLMs through micro-level preference learning processes may result
in models that are better aligned with each user. However, there are several
normative challenges in defining the bounds of a societally-acceptable and safe
degree of personalisation. In this paper, we ask how, and in what ways, LLMs
should be personalised. First, we review literature on current paradigms for
aligning LLMs with human feedback, and identify issues including (i) a lack of
clarity regarding what alignment means; (ii) a tendency of technology providers
to prescribe definitions of inherently subjective preferences and values; and
(iii) a 'tyranny of the crowdworker', exacerbated by a lack of documentation in
who we are really aligning to. Second, we present a taxonomy of benefits and
risks associated with personalised LLMs, for individuals and society at large.
Finally, we propose a three-tiered policy framework that allows users to
experience the benefits of personalised alignment, while restraining unsafe and
undesirable LLM-behaviours within (supra-)national and organisational bounds.
No ratings yet
View paper →Position: Episodic Memory is the Missing Piece for Long-Term LLM Agents
Mathis Pink, Qinyuan Wu, Vy Ai Vo, Javier Turek, Jianing Mu, Alexander Huth, Mariya Toneva
arXiv·2025
As Large Language Models (LLMs) evolve from text-completion tools into fully
fledged agents operating in dynamic environments, they must address the
challenge of continually learning and retaining long-term knowledge. Many
biological systems solve these challenges with episodic memory, which supports
single-shot learning of instance-specific contexts. Inspired by this, we
present an episodic memory framework for LLM agents, centered around five key
properties of episodic memory that underlie adaptive and context-sensitive
behavior. With various research efforts already partially covering these
properties, this position paper argues that now is the right time for an
explicit, integrated focus on episodic memory to catalyze the development of
long-term agents. To this end, we outline a roadmap that unites several
research directions under the goal to support all five properties of episodic
memory for more efficient long-term LLM agents.
No ratings yet
View paper →Localizing Lying in Llama: Understanding Instructed Dishonesty on True-False Questions Through Prompting, Probing, and Patching
James Campbell, Richard Ren, Phillip Guo
arXiv·2023
Large language models (LLMs) demonstrate significant knowledge through their
outputs, though it is often unclear whether false outputs are due to a lack of
knowledge or dishonesty. In this paper, we investigate instructed dishonesty,
wherein we explicitly prompt LLaMA-2-70b-chat to lie. We perform prompt
engineering to find which prompts best induce lying behavior, and then use
mechanistic interpretability approaches to localize where in the network this
behavior occurs. Using linear probing and activation patching, we localize five
layers that appear especially important for lying. We then find just 46
attention heads within these layers that enable us to causally intervene such
that the lying model instead answers honestly. We show that these interventions
work robustly across many prompts and dataset splits. Overall, our work
contributes a greater understanding of dishonesty in LLMs so that we may hope
to prevent it.
No ratings yet
View paper →Reinforcement Pre-Training
Qingxiu Dong, Li Dong, Yao Tang, Tianzhu Ye, Yutao Sun, Zhifang Sui, Furu Wei
arXiv·2025
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling
paradigm for large language models and reinforcement learning (RL).
Specifically, we reframe next-token prediction as a reasoning task trained
using RL, where it receives verifiable rewards for correctly predicting the
next token for a given context. RPT offers a scalable method to leverage vast
amounts of text data for general-purpose RL, rather than relying on
domain-specific annotated answers. By incentivizing the capability of
next-token reasoning, RPT significantly improves the language modeling accuracy
of predicting the next tokens. Moreover, RPT provides a strong pre-trained
foundation for further reinforcement fine-tuning. The scaling curves show that
increased training compute consistently improves the next-token prediction
accuracy. The results position RPT as an effective and promising scaling
paradigm to advance language model pre-training.
No ratings yet
View paper →