Lowest Rated Papers
Diversity Is the Strength of the AI Crowd
Unknown Author
openreview.net
Top AI forecasting systems are approaching superforecaster-level accuracy on future world events, but still rely primarily on off-the-shelf
LLMs combined with forecasting-specific context gathering and scaffolding. We study how to
improve this recipe through ensembling: given a fixed number of samples, which off-the-shelf
model forecasts should be combined to maximize accuracy? On binary questions from the Metaculus AI Benchmark, we find that individual accuracy is not enough: many frontier LLMs make highly correlated predictions, limiting the value
of additional forecasts from the same or similar models. Instead, the strongest ensembles combine accurate but diverse forecasters, with models such as Grok 4 contributing disproportionately
because their predictions are less correlated with other frontier LLMs. These results suggest that the strength of the AI crowd comes not from sampling more forecasts indiscriminately, but from combining forecasts across models with complementary errors, motivating forecasting systems that explicitly optimize for both model quality and diversity.
No ratings yet
View paper →Scaling FP8 training to trillion-token LLMs
Maxim Fishman, Brian Chmiel, Ron Banner, Daniel Soudry
arXiv·2024
We train, for the first time, large language models using FP8 precision on datasets up to 2 trillion tokens -- a 20-fold increase over previous limits. Through these extended training runs, we uncover critical instabilities in FP8 training that were not observable in earlier works with shorter durations. We trace these instabilities to outlier amplification by the SwiGLU activation function. Interestingly, we show, both analytically and empirically, that this amplification happens only over prolonged training periods, and link it to a SwiGLU weight alignment process. To address this newly identified issue, we introduce Smooth-SwiGLU, a novel modification that ensures stable FP8 training without altering function behavior. We also demonstrate, for the first time, FP8 quantization of both Adam optimizer moments. Combining these innovations, we successfully train a 7B parameter model using FP8 precision on 256 Intel Gaudi2 accelerators, achieving on-par results with the BF16 baseline while delivering up to a $\sim 34 \%$ throughput improvement. A reference implementation is supplied in https://github.com/Anonymous1252022/Megatron-DeepSpeed.
No ratings yet
View paper →Gaza to Donbass: How Israel and Ukraine Built a Fascist, Transnational War Machine | Black Agenda Report
Sarah B.
Black Agenda Report·2025
From Bandera to Ben-Gurion, an axis of ethno-supremacy is rising, fueled by U.S. backing. Same guns. Same flags. Same ideology. Gaza and Donbass are not separate wars. They are one machine.
No ratings yet
View paper →