Lowest Rated Papers
The Computational Limits of Deep Learning
Neil C. Thompson, Kristjan Greenewald, Keeheon Lee, Gabriel F. Manso
arXiv·2020
Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image classification, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article catalogs the extent of this dependency, showing that progress across a wide variety of applications is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.
No ratings yet
View paper →The Impact of an Experimental Guaranteed Income on Crime and Violence
David Calnitsky, Pilar Gonalons-Pons
Social Problems·2020·20 citations
Would unconditional cash payments reduce crime and violence? This paper examines data on crime and violence in the context of an understudied social experiment from the late 1970s called the Manitoba Basic Annual Income Experiment, or Mincome. We combine town-level crime statistics for all medium-sized Canadian Prairie towns with town-level socio-demographic data from the census to study how an experimental guaranteed income affected both violent crime and total crime. We find a significant negative relationship between Mincome and both outcomes. We also decompose total crime and analyze its main components, property crime and “other” crime, and find a significant negative relationship between Mincome and property crime. While the impact on property crime is theoretically straightforward, we close by speculating on the mechanisms that might link the availability of guaranteed annual income payments to a decline in violence, focusing on the mechanisms that shape patterns of inter-partner violence.
No ratings yet
View paper →Going Postal – Bookforum Magazine
Unknown Author
Bookforum·2020
A psychoanalytic reading of social media and the death drive – Max Read
No ratings yet
View paper →Why Enterprises Need Specialized RL Agents | Scale
Unknown Author
Scale AI·2023
General AI models struggle with enterprise workflows. Learn how Scale's specialized RL agents achieve superior accuracy.
No ratings yet
View paper →Enterprise Reinforcement Learning with Rubrics as Rewards | Scale
Unknown Author
Scale AI·2023
Scale's RaR trains smaller models to outperform bigger models on enterprise tasks.
No ratings yet
View paper →Closing the Gap Between AI Promise and Enterprise Reality | Scale
Unknown Author
Scale AI·2023
Yuan Xue joins Scale AI to bridge the gap between AI’s promise and real enterprise results.
No ratings yet
View paper →Improved Small Set Expansion in High Dimensional Expanders
Tali Kaufman, David Mass
eccc.weizmann.ac.il
Small set expansion in high dimensional expanders is of great importance, e.g., towards
proving cosystolic expansion, local testability of codes and constructions of good quantum
codes.
In this work we improve upon the state of the art results of small set expansion in high
dimensional expanders. Our improvement is either on the expansion quality or on the size
of sets for which expansion is guaranteed.
One line of previous works [KM22, DD24] has obtained weak expansion for small sets,
which is sufficient for deducing cosystolic expansion of one dimension below. We improve
upon their result by showing strong expansion for small sets.
Another line of works [KKL14, EK16, KM21] has shown strong expansion for small sets.
However, they obtain it only for very small sets. We get an exponential improvement on
the size of sets for which expansion is guaranteed by these prior works.
Interestingly, our result is obtained by bridging between these two lines of works. The
works of [KM22, DD24] use global averaging operators in order to obtain expansion for
larger sets. However, their method could be utilized only on sets that are cocycle-like. We
show how to combine these global averaging operators with ideas from the so-called “fat
machinery” of [KKL14, EK16, KM21] in order to apply them for general sets.
No ratings yet
View paper →Scaling Laws and Symmetry, Evidence from Neural Force Fields
Khang Ngo, Siamak Ravanbakhsh
arXiv·2025
We present an empirical study in the geometric task of learning interatomic potentials, which shows equivariance matters even more at larger scales; we show a clear power-law scaling behaviour with respect to data, parameters and compute with ``architecture-dependent exponents''. In particular, we observe that equivariant architectures, which leverage task symmetry, scale better than non-equivariant models. Moreover, among equivariant architectures, higher-order representations translate to better scaling exponents. Our analysis also suggests that for compute-optimal training, the data and model sizes should scale in tandem regardless of the architecture. At a high level, these results suggest that, contrary to common belief, we should not leave it to the model to discover fundamental inductive biases such as symmetry, especially as we scale, because they change the inherent difficulty of the task and its scaling laws.
No ratings yet
View paper →Beyond Attention as a Graph - The Tensor Throne - Obsidian Publish
Unknown Author
publish.obsidian.md
Beyond Attention as a Graph - The Tensor Throne - Powered by Obsidian Publish.
No ratings yet
View paper →BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova
Proceedings of the 2019 Conference of the North·2019·4765 citations
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models (Peters et al., 2018a; Radford et al., 2018), BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5 (7.7 point absolute improvement), MultiNLI accuracy to 86.7% (4.6% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).
No ratings yet
View paper →RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R. Sanketi, Grecia Salazar, Michael S. Ryoo, Krista Reymann, Kanishka Rao, Karl Pertsch, Igor Mordatch, Henryk Michalewski, Yao Lu, Sergey Levine, Lisa Lee, Tsang-Wei Edward Lee, Isabel Leal, Yuheng Kuang, Dmitry Kalashnikov, Ryan Julian, Nikhil J. Joshi, Alex Irpan, Brian Ichter, Jasmine Hsu, Alexander Herzog, Karol Hausman, Keerthana Gopalakrishnan, Chuyuan Fu, Pete Florence, Chelsea Finn, Kumar Avinava Dubey, Danny Driess, Tianli Ding, Krzysztof Marcin Choromanski, Xi Chen, Yevgen Chebotar, Justice Carbajal, Noah Brown, Anthony Brohan, Montserrat Gonzalez Arenas, Kehang Han
PMLR·2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic ControlBrianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao,&...
No ratings yet
View paper →Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems?
Kai Yan, Yufei Xu, Zhengyin Du, Xuesong Yao, Zheyu Wang, Xiaowen Guo, Jiecao Chen
arXiv·2025
The rapid escalation from elementary school-level to frontier problems of the difficulty for LLM benchmarks in recent years have weaved a miracle for researchers that we are only inches away from surpassing human intelligence. However, is the LLMs' remarkable reasoning ability indeed comes from true intelligence by human standards, or are they simply reciting solutions witnessed during training at an Internet level? To study this problem, we propose RoR-Bench, a novel, multi-modal benchmark for detecting LLM's recitation behavior when asked simple reasoning problems but with conditions subtly shifted, and conduct empirical analysis on our benchmark. Surprisingly, we found existing cutting-edge LLMs unanimously exhibits extremely severe recitation behavior; by changing one phrase in the condition, top models such as OpenAI-o1 and DeepSeek-R1 can suffer 60 percent performance loss on elementary school-level arithmetic and reasoning problems. Such findings are a wake-up call to the LLM community that compels us to re-evaluate the true intelligence level of cutting-edge LLMs.
No ratings yet
View paper →