Lowest Rated Papers
LiteAttention: A Temporal Sparse Attention for Diffusion Transformers
Dor Shmilovich, Tony Wu, Aviad Dahan, Yuval Domb
arXiv·2025
Diffusion Transformers, particularly for video generation, achieve remarkable quality but suffer from quadratic attention complexity, leading to prohibitive latency. Existing acceleration methods face a fundamental trade-off: dynamically estimating sparse attention patterns at each denoising step incurs high computational overhead and estimation errors, while static sparsity patterns remain fixed and often suboptimal throughout denoising. We identify a key structural property of diffusion attention, namely, its sparsity patterns exhibit strong temporal coherence across denoising steps. Tiles deemed non-essential at step $t$ typically remain so at step $t+δ$. Leveraging this observation, we introduce LiteAttention, a method that exploits temporal coherence to enable evolutionary computation skips across the denoising sequence. By marking non-essential tiles early and propagating skip decisions forward, LiteAttention eliminates redundant attention computations without repeated profiling overheads, combining the adaptivity of dynamic methods with the efficiency of static ones. We implement a highly optimized LiteAttention kernel on top of FlashAttention and demonstrate substantial speedups on production video diffusion models, with no degradation in quality. The code and implementation details will be publicly released.
No ratings yet
View paper →‘They are watching you do everything online’: Children's perceptions of social media surveillance
Claire Kathryn Pescott
Children & Society·2024·3 citations
<jats:title>Abstract</jats:title><jats:p>Through a Foucauldian lens, this qualitative study explored the perspectives and lived conditions of children's experiences of social media surveillance. Sixteen children between the ages of 10 and 11 years old participated in the creative method of collaging with an unstructured interview in four schools in South Wales, UK. Visual combined with verbal analysis found a nuanced picture of how social media surveillance has influenced children's cultural and social practices of their childhood. Despite the challenges of peer and adult control exemplified, children did demonstrate agency within their digital spaces. Policy implications should involve a stronger emphasis on developing children's emotional resilience and discernment surrounding perceived surveillance.</jats:p>
No ratings yet
View paper →Dark Matter Velocity Distributions for Direct Detection: Astrophysical Uncertainties Are Smaller Than They Appear
Dylan Folsom, Carlos Blanco, Mariangela Lisanti, Lina Necib, Mark Vogelsberger, Lars Hernquist
Physical Review Letters·2025
<jats:p>
The sensitivity of direct detection experiments depends on the phase-space distribution of dark matter near the Sun, which can be modeled theoretically using cosmological hydrodynamical simulations of Milky Way–like galaxies. However, capturing the halo-to-halo variation in the local dark matter speeds—a necessary step for quantifying the astrophysical uncertainties that feed into experimental results—requires a sufficiently large sample of simulated galaxies, which has been a challenge. In this Letter, we quantify this variation with nearly 100 Milky Way–like galaxies from the 50 simulation, the largest sample to date at this resolution. Moreover, we introduce a novel phase-space scaling procedure that endows every system with a reference frame that accurately reproduces the local standard-of-rest speed of our Galaxy, providing a principled way of extrapolating the simulation results to real-world data. The ensemble of predicted speed distributions is well characterized by the standard halo model, a Maxwell-Boltzmann distribution truncated at the escape speed, though the individual distributions can deviate from it, especially at high speeds. The dark matter–nucleon cross section limits placed by these speed distributions vary by
<a:math xmlns:a="http://www.w3.org/1998/Math/MathML" display="inline">
<a:mo>∼</a:mo>
<a:mn>60</a:mn>
<a:mo>%</a:mo>
</a:math>
about the median. This places the
<c:math xmlns:c="http://www.w3.org/1998/Math/MathML" display="inline">
<c:mrow>
<c:mn>1</c:mn>
<c:mi>σ</c:mi>
</c:mrow>
</c:math>
astrophysical uncertainty at or below the level of the systematic uncertainty of current ton-scale detectors, even down to the energy threshold. The predicted uncertainty remains unchanged when subselecting on those 50 galaxies with merger histories similar to the Milky Way. Tabulated speed distributions, as well as Maxwell-Boltzmann fits, are provided for use in computing direct detection bounds or projecting sensitivities.
</jats:p>
No ratings yet
View paper →Cosmological Simulations of Stellar Halos with Gaia Sausage-Enceladus Analogues: Two Sausages, One Bun?
Dylan Folsom, Mariangela Lisanti, Lina Necib, Danny Horta, Mark Vogelsberger, Lars Hernquist
arXiv·2024
Observations of the Milky Way's stellar halo find that it is predominantly comprised of a radially-biased population of stars, dubbed the Gaia Sausage--Enceladus, or GSE. These stars are thought to be debris from dwarf galaxy accretion early in the Milky Way's history. Though typically considered to be from a single merger, it is possible that the GSE debris has multiple sources. To investigate this possibility, we use the IllustrisTNG50 simulation to identify stellar accretion histories in 98 Milky Way analogues -- the largest sample for which such an identification has been performed -- and find GSE-like debris in 32, with two-merger GSEs accounting for a third of these cases. Distinguishing single-merger GSEs from two-merger GSEs is difficult in common kinematic spaces, but differences are more evident through chemical abundances and star formation histories. This is because single-merger GSEs are typically accreted more recently than the galaxies in two-merger GSEs: the median infall times (with 16th and 84th percentiles) are $5.9^{+3.3}_{-2.0}$ and $10.7^{+1.2}_{-3.7}$ Gyr ago for these scenarios, respectively. The systematic shifts in abundances and ages which occur as a result suggest that efforts in modeling these aspects of the stellar halo prove ever-important in understanding its assembly.
No ratings yet
View paper →The HoTT Book | Homotopy Type Theory
Unknown Author
Homotopy Type Theory·2013
Homotopy Type Theory: Univalent Foundations of Mathematics The Univalent Foundations Program Institute for Advanced Study Buy a hardcover copy for $21.00. [620 pages, 6" × 9" size, hardcover, first-edition-1277-g3274cb3] Buy a paperback copy for $14.00. [620 pages, 6" × 9" size, paperback, first-edition-1277-g3274cb3] Download PDF for on-screen viewing. [490+ pages, letter size, in color, with…
No ratings yet
View paper →Mixture-of-Depths: Dynamically allocating compute in transformer-based language models | alphaXiv
Unknown Author
alphaxiv.org
Transformer-based language models spread FLOPs uniformly across input sequences. In this work we demonstrate that transformers can instead learn to dynamically allocate FLOPs (or compute) to specific positions in a sequence, optimising the allocation along the sequence for different layers across the model depth. Our method enforces a total compute budget by capping the number of tokens (𝑘) that can participate in the self-attention and MLP computations at a given layer. The tokens to be processed are determined by the network using a top-𝑘 routing mechanism. Since 𝑘 is defined a priori, this simple procedure uses a static computation graph with known tensor sizes, unlike other conditional computation techniques. Nevertheless, since the identities of the 𝑘 tokens are fluid, this method can expend FLOPs non-uniformly across the time and model depth dimensions. Thus, compute expenditure is entirely predictable in sum total, but dynamic and context-sensitive at the token-level. Not only do models trained in this way learn to dynamically allocate compute, they do so efficiently. These models match baseline performance for equivalent FLOPS and wall-clock times to train, but require a fraction of the FLOPs per forward pass, and can be upwards of 50% faster to step during post-training sampling.
No ratings yet
View paper →Contrastive independent component analysis for salient patterns and dimensionality reduction
Kexin Wang, Aida Maraj, Anna Seigal
Proceedings of the National Academy of Sciences·2025
<jats:p>In recent years, there has been growing interest in jointly analyzing a foreground dataset, representing an experimental group, and a background dataset, representing a control group. The goal of such contrastive investigations is to identify salient features in the experimental group relative to the control. Independent component analysis (ICA) is a powerful tool for learning independent patterns in a dataset. We generalize it to contrastive ICA (cICA). For this purpose, we devise a linear algebra–based tensor decomposition algorithm, which is more expressive but just as efficient and identifiable as other linear algebra–based algorithms. We establish the identifiability of cICA and demonstrate its performance in finding patterns and visualizing data, using synthetic, semisynthetic, and real-world datasets, comparing the approach to existing methods.</jats:p>
No ratings yet
View paper →Project MUSE -- Verification required!
Unknown Author
muse.jhu.edu
No abstract available
No ratings yet
View paper →How to Build an Empirical Speed Distribution for Dark Matter in the Solar Neighborhood
Tal Shpigel, Dylan Folsom, Mariangela Lisanti, Lina Necib, Mark Vogelsberger, Lars Hernquist
arXiv·2025
The dark matter flux in a direct detection experiment depends on its local speed distribution. This distribution has been inferred from simulations of Milky Way-like galaxies, but such models serve only as proxies given that no simulation directly captures the detailed evolution of our own Galaxy. This motivates alternative approaches which obtain this distribution directly from observations. In this work, we utilize 98 Milky Way analogues from the IllustrisTNG50 simulation to develop and validate a procedure for inferring the dark matter speed distribution using the kinematics of nearby stars. We find that the dark matter that originated from old mergers, plus that from recent non-luminous accretions, is well described by a Maxwell-Boltzmann speed distribution centered at the local standard-of-rest velocity. Meanwhile, recently accreted dark matter from massive mergers has speeds that can be traced from the associated stellar debris of these events. The stellar populations systematically underestimate the velocity dispersion of their dark matter counterparts, but a simple kinematic boost brings the two into good alignment. Using the TNG50 host galaxies, we demonstrate that combining these two contributions provides an accurate reconstruction of the local dark matter speeds. As an application of the procedure to our own Galaxy, we utilize stellar kinematic data from Gaia to quantify how the dark matter remnants from the Milky Way's last major merger impact its speed distribution in the Solar neighborhood.
No ratings yet
View paper →Model theory for modal logic part I?The De re/de dicto distinction
Kit Fine
Journal of Philosophical Logic·1978·13 citations
No abstract available
No ratings yet
View paper →Dark Matter Velocity Distributions for Direct Detection: Astrophysical Uncertainties are Smaller Than They Appear
Dylan Folsom, Carlos Blanco, Mariangela Lisanti, Lina Necib, Mark Vogelsberger, Lars Hernquist
arXiv·2025
The sensitivity of direct detection experiments depends on the phase-space distribution of dark matter near the Sun, which can be modeled theoretically using cosmological hydrodynamical simulations of Milky Way-like galaxies. However, capturing the halo-to-halo variation in the local dark matter speeds -- a necessary step for quantifying the astrophysical uncertainties that feed into experimental results -- requires a sufficiently large sample of simulated galaxies, which has been a challenge. In this work, we quantify this variation with nearly one hundred Milky Way-like galaxies from the IllustrisTNG50 simulation, the largest sample to date at this resolution. Moreover, we introduce a novel phase-space scaling procedure that endows every system with a reference frame that accurately reproduces the local standard-of-rest speed of our Galaxy, providing a principled way of extrapolating the simulation results to real-world data. The predicted speed distributions are consistent with the Standard Halo Model, a Maxwell-Boltzmann distribution peaked at the local circular speed and truncated at the escape speed. The dark matter-nucleon cross section limits placed by these speed distributions vary by ~60% about the median. This places the 1-sigma astrophysical uncertainty at or below the level of the systematic uncertainty of current ton-scale detectors, even down to the energy threshold. The predicted uncertainty remains unchanged when sub-selecting on those TNG galaxies with merger histories similar to the Milky Way. Tabulated speed distributions, as well as Maxwell-Boltzmann fits, are provided for use in computing direct detection bounds or projecting sensitivities.
No ratings yet
View paper →Probabilistic Inference of the Structure and Orbit of Milky Way Satellites with Semi-Analytic Modeling
Dylan Folsom, Oren Slone, Mariangela Lisanti, Fangzhou Jiang, Manoj Kaplinghat
arXiv·2023
Semi-analytic modeling furnishes an efficient avenue for characterizing the properties of dark matter halos associated with satellites of Milky Way-like systems, as it easily accounts for uncertainties arising from halo-to-halo variance, the orbital disruption of satellites, baryonic feedback, and the stellar-to-halo mass (SMHM) relation. We use the SatGen semi-analytic satellite generator -- which incorporates both empirical models of the galaxy-halo connection in the field as well as analytic prescriptions for the orbital evolution of these satellites after they enter a host galaxy -- to create large samples of Milky Way-like systems and their satellites. By selecting satellites in the sample that match the observed properties of a particular dwarf galaxy, we can then infer arbitrary properties of the satellite galaxy within the Cold Dark Matter paradigm. For the Milky Way's classical dwarfs, we provide inferred values (with associated uncertainties) for the maximum circular velocity $v_{max}$ and the radius $r_{max}$ at which it occurs, varying over two choices of feedback model and two prescriptions for the SMHM relation that populate dark matter halos with physically distinct galaxies. While simple empirical scaling relations can recover the median inferred value for $v_{max}$ and $r_{max}$, this approach provides realistic correlated uncertainties and aids interpretability through variation of the model. For these different models, we also demonstrate how the internal properties of a satellite's dark matter profile correlate with its orbit, and we show that it is difficult to reproduce observations of the Fornax dwarf without strong baryonic feedback. The technique developed in this work is flexible in its application of observational data and can leverage arbitrary information about the satellite galaxies to make inferences about their dark matter halos and population statistics.
No ratings yet
View paper →