Atri Rudra's Publications
*
[DBLP Listing] [Google Scholar listing]
(Papers are ordered in reverse chronological order of
first publication)
Drafts
2024
- Simple linear attention language models balance the recall-throughput tradeoff
-
Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, Dylan Zinsley, James Zou, Atri Rudra, Christopher Re.
-
Proceedings of the 41st International Conference on Machine Learning (ICML) July 2024.
-
Spotlight
-
ES-FoMo@ICML2024 Best Paper Award
-
[arXiv], [Code], [Blogpost: one, two]
- Zoology: Measuring and Improving Recall in Efficient Language Models
-
Simran Arora, Sabri Eyuboglu, Aman Timalsina, Isys Johnson, Michael Poli, James Zou, Atri Rudra, Christopher Re.
-
Proceedings of 12th International Conference on Learning Representations (ICLR) May 2024.
-
[arXiv], [Code], [Blogpost: one, two]
2023
- Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
-
Daniel Y. Fu, Simran Arora, Jessica Grogan, Isys Johnson, Sabri Eyuboglu, Armin W. Thomas, Benjamin Spector, Michael Poli, Atri Rudra and Christopher Re.
-
Proceedings of the 36th Neural Information Processing Systems Conference (NeurIPS) December 2023.
-
Oral
-
[arXiv], [Code], [Blogpost]
- Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
-
Stefano Massaroli, Michael Poli, Daniel Y. Fu, Hermann Kumbong, Rom N. Parnichkun, Aman Timalsina, David W. Romero, Quinn McIntyre, Beidi Chen, Atri Rudra, Ce Zhang, Christopher Re, Stefano Ermon and Yoshua Bengio
-
Proceedings of the 36th Neural Information Processing Systems Conference (NeurIPS) December 2023.
-
[arXiv]
- Simple Hardware-Efficient Long Convolutions for Sequence Modeling
-
Daniel Y. Fu, Elliot L. Epstein, Eric Nguyen, Armin W. Thomas, Michael Zhang, Tri Dao, Atri Rudra and Christopher Re.
-
Proceedings of the 40th International Conference on Machine Learning (ICML) July 2023.
-
[arXiv], [Code]
- Hungry Hungry Hippos: Towards Language Modeling with State Space Models
-
Tri Dao, Daniel Y. Fu, Khaled K. Saab, Armin W. Thomas, Atri Rudra and Christopher Re.
-
Proceedings of the 11th International Conference on Learning Representations (ICLR) May 2023.
-
Spotlight
-
[arXiv], [Code]
- How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
-
Albert Gu*, Isys Johnson*, Aman Timalsina, Atri Rudra and Christopher Re.
-
Proceedings of the 11th International Conference on Learning Representations (ICLR) May 2023.
-
[arXiv], [Code]
- Arithmetic Circuits, Structured Matrices and (not so) Deep Learning
-
Atri Rudra
-
Theory of Computing Systems, volume 67, pages 592-626, 2023.
-
[arXiv]
- Technical Perspective: (Pre-) Semirings Come to the Recursion Party
-
Atri Rudra
-
SIGMOD Record, March 2023.
-
2022
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
-
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra and Christopher Re.
-
Proceedings of the 35th Neural Information Processing Systems Conference (NeurIPS) December 2022.
-
[arXiv] [Code] [IEEE Specturm article] [Known usage]
-
- Monarch: Expressive Structured Matrices for Efficient and Accurate Training
-
Tri Dao, Beidi Chen, Nimit Sohoni, Arjun Desai, Michael Poli, Jessica Grogan, Alexander Liu, Aniruddh Rao, Atri Rudra and Christopher Re.
-
Proceedings of the 39th International Conference on Machine Learning (ICML) July 2022.
-
Long talk
-
Outstanding Paper Runner Up
-
[arXiv] [Code]
-
- A qualitative, network-centric method for modeling socio-technical systems, with applications to evaluating interventions on social media platforms to increase social equality
-
Kenneth Joseph, Huei-Yen Winnie Chen, Stefania Ionescu, Yuhao Du, Pranav Sankhe, Aniko Hannak and Atri Rudra
-
Applied Network Science volume 7, Number 49 (2022)
-
- Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
-
Beidi Chen, Tri Dao, Kaizhao Liang, Jiaming Yang, Zhao Song, Atri Rudra and Christopher Re.
-
Proceedings of the 10th International Conference on Learning Representations (ICLR) May 2022.
-
Spotlight
-
[arXiv]
[Code]
[Blogpost]
-
2021
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers
-
Albert Gu, Isys Johnson, Karan Goel, Khaled Kamal Saab, Tri Dao, Atri Rudra and Christopher Re
-
Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS) December 2021.
-
[arXiv][Code]
- Scatterbrain: Unifying Sparse and Low-rank Attention
-
Beidi Chen, Tri Dao, Eric Winsor, Zhao Song, Atri Rudra and Christopher Re
-
Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS) December 2021.
-
[arXiv][Code]
2020
- HiPPO: Recurrent Memory with Optimal Polynomial Projections
-
Albert Gu Tri Dao, Stefano Ermon, Atri Rudra and Christopher Re
-
To appear in the Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS) December 2020.
-
Spotlight.
-
[arXiv][Code][Blog post]
- Transformative Social Innovation as a Lens for ML for Good
-
Melanie Sage, Atri Rudra, Kenneth Joseph, Huei-Yen Chen and Varun Chandola
-
In CSCW 2020 Workshop: Collective Organizing and Social Responsibility, October 2020.
-
Not peer-reviewed
-
Sparse Recovery for Orthogonal Polynomial Transforms
-
Anna Gilbert Albert Gu, Christopher Re, Atri Rudra and Mary Wootters
-
In the Proceedings of 47th International Colloquium on Automata Languages and Programming (ICALP), July 2020.
-
[arXiv]
- Kaleidoscope: An Efficient Learnable Representation For All Structured Linear Maps
-
Tri Dao Nimit Sohoni, Albert Gu, Matthew Eichhorn, Amit Blonder, Megan Leszczynski, Atri Rudra, Christopher Re
-
In the Proceedings of 8th International Conference on Learning Representations (ICLR) April 2020.
-
Spotlight.
-
[Code][Blog post]
2019
2018
- Learning Compressed Transforms with Low Displacement Rank
-
Anna T. Thomas* Albert Gu*, Tri Dao, Atri Rudra, Christopher Re
-
Proceedings of the 31st Neural Information Processing Systems Conference (NeurIPS) December 2018.
-
[arXiv][Code]
- Hypertree Decompositions Revisited for PGMs
-
Aarthy Shivram Arun Sai Vikneshwar Mani Jayaraman, Christopher Re and Atri Rudra
-
Star AI July 2018.
-
[arXiv]
- General Strong Polarization
-
Jaroslaw Blasiok Venkatesan Guruswami, Preetum Nakkiran, Atri Rudra and Madhu Sudan
-
Proceedings of the 50th Annual ACM Symposium on the Theory of Computing (STOC) June 2018.
-
[arXiv ECCC]
- Learning Invariance with Compact Transforms
-
Anna T. Thomas Albert Gu, Tri Dao, Atri Rudra and Christopher Re
-
ICLR Workshop Track April 2018.
-
Full version in NeurIPS 2018 above.
- A Two-pronged Progress in Structured Dense Matrix Vector Multiplication
-
Christopher De Sa Albert Gu, Rohan Puttagunta, Christopher Re and Atri Rudra
-
Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) January 2018.
-
[arXiv]
- Average-radius list-recoverability of random linear codes
-
Atri Rudra and Mary Wootters
-
Proceedings of the Twenty-Ninth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) January 2018.
-
[arXiv]
2017
2016
2015
- The Range of Topological Effects on Communication
-
Arkadev Chattopadhyay and Atri Rudra
-
Proceedings of ICALP (Track C) July 2015.
-
[arXiv]
- A Multiple Server Scheme for Fingerprint Fuzzy Vaults
-
Jesse Hartloff Matthew Morse, Bingsheng Zhang, Thomas Effland, Jennifer Cordaro, Jim Schuler, Sergey Tulyakov, Atri Rudra and Venu Govindaraju
-
Proceedings of CVPR Workshop on Biometrics June 2015.
-
[PDF]
- Join Processing for Graph Patterns: An Old Dog with New Tricks
-
Dung Nguyen Molham Aref, Martin Bravenboer, George Kollias, Hung Q. Ngo, Christopher Re and Atri Rudra
-
Proceedings of GRADES 2015 (co-located with SIGMOD/PODS 2015) June 2015.
-
[arXiv]
-
Joins via Geometric Resolutions: Worst-case and Beyond
-
Mahmoud Abo Khamis Hung Q. Ngo, Christopher Re and Atri Rudra
-
Proceedings of the 34th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS) June 2015.
-
Journal version: ACM Transactions on Database Systems (TODS) 41(4), December 2016.
-
[arXiv]
-
[TODS version]
-
It'll probably work out: improved list-decoding through random operations
-
Atri Rudra and Mary Wootters
-
Proceedings of 6th Innovations in Theoretical Computer Science (ITCS) January 2015.
-
[arXiv ECCC]
2014
-
Secure fingerprint hashes using subsets of local structures
-
Tom Effland Mariel Schneggenburger, Jim Schuler, Bingsheng Zhang, Jesse Hartloff, Jimmy Dobler, Sergey Tulyakov, Atri Rudra and Venu Govindaraju
-
Proceedings of SPIE 9075 Biometric and Surveillance Technology for Human and Activity Identification XI, May 2014.
-
[PDF]
- Beyond worst-case analysis for joins with minesweeper
-
Hung Q. Ngo Dung T. Nguyen, Christopher Re and Atri Rudra
-
Proceedings of the 33rd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS) June 2014.
-
[arXiv]
-
Every list-decodable code for high noise has abundant near-optimal rate puncturings
-
Atri Rudra and Mary Wootters
-
Proceedings of the 46th Annual Symposium on the Theory of Computing (STOC) June 2014.
-
(Blog mentions: one two)
-
[arXiv ECCC]
-
Secure Fingerprint MatchingWith Generic Local Structures
-
Matthew Morse Jesse Hartloff, Thomas Effland, Jim Schuler,
Jennifer Cordaro Sergey Tulyakov, Atri Rudra and Venu Govindaraju
-
Proceedings of IEEE Computer Society Workshop on Biometrics (CVPR-W) June 2014.
-
[PDF]
-
Energy Aware Algorithmic Engineering
-
Swapnoneel Roy Atri Rudra and Akshat Verma
-
Proceedings of the IEEE 22nd International Symposium on Modeling Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS), September 2014.
-
[PDF]
-
Topology matters in communication
-
Arkadev Chattopadhyay Jaikumar Radhakrishnan and Atri Rudra
-
Proceedings of 55th Annual IEEE Symposium on
Foundations of Computer Science (FOCS) October 2014.
-
[ECCC]
2013
- An energy complexity model for algorithms
-
Swapnoneel Roy Atri Rudra and Akshat Verma
-
Proceedings of the 4th Innovations in Theoretical Computer Science (ITCS) January 2013.
-
[PDF]
-
Security analysis for fingerprint fuzzy vaults
-
Jesse Hartloff Maxwell Bileschi, Sergey Tulyakov, Jimmy Dobler, Atri Rudra and Venu Govindaraju
-
SPIE (Conference on Biometric and Surveillance Technology for Human and Activity Identification X) volume 8712, May 2013.
-
[PDF]
-
Towards fingerprints as strings: Secure indexing for fingerprint matching
-
Jesse Hartloff Jimmy Dobler, Sergey Tulyakov, Atri Rudra and Venu Govindaraju
-
Proceedings of the 6th IAPR International Conference on Biometrics (ICB) June 2013.
-
[PDF]
- l2/l2-Foreach Sparse Recovery with Low Risk
-
Anna C. Gilbert Hung Q. Ngo, Ely Porat, Atri Rudra and Martin J. Strauss
-
Proceedings of the 40th International Colloquium on Automata Languages, and Programming (ICALP), July 2013.
-
[arXiv]
-
Accurate Decoding of Pooled Sequenced Data Using Compressed Sensing
-
Denisa Duma Mary Wootters, Anna C. Gilbert, Hung Q. Ngo, Atri Rudra, Matthew Alpert, Timothy J. Close, Gianfranco Ciardo and Stefano Lonardi
-
Proceedings of the 13th International Workshop on Algorithms in Bioinformatics (WABI) September 2013.
-
[arXiv]
-
Skew strikes back: new developments in the theory of join algorithms
-
Hung Q. Ngo Christopher Re and Atri Rudra
-
SIGMOD Record 42(4) Pgs. 5-16, December 2013.
-
Invited paper.
-
[Blog mention: one]
-
[arXiv]
2012
2011
2010
- Efficiently Decodable Non-adaptive
Group Testing
-
Piotr Indyk Hung Q. Ngo and Atri Rudra
-
Proceedings of the 21st Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). January 2010.
- [PDF]
- Analyzing Nonblocking Switching Networks using Linear Programming (Duality)
-
Hung Q. Ngo Atri Rudra, Anh N. Le and Thanh-Nhan Nguyen
-
Proceedings of the 29th IEEE International Conference on Computer Communications (INFOCOM) March 2010.
-
[arXiv]
- Data Stream Algorithms for Codeword Testing
-
Atri Rudra and Steve Uurtamo
-
Proceedings of the 37th International Colloquium on Automata Languages and Programming (ICALP), July 2010.
-
[arXiv]
- k+ Decision Trees
-
James Aspnes Eric Blais, Murat Demirbas, Ryan O'Donnell, Atri Rudra and Steve Uurtamo
-
Proceedings of the 6th International Workshop on Algorithms for Sensor Systems Wireless Ad Hoc Networks and Autonomous Mobile Entities (ALGOSENSORS), Pgs. 74-88. July, 2010.
- [Extended Abstract Full version]
- Two Theorems on List Decoding
-
Atri Rudra and
Steve Uurtamo
-
Proceedings of the 14th International Workshop on Randomization and Computation (RANDOM) September 2010.
-
[arXiv ECCC]
- When LP Is the Cure for Your Matching Woes: Improved Bounds for Stochastic Matchings
-
Nikhil Bansal Anupam Gupta, Jian Li, Julián Mestre, Viswanath Nagarajan and Atri Rudra
-
Algorithmica 63(4) Pgs. 733-762, 2012. (ESA special issue)
-
Preliminary version in Proceedings of the 18th Annual European Symposium on Algorithsm (ESA) September 2010.
-
Co-winner of the ESA Best paper award
-
[arXiv]
2009 and earlier (with abstracts)
2009
2008
2007
2006
2005
2004
- Floodlight Illumination of Infinite Wedges
-
Matthew Cary Ashish Sabharwal , Atri Rudra and Erik Vee
-
Abstract in the 14th Annual Fall Workshop on Computational Geometry. November 2004.
-
Full version invited to Computation Geometry: Theory and Applications (CGTA)
-
Testing Low-Degree Polynomials Over Prime Fields
-
Charanjit S. Jutla Anindya C. Patthak, Atri Rudra and David Zuckerman
-
Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science (FOCS) Pgs 423-432. October 2004.
2003
2001
Unpublished Manuscripts and Technical Reports
Copyright notice: The documents distributed by this server have
been
provided as a means to ensure timely dissemination of scholarly and
technical work on a non-commercial basis. Copyright © and all
rights therein are maintained by the authors or by other copyright
holders notwithstanding that they have offered their works here
electronically. It is understood that all persons copying this
information will adhere to the terms and constraints invoked by each
author's copyright. These works may not be reposted without the
explicit permission of the copyright holders. ACM published documents
are ©
Copyright 199x by ACM Inc.; Springer-Verlag published documents
are ©
Springer-Verlag; and IEEE published documents are ©
199x IEEE under these
conditions.