# Research

Posted: 2017-08-07 , Modified: 2020-07-12

Tags: math

Parent: Math

Children: Online Sampling from Log-Concave Distributions, Simulated tempering Langevin Monte Carlo, l-adic properties of partition functions

Posted: 2017-08-07 , Modified: 2020-07-12

Tags: math

Parent: Math

Children: Online Sampling from Log-Concave Distributions, Simulated tempering Langevin Monte Carlo, l-adic properties of partition functions

I received my Ph.D. from Princeton, where I was advised by Sanjeev Arora (research group page, ML theory at Princeton).

I focus on machine learning theory and applied probability, and also have broad interests in theoretical computer science and related math.

Although machine learning (and deep learning in particular) has made great advances in recent years, our mathematical understanding of it is shallow. Learning problems can be highly nonconvex, yet tractable in practice. What hidden structure do these problems have, and how can we design algorithms to take advantage of it?

Current interests include:

**Bayesian inference/Probabilistic modeling**, with a focus on**sampling (MCMC) algorithms**: How to design provable algorithms for learning probability distributions and sampling from them? How can we improve classical algorithms like Markov Chain Monte Carlo, or test the quality of the samples?**Control theory and reinforcement learning**, with a focus on**learning dynamical systems**: It is a well-studied problem how to find the optimal control for known linear dynamical system. However, reinforcement learning deals with learning how to act unknown, combinatorially complex systems; algorithms are heuristic and slow. How to bridge this gap?**Neural networks**: Neural networks tackle highly nonconvex problems but do very well in practice. Why? What kind of algorithmic improvements can we come up with by understanding their theoretical foundations more deeply?**Natural language understanding**: Language is a fundamental part of human intelligence and a big frontier for machine learning. How do we create machines that can understand “grammar” and “semantics”?

The publication list is available as pdf.

[A] denotes alphabetical order of authors.

**Estimating Normalizing Constants for Log-Concave Distributions: Algorithms and Lower Bounds**[A] Rong Ge,

**Holden Lee**, and Jianfeng Lu.STOC 2020. [arXiv, pdf, STOC 2020:579–586, slides, video]

**Online Sampling from Log-Concave Distributions**[A]

**Holden Lee**, Oren Mangoubi, and Nisheeth Vishnoi.**Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo.**webpage[A] Rong Ge,

**Holden Lee**, and Andrej Risteski.

**No-Regret Prediction in Marginally Stable Systems**[A] Udaya Ghai,

**Holden Lee**, Karan Singh, Cyril Zhang, and Yi Zhang. [arxiv, pdf, slides, summary slide, videos]COLT 2020.

**Statistical Guarantees for Learning an Autoregressive Filter**[A]

**Holden Lee**and Cyril Zhang. [arxiv, pdf]ALT 2020.

**Spectral Filtering for General Linear Dynamical Systems**[A] Elad Hazan,

**Holden Lee**, Karan Singh, Cyril Zhang, and Yi Zhang. [arxiv, pdf]NeurIPS 2018 (oral).

**Towards Provable Control for Unknown Linear Dynamical Systems.**[A] Sanjeev Arora, Elad Hazan,

**Holden Lee**, Karan Singh, Cyril Zhang, and Yi Zhang. [ICLR page, pdf]ICLR workshop 2018.

**Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets.**Rohith Kuditipudi, Xiang Wang,

**Holden Lee**, Yi Zhang, Zhiyuan Li, Wei Hu, Rong Ge, and Sanjeev Arora.**On the Ability of Neural Nets to Express Distributions.****Holden Lee**, Rong Ge, Tengyu Ma, Andrej Risteski, and Sanjeev Arora. [arXiv, pdf, PMLR 65:1271-1296, webpage]COLT 2017.

**Quadratic polynomials of small modulus cannot represent OR.****Holden Lee**

**l-adic properties of partition functions.**[A] Eva Belmont,

**Holden Lee**, Alexandra Musat, and Sarah Trebat-Leder.Monatshefte für Mathematik, 173(1), 1-34, 2014. [arXiv, pdf, presentation, webpage]