I am a postdoctoral fellow with Prof Finale Doshi-Velez in the Data to Actionable Knowledge group.

I obtained my PhD in the Computational and Biological Learning lab at the University of Cambridge, supervised by Prof Richard E Turner and advised by Prof Carl Rasmussen. During my PhD, I designed algorithms for large-scale machine learning systems that learn sequentially without revisiting past data, are private over user data, and are uncertainty-aware. My PhD was funded by an EPSRC DTP award and a Microsoft Research EMEA PhD Award. I also held an Honorary Vice-Chancellor’s Award from the Cambridge Trust.

Quick Links: Publications, Google Scholar, CV.


Research summary

I am interested in quickly and efficiently adapting machine learning models, and usually prefer a Bayesian approach to such problems. This includes scenarios such as continual / lifelong learning (data arrives sequentially and past data cannot all be revisited), federated learning (data is split among many different clients), and unlearning (a subset of the training data must be forgotten). An example application of federated learning is when training a healthcare model on patient data from different hospitals, where sensitive patient data must not leave local hospital databases. Bayesian-inspired approaches have a lot of potential to work well in such adaptation scenarios, and I therefore also work on understanding and improving approximate Bayesian inference techniques.

I am also interested in human-computer interaction, looking at what ML model outputs to show a human in order to maximise the human-AI team’s performance (going beyond the usual accuracy metrics to jointly consider other objectives such as time taken and the human’s learning/enjoyment).


News

Nov 2023 Paper published in Transactions on Machine Learning Research, Improving Continual Learning by Accurate Gradient Reconstructions of the Past.
Jul 2023 Paper at Advances in Approximate Bayesian Inference Symposium 2023, Improving Continual Learning by Accurate Gradient Reconstructions of the Past.

Paper at Duality Principles for Modern ML Workshop at ICML 2023, Memory Maps to Understand Models.

Two papers at the AI&HCI workshop at ICML 2023, Adaptive interventions for both accuracy and time in AI-assisted human decision making, and Discovering User Types: Mapping User Traits by Task-Specific Behaviors in Reinforcement Learning.

Paper at Challenges of Deploying Generative AI Workshop at ICML 2023, Soft prompting might be a bug, not a feature.
Jun 2023 Invited speaker at the Bayes-Duality Workshop in Tokyo, Japan.
Apr 2023 Paper published in Transactions on Machine Learning Research, Differentially private partitioned variational inference.
Dec 2022 Organised the Continual Lifelong Learning Workshop at the Asian Conference on Machine Learning, 2022. As part of the conference, I also was a mentor in the Mentorship Program, chaired an invited talk session, and chaired a paper talks session.
Jul 2022 I have started as a postdoctoral fellow in Harvard University, with Prof Finale Doshi-Velez in the Data to Actionable Knowledge group.
Jun 2022 Invited talk at the Workshop on Continual Learning in Computer Vision at CVPR 2022 on “Knowledge-adaptation priors for continual learning”.
Mar 2022 I successfully defended my PhD thesis, “Probabilistic Continual Learning using Neural Networks”, available online here.
Dec 2021 Paper at NeurIPS 2021, Knowledge-Adaptation Priors.

Second paper at NeurIPS 2021, Collapsed Variational Bounds for Bayesian Neural Networks.

Gave part of an invited talk at Bayesian Deep Learning workshop (at NeurIPS 2021), jointly with Emtiyaz Khan and Dharmesh Tailor, Adaptive and Robust Learning with Bayes.
Nov 2021 I have written two blog posts on natural-gradient variational inference (NGVI). The first part motivates and derives equations for NGVI on neural networks. The second part scales to large datasets/architectures such as ImageNet/ResNets, following Osawa et al. (2019).
Jul 2021 Invited oral at Theory and Foundations of Continual Learning workshop (ICML 2021), “Continual Deep Learning with Bayesian Principles”.

Two invited talks at Microsoft Research Cambridge, UK, at the Machine learning reading group and Healthcare intelligence reading group, on “Continual Deep Learning with Bayesian Principles”.
Jun 2021 Invited talks at University of Toronto, Canada; DtAK lab, Harvard University, USA; and OATML, University of Oxford, UK, on Continual Deep Learning by Functional Regularisation of Memorable Past (Pan et al., 2020).
Jan 2021 Paper at ICLR 2021, Generalized Variational Continual Learning.
Dec 2020 Oral presentation at NeurIPS 2020, Continual Deep Learning by Functional Regularisation of Memorable Past (top 1% of submissions, 105/10K).

Paper at NeurIPS 2020, Efficient Low Rank Gaussian Variational Inference for Neural Networks.
Jul 2020 Oral at LifeLongML Workshop (ICML 2020), Combining Variational Continual Learning with FiLM Layers.

Second oral at LifeLongML Workshop (ICML 2020), Continual Deep Learning by Functional Regularisation of Memorable Past.
Jun 2020 I have been awarded a Microsoft Research EMEA PhD Award to fund my research on function-space Bayesian neural networks for continual learning and federated learning. Also see news article.
Dec 2019 Paper at NeurIPS 2019, Practical Deep Learning with Bayesian Principles.
Dec 2018 Oral at Continual Learning Workshop (NeurIPS 2018), Improving and Understanding Variational Continual Learning.

Spotlight at Bayesian Deep Learning Workshop (NeurIPS 2018), Partitioned Variational Inference: A unified framework encompassing federated and continual learning.

Paper at Advances in Approximate Bayesian Inference Symposium (2018), Neural network ensembles and variational inference revisited.
Jun 2018 Internship at Microsoft Research, Cambridge, supervised by John Winn and Martin Kukla, on knowledge-base construction.
Oct 2017 Started my PhD with Professor Richard Turner at the Machine Learning Group.