Hire Dante L. - Prompt Engineer - San Francisco

* Investigating the effect of architectural inductive biases on feature learning in different self-supervised learning methods, MatrixSSL and Spectral Contrastive Learning (SCL), which focus on aligning second-order vs first-order statistics in the representations respectively. * Found a second-order moment-based assumption of embeddings that minimizes the alignment loss in * MatrixSSL, analogous to mean-based assumption for SCL from prior studies. Analyzed the * implications of these assumptions (for various embedding functions and toy distributions) on feature learning. * Trained and evaluated representations learnt by these methods on synthetic datasets with various * augmentation schemes. Code available at https://github.com/dante-hl/matrixssl-inductive * Analysis and simulations suggest that, when data are marginally Gaussian with correlations amongst positive pairs in specific feature directions, and embedding functions are linear or two layer ReLU networks, neither method outperforms the other at learning representations. * Proved that under a simple quadratic transformation and a modification to the loss terms, any solution to a second-order moment loss could be in principle achieved by SCL

* Worked directly under postdoc Richard Lange * Implemented Automated Differentiation Variational Inference (ADVI) for the Bayes-kit library with constant and adaptive learning rates. * Developed scripts to run inference algorithms including Hamiltonian Monte Carlo (HMC), ADVI, and the novel algorithm Interpolated Sampling and Variational Inference (ISVI) on a variety of distributions from posteriordb. * Investigated the performance of ISVI against HMC and ADVI by plotting MMD and Wasserstein Distance against time (quality of samples from the resulting algorithms relative to reference samples). * Developed a working proficiency in git and Docker.