Manasi Sharma

Hi there! I am a second year MS student in Computer Science at Stanford University on the Artificial Intelligence track. I am currently member of the Stanford Vision Lab under Prof. Fei-Fei Li and Prof. Prof. Jiajun Wu, where I work on the BEHAVIOR project in robotic simulation benchmarking and the ADDA (Attention-driven data augmentation) project which aims to improve image classification performance using a saliency-map based augmentation. I consider myself a generalist in AI and am interested in the broad domains of Deep Learning, Computer Vision, Explainability, Reinforcement Learning & Decision Making, Graph Neural Networks. I have also recently joined the ILIAD Lab under Prof. Dorsa Sadigh, and I am heading a new project on 'Gaze & Language-assisted Instruction Corrections for Imitation Learning'.

I graduated in 2021 from Columbia University with a Bachelor's degree in Computer Science and Physics. As an undergrad, I was supervised by Prof. Daniel Hsu and Prof. Zoltan Haiman on interpreting astrophysical deep learning models for weak lensing using an array of saliency map methods. I started off my undergraduate program with a broad interest in computational physics and first pursued the direction of AI at Caltech in 2019, under Prof. Mansi Kasliwal in which I developed a real/bogus image astrophysical source classifier for the IR-Gattini telescope that performed so well that it was permanently included in the telescope's data processing pipeline.

As I entered graduate school, unfulfilled by the relatively limited impact astrophysics can have on current-life and real-world affairs, my interest in AI broadened to include more real-world applications that had a greater focus on human and human-computer interaction, such as robotics and AV. I am now looking for full-time roles (starting 2023) and am very excited to learn about opportunities for the application of deep-learning and/or perception-based systems to a variety of scenarios!

Email  /  CV  /  Github  /  Google Scholar  /  LinkedIn  /  Twitter



Stanford University, School of Engineering

Master's in Computer Science with a Research Distinction (Thesis)
Enrolled in September 2021
Advisors: Fei-Fei Li, Jiajun Wu (Stanford Vision Lab)

Columbia University, Columbia College

BA in Computer Science (major) and Physics (minor)
Graduated in May 2021
Advisors: Daniel Hsu, Zoltan Haiman

Professional / Research Experience


Nissan-Renault-Mitsubishi: Alliance Innovation Laboratory (AIL-SV)

[June 2022 -- Sep 2022]

  • Implemented LiDAR point-cloud classification of for Autonomous Systems using the 'SimpleView' supervised learning algorithm, which projects a 3D point-cloud onto the 6 2D frames and passes it through a CNN (ResNet backbone).
  • The model was able to classify cars, pedestrians, cyclists and vegetation with over 95% accuracy on real-world data and within the 10 Hz LiDAR processor timeframe.
  • Experimented with a number of architechtures including Graph-based methods like Grid-GCN and point-wise MLP methods like PointNet++, finally settling on the lightweight convolution based method SimpleView.
  • LiDAR point-cloud classification is exceedingly difficult and I encountered numerous challenges-- I was finally able to improve model performance and speed by sparisfying the network, using saliency-based weight-freezing and training on hard-negative samples.
  • The model will be deployed in Nissan Autonomous Vehicles beginning October '22.
  • obj

    Stanford University, Stanford Vision Laboratory

    [Oct 2021 -- present]

    BEHAVIOR Project:
  • Led the development of the Knowledgebase for iGibson and BEHAVIOR-1K, an ImageNet-scale robotic simulation benchmark with a specifc-focus on human-relevant design. Paper accepted to the Conference of Robotics and Learning '22.
  • Mobilized ~20 crowd-workers to categorize ~5000 "how-to" articles and used zero-shot Natural Language Processing techniques with GPT-3 to annotate the Virtual Reality videos and generate >97% quality activity definitions in a predicate logic-based language.
  • ADDA (Attention-driven data augmentation):
  • Oversaw 5 person team project on 'Modulated Attention Dropout' technique to allow for better generalization of RL policies through task-importance aware dataset augmentation. Results showed a 2% increase on baseline Behavioral Cloning results.
  • obj

    Columbia University, Data Science Institute

    [Sep 2019 -- June 2021]
  • Under the guidance of Prof.'s Daniel Hsu and Zoltan Haiman, worked on targeting the "explainability" & trustworthiness of neural networks in the more traditional field of Astronomy.
  • Discovered that 89% of the output of a popular neural network that uses gravitational lensing maps to predict cosmological parameters (omega_m and sigma_8) was counterintuitively attributable to negative image regions (voids, black holes, etc.) as opposed to the bright image regions (stars, galaxies, etc.). Published results in APS Physical Review '20.
  • Also ran sanity checks on a number of popular saliency methods and found that gradient-based methods (eg. Grad-CAM / Input x Gradients, etc.) were most robust to model parameters.
  • obj

    California Institute of Technology, Division of Physics, Mathematics and Astronomy

    [Jun 2019 -- Aug 2019]
  • Under the guidance of Prof. Mansi Kasliwal at Caltech, pioneered the development of a flagship CNN-based real/bogus image classification system for Caltech's Gattini-IR Telescope using TensorFlow (link), which achieved ~97.5% accuracy on thousands of cosmic transient sources, and published results in PASP '20.
  • The model worked so well that it was deployed the model in the Telescope's data processing pipeline (still active), replacing the manual classification process.
  • I also used the results of the model to identify high-confidence transient sources that I performed optical followup on by operating the 200-inch telescope at the Palomar Observatory in Southern California.
  • obj

    Columbia University, Department of Physics

    [Jun 2018 -- May 2019]
  • Worked with Prof. Charles Hailey in the NuSTAR Group as a Laidlaw Research Intern.
  • Modeled/analyzed data from NASA's NuSTAR telescope for 'AM Her' & 'HU Aqr' sources to determine key parameters such as temperature and periodicity.
  • Publications

    obj BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation
    Chengshu Li, Cem Gokmen, ... Manasi Sharma...,
    Conference on Robot Learning (CoRL), 2022
    Paper link will be posted when it's made publicly available on arXiv.

    We present BEHAVIOR-1K, a comprehensive simulation benchmark for human-centered robotics, motivated by the results of an extensive survey on `what do you want robots to do for you?'. It includes the definition of 1,000 everyday activities, grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more than 3,000 objects annotated with physical and semantic properties. It also includes OmniGibson, a novel simulator that supports these activities via realistic physics simulation and rendering of rigid bodies, deformable bodies, and liquids.

    obj Interpreting deep learning models for weak lensing
    Jose Manuel Zorrilla Matilla, Manasi Sharma, Daniel Hsu, Zoltan Haiman
    American Physical Society, PHYSICAL REVIEW D, 2020
    Paper link

    Deep neural networks (DNNs) are powerful algorithms that have been proven capable of extracting non-Gaussian information from weak lensing (WL) datasets. We apply a series of well-established saliency methods to interpret the DNN and find that the most relevant pixels are those with extreme K values. For noiseless maps, regions with negative K account for 69%-89%. of the attribution of the DNN output, defined as the square of the saliency in input space. In the presence of shape noise, the attribution concentrates in high-convergence regions, with 36%-68% of the attribution in regions with highest 3rd standard deviation of K values.

    obj Palomar Gattini-IR: Survey Overview, Data Processing System, on-Sky Performance and First Results
    Kishalay De, Matthew J. Hankins, ... Manasi Sharma, ...,
    Publications of the Astronomical Society of the Pacific, vol. 132., 2020
    Paper link

    Palomar Gattini-IR is a new wide-field, near-infrared (NIR) robotic time domain survey operating at Palomar Observatory. Using a 30 cm telescope mounted with a H2RG detector, Gattini-IR achieves a field of view (FOV) of 25 sq. deg. with a pixel scale of 8 7 in J-band. Here, we describe the system design, survey operations, data processing system and on-sky performance of Palomar Gattini-IR. To automatically distinguish between an astrophysical source and image subtraction artifacts, we use a ML based real-bogus (RB) classification scheme. Bogus candidates were compiled using a labeling scheme on Zooniverse, a citizen science web portal which allows set up of individual projects usually pertaining to classification and data visualization. The performance of the model was evaluated using the following metrics: accuracy on the test set of 0.975, a Matthews correlation coefficient of 0.949 and an F1 score of 0.977.

    Teaching Experience


    Stanford School of Engineering, CS231N Course (Deep Learning for Computer Vision, Prof. Fei-Fei Li)

    [Mar 2022 -- Jun 2022]
    Graduate Teaching Assistant
  • Managed weekly 'Discussion Sections' of 75+ students for one of the most popular CS courses at Stanford (>500 students); held office hours, constructed & graded HWs. Received >95% excellent reviews ('Very/Extremely Effective').
  • Taught and covered topics such as backpropagation, convolutional neural networks, visual transformers & attention, RNNs, YOLO, object detection, etc.
  • obj

    Columbia University, MATH 1201-1202 (Calculus III and IV)

    [Mar 2022 -- Jun 2022]
    Columbia University, Department of Mathematics
  • Graded assignments, led weekly office hours, etc., for ~150 students in the undergraduate Calculus III class. Consistently received >80% excellent reviews.
  • Taught and covered topics such as multiple integrals, Green's theorem, vector calculus, Fourier analysis, etc.
  • AI Blog-- Manasi's Musings

      [June 2022 -- present]
    • In the summer of 2022, I started writing a bi/tri-weekly blog called 'Manasi's Musings' about an array of topics, from my personal journey to my thoughts on LiDAR vs. camera data in AVs to robotic benchmarking, etc. The goal is to cover a sub-topic or question of interest in bite-sized pieces, a maximum 3 min read. The readers are currently mostly friends, colleagues and peers in my network but I hope to reach a wider audience through social media. I started the blog as I missed the process of writing-- I had actually maintained an Astrophysics blog during my high school and early undergraduate program, but stopped due to the commitments of completing my degree. I am so excited to restart, and you can check it out here!


    Stanford Course Projects

      Crowd Aware Intent-based Reinforcement Learning - CS333 (Algorithms for Interactive Robotics)

      Winter '22
    • Reduced collision rate in crowd navigation by 50% by leveraging human latent intent reinforcement learning (link).
    • Predicting Drug Interactions with Graph Neural Networks - CS224W (Machine Learning with Graphs)

      Fall '21
    • Used the Graph Isomorphism Network to exceed 11th place on ogbl-ddli leaderboard (link, selected for course website).
    • Debiasing Models for Out-of-domain Generalization - CS224N (NLP for Deep Learning)

      Winter '22
    • Exceeded BERT's performance on out-of-domain question-answering data by 2.5% by using debiasing models (link).
    • Optimizing Wind Turbine Placement Subject to Turbine Wakes - CS238 (Decision Making Under Uncertainty)

      Fall '21
    • Applied Q-Learning to windfarms to generate sensible layouts that maximize power, subject to wake constraints (link).
    • LIMES: LIME for Image Segmentation - CS329T (Trustworthy Machine Learning)

      Spring '22
    • Devised a LIME algorithm variant for facial segmentation that achieves explainability like gradient-based methods.
    • Monte-Carlo Tree Search Player - CS227B (General Game Playing)

      Spring '22
    • Designed a player to play any game, using MCTS, multi-threating, grounding, etc.; placed 8th in the class (link).

    Honors & Awards

    • 1 of 18 accepted to the GFSD (Graduate Fellowships for STEM Diversity) Program. -- [Mar '22]
    • 1 of 50 accepted into Google's CS Research Mentorship Program (CSRMP), Class of 2022A. -- [Feb '22]
    • Selected for the final round of the GEM Fellowship. -- [Jan '22]
    • Dean's List (in 6 out of 7 graded semesters, awarded to top 20%), Columbia University. -- [Fall '17 - Fall '20]
    • Columbia Undergraduate Research Fellowship (URF), Columbia College Summer Funding Program. -- [May '20]
    • Visiting Undergraduate Research Program (VURP) Award, California Institute of Technology. -- [May '19]
    • 1 of 25 awarded Laidlaw Undergraduate Research & Leadership Scholarship, Columbia University. -- ['18 - '19]
    • Andy Grove Scholarship for Intel Employees' Children, Intel Foundation. -- [Feb '19]

    Leadership Roles and Extra-curriculars

    • Graduate Community Chair, Women in Computer Science, Stanford University. -- [Jun '22 - present]
    • Elected as Social Chair for GradSWE, Stanford University. -- [Jun '22]
    • Founder & Project Leader, COVID-19 Public Hub website highlighting Columbia research. -- [Jan '22]
    • Corporate Chair, Women in Computer Science, Columbia University. -- [Apr '20 - Jun '20]
    • Class 3 Curriculum Developer (AI section), Girls Who Code, Columbia University. -- [Feb '20 - Aug '20]
    • Executive Board UG Student Coordinator, Columbia Society for Women in Physics. -- [Sep '18 - Sep '19]
    • Captain, 'Columbia Raas' Dance Team (member since Sep 2017), Columbia University. -- [Apr '20 - Jun '21]
    • Crew Captain, New Student Orientation Program, Columbia University. -- [Aug '20 - Sep '20]
    • Jam Leader, Columbia Design Jam. -- [May '20 - Jul '20]
    • Co-President, 'Symposium in India' Student Club, Columbia University. -- [Sep '18 - May '19