Shreyansh Singh
  • About
  • Posts
  • Projects
  • Publications
  • Bookshelf
  • CV
  • mlsys
  • •

  • transformer
  • •

  • paper-summaries
  • •

  • MLSys
  • •

  • LLMs
  • •

  • PPML
  • Paper Summary #9 - Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

    Understanding Sophia - A new fast, scalable second-order optimizer which beats Adam on LLM pretraining.

    31 min read   ·   May 28, 2023

    2023   ·   transformer   optimizer   deep-learning   paper-summaries   ·   Deep Learning   ML Theory

  • Paper Summary #8 - FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

    Understanding FlashAttention which is the most efficient exact attention implementation out there, which optimizes for both memory requirements and wall-clock time.

    37 min read   ·   March 26, 2023

    2023   ·   mlsys   transformer   efficiency   attention   paper-summaries   ·   MLSys

  • Paper Summary #7 - Efficient Transformers: A Survey

    A survey paper of improvements over the original Transformer architecture in terms of memory-efficiency.

    24 min read   ·   October 10, 2022

    2022   ·   mlsys   transformer   efficiency   attention   paper-summaries   ·   MLSys   LLMs

  • Deploying Machine Learning models using GCP's Google AI Platform - A Detailed Tutorial

    A step-wise tutorial to demonstrate the steps required to deploy a ML model using GCP, specifically the Google AI Platform and use Streamlit to access the model through a UI.

    21 min read   ·   March 06, 2022

    2022   ·   model-deployment   gcp   streamlit   ·   MLOps

  • Deploying Machine Learning models using AWS Lambda and Github Actions - A Detailed Tutorial

    A step-wise tutorial to demonstrate the steps required to deploy a ML model using AWS Lambda, Github Actions, API Gateway and use Streamlit to access the model API through a UI.

    21 min read   ·   January 23, 2022

    2022   ·   model-deployment   aws   streamlit   github-actions   ·   MLOps

  • PPML Series #3 - Federated Learning for Mobile Keyboard Prediction

    Understanding how your mobile keyboard (Gboard, specifically) performs the next word prediction task and performs model training and updates

    17 min read   ·   December 27, 2021

    2021   ·   federated learning   ppml   paper-summaries   ·   PPML

  • PPML Series #2 - Federated Optimization Algorithms - FedSGD and FedAvg

    A mathematical deep dive on a Federated Optimization algorithm - FedAvg and comparing it with a standard approach - FedSGD.

    16 min read   ·   December 18, 2021

    2021   ·   federated learning   ppml   paper-summaries   ·   PPML

  • PPML Series #1 - An introduction to Federated Learning

    A short general introduction to Federated Learning (FL) for folks interested in privacy-preserving machine learning (PPML).

    10 min read   ·   December 11, 2021

    2021   ·   federated learning   ppml   paper-summaries   ·   PPML

  • Paper Summary #6 - Language Models are Unsupervised Multitask Learners

    The GPT2 model which aimed to perform complex NLP tasks while relying only on a language model trained in a completely unsupervised fashion.

    19 min read   ·   May 23, 2021

    2021   ·   llm   transformer   paper-summaries   ·   LLMs

  • Paper Summary #5 - XLNet: Generalized Autoregressive Pretraining for Language Understanding

    XLNet tries to overcome the limitations of BERT by having a autoregressive component while also capturing the bidirectional context.

    28 min read   ·   May 16, 2021

    2021   ·   llm   transformer   paper-summaries   ·   LLMs   NLP

  • Newer
  • 1
  • 2
  • 3
  • 4
  • Older
© Copyright 2025 Shreyansh Singh.