Repositories list

• ZClip

  Public
  Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".

  lossgradient-clippingadaptive-gradient-clipping+ 10

  lossgradient-clippingadaptive-gradient-clippingloss-spikeadaptive-clippingzclipstable-llm-pretrainingenable-high-learning-ratetraning-stabilitypre-training-stability+ 3

  Python

  •

  Apache License 2.0

  •1111 forks•153153 stars•33 issues•44 pull requests•Updated Jul 20, 2026Jul 20, 2026

• perftest

  Public
  Shell

  •00 forks•00 stars•00 issues•00 pull requests•Updated Nov 3, 2025Nov 3, 2025

• weight_rescaling

  Public
  Official implementation of the "Variance control via weight rescaling in LLM pretraining" paper.

  Python

  •

  Apache License 2.0

  •00 forks•55 stars•00 issues•00 pull requests•Updated Jun 29, 2025Jun 29, 2025

• refine_massive_activations

  Public
  Official implementation of the paper: "A Refined Analysis of Massive Activations in LLMs".

  Python

  •

  MIT License

  •33 forks•1111 stars•00 issues•00 pull requests•Updated May 21, 2025May 21, 2025

• raydp

  Public
  RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.

  Scala

  •

  Apache License 2.0

  •8484 forks•00 stars•00 issues•00 pull requests•Updated Apr 22, 2025Apr 22, 2025

• Megatron-LM

  Public
  Ongoing research training transformer models at scale

  Python

  •

  Other

  •4.6k4.6k forks•11 star•00 issues•00 pull requests•Updated Mar 21, 2025Mar 21, 2025

• vllm-production-stack

  Public
  vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

  Python

  •

  Apache License 2.0

  •514514 forks•00 stars•00 issues•00 pull requests•Updated Mar 14, 2025Mar 14, 2025

• lingua

  Public
  Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •269269 forks•00 stars•00 issues•00 pull requests•Updated Feb 6, 2025Feb 6, 2025

• flash-attention

  Public
  Fast and memory-efficient exact attention

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •3.1k3.1k forks•00 stars•00 issues•00 pull requests•Updated Jan 16, 2025Jan 16, 2025

• torchx_nccl_test

  Public
  Python

  •00 forks•00 stars•00 issues•00 pull requests•Updated Jan 15, 2025Jan 15, 2025

• blt

  Public
  Code for BLT research paper

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •196196 forks•00 stars•00 issues•00 pull requests•Updated Jan 10, 2025Jan 10, 2025

• check_nccl

  Public
  Python

  •00 forks•00 stars•00 issues•00 pull requests•Updated Nov 1, 2024Nov 1, 2024

• netifaces

  Public
  C

  •

  MIT License

  •8787 forks•00 stars•00 issues•00 pull requests•Updated Oct 7, 2024Oct 7, 2024

• AdEMAMix-Optimizer-Pytorch

  Public
  The AdEMAMix Optimizer: Better, Faster, Older.

  Python

  •

  MIT License

  •99 forks•00 stars•00 issues•00 pull requests•Updated Oct 7, 2024Oct 7, 2024

• wheels

  Public
  00 forks•00 stars•00 issues•00 pull requests•Updated Oct 4, 2024Oct 4, 2024

• beam

  Public
  Apache Beam is a unified programming model for Batch and Streaming data processing.

  Java

  •

  Apache License 2.0

  •4.7k4.7k forks•00 stars•00 issues•00 pull requests•Updated Oct 4, 2024Oct 4, 2024

• lighteval

  Public
  LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and…

  Python

  •

  MIT License

  •567567 forks•00 stars•00 issues•00 pull requests•Updated Aug 5, 2024Aug 5, 2024

• rules_python

  Public
  Bazel Python Rules

  Starlark

  •

  Apache License 2.0

  •725725 forks•00 stars•00 issues•00 pull requests•Updated Jul 31, 2024Jul 31, 2024

• flash-attention-wheels

  Public
  C++

  •22 forks•00 stars•00 issues•00 pull requests•Updated Jul 29, 2024Jul 29, 2024