Skip to content

@neuralmagic Neural Magic

All

    Repositories list

    114 repositories

    • vllm-vcr

      Public
      vLLM inference simulator and replay machine
      rustinferencevllm
      rustinferencevllm
      Rust
      •
      Apache License 2.0
      •2•10•15•15•Updated Oct 17, 2026Oct 17, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •23k•17•0•38•Updated Oct 16, 2026Oct 16, 2026

    • vllm-openshift-recipes

      Public
      0•0•0•0•Updated Oct 16, 2026Oct 16, 2026

    • compressed-tensors

      Public
      A safetensors extension to efficiently store sparse quantized tensors on disk
      Python
      •
      Apache License 2.0
      •135•0•0•0•Updated Oct 16, 2026Oct 16, 2026

    • llm-compressor

      Public
      State-of-the-art LLM compression, built for production inference with vLLM
      Python
      •
      Apache License 2.0
      •695•0•0•0•Updated Oct 16, 2026Oct 16, 2026

    • inspect_evals

      Public
      Collection of evals for Inspect AI
      Python
      •
      MIT License
      •461•0•0•10•Updated Oct 16, 2026Oct 16, 2026

    • speculators-dashboard

      Public
      HTML
      •0•1•34•0•Updated Oct 14, 2026Oct 14, 2026

    • nm-vllm-omni-ent

      Public
      A framework for efficient model inference with omni-modality models
      Python
      •
      Apache License 2.0
      •1.9k•3•0•4•Updated Oct 14, 2026Oct 14, 2026

    • DeepGEMM

      Public
      DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
      Cuda
      •
      MIT License
      •1.4k•0•0•5•Updated Oct 13, 2026Oct 13, 2026

    • evals-refresh

      Public
      Shell
      •2•0•0•2•Updated Oct 13, 2026Oct 13, 2026

    • vllm-beam-search-plugin

      Public
      Beam search scheduler plugin for vLLM v1 with CoW block table forking
      Python
      •
      Apache License 2.0
      •2•4•1•0•Updated Oct 13, 2026Oct 13, 2026

    • vllm-envs

      Public
      Fast disposable vLLM dev environments from any branch/commit, built on content-addressed layer caches
      Python
      •1•3•2•4•Updated Oct 12, 2026Oct 12, 2026

    • axolotl

      Public
      Go ahead and axolotl questions
      Python
      •
      Apache License 2.0
      •1.5k•0•0•6•Updated Oct 11, 2026Oct 11, 2026

    • transformers-gdm

      Public
      🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference an…
      Python
      •
      Apache License 2.0
      •35k•0•0•1•Updated Oct 9, 2026Oct 9, 2026

    • crucible

      Public
      Crucible: an autonomous goal-directed research loop engine
      kubernetesrustagents
      kubernetesrustagentsautonomous-researchllm
      Rust
      •
      Apache License 2.0
      •3•10•7•8•Updated Oct 9, 2026Oct 9, 2026

    • inspect_k8s_sandbox

      Public
      A Kubernetes sandbox environment for use with inspect_ai
      Python
      •
      MIT License
      •38•0•0•8•Updated Oct 8, 2026Oct 8, 2026

    • model-validation-configs

      Public
      2•2•1•6•Updated Oct 8, 2026Oct 8, 2026

    • eval-hub-contrib

      Public
      Community-contributed evaluation framework adapters for eval-hub
      Python
      •
      Apache License 2.0
      •27•0•0•2•Updated Oct 8, 2026Oct 8, 2026

    • DeepEP

      Public
      DeepEP: an efficient expert-parallel communication library
      Cuda
      •
      MIT License
      •1.5k•1•0•1•Updated Oct 7, 2026Oct 7, 2026

    • eval-hub

      Public
      Go
      •
      Apache License 2.0
      •28•0•0•1•Updated Oct 7, 2026Oct 7, 2026

    • nyann-bench

      Public
      Go
      •
      Apache License 2.0
      •2•2•0•12•Updated Oct 5, 2026Oct 5, 2026

    • llm-manifesto

      Public
      Declarative Kubernetes renderer for llm-d and vLLM deployments
      Python
      •
      Apache License 2.0
      •0•2•0•0•Updated Oct 1, 2026Oct 1, 2026

    • rh-custom-models-plugin

      Public
      Python
      •
      Apache License 2.0
      •0•0•0•0•Updated Oct 1, 2026Oct 1, 2026

    • research

      Public
      Repository to enable research flows
      Python
      •2•4•0•4•Updated Sep 17, 2026Sep 17, 2026

    • gpu-pruner

      Public
      Non-destructive GPU based idle-culler for RHOAI/Kubeflow workloads
      kubernetesrustopenshift
      kubernetesrustopenshiftgpuk8skubernetes-operatorkubeflowrhoaikubeflow-notebooks
      Rust
      •
      MIT License
      •4•6•0•3•Updated Sep 11, 2026Sep 11, 2026

    • fs-offload-experiments

      Public
      Benchmark sweep experiments for FS Offloading
      HTML
      •0•0•0•0•Updated Sep 3, 2026Sep 3, 2026

    • nm-hard-tools

      Public
      MCP services for rigorous inference evaluation and benchmarking
      Python
      •
      Apache License 2.0
      •0•0•0•2•Updated Sep 1, 2026Sep 1, 2026

    • tokenspeed

      Public
      TokenSpeed is a speed-of-light LLM inference engine.
      Python
      •
      MIT License
      •354•0•0•3•Updated Aug 19, 2026Aug 19, 2026

    • tpu-inference

      Public
      TPU inference for vLLM, with unified JAX and PyTorch support.
      Python
      •
      Apache License 2.0
      •344•0•0•0•Updated Aug 18, 2026Aug 18, 2026

    • speculators

      Public
      A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
      Python
      •
      Apache License 2.0
      •260•0•0•1•Updated Aug 13, 2026Aug 13, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.