🤗 Transformers Serve

A Gradio frontend that calls transformers serve over HTTP. Everything starts automatically — just run python app.py.

Serve API: http://127.0.0.1:8000
Model: google/gemma-4-26B-A4B-it
Mode: ZeroGPU (dynamic GPU allocation)
API endpoints: /v1/chat/completions (streaming + non-streaming)

Architecture

This Gradio app pre-downloads the model, then starts transformers serve as a child process with eager model loading. The model is ready before any request arrives:

Gradio UI  ──HTTP──▶  transformers serve  ──▶  GPU
  (app.py)              (child process, eager load + pre-download)

ZeroGPU Spaces are detected automatically by transformers serve via the SPACE_ID environment variable.

Quick start

pip install -r requirements.txt
python app.py        # ← starts both the Gradio UI AND serve