Repository navigation
Feat/wrapper auth - #16570
Closed
shivanshtalwar0 wants to merge 44 commits into
Closed
Feat/wrapper auth#16570shivanshtalwar0 wants to merge 44 commits into
shivanshtalwar0 wants to merge 44 commits into
Conversation
…FloStudio workers
- WRAPPER_AUTH_TOKEN: when set, every route (wrapper, native /queue,
/prompt, /view, /history, /ws and static) requires
`Authorization: Bearer <token>` (constant-time compare; ?token= only on
websockets; OPTIONS passes). Unset = unchanged behaviour. Needed because
RunPod pod URLs are public.
- progress: GET /api/wrapper/jobs/{id} reports a `progress` object
(queue position while pending, sampler progress while running).
- cleanup: DELETE /api/wrapper/jobs/{id} removes the job's outputs,
uploaded inputs and history entry. Wrapper outputs now go to
output/wrapper/<job_id>/ and uploads to input/wrapper/<job_id>/ so they
can be found after a restart.
- README, .env.example and docker-compose (healthcheck sends the bearer
when set) document it; tests for auth, progress and cleanup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
api_wrapper: bearer auth, job progress and per-job cleanup (FloStudio GPU workers)
… Hub publish (#2) One image for the FloStudio GPU workers: the local rig (docker compose) and on-demand RunPod pods (which run the image's own CMD, no compose). - Dockerfile: python:3.12-slim + PyTorch cu128 wheels (Blackwell/RTX 5090, driver >= 570), installed before requirements.txt; the build fails if the final torch is not a CUDA build. tini + docker/entrypoint.sh, HEALTHCHECK that sends the bearer token when WRAPPER_AUTH_TOKEN is set. - docker/entrypoint.sh: persistent store from COMFYUI_DATA_DIR, defaulting to RunPod's /workspace network volume (models, HF_HOME, Triton kernel cache); bind mounts win. Builds the ComfyUI flags from env (the logic that used to be the compose `command`), `prefetch` subcommand, and runs any other command as is (developer-api's uvicorn). - docker/prefetch_models.py: download the wrapper's H3 (fl2va/ref2va, any quantization) and FLUX.2 klein checkpoints ahead of time, so a new network volume is filled once instead of during a customer's render. - .dockerignore: exclude every top-level dot-entry (.venv 1.5 GB, .reasonix 1.5 GB, .claude worktrees, .env) and tests; the COPY layer went from 1.64 GB to 80 MB. - docker-compose.yml: uses shivanshtalwar0/comfyui (build fallback), no command (entrypoint), configurable host port (8282 on the rig), passes WRAPPER_AUTH_TOKEN / AUTO_DOWNLOAD_MODELS / COMFYUI_ARGS. - .github/workflows/docker-publish.yml: build on ubuntu-latest, smoke test (CUDA torch, prefetch --dry-run, server boot + wrapper 200 with / 401 without bearer), then push to Docker Hub on master (latest, sha-), vX.Y.Z tags and manual runs; PRs build and test only. Needs DOCKERHUB_TOKEN. - README / .env.example: Docker, RunPod and prefetch docs. Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
PR #2 was stacked on feat/wrapper-auth and merged there after #1 had already landed, so the Docker work never reached master and the Docker image workflow never ran. Conflicts in README.md, .env.example and docker-compose.yml were all the pre-Docker text on master's side; resolved to the Docker side (master's README <div> edit is kept). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docker/prefetch_models.py: say() helper instead of print() (T201 is enabled for the repo). - Pre-existing fork lint on master: unused `files` in api_wrapper/workflows.py, unused HfHubHTTPError import and `start` in comfy/model_downloader.py, unused imports + trailing newline in comfy/sync_api.py. `ruff check .` passes. - tests-unit/wrapper_api_test/test_docker.py: compose no longer carries the server `command`; assert the flags in docker/entrypoint.sh instead, plus the CUDA-only torch install, the entrypoint/healthcheck wiring, the published image, and the .dockerignore dot-entry exclusion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Land the Docker image on master (re-land #2) + lint fixes
…tore The Hugging Face paths downloaded into the HF cache and then copied the file into the models dir, so every checkpoint existed twice. With the worker image keeping HF_HOME on the RunPod network volume, the default H3 + FLUX.2 klein set needed 164 GB instead of 82 GB: over the configured volume, so a prefetch would fail with a full disk, and double the per-GB storage bill. - COMFY_HF_DOWNLOAD_MODE=direct: hf_hub_download(local_dir=...) into <models>/.hf-staging/<id>/ (same disk, outside every scanned folder), then move into place and remove the staging dir. Size verification is kept, and a truncated direct download is deleted from the models dir. - Default stays `cache` (unchanged behaviour), which lets a HF cache shared between projects serve repeat downloads on the local rig. - docker/entrypoint.sh: `direct` by default when a data dir / RunPod volume is in use. - Tests: direct mode lands one file and cleans staging, the truncated download is removed, the at-format path, and cache mode is the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
model_downloader: one copy per checkpoint on the RunPod volume (82 GB, not 164)
…volume Measured on RunPod EU-RO-1: pulling the 5.15 GB full image took >9 min (<10 MB/s from Docker Hub) before the container even started. The slim `runpod` target is 383 MB compressed: the apt layer, a pinned uv, the source and the entrypoint; no torch, no requirements. - Env key (16-char sha256 of recipe, Python, arch, Debian, torch index and requirements.txt) baked at build time into docker/env-key. - entrypoint.sh (runpod variant): the env lives at $DATA_DIR/envs/<key>, built once with uv (torch from the cu128 index, then requirements, CUDA asserted, freeze file, completion marker) under an mkdir lock with a heartbeat and stale-lock takeover, so concurrent pods never build twice; uv cache on the container disk (nothing cached twice on the per-GB volume). Server, prefetch and passthrough commands run inside the env. Exits 64 without a volume. `prefetch --env-only` pre-builds it. - The full `comfyui` target is unchanged and stays the default build. - docker-publish.yml: parallel full and runpod jobs; runpod pushes `runpod`, `runpod-sha-<short>`, `runpod-X.Y.Z`; its smoke test builds the real CUDA env on a fresh volume, checks 200/401 on the wrapper, then a second container must reuse the env and come up faster. - test_docker.py structural tests; README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…me (#5) Slim RunPod image (383 MB vs 5.15 GB): Python env on the network volume
… on the volume (#6) Pulling our image from Docker Hub ran at ~10 Mbps on EU-RO-1 hosts: 4-5 min for the 370 MB slim image, 9+ min for the full one. RunPod keeps its own PyTorch image cached; a pod on runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404 was running 2 s after creation. - docker/runpod-bootstrap.sh: the pod's start command on RunPod's image. Fetches this repo at COMFY_CODE_REF (a commit) onto the volume once per commit, unpacks it to /opt/ComfyUI and runs the entrypoint. - Python env on the volume is now one archive (<data>/envs/<key>.tar): built on the container disk, published under a temp name, unpacked per pod. MooseFS streams big files at ~800 MB/s but was still writing the ~42k-file env in place after 16 minutes. - docker/env-inputs.sh prints the env key inputs for both the image build and the boot-time key (code fetched by the bootstrap); recipe 2, and the base interpreter path is part of the key. - CI: the runpod job also boots through the bootstrap from a git-archive tarball, reusing the env archive the earlier steps published, then boots again with the code server gone. Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A browser cannot send the WRAPPER_AUTH_TOKEN bearer header, so admins could not open a pod's ComfyUI UI at all. Whoever holds the token (voxmin-backend) now signs a short-lived link, ?comfy_login=<exp>.<HMAC-SHA256(token, "comfy-login:<exp>")>, at most 10 minutes ahead. A valid link redirects to the clean URL with an HttpOnly, SameSite=Lax session cookie (12 h, signed the same way; Secure behind an https proxy). Pages, API calls and the WebSocket then authenticate with the cookie; the token never reaches the browser. Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…f_images (#8) * fix(wrapper): H3 references reach the model; FLUX edits take ref_images The ref2va builder wired its Autogrow inputs as a nested dict ({"ref_images": {"ref_image_0": ...}}). ComfyUI addresses an Autogrow slot by its flat dotted id ("ref_images.ref_image_0"); the nested key is not an input the node declares, so execution dropped it without a validation error and MiniMaxH3ReferenceToVideo ran with ref_images={} and ref_audios={}. Every reference render since the wrapper landed was text-only. - ref2va emits flat ids for images, videos, video audio and audio - a reference video goes through GetVideoComponents: the slot takes IMAGE frames, LoadVideo gives a VIDEO (the nested shape hid this mismatch too) - test_reference_inputs_reach_the_node runs get_input_data + build_nested_inputs and checks what the node actually receives flux2klein9b (the edit) only accepted one `image`, so a still sent with `ref_images` (what Flow sends) was a 400 "No image provided": - accepts `image` and/or up to 3 `ref_images`, one ReferenceLatent each (FLUX.2 multi-reference edit), image first - optional width/height set the output canvas (a video frame), instead of the first reference's size - references upscale with lanczos, not nearest-exact - requires_image is satisfied by any of the task's image_fields; the OpenAPI form no longer marks `image` required where ref_images also works Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(wrapper): cover-crop H3 keyframes; report each job's seed and weights - i2v/fl2v: first and last frames are cover-cropped to the canvas (ImageScale crop=center, lanczos) before the H3 node, which stretches a first frame to fit. An off-aspect still no longer squashes the faces in it, and a pair from one source no longer warps differently at each end. - provenance: every job's seed, checkpoint files, settings, advisory note and (when the LLM rewrote it) the prompt it actually ran with are returned in an async submit's 504 body and on GET /wrapper/jobs/{id}. A caller that polls never saw the X-Wrapper-* headers, so none of this was recorded anywhere. X-Wrapper-Settings now carries the seed too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(prefetch): default to the rig's int8 H3 UNETs; fetch the encoder the wrapper loads RunPod pods ran the fp8 H3 UNETs (the nvfp4 profile maps to fp8_scaled) while the RTX 5090 rig runs int8, so neighbouring shots of one film could come from different weights and differ in skin, texture and colour. - DEFAULT_SPECS: minimaxh3:int8 and minimaxh3-ref:int8 (was nvfp4 = fp8) - an H3 spec fetches the text encoder the picker loads at render time (the smallest on disk: nvfp4, 16 GB), not its quantization's own: int8 used to pull the 34 GB int8 encoder that nothing would load - README and .env.example follow With int8 requested and only fp8 on a volume, the picker keeps using fp8 (no download mid-render), so flipping the pod profiles to int8 is safe before the volume is refilled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ult (#9) Xiaomi released MiMo V2.6 Flash and Pro on 2026-09-22. Flash is omnimodal, costs the same as mimo-v2.5, and Xiaomi rates it above mimo-v2.5-pro, so `auto` now sends every rewrite to mimo-v2.6-flash instead of splitting image rewrites (mimo-v2.5) from text rewrites (mimo-v2.5-pro). - llm_model accepts mimo-v2.6-flash and mimo-v2.6-pro; the V2.5 ids stay. - Only mimo-v2.5-pro is text-only. Checked on the live API on 2026-09-25: both V2.6 models read an image. - resolve_model no longer takes has_image, since auto no longer depends on it. Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…10) The reference task now takes an optional 'image' (first frame) and 'last_frame', anchored with MiniMaxH3AddGuide at frame 0 / -1 on top of the reference conditioning, cover-cropped to the canvas like the i2v path. A shot can then continue from the previous shot's final frame and hold its cast to their character sheets in one render. Measured on the 5090 (864x480, 3 s, int8, same seed): a pinned first frame opens at 33 dB PSNR to the anchor, the same as image-to-video; naming it only in the prompt gives 22 dB. When the anchor hides a face, i2v re-invents it; the pinned reference render recovers it from the sheet. Co-authored-by: shivanshtalwar00 <shivansh@finasium.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
shivanshtalwar0
requested review from
alexisrolland,
comfyanonymous,
guill,
kijai and
rattus128
as code owners
September 25, 2026 23:06
Contributor
|
🎉 Thank you for your contribution, we really appreciate it! 🎉 Like many open source projects, we require contributors to sign our Contributor License Agreement (CLA). A CLA makes the ownership of contributions explicit, so contributors and the project share a clear understanding of how the code can be used. By signing, you:
CLAs are standard practice across major open source projects including those under the Apache Software Foundation and the Linux Foundation. Ours is based on the Apache Software Foundation's CLA. Most importantly, it would enable us to relicense the project under a more permissive license in the future, giving the project and its community greater flexibility. ✍ To sign, please post a new comment on this PR with exactly the following text: ✍ I have read and agree to the Contributor License Agreement You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.