Nvidia NIM support #226
Description
This is a draft PR in github:
it's some initial work to make NIMs a first-class citizen in DMR. But we need someone with Nvidia hardware to grab this patch and make it their own. How Nvidia recommend you run a NIM today is here https://build.nvidia.com/google/gemma-3-1b-it/deploy
docker login nvcr.io
export NGC_API_KEY=<PASTE_API_KEY_HERE>
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
docker run -it --rm \
--gpus all \
--shm-size=16GB \
-e NGC_API_KEY \
-v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
-u $(id -u) \
-p 8000:8000 \
nvcr.io/nim/google/gemma-3-1b-it:latest
This work would make NIMs a first class citizen of DMR so that would simply become:
docker login nvcr.io
export NGC_API_KEY=<PASTE_API_KEY_HERE>
docker model run nvcr.io/nim/google/gemma-3-1b-it:latest