ML Infrastructure Engineer for Air-Gapped Deployment
embedded-systems, cisco-routers, computer-networking, microcontroller-programming
Preferred qualifications
- Experience: Expert
We need an India-based ML Infrastructure Engineer to deploy an open-model LLM inference stack on cloud and an air-gapped workstation. Phase 1, (2–3 weeks) includes setting up a Linux GPU box with CUDA, Docker, and vLLM or similar, serving open LLMs and a small vision model behind one API, integrating with our pipeline, running benchmarks, and building an offline installer. Phase 2 (1–2 weeks) involves preparing a 48 GB NVIDIA workstation, installing from the offline installer, verifying and tuning the setup, and supporting the pilot remotely. Must have experience with quantized models, NVIDIA/CUDA setup, Docker, Python, install scripting, and offline deployment. Two senior engineers will guide the work. Please include the largest open model you have served, one offline deployment you have done, your location, and on-site availability in the next 6–8 weeks.
Open job
AI proposal draft
Generate a short cover letter to copy into the offer. Says you are interested and ready to work.
Sign in to generate an AI proposal draft.
Log in