LLM Ops Engineer

\u003cp\u003e\u003c/p\u003e\u003cp\u003eWe are looking for an experienced\u0026nbsp;\u003cstrong\u003eLLM Ops Engineer\u003c/strong\u003e\u0026nbsp;for our client, an international AI development company based in New York.\u003c/p\u003e\u003cp\u003eIn this role, you will work closely with\u0026nbsp;\u003cstrong\u003eLLM Engineers, ML Engineers, Data Engineers, and Software Engineers\u003c/strong\u003e\u0026nbsp;to productionize Generative AI and LLM-based solutions. You will help build reliable, scalable, and secure systems at the intersection of\u0026nbsp;\u003cstrong\u003eAI Engineering, MLOps, DevOps, and Cloud Infrastructure\u003c/strong\u003e.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eWhat You’ll Do\u003c/strong\u003e\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eDesign and maintain scalable infrastructure for\u0026nbsp;\u003cstrong\u003eLLM and Generative AI applications\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eBuild and manage\u0026nbsp;\u003cstrong\u003eLLMOps/MLOps pipelines\u003c/strong\u003e\u0026nbsp;for model, prompt, and application deployment.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eDevelop and maintain\u0026nbsp;\u003cstrong\u003eCI/CD pipelines\u003c/strong\u003e\u0026nbsp;for AI/ML workloads.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eDeploy and manage\u0026nbsp;\u003cstrong\u003eLLM inference services\u003c/strong\u003e\u0026nbsp;and model-serving infrastructure.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eImplement monitoring and observability for:\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eModel performance\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eLatency\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eAvailability\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eErrors\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eResource utilization\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMonitor and optimize\u0026nbsp;\u003cstrong\u003eLLM costs, token consumption, latency, and inference performance\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEstablish processes for\u0026nbsp;\u003cstrong\u003emodel, prompt, dataset, and configuration versioning\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eAutomate deployment, testing, rollback, and release processes.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eBuild infrastructure using\u0026nbsp;\u003cstrong\u003eInfrastructure as Code (IaC)\u003c/strong\u003e\u0026nbsp;and cloud-native technologies.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eImplement appropriate\u0026nbsp;\u003cstrong\u003esecurity, access controls, secrets management, and data protection practices\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eTroubleshoot production issues and continuously improve system reliability and scalability.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEstablish best practices for\u0026nbsp;\u003cstrong\u003eLLM evaluation, observability, and production monitoring\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eContinuously evaluate emerging LLM infrastructure, serving technologies, and AI platform capabilities.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eWhat You’ll Bring\u003c/strong\u003e\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003e\u003cstrong\u003e7+ years of experience\u003c/strong\u003e\u0026nbsp;in DevOps, MLOps, ML Engineering, Platform Engineering, or a related field.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e\u003cstrong\u003e4+ years of hands-on experience\u003c/strong\u003e\u0026nbsp;supporting LLM or Generative AI workloads in production.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStrong experience with at least one major cloud platform:\u0026nbsp;\u003cstrong\u003eAWS, Azure, or GCP\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStrong knowledge of\u0026nbsp;\u003cstrong\u003eDocker and Kubernetes\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience building and maintaining\u0026nbsp;\u003cstrong\u003eCI/CD pipelines\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStrong\u0026nbsp;\u003cstrong\u003ePython and scripting skills\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience with\u0026nbsp;\u003cstrong\u003eInfrastructure as Code\u003c/strong\u003e, preferably\u0026nbsp;\u003cstrong\u003eTerraform\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience with\u0026nbsp;\u003cstrong\u003eML/LLM deployment and model-serving frameworks\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eSolid understanding of:\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eLLM architectures\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eInference workflows\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eEmbeddings\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eRAG\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eVector databases\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience with\u0026nbsp;\u003cstrong\u003eobservability and monitoring tools\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStrong understanding of\u0026nbsp;\u003cstrong\u003eAPIs, microservices, networking, and distributed systems\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience with\u0026nbsp;\u003cstrong\u003eGit and modern software development practices\u003c/strong\u003e.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eStrong troubleshooting and problem-solving skills.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eNice to Have\u003c/strong\u003e\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eExperience with platforms and tools such as:\u003c/p\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eMLflow\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eKubeflow\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eLangChain\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eLangGraph\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eRay\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003evLLM\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eor similar technologies\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience with LLM APIs such as\u0026nbsp;OpenAI, Anthropic, Google, or Azure OpenAI.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eExperience implementing\u0026nbsp;LLM evaluation and quality monitoring frameworks.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003cp\u003e\u003c/p\u003e