Job Description

• Scale Inference: build and deploy multimodal AI models into a realtime API platform.
• Infrastructure management using Terraform or alternative infrastructure as code.
• Optimize deployment pipelines, CI/CD pipelines for seamless integration of machine learning models.
• Design and build RESTful/WebRTC APIs to facilitate model serving and application development.
• Identify bottlenecks and optimize the performance of the inference lifecycle.
• Work closely with Research team to understand and productize latest model architectures.
• Automate documentation, processes, and systems to facilitate knowledge sharing within the team.

Job Requirements

• Production level MLOps or Python experience.
• Strong fundamental understanding of backend concepts like: concurrent programming, event driven architectures, caching, kernel/thread/low level routines.
• Experience scaling software with Docker, Kubernetes (K8s), or equivalent containerization platforms.
• Experience designing and building developer APIs or SDKs.
• Cloud experience working with AWS, GCP, Azure or Self hosted On-Premise environments.
• Basic understanding of LLM inference lifecycles, e.g. kv cache, paged attention, etc.
• Bonus: Triton Inference Server, VLLM, Kubeflow, etc.

Skills