Lead AI Inference Engineer 100% Remote

About the job

You'll lead a cross-functional pod that spans the full stack, from C++ inference engines to JavaScript applications. Your responsibility is to ensure that local AI capabilities ship reliably and perform well across devices. You'll balance hands on technical work with team coordination, guiding foundation and middleware engineers toward shared goals.

This role is ideal for someone who understands both the low-level challenges of edge AI and the product-facing needs of app developers, and wants to drive the delivery of cohesive, production ready local AI systems.

Responsibilities

• Work on deploying machine learning models to edge devices using the frameworks llama.cpp, ggml, onnx
• Collaborate closely with researchers to assist in coding, training and transitioning models from research to production environments
• Integrate AI features into existing products, enriching them with the latest advancements in machine learning
• Manage a cross functional team (pod) made of middleware (JS), foundation (C++), QA and documentation engineers to produce high quality deliverables
• Regularly assess, both qualitatively and quantitatively, our position in the market with regards to similar products or platforms
• Leverage the expertise of technical architects to ensure robust architectural choices and code quality
• Ensure stable releases by following precise internal release processes

Qualifications

• Excellent programming skills in C++
• Strong experience with Llama.cpp and ggml inference engines, facilitating deployment of models to specific GPU architectures
• Good understanding of deep learning concepts and model architectures
• Experience with transformers, LLMs, Diffusion Models
• Demonstrated ability to rapidly assimilate new technologies and techniques
• Experience managing a small, specialized, cross functional team (pod) of 3-5 people
• Genuine passion for building good products that improve people's lives
• Degree in Computer Science, AI, Machine Learning, or related field, complemented by a solid track record in AI R&D

Bonus points if

• Extensive experience with Javascript/Typescript
• Understanding of difficulties, nuances and importance of p2p technology
• Experience with Vulkan, Metal and OpenCL
• Experience productionizing models