AI Inference Engineer

• Develop APIs for AI inference that will be used by both internal and external customers
• Benchmark and address bottlenecks throughout our inference stack
• Improve the reliability and observability of our systems and respond to system outages
• Explore novel research and implement LLM inference optimizations

• Experience with ML systems and deep learning frameworks (e.g. PyTorch, TensorFlow, ONNX)
• Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization, etc.)
• Understanding of GPU architectures or experience with GPU kernel programming using CUDA

Equity may be part of the total compensation package