Job Description

Responsibilities

• Diagnose performance problems across orchestration, scheduling, placement, networking, autoscaling, inference internals, runtimes, host operating systems, and accelerators.
• Isolate root causes to specific system layers and owners, fixing issues directly or handing them off with reproducible findings and evidence.
• Protect throughput, latency, and stability targets by identifying gaps between orchestrated workloads and bare-metal performance.
• Assess fleet-scale deployment concerns including routing, placement, and shared-resource contention.
• Capture CPU placement, memory, NUMA locality, and network-path requirements in orchestration configuration.
• Build lightweight telemetry and benchmarks for prototype stacks.
• Convert recurring investigations into playbooks and mentor engineers in a profiling-first performance methodology.

Requirements

• 8+ years of hands-on performance or systems engineering experience, including resolving cross-layer performance regressions.
• Demonstrated ability to profile unfamiliar, sparsely documented systems and identify performance overhead.
• Deep knowledge of Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths.
• Experience running performance-sensitive workloads on Kubernetes and addressing overhead from scheduler defaults, network overlays, and orchestration abstractions.
• Working knowledge of LLM inference concepts including batching, KV-cache, prefill/decode, quantization, and speculative decoding.
• Proficiency in Python and C/C++ for instrumentation, debugging, and cross-stack patching.
• Strong technical writing for hardware, firmware, and platform teams, plus sound judgment about ownership and handoffs.
• Fluency with AI and agentic tools for investigation, code search, log triage, and instrumentation drafting, with the ability to verify results against real data.
• Preferred experience with vLLM, SGLang, TensorRT-LLM, speculative decoding, disaggregated or heterogeneous serving, KV-cache management, request and batch scheduling, weight prefetch and residency, GPU programming and profiling, pre-production hardware, or Rust.
• Startup experience or comfort working in an environment with shifting priorities.

Benefits

• Location is Santa Clara, California or remote within the United States.
• The company describes an inclusive, collaborative, and equal-opportunity workplace.

Job Tags

Similar Jobs