Inference System & Performance - Member of Technical Staff
Callosum
midpermanentbackenddevops London 94 days ago via Arbeitnow
See how well this job matches your profile
Sign up to get an AI match score and generate a tailored application in seconds.
Get your match scoreTags
LLM InferenceKV CacheGPU KernelsMemory ManagementBatching StrategiesDistributed InferenceNVIDIA NCCLRDMAInfiniBandSpeculative Decoding
About the role
Role overview
Inference System & Performance – Member of Technical Staff at Callosum. You will own end-to-end performance for the company’s inference platforms in a highly heterogeneous environment spanning multi-GPU and multi-node systems.
What you’ll build
- Design and optimize inference serving systems for heterogeneous multi-GPU and multi-node environments
- Build and optimize KV cache lifecycle management, including:
- Batching strategies
- Memory allocation to maximize throughput and minimize latency
- Profile and tune GPU kernels by identifying bottlenecks across:
- compute
- memory
- network and implement targeted optimizations
- Build/improve scheduling logic for:
- continuous batching
- disaggregated prefill/decode
- speculative decoding
- Work with distributed networking primitives to optimize communication:
- NCCL, NVLink, RDMA, InfiniBand, RoCE
- Develop tooling for performance visibility, regression detection, and benchmarking across hardware configurations
What you bring
- Deep understanding of LLM inference internals, including:
- KV cache lifecycle
- memory management
- attention mechanisms
- serving architectures
Nice-to-haves (implied by the role)
- Strong experience profiling/tuning GPU performance and distributed inference communication
- Familiarity with continuous batching, speculative decoding, and disaggregated prefill/decode
About Callosum
Callosum is an “intelligent systems” company building infrastructure and software orchestration that unifies heterogeneous compute across the full AI stack. It focuses on tailoring AI inference to each workload by co-evolving models, workflows, and silicon, working closely with frontier and specialized hardware accelerators.
Scraped 8/20/2026