English

GPU Server Sizing Methodology For LLM Inference Solutions: A Formalized Approach To Infrastructure Capacity Planning

This technical methodology document presents a formalized approach to infrastructure capacity planning for Large Language Model (LLM) inference deployments. Developed by AI engineering specialists at OTP Bank (Moscow, Russia) in 2026, the methodology provides systematic formulas and procedures for calculating the required number of GPU servers based on memory constraints (model weights, KV-cache) and computational throughput requirements. Key contributions include: • Mathematical framework for GPU memory estimation with Tensor Parallelism optimization • Analytical and empirical models for inference throughput prediction (prefill/decode phases) • SLA-compliance verification procedures for TTFT (Time-To-First-Token) and end-to-end latency • Architectural pattern catalog for token load estimation in agent-based AI systems (RAG, ReAct, CoT, multi-agent frameworks) • Calibration coefficients for real-world deployment adjustments based on pilot testing The methodology supports CapEx forecasting, TCO estimation, and ROI analysis for enterprise AI infrastructure projects. All formulas are designed for reproducibility and include correction factors for framework overhead, network latency, a

Sign in with your free account to read, download EPUBs or listen.