Optimizing LLM Inference at Scale