Skip to Content
GeneratedSkillsfai-inference-optimization

fai-inference-optimization

Optimize model inference latency and throughput — batching, quantization, caching, KV-cache reuse, and hardware/SKU selection. Use this skill when responses are too slow, cost per token is too high, or scaling under load — even if the user only says “make the model faster” or “reduce latency”.

Overview

PropertyValue
TypeSkill
Folderfai-inference-optimization
WAF AlignmentN/A
Compatible PlaysN/A
Lines157

Usage

Reference this skill in your fai-manifest.json:

{ "primitives": { "skills": ["../../skills/fai-inference-optimization/SKILL.md"] } }

Source


Auto-generated from the FrootAI primitive catalog .

Last updated on