fai-inference-optimization
Optimize model inference latency and throughput — batching, quantization, caching, KV-cache reuse, and hardware/SKU selection. Use this skill when responses are too slow, cost per token is too high, or scaling under load — even if the user only says “make the model faster” or “reduce latency”.
Overview
| Property | Value |
|---|---|
| Type | Skill |
| Folder | fai-inference-optimization |
| WAF Alignment | N/A |
| Compatible Plays | N/A |
| Lines | 157 |
Usage
Reference this skill in your fai-manifest.json:
{
"primitives": {
"skills": ["../../skills/fai-inference-optimization/SKILL.md"]
}
}Source
Auto-generated from the FrootAI primitive catalog .
Last updated on