Building a Kid-Safe Audiobook Assistant on Cloudflare Workers AI

September 2026

A local model on an old mini PC was too slow to hold a five-year-old's attention. Moving the matching step to a Cloudflare Worker fixed the latency — then a second pass cut its token usage by 95% without changing what it actually does.

My son has a Home Assistant Voice Preview in his room, wired to Music Assistant, pointed at an Audiobookshelf library. The goal was narrow: he says a book title, it plays. Getting there took two different architectures and a handful of bugs that only showed up once real hardware and a real kid were involved.

The problem: local models were too slow

The first version ran entirely on an old mini PC I had sitting idle. Home Assistant would hand a small local model the book list and whatever he said, and the model would pick a match. It worked, but even the lightest model I could get running at usable quality took several seconds to answer. For a voice interface, that's disqualifying — a five-year-old doesn't wait out a multi-second pause to find out if the speaker heard him. Latency isn't a nice-to-have here; it's the whole interaction.

The architecture

The fix was to stop asking a general-purpose local model to do this locally at all, and move only the matching step to a Cloudflare Worker calling Workers AI. Everything else stayed in Home Assistant, where it already belonged:

Only titles and a phrase go out; only a number comes back. Home Assistant does the rest.

The book list itself comes from a "kids" playlist I curate directly in Music Assistant, not from genre metadata — the library's own genre tags turned out to be inconsistent enough (Diary of a Wimpy Kid tagged as children's audio, Harry Potter not) that the playlist is the more reliable allow-list.

Three bugs before any of it worked

• The trigger didn't fire. I'd planned to catch anything Assist couldn't handle by listening for a conversation_end event. It never fired — confirmed by opening a live event listener and getting zero events across a real test. Current Home Assistant expects a wildcard conversation sentence trigger instead; same intent, different mechanism.
• Music Assistant changed a default. An empty search query used to return the whole library. A newer version hard-returns nothing for an empty query by design, so the "fetch everything" call silently returned zero books.
• The model's response shape didn't match the code. Gemma replies in an OpenAI-style choices[0].message.content shape, not the flatter field some other Workers AI models use. Every request was quietly getting back an empty string — "no match" was actually "never read the answer."

Compressing the prompt

Once matching worked, it was slow in a new way. Gemma reasons step-by-step before answering by default, and against a real 60-plus title list that reasoning chain alone was routinely over 800 tokens — occasionally enough to exceed Home Assistant's own request timeout before the model ever wrote its answer. Three changes fixed it, none of which touch what the assistant actually does:

// index.js
env.AI.run("@cf/google/gemma-4-26b-a4b-it", {
 messages,
 max_tokens: 512,
 chat_template_kwargs: { enable_thinking: false }
});

• Disabled thinking mode outright. Matching a phrase against a title list doesn't need a visible reasoning chain, and the model answers just as accurately without one.
• Stopped asking the model to echo back a full audiobook URI — fifty to ninety characters, provider-specific. It now returns a plain index into the list, resolved locally on the Worker. It can't hallucinate a URI it never has to type.
• Strip (Unabridged) and duplicate | Series, Book N suffixes from every title before it reaches the prompt.

Staying under the free tier

Workers AI gives every account 10,000 free neurons a day. At roughly 15 neurons a request, that's over 650 requests a day before anything is billed, and past that Cloudflare charges $0.011 per 1,000 neurons — the 651st request that day costs a fraction of a cent. A kid asking for the same book four nights running was never going to be what pushes this over budget.

The lesson wasn't "use a bigger model" or "use a smaller one" — it was that a narrow, well-scoped task doesn't need a model's full reasoning budget, and turning that budget off is often the highest-leverage optimization available before touching anything else.