Using an LLM as a Heuristic Dispatcher, Not a Cloud API Wrapper

When someone reports a broken vehicle in MobiShare’s chat, something has to decide which technician gets the ticket. There’s no scoring model, no trained classifier, no queue algorithm — a local LLM reads a JSON list of technicians with their current workload and recent history, and replies with one GUID. It’s a real, working pattern for a small ranking decision, not a toy. It also ran for months always ranking on half its intended input, because of a single hardcoded string nobody’s tests caught, because there were no tests.

The decision doesn’t need a model, just a heuristic

Building an actual assignment model for “which technician should fix this” needs things MobiShare doesn’t have at hackathon scale: enough historical tickets to learn a pattern from, labeled outcomes to validate against, and a real reason to believe a learned ranking beats a simple one. What the app does have is an LLM already loaded for the chatbot, and a decision that’s exactly the shape LLMs are good at when you keep them on a short leash: given a short list of options, each with a few relevant attributes, pick the best one — which is closer to “read a table and make a judgment call” than to “generate free text,” even though the underlying mechanism is identical.

The distinction that makes this safe to ship is how much you trust the output. This isn’t the model deciding whether a payment goes through, or writing to the database directly — it’s the model producing one input to a decision that gets verified before anything happens as a result. That verification step is the part of this design worth copying regardless of what you’re routing.

Feed it a summary, verify the reply before acting on it

ReportVehicleIssueAsync builds the technician list, asks the model, and treats the reply as an ID to look up — never as text to interpret or act on directly:

var getTechnicians = await _httpClient.GetFromJsonAsync<IEnumerable<TechnicianReports>>(
    "api/TechnicianApis/GetTechniciansReports");

var promptText = new PromptCollections().ReportPrompt(getTechnicians, description);

var technicianId = "";
await foreach (var response in _chat.SendAsync(promptText))
{
    technicianId += response;
}

var verifiedTechnician = HttpClientContext.UserManagerController.FindByIdAsync(technicianId).Result;
if (verifiedTechnician == null)
{
    return $"Nessun tecnico trovato per gestire il report: {description}";
}

Each technician in the TechnicianReports payload carries AssignedReports (current open workload) and LastClosedReports (their five most recent resolved issues), so the prompt gives the model something to weigh “already busy” against “has fixed something like this before.” If the model’s reply doesn’t resolve to a real, existing user ID, the assignment is rejected outright rather than created against garbage — no ticket gets silently routed to a hallucinated technician. That FindByIdAsync check is a one-line guard, and it’s the single most important line in the feature: it’s the difference between “the model suggests, the database confirms” and “the model has a side effect.”

The bug wasn’t in the model, or the prompt

LastClosedReports — half of what the prompt is built to weigh — comes from a query meant to pull each technician’s five most recently resolved tickets:

LastClosedReports = _dbContext.ReportAssignments
    .Where(a => a.UserId == user.Id && a.Report.Status == "Chiuso")
    .OrderByDescending(a => a.Report.CreatedAt)
    .Take(5)
    .Select(a => new ReportSolution { /* ... */ })
    .ToList()

"Chiuso" is Italian for “closed.” The actual status the rest of the codebase writes and reads is the English enum value:

public enum ReportStatus { Pending, Assigned, Closed }

Every ticket that’s ever been resolved carries Status == "Closed". The Where clause above compares that against "Chiuso" and never matches a single row, anywhere, for any technician, since the query was written. LastClosedReports has been an empty list unconditionally — not sometimes, not for edge cases, always — and the “who’s actually good at this kind of issue” half of the prompt’s reasoning has been silently absent since day one.

The model doesn’t fail loudly when this happens. It doesn’t error, doesn’t refuse, doesn’t flag that it’s missing context — it just makes the best decision it can from whatever’s left, which here means falling back to workload alone every single time. An empty list isn’t an exception; it’s a perfectly well-typed, perfectly valid, silently wrong answer, and nothing in the pipeline — no log line, no test, no assertion — was in a position to notice the difference between “no technician has closed anything relevant” and “this query has never worked.”

Why this is a data bug wearing an AI costume

It’s tempting to file this under “prompt engineering problem” because the symptom shows up in what the LLM decides. It isn’t one, and better prompting wouldn’t have caught it — the model was working correctly on the input it received; the input was wrong before the model ever saw it. This is an ordinary application bug: an untranslated string literal compared against an enum that was never Italian to begin with, sitting in a LINQ query that would have failed the same way whether the caller downstream was an LLM, a scoring formula, or a human reading a dashboard. The fact that it fed an AI feature doesn’t make it an AI bug, and treating it like one is exactly how it would keep hiding — “the model’s ranking seems fine” is true and also irrelevant, because the model was never the layer with the defect.

What would have caught it is the same thing that would have caught the fake-success bug in the CQRS post: one test asserting that LastClosedReports returns something non-empty for a technician with a known closed ticket in the seed data. Mobishare.Tests had a folder scaffolded for exactly this class of query and zero tests in it — the gap wasn’t a hard-to-test AI pipeline, it was a completely ordinary EF Core query that happened to sit one hop upstream of a chatbot.

Verdict

Using an LLM as a lightweight ranking heuristic over a small, explicit JSON context is a legitimate pattern, not a shortcut you’ll regret — you don’t need a trained model to make a “pick the best of five options” decision, and the habit worth keeping from this implementation is verifying the model’s output before it causes any side effect, rather than trusting free text. But an LLM making a decision is only as good as the data path feeding it, and that path is regular application code, with all the regular ways application code breaks: an untranslated string, an untested query, a silently empty result that never throws. Don’t let “there’s an LLM in this feature” change how carefully you test the plumbing around it — if anything, test it more carefully, because a wrong answer from a heuristic doesn’t look like a bug. It looks like a decision.