AgileV's errors seen from your client: six rules

Who this is for: anyone calling AgileV from their own code — and in particular anyone who writes the client in house, which seems to be almost all of us: the harness of the Agile boxes forbids running code downloaded from outside. Where it comes from: on 28/9 trpg, looking into something we had flagged, found two defects in their own client. They are not defects of theirs: they are defects of a class, and we found them in the three reference clients we hand out as well. This page is what we learned, with the measurement underneath; if you wrote your client yourself, check these points before waiting for them to show up on their own.

What had happened at trpg

Their client read detail and error from our response and stuffed them into a string. Here they are structures, not strings: so every AgileV error became, at their end, the word «Array» — literally Voyage API error: Array. And it was not a log: that message is read by the end user when the indexing of a document fails. In practice, since they moved to us, KEY_MISSING, KEY_INVALID and RATE_LIMIT never reached the reader: if their key expired tomorrow, the customer would read «Array».

The second defect is worse because it makes no noise: a 200 of ours with an unexpected shape passed as a success with zero vectors, and the caller received NULL in silence.

Rule 1 — branch on error.code, which is a structure

Every AgileV error but one has this body, and the code is stable: it is the only thing worth branching on (the HTTP status tells you the family, the code tells you what to do).

{"error": {"code": "KEY_REVOKED", "message": "chiave revocata"}}

error and detail are objects: in PHP json_decode($t, true) gives them to you as arrays, and concatenating them into a string produces «Array» plus a warning. In Python, str(e["error"]) produces the repr of a dictionary. Build the message to display yourself, out of three pieces only: HTTP status, error.code, error.message.

A historical note: some versions nested the same object under detail ({"detail":{"error":{…}}}). A robust client reads error at the top or under detail. Today it always arrives at the top.

Where the table is: every code you can receive, with what to do about it, is in the error table. Look at it once for the 403s that do not speak about your key but about the contract it belongs to: every key of the same contract receives them at the same instant, and they are the only ones where asking us for a new key is not the remedy. We do not copy them here: there is one table only, so it cannot fall behind.

Rule 2 — the 422 is different: it has no code, and it sends your texts back to you

The 422 is the error a new client meets first (a wrong field in the body: input on /embed instead of texts, documents missing on /v1/rerank). It arrives before authentication — a wrong body and a missing key give 422, not 401 — it consumes nothing, it carries no X-RateLimit-* headers, and it is the only response that does not have our shape: it is FastAPI's body.

{"detail": [{"type": "missing", "loc": ["body", "input"],
             "msg": "Field required", "input": {"texts": ["…YOUR TEXT…"]}}]}

Two consequences. The first: if you branch only on error.code, for the 422 you find nothing and your message goes mute. Read detail[].loc and detail[].msg: they tell you which field is wrong, which is exactly the information you need. The second: detail[].input was the echo of what you sent, and since 17/10 at 03:37:52Z it is gone. Until that day a batch of 64 texts (89,600 characters) came back as 89,909 bytes of error; trpg had measured it from their own box with 16 texts, 25,969 bytes sent and 26,085 bytes of error returned. Now the response weighs 402 bytes for one text and for sixty-four, measured at the edge, and inside there is no piece of your body. So we withdraw the warning this row used to give you — «do not log error bodies in full» — because it was a risk of ours that we were unloading onto you: today, logging one of our error bodies in full does not put a single character of your texts into your logs.

Rule 3 — 200 does not mean «all the vectors are there»

Count. data must have as many elements as the texts you sent, and the index values must be 0…n-1. If fewer arrive and you pair texts to vectors with a zip (or a parallel foreach), you index the wrong text with the wrong vector and never notice: the fault shows up months later, as search quality getting worse. On the rerank the same rule applies with one caveat: results has min(top_n, documents) elements when top_n is positive — fewer than documents is legitimate if you asked for top_n, never more — and every index must be inside the range of the documents you sent. A top_n that is zero or negative is not applied: every document comes back and the response says so in ignored_values: ["top_n"], so do not compute the expected count with a bare min — min(-1, 4) is -1, and a client that stopped there blamed the service (attesi -1 elementi in results, ricevuti 3) for a wrong value of its own. That was the case of our own two published clients, fixed on 7/10 together with the service.

Rule 4 — an empty list is not an answer

This one was found by trpg in their own house on 28/9, pulling the thread of a warning of ours, and it is the nastiest of the four because it produces no error and no result. Their search silently discarded — a bare continue, in two places — every chunk whose dimension did not match the question's. Consequence: an index built with one model and queried with another does not give an error, it gives zero results, and the assistant answers with no sources, plausibly and pointlessly. Measured by them read-only, with a fake service in our place: a 512-value question on a 1024 index, no exception, 0 results; an old index after a change of model, 0 results out of 471 chunks all discarded, without a trace anywhere. The way in, at their end, was the installation packages with the model name hard-coded and the service address absent: an installation at a customer's site could have AgileV in its configuration and still build the index on the historical 512-value provider, with no signal at all.

Five countermeasures, all at zero cost except the last. (a) and (e) hold on the index you ALREADY have: (b), (c) and (d) protect what you write from tomorrow on, while (a) applies to chunks written months ago, because their dimension is measured and does not require anyone to have declared it. It was trpg who pointed this out to us by putting it into operation, and for them it is worth more than the others for exactly that reason. (e) came later, and it is the remedy for whoever does not have (d).

• (a) At start-up, one single check — on the DIMENSION, not on the name: GET /health needs no key, consumes nothing and declares "dim" (today 1024). Compare it with the dimension of your index and stop at once if it does not match, instead of finding out once the index is built.
  The name, NO, and it has to be said because it is a real stumble (trpg reported it to us on 28/9, after putting (a) into operation): in /health the model field is the underlying model (bge-m3), while /v1/models lists the catalogue name (agilev-m3), which is also the one that comes back from /v1/embeddings. They are two vocabularies and the two values are not comparable: whoever reads this line as «compare model and dim» and compares the name finds a mismatch at every start-up, with the service perfectly healthy. The counter-proof, measured by us: /v1/embeddings says agilev-m3, /embed says bge-m3, and the two vectors are identical bit for bit. The name is only compared within the same endpoint — and that is (b).
  And the dimension is counted on the floats. With output_dtype binary or ubinary a 1024 vector arrives as 128 values (1024 bits packed), and with encoding_format: base64 it arrives as a string: len(vector) == dim only holds on float. Measured: 7 combinations out of 10 would trip the guard wrongly — and a guard that discards wrongly, in silence, is exactly the damage this rule is about.
• (b) Look at the model of the RESPONSE, not the one in the request: every response declares the model used (/v1/embeddings → agilev-m3, /v1/rerank → agilev-rerank, /embed → bge-m3). What you sent is no guarantee: the field is not validated today and /embed does not look at it at all. The comparison holds because the two values come from the same vocabulary: that of the endpoint you called. Do not compare the name of one endpoint with another's.
• (c) Never discard in silence. If a chunk has the wrong dimension, count it and raise your voice: zero results on a full index is a fault, not a search without matches.
• (d) Write the model NEXT TO THE CHUNK, at the moment you build the index. It is the corollary trpg sent us on the evening of 28/9, and they paid for it: their index has 442 chunks from one provider and 29 of ours, both at 1024 values, and the chunk table does not record the model anywhere. No guard on the dimension tells them apart, and the information has never been declared anywhere: they tried to derive it from search measurements and the result is undecidable. One text field per chunk, written when the vector is born, costs nothing today; deriving it tomorrow is possible (e), but it costs one call per chunk. Here we had written «impossible to recover tomorrow»: that was false, and we had written it without measuring it. (e) is the correction.
• (e) If you did not write the model down, DERIVE IT: the stored vector against the stored text. Re-embed the chunk's text and compute the cosine with the vector you hold in house. Measured on 28/9 with the real model on 64 chunks, 13 scenes, 0 failures: the same text and our model give cosine 1 (maximum deviation 2.2·10^-12, and 3.9·10^-12 even across our deploy from 1.1 to 1.2); another space of 1024 values gives |cosine| ≤ 0.10; compressed storage holds (float16 0.99999998, which even holds the exact verdict; int8 0.9998, one bit 0.75) and so does a text edited afterwards (0.9755). Between ours and the strangers there is a factor of 7 and nothing in between: the verdict is decidable. 29 chunks sampled are enough to notice a 10 % contamination with 95 % confidence, and an index of 471 chunks is labelled entirely in two minutes of service. Thresholds, traps (the most insidious: re-embedding the raw text when the indexer prepared a different one) and a protocol to rewrite in house: the provenance of vectors.

Rule 5 — no response may leave your client with an «I do not know» code

This one we found ourselves, on 29/9, by counting how many shapes an AgileV response can take. The 422 of Rule 2 is not an isolated case: the responses without error.code were four, on 5/10 we found a fifth and on 8/10 a sixth we had not counted, and AgileV does not write them — they are written by the framework beneath us or the edge in front of us, before our code sees the request.

There were four; then, measuring in a cage, we found two more, and today none remains open. The last one was the 422 of the invalid body, and we closed it ourselves on 17/10 at 03:37:52Z: it now has error.code BODY_INVALID, and it no longer sends back the texts you sent us — which was the reason this page told you not to log error bodies in full. The three before it — the 405, the 413 of the oversized body (found on 5/10) and the three 50x of the engine that does not answer (found on 8/10) — have been closed since 9/10 at 07:31Z. In all of them detail was a string (or the body was not even JSON, or there was no body at all), so they all fell into the same branch of your error reader: the one that does not know what to do. If you put ? there, whoever calls you receives an alarm that cannot be routed or counted — it is Rule 1 dressed up as an edge case. The two that were closed were closed by the edge, without touching the app: on 29/9 the four points where it stops a request by itself, and on 5/10 at 06:40:01Z — written on 1/10, live with the reload of 5/10 — the 404 that comes back to it from the engine — /v1/* passes whole, so /v1/embed or /v1/keys reached FastAPI and came back without a code (measured: 0 out of 7 → 7 out of 7 among the groping requests under /v1/). The 405 stayed until 8/10, and the reason was this: the endpoint exists, and telling it «endpoint does not exist» would be false advice. The reason still holds — the 405 does not reuse the body of the 404 — but it ruled out one cure, not the cure: since 9/10 at 07:31:16Z it has a code of its own, METHOD_NOT_ALLOWED, and the message says which method that endpoint wants, taken from the Allow the framework already sends. And these are not hypotheses: on 13/9/2026 at 01:26:32Z a customer called /v1/embeddings with GET and received the 405; on 15/9 at 12:43:55Z another received a 502 from the edge because two seconds earlier we had restarted the service. Neither of them reported it to us: we read it ourselves in the edge log, and from their side it was an error without a name.

What to do. Do not guess from the body: those bodies are not ours and nobody promised what would be put inside them. Deduce the code from the HTTP status, which is the only guaranteed part — and these are five different behaviours, which with ? were a single one:

The corollary, and it is the part that gets forgotten: the body of those responses must not be put into the message. An error page from a proxy in the middle can contain anything, and it would end up in your logs labelled as an AgileV error. It is the same rule as the 422 echo (Rule 2), applied to bodies that are not ours.

The trailing slash: the 307 that is not an error (1/10)

A real endpoint called with a trailing slash — POST /v1/embeddings/ instead of POST /v1/embeddings — does not answer 200 and does not answer with an error of ours: it answers 307, empty body, with Location: pointing at the same endpoint without the slash. It is written by the framework (redirect_slashes), not by our code, and the 307 preserves method and body: a client that follows redirects does not even notice. A client that does not follow them — and many disable them on POSTs, which is a prudent choice, because a redirect on a POST re-sends the body — sees a status that is not 200 and has no error.code. It is not a third shape without a code: it is not an error, it is the same answer at the right address.

What you need to know, all of it measured (bench/misura-la-barra-in-fondo.py): following it is safe, because the Location is absolute and in https and the body of the 307 is empty (0 bytes: nothing of what you sent comes back); it does not consume (the same call counts 1 request without the slash, 1 following the redirect, 0 stopping at the 307: the redirect is not billed); it does not carry the X-RateLimit-* headers, so whoever paces themselves on X-RateLimit-Remaining reads nothing on a 307; and it does not behave the same way on every endpoint — GET /health/ and POST /embed/ do not even reach the app and receive the edge's 404 NOT_FOUND, which says «endpoint does not exist» about an endpoint that does. Live on 1/10, on the eight public endpoints: 5 answer 307, two the edge's 404.

What to do: do not write the trailing slash. The three clients we hand out cannot add it by themselves (they strip the slash from the base address) and since 1/10 they have the 3xx branch of the table above, which returns REDIRECT with the remedy instead of an «I do not know» code: measured 1 client out of 3 → 3 out of 3 that get something routable out of it.

The measurement, on the clients we hand out

It is not advice by eye: it is what we found on our own doorstep. bench/misura-errori-client.py captures the real error bodies of the service (from a copy of the code loaded into memory, without sending a request to production), replays them from a fake local server and records what the caller sees in the three reference clients: on 28/9 12 scenes × 3 clients = 36 rows, since 29/9 15 scenes × 3 clients = 45 rows (the three added ones are the 405, the app's 404 and the edge's plain-text 404: real shapes nobody had ever passed to the clients). The last two no longer arrive that way today — the edge rewrites them — but the scenes stay: a client must know what to do with the old shape too, because an old client talks to a new edge and vice versa, and because Rule 5 (the code is deduced from the status) is written precisely so as not to depend on the body.

All three needed fixing, and in the same way. The 422 becomes the code BAD_BODY, with the message corpo della richiesta non valido - body.input: Field required (only loc, msg and type: input is never read). A mutilated or missing data becomes BAD_RESPONSE, with the message attesi 2 elementi in data, ricevuti 1. Since 29/9 the four shapes without a code come out as NOT_FOUND, BAD_METHOD, UPSTREAM_ERROR and NO_CODE, and the body no longer enters the message. These codes are put there by the client, not by the service: they are for you to branch on, and they are the same in the Python, in the PHP and in the LangChain adapter — word for word.

The same 45 rows measured by capturing the error bodies from the 1.1 code that is in production today and from 1.2 give the same result: you can copy the clients now, you do not have to wait for a deploy of ours.

And how many times it really happened, read from our edge log for September, read-only (no request to the service, no address in the output): out of 4,310 requests served, 20 responses without a contract code reached somebody who actually uses the service — 13 on a real endpoint and 7 while looking for the endpoint. The honest count cost two corrections: the path is not enough to say who someone is (a scanner trying /v1/.env is not a customer) and 131 of those "customers in trouble" were our own acceptance bench, which asks for 404s on purpose in order to check them. From 151 to 20. If we do not tell you it this way, the number looks like an alarm and it is not.

17 new tests hold the behaviour in place (among them one that passes eight real errors of the service to the three readers and demands that none of them lets the echo out), and 11 mutations of the clients' code make them all fail: if somebody puts the old reader back, the bench sees it. For Rule 5, another 9 tests and 16 mutations (15 that must fail and one that, changing nothing, must not fail), tested on all three clients — three copies of the same rule are three places where the old one can survive.

Rule 6 — the configuration that disappears inside a web server

This one was not taught to us by an error: it was taught to us by trpg measuring in their own house on 29/9. In their container, under Apache $_SERVER['LLM_API_URL'] is null while getenv() and $_ENV carry the value. From the command line, on our box, it is the opposite: getenv() and $_SERVER carry the value and $_ENV is empty (the default variables_order, without «E»). And php-fpm with clear_env = yes, which is the default value, empties the process environment: there only $_SERVER is left, filled from the pool's env[...] lines. Three rooms, three different results for the same variable: whoever reads only one is betting on the room, and when they lose the bet there is no error at all — there is a configuration that works in every command-line test and is no longer there inside the web server.

Why it is worse for AgileV than for others: the variable that disappears is the address. If you pointed the client at a private front end — a WireGuard, a second box, the router — and AGILEV_URL does not arrive, the client does not stop: it takes the public default and your texts leave onto the Internet, without a word, with all the tests green.

What we found in our own house, before writing this rule. Surveying the six examples we publish: the key stopped 6 times out of 6, the address never. Four cells ended up silently on the public address; two examples, with an empty AGILEV_KEY (not absent: empty), sent the text to public production with an empty Bearer; and in ricerca_ibrida.php the address was decided inside embed(), so it was not checkable without making a request, not even by us. Now it is 0, 0 and 0 (bench/misura-indirizzo-che-sparisce.py, control group taken from git, 22 mutations out of 22 caught).

The four rules, if you write the client yourself: (1) read the environment from all the sources the room may have — in PHP getenv(), $_SERVER, $_ENV, in that order, because getenv() is the process environment and not something that arrives with the request; (2) a value that is present but empty is an error that names the variable, never a fallback to the default; (3) always say where the address came from, and for the key say the source and never the value; (4) if the address must be private, put in a guard that stops on the default — and write it in the code, not in an environment variable: a guard that lives in the same environment disappears together with what it is supposed to protect, precisely in the room where it is needed.

The diagnosis in one command, when «it works from the command line and not under the web server»: php ricerca_ibrida.php --dove or python agilev_client.py --prova print the address and its source (argomento|getenv|server|env|ambiente|predefinito) and never print the key.

What we do not claim: we did not measure the php-fpm room ourselves — php-fpm is not installed on our box, and that line is documentation, not a measurement of ours. The Apache one is trpg's. Ours is the command line. The three rows of the table say who measured what, on purpose.

What you have to do

If you use our clients (agilev_client.py, agilev_client.php, agilev_langchain.py): copy them again. The fix is inside, it changes no signature and changes nothing on the happy path. (a) is in there too, done the way it holds — a single line, at start-up:

av.controlla_dim(1024)        # Python     $av->controllaDim(1024)   # PHP
emb.controlla_dim(1024)       # LangChain (AgileVEmbeddings)

It reads dim from /health (no key, no consumption), stops with DIM_MISMATCH if it is not your index's dimension, and does not look at the name. From that moment it also does (c) for you: a vector of the wrong size becomes a BAD_RESPONSE («attesi vettori da 1024 valori, ricevuti da 512») instead of entering the index, and two different lengths in the same batch are an error even without having called controlla_dim.

If you wrote the client yourself: the points above are to be gone through one by one. The quickest is Rule 2 — look in your code for whether you log the body of an error response in full. The most important is Rule 4, because it is the one that dirties an index without making any noise; and read it all, including the paragraph about the name: a guard written on the wrong value is worse than no guard at all, because it makes you switch off the good one as well.

In both cases, a one-minute test: make a call fail for real (a wrong key, and a body with a missing field) and look with your own eyes at the message that reaches the reader — a user, a ticket, a log. If it contains the word «Array», the repr of a dictionary, a ? or a piece of your texts, you have found the defect. It is what trpg did.

None of this changes your client, and that is the point. The 404 has had our shape since 5/10 and the 405 since 9/10 at 07:31:16Z: both were closed by the edge, without touching the app and without you copying anything, because you deduce the code from the status. And the 422 does not remain: we gave it our shape ourselves, in the app, on 17/10 at 03:37:52Z — a stable code (BODY_INVALID) and no echo of your texts — and it was not a change of contract: the contract remained 1.2, and no key had to be reissued. Details in API.md, section «Errors». This page is self-contained, with no scripts and no remote resources: you can read it and copy it wherever you like.

questa pagina in italiano