Building a cross-bid buyer dossier for a bid-response system, starting from the discovery that the entity in the feature's first sentence did not exist in the data model.
Go
MongoDB
Vue 3
AWS Bedrock
OpenSearch
Four asks arrived together and the first one was historic buyer activity. Every requirement that starts with the words "this buyer" needs there to be a buyer. There wasn't one. The customer was a free-text string on each bid, the only lookup was a case-insensitive substring match, and nothing anywhere recorded whether a bid had been won or lost.
0external integrations in the final design, after the CRM turned out to be a different product entirely
The problem
A bid manager opening a new tender wants one thing before anything else: what do I need to know to
bid to these people? Have we sold to them, who owned it, what did we send, did we win, and what did
they ask for that they always ask for.
The card was already on the page, and had been for months, with the right heading, the right sentence
from the request, and a disabled button labelled Coming soon. So the ask was not “add a button”.
It was everything behind it. And behind it there was less than the request assumed.
01
The buyer was a string
A free-text customer name per bid. Nothing normalised it, nothing deduplicated it, and the only lookup was a case-insensitive substring regex. So an acronym and the full legal name it stands for were two different buyers, and a short name that is a prefix of an unrelated organisation matched it.
02
There was no outcome
The status field runs draft to archived and has no won-or-lost concept anywhere. A dossier built on it can say "we have bid to this buyer three times". It cannot say the sentence a bid manager actually wants, which is "we lost the last one".
03
The history store expires
The obvious source, the per-bid activity feed, carries a one-year expiry index. The two-year-old engagement is precisely the row that has already been deleted, so building buyer history on it produces a history that thins as it becomes valuable.
04
Scope creep is one question away
Renewal timing, whitespace analysis and upsell suggestions all came up. They are account planning, and there is already a CRM. Ruling them out is what turned the spine of the page into material we had produced ourselves.
What I built
This one shipped. The dossier, the identity model, the outcome field and the buyer-level routes are
all in the codebase. The external market-data phase is deliberately still a verification step rather
than an integration.
Already existed
The card, on the page, disabled, with the exact heading and sentence the finished page kept
Bids readable group-wide behind a single route permission, so a cross-bid page raised no new access-control question
A shipped customer-name normaliser behind the content library and the reference register: whitespace and case only, no aliases
A content-library upload flow with customer tagging and a withhold-from-answering flag, and a documented boot-backfill pattern guarded by a distributed lock
An app-only credential and tenant plumbing for the productivity suite, able to read meetings and transcripts, which is the tedious half of a source nobody had used yet
What I added
A buyer record keyed on a normalised name, created lazily on first view, with hand-added aliases and the observed spellings behind each of them
A stored buyer key on bids, stamped on create, re-derived on rename, indexed, and backfilled once under the existing lock pattern
The same key on bid-less questionnaires only, with all three write paths that can invalidate it clearing or re-deriving it
A won / lost / no-bid / unknown outcome on the bid, with a validator, distinguishing "not recorded yet" from an explicit "unknown"
A per-request read model at a buyer-level address rather than a persisted report behind a per-bid one
Projected, batched repository reads and a capped prior-bid list whose truncation is logged rather than silent
A hand-written customer-knowledge block, the part that makes the page worth opening, and later and on conditions, a never-persisted AI overview paragraph over the assembled facts
One buyer record, carrying the two representations its data stores can actually match on
Five problems worth describing
01
An identity function and a search affordance are not the same thing
Symptom
Every query the feature needed began “for this buyer”, and the only buyer-shaped predicate in the
codebase was a case-insensitive substring regex behind a list filter, already wired end to end,
from a query parameter in the browser client through to the repository.
Diagnosis
That filter is correct as what it is: a search box. As an identity function it fails in both
directions. It splits one buyer into many, because an acronym and the full legal name behind it
share no substring. And it merges buyers that are not the same, because any short name that is a
prefix of a longer, unrelated organisation’s name matches it, which in a dossier means listing
another company’s tenders as this buyer’s history.
The dangerous part is that it was the path of least resistance. Reusing it required writing no new
query at all, so the wrong turn was available to anyone who never asked whether the existing
filter was an identity function.
Decision
Store a normalised key on the bid and match on equality, never on a substring. A small buyer
collection keyed on that value, created lazily on first view, holding the alias list, because
aliases are the whole problem and there is nowhere else to put them, and per-bid copies of buyer
facts are how three bids end up with three drifting versions of the same knowledge.
Two deliberate limits. Aliases are added by hand, by whoever notices the duplicate: automatic
entity resolution over a few hundred bids is solving a problem that does not exist yet, and its
failure mode is silently merging two real buyers. And the key is re-derived when the customer
name is edited, not only stamped on create. The name is editable, and a stale key is a bid that
silently drops out of its own buyer’s history.
The outcome field is the cheapest high-value thing in the whole plan and the only one that is not
free. Adding won / lost / no-bid / unknown with a validator is an hour; getting it populated is a
standing data-entry ask on a team that has no habit of closing a bid out in this system. It is
written up as a data-entry cost rather than a code cost for that reason. The enum also
distinguishes an unset value, meaning not recorded yet, from an explicit unknown, because on a
dossier those two mean very different things and collapsing them would make every historic bid look
deliberately unresolved.
02
You cannot compose a normaliser inside a query
Symptom
A normaliser already existed and shipped, doing whitespace collapse and case-insensitive matching,
backing the customer field on both the content library and the reference register. The new buyer
key needed punctuation and legal-suffix stripping and alias resolution on top. The obvious move is
to make the new one a superset and compose it over the stored value.
Diagnosis
A superset is right, and composing it at query time is not a query. Both of those collections
normalise on write and keep the original spelling, and they match it with an anchored
case-insensitive comparison. So “normalise the stored value, then compare” means loading every
content-library document and every reference on each page load and normalising in application
code, which is the exact scan the identity model exists to remove, reintroduced on two more
collections.
The same trap has a second door: the reference register’s own search filter is an unanchored
substring regex, which is the collision bug of exhibit 1 on a different collection.
Decision
Keep both representations on the buyer record, and be explicit about which store consumes which.
Aliases are normalised keys, and they are what the keyed collections match on. Spellings are
the observed original spellings, the display name plus whatever each alias was typed as, and
they are what the two spelling-keeping collections match on, as a set of anchored patterns, which
is a clause those repositories already build.
That also keeps the human action singular: whoever adds an alias supplies the spelling they saw.
One action, two representations, rather than two alias mechanisms to keep in step.
Matching the same buyer across stores that hold different things
One canonical key
normalise(stored value) == key
reads as clean. is not a predicate:
→ load every library document
→ load every reference
→ normalise in Go, per page load
i.e. the scan this model removes,
on two more collections
Keys and spellings, per store
stored key ∈ {key, …aliases}
→ bids, bid-less questionnaires
→ indexed set membership
customer ∈ {anchored spellings}
→ library, reference register
→ a clause they already build
one human action: the alias, and
the spelling it was typed as
And one non-goal, stated so nobody chases it: an anchored case-insensitive comparison cannot be
served from an index, so the two spelling-matched blocks will not produce an index scan however
they are written. That is pre-existing behaviour for every customer filter in the application, not
a regression introduced here, so the validation item that requires an index scan is scoped to the
two keyed collections only. An unqualified “all queries must be indexed” would have sent someone
after an impossible one.
03
The thing that goes stale when a different record changes
Symptom
Two report types already existed on a bid: one model call over that bid’s tender index, persisted,
with tender-scoped citations, a generation job, a status endpoint and a card state machine. The
obvious implementation is a third one.
Diagnosis
It fits none of that. This dossier’s provenance is record links, meaning bids, documents and
register entries, and the citation scopes on the existing reports are all search indexes. But the
deciding argument is staleness, and it is a structural one: this artefact changes when a
different bid changes. Persist it and it is silently wrong the moment a colleague closes out
another deal for the same buyer, and nothing in the system would know to regenerate it.
Computed per request, it is always current, and the entire apparatus disappears: no generation
job, no template type, no status polling, no stale badge. The card navigates. It never says
Generating….
The address had the same shape of error in the first version of my own plan, which specified the
dossier under the bid. Nothing on the page is a fact about this tender; it is all facts about
the buyer. Hang it off a bid and three bids with the same buyer produce three URLs rendering
identical pages, with the notes and aliases stored against whichever bid someone happened to be
looking at, so saving a note from one leaves it invisible from the others. It also cannot be
reached before a bid exists, which is exactly when someone wants to look a buyer up.
Decision
A read model at a buyer-level address, linked from the bid card. Same query, same blocks, same
permission: a choice of address rather than a bigger build, and five minutes now against a URL
migration later. The rejected report type is written down with its full cost: a generate handler,
a status handler, a template branch, a repository case and a card state machine, to deliver a page
that is a database query.
Empty state matters more on this page than anywhere else in the feature, because a first-time
buyer is the common case. “No prior activity for this buyer” has to read as a fact about the
buyer rather than a broken page, and it is the honest place to put the is this the same buyer
as…? alias affordance. Related discipline: the card had shipped disabled for months, and a live
button over an empty page is worse than a disabled one, so the empty state was finished as part
of the same change that removed disabled, not after it.
04
Three counts that would each have been wrong, and one that was a disclosure
Symptom
“What we answered for them” sounds like a count. Three separate facts about the data model make
the obvious count wrong, and they are only visible if you go and look.
Diagnosis
One workbook is sixteen documents. A returnable parent copies the bid’s customer name, and
every sheet copies the parent’s name and its bid id. So a fifteen-sheet workbook matches a
naive count sixteen times, and the honest-looking sentence “we answered forty-seven
questionnaires for this buyer” can be three workbooks.
A missing bid link means “shared with everything”, not “orphaned”. Reusable-answer provenance
is declared on the model and written by none of the five import or creation paths. And the
repository treats a nil bid id as globally shared, so the obvious provenance query does not
return nothing. It returns the entire corpus, labelled as this buyer’s answers. That is a
confidentiality problem wearing a wrong count’s clothes.
The early-engagement work is not reachable through bids. A questionnaire carries its own
customer name, deliberately independent of any bid, and is routinely answered before a bid exists.
A dossier that walks only bids omits exactly the work the block is meant to surface.
Decision
Exclude returnable sheets and count parents, as the returnables screen already does. Cut the
reusable-answers block to a labelled empty state rather than ship a query that is wrong in the
worst direction, with the provenance stamping named as a prerequisite ticket rather than a
nice-to-have. Add the key to bid-less questionnaires only, and match the attached ones by bid
id instead: a questionnaire under a bid has its customer name copied from that bid, so keying it
too would create a second source of truth that goes stale on rename. Splitting on that makes the
two branches disjoint by construction, so the union needs no deduplication pass.
Then bound the whole thing. Every repository read on the collections involved took a single bid
id, so the dossier as first sketched was a per-prior-bid fan-out. The fix is set-membership
variants plus projections, a cap on prior bids shown newest-first, and logged truncation, so
“3 prior bids” never silently means three of eleven.
Three write paths can invalidate a bid-less questionnaire’s key, and only the first is obvious.
Attaching it to a bid must clear it. Attaching a returnable parent must clear its sheets too,
because they move by a different method than the one the parent uses. And editing the customer
fields long after upload rewrites the name via a targeted update, where an explicitly empty name
is a clear rather than a no-op. That third one is the path most likely to be missed, precisely
because it is neither the upload nor the attach.
05
The cheapest step in the plan was the one that leaked
Symptom
The hand-written half of the page is the half worth opening: relationship state, named people and
roles, the incumbent and competitive read, how this buyer buys. All of it already exists, in
kickoff decks. Rather than build an upload control, reuse the content-library upload that exists:
tag the deck to this buyer and the page reads whatever is tagged. One upload, two payoffs.
Diagnosis
The set of content classes withheld from the answering lane is exactly two, and the comment above
it says plainly that a file with no class tag is not withheld. So a kickoff deck tagged only
with a customer name is chunked, indexed and retrievable by the answering lane, and that deck is
the single worst document in the corpus to make quotable. Its weaknesses column is our
weakness, not the buyer’s, and its margin notes are working shorthand about the buyer written for
an internal audience. A citation puts either in front of a customer.
The cheapest step in the plan turned out to be the one whose failure mode is a buyer reading our
competitive read on them.
Decision
The reuse stands, but the upload is not the bare existing flow. Withhold-from-answering is not
optional on this path, and it is a checklist item with a test: upload a deck for one buyer, then
run answering on an unrelated bid and confirm no chunk of it is retrieved or cited. And the retag
case is covered explicitly. If a deck goes up un-withheld and is flipped afterwards, the cached
answers derived from it must be invalidated, or a source that has been withdrawn keeps being
served from cache.
The AI half is deliberately narrow. Drafting the knowledge fields from a deck is
suggest-then-confirm: editable unsaved values, nothing written to the record without a human
saving it, never re-run over a field that already holds a human-entered value, and only ever over
the one document the user picked. No background crawl of everything findable about this buyer.
An LLM reading a deck densely packed with shorthand and judgement calls will produce fluent,
plausible, wrong values, and a suggestion a human deletes costs nothing while a silent auto-fill
costs the correction nobody knows to make.
I rejected a generated narrative summary in the plan, then built one, which is worth stating as a
reversal rather than glossing. The rejection was that once the knowledge fields are filled the
dossier already reads as the answer, so a paragraph restating it adds translation risk for little
gain. What changed my mind is that the paragraph is the thing a bid manager reads in the ten
seconds they actually have. It shipped on conditions: computed on request and never persisted,
grounded strictly on the dossier’s own assembled facts plus a small capped set of the buyer’s own
tagged documents, with a prompt that forbids inventing names, dates, outcomes or relationships and
instructs it to say the material is thin rather than pad it out. No tender content reaches it,
which was the original objection’s real target.
Where it landed
16questionnaire documents in one fifteen-sheet workbook, each carrying the buyer’s name, so a naive count multiplies
3write paths that can invalidate a bid-less questionnaire’s buyer key; only the first is obvious
2content classes withheld from answering by default. A customer tag alone is not one of them
0external integrations in the final design, after the CRM turned out to be a different product entirely
Verified
A buyer record keyed on a normalised name that is a strict superset of the existing customer normaliser, so the two do not become incompatible keys for the same fact
The key is stored on bids, re-derived on rename, indexed, and backfilled once under the same distributed-lock boot pattern the codebase already used for a prior backfill
Bid-less questionnaires carry the key and attached ones deliberately do not, with all three invalidating write paths clearing or re-deriving it, and the two branches disjoint so no deduplication is needed
A won / lost / no-bid / unknown outcome exists on the bid with a validator, and an unset value is distinguishable from an explicit unknown
The dossier is a per-request read model at a buyer-level address: no generation job, no status polling, no stale badge, and two bids for the same buyer open the same URL
Prior bids are capped newest-first and truncation is logged, so a shown count is never quietly a partial one
The page issues no decrypt: the encrypted bid field is not read at all, rather than being read and discarded
The reusable-answers block ships as a labelled empty state rather than as a query that would have returned the whole corpus as one buyer’s answers
The AI overview is computed on request, never persisted, and grounded only on the dossier’s own facts and the buyer’s own tagged documents
What this doesn't solve
External award history is still a verification step, not an integration. The public feed publishes contract notices, meaning who won rather than who bid and lost, so it can never show "their past bids", and coverage may miss state, university and local-government buyers, which is a large share of the pipeline. A coverage sample against ten real buyers comes before any data partnership, and the UI wording has to say prior awards rather than prior bids.
Reusable-answer provenance is still unwritten, so the strongest block on the page is still empty. It is a prerequisite ticket rather than an enhancement, and the empty state says so.
Outcome is only as good as the data entry behind it. The field exists; the habit of closing a bid out in this system does not yet, and no amount of code creates it.
A per-bid response tone is fully built, validated and applied in both answering tiers, and completely invisible: the browser client’s metadata parser enumerates keys explicitly and drops it. Exposing it is not a select element. There is no bid edit path at all, and choosing a tone explicitly rather than leaving it unset silently stops answer-cache population for that bid. Recorded and deliberately not built here.
There are three distinct tone concepts in flight: the per-bid one above, a reserved account-level setting, and a per-question register the answering path already controls. Nobody should build a tone surface before reconciling them, which is why this page treats a buyer’s preferred tone as an informational field that drives nothing.
The reference-register block links to a list rather than a record, because the register has no per-record route or deep link. Stated as a limitation rather than written as if the target exists.
Buyer identity is hand-curated: aliases are added by whoever notices a duplicate. That is the right call at this corpus size and it will need revisiting, not extending, at ten times it.
What I'd do differently
Grade what exists before designing anything, and grade it honestly. Four things arrived as one
ask. One was already fully built and merely unreachable from the interface; one named an integration
that existed against the wrong product of the same vendor and only as per-session agent actions, not
a service the page could call; one needed a data partnership that may not cover most of our buyers;
and one was the actual work. Half a day of reading changed the shape of the plan more than any
design decision in it.
Check that the noun in the requirement exists. “Historic buyer activity” has a buyer in it, and
the system had a string. Everything hard about this feature followed from that one absence, and it
was findable in ten minutes by grepping for how the customer name is queried. I would now read a
feature request first as a list of the entities it assumes.
Ask whether a source is populated, not whether it exists. The CRM has a lost-reason field, so
the temptation is to design around it. The right question is whether anyone fills it in, because a
field that is ninety per cent blank is not a source. The same test disposed of several candidate
document locations: one person’s cloud drive is not a source, it is someone to interview. Filtering
every candidate through consistently populated, and in a shared place removed the entire external
half of the design and left something that could ship.