Self-Serve Billing Needed an Immutable Price, Fully Audited
A customer who can’t find their own invoice without emailing support is a customer who assumes something is being hidden from them. ZopNight’s Settings now has a Billing page that brings the plan a customer is on, every invoice issued, and their full payment history into one place. Each invoice opens as a complete document, printable or downloadable, and anything still outstanding can be paid immediately through a secure provider that ZopNight itself never touches for card details.
A stage run found 4 bugs in the cron that actually issues these invoices, one day before this shipped. Credit notes and refunds sit next to the charges they adjust, so a customer can trace any correction back to its source without asking. None of that visible surface is difficult to build. What’s difficult sits underneath it, in a service that has to agree with itself about what a customer owes, forever, even after the price list has changed six times.
Identity Is a Pair, and the Identifier Stays Opaque
The new billing service treats a tenant, meaning the SaaS vendor itself, as the top-level scope. Below that, a customer’s true identity inside billing is a pair: the tenant, which is read from the route on every request, and a customer identifier that billing treats as completely opaque. It doesn’t parse the identifier, infer any structure from it, or enumerate it looking for patterns. The cross-tenant case is tested directly rather than assumed: both zopdev and a second tenant, gofr, have a customer literally named acme, and the system is required to keep them apart on that identity pair alone.
That identity model is what makes the catalog, orders, GST-compliant invoicing, credit notes, promotions, metered usage, and entitlement resolution over HTTP and gRPC all safe to build on top of, with nothing extra needed for what happens when two tenants both have a customer with the same name.
A Price That Can Change Is a Price That Can’t Be Trusted
An order line records the exact price id it was bought through, not just a dollar amount at the time. That single design decision is what makes the rest of the pricing model enforceable. Creating a second live price for the same version, currency, and billing interval is refused outright with a 409 response naming the existing id that already holds that slot. A price that needs to change isn’t edited. It’s retired through a dedicated delete endpoint, and a new price takes its place.
| Situation | With a mutable price | With an immutable price |
|---|---|---|
| Price changes after a customer’s order | That customer’s past invoice total can silently shift | Past invoice stays tied to the exact price id it was issued against |
| Two price rows exist for one plan tier | Which one applies depends on write order in the database | A second live price for the same slot is refused with a 409 |
| Auditing a disputed charge | Requires reconstructing the price table’s state at order time | The order line already names the price id directly |
A related gap closed in the same release: the service had been writing an audit trail on every meaningful action since it was first built, with no endpoint to actually read them back. A new tenant-scoped endpoint exposes that history now, which matters because a billing dispute is exactly the kind of disagreement that gets resolved by looking at a log, not by asking someone to remember.
Four Bugs a Stage Run Found in One Night
A full end-to-end run of the invoice-generation cron against stage, one day before this shipped, surfaced four separate bugs in the exact code path that actually issues invoices on a schedule. All four were fixed together in the same release, because they shared enough of that path that fixing one in isolation would have left the system in a confusing half-repaired state.
| Bug | Severity | What actually happened |
|---|---|---|
| Lock name over 64 characters | High | Every customer identified by a UUID failed to issue an invoice at all, in both the scheduled cron and the webhook payment-confirmation path that shares the same lock |
| Free-plan order missing | Medium | A free-plan account generates no order at a zero subtotal, but the issue step still required one, rejecting 46 of 47 customers on every single tick |
| Empty order-lines logged as an error | Low | A receivable invoice legitimately carries no line items by design, but the empty result was logged as a hard error, creating pure noise |
| Dunning fails open with no notification service | High, safety | With the notification service URL unset, the cron still advanced a customer through dunning stages and revoked their access, having never actually sent them a warning |
The lock-name bug is a good example of how a limit that seems like a pure implementation detail becomes a customer-facing outage. MySQL’s advisory lock names cap at 64 characters, and the identifier this system built for a UUID-based customer ran ten characters over that limit on every single attempt, which is not a rare edge case, it’s every customer of that shape. The fix hashes the identity portion of the lock name instead of using it raw.
Fail Closed, Not Fail Open, When Dunning Can’t Notify Anyone
The dunning fix is the one worth sitting with longest, because it’s the one where the old behavior was actively dangerous rather than merely broken. Dunning exists to warn a customer before their access is revoked for non-payment. If the service responsible for sending that warning has no configured URL, silently continuing to advance the customer through dunning stages and eventually cutting off their access means the warning step ran in name only. The customer loses access having never been told anything was wrong.
This works when the notification service is reachable and configured, which is the assumed steady state. The moment that assumption isn’t true, it now refuses loudly to advance dunning at all, because a silent fail-open in this specific path turns a missing environment variable into a customer-facing account suspension with zero warning. None of these four bugs had reached production. They were caught precisely because the release didn’t skip the step of running the actual cron, end to end, against something that resembled real account data before calling it done. Check Settings → Billing to see the invoice and payment history on your own account.
