Software Factories on AWS

Self-hosted platform for standing up and running software factories on AWS: complete DevSecOps environments with secure delivery pipelines, supply-chain controls, compliance evidence and multi-tenant governance — in minutes, not months.

Plan — Charging for compute an organization triggers

Compute is the one cost a tenant can run up without limit. Every other expensive dimension is metered — API calls, AI calls, automation calls, plugin builds by count — but nothing charges for how long that work occupies a core.

Status: PLAN ONLY — nothing here is built.

Direction: a new per-period quota metered at the work boundary, sized per tier, with a stackable pack for more. Named buildMinutes rather than “CPU credits”, for the reason in D1.

Why this is possible at all

Not from infrastructure metrics. container_cpu_usage_seconds_total is scraped (with container_spec_cpu_quota, the CFS throttle counters and machine_cpu_cores), so CPU seconds exist — labelled by container and pod, which is the wrong axis. Tenant work does not run in per-tenant containers:

No PromQL recovers an organization from a metric whose finest label is a pod shared by every organization.

So meter at the work boundary instead — when a build finishes, charge its duration to the org that asked for it. Two consequences, both good:

  1. Platform services are excluded by construction. Billing, frontend, platform, quota, message, reporting, the registry, Prometheus, Grafana, Loki, Thanos, Postgres, Mongo and Redis never appear, because none of them is tenant-triggered work. There is no denylist to maintain and nothing to keep in sync — the opposite of filtering infrastructure metrics, where every new platform component is a new exclusion somebody must remember.
  2. The signal already exists. plugin_build_duration_seconds{org_id} is already observed per org, and plugin_builds_total{status, org_id} already counts per org. The measurement is not new work; the accounting is.

What the machinery already gives us

Adding a quota type is compiler-guided. QuotaType is a union and QUOTA_TYPE_LABELS, QuotaTierLimits, DEFAULT_TIER_LIMITS and the frontend’s QUOTA_TYPE_LABEL are all Record<QuotaType, …> — add the member and tsc enumerates every table that must be filled. Nothing silently defaults.

Everything downstream is generic over VALID_QUOTA_TYPES and comes along for free: the 80%/100% threshold notices (inbox + email + webhook, with the existing opt-out), the usage ledger and its CSV, pooling at the account root, the at-risk views, the 429 with an upgradeUrl, and the add-on/bundle plumbing.

This is the opposite of the storage quota’s position (see the storage plan): storage is a stock that fits the per-period model badly and needs special handling everywhere. Build minutes are a per-period flow, exactly like every other quota here, so they inherit the whole apparatus.

D1 — Call it buildMinutes, not cpuCredits

Decision: buildMinutes, scoped to plugin builds, for the first release.

“CPU credits” promises a measurement we cannot make and invites an expectation the scope violates: pipeline executions run in AWS and are not metered at all — pipeline_executions_total does not even carry org_id. A quota called “CPU credits” that ignores the largest compute an org can trigger would charge the org that builds plugins and let the org running hundreds of pipeline executions pay nothing. That is not a rounding error; it is a fairness inversion customers will notice and be right about.

buildMinutes is self-documenting: it says what it counts, and nobody reasonably expects pipeline runs to be in it. Ingesting CodeBuild is then an honest follow-on (Phase 4) rather than a correction to a promise already made.

D2 — One minute of one vCPU, charged on wall-clock × allocation

Decision: a credit is a vCPU-minute; a build is charged ceil(duration_seconds × buildkit_vcpu / 60).

Per-build CPU is not observable — buildkit is a shared sidecar — but slot occupancy is, and it is what the platform actually pays for. An org that holds a build slot for ten minutes costs that slot whether it pegs the core or not, so occupancy is both the measurable quantity and the fairer basis.

buildkit_vcpu comes from the deployment: the minikube tree sets the buildkitd container to 1500m, i.e. 1.5 vCPU, so a 10-minute build costs 15 buildMinutes. It must be read from configuration rather than hardcoded, since an operator who raises the limit raises the real cost per minute.

D3 — Reserve an estimate, then true up

Decision: reserve a pessimistic estimate at admission; settle the difference on completion. This replaces an earlier draft of this plan that said “meter after, refuse the next one” — that was wrong, and the reason is worth keeping.

Build cost is unknown until the build ends, which argues for charging afterwards. But a budget checked at admission and charged at completion bounds nothing: N builds can all be admitted while the budget allows one. A Developer org with 15 credits left can still have twenty builds admitted; at a 10-minute build (15 credits) that is 300 credits spent against 15 remaining — a 20x overshoot, not a rounding error. Every other expensive quota here avoids exactly this by reserving atomically before the work.

So:

The two gates must unwind together. upload-plugin.ts already wraps the upload in a plugins reservation, with the discipline spelled out in its own comment: a throw refunds automatically, and markConsumed() at hand-off stops that refund once the queued job owns the slot. Adding buildMinutes makes two reservations on one submission, so a refusal of either must release the other — state the order, and unwind both in one place. Two budgets reserved together and released separately is the defect shape that leaks a slot on every refusal.

D4 — Charge failed builds

Decision: a build is charged for the time it ran, whatever its outcome — with one carve-out.

createBuildFailedHandler is a separate path from completedHandler, and a build that fails after eight minutes consumed eight minutes. If only successes are charged, the cheapest way to obtain free compute is to fail — so both handlers meter, and a cancelled build is charged for the time it ran.

The carve-out is whose fault it was. A failure caused by tenant content (a broken Dockerfile, a failing smoke test, a refused scan) is the tenant’s compute and is charged. A retry forced by infrastructure — a worker eviction, a transient registry error — is the platform’s, and charging an org three times for one submission because the platform retried it twice is indefensible.

The field for that line already exists: FailureCategory is 'retryable' | 'permanent' (api/plugin/src/helpers/plugin-helpers.ts), carried on the job and set by the failure handler. retryable is the platform’s — do not charge the re-attempt; permanent is the tenant’s. The multiplier is real rather than hypothetical: PLUGIN_BUILD_MAX_ATTEMPTS defaults to 2 on the main queue, so one submission can burn two attempts with no further human action. Phase 2 should name the field, not restate the principle.

The charged-on-failure rule wants saying out loud in the product copy, because it is the part of this scheme a customer is most likely to find surprising.

D5 — A per-build ceiling, and no sub-allocation yet

Decision: cap what one build may cost; do not carve the budget per team.

Build minutes pool at the account root like every flow quota, so one team’s runaway build loop consumes the whole account’s compute — and before D7 cut the period to a day, that meant up to three days in which no team in the account could build. apiCalls tolerates this because it is cheap and large (25k–1.5M); one spike is absorbed. Build minutes are few, expensive, and slow to refill, so the same pooling behaves much worse.

Two mitigations, and only one is worth building now:

The residual risk after the ceiling is many legitimate builds from one team crowding out another. That is the pooled model working as designed, and it is visible — the at-risk views and the 80% notice name the account, not the team.

D6 — The admission check fails CLOSED

Decision: refuse the build when the budget cannot be read.

Quota checks “time out fast (fail open)” after QUOTA_CHECK_TIMEOUT_MS (300 ms), counted on quota_fail_open_total, which alerts. That is right for apiCalls: breaking the console because the quota service blinked is worse than serving a few unmetered requests.

storageBytes is deliberately the exception — a longer read budget behind a fail-closed push gate, on the reasoning that an unverifiable cap must not let an over-quota org consume. Build minutes sit on that side of the line: expensive, deferrable by seconds without harm, and the thing being protected is real compute. A build refused because the budget could not be read is a retry; a build admitted on an unreadable budget is capacity given away.

Give the check the longer budget storage gets rather than the 300 ms hot-path one — an admission check runs once per submission, not once per request, so it can afford to wait for an answer instead of guessing.

D7 — A period of its own, and a MONTH

Decision: give buildMinutes its own period and make it a calendar month, aligned to the subscription. Leave QUOTA_RESET_DAYS alone for every other quota.

This settles what an earlier draft left open (“accept the three-day window, or argue for a per-type period”). Three findings decided that it needs its OWN period; D10 decides the LENGTH, and a one-day window — which two earlier drafts of this decision chose — is the wrong answer for a reason neither of them weighed. The reasoning is kept below rather than deleted, because the two structural findings are what make a per-type period possible at all and they are unchanged.

The invariant that was supposed to make the window harmless does not hold. Phase 1 used to claim plugins “stays the primary limit on every tier by construction: a tier runs out of builds before it runs out of minutes”. That is false on the failure path, and the failure path is the one that matters. releasePluginQuota (api/plugin/src/queue/build-quota.ts) refunds the plugins slot on every terminal failure, while D4 charges buildMinutes for a tenant-caused one. So a broken Dockerfile costs minutes and no plugins. On Developer’s old 300, a 15-minute failing build at 1.5 vCPU is ~22 credits, so roughly thirteen failed iterations exhausted the budget with the plugins counter still at 0/10 — and then nobody in the account built for up to three days.

The two quotas cannot be reconciled, either. Charging plugins for failures would give a Developer ten attempts per period, which is worse for the exact workflow Phase 1 already flags as that tier’s weak point (someone iterating on a Dockerfile, no warm cache). Refunding buildMinutes is the free-compute hole D4 exists to close. So they diverge permanently, the claim is struck, and window length becomes the only lever.

“Aligned to the subscription” is not implementable as worded, and that is a real constraint rather than a quibble. nextQuotaResetDate does DAY arithmetic (date.setDate(date.getDate() + days)), and the Mongo rollover adds unit: 'day'. Worse, the reset is LAZY — it fires on the first increment after expiry — so the boundary follows the org’s own activity rather than a calendar. At three days nobody notices; over a month two things become visible and arguable:

Two honest options. Either accept a 30-day rolling window and say so in the docs and the UI copy — in which case drop “aligned to the subscription”, because it will not be — or switch the reset to month arithmetic ($dateAdd does support unit: 'month') and make periodDays carry a unit rather than a day count. The second is the better product and the larger change: it is a contract change on the field the dashboard reads, not just a different number in it. Decide before Phase 1 ships, because the field shape is what Phase 1 moves into QuotaSummary.

A per-type period costs much less than assumed. The earlier draft called it “a change to the quota service, not a config tweak”. It is nearer the latter: resetDays is already a parameter of both rollover stages (resetIfExpiredStage(usagePath, amount, resetDays) and its pooled twin in api/quota/src/services/quota-service.ts), and resetAt is already stored per counter, not per org (api/quota/src/models/organization.ts). Rollover is lazy and evaluated per type inside the Mongo pipeline, so there is no sweep to change and no schema migration. The real work is one response-contract change: periodDays is a single top-level field on OrgQuotaResponse that the dashboard’s “per N days” copy and its usage forecast read, and it has to move into each QuotaSummary.

Why a MONTH and not a day, which is where the earlier drafts went wrong. Shortening the window was argued as “same throughput, a third of the lockout”: a day gives the same minutes over three days and caps the worst-case wait at one day. That is true and it is not the whole trade. A window does two jobs — it bounds the burn AND it decides how much of your allowance you may spend at once — and the earlier reasoning only priced the first.

Build work is bursty by nature, and this plan says so elsewhere without drawing the conclusion: changing a base image rebuilds everything that sits on it, while publishing another version costs little. So the expensive day is a base-image bump, a CVE sweep or a release — a day on which an organization wants its whole catalogue rebuilt at once.

Price that against a daily window at Pro: a full catalogue rebuild is 25 builds x 24 wall-minutes x 1.5 = 900 credits in one day against a 300/day cap. The account is refused on the first morning of its release while holding 8,100 unspent credits for the month. A daily window does not ration that work, it forbids it — and the remedy it leaves (buy a pack) charges a customer for capacity it has already paid for and cannot reach.

A monthly window is the same total capacity with the accumulation restored, and D10 is the requirement that settles it: the budget has to cover a month of normal activity, which it cannot do if a normal month’s shape is refused.

What a month costs, and the mitigations that make it affordable. The honest objection is the one D7 was built on: a long window means a long lockout. Four things bound it, and the last is new:

Two riders this creates, both small and both real:

Also extend the existing grace band. enforcedQuotaLimit (packages/api-core/src/types/quota-tiers.ts) already implements a soft overrun above the nominal cap, and it was built for this complaint in these words: “a self-mutating pipeline that hard-stopped at 100% stayed stopped until the period reset, up to 3 days.” Today it is a hardcoded quotaType !== 'automationCalls'. Generalise it to a per-type grace lookup fed by QUOTA_AUTOMATION_GRACE_PCT and a new QUOTA_BUILD_MINUTES_GRACE_PCT, rather than adding a second special case to the predicate. The band does not solve a lockout — it is 10% — but it removes the cliff at exactly 100%, which is the half of the complaint it was designed for.

D8 — One build budget, not a human/automation split

Decision: do not mirror automationCalls. Add a trigger label to the build metric, and revisit only if the data says to.

The symmetry argument is tempting and was worth writing down: automationCalls exists “so a fleet of pipelines or CI scripts can’t lock people out of the console”, and CI-triggered builds could equally lock out a human iterating on a plugin. But the analogy breaks on what each split protects.

automationCalls separates two different surfaces. Automation spends on writes; what is being defended is a human’s ability to read the console — a different activity, which the split genuinely protects. Build minutes have only one surface: building. A separate automation budget does not give the human a build slot (both contend for the same buildkitd regardless), it only gives them a smaller private budget. That is a sub-allocation, and D5 already deferred per-team sub-allocation for precisely this reason: it is an administration surface nobody has asked for.

It is also not the cheap mirror it looks like. The apiCalls / automationCalls choice is made in middleware, where the principal is in hand. A build’s charge lands in a queue handler, and PluginBuildJobData (api/plugin/src/helpers/plugin-helpers.ts) carries orgId, userId and an access snapshot but no principal type. The split needs a new payload field threaded through the upload route, the DLQ replay and the failed-build retry — paths already carrying delicate snapshot semantics (reservedResetAt, quotaReleased).

And the deciding evidence is one label away. The earlier draft said the org_id-labelled build metrics “could answer today” how much volume is automated. They cannot, quite: plugin_builds_total carries {status, org_id} only (api/plugin/src/queue/plugin-build-queue.ts, api/plugin/src/queue/failure-handler.ts). Adding a bounded trigger: 'user' | 'automation' label is one line in each of those two call sites and makes the question answerable before anyone builds a second budget, two more tier numbers, a second pack and two more frontend rows. Do that in Phase 2 and leave the split unbuilt.

If the data ever justifies it, note that the cheaper remedy comes first: a per-build ceiling (D5) already bounds one runaway automated build, and the grace band (D7) already removes the hard stop that makes a shared budget feel punitive.

D9 — Tiers are bounded by SUPPLY, not only by demand

Decision: no tier’s daily budget may exceed what the deployment’s buildkitd can actually deliver, and the ladder below must be checked against that before it is sold.

This is the one decision that changes the plan’s conclusions rather than its details, and it was missed because the Phase 1 table is derived from the plugins cap — a DEMAND model — with nothing on the other side of the equation.

The arithmetic, and there are TWO numbers — a floor and a ceiling. A credit is a vCPU-minute (D2). deploy/*/k8s/plugin.yaml gives the buildkitd sidecar limits.cpu: 1500m but requests.cpu: 300m, one sidecar per plugin pod, with replicas: 1. Kubernetes schedules on REQUESTS, so:

Ceiling (the limit, available only when the node is otherwise idle): 1.5 vCPU x 1440 min x 30 days = 64,800 vCPU-minutes per month

Floor (the request, which is what is actually reserved): 0.3 vCPU x 1440 min x 30 days = 12,960 vCPU-minutes per month

The ceiling is the number a first draft of this decision used, and it is the optimistic one: a build can burst to 1.5 vCPU only if nothing else on the node wants the cores. The guaranteed figure is the floor, and against it even Team (24,000) is already 185% — not just the top two rows. Quote both, because selling against the ceiling is selling capacity the scheduler never reserved.

The first fix is therefore to raise requests, not to argue about the limit. A quota that promises 1.5 vCPU of throughput against a 0.3 vCPU reservation is promising what the scheduler has not set aside; making requests match the capacity being sold is what turns the ceiling into the floor.

against the demand-derived ladder (the ratios are period-independent, so this reads the same whatever window D7 picks):

Tier Cap/month Share of the ENTIRE pool
Developer 3,000 5%
Pro 9,000 14%
Team 24,000 37%
Business 60,000 93%
Enterprise 135,000 208%

One Enterprise tenant’s cap is twice the platform’s total capacity — undeliverable with no other tenant on the system at all. One Business tenant leaves 4,800 vCPU-minutes a month for everybody else. Two Team tenants take 74%.

This is not ordinary oversubscription. Selling more than you have is normal and healthy where nobody uses their whole cap — that is exactly the reasoning behind the storage plan’s StorageCommitmentExceedsCapacity, which fires only past ~10x. Here the top tier’s cap cannot be met in isolation, and the failure mode is different in kind: a full disk stops everyone, while a saturated build pool QUEUES everyone. The quota as designed caps a VOLUME and says nothing about concurrency, so two tenants with budget to spare still wait behind one buildkitd.

Scaling the ladder down does not work, and that is the useful finding. Hold Enterprise to half the pool (32,400/month) and the whole table scales by 0.24: Developer lands at 720 credits/month, which at 20 wall-minutes and 1.5 vCPU is 24 builds a month against a plugins cap of 100 — so buildMinutes would become the binding limit on the cheapest tier, which is exactly the inversion Phase 1 rejected when it rejected the first draft’s numbers. The ladder and the shipped capacity are not reconcilable by re-picking numbers.

So SUPPLY is the variable that has to move, and the plan must say by how much:

And here is the part that reframes this whole decision: the fix is cheap. Priced at us-east-1 on-demand (checked October 2026), a vCPU-hour is ~$0.036 on Graviton (c7g.large, $0.0725/hr for 2 vCPU) and ~$0.045 on x86 (c7i.large, $0.0892/hr); EKS Auto Mode adds ~12% on top, and charges it at the on-demand rate even under a Savings Plan. So:

Reserved capacity Serves Cost/month (c7i, +Auto Mode)
1.5 vCPU (today’s limit) Business, as sole tenant ~$55
3.125 vCPU Enterprise, as sole tenant ~$114
6.25 vCPU Enterprise at half the pool ~$228

Removing this constraint entirely costs on the order of $150–180 a month more than the platform spends today, against a single Enterprise subscription at $899. So D9 is a PROVISIONING decision with a known and small price, not a product limit — and it should be read that way rather than as a reason to shrink the ladder. The revenue side makes the same point: the ceiling’s 64,800 credits/month sold as packs at $4.99 per 500 is ~$647/month against ~$55 of cost.

What stays true is the discipline, not the pessimism: the ladder must not be published without the check below, because the failure mode of getting it wrong is selling a cap nobody can serve.

What to build, rather than what to assert:

  1. Read the vCPU figure from configuration, which D2 already requires, and derive periodSupply = buildkit_vcpu x 1440 x periodDays from it.
  2. A boot-time check and a test: max(tier buildMinutes) <= periodSupply x SINGLE_TENANT_SHARE. A deployment whose ladder exceeds its own capacity should refuse to sell the tiers it cannot serve, or at minimum say so loudly at startup — the same shape as the tier-config echo already in the quota service’s boot path.
  3. The sold-vs-supply gauge and alert, copied from the storage work rather than invented: quota_buildminutes_committed over periodSupply, warning only, not routed to any actuator. Compute cannot be auto-expanded the way a PVC can, so there is nothing for an actuator to do.
  4. Document the vCPU each tier REQUIRES, so an operator who enables Enterprise knows it is a node-sizing decision and not a price-list edit.

One wrinkle for whoever sizes it. buildkitd is a sidecar of the plugin pod, so supply rises either by raising that container’s limit (a bigger node — and rootless buildkit already constrains the node choice: see the Fargate note in the deploy docs) or by scaling plugin replicas, which multiplies buildkitd instances. The second breaks D2’s “shared sidecar” premise: per-pod capacity becomes the unit, and a tenant’s builds may land on either, so the ceiling is per-pod while the quota is per-account. Prefer the bigger limit until that is designed.

D10 — The budget must cover a MONTH of normal activity, with room for a burst

Decision: size every tier so a standard month of work fits with roughly 3x headroom, and state the activity model rather than the number.

The ladder was derived from the plugins cap — one internal number from another — which says nothing about whether a customer can get a month’s work done. This is the requirement that makes the table answerable, and it follows AUTOMATION_CALLS_SIZING’s example of writing the arithmetic down “so the next person argues with the arithmetic, not with the numbers”.

What a normal month looks like. A tier’s plugins cap is a per-3-day build RATE, so it already asserts a monthly build count (x10). Normal activity is not the cap — the cap exists for peaks — so take normal as 30% utilisation:

Tier Builds/month at the cap Normal (30%) x wall-min x 1.5 vCPU Normal credits/month
Developer 100 30 20 900
Pro 250 75 24 2,700
Team 500 150 32 7,200
Business 1,000 300 40 18,000
Enterprise 1,500 450 60 40,500

Against the Phase 1 ladder those land at a uniform 3.3x headroom — the same factor on every tier, because both sides are linear in the same two assumptions. So the answer to “is it enough for a month of standard activity?” is yes, in aggregate, with room for a month that runs three times hot.

The aggregate was never the problem; the SHAPE was. The same figures under a one-day window refuse a Pro account’s release day while it holds 8,100 unspent credits — see D7, which this decision reverses to a monthly period for exactly this reason. A budget that covers a month of normal activity has to be spendable the way a month of normal activity actually arrives: unevenly.

Two consequences worth naming:

Phase 1 — The quota type

Tier sizing — DERIVED, so the argument is about the principle. An earlier draft picked a number per tier and asserted it was “generous relative to the plugins count quota”. Checked against the real limits (10 / 25 / 50 / 100 / 150) that was true on four tiers and false on the one that matters: it gave Developer 8 wall-minutes per permitted build against 24–67 everywhere else, so the new quota became the binding constraint on the only tier that cannot pay — and on the tier where builds are slowest, with no warm cache and someone iterating on a Dockerfile.

So the ladder is derived rather than chosen. The budget is per month (D7) while plugins is per three days, so the derivation goes through builds per month:

monthly credits = (plugins_limit x 10) x per-build wall-minutes x buildkit_vcpu

Tier plugins / 3 days Builds / month Wall-min per build buildMinutes / month
Developer 10 100 20 3,000
Pro 25 250 24 9,000
Team 50 500 32 24,000
Business 100 1,000 40 60,000
Enterprise 150 1,500 60 135,000
Unlimited — — — -1

Every row is exact at buildkit_vcpu = 1.5. D10 checks these against a month of normal activity and finds a uniform 3.3x headroom, which is the point of the sizing: a standard month fits, and a month that runs three times hot still fits.

But the table is a DEMAND model and is not sellable as it stands. Nothing in this derivation looks at supply, and D9 shows the top two rows exceed what the shipped buildkitd can deliver — Business takes 93% of the whole pool and Enterprise 208% of it. The ratios are period-independent, so moving to a monthly window changes nothing about that: 1.5 vCPU delivers 64,800 credits a month, and Enterprise’s 135,000 is still twice it. Treat the numbers here as the shape of the ladder and D9’s check as the gate on publishing it.

The only judgement left is the wall-minutes column — how long a build on each tier is allowed to take on average — which is a question somebody can actually hold an opinion about. It rises with tier because larger customers build larger images, not because larger customers deserve more slack.

What this table does NOT give you is a guarantee that plugins binds first. An earlier draft claimed it did, “by construction”. D7 records why that is false: a terminal build failure refunds the plugins slot and still charges the minutes, so the two quotas diverge on exactly the path a Developer hits most (iterating on a Dockerfile). Size the table for normal use, and read the “how we would know it worked” check as a detector of mis-sizing rather than as a proof it cannot happen.

Developer deserves a sanity pass as a row, not as a cell. It already carries 10 builds per period and 2 GB of registry storage — about four images at typical size. So a Developer can build ten times and store four. Adding a third, differently-shaped cap to the same workflow is worth reviewing against the other two rather than in isolation.

Phase 2 — Metering and admission

Phase 3 — The pack

b('build_minutes_pack', 'Build Minutes Pack (+500 min)',
  '500 additional build minutes / month', { buildMinutes: 500 },
  499, PAID, <sort>,
  { volumeTiers: SEAT_STYLE_VOLUME, maxQuantity: <from D9 supply> })

+500 minutes a month, granular, with seat-style volume breaks and a quantity cap. Each of those four is a decision the earlier drafts got wrong or left open; the reasoning is below. A pack grant raises the cap per period, so the monthly period (D7) sets the unit: +500/month is about fourteen extra builds at Pro’s assumed 24 wall-minutes.

Three things to get right, all learned from the existing catalogue:

Why granularity was the decision behind all four. Every add-on is quantity x a fixed grant, so the grant size is simply the step the buyer types: seat grants 1 and typing 8 means 8 seats; storage_pack grants 10 GB and is typed in GB. Choosing a coarse grant is what put the earlier draft on the ratio bound, made CAPACITY_VOLUME look right, and left the typed unit unstated. One choice fixed three symptoms, which is usually the sign it was the real decision.

Phase 3b — Every existing capacity pack now under-grants

Adding a dimension that is DERIVED from another one silently breaks the packs that grant the other one. The formula is credits/month = (plugins x 10) x wall-minutes x vcpu, so:

Pack Grants Build minutes it needs Grants today
plugin_pack +10 plugin builds / period ~3,600/month 0
pipeline_pack +5 pipelines, +25 plugins (Team) ~12,000/month 0

A buyer of either gets builds they have no minutes to run. This repository has already made this exact mistake twice and fixed it twice, and both comments are still in the catalogue:

So extend both grants, and add the invariant test beside the one that already holds “every tier keeps 5 plugin builds per pipeline, packs included” — a minutes-per-plugin-build ratio is the same guard on the same relationship.

One conflict to resolve first: RATE_PACKS requires a pack to grant exactly ONE dimension (so BILLING_BUNDLE_<ID>_GRANT keeps working), which is why pipeline_pack is classed as a differentiator and exempt. A plugin_pack that also granted minutes would have to leave RATE_PACKS the same way — losing the ratio bound and the free-tier guard with it. Either accept that trade, or decide deliberately that plugin builds and their minutes are sold separately and say so where a buyer can read it.

Phase 3c — Capped, not metered. A decision, not an omission

Build minutes are the first quota with genuine per-unit marginal cost. Every other dimension is a rate limit protecting shared infrastructure; this one is compute the platform actually pays for. That invites a pricing model the plan never weighs: meter it and bill overage, rather than cap it and sell packs.

Capped is still right, for a reason worth writing down rather than assuming: there is no overage-billing machinery in this codebase at all (the storage plan recorded the same thing — “there is no overage billing”). A cap plus packs reuses the whole bundle apparatus; metering is a new subsystem with its own invoicing, disputes and proration. Revisit only if build minutes become a material share of revenue.

Phase 3d — The over-cap guard must take buildMinutes

checkEntitlementOvercap refuses a downgrade or pack removal that would strand an account over a cap it cannot shrink into. It guards seats, plugins, pipelines, listings and storageBytes, and excludes the per-period rate quotas with a stated reason: they reset, so landing over a reduced cap costs at most one period.

The monthly window (D7) invalidates that reasoning for this dimension. “At most one period” is three days for apiCalls and a month for buildMinutes — so a mid-month downgrade, or cancelling a Build Minutes Pack, can refuse every build for the rest of the month. That is a month-long outage caused by a billing click, which is precisely the failure the guard exists to prevent.

So add buildMinutes to the guarded list. Note that this is the same correction storage needed, for the same reason and with the same shape: storage was grouped with the rate quotas on the “they reset” rule, the rule did not apply to it, and the fix was one entry in that list. The lesson generalises — the exclusion is about period LENGTH, not about being a rate — and the guard’s comment should say so, or the next long-window quota repeats it.

One consequence to design, not just assert: unlike plugins, build minutes are spent continuously rather than held, so “current usage” against a reduced cap is a point-in-time reading that will keep rising. Refusing the downgrade outright is the conservative answer; the alternative — allow it and let the grace band plus the forecast carry the account to the period boundary — is defensible but needs the notice to say what is about to happen.

Phase 4 — Ingest CodeBuild, and rename

The honest completion of D1. AWS reports duration and compute type per build, and pipeline-manager already talks to AWS, so the data is reachable. Once pipeline execution compute is metered:

  1. add org_id to pipeline_executions_total — one label, and executions become attributable by count even before any of this;
  2. charge CodeBuild duration × its compute class into the same quota;
  3. then the name cpuCredits becomes accurate, and the rename is a migration with a reason rather than a promise made early.

Until then the quota is honestly scoped and honestly named.

Both open questions are now decided

The two questions this plan used to leave open — the three-day period (Q3) and a separate budget for automated builds (Q5) — are settled in D7 and D8. Each was resolved by checking the code rather than by picking a side: Q3 turned on finding that the invariant meant to make it harmless is false and that a per-type period is far cheaper than assumed; Q5 turned on finding that the surface being protected is not analogous and that the payload carries no principal to split on.

The remaining judgement is the wall-minutes column of the Phase 1 table, which is flagged there and is a product opinion rather than a question about the code.

Checked and NOT a gap: the seat pack

seat carries a per-seat capacity invariant — every paid tier holds exactly 100,000 API calls and 200 AI calls per seat, and the pack grants that share so a bought seat is worth a tier seat. The obvious question is whether it must now grant build minutes too.

It must not, because there is no invariant to preserve: per-seat minutes are not constant across the ladder — Pro 9,000, Team 8,000, Business 7,500, Enterprise 9,000. Recorded so nobody re-raises it.

Worth noticing while here, though: Pro and Enterprise already sit at exactly 9,000 per seat. Re-deriving Team to 27,000 and Business to 72,000 would make the whole column constant, let seat carry build minutes like it carries the other two, and bring the dimension under the existing invariant test for free. That is an improvement rather than a defect — and it collides head-on with D9, since 72,000/month is above what the shipped pool can deliver at all. Decide the supply question first; this one is downstream of it.

Out of scope, and bounded: quarantine builds

Anonymous submissions build on a separate quarantine buildkitd (1000m), and their jobs carry “nothing but a submission id” — no orgId — so that compute falls on the platform rather than a tenant. It is deliberately not charged, and it is not a hole: the isolated pool is small by design and runs one build at a time, which bounds it without a quota.

Recorded for the same reason the storage plan records that quarantine storage is bounded by an unconditional 30-day sweep — so the next reader does not have to re-derive the answer.

Docs and frontend

Enforced contracts, not polish — the same list the storage plan carries:

Item Contract
New env vars docs/environment-variables.md, or env-documented.test.ts fails
Any doc edit the help corpus is SHA-pinned to its source doc; regenerate or help-corpus.test.ts fails
Quota type the Record<QuotaType, …> tables are compiler-forced; the frontend label map is one of them

Frontend

An earlier draft called this work “free” twice and was wrong twice. Two of the five deliverables have since become nearly free, for a reason worth recording: the quota display catalog (packages/api-core/src/types/quota-catalog.ts) was built after that draft and collapsed the seven label/unit maps the frontend used to carry. F1 and F2 are now catalog entries rather than new mechanisms; F3–F5 are the real work.

F1 — a minutes render mode (now one entry, compiler-enforced). The draft said QuotaCard formats everything through fmtNum and that build minutes would be the second quota needing unit-aware display. That display is built: QuotaRender is 'count' | 'bytes', formatQuotaValue switches on it exhaustively, and QuotaCard, UsageCard, TeamUsageCard and both at-risk lists all route through it. So adding 'minutes' to the union makes tsc name every surface that must handle it — the case the exhaustive switch was written for. The work is the mode plus its formatter (“2h 30m”), not a new pathway.

F2 — a packUnit on the catalog entry. The draft described PACK_UNITS as a hardcoded map with one member (storageBytes → GB), and framed adding build minutes as a judgement its comment had already set up. That map is gone; the unit is packUnit on the catalog entry and AddonGrid reads it through quotaPackUnit. The judgement still stands and still falls the same way — a +500/month pack reading “500 minutes” says far more than “1 pack” — and with packUnit declared the field is TYPED in minutes, so the buyer enters the number they have in mind rather than a pack count. It is a catalog field now, not a map to extend.

F3 — show in-flight reservations, or the number appears to go backwards. D3 reserves an estimate at admission and settles on completion, so while a build runs, used includes a reservation that may be refunded if the build beats the estimate. The page will show usage climb and then drop. No quota does that today — AI reservations settle in seconds, builds in minutes — so there is no existing affordance for it, and without one it reads as a bug and generates support traffic about wrong numbers. Either show “N in flight” explicitly or render the reserved portion distinctly from the settled portion. This is a requirement on D3, not a frontend nicety: a quota that trues up on a human-visible timescale has to say so.

Post-purchase latency is bounded, and worth saying (Q6). A denied check reading is cached until the period resets, capped at QUOTA_CHECK_DENIED_CACHE_MS (60 s), explicitly “so an upgrade, a bundle or an operator reset is honoured within a minute”. So an org that buys a pack to unblock a build may wait up to a minute. That is deliberate and bounded, not a defect — but the purchase confirmation should say so, or it will be reported as one.

F4 — two refusals, two messages. D5 adds a per-build ceiling, so there are now two distinct rejections: budget exhausted (buy a pack, or wait for reset) and this single build is too expensive (fix the build — a pack will not help). One shared 429 message sends people to buy capacity that cannot solve their problem.

F5 — a cost hint before the build, modelled on AIQuotaHint. The closest precedent in the codebase, built for exactly this: “the one line every AI surface shows BEFORE a generation: how many AI calls are left and when they reset, said up front instead of discovered as a 429”. It already handles the not-charged case, disables the action on the same predicate the caller uses, and fails open when the quota cannot be read.

A build submission deserves it more than an AI call does: one build is ~15 credits where an AI call is one unit, and a base-image bump submits dozens at once. The monthly period (D7) changes what the hint should SAY rather than whether it is needed: not “15 of your 85 remaining today” — which a daily window would have made the honest phrasing — but “about 15 of your 2,900 remaining this month”, alongside the forecast that D7 leans on as its main safety mechanism. Highest-value frontend item here, and the one an earlier draft of this plan omitted entirely while calling the frontend work free.

Docs

A build-minutes section in docs/quotas.md: the unit and how it is derived, the per-tier table, that failed builds are charged (and which retries are not), the per-build ceiling, and what the pack buys.

How we would know it worked

Rough sizing

Phase Size Why
1 Small–Medium A union member plus compiler-named tables; D7 adds a per-type period (a lookup threaded to two call sites) and moves periodDays into QuotaSummary, which touches the dashboard. D10’s utilisation figure is a guess until Phase 2 measures it
Frontend Medium F1/F2 are now catalog entries (the display catalog landed); the work is in-flight reservations, two refusal messages, and the pre-build cost hint
2 Medium Two metering hooks, a reserve-and-true-up settle shared with the plugins gate, retry pricing, exemptions, ledger attribution
3 Small–Medium One catalogue entry, two test-list additions, a Marketplace dimension, a maxQuantity, and the grant-granularity decision (which also sets the volume breaks)
3b Small Extending plugin_pack / pipeline_pack grants, plus the ratio invariant test — but it needs the RATE_PACKS single-dimension conflict resolved first
3d Hours One entry in checkEntitlementOvercap, plus the notice copy for what a refused downgrade means
4 Largest A new AWS ingestion path, plus a rename migration

What this plan deliberately does not do

Sequencing

D9 FIRST, before anything is published. It is the only item that can invalidate the tier table rather than refine it: at the shipped 1.5 vCPU the top two rows are undeliverable, and no amount of Phase 1–3 work makes a cap the hardware cannot serve into a real product. Sizing the pool, or trimming the ladder to it, is the gate on the rest.

D10 decides what the numbers mean, and D7’s window decides whether they are reachable. Both are paper decisions — settle them with D9 rather than after it, because all three move the same table and re-deriving it three times is how a ladder ends up internally inconsistent.

Then 1 → 2 → 3 (with 3b), then 4 when pipeline compute matters enough.

One ordering note that only became visible once the window was monthly: Phase 2 is what turns D10’s 30% utilisation guess into a measurement. Until build_minutes_charged_total has run for a month, every number in the Phase 1 table rests on an assumption nobody has tested, so treat the first month after Phase 2 as the point at which the ladder is re-derived — not as confirmation that it was right.

Phase 1 alone is inert — a quota with no meter. Phase 2 makes it real. Phase 3 without Phase 2 would sell capacity against a limit nothing enforces, which is worse than not selling it.

Phase 4 is independent of 1–3 in mechanism but dependent in meaning: it is what turns an honestly-scoped build quota into a complete compute quota. Its first step (an org_id label on pipeline_executions_total) is worth landing early regardless, because it is one label and it makes the gap measurable.