The deploy button had been the owner’s since 13 October: the manager asked, the owner pressed. On 15 October, with a fresh build ready and the owner away, they ended that arrangement in one line. The manager ran the deploy, the database was backed up first as the runbook requires, and the new build was live in under a minute.
What the check found
Each deploy ends with a short list of checks against the live site. One failed. The analytics change from slice 19 had added two hosts to the site’s content security policy, and the live policy did not have them. Nine slices had been deployed since, every one of them reporting success.
The cause was in how the proxy is run. Its configuration file is mounted into the container read-only and its admin interface is off, so it reads the file once, at start. The deploy script brings the containers up with a command that recreates a container when its image or settings change, but a changed file behind a mount is neither. The proxy had been serving the configuration from before slice 19 the whole time.
The fix
The deploy script now keeps a note on the server of which configuration file the running proxy started with. After the app is healthy it compares. A container the deploy just created already has the new file; an unchanged file does nothing; otherwise the new configuration is validated in a throwaway container and, if it passes, the proxy is recreated. A rejected configuration is never applied: the old one keeps serving, the run prints that it is live and then fails, so the summary shows the error. The rollback script follows the same rule.
Because the server runs its old deploy script during the deploy that ships a new one, the fix needed two deploys to take effect. The second printed that the configuration had changed and recreated the proxy, and the analytics hosts were finally served.
What the owner decided
Two decisions sit on this day. The deploy is the manager’s to run from now on, recorded as a rule in the project’s memory. And the analytics itself, along with switching payments on, went onto the back burner at the owner’s word: the setup steps stop appearing as open items until they raise them.