Automate post-import refresh and validation workflow

2026-05-04 09:49:47 -04:00
parent fba494dada
commit faad70872b
13 changed files with 995 additions and 136 deletions
@@ -15,6 +15,92 @@ redMCP testing.
 - Mailpit ports: HTTP `8025`, SMTP `1025`, POP3 `1110`
 - POP3 credentials: `test` / `testpass`
 - SMTP authentication: none
+- Shared scratch path: `/opt/lanscratch`
+- Post-import payload path:
+  `/opt/lanscratch/redmine-post-import/repo`
+- Post-import status path:
+  `/opt/lanscratch/redmine-post-import/status`
+
+## Automated Daily Post-Import
+
+Stage the post-import payload from this host into the shared LAN scratch folder:
+
+```sh
+./stage_post_import_payload.py
+```
+
+The default staging mode is a dry run. Review the `rsync` command, then apply:
+
+```sh
+./stage_post_import_payload.py --apply
+```
+
+After the fresh production database and `/usr/share/redmine` tree have been
+copied onto the LAN test host, the test host should run the automation locally:
+
+```sh
+cd /opt/lanscratch/redmine-post-import/repo
+./post_import_refresh.py --local --apply
+```
+
+For manual review on the test host, omit `--apply` first:
+
+```sh
+cd /opt/lanscratch/redmine-post-import/repo
+./post_import_refresh.py --local
+```
+
+This host can check completion by reading:
+
+```text
+/opt/lanscratch/redmine-post-import/status/latest.json
+/opt/lanscratch/redmine-post-import/status/latest-success.json
+```
+
+The automation:
+
+- verifies the tracked plugin source directories exist locally and that the
+  remote Redmine path exists;
+- overlays remote dev-only files from `/home/reddev/redmine-dev-overrides` when
+  that directory exists;
+- reapplies `redmine_event_outbox`, `redmine_contacts`, and
+  `redmine_contacts_helpdesk` from this repository into
+  `/usr/share/redmine/plugins/`;
+- runs `RAILS_ENV=production bundle exec rake redmine:plugins:migrate`;
+- fixes group-write permissions on attachment, `tmp`, and `log` paths;
+- runs `reset_helpdesk_mail_settings.py` unless `--skip-helpdesk-reset` is
+  passed;
+- restarts Passenger with `touch tmp/restart.txt`;
+- runs `validate_test_instance.py`;
+- checks outbox status and dry-runs a small outbox batch;
+- runs a semantic-index dry-run smoke check only.
+
+Each applied run writes status JSON under
+`/opt/lanscratch/redmine-post-import/status/runs/`, updates `latest.json` after
+each step, and updates `latest-success.json` only after every step exits
+successfully. The JSON includes the run id, host, execution mode, Redmine path,
+repo root, failed step when applicable, and per-command return codes.
+
+Remote write and permission steps use `sudo` by default because a fresh
+production file copy may leave `/usr/share/redmine` or attachment paths owned by
+another user. This applies in both local and SSH modes. If the dev host already
+gives the runner write access to those paths, pass `--no-remote-sudo`.
+
+The older SSH orchestration path from this host remains available:
+
+```sh
+./post_import_refresh.py
+./post_import_refresh.py --apply
+```
+
+The automation deliberately does **not** run a semantic-index apply refresh,
+does **not** use `--force-rebuild`, and does **not** enable the semantic-index
+refresh timer. After a fresh database clone, treat semantic-index writes or a
+Qdrant rebuild as a separate manual maintenance action with a snapshot or
+isolated dev collection first.
+
+Use the manual sections below for troubleshooting individual steps or for
+running the sequence by hand.

 ## 1. Validate The Fresh Import