redmine/docs/test_instance_post_import.md

Test Instance Post-Import Runbook

Use this after loading a production database backup into the LAN Redmine test instance. The goal is to make the test copy safe for Helpdesk, Mailpit, and redMCP testing.

Defaults

• Redmine URL: http://192.168.50.170
• SSH host: reddev@192.168.50.170
• SSH key: /tmp/reddev
• Remote Redmine path: /usr/share/redmine
• Attachment files root: /var/lib/redmine/default/files
• Mailpit host: 192.168.1.105
• Mailpit ports: HTTP 8025, SMTP 1025, POP3 1110
• POP3 credentials: test / testpass
• SMTP authentication: none
• Shared scratch path: /opt/lanscratch
• Post-import payload path: /opt/lanscratch/redmine-post-import/repo
• Post-import status path: /opt/lanscratch/redmine-post-import/status

Automated Daily Post-Import

Stage the post-import payload from this host into the shared LAN scratch folder:

./stage_post_import_payload.py

The default staging mode is a dry run. Review the rsync command, then apply:

./stage_post_import_payload.py --apply

After the fresh production database and /usr/share/redmine tree have been copied onto the LAN test host, the test host should run the automation locally:

cd /opt/lanscratch/redmine-post-import/repo
./post_import_refresh.py --local --apply

For manual review on the test host, omit --apply first:

cd /opt/lanscratch/redmine-post-import/repo
./post_import_refresh.py --local

This host can check completion by reading:

/opt/lanscratch/redmine-post-import/status/latest.json
/opt/lanscratch/redmine-post-import/status/latest-success.json

The automation:

• verifies the tracked plugin source directories exist locally and that the remote Redmine path exists;
• overlays remote dev-only files from /home/reddev/redmine-dev-overrides when that directory exists;
• reapplies redmine_event_outbox, redmine_contacts, and redmine_contacts_helpdesk from this repository into /usr/share/redmine/plugins/;
• runs RAILS_ENV=production bundle exec rake redmine:plugins:migrate;
• fixes group-write permissions on attachment, tmp, and log paths;
• runs reset_helpdesk_mail_settings.py unless --skip-helpdesk-reset is passed;
• restarts Passenger with touch tmp/restart.txt;
• runs validate_test_instance.py;
• checks outbox status and dry-runs a small outbox batch;
• runs a semantic-index dry-run smoke check only.

Each applied run writes status JSON under /opt/lanscratch/redmine-post-import/status/runs/, updates latest.json after each step, and updates latest-success.json only after every step exits successfully. The JSON includes the run id, host, execution mode, Redmine path, repo root, failed step when applicable, and per-command return codes.

Remote write and permission steps use sudo by default because a fresh production file copy may leave /usr/share/redmine or attachment paths owned by another user. This applies in both local and SSH modes. If the dev host already gives the runner write access to those paths, pass --no-remote-sudo.

The older SSH orchestration path from this host remains available:

./post_import_refresh.py
./post_import_refresh.py --apply

The automation deliberately does not run a semantic-index apply refresh, does not use --force-rebuild, and does not enable the semantic-index refresh timer. After a fresh database clone, treat semantic-index writes or a Qdrant rebuild as a separate manual maintenance action with a snapshot or isolated dev collection first.

Use the manual sections below for troubleshooting individual steps or for running the sequence by hand.

1. Validate The Fresh Import

Run the read-only validator first:

./validate_test_instance.py

This checks SSH access, Redmine paths, Mailpit connectivity, attachment directory permissions, controlled test projects, Helpdesk mail settings, and redMCP Composer metadata when Composer is available.

If Composer is not installed globally, pass a known PHAR:

./validate_test_instance.py --composer-bin /home/iadnah/projects/redMCP/composer.phar

2. Fix Attachment Directory Permissions

If the validator reports attachment directory failures, run this on the Redmine test host:

sudo chmod -R g+rwX /var/lib/redmine/default/files
sudo find /var/lib/redmine/default/files -type d -exec chmod g+s {} +

Verify:

stat -c "%U %G %a %n" \
  /var/lib/redmine/default/files \
  /var/lib/redmine/default/files/2026 \
  /var/lib/redmine/default/files/2026/04

Directories should normally show group-write permissions, such as 2775.

3. Reset Helpdesk Mail Settings

Preview first:

./reset_helpdesk_mail_settings.py --dry-run

Apply:

./reset_helpdesk_mail_settings.py

This rewrites all active projects to use Mailpit for Helpdesk POP3 and SMTP, even if the Helpdesk module is currently disabled for a project. That prevents imported real mail credentials from being used accidentally in the test instance. Passwords are written but not printed.

4. Restart Passenger

After plugin changes or other code updates, trigger Passenger reload on the Redmine host:

ssh -i /tmp/reddev -o IdentitiesOnly=yes reddev@192.168.50.170 \
  'cd /usr/share/redmine && touch tmp/restart.txt'

Then hit the site in a browser or with curl so Passenger reloads the app.

5. Validate Helpdesk Mail

In Redmine, use project fud-helpdesk for Helpdesk mail tests.

Incoming test:

1. Send a message into Mailpit.
2. Open http://192.168.50.170/projects/fud-helpdesk/settings.
3. Click Get Mail.
4. Confirm a Helpdesk issue is created.

Outgoing test:

1. Open the imported Helpdesk issue.
2. Send a Helpdesk response from Redmine.
3. Confirm it appears in Mailpit at http://192.168.1.105:8025.

Useful logs:

ssh -i /tmp/reddev -o IdentitiesOnly=yes reddev@192.168.50.170 \
  'tail -n 100 /usr/share/redmine/log/redmine_helpdesk.log'

6. Validate redMCP Safe CRUD

Use fud-nohelpdesk for issue create/update/delete tests that should not touch the Helpdesk plugin. Keep API keys in redMCP/.env; do not commit that file.

Minimum checks:

php -l redMCP/app/RedmineClient.php
php -l redMCP/app/redmineClient.php

Use the existing redMCP examples in redMCP/README.md for read and CRUD smoke tests against the LAN Redmine copy.

7. Run The Helpdesk Smoke Test

After the post-import checks pass, run the live Helpdesk/redMCP smoke test:

./helpdesk_smoke_test.py

This imports a controlled Helpdesk email, verifies issueWithHelpdesk(), checks non-Helpdesk CRUD, confirms default updates do not send Helpdesk email, verifies explicit outbound Mailpit delivery, and closes the created test ticket. Details are in docs/helpdesk_smoke_test.md.

8. Validate Helpdesk Outbox Worker Enrichment

When Helpdesk/outbox behavior matters, run the repeatable live validator:

./validate_helpdesk_outbox_worker.py

This creates one controlled Helpdesk ticket through Mailpit, verifies redMCP Helpdesk behavior, checks the matching event_outbox_events rows, and dry-runs worker enrichment without claiming or marking rows processed. Details are in docs/helpdesk_outbox_worker_validation.md.

9. Check Or Process The Outbox Worker

Inspect outbox queue state:

./redmine_outbox_worker.py --status

Preview worker output without marking rows processed:

./redmine_outbox_worker.py --dry-run --batch-size 10

Process a small bounded batch only after the dry-run output looks correct:

./redmine_outbox_worker.py --batch-size 5

The default JSONL output path is /tmp/redmine-outbox/derived_documents.jsonl. Use --output to override it. Processed rows are retained by default. To preview test-instance cleanup:

./redmine_outbox_worker.py --purge-processed-days 30

10. Re-Run The Read-Only Validator

Finish by running:

./validate_test_instance.py

The expected result is no FAIL lines. WARN is acceptable only for optional local tooling, such as missing global Composer.