redmine/docs/test_instance_post_import.md

# Test Instance Post-Import Runbook

Use this after loading a production database backup into the LAN Redmine test
instance. The goal is to make the test copy safe for Helpdesk, Mailpit, and
redMCP testing.

## Defaults

- Redmine URL: `http://192.168.50.170`
- SSH host: `reddev@192.168.50.170`
- SSH key: `/tmp/reddev`
- Remote Redmine path: `/usr/share/redmine`
- Attachment files root: `/var/lib/redmine/default/files`
- Mailpit host: `192.168.1.105`
- Mailpit ports: HTTP `8025`, SMTP `1025`, POP3 `1110`
- POP3 credentials: `test` / `testpass`
- SMTP authentication: none
- Shared scratch path: `/opt/lanscratch`
- Post-import payload path:
  `/opt/lanscratch/redmine-post-import/repo`
- Post-import status path:
  `/opt/lanscratch/redmine-post-import/status`

## Automated Daily Post-Import

Stage the post-import payload from this host into the shared LAN scratch folder:

```sh
./stage_post_import_payload.py
```

The default staging mode is a dry run. Review the `rsync` command, then apply:

```sh
./stage_post_import_payload.py --apply
```

After the fresh production database and `/usr/share/redmine` tree have been
copied onto the LAN test host, the test host should run the automation locally:

```sh
cd /opt/lanscratch/redmine-post-import/repo
./post_import_refresh.py --local --apply
```

For manual review on the test host, omit `--apply` first:

```sh
cd /opt/lanscratch/redmine-post-import/repo
./post_import_refresh.py --local
```

This host can check completion by reading:

```text
/opt/lanscratch/redmine-post-import/status/latest.json
/opt/lanscratch/redmine-post-import/status/latest-success.json
```

The automation:

- verifies the tracked plugin source directories exist locally and that the
  remote Redmine path exists;
- overlays remote dev-only files from `/home/reddev/redmine-dev-overrides` when
  that directory exists;
- reapplies `redmine_event_outbox`, `redmine_contacts`, and
  `redmine_contacts_helpdesk` from this repository into
  `/usr/share/redmine/plugins/`;
- runs `RAILS_ENV=production bundle exec rake redmine:plugins:migrate`;
- fixes group-write permissions on attachment, `tmp`, and `log` paths;
- runs `reset_helpdesk_mail_settings.py` unless `--skip-helpdesk-reset` is
  passed;
- restarts Passenger with `touch tmp/restart.txt`;
- runs `validate_test_instance.py`;
- checks outbox status and dry-runs a small outbox batch;
- runs a semantic-index dry-run smoke check only.

Each applied run writes status JSON under
`/opt/lanscratch/redmine-post-import/status/runs/`, updates `latest.json` after
each step, and updates `latest-success.json` only after every step exits
successfully. The JSON includes the run id, host, execution mode, Redmine path,
repo root, failed step when applicable, and per-command return codes.

Remote write and permission steps use `sudo` by default because a fresh
production file copy may leave `/usr/share/redmine` or attachment paths owned by
another user. This applies in both local and SSH modes. If the dev host already
gives the runner write access to those paths, pass `--no-remote-sudo`.

The older SSH orchestration path from this host remains available:

```sh
./post_import_refresh.py
./post_import_refresh.py --apply
```

The automation deliberately does **not** run a semantic-index apply refresh,
does **not** use `--force-rebuild`, and does **not** enable the semantic-index
refresh timer. After a fresh database clone, treat semantic-index writes or a
Qdrant rebuild as a separate manual maintenance action with a snapshot or
isolated dev collection first.

Use the manual sections below for troubleshooting individual steps or for
running the sequence by hand.

## 1. Validate The Fresh Import

Run the read-only validator first:

```sh
./validate_test_instance.py
```

This checks SSH access, Redmine paths, Mailpit connectivity, attachment
directory permissions, controlled test projects, Helpdesk mail settings, and
redMCP Composer metadata when Composer is available.

If Composer is not installed globally, pass a known PHAR:

```sh
./validate_test_instance.py --composer-bin /home/iadnah/projects/redMCP/composer.phar
```

## 2. Fix Attachment Directory Permissions

If the validator reports attachment directory failures, run this on the Redmine
test host:

```sh
sudo chmod -R g+rwX /var/lib/redmine/default/files
sudo find /var/lib/redmine/default/files -type d -exec chmod g+s {} +
```

Verify:

```sh
stat -c "%U %G %a %n" \
  /var/lib/redmine/default/files \
  /var/lib/redmine/default/files/2026 \
  /var/lib/redmine/default/files/2026/04
```

Directories should normally show group-write permissions, such as `2775`.

## 3. Reset Helpdesk Mail Settings

Preview first:

```sh
./reset_helpdesk_mail_settings.py --dry-run
```

Apply:

```sh
./reset_helpdesk_mail_settings.py
```

This rewrites all active projects to use Mailpit for Helpdesk POP3 and SMTP,
even if the Helpdesk module is currently disabled for a project. That prevents
imported real mail credentials from being used accidentally in the test
instance. Passwords are written but not printed.

## 4. Restart Passenger

After plugin changes or other code updates, trigger Passenger reload on the
Redmine host:

```sh
ssh -i /tmp/reddev -o IdentitiesOnly=yes reddev@192.168.50.170 \
  'cd /usr/share/redmine && touch tmp/restart.txt'
```

Then hit the site in a browser or with `curl` so Passenger reloads the app.

## 5. Validate Helpdesk Mail

In Redmine, use project `fud-helpdesk` for Helpdesk mail tests.

Incoming test:

1. Send a message into Mailpit.
2. Open `http://192.168.50.170/projects/fud-helpdesk/settings`.
3. Click **Get Mail**.
4. Confirm a Helpdesk issue is created.

Outgoing test:

1. Open the imported Helpdesk issue.
2. Send a Helpdesk response from Redmine.
3. Confirm it appears in Mailpit at `http://192.168.1.105:8025`.

Useful logs:

```sh
ssh -i /tmp/reddev -o IdentitiesOnly=yes reddev@192.168.50.170 \
  'tail -n 100 /usr/share/redmine/log/redmine_helpdesk.log'
```

## 6. Validate redMCP Safe CRUD

Use `fud-nohelpdesk` for issue create/update/delete tests that should not touch
the Helpdesk plugin. Keep API keys in `redMCP/.env`; do not commit that file.

Minimum checks:

```sh
php -l redMCP/app/RedmineClient.php
php -l redMCP/app/redmineClient.php
```

Use the existing redMCP examples in `redMCP/README.md` for read and CRUD smoke
tests against the LAN Redmine copy.

## 7. Run The Helpdesk Smoke Test

After the post-import checks pass, run the live Helpdesk/redMCP smoke test:

```sh
./helpdesk_smoke_test.py
```

This imports a controlled Helpdesk email, verifies `issueWithHelpdesk()`, checks
non-Helpdesk CRUD, confirms default updates do not send Helpdesk email, verifies
explicit outbound Mailpit delivery, and closes the created test ticket. Details
are in `docs/helpdesk_smoke_test.md`.

## 8. Validate Helpdesk Outbox Worker Enrichment

When Helpdesk/outbox behavior matters, run the repeatable live validator:

```sh
./validate_helpdesk_outbox_worker.py
```

This creates one controlled Helpdesk ticket through Mailpit, verifies redMCP
Helpdesk behavior, checks the matching `event_outbox_events` rows, and dry-runs
worker enrichment without claiming or marking rows processed. Details are in
`docs/helpdesk_outbox_worker_validation.md`.

## 9. Check Or Process The Outbox Worker

Inspect outbox queue state:

```sh
./redmine_outbox_worker.py --status
```

Preview worker output without marking rows processed:

```sh
./redmine_outbox_worker.py --dry-run --batch-size 10
```

Process a small bounded batch only after the dry-run output looks correct:

```sh
./redmine_outbox_worker.py --batch-size 5
```

The default JSONL output path is
`/tmp/redmine-outbox/derived_documents.jsonl`. Use `--output` to override it.
Processed rows are retained by default. To preview test-instance cleanup:

```sh
./redmine_outbox_worker.py --purge-processed-days 30
```

## 10. Re-Run The Read-Only Validator

Finish by running:

```sh
./validate_test_instance.py
```

The expected result is no `FAIL` lines. `WARN` is acceptable only for optional
local tooling, such as missing global Composer.