diff --git a/AGENTS.md b/AGENTS.md index 80ec7b8..3af9a4c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,9 +11,10 @@ environment. The canonical plugin sources live in `plugins/`: Top-level Python helpers such as `redmine_outbox_worker.py` and `reset_helpdesk_mail_settings.py` support LAN test-instance operations. -`validate_test_instance.py` performs read-only post-import checks. Project -notes and design records live in `docs/`. `redMCP/` contains the PHP -Redmine API/MCP wrapper. `dist/*.MANIFEST.md` files are tracked; rollback +`validate_test_instance.py` performs read-only post-import checks, and +`validate_helpdesk_outbox_worker.py` runs the controlled Helpdesk/outbox live +validator. Project notes and design records live in `docs/`. `redMCP/` +contains the PHP Redmine API/MCP wrapper. `dist/*.MANIFEST.md` files are tracked; rollback tarballs are intentionally ignored. `redmine-copy/` is an ignored working/reference copy, not the source of truth. @@ -25,7 +26,7 @@ Use targeted syntax checks before committing: ruby -c plugins/redmine_contacts/lib/redmine_contacts/utils/check_mail.rb ruby -c plugins/redmine_contacts_helpdesk/app/models/helpdesk_mailer.rb ruby -c plugins/redmine_event_outbox/lib/redmine_event_outbox.rb -python3 -m py_compile redmine_outbox_worker.py reset_helpdesk_mail_settings.py validate_test_instance.py helpdesk_smoke_test.py +python3 -m py_compile redmine_outbox_worker.py reset_helpdesk_mail_settings.py validate_test_instance.py helpdesk_smoke_test.py validate_helpdesk_outbox_worker.py cd redMCP && php -l app/RedmineClient.php && composer validate ``` @@ -35,6 +36,7 @@ Dry-run operational helpers before applying changes: ./reset_helpdesk_mail_settings.py --dry-run ./validate_test_instance.py ./helpdesk_smoke_test.py +./validate_helpdesk_outbox_worker.py ./redmine_outbox_worker.py --dry-run --batch-size 10 ``` @@ -53,7 +55,8 @@ against the LAN Redmine copy using controlled projects such as `fud-helpdesk` or `fud-nohelpdesk`. Record notable manual validation in `docs/` when behavior or deployment assumptions change. The post-import workflow lives in `docs/test_instance_post_import.md`; the Helpdesk/redMCP live smoke test is -documented in `docs/helpdesk_smoke_test.md`. +documented in `docs/helpdesk_smoke_test.md`; the Helpdesk outbox worker +validator is documented in `docs/helpdesk_outbox_worker_validation.md`. ## Commit & Pull Request Guidelines diff --git a/README.md b/README.md index 0bd31ef..cecf033 100644 --- a/README.md +++ b/README.md @@ -138,19 +138,20 @@ Tested event types on the LAN copy: - `journal.created` - `contact.created` - `contact.updated` - -Planned/implemented locally but not fully LAN-validated as a complete workflow: - - `helpdesk_ticket.created` - `helpdesk_ticket.updated` - `journal_message.created` + +Planned/implemented locally but not yet observed in the controlled LAN +validation workflow: + - `journal_message.updated` The Helpdesk user workflow itself is now live-smoke-tested, including inbound Mailpit import, `issueWithHelpdesk()` metadata, default non-email updates, and -explicit customer-visible Helpdesk replies. The remaining gap is validating that -those same controlled actions produce the expected outbox rows and derived -worker documents. +explicit customer-visible Helpdesk replies. The repeatable outbox validation +uses that same controlled workflow to verify Helpdesk ticket/message outbox rows +and worker-derived documents without marking rows processed. ### 3. Local Helpdesk Plugin Fork Changes @@ -259,6 +260,17 @@ That test is documented in: - [docs/helpdesk_smoke_test.md](/home/iadnah/redmine/docs/helpdesk_smoke_test.md:1) +Run the Helpdesk outbox worker validation when changing outbox hooks, worker +enrichment, or Helpdesk/redMCP behavior: + +```sh +./validate_helpdesk_outbox_worker.py +``` + +That test is documented in: + +- [docs/helpdesk_outbox_worker_validation.md](/home/iadnah/redmine/docs/helpdesk_outbox_worker_validation.md:1) + Preview the affected projects and settings: ```sh @@ -323,18 +335,14 @@ The deployed helpdesk search routes were verified with: ## What Is Not Finished Yet -### 1. Worker Validation +### 1. Worker Processing Policy -This is the next implementation milestone. +The Helpdesk outbox worker now has a repeatable live validator. The next worker +decision is operational policy: -Still needed: - -- run and document end-to-end validation of `redmine_outbox_worker.py` against - controlled Helpdesk activity on the LAN copy -- prove that imported Helpdesk issues, Helpdesk replies, and normal issue - updates create the expected event rows -- verify the derived JSONL document shape is useful and does not leak unsafe - content +- choose when to mark rows processed on the test instance +- decide where durable JSONL/index output should live +- define retention or replay expectations for `event_outbox_events` ### 2. External Search Index diff --git a/docs/helpdesk_outbox_worker_validation.md b/docs/helpdesk_outbox_worker_validation.md new file mode 100644 index 0000000..59950bb --- /dev/null +++ b/docs/helpdesk_outbox_worker_validation.md @@ -0,0 +1,60 @@ +# Helpdesk Outbox Worker Validation + +Use `validate_helpdesk_outbox_worker.py` to verify that controlled Helpdesk +activity creates the expected `event_outbox_events` rows and that +`redmine_outbox_worker.py` can enrich those rows without processing them. + +## Run + +```sh +./validate_helpdesk_outbox_worker.py +``` + +The script first runs the full Helpdesk/redMCP smoke path: + +- sends an inbound email to Mailpit for `fud-helpdesk` +- triggers Helpdesk mail import on the LAN Redmine host +- verifies `issueWithHelpdesk()` metadata +- verifies normal `updateIssue()` does not send customer mail +- verifies explicit Helpdesk reply delivery through Mailpit +- closes the controlled Helpdesk issue unless `--keep-open` is passed + +It then queries `event_outbox_events` for the created issue and dry-runs +`redmine_outbox_worker.enrich_event()` against those exact rows. + +## Expected Checks + +The validator fails if any of these are missing: + +- `helpdesk_ticket.created` +- `helpdesk_ticket.updated` +- `journal_message.created` +- `journal.created` +- `issue.updated` +- derived worker document types: `event`, `ticket`, and `message` + +It also confirms the new rows have not been locked or marked processed, and +that worker documents do not expose raw `bcc_address` data. + +## Configuration + +Defaults match the LAN test setup: + +- Redmine: `http://192.168.50.170` +- SSH: `reddev@192.168.50.170`, key `/tmp/reddev` +- remote Redmine path: `/usr/share/redmine` +- Mailpit: `192.168.1.105`, SMTP `1025`, HTTP `8025` +- API key source: `REDMINE_API_KEY`, `REDMNINE_API_KEY`, or `redMCP/.env` + +Useful options: + +```sh +./validate_helpdesk_outbox_worker.py --skip-preflight +./validate_helpdesk_outbox_worker.py --keep-open +``` + +## Expected Output + +The final summary should include the created issue id, Helpdesk ticket ids, +journal message ids, event row ids/types, and derived document type counts. +The worker mode should report dry-run enrichment only. diff --git a/docs/helpdesk_smoke_test.md b/docs/helpdesk_smoke_test.md index d3d19fa..1f3728a 100644 --- a/docs/helpdesk_smoke_test.md +++ b/docs/helpdesk_smoke_test.md @@ -48,3 +48,7 @@ Smoke test passed. Closed smoke-test tickets are intentionally retained in `fud-helpdesk` as audit evidence. Their subjects start with `[redMCP-smoke ...]`. + +For outbox and worker coverage, run `./validate_helpdesk_outbox_worker.py`. +That script reuses this same live smoke path, then checks the matching +`event_outbox_events` rows and dry-runs worker enrichment. diff --git a/docs/redmineup_local_fork_changelog.md b/docs/redmineup_local_fork_changelog.md index ac61e51..f81a41f 100644 --- a/docs/redmineup_local_fork_changelog.md +++ b/docs/redmineup_local_fork_changelog.md @@ -27,11 +27,38 @@ environment. Before risky edits, archive the current plugin directories in manual browser testing. - Helpdesk mail import, Helpdesk metadata lookup, default non-email updates, and explicit outbound Helpdesk replies are live-smoke-tested through redMCP. - - Helpdesk outbox/worker validation is still pending. + - Helpdesk outbox/worker validation now has a repeatable live script. - Next meaningful milestone: - - Validate `redmine_outbox_worker.py` end to end against controlled Helpdesk - activity, document the derived JSONL shape, then choose the external index - target. + - Run `validate_helpdesk_outbox_worker.py` after outbox or worker changes, + then choose the external index target and durable processing policy. + +## 2026-04-25 - Helpdesk Outbox Worker Validation + +- Touched areas: + - Helpdesk test tooling + - External outbox worker validation docs +- Purpose: + - Make Helpdesk outbox coverage repeatable after database refreshes and + worker changes. + - Validate worker enrichment against a real Mailpit-imported Helpdesk ticket + without claiming rows or marking them processed. +- Behavior checked: + - Controlled Helpdesk import produces `helpdesk_ticket.created`, + `helpdesk_ticket.updated`, `journal_message.created`, `journal.created`, + and `issue.updated` rows. + - Worker dry-run enrichment emits `event`, `ticket`, and `message` documents. + - Derived message documents expose `has_bcc_address` but do not expose raw + `bcc_address`. +- LAN test result: + - `./validate_helpdesk_outbox_worker.py` passed against `fud-helpdesk` and + `fud-nohelpdesk`. + - Latest documented passing run created and closed controlled Helpdesk issue + `#39873`. + - Observed outbox rows: 1 `helpdesk_ticket.created`, 3 + `helpdesk_ticket.updated`, 1 `journal_message.created`, 3 + `journal.created`, and 2 `issue.updated`. + - Worker dry-run enrichment produced 10 `event`, 6 `ticket`, and 2 + `message` documents without claiming or marking rows processed. ## 2026-04-24 - Helpdesk/redMCP Smoke Validation diff --git a/docs/test_instance_post_import.md b/docs/test_instance_post_import.md index a2b1f58..ad032d8 100644 --- a/docs/test_instance_post_import.md +++ b/docs/test_instance_post_import.md @@ -136,7 +136,20 @@ non-Helpdesk CRUD, confirms default updates do not send Helpdesk email, verifies explicit outbound Mailpit delivery, and closes the created test ticket. Details are in `docs/helpdesk_smoke_test.md`. -## 8. Re-Run The Read-Only Validator +## 8. Validate Helpdesk Outbox Worker Enrichment + +When Helpdesk/outbox behavior matters, run the repeatable live validator: + +```sh +./validate_helpdesk_outbox_worker.py +``` + +This creates one controlled Helpdesk ticket through Mailpit, verifies redMCP +Helpdesk behavior, checks the matching `event_outbox_events` rows, and dry-runs +worker enrichment without claiming or marking rows processed. Details are in +`docs/helpdesk_outbox_worker_validation.md`. + +## 9. Re-Run The Read-Only Validator Finish by running: diff --git a/helpdesk_smoke_test.py b/helpdesk_smoke_test.py index b889185..0a114bf 100755 --- a/helpdesk_smoke_test.py +++ b/helpdesk_smoke_test.py @@ -54,6 +54,16 @@ class SmokeConfig: keep_open: bool +@dataclass(frozen=True) +class SmokeResult: + token: str + subject: str + issue_id: int + from_address: str + reply_token: str + control_issue_id: int + + def main() -> int: parser = argparse.ArgumentParser(description="Run a live Helpdesk/redMCP smoke test against the LAN Redmine copy.") parser.add_argument("--redmine-url", default=os.getenv("REDMINE_URL", DEFAULT_REDMINE_URL)) @@ -101,7 +111,7 @@ def main() -> int: return 1 -def run_smoke(config: SmokeConfig) -> None: +def run_smoke(config: SmokeConfig) -> SmokeResult: token = time.strftime("%Y%m%d%H%M%S") subject = f"[redMCP-smoke {token}] Helpdesk inbound validation" from_address = f"redmcp-smoke-{token}@example.test" @@ -158,6 +168,14 @@ def run_smoke(config: SmokeConfig) -> None: print("\nSmoke test passed.") print(f" issue_id: {issue_id}") print(f" subject: {subject}") + return SmokeResult( + token=token, + subject=subject, + issue_id=issue_id, + from_address=from_address, + reply_token=reply_token, + control_issue_id=control_id, + ) def load_env_file(path: Path) -> dict[str, str]: diff --git a/validate_helpdesk_outbox_worker.py b/validate_helpdesk_outbox_worker.py new file mode 100755 index 0000000..8d00a9f --- /dev/null +++ b/validate_helpdesk_outbox_worker.py @@ -0,0 +1,228 @@ +#!/usr/bin/env python3 +"""Live Helpdesk outbox/worker validation for the LAN Redmine copy. + +This creates one controlled Helpdesk ticket through Mailpit, then inspects the +outbox rows for that issue and dry-runs worker enrichment without claiming or +marking rows processed. +""" + +from __future__ import annotations + +import argparse +import json +import os +import sys +from collections import Counter +from dataclasses import dataclass +from pathlib import Path +from typing import Any + +from helpdesk_smoke_test import ( + DEFAULT_CONTROL_PROJECT, + DEFAULT_MAILPIT_HOST, + DEFAULT_MAILPIT_HTTP_PORT, + DEFAULT_MAILPIT_SMTP_PORT, + DEFAULT_PROJECT, + DEFAULT_REDMINE_URL, + DEFAULT_REMOTE_REDMINE, + DEFAULT_SSH_HOST, + DEFAULT_SSH_KEY, + SmokeConfig, + SmokeError, + load_env_file, + run_smoke, +) +from redmine_outbox_worker import OutboxWorkerError, RemoteRedmine, enrich_event, sql_int + + +EXPECTED_EVENT_TYPES = { + "helpdesk_ticket.created", + "helpdesk_ticket.updated", + "journal_message.created", + "journal.created", + "issue.updated", +} + + +class ValidationError(RuntimeError): + pass + + +@dataclass(frozen=True) +class ValidationConfig: + smoke: SmokeConfig + remote: RemoteRedmine + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Create a controlled Helpdesk issue and validate outbox worker enrichment for it." + ) + parser.add_argument("--redmine-url", default=os.getenv("REDMINE_URL", DEFAULT_REDMINE_URL)) + parser.add_argument("--api-key", default=os.getenv("REDMINE_API_KEY") or os.getenv("REDMNINE_API_KEY")) + parser.add_argument("--env-file", type=Path, default=Path("redMCP/.env")) + parser.add_argument("--mailpit-host", default=DEFAULT_MAILPIT_HOST) + parser.add_argument("--mailpit-http-port", type=int, default=DEFAULT_MAILPIT_HTTP_PORT) + parser.add_argument("--mailpit-smtp-port", type=int, default=DEFAULT_MAILPIT_SMTP_PORT) + parser.add_argument("--project", default=DEFAULT_PROJECT) + parser.add_argument("--control-project", default=DEFAULT_CONTROL_PROJECT) + parser.add_argument("--ssh-host", default=os.getenv("REDMINE_SSH_HOST", DEFAULT_SSH_HOST)) + parser.add_argument("--ssh-key", type=Path, default=Path(os.getenv("REDMINE_SSH_KEY", str(DEFAULT_SSH_KEY)))) + parser.add_argument("--remote-redmine", default=os.getenv("REDMINE_REMOTE_PATH", DEFAULT_REMOTE_REDMINE)) + parser.add_argument("--skip-preflight", action="store_true", help="Skip validate_test_instance.py.") + parser.add_argument("--keep-open", action="store_true", help="Do not close the created Helpdesk issue.") + args = parser.parse_args() + + try: + config = build_config(args) + run_validation(config) + return 0 + except (SmokeError, OutboxWorkerError, ValidationError) as exc: + print(f"error: {exc}", file=sys.stderr) + return 1 + + +def build_config(args: argparse.Namespace) -> ValidationConfig: + env = load_env_file(args.env_file) + redmine_url = args.redmine_url or env.get("REDMINE_URL") or DEFAULT_REDMINE_URL + api_key = args.api_key or env.get("REDMINE_API_KEY") or env.get("REDMNINE_API_KEY") + if not api_key: + raise ValidationError("missing Redmine API key. Set REDMINE_API_KEY or redMCP/.env.") + + smoke = SmokeConfig( + redmine_url=redmine_url.rstrip("/"), + api_key=api_key, + mailpit_host=args.mailpit_host, + mailpit_http_port=args.mailpit_http_port, + mailpit_smtp_port=args.mailpit_smtp_port, + project=args.project, + control_project=args.control_project, + ssh_host=args.ssh_host, + ssh_key=args.ssh_key, + remote_redmine=args.remote_redmine, + skip_preflight=args.skip_preflight, + keep_open=args.keep_open, + ) + remote = RemoteRedmine(args.ssh_host, args.ssh_key, args.remote_redmine) + return ValidationConfig(smoke=smoke, remote=remote) + + +def run_validation(config: ValidationConfig) -> None: + smoke_result = run_smoke(config.smoke) + events = fetch_issue_events(config.remote, smoke_result.issue_id) + assert_expected_events(events) + assert_events_unprocessed(events) + + documents: list[dict[str, Any]] = [] + for event in events: + documents.extend(enrich_event(config.remote, event)) + + assert_worker_documents(events, documents) + assert_no_bcc_address_leak(documents) + print_summary(smoke_result.issue_id, events, documents) + + +def fetch_issue_events(remote: RemoteRedmine, issue_id: int) -> list[dict[str, Any]]: + events = remote.mysql_json_lines( + f""" +SELECT HEX(CAST(JSON_OBJECT( + 'id', id, + 'event_type', event_type, + 'source_type', source_type, + 'source_id', source_id, + 'project_id', project_id, + 'issue_id', issue_id, + 'journal_id', journal_id, + 'user_id', user_id, + 'occurred_at', DATE_FORMAT(occurred_at, '%Y-%m-%dT%H:%i:%sZ'), + 'attempts', attempts, + 'processed_at', IF(processed_at IS NULL, NULL, DATE_FORMAT(processed_at, '%Y-%m-%dT%H:%i:%sZ')), + 'locked_at', IF(locked_at IS NULL, NULL, DATE_FORMAT(locked_at, '%Y-%m-%dT%H:%i:%sZ')), + 'payload', payload +) AS CHAR)) AS document +FROM event_outbox_events +WHERE issue_id = {sql_int(issue_id)} +ORDER BY id; +""" + ) + if not events: + raise ValidationError(f"no event_outbox_events rows found for issue #{issue_id}") + return events + + +def assert_expected_events(events: list[dict[str, Any]]) -> None: + found = {str(event.get("event_type") or "") for event in events} + missing = sorted(EXPECTED_EVENT_TYPES - found) + if missing: + raise ValidationError(f"missing expected Helpdesk outbox event types: {', '.join(missing)}") + + +def assert_events_unprocessed(events: list[dict[str, Any]]) -> None: + mutated = [ + event + for event in events + if event.get("processed_at") is not None or event.get("locked_at") is not None + ] + if mutated: + ids = ", ".join(str(event.get("id")) for event in mutated) + raise ValidationError(f"new smoke-test outbox rows were already processed or locked: {ids}") + + +def assert_worker_documents(events: list[dict[str, Any]], documents: list[dict[str, Any]]) -> None: + doc_types = {str(document.get("doc_type") or "") for document in documents} + required_doc_types = {"event", "ticket", "message"} + missing_doc_types = sorted(required_doc_types - doc_types) + if missing_doc_types: + raise ValidationError(f"worker dry-run did not derive document types: {', '.join(missing_doc_types)}") + + event_doc_ids = {document.get("event_id") for document in documents if document.get("doc_type") == "event"} + missing_events = [str(event.get("id")) for event in events if event.get("id") not in event_doc_ids] + if missing_events: + raise ValidationError(f"worker dry-run did not emit event documents for rows: {', '.join(missing_events)}") + + +def assert_no_bcc_address_leak(documents: list[dict[str, Any]]) -> None: + for document in documents: + if contains_key(document, "bcc_address"): + raise ValidationError(f"worker document leaked raw bcc_address: {document.get('doc_id')}") + + +def contains_key(value: Any, needle: str) -> bool: + if isinstance(value, dict): + return any(key == needle or contains_key(child, needle) for key, child in value.items()) + if isinstance(value, list): + return any(contains_key(child, needle) for child in value) + return False + + +def print_summary(issue_id: int, events: list[dict[str, Any]], documents: list[dict[str, Any]]) -> None: + event_types = Counter(str(event.get("event_type") or "") for event in events) + doc_types = Counter(str(document.get("doc_type") or "") for document in documents) + ticket_ids = sorted( + { + int(document["helpdesk_ticket_id"]) + for document in documents + if document.get("doc_type") == "ticket" and document.get("helpdesk_ticket_id") + } + ) + message_ids = sorted( + { + int(document["journal_message_id"]) + for document in documents + if document.get("doc_type") == "message" and document.get("journal_message_id") + } + ) + event_rows = [f"{event.get('id')}:{event.get('event_type')}" for event in events] + + print("\nHelpdesk outbox worker validation passed.") + print(f" issue_id: {issue_id}") + print(f" helpdesk_ticket_ids: {json.dumps(ticket_ids)}") + print(f" journal_message_ids: {json.dumps(message_ids)}") + print(f" event_rows: {', '.join(event_rows)}") + print(f" event_type_counts: {json.dumps(dict(sorted(event_types.items())), sort_keys=True)}") + print(f" derived_doc_type_counts: {json.dumps(dict(sorted(doc_types.items())), sort_keys=True)}") + print(" worker_mode: dry-run enrichment only; rows were not claimed or marked processed") + + +if __name__ == "__main__": + raise SystemExit(main())