Add outbox worker processing policy

README.md

@@ -222,16 +222,22 @@ It runs outside Redmine and consumes `event_outbox_events` over SSH/MySQL. The
 initial output target is deterministic local JSONL rather than a live search
 service:
 
-- default output: `.cache/redmine_outbox/derived_documents.jsonl`
+- default output: `/tmp/redmine-outbox/derived_documents.jsonl`
 
 Current behavior:
 
 - dry-runs pending rows without marking them processed
+- reports queue counts with `--status`
 - claims bounded batches with `locked_at` and `locked_by`
 - enriches helpdesk ticket/message/contact-related events with read-only joins
 - writes derived event/ticket/message/contact documents as JSONL
 - marks rows processed only after a successful local write
 - increments `attempts` and writes `last_error` when processing fails
+- previews or applies processed-row cleanup with `--purge-processed-days`
+
+The worker processing policy is documented in:
+
+- [docs/outbox_worker_policy.md](/home/iadnah/redmine/docs/outbox_worker_policy.md:1)
 
 ### 7. Test Helpdesk Mail Reset
 
@@ -335,18 +341,10 @@ The deployed helpdesk search routes were verified with:
 
 ## What Is Not Finished Yet
 
-### 1. Worker Processing Policy
+### 1. External Search Index
 
-The Helpdesk outbox worker now has a repeatable live validator. The next worker
-decision is operational policy:
-
-- choose when to mark rows processed on the test instance
-- decide where durable JSONL/index output should live
-- define retention or replay expectations for `event_outbox_events`
-
-### 2. External Search Index
-
-We have not yet built the actual external index.
+The worker can now publish bounded JSONL batches and mark successful rows
+processed. We have not yet built the actual external index.
 
 Planned direction:
 
@@ -358,7 +356,7 @@ Planned direction:
 This should wait until the worker validation pass has documented the derived
 document shape that will feed the index.
 
-### 3. Pre-Existing UI/Plugin Bugs
+### 2. Pre-Existing UI/Plugin Bugs
 
 We discovered old plugin issues while working:
 
@@ -401,10 +399,9 @@ If continuing this project, the next best work is:
 2. Generate controlled Helpdesk activity in `fud-helpdesk`.
 3. Inspect `event_outbox_events` for the corresponding issue, journal,
    Helpdesk ticket, and journal message rows.
-4. Run `redmine_outbox_worker.py --dry-run --batch-size 10` and document the
-   derived JSONL output shape.
-5. Process a bounded batch only after the dry-run output is correct.
-6. Choose the first external index target after the worker output is validated.
+4. Check worker queue state with `redmine_outbox_worker.py --status`.
+5. Process a bounded JSONL batch after dry-run output is correct.
+6. Choose the first external index target after JSONL publishing is stable.
 
 ## Practical Commands
 
@@ -448,12 +445,24 @@ processed:
 ./redmine_outbox_worker.py --dry-run --batch-size 10
 ```
 
-Process a bounded outbox batch into local JSONL and mark successful rows:
+Check queue status:
+
+```sh
+./redmine_outbox_worker.py --status
+```
+
+Process a bounded outbox batch into JSONL and mark successful rows:
 
 ```sh
 ./redmine_outbox_worker.py --batch-size 20
 ```
 
+Preview processed-row cleanup:
+
+```sh
+./redmine_outbox_worker.py --purge-processed-days 30
+```
+
 Search the cached helpdesk documents:
 
 ```sh
@@ -474,7 +483,6 @@ not cosmetic UI work. It is the architectural clarification that helpdesk data
 is the authoritative customer communication layer, plus the first safe event and
 export tooling around it.
 
-The repo is now at the point where the next meaningful leap is an external
-worker validation pass, not more Redmine UI surface. The Qdrant/OpenAI index
-work should follow once the worker output is proven against controlled Helpdesk
-activity.
+The repo is now at the point where bounded external worker publishing is proven
+against the LAN test instance. The next meaningful leap is choosing and building
+the first external search/index target, likely Qdrant with OpenAI embeddings.