Jason Thistlethwaite
64 lines
1.5 KiB
Markdown
64 lines
1.5 KiB
Markdown
# Outbox Worker Policy
|
|
|
|
`redmine_outbox_worker.py` is the first external publisher for
|
|
`event_outbox_events`. It currently publishes enriched documents to JSONL, but
|
|
the processing policy is intended to carry forward to a later message bus.
|
|
|
|
## Default Output
|
|
|
|
The default output is:
|
|
|
|
```sh
|
|
/tmp/redmine-outbox/derived_documents.jsonl
|
|
```
|
|
|
|
Use `--output /path/to/file.jsonl` to override it. The worker creates parent
|
|
directories as needed.
|
|
|
|
## Processing Semantics
|
|
|
|
- Read eligible pending rows globally in id order.
|
|
- Claim a bounded batch with `locked_at` and `locked_by`.
|
|
- Enrich each event with read-only joins.
|
|
- Append derived JSON documents to the output.
|
|
- Mark a row `processed_at` only after the output write succeeds.
|
|
- On failure, increment `attempts`, write `last_error`, and release the lock.
|
|
|
|
This is at-least-once delivery. Downstream consumers must tolerate duplicate
|
|
documents keyed by `event_id` or `doc_id`.
|
|
|
|
## Operations
|
|
|
|
Inspect queue state:
|
|
|
|
```sh
|
|
./redmine_outbox_worker.py --status
|
|
```
|
|
|
|
Preview a batch without claiming or marking rows processed:
|
|
|
|
```sh
|
|
./redmine_outbox_worker.py --dry-run --batch-size 10
|
|
```
|
|
|
|
Process a bounded batch:
|
|
|
|
```sh
|
|
./redmine_outbox_worker.py --batch-size 20
|
|
```
|
|
|
|
Preview processed-row cleanup:
|
|
|
|
```sh
|
|
./redmine_outbox_worker.py --purge-processed-days 30
|
|
```
|
|
|
|
Apply cleanup explicitly:
|
|
|
|
```sh
|
|
./redmine_outbox_worker.py --purge-processed-days 30 --apply-purge
|
|
```
|
|
|
|
Processed rows are kept by default. Purge is an administrative cleanup tool for
|
|
test instances and should not be part of normal publishing.
|