Add outbox worker processing policy
This commit is contained in:
@@ -384,7 +384,7 @@ row exists.
|
||||
|
||||
## Worker/Rake Task
|
||||
|
||||
Current implemented worker-facing command:
|
||||
Current implemented Redmine-side worker-facing command:
|
||||
|
||||
```sh
|
||||
bundle exec rake redmine_event_outbox:dump RAILS_ENV=production
|
||||
@@ -392,7 +392,7 @@ bundle exec rake redmine_event_outbox:dump RAILS_ENV=production
|
||||
|
||||
This prints pending rows as JSON and does not mark them processed.
|
||||
|
||||
Next worker command:
|
||||
Historical planned Redmine-side publish command:
|
||||
|
||||
```sh
|
||||
bundle exec rake redmine_event_outbox:publish RAILS_ENV=production
|
||||
@@ -411,10 +411,11 @@ bundle exec rake redmine_event_outbox:publish
|
||||
bundle exec rake redmine_event_outbox:publish RETRY=1
|
||||
```
|
||||
|
||||
Initial publisher target can be stdout or a local JSONL file. That lets us test
|
||||
the Redmine side before choosing Redis Streams or RabbitMQ.
|
||||
The current external publisher is `redmine_outbox_worker.py`. Its initial
|
||||
publisher target is JSONL at `/tmp/redmine-outbox/derived_documents.jsonl`,
|
||||
which lets us test the Redmine side before choosing Redis Streams or RabbitMQ.
|
||||
|
||||
The next implementation should keep the worker small and conservative:
|
||||
The worker stays small and conservative:
|
||||
|
||||
- select pending rows in id order
|
||||
- lock or claim a bounded batch
|
||||
@@ -424,6 +425,19 @@ The next implementation should keep the worker small and conservative:
|
||||
- leave failed rows available for retry
|
||||
- make duplicate delivery acceptable to consumers
|
||||
|
||||
Current external worker commands:
|
||||
|
||||
```sh
|
||||
./redmine_outbox_worker.py --status
|
||||
./redmine_outbox_worker.py --dry-run --batch-size 10
|
||||
./redmine_outbox_worker.py --batch-size 20
|
||||
./redmine_outbox_worker.py --purge-processed-days 30
|
||||
./redmine_outbox_worker.py --purge-processed-days 30 --apply-purge
|
||||
```
|
||||
|
||||
Processed rows are retained by default. Purge is explicit, age-gated, and
|
||||
intended for test-instance cleanup, not normal delivery.
|
||||
|
||||
Later publisher targets:
|
||||
|
||||
- Redis Streams
|
||||
@@ -550,7 +564,8 @@ decision code fails.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- How long should processed outbox rows be retained?
|
||||
- Do production processed rows need a formal retention window, or should they
|
||||
be retained until external index/message-bus replay policy is settled?
|
||||
- Do we need a Redmine admin page to inspect outbox health, or is a rake task/log enough?
|
||||
- What is the current production mail ingestion path?
|
||||
- Which vector-capable external search index should be used first?
|
||||
|
||||
Reference in New Issue
Block a user