ldr/redmine
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Organized

Browse Source
This commit is contained in:
Jason Thistlethwaite
2026-10-17 20:50:35 -04:00
parent bd26c8894f
commit 9ee9eb3ec6
130 changed files with 24172 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/raw-notes/Untitled.md
+33
View File
@@ -0,0 +1,33 @@
@everyone I think we should do some tune-ups and checks on the "advanced" Redmine CLI located in ~/projects/redmine/ that is pointed to by ~/projects/redmine/bin/rtl
We kind of had a history with this tool:
- Initially, it kicked ass and was very fast and powerful, but it lacked a nice TUI
- We attempted to build a TUI using curses, and found that had severe limitations on the UI; the functionality was powerful, but the UI/UX sucked ass.
- We tried rebuilding a TUI using more modern libraries and apps as reference points, and somehow totally lost the plot. Not only did we end up with incomplete features agents swore they had completed, we ended up regressing the codebase terribly, to the point even the original CLI features became unreliable, didn't do what the docs/specs said they did, and all kinds of problems. Nobody involved was able to identify a straight path to fixing that.
- We only recently seem to have restored the basic CLI functionality of the tool. From what I understand, the majority of the problem(s) turned out to be:
a) Unreachable code in CLI argument parsing
b) Something to do with order of arguments supplied before or after others would cause silent failures / weird behavior
c) Unverified, but we've seen evidence there is a bug that can leave the CACHE in an unusable state if the script is terminated before it finishes a sync, and the code is seems to be unable to recover from or _recognize_ that problem. On subsequent runs, it will act like there are no problems, it just doesn't give correct output if this happens.
### As far as I know, at this point, the following things works correctly:
- `jason@onyx > ./bin/rtl sync --remote-status-id '*' --remote-updated-since 30d --run-log --run-log-keep 20` run every 5-10 minutes to keep the local cache updated
- the `walk` verb, except it does not have a help menu for what commands it supports (n, p, q, r)
- view verb, but unclear if it displays local cache or fetches live
- `materialize-queues` seems to work
- `contacts-sync`
- Need to make sure this _does not_ delete old contacts if run, for instance, with something like `--nax0contacts 50`
-
### Minor issues
- `draft` verb:
- doesn't have mechanism to _show_ a drafted change
- should have simple way to clear old drafts or something
-
### Definitely not working:
- triage verb
-
docs/raw-notes/features-for-redmine_tickets-sync.md
+332
View File
@@ -0,0 +1,332 @@
# Features / Roadmap: power-user Redmine client (`redmine_tickets` plane)
> [!todo] Keep this file as the **index**: status, priority, and short summaries.
> Long implementation detail lives in `docs/raw-notes/features/*` and is linked from here.
> Update checkboxes when work lands; do not re-expand full specs into this file.
**Last status refresh:** 2026-07-15 (post browse-loop + attachments + entities + contacts bridge)
---
## Product intent (what this is)
A **scriptable power-user Redmine client** for humans and agents:
- durable local ticket (+ contact) cache as the **ops workbench**
- property-based local search, saved filters, filesystem queues
- CLI / TUI-lite read path (Markdown + `glow`/`bat`/`fzf`) then **queued** writes
- place to invent/test filters, tools, and agent loops without living in the web UI
**Not** the system of record (Redmine is).
**Not** the vault/wiki (curated memory).
**Not** Qdrant (optional later semantic layer).
First project focus: **`customer-service`**.
| Env | Redmine | Cache | Wrapper |
|-----|---------|--------|---------|
| Prod | `https://rm.ldrprep.com` | `/home/jason/projects/redmine/cache/redmine_tickets` | `bin/rtl` |
| Dev / mutation testing | `http://192.168.50.170` | `/home/jason/projects/redmine/cache-dev/redmine_tickets` | `bin/rtd` |
**Safety defaults**
- Writes/applies prefer LAN; prod needs parent `--allow-prod-writes`
- `note` never emails; `customer-email` always emails (ugly confirm flag)
- Browse/actions **queue drafts** by default (apply later)
- Dual tools OK for now: tickets workbench vs full `redmine_contacts.py` CRM CLI, with bridges
---
## Priority stack (current)
| Priority | Area | Status |
|----------|------|--------|
| P0 | Durable incremental ticket sync + remote filters | **Done** |
| P0 | Cron for CS refresh (+ queue rematerialize) | **Running** (6h window) |
| P1 | Property-based local search (`--where`) | **Done** |
| P1 | Markdown view + walk | **Done** |
| P1 | Browse loop (fzf → pager → action menu) | **MVP done** (polish open) |
| P2 | Saved filters + symlink queues | **Done** |
| P2 | Queued notes / customer-email / set-fields | **Done** |
| P2 | Config wrappers (`--config`, `rtl`/`rtd`) | **Minimal done** |
| P3 | Local control tags + suggest + JSONL export | **Done** (LLM refine open) |
| P3 | Contacts mirror + contact-set + ticket-contact | **Done** (cache unify optional) |
| P4 | Attachments list/download/upload | **Done** |
| P4 | Entity extract (patterns) + NMI bridge CLI | **MVP done** |
| Later | Full curses TUI / nested config DSL / master layout | Deferred |
| Adjacent | Semantic index / vault decompose / OCR rename skill | Separate tracks |
---
## Capability map (what you can run today)
| Need | Command(s) |
|------|------------|
| Refresh tickets | `sync` · cron: `cron-scripts/sync-live-customer-service.sh` |
| Search cache | `search --where '…'` · `@saved` · `--control-tag` · `--has-attachments` |
| Read ticket | `view ID` · `walk` · **`browse`** (loop) |
| Internal note | `note` (draft → `draft apply`) |
| Customer email | `customer-email` + confirm flag |
| Fields | `set-fields` (`--status Closed`, etc.) · `meta-refresh` |
| Local labels | `control-tag add|suggest|export|…` |
| Contacts | `contacts-sync` · `contacts-search` · `contact-show` · `contact-set` · `ticket-contact --fetch` |
| Files | `attachments list|download|upload` |
| Tracking/order hints | `entities show|scan` · `entities nmi` · browse `m` |
| OCR scan rename | `ocr-rename --issue-id` / `--where '@scanned-document'` |
| Categories | `categories list [--refresh]` |
| Queues on disk | `materialize-queues --config queues.example.json` |
Cookbook: **`docs/cli_power_user_client.md`**
---
## P0 — Sync foundation
**Detail:** [[features/sync-foundation]]
- [x] Stable cache path (not per-run timestamped dirs)
- [x] Incremental full-fetch skip when `updated_on` unchanged
- [x] Server-side `updated_on` + `status_id` (remote flags)
- [x] Local post-filters retained
- [x] Sync progress + `--quiet` for cron
- [x] Cache never auto-deletes old ticket JSON
- [x] Run-stats ring buffer (`--run-log`)
- [x] Cron: live + dev CS scripts; **6h** remote window; rematerialize queues after sync
- [x] Optional contacts cron: `cron-scripts/sync-live-contacts.sh`
- [ ] Confirm long-term cadence (currently ~every few minutes) is still desired
- [x] Full fetch includes `attachments` metadata (not just journals/helpdesk)
```bash
./bin/rtl sync --remote-status-id '*' --remote-updated-since 2h --run-log --quiet
```
---
## P1 — Local search
**Detail:** [[features/local-search-and-cli-client]] · `redmine_ticket_query.py`
**Historical thin search:** [[../2026-07-14_localsearch_before_sanity]]
- [x] JSON path filters + `includes()` / `since()` / AND/OR / parens
- [x] Nested helpdesk/contact paths
- [x] `--ids-only`, ordering, max caps
- [x] Table columns: assignee, contact, control tags
- [x] `--has-attachments` (cache metadata)
- [x] **No default status filter** — closed tickets appear if cached
- [ ] Optional default “open-ish” profile (if desired later)
---
## P1 — Read path + browse loop
**Detail:** [[features/local-search-and-cli-client]] · `redmine_ticket_view.py` · `redmine_ticket_browse.py`
- [x] Ticket → Markdown (`view`) + MD cache under `markdown/`
- [x] Pagers: glow / bat / batcat / cat
- [x] `walk` (n/p/q)
- [x] **`browse` loop:** fzf (or numbered) → pager → action menu → list
- Actions: view again, queue note/private note, queue fields, control-tag, contact handoff, entities, print id
- Mutations **queue only**
- `--once`, `--pager`, filters same family as search
- [ ] Browse polish (user-noted “minor things”) — collect below under **Open polish**
- [ ] Full curses/TUI (linear-tui style) still **later**, not blocking ops
```bash
./bin/rtl browse --where 'assigned_to IS NULL' --max-tickets 80 --pager glow
```
---
## P2 — Filters, queues, mutations
**Detail:** [[features/local-search-and-cli-client]] · drafts in `redmine_ticket_drafts.py`
- [x] `saved-filters/*.expr` + `@name`
- [x] `materialize-queues` symlink folders
- [x] `note` vs `customer-email` (hard separation; Helpdesk `email_note` path)
- [x] Draft list/cancel/apply; draft binds `base_url`; obvious `draft_id=`
- [x] `set-fields` + name→id (`meta-refresh`)
- [x] Parent `--allow-prod-writes` (not subparser-shadowed)
- [x] Minimal `--config` + `bin/rtl` / `bin/rtd` / `bin/qaa`
- [ ] Nested config defaults (`search.max-tickets` etc.) — [[features/config-files]]
- [ ] Note body from file + optional pandoc→Textile (steering list)
```bash
./bin/rtl note --issue-id 41934 --body 'staff context' --private
./bin/rtl customer-email --issue-id 41934 --body '…' --i-understand-this-emails-the-customer
./bin/rtl --allow-prod-writes draft apply <id>
```
---
## P3 — Control tags
**Detail:** [[features/control-tags]] · export: `redmine_ticket_control_export.py`
- [x] Local wipeable store under `control-tags/tags.json`
- [x] add / remove / list / show / for-issue / wipe
- [x] suggest (`--where` rules, optional `--apply` local)
- [x] `export` → JSONL for training / refine
- [ ] LLM refine loop (consume export, propose tags, human correct/incorrect)
---
## P3 — Contacts
**Detail:** [[features/contacts-sync-integration]] · `redmine_ticket_contacts.py`
**Full CRM still:** `redmine_contacts.py` (separate cache/CLI by design for now)
- [x] Durable mirror: `<cache>/contacts/`
- [x] `contacts-sync` / `contacts-search` / `contact-show`
- [x] `contact-set` draft/apply (tags, company, names, `--set`)
- [x] `ticket-contact` bridge; **`--fetch` implies show+refresh** (bugfix)
- [x] Contact column in ticket search
- [ ] Optional unify on-disk layout with `redmine_contacts.py` (only if dual-cache pain grows)
---
## P4 — Attachments
**Detail:** [[features/attachments]] · `redmine_ticket_attachments.py`
- [x] `attachments list` (cache-first, `--refresh`)
- [x] `attachments download` (one / `--all`)
- [x] `attachments upload --apply` (POST `/uploads.json` + issue `uploads`; no email)
- [x] OCR rename workflow CLI (`ocr-rename`) — [[features/ocr-rename]]
---
## P4 — Entities + NMI
**Detail:** [[features/entity-type-mapper]] · `redmine_ticket_entities.py` · `redmine_ticket_nmi.py`
- [x] Pattern extract: UPS/USPS, order/SKU/account + LDR `item_no`
- [x] Shown in Markdown view
- [x] `entities show|scan`
- [x] `entities nmi` bridge to `mrscClient-php bin/nmi` (cs-brief / resolve)
- [x] Noise filters on medium-confidence `order_id` hits
- [ ] Richer NMI membership index (known order/item windows) if CLI alone isn’t enough
- [x] One-key NMI from browse menu (`m` → private note draft)
---
## Open polish / small bugs (browse + ops)
- [x] Browse: fzf header/keybinding discoverability
- [x] Browse: optional preview window in fzf (`--no-preview` to disable)
- [x] Browse: louder draft queue banner (`announce_draft`)
- [x] Browse: `m` → entities nmi brief → private note draft
- [x] Entity noise filters + LDR 9-digit `item_no`
- [x] Note `--body-file` (already) + `--to-textile` (pandoc)
- [x] `search`/`browse`/`walk` `--open-only`
- [x] OCR attachment → rename (`ocr-rename`) MVP
- [ ] OCR cron pack for HR “Scanned document” inboxes
- [ ] categories create/update (list only for now)
---
## Steering / next decisions (pick a lane)
### A — Daily-driver polish — **done for now**
Reopen if browse UX still nags.
### B — CS frontage / NMI / OCR — **MVP+ done**
Optional: OCR cron for HR; NMI membership index.
### C — CRM / spam tooling **← good next lane**
1. Deeper `contact-set` (custom fields) or keep routing to `redmine_contacts.py`
2. Control-tag LLM refine from JSONL export
3. Issue categories create/update for triage taxonomy
### D — Bigger product bets (defer unless intentional)
- Full curses TUI
- Unified contact cache
- Composable remote/local filter DSL
- Redmine project/version/custom-field admin
---
## Explicitly out of this index (or later)
| Item | Notes |
|------|--------|
| Semantic index v2 | `semantic_index/` — feed from this cache |
| Vault decompose | separate track |
| Redmine admin: versions / custom field *definitions* / project modules | not ticket ops |
| Master global JSON + symlink org | nice-to-have |
---
## Related docs
- `docs/cli_power_user_client.md` — **usage cookbook**
- `docs/ticket_sync_tool_plan.md` — original sync plan
- `docs/2026-07-14_localsearch_before_sanity.md` — historical
- `docs/ticket_spam_cleanup_handoff_2026-07-09.md` — spam consumers
- `docs/raw-notes/features/*.md` — detail files
- `configs/live.json`, `configs/dev.json` · `bin/rtl` · `bin/rtd`
### Feature detail files
| File | Topic |
|------|--------|
| [[features/sync-foundation]] | Sync / cache / remote filters |
| [[features/local-search-and-cli-client]] | Search, view, walk, queues, writes |
| [[features/control-tags]] | Local control tags |
| [[features/contacts-sync-integration]] | Contacts mirror + bridge |
| [[features/attachments]] | File attachments |
| [[features/entity-type-mapper]] | Entities + NMI |
| [[features/nmi-bridge]] | NMI env + browse `m` |
| [[features/ocr-rename]] | OCR subject rename |
| [[features/browse-loop]] | fzf browse loop |
| [[features/config-files]] | Config / aliases |
---
## Maintenance rule
1. Put full detail in `docs/raw-notes/features/<name>.md`
2. Leave status + short bullets + link here
3. Prefer updating the feature file over re-pasting long prose into this index
4. When fixing a bug from **Bug reports**, check the box and leave the minimal repro snippet
---
## Bug reports and QOL (resolved log)
### --allow-prod-writes unrecognized / ignored
- [x] Parent flag only; subparser copies removed (child defaults overwrote parent)
### Draft id not obvious
- [x] `draft_id=` / `apply_hint=` key=value output
### Drafts ignore create-time base-url
- [x] Apply prefers draft `base_url`; conflict needs `--allow-base-url-override`
### Configs / short aliases
- [x] Minimal `--config` + `bin/rtl|rtd|qaa`
### `ticket-contact --fetch` alone did nothing
- [x] `--fetch` implies show + live refresh into contacts cache
---
## Modules (code map)
| Module | Role |
|--------|------|
| `redmine_tickets.py` | CLI entry + sync + routing |
| `redmine_ticket_query.py` | Local AST filters |
| `redmine_ticket_view.py` | Markdown render / pager |
| `redmine_ticket_browse.py` | fzf loop + action menu |
| `redmine_ticket_filters.py` | Saved filters / queues |
| `redmine_ticket_drafts.py` | Draft queue + write safety hosts |
| `redmine_ticket_meta.py` | Name→id meta cache |
| `redmine_ticket_control_tags.py` | Local tags store |
| `redmine_ticket_control_export.py` | Tag JSONL export |
| `redmine_ticket_contacts.py` | Contact mirror |
| `redmine_ticket_attachments.py` | Upload/download helpers |
| `redmine_ticket_entities.py` | Pattern extract |
| `redmine_ticket_nmi.py` | NMI CLI bridge |
| `redmine_ticket_ocr.py` | Attachment OCR + subject suggest |
| `redmine_contacts.py` | Full CRM CLI (sibling tool) |
docs/raw-notes/features/attachments.md
+28
View File
@@ -0,0 +1,28 @@
# Feature detail: Ticket attachments
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** MVP 2026-07-15
**Module:** `redmine_ticket_attachments.py`
## Commands
```bash
./bin/rtl attachments list --issue-id 39305 --refresh
./bin/rtl attachments download --issue-id 39305 --attachment-id 55917
./bin/rtl attachments download --issue-id 39305 --all
./bin/rtd attachments upload --issue-id 39305 --file ./scan.pdf # dry-run
./bin/rtd attachments upload --issue-id 39305 --file ./scan.pdf --apply --notes 'scan attached'
```
| Op | Emails customer? | Notes |
|----|------------------|--------|
| list | no | cache-first; `--refresh` pulls `include=attachments` |
| download | no | needs API key; default dir `<cache>/<project>/attachments/<issue_id>/` |
| upload | **no** | Redmine `POST /uploads.json` + issue `uploads` token; prod needs `--allow-prod-writes` |
Sync now also requests `attachments` on full issue fetch so metadata lands in cache over time.
## Still open
- OCR rename workflow (see [[../ocr-ticket-attachment-to-rewrite-update-ticket]])
- Draft-queue binary uploads (paths) if desired
docs/raw-notes/features/browse-loop.md
+44
View File
@@ -0,0 +1,44 @@
# Feature detail: Interactive browse loop
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** MVP+ polish 2026-07-15
**Module:** `redmine_ticket_browse.py`
## Flow
1. Filtered ticket list via **fzf** (preview pane) or numbered pick
2. **Enter** → Markdown pager (`glow` / `bat` / `batcat` / `cat`)
3. After pager → **action menu**
4. Back to list until Esc/q
## Actions (queue-safe)
| Key | Behavior |
|-----|----------|
| v / Enter | View again |
| n / p | Queue internal / private note |
| f | Queue field update (names) |
| t | Local control-tag |
| c | ticket-contact handoff |
| e | Pattern entities |
| **m** | NMI cs-brief → confirm → **private** note draft |
| i | Print id |
| l / q | List / quit |
## Flags
- `--where`, `--status`, `--control-tag`, `--has-attachments`, **`--open-only`**
- `--max-tickets`, `--order`, `--pager`, `--once`, **`--no-preview`**
## Done polish
- [x] fzf header lists actions
- [x] fzf preview pane
- [x] loud draft banner
- [x] NMI menu action
## Non-goals
- Full curses app (separate spike)
- Customer-email from browse (CLI-only by design)
- Applying prod writes from browse without draft step
docs/raw-notes/features/config-files.md
+41
View File
@@ -0,0 +1,41 @@
# Config files / short aliases (minimal)
**Parent:** [[../features-for-redmine_tickets-sync]]
**Status:** minimal MVP (global defaults + bin wrappers). Nested per-command defaults deferred.
## Runtime config
```bash
./redmine_tickets.py --config configs/live.json search --max-tickets 10
./redmine_tickets.py --config configs/dev.json note --issue-id 39305 --body 'x' --apply
```
JSON (always works) or YAML if PyYAML is installed.
Supported top-level keys (also under `redmine_tickets:`):
- `base-url` / `base_url`
- `project`
- `cache-dir` / `cache_dir`
- `allow-prod-writes` (bool)
CLI flags already present win over config.
## Short wrappers (`bin/`)
| Command | Maps to |
|---------|---------|
| `bin/rtl` | live/prod CS (`configs/live.json`) |
| `bin/rtd` | LAN/dev CS (`configs/dev.json`) |
| `bin/qaa` | alias → `rtl` (muscle-memory example) |
```bash
./bin/rtl search --where 'assigned_to IS NULL' --max-tickets 5
./bin/rtl --allow-prod-writes draft apply <id>
./bin/rtd note --issue-id 39305 --body 'LAN note' --apply
```
## Not yet
- Nested `search.max-tickets` / command-specific defaults from YAML namespaces
- Auto-discovery of many named profiles beyond live/dev
docs/raw-notes/features/contacts-handling.sh
+5
View File
@@ -0,0 +1,5 @@
## Contact Triage / Fixing
- [ ] Properly link "call transcript" and "voicemail" tickets to the correct person by searching the contact database with the phone number involved
- [ ] Ingest call transcripts into our timeline
- [ ] Fix contact company is set to their email address, when the name of their business is easily found through context in tickets
docs/raw-notes/features/contacts-sync-integration.md
+38
View File
@@ -0,0 +1,38 @@
# Feature detail: Contacts sync + mutation (tie-in)
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** Sync/search/show + draftable `contact-set` + `ticket-contact` bridge (2026-07-15)
**Module:** `redmine_ticket_contacts.py`
**Still separate from:** full `redmine_contacts.py` CRM CLI (batch/loops/custom-field tooling)
## Why not one tool
- Tickets workbench wants co-located per-id JSON + draft/apply safety next to tickets.
- `redmine_contacts.py` already owns heavy CRM workflows and a different cache layout.
- Do not rewrite/delete the CRM CLI; add bridges so a ticket-started workflow can end on contacts.
## Implemented
```bash
./bin/rtl contacts-sync --max-contacts 100
./bin/rtl contacts-search --query ada
./bin/rtl contact-show --contact-id 14 --fetch
# Bridge ticket → contact id
./bin/rtl ticket-contact --issue-id 41934
./bin/rtl ticket-contact --issue-id 41934 --show --fetch
# CRM mutation (draft by default; same prod/LAN gates as ticket writes)
./bin/rtl contact-set --contact-id 4515 --add-tag maybe-customer
./bin/rtd contact-set --contact-id 2284 --add-tag maybe-customer --apply --refresh-cache
```
- Durable path: `<cache-dir>/contacts/<id>.json` + `manifest.json`
- Draft kind: `contact_fields` (apply via `draft apply`)
- Handoff hints print `redmine_contacts.py show …` for full CRM when needed
## Still open
- Richer custom-field helpers (use `redmine_contacts.py` or extend `--set`)
- Optional cron for contacts next to ticket sync
- Shared library / single on-disk layout if dual-cache friction becomes painful
docs/raw-notes/features/control-tags.md
+31
View File
@@ -0,0 +1,31 @@
# Feature detail: Local control tags (filter-assist labeling)
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** MVP implemented 2026-07-15 (`control-tag` CLI + `search --control-tag`)
**Module:** `redmine_ticket_control_tags.py`
**Critical distinction:** these tags are **local-only**. They are **not** Redmine issue tags, **not** contact CRM tags.
## Storage
`<cache-dir>/control-tags/tags.json` — wipe without touching Redmine or ticket JSON.
## CLI
```bash
./redmine_tickets.py ... control-tag add --tag bavaria --issue-id 41934 \
--property subject --property helpdesk_ticket.contact.company \
--annotation 'BavariaTrachten shape'
./redmine_tickets.py ... control-tag list
./redmine_tickets.py ... control-tag show --tag bavaria
./redmine_tickets.py ... control-tag for-issue --issue-id 41934
./redmine_tickets.py ... control-tag remove --tag bavaria --issue-id 41934
./redmine_tickets.py ... control-tag wipe --i-understand-this-wipes-local-control-tags
./redmine_tickets.py ... search --control-tag bavaria
```
## Still open
- Suggest/refine loop (LLM or rules + human correct/incorrect)
- Training-export format for classifier experiments
docs/raw-notes/features/entity-type-mapper.md
+25
View File
@@ -0,0 +1,25 @@
# Feature detail: External entity type mapper / identifier
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** pattern MVP + NMI CLI bridge + browse `m` (2026-07-15)
**Modules:** `redmine_ticket_entities.py`, `redmine_ticket_nmi.py`
**NMI env docs:** [[nmi-bridge]]
## Commands
```bash
./bin/rtl entities show --issue-id 41394
./bin/rtl entities scan --where 'subject.includes("1Z")' --max-tickets 20
./bin/rtl entities nmi --issue-id 41394
```
## Done
- Pattern types: UPS/USPS, FedEx candidates, Amazon order, order_id, item_or_sku, **item_no** (LDR 9-digit)
- Markdown view section: **Detected entities**
- NMI bridge + browse **`m`** → private note draft
## Open
- [ ] Optional membership index of recent NMI order/item ids
- [ ] Postal addresses / phones / social (much later)
docs/raw-notes/features/local-search-and-cli-client.md
+39
View File
@@ -0,0 +1,39 @@
# Feature detail: Local search + CLI Redmine client
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** MVP+ (2026-07-14 … 2026-07-15) — ops-usable
**Cookbook:** `docs/cli_power_user_client.md`
## Modules
| Module | Role |
|--------|------|
| `redmine_ticket_query.py` | Property AST (`--where`) |
| `redmine_ticket_view.py` | Markdown + pager |
| `redmine_ticket_browse.py` | fzf → pager → action menu loop |
| `redmine_ticket_filters.py` | Saved filters + symlink queues |
| `redmine_ticket_drafts.py` | Draft queue + host safety |
| `redmine_ticket_meta.py` | Name→id for set-fields |
| `redmine_ticket_ocr.py` | Attachment text extract + subject suggest |
## Done
- [x] Local search / `@saved` filters / queues
- [x] view / walk (`--open-only` on search/browse/walk)
- [x] note vs customer-email vs set-fields (draft/apply)
- [x] `--body-file` + `--to-textile` (pandoc)
- [x] browse **loop** with action menu (queue-only mutations + NMI `m`)
- [x] configs: `bin/rtl`, `bin/rtd`, `--config`
- [x] `ocr-rename` + `@scanned-document`
- [x] `categories list`
## Open
- [ ] Nested config defaults
- [ ] Full curses TUI (optional later)
- [ ] categories create/update
## Non-goals
- Replacing Redmine web UI for everything
- Silent customer email from browse
docs/raw-notes/features/nmi-bridge.md
+41
View File
@@ -0,0 +1,41 @@
# NMI bridge env + usage
**Parent:** [[entity-type-mapper]] · module `redmine_ticket_nmi.py`
## Environment
| Variable | Default | Purpose |
|----------|---------|---------|
| `NMI_CLI` | `/home/jason/projects/mrscClient-php/bin/nmi` | Path to NMI PHP CLI |
| `NMI_ENV_FILE` | `/home/jason/projects/mrscClient-php/examples/modern/.env` | Env file for credentials/endpoint |
| `NMI_MODE` | `dev` | CLI `--mode` (dev/prod as supported by nmi) |
Requires `php` on PATH.
## CLI
```bash
# Pattern extract only
./bin/rtl entities show --issue-id 41394
./bin/rtl entities scan --type tracking_ups --max-tickets 20
# Resolve / cs-brief via NMI CLI
./bin/rtl entities nmi --issue-id 41394
./bin/rtl entities nmi --issue-id 41394 --nmi-mode dev
./bin/rtl entities nmi --issue-id 41394 --as-private-note-draft # if wired; else use browse `m`
```
## Browse
After opening a ticket in `browse`, press **`m`**:
1. Runs NMI brief from extracted entities
2. Shows brief text
3. Confirms → queues **private** internal note draft
Never emails the customer.
## Safety
- NMI is read-side ops context, not customer mail
- Private note drafts still need `draft apply` (+ prod flags) to land in Redmine
docs/raw-notes/features/ocr-rename.md
+46
View File
@@ -0,0 +1,46 @@
# Feature detail: OCR attachment → subject rename
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** MVP 2026-07-15
**Module:** `redmine_ticket_ocr.py` · CLI: `ocr-rename`
## Intent
Scanned docs land in Helpdesk as tickets titled “Scanned document…”.
Ops need: open attachment → understand contents → rename subject (+ optional notes).
## Commands
```bash
# One ticket: download OCR-able attachments, propose subject, queue private note (+ subject draft)
./bin/rtl ocr-rename --issue-id 12345
# Scan cache for generic scan subjects
./bin/rtl ocr-rename --where '@scanned-document' --only-generic-subjects --max-tickets 20
# Apply subject now (LAN/prod safety gates); still queues private OCR note by default
./bin/rtd ocr-rename --issue-id 12345 --apply-subject
# Extract only (no drafts)
./bin/rtl ocr-rename --issue-id 12345 --no-queue-subject --no-queue-note --json
```
## Pipeline
1. Load ticket + attachments (`include=attachments` refresh if needed)
2. Download PDF/image candidates (multi-URL fallback)
3. Extract text: `pdftotext` → else `pdftoppm` + `tesseract`
4. Heuristic `suggest_subject_from_text` (doc type, person/company, date)
5. Queue drafts:
- `fields` subject change (unless `--no-queue-subject`)
- private `internal_note` with OCR snippet (unless `--no-queue-note`)
## Saved filter
`saved-filters/scanned-document.expr` → `@scanned-document`
## Still open
- [ ] Stronger LLM subject summarization (optional; keep heuristics default)
- [ ] Cron wrapper that only processes new generic-scan tickets
- [ ] HR-project default project config example
docs/raw-notes/features/p5-exploration.md
+3
View File
@@ -0,0 +1,3 @@
- [ ] Download voicemails and call recordings from voip.ms automatically
- [ ] helper PHP client library https://git.int.ldrprep.com/dusk/php-voip.ms
docs/raw-notes/features/sync-foundation.md
+91
View File
@@ -0,0 +1,91 @@
# Feature detail: Ticket sync foundation
**Parent roadmap:** [[../features-for-redmine_tickets-sync]]
**Status:** largely implemented (2026-07-14); cron wiring still open
**Primary script:** `redmine_tickets.py`
**Target project (first):** `customer-service`
## Intent
Durable, incremental local mirror of Redmine issues so humans and agents can work offline-first, cron-refresh frequently, and not re-pull full payloads for unchanged tickets.
## Requirements
- Sync tickets updated within a window (e.g. past 30 days) into a **stable** cache path (not timestamped per run).
- Skip full fetch when cached `updated_on` still matches the list stub.
- Support frequent refresh (every few minutes / couple times per hour) using a tight remote window after the initial warm-up.
- Never delete old ticket JSON unless explicitly asked.
- Optional small run-stats ring buffer for cron observability.
## Cache locations
| Environment | Path |
|-------------|------|
| Production (`https://rm.ldrprep.com`) | `/home/jason/projects/redmine/cache/redmine_tickets` |
| Dev / LAN (`http://192.168.50.170`) | `/home/jason/projects/redmine/cache-dev/redmine_tickets` |
Layout:
```text
<cache-dir>/
manifest.json
run-stats.json # optional, with --run-log
<project>/
<issue-id>.json
```
## Server-side vs local filters (sync)
| Flag | Side | Meaning |
|------|------|---------|
| `--status-id` / `--remote-status-id` | **remote** | Redmine API `status_id` (`*`, `open`, `closed`, numeric) |
| `--remote-updated-on` | **remote** | Raw Redmine syntax: `>=2026-06-14`, `>=2014-01-02T08:12:32Z`, `><a\|b` |
| `--remote-updated-since` | **remote** | Convenience → `>=...` (YYYY-MM-DD, ISO, `30d`, `2 days ago`, …) |
| `--remote-updated-before` | **remote** | Convenience → `<=...`; with since → `><since\|before` |
| `--updated-since` / `--updated-before` / `--status` / `--category` | **local** | Post-filters after API list only |
Implementation notes:
- List stubs intentionally omit heavy `include=journals,helpdesk`; full include is only on per-ticket fetch when `should_fetch` is true.
- Progress on by default; `--quiet` for cron.
- Unit tests: `tests/test_redmine_tickets_remote_filters.py`
- Run log: `--run-log` → default `<cache-dir>/run-stats.json`, keep last N (`--run-log-keep`, default 3).
## Recommended commands
```bash
cd /home/jason/projects/redmine
# Initial 30-day window (server-side)
./redmine_tickets.py \
--base-url https://rm.ldrprep.com \
--project customer-service \
--cache-dir /home/jason/projects/redmine/cache/redmine_tickets \
sync \
--remote-status-id '*' \
--remote-updated-since 30d \
--run-log
# Frequent refresh
./redmine_tickets.py \
--base-url https://rm.ldrprep.com \
--project customer-service \
--cache-dir /home/jason/projects/redmine/cache/redmine_tickets \
sync \
--remote-status-id '*' \
--remote-updated-since 2h \
--run-log \
--quiet
```
Raw Redmine form:
```bash
./redmine_tickets.py ... sync --remote-updated-on '>=2026-06-14' --status-id '*'
```
## Open
- [x] Scripts under `cron-scripts/` installed + crontab running (live + dev)
- [ ] **ADJUST LATER (do not forget):** cron still uses `--remote-updated-since 30d` on a few-minute cadence. Fine while validating (~3–4s/run after warm cache) but should tighten to `2h`/`6h` (or marker-based since-last-success) so list pages stay cheap long-term. `run-log-keep 20` is already set.
- [ ] Optional: master storage + symlink organization of JSON (see parent roadmap)
docs/raw-notes/ocr-ticket-attachment-to-rewrite-update-ticket.md
+31
View File
@@ -0,0 +1,31 @@
I want a skill (or set of skills) that accomplishes the following:
## Context
We have found that using networked printers like HPs to scan various documents have them emailed to inboxes wired up to the Redmine Helpdesk plugin a very useful way to avoid losing paperwork and speed up finding it later. That has been instrumental on several occassions, where we've had the printer in my office configured to email scanned files to hr@ldrprep.com, where they get imported as tickets with the scanned document as a file attachment.
Even with that, to make it truly useful, somebody still has to typically do this:
1. Log into redmine and go to the hr project's issues.
2. Identify scanned document emails (usually the subject like says something like "Scanned document...")
3. Open the attachment to see what it is.
4. Edit the ticket and change the subject line so if we search later on, we'll most likely be able to find the ticket with the attached file.
5. Optionally, transcribe key points or info from the document and add them to the ticket.
## Goal: automate this process
- [ ] **Convert target ticket**: When passed a redmine issue id and asked to peform this action, the following should happen:
- Fetch the ticket and check if it has any PDF files attached to it.
- Download the PDFs and use OCR or similar technology to extract the main topics and entities mentioned within the document(s).
- Update the ticket by changing it's subject line to reflect what the attached documents are. If they're mainly about one particular person or company, the subject line should include those.
- [ ] **Scan project**: When asked, search/scan a specified Redmine project to locate tickets where this hasn't been done yet, and then do it for all of the tickets found.
- [ ] **Cron-usable skill**: Once this is done, I want to have it as a skill I could easily add to an agentic cronjob, what will enable it to scan/check a specified list of Redmine projects whenever it runs, and then automatically perform this action.
## Testing parameters
- The test instance of redmine located at `192.168.50.170` can be used to test functionality. It supports the same authentication info as the live redmine.
- The email aspect of this is not important at all. It would be fine to simulate it as follows:
- Create a new ticket simulating what an ideal target would look like
- Upload a pdf to it
- Then, run the relevant workflow against the test ticket to check results