redmine/docs/cli_power_user_client.md

# Power-user Redmine client — usage (`redmine_tickets.py`)

Scriptable local client for **humans and agents**: durable ticket cache, property search, Markdown view, and interactive walk.

| Env | Redmine | Cache |
|-----|---------|--------|
| **Prod** | `https://rm.ldrprep.com` | `/home/jason/projects/redmine/cache/redmine_tickets` |
| **Dev / mutation testing** | LAN (e.g. `192.168.50.170` / `retired.int.ldrprep.com`) | `/home/jason/projects/redmine/cache-dev/redmine_tickets` |

**Safety:** treat prod cache as read/sync. Test any **writes** on LAN first. Mutations impersonate `rebot` when implemented.

Related:

- Roadmap index: [raw-notes/features-for-redmine_tickets-sync.md](raw-notes/features-for-redmine_tickets-sync.md)
- Pre-MVP search snapshot: [2026-07-14_localsearch_before_sanity.md](2026-07-14_localsearch_before_sanity.md)
- Sync plan: [ticket_sync_tool_plan.md](ticket_sync_tool_plan.md)
- Cron scripts: `cron-scripts/sync-live-customer-service.sh`, `cron-scripts/sync-dev-customer-service.sh`

---

## Setup

```bash
cd /home/jason/projects/redmine
export REDMINE_API_KEY=…   # required for sync / live API only
CACHE=/home/jason/projects/redmine/cache/redmine_tickets
PROJECT=customer-service
```

Shorthand used below:

```bash
RT="./redmine_tickets.py --project $PROJECT --cache-dir $CACHE"
```

---

## Sync (fill / refresh cache)

Remote filters hit Redmine; local cache is updated only for tickets whose `updated_on` changed.

```bash
# Initial / catch-up window
$RT sync --remote-status-id '*' --remote-updated-since 30d --run-log

# Frequent refresh (preferred after warm cache)
$RT sync --remote-status-id '*' --remote-updated-since 2h --run-log --quiet
```

| Flag | Meaning |
|------|---------|
| `--remote-status-id` / `--status-id` | Server: `*`, `open`, `closed`, or numeric |
| `--remote-updated-since` | Server: `30d`, `2h`, `YYYY-MM-DD`, … → `updated_on >= …` |
| `--remote-updated-on` | Server: raw Redmine operator string |
| `--run-log` | Ring-buffer stats → `$CACHE/run-stats.json` |
| `--run-log-keep N` | Keep last N runs (default 3; cron uses 20) |
| `--quiet` | Less progress noise |

**Do not** use local `--updated-since` as a substitute for remote filters on large projects (local only post-filters stubs already listed).

Cron note: live scripts currently still use `30d` on a short interval (~3–4s after warm-up). **Tighten later** to `2h`/`6h`.

---

## Search (local cache only)

No API key required. Only sees tickets already on disk.

## Search table columns

Default table (tab-separated):

```text
#id  updated_on  status  assignee  contact  category  control_tags  subject
```

Contact comes from `helpdesk_ticket.contact.email` (or name/company/from_address).

### Classic filters

```bash
$RT search --status New --max-tickets 50
$RT search --updated-since 2026-07-01 --order newest-updated-first
```

### Property queries (`--where`)

Engine: `redmine_ticket_query.py`. Repeatable `--where` = **AND**.

```bash
$RT search --where 'subject.includes("account")' --max-tickets 20

$RT search --where 'assigned_to IS NULL' --max-tickets 10

$RT search \
  --where 'helpdesk_ticket.contact.email == helpdesk_ticket.contact.company' \
  --ids-only

$RT search \
  --where '(subject.includes("verification code") OR subject.includes("magic link")) AND assigned_to IS NULL'
```

| Construct | Example |
|-----------|---------|
| Equality | `status.name == "New"` |
| Inequality | `assigned_to != "Jason T."` |
| Null | `assigned_to IS NULL` / `IS NOT NULL` |
| Substring | `subject.includes("rate sheet")` |
| Relative time | `updated_on.since("5 days")` |
| Path == path | `helpdesk_ticket.contact.email == helpdesk_ticket.contact.company` |
| Boolean | `AND` `OR` `&&` `||` + `()` |

Nested dicts compared to strings use `.name` (or email/id) when present.

### Output modes

| Flag | Output |
|------|--------|
| (default) | Table: id, updated_on, status, category, subject |
| `--json` | Full issue JSON array |
| `--jsonl` | One full issue JSON per line |
| `--ids-only` | Issue ids only (best for scripts) |

Match count goes to **stderr** on table mode so stdout stays pipe-clean for `--ids-only` / jsonl.

### Automation pattern

```bash
$RT search --where 'status.name == "New"' --where 'assigned_to IS NULL' --ids-only \
  | while read -r id; do
      $RT view "$id" --print > "/tmp/ticket-$id.md"
    done
```

---

## Saved filters (`saved-filters/`)

Put one expression per file (comments with `#` allowed):

```text
saved-filters/unassigned.expr
saved-filters/helpdeskEmailMatchCompany.expr
saved-filters/triage-needed-5d.expr
```

Reference them from CLI:

```bash
$RT search --where '@unassigned' --max-tickets 20
$RT search --where 'alias.helpdeskEmailMatchCompany' --ids-only
$RT walk --where '@triage-needed-5d' --max-tickets 10 --pager cat
```

Bare names that match a file also work: `--where unassigned`.

### Short configs / wrappers

```bash
./bin/rtl search --where 'assigned_to IS NULL' --max-tickets 10   # prod CS
./bin/rtd note --issue-id 39305 --body 'LAN' --apply              # dev CS
./redmine_tickets.py --config configs/live.json ...
```

`--allow-prod-writes` is a **parent** flag (before the subcommand):

```bash
./bin/rtl --allow-prod-writes draft apply <draft_id>
```

---

## Named queue folders (`materialize-queues`)

Config: [queues.example.json](../queues.example.json)

```bash
$RT materialize-queues --config queues.example.json
# writes: $CACHE/queues/<name>/*.json → symlinks into $CACHE/$PROJECT/<id>.json
```

Agents that only understand files can list `$CACHE/queues/unassigned/`.

Dry-run:

```bash
$RT materialize-queues --config queues.example.json --dry-run
```

Refresh after cron sync if you want queues kept current (optional second cron step).

---

## View (single ticket → Markdown)

Cache-first. Does not call Redmine unless you add that later.

```bash
$RT view 41934 --print
$RT view --issue-id 41934 --pager auto    # glow → bat → batcat → cat
$RT view 41934 --write-md --pager cat
```

| Flag | Meaning |
|------|---------|
| `--print` | Markdown to stdout (no pager) |
| `--pager auto\|glow\|bat\|batcat\|cat` | How to display when not `--print` |
| `--write-md` | Force write under markdown cache |
| `--force-md` | Regen even if `updated_on` stamp matches |

Markdown cache path:

```text
$CACHE/$PROJECT/markdown/<id>.md
$CACHE/$PROJECT/markdown/<id>.md.updated_on
```

Regenerates when ticket `updated_on` changes (stamp file).

---

## Walk (interactive feed)

```bash
# From property search
$RT walk --where 'assigned_to IS NULL' --max-tickets 10 --pager cat

# Explicit ids
$RT walk --ids 41934,41899 --pager cat
```

Controls: `n` / Enter = next, `p` = previous, `q` = quit, `r` = redraw.

Same filter flags as search (`--where`, `--status`, `--max-tickets`, …). `--ids` overrides search selection.

---

## Internal note vs customer email (critical)

These are **different Redmine/Helpdesk operations**. The CLI uses **separate commands** so they are hard to confuse.

| | `note` | `customer-email` |
|--|--------|------------------|
| **What it is** | Internal journal note on the issue | Helpdesk “send note” / customer reply |
| **Emails customer?** | **No** | **Yes** (when applied) |
| **API** | `PUT /issues/:id.json` with `notes` (+ optional `private_notes`) | `POST /helpdesk/email_note.json` with `message.content` |
| **Private?** | Optional `--private` | **Never** (private cannot email) |
| **Confirm flag** | None needed | **Required:** `--i-understand-this-emails-the-customer` |
| **Default** | Queue local draft | Queue local draft |
| **Apply now** | `--apply` | `--apply` (+ confirm flag already required) |
| **Prod writes** | Need `--allow-prod-writes` | Need `--allow-prod-writes` |
| **Impersonation** | `--impersonate-user rebot` (default) on apply | same |

### Queue an internal note (safe default — no email)

```bash
$RT note --issue-id 41934 --body 'Staff-only follow-up' --private
# later:
$RT draft apply <draft-id>   # still does NOT email
```

### Apply internal note immediately (LAN/test preferred)

```bash
$RT --base-url http://192.168.50.170 note --issue-id 123 \
  --body 'LAN test note' --apply
```

### Queue a customer email (explicit, scary name + confirm flag)

```bash
$RT customer-email --issue-id 41934 \
  --body 'Thanks for writing — we received your request.' \
  --i-understand-this-emails-the-customer
# later:
$RT draft apply <draft-id> --i-understand-this-emails-the-customer
```

### Send customer email immediately

```bash
$RT --base-url http://192.168.50.170 customer-email --issue-id 123 \
  --body 'LAN test customer reply' \
  --i-understand-this-emails-the-customer \
  --apply
```

### Production writes

Writes to hosts outside the LAN allow-list (e.g. `https://rm.ldrprep.com`) are **refused** unless you pass `--allow-prod-writes`. Prefer testing on LAN first.

### Draft list

```bash
$RT draft list
# columns include EMAILS_CUSTOMER vs no-email
```

**Do not** use `note` when you mean to email a customer. **Do not** use `customer-email` for staff chatter.

---

## set-fields (status / assignee / priority / category / due date)

Does **not** email the customer. Default queues a draft; `--apply` writes (LAN-safe by default).

```bash
# By numeric id
$RT set-fields --issue-id 41934 --status-id 5 --assigned-to-id 0

# By human name (uses meta cache; run meta-refresh once)
$RT meta-refresh --force
$RT set-fields --issue-id 41934 --status Closed --assigned-to unassigned \
  --notes 'Unassign + close' --private
```

| Flag | Meaning |
|------|---------|
| `--status` / `--status-id` | name or numeric status |
| `--assigned-to` / `--assigned-to-id` | name/login or id; `unassigned` / `0` clears |
| `--priority` / `--priority-id` | name or id |
| `--category` / `--category-id` | name or id; `none` / `0` clears |
| `--due-date` | `YYYY-MM-DD` |
| `--subject` | replace subject |
| `--notes` / `--private` | optional internal journal note with the update |
| `--refresh-meta` | force name→id cache refresh |

Meta cache: `$CACHE/meta/<project>/`

---

## Contacts (local mirror + light mutations)

Separate from full `redmine_contacts.py` CRM CLI on purpose (different cache + surface), but **workflows can hand off**.

```bash
# Mirror / search / show
$RT contacts-sync --max-contacts 100
$RT contacts-search --query bavaria --max-contacts 20
$RT contact-show --contact-id 1890

# Ticket → contact bridge (starts on ticket, ends on contact tools)
$RT ticket-contact --issue-id 41934
$RT ticket-contact --issue-id 41934 --show --fetch

# Contact mutation (draft/apply, same safety as tickets; does NOT email)
$RT contact-set --contact-id 4515 --add-tag maybe-customer
$RT --allow-prod-writes draft apply <id>          # if draft was for prod
# LAN apply:
$RT --base-url http://192.168.50.170 contact-set --contact-id 2284 \
  --add-tag maybe-customer --apply --refresh-cache
```

| Piece | Path |
|-------|------|
| Local mirror | `$CACHE/contacts/<id>.json` |
| Full CRM (batch/loops/custom fields) | `redmine_contacts.py` |
| Ticket workbench contacts | `contacts-*`, `contact-set`, `ticket-contact` |

Store: `$CACHE/contacts/<id>.json` + `manifest.json`.

---

## Local control tags (not Redmine tags)

```bash
$RT control-tag add --tag bavaria --issue-id 41934 --property subject --annotation '…'
$RT control-tag list
$RT search --control-tag bavaria
$RT control-tag suggest --tag account --where 'subject.includes("account")' --max-tickets 20
$RT control-tag suggest --tag account --where 'subject.includes("account")' --apply
$RT control-tag wipe --i-understand-this-wipes-local-control-tags
```

Store: `$CACHE/control-tags/tags.json`

---

## Attachments

```bash
$RT attachments list --issue-id 39305 --refresh
$RT attachments download --issue-id 39305 --all
$RT attachments upload --issue-id 39305 --file ./doc.pdf          # dry-run
$RT --base-url http://192.168.50.170 attachments upload \
  --issue-id 39305 --file ./doc.pdf --apply --notes 'attached scan'
```

Default download dir: `$CACHE/$PROJECT/attachments/<issue_id>/`.  
Does **not** email the customer. Prod write needs `--allow-prod-writes`.

---

## Entities (tracking / order patterns)

```bash
$RT entities show --issue-id 41394
$RT entities scan --where 'subject.includes("1Z")' --max-tickets 20
$RT entities scan --type tracking_ups --ids-only
```

Heuristic only — not NMI-backed. Also appears in `view` Markdown.

---

## Browse (interactive loop: list → pager → actions)

```bash
# Default: fzf (preview pane) → glow/bat pager → action menu → back to fzf
./bin/rtl browse --where 'assigned_to IS NULL' --max-tickets 80 --open-only

# Explicit pager / no preview
./bin/rtl browse --where '@unassigned' --pager glow
./bin/rtl browse --status New --no-preview

# One-shot (pick once, page, exit — no menu/loop)
./bin/rtl browse --where 'subject.includes("invoice")' --once
```

### Loop keys

| Step | What happens |
|------|----------------|
| **List** | fzf list (Esc quits). Header lists actions. Preview shows ticket snippet. |
| **Enter** | Open ticket Markdown in glow/bat/batcat (`--pager`) |
| **After pager** | Action menu for that ticket |

### Action menu

| Key | Action |
|-----|--------|
| Enter / `v` | View again |
| `n` | Queue **internal note** (no email) |
| `p` | Queue **private** internal note |
| `f` | Queue **field update** (status/assignee by name) |
| `t` | Add **local control-tag** |
| `c` | Show ticket→contact handoff |
| `e` | Extract entities (patterns) |
| `m` | **NMI brief** → confirm → queue as **private** note draft |
| `i` | Print issue id |
| `l` | Back to list |
| `q` | Quit browse |

Mutations from browse **queue drafts only** (loud `draft_id=` banner). Apply later with `draft apply`.  
Customer email is **not** in the menu (use CLI deliberately).

```bash
# Notes from file (+ optional Textile via pandoc)
./bin/rtl note --issue-id 41934 --body-file ./reply.md --private
./bin/rtl note --issue-id 41934 --body-file ./reply.md --to-textile --private

# Open tickets only (search/browse)
./bin/rtl search --open-only --where 'assigned_to IS NULL' --max-tickets 20
```

Non-interactive (no TTY): prints list lines and exits.

---

## OCR rename (scanned attachments → better subject)

```bash
# One ticket
./bin/rtl ocr-rename --issue-id 12345

# Scan for generic "Scanned document…" tickets
./bin/rtl ocr-rename --where '@scanned-document' --only-generic-subjects --max-tickets 15

# Apply subject on LAN immediately (note still queued private by default)
./bin/rtd ocr-rename --issue-id 12345 --apply-subject
```

Uses `pdftotext`, else `pdftoppm`+`tesseract`. Queues subject `fields` draft + private OCR note (disable with `--no-queue-subject` / `--no-queue-note`).

---

## Categories

```bash
./bin/rtl categories list
./bin/rtl categories list --refresh
```

---

## Entities + NMI

```bash
./bin/rtl entities show --issue-id 41394
./bin/rtl entities nmi --issue-id 41394
# Env: NMI_CLI, NMI_ENV_FILE, NMI_MODE — see docs/raw-notes/features/nmi-bridge.md
# Browse action `m` queues NMI brief as private note draft
```

---

## Local drafts (queue + apply)

```bash
$RT draft list
$RT draft cancel <id>
$RT draft cancel --all-pending
$RT draft apply <id>   # internal notes
$RT draft apply <id> --i-understand-this-emails-the-customer  # customer_email only
```

Stored under `$CACHE/drafts/*.json`.

---

## Other subcommands (ops)

| Command | Role |
|---------|------|
| `spam-check` / `close-if-spam` | Deterministic spam tooling (LAN for apply) |
| `known-spam-scan` / `known-spam-close` | Profile-based spam |
| `tag-maybe-customer` | Contact tagging from inquiry evidence |
| `decompose` | Dry-run vault-oriented plan for one ticket |

---

## Modules

| File | Role |
|------|------|
| `redmine_tickets.py` | CLI entry: sync, search, view, walk, spam, … |
| `redmine_ticket_query.py` | Property expression compiler |
| `redmine_ticket_view.py` | Markdown render + pager |
| `redmine_contacts.py` | Shared Redmine client / contacts |

Tests: `tests/test_redmine_ticket_query_and_view.py`, `tests/test_redmine_tickets_remote_filters.py`.

---

## What is **not** here yet

- Queued notes / helpdesk replies / field mutations (P2 remaining)
- Free-text semantic search (Qdrant / semantic_index — separate track)
- SQLite projection layer (planned architecture, not shipped)

**Shipped for queues/filters:** saved-filters + `materialize-queues` (see above).

---

## Quick “is the client alive?” check

```bash
$RT search --max-tickets 3
$RT search --where 'assigned_to IS NULL' --max-tickets 3 --ids-only
ID=$($RT search --max-tickets 1 --ids-only | head -1)
$RT view "$ID" --print | head -30
```

---

## TUI (curses keyboard-driven batch processor)  [NEW]

Fast single-screen TUI for the exact workflow you described: queue a template of changes, scroll/cycle tickets, paint the template onto many tickets with one key, then one button to write everything.

```bash
./bin/rtl tui --where 'assigned_to IS NULL' --max-tickets 60 --open-only
./bin/rtl tui --control-tag triage-needed --max-tickets 30
```

**Workflow in the TUI:**
1. **Set template once** (optional fields; menus or type+filter):
   - `c` : Category picker (from meta cache; type to filter)
   - `a` : Assignee picker (unassigned=0)
   - `s` : Status picker
   - `p` : Priority (type a number, e.g. 2; hotkeys 1/2/3 also work)
   - `P` : Project ID (for moving the ticket)
   - `t` : Local control tag (string)
   - `C` : Add contact tag (repeatable; creates if new)
   - `n` : Private note (applies to painted tickets)
   - `e` / `m` : Toggle entity-extraction / NMI-brief flags (become control tags + notes)

2. **Navigate the list**:
   - `j` / `k` / arrows : move
   - `/` : filter the visible list (live)
   - `r` : redraw

3. **Paint / apply**:
   - `SPACE` or `Enter` on a ticket: **apply the current template** to it (records per-ticket pending: fields, note, tags, toggles). Shows marker [FNTEM] etc.

4. **Commit**:
   - `A` : Queue drafts for all painted tickets (safe default).
   - (If you passed `--apply` and have `--allow-prod-writes` on LAN/prod, `S` for direct apply — use with care.)

Other keys: `x` clear pending on current, `R` force meta refresh, `q` quit (prompts if pending), `?` help.

**What gets written on commit:**
- Field updates (status/assignee/priority/category/project) via KIND_FIELDS draft (or direct).
- Private note (per ticket or from template).
- Local control tags (via control-tag add).
- Contact tags (via contact_fields drafts for the helpdesk contact).
- Special toggles → control tags ("entity-extract") + placeholder NMI note if needed.

All the safety rules (drafts vs apply, prod gate, impersonate rebot) still apply. The TUI just removes the per-ticket menu tax.

**Tips:**
- Run `meta-refresh --force` once for full picker lists.
- Combine with saved filters: `./bin/rtl tui --where '@unassigned'`
- The TUI loads from the same cache/search logic as browse/walk.


## TUI Commit behavior update (auto-apply on A)

As of latest change:

- Pressing **A** in the TUI now:
  1. Shows a **single** "Are you SURE?" confirmation for the whole batch.
  2. Queues the drafts for every painted ticket.
  3. Immediately applies those drafts (using the normal apply_draft_to_redmine + mark_applied path).

This removes the need to manually run `draft apply <id>` for each ticket.

`S` remains available for the old direct-apply (bypass draft) path if you ever want it.

All normal safety (LAN/prod gate, --allow-prod-writes, impersonate rebot, no customer emails from this path) remains in force.

See the module docstring in redmine_ticket_tui.py or run the TUI and press `?` for current keys.


## NMI Enrichment Improvements (entities nmi)

The `entities nmi` command (and the NMI path used by browse 'm' and TUI) now:

- Automatically generates rich deep links when it sees item numbers, tracking, request ids, order ids, accounts:
  - Item Manager + Item Statement for item_no
  - Box History (boxFinder) for tracking numbers
  - Order summary and package/request history pages
  - User Info for account numbers
  (See also the reference note: [[INBOX/WIP/Linking to Entities Within NMI]] )

- When an item_no is resolved, it calls the new `getItemHistory` API endpoint via `nmi item history --item-no=... --snapshot`:
  - Returns merged timeline of product_history (moves, condition changes), service_fees (billings), and comments.
  - The private note / cs-brief output now includes a "=== NMI Deep Links ===" section and a "=== Item History (getItemHistory) ===" section (truncated for notes; full in --format json).

Usage:
```
rtl entities nmi --issue-id 123456 --format note
rtl entities nmi --issue-id 123456 --format json   # full history payload
rtl entities nmi --issue-id 123456 --no-history
```

The same enrichment logic feeds the TUI and browse NMI quick action.


---

## Post-apply cache refresh (anti-staleness)

After any mutation that hits live Redmine (TUI `A`, `draft apply`, `set-fields --apply`, direct notes, etc.) the affected ticket's JSON is immediately re-fetched and overwritten in the local cache.

This means:
- Next `rtl browse`, `rtl tui`, or search sees the new status / assignee / journals / updated_on right away.
- No need to wait for the next full sync for the tickets you just touched.

In the TUI:
- `r` → local redraw (fast)
- `R` → force live refresh of the **currently highlighted** ticket (pulls fresh from Redmine + updates cache + view)

This directly supports the rule: "update the existing cached file when the ticket has changed" and "agents update the ticket when their work is done."


### TUI "S" key (direct apply)

`S` is the **direct/bypass** path (still requires write permission).

- It skips the draft queue for field changes and does an immediate `update_issue`.
- We now also call the same live cache refresh that `A` uses, so the TUI and later `browse` see the change.
- Behavior difference from `A`:
  - No "are you sure?" batch confirmation.
  - No draft records created for the main fields.
  - Still gated by `--allow-prod-writes` (or `--apply`).

Current recommended path for most batch work remains **A** (one confirm, goes through drafts for auditability, then auto-applies).

### TUI safety / UX improvements (2026-07)

- If you start without `--allow-prod-writes`, a clear **READ-ONLY** banner appears near the top.
- Bottom status/help line is now wrapped to avoid `_curses.error` on narrow/short terminals (common when ssh'ing from Windows Terminal, Tabby, or split panes).
- After confirming `A`, the screen immediately shows "COMMIT + APPLY IN PROGRESS" feedback so it doesn't look hung while the writes happen.
- `S` now shows "fields updated DIRECTLY" + cache sync messages when it actually performs a live write.