redmine/AGENT.md

AGENT.md

This file is a restart brief for future agent sessions on this repository.

Project Identity

This repository supports a legacy, business-critical Redmine 3.4.4 installation. The real purpose is to make CRM/helpdesk communication data searchable and automatable without a risky near-term platform migration.

The LAN copy used for testing is:

http://192.168.50.170/

This is not a greenfield app. Treat it as a local-fork-and-integration project around an existing Redmine install and old RedmineUP plugins.

Tracked local plugin source lives in plugins/. The full redmine-copy/ tree is ignored and should be treated as a working/reference copy, not the source of truth for local plugin changes.

Original Motivation

The business stores most external communication inside Redmine through:

• redmine_contacts
• redmine_contacts_helpdesk

The default UI/API is not good enough for:

• efficient contact search
• light contact maintenance
• helpdesk message search
• customer communication timeline lookup
• future semantic/vector search

The key realization from prior work:

Helpdesk-created issues may have Anonymous issue authors.
The real customer identity lives in helpdesk_tickets, journal_messages, and contacts.

Never design the search/index layer as if issues.author were enough.

High-Level Architecture Decision

The intended architecture is:

1. Redmine writes local outbox rows.
2. External worker reads those rows.
3. Worker enriches from read-only MySQL joins.
4. Worker builds ticket/message documents.
5. External index stores/searches those documents.

Safety rule:

External failures must not break Redmine saves.

That is why the event boundary is local DB outbox, not request-time webhooks or embedding calls.

Redmine And Plugin Baseline

• Redmine version: 3.4.4
• Old RedmineUP plugin versions in use:
  • redmine_contacts 4.1.2 PRO
  • redmine_contacts_helpdesk 3.0.9 PRO
• Near-term upgrade to newer Redmine/RedmineUP is not the current goal.
• Treat these plugins as local legacy code when necessary.

Repository Landmarks

Top-level docs:

• README.md
• docs/event_outbox_spec.md
• docs/redmineup_local_fork_changelog.md
• docs/pre_existing_issues.md

Main scripts:

• redmine_contacts.py
• redmine_helpdesk_search.py
• redmine_outbox_worker.py

Local Redmine copy:

• redmine-copy

Important local plugin paths:

• redmine-copy/plugins/redmine_event_outbox
• redmine-copy/plugins/redmine_contacts_helpdesk

What Has Already Been Done

Contact CLI

redmine_contacts.py exists and supports:

• contact fetch to local cache
• fuzzy-ish contact search
• light contact updates
• helpdesk read API calls:
  • ticket by issue
  • issues by contact
  • messages by issue
  • contact timeline

Event Outbox Plugin

redmine_event_outbox exists and the known-good archive is:

• dist/redmine_event_outbox-0.0.1-known-good-20260421T143957Z.tar.gz

Known-good tested LAN events:

• issue.created
• issue.updated
• journal.created
• contact.created
• contact.updated

Local code also adds optional helpdesk events:

• helpdesk_ticket.created
• helpdesk_ticket.updated
• journal_message.created
• journal_message.updated

These helpdesk event paths are implemented locally but should still be treated as needing fuller end-to-end validation.

Helpdesk Read API In Local Plugin Fork

The local redmine_contacts_helpdesk fork includes:

• helpdesk_search_controller.rb
• routes under /helpdesk_search/*
• alias/usage routes to avoid noisy routing errors

This was deployed to the LAN copy and route-loaded successfully.

Helpdesk Export/Search CLI

redmine_helpdesk_search.py was created to prove the data model and export path. It:

• SSHes to the LAN Redmine host
• reads remote MySQL credentials from config/database.yml
• runs read-only MySQL queries
• exports ticket/message docs to local JSONL cache
• provides rough local search/timeline/issues-by-contact commands

Important:

This script is a diagnostic/export tool, not the final search architecture.

Do not get dragged into treating local CLI search speed as the core mission.

External Outbox Worker Prototype

redmine_outbox_worker.py now exists as the first worker/indexer boundary. It:

• reads and optionally claims pending event_outbox_events over SSH/MySQL
• supports --dry-run for non-mutating previews
• uses locked_at, locked_by, processed_at, attempts, and last_error
• enriches helpdesk ticket/message/contact events with read-only joins
• writes derived JSONL to /tmp/redmine-outbox/derived_documents.jsonl
• marks rows processed only after a successful local write

This is still a prototype output target, not the final vector index.

LAN Host Facts

These were verified previously:

• SSH host/user: reddev@192.168.50.170
• SSH key used previously: /tmp/reddev
• remote Redmine path: /usr/share/redmine
• remote Ruby: 2.5.1
• remote Bundler: 1.17.3
• remote DB: MySQL 5.7
• remote plugin rollback archives were stored in:
  • /home/reddev/redmine-plugin-backups/

If using those exact credentials again, verify they still exist before relying on them.

Remote Plugin Changes Previously Deployed

Changes were copied to the LAN Redmine host for:

• plugins/redmine_event_outbox/...
• plugins/redmine_contacts_helpdesk/...

Passenger was restarted with:

touch /usr/share/redmine/tmp/restart.txt

Routes were verified by remote rake routes.

Fork Hygiene Rules

Before editing RedmineUP plugin code:

1. create a rollback archive in dist/
2. record the change in:
   • docs/redmineup_local_fork_changelog.md
   • plugin-local changelog if appropriate
3. keep edits scoped
4. prefer read-only APIs/endpoints first
5. validate on the LAN copy before claiming the change is done

Existing rollback archives:

• dist/redmine_contacts-4.1.2-local-before-helpdesk-search-20260421T215548Z.tar.gz
• dist/redmine_contacts_helpdesk-3.0.9-local-before-helpdesk-search-20260421T215548Z.tar.gz

Important Pre-Existing Issues

Read:

• docs/pre_existing_issues.md

Especially remember:

1. duplicate id="avatar" is likely a real long-standing UI bug
2. attachments/contacts_thumbnail digest warning is probably log noise
3. acts_as_list warning is an upgrade-compatibility note, not a current blocker

Do not confuse those with the main search deliverable unless they directly block the work in front of you.

Current Strategic State

The project is past the "can we get data out?" phase.

We now know:

• contact JSON access exists
• helpdesk/customer identity can be extracted
• outbox events can be recorded safely
• helpdesk read API routes can be added locally
• full ticket/message export is feasible

The most useful next step is:

Build the external worker/indexer pipeline.

Not:

• polishing a local CLI into a full search product
• building a big Redmine admin UI
• chasing unrelated legacy plugin cleanup

Recommended Next Steps For A Future Session

1. Re-read:
   • README.md
   • docs/event_outbox_spec.md
   • docs/redmineup_local_fork_changelog.md
   • docs/pre_existing_issues.md
2. Verify local and remote state:
   • plugin files present
   • LAN host reachable
   • remote routes still load
3. Validate redmine_outbox_worker.py end to end against the LAN copy
4. Refine read-only joins that enrich:
   • helpdesk_tickets
   • journal_messages
   • contacts
   • issues
   • journals
5. Confirm deterministic ticket/message/contact JSONL output
6. Only after that, connect Qdrant and embeddings

Query/Data Model Notes

Authoritative helpdesk context lives in:

• helpdesk_tickets
  • issue_id
  • contact_id
  • from_address
  • to_address
  • cc_address
  • message_id
  • ticket_date
  • source
• journal_messages
  • journal_id
  • contact_id
  • from_address
  • to_address
  • cc_address
  • bcc_address
  • message_id
  • is_incoming
  • message_date
• contacts
• issues
• journals

The worker/indexer should prefer direct DB joins as the source of truth for this layer.

What To Be Careful About

• Do not assume Redmine core issue API data is enough for helpdesk search.
• Do not treat old RedmineUP code as untouchable vendor code.
• Do not claim a plugin change is validated just because ruby -c passes.
• Do not burn time optimizing local CLI search if the real goal is external indexing.
• Do not revert unrelated repo changes; the Redmine tree may be dirty.

If You Need A One-Paragraph Summary

This project exists to safely extract and search real CRM/helpdesk communication history from a legacy Redmine 3.4.4 install. We already have a contact CLI, a local event outbox plugin, local helpdesk read API routes, and a rough helpdesk-export CLI. The next meaningful milestone is an external worker that consumes outbox rows, enriches from helpdesk/contact MySQL tables, and builds deterministic ticket/message documents for a future Qdrant/OpenAI-backed search index.