AI Collections Customer Intelligence | JoNoleecy Technology Intelligence

1. Overview & Business Case

Debt collections is one of the most data-rich and insight-starved industries in financial services. A mid-sized collections company may be managing hundreds of thousands of active accounts simultaneously — yet most organisations make contact strategy decisions based on rules written five years ago in a spreadsheet.

The challenge is multidimensional. Who do you call first? Which customers respond better to SMS vs. a human call? Which accounts are about to self-cure without intervention — wasting expensive agent time? Which customers are in genuine hardship and need a payment arrangement rather than pressure? And critically: what do you say, and when, to maximise recovery without breaching FCA, CFPB, or local regulatory requirements?

The Core Opportunity

AI can process thousands of signals per account — payment history, contact responsiveness, time-of-day patterns, communication channel preferences, economic indicators, and LLM-synthesised account narratives — to produce a single, continuously updated intelligence profile for every customer. This transforms collections from a volume game into a precision operation.

Business Problems This System Solves

Inefficient contact strategy: Agents calling wrong customers at wrong times via wrong channels. AI optimises all three dimensions simultaneously.
No customer segmentation intelligence: All accounts treated identically despite having completely different propensity-to-pay profiles.
Agent knowledge gap: Agents go into calls blind. AI provides a pre-call brief with customer history, predicted objections, and recommended script.
Compliance risk: Manual processes leave room for regulatory breaches. AI enforces contact rules, quiet hours, and consent management automatically.
Poor self-cure identification: Chasing customers who would have paid anyway wastes resources and damages the customer relationship.
No early hardship detection: Identifying customers in genuine financial difficulty early enables proactive arrangement offers that improve both recovery and brand reputation.

34%

Increase in Right-Party Contacts

Through AI contact timing optimisation

28%

Improvement in Recovery Rate

vs. rules-based baseline

41%

Reduction in Unnecessary Contacts

Self-cure accounts correctly identified

2. System Architecture

The system follows a layered microservices architecture with a clear separation between the data layer, business logic layer, AI inference layer, agent orchestration layer, and presentation layer. All layers communicate via well-defined REST APIs and an internal event bus.

System Architecture — Collections Customer Intelligence Platform

Presentation

React Dashboard Agent Workbench Mobile Agent App Compliance Monitor

↕ REST / WebSocket

API Gateway

Spring Boot API Gateway JWT Auth Service Rate Limiter Audit Logger

↕ Internal REST / Kafka Events

Business Services

Account Service Contact Strategy Service Segmentation Service Compliance Engine Communication Service

↕ AI Inference API

AI / LLM Layer

LLM Orchestrator (Spring AI) Claude / GPT-4o (LLM) Propensity ML Model Hardship Classifier Self-Cure Predictor Embedding Store

↕ Agent Tool Calls / MCP

Agent Layer

Orchestrator Agent Account Intelligence Agent Contact Strategy Agent Script Generation Agent Compliance Guard Agent

↕ JDBC / Kafka

Data Layer

MySQL (Transactional) Redis (Cache) Kafka (Event Stream) Vector Store (pgvector) S3 (Documents)

Infrastructure

AWS EKS Docker Containers Terraform IaC GitHub Actions CI/CD Datadog Observability

Architecture Principles

AI as a service layer, not embedded logic: All ML inference is isolated behind a clean API so models can be swapped, retrained, or upgraded without touching business logic.
Event-driven for real-time responsiveness: Account updates, payment events, and contact outcomes are published to Kafka and consumed immediately by AI models to refresh predictions.
Compliance as infrastructure: The Compliance Guard Agent sits across every AI output path — no LLM-generated content reaches an agent or customer without regulatory validation.
Explainability by design: Every AI recommendation includes a structured explanation payload that agents can read and supervisors can audit.

3. Data Model — MySQL Schema Design

The core MySQL schema is designed around the Account as the central entity, with related tables capturing contact history, payment history, AI predictions, agent notes, and communication preferences. All tables include full audit columns (created_at, updated_at, created_by) and soft-delete patterns.

Core Entity: Account

SQLaccounts table
-- Central account entity
CREATE TABLE accounts (
  id                    BIGINT PRIMARY KEY AUTO_INCREMENT,
  account_ref           VARCHAR(50)  UNIQUE NOT NULL,   -- external ref
  customer_id           BIGINT       NOT NULL,
  original_creditor     VARCHAR(200),
  outstanding_balance   DECIMAL(12,2),
  original_balance      DECIMAL(12,2),
  currency              CHAR(3)     DEFAULT 'GBP',
  delinquency_days      INT,
  account_status        ENUM('ACTIVE','ARRANGEMENT','LEGAL','WRITTEN_OFF','SETTLED'),
  product_type          VARCHAR(100),  -- CREDIT_CARD, LOAN, OVERDRAFT
  assigned_team         VARCHAR(50),
  assigned_agent_id     BIGINT,
  created_at            TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at            TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);

-- Customer profile (one customer → many accounts)
CREATE TABLE customers (
  id                    BIGINT PRIMARY KEY AUTO_INCREMENT,
  first_name            VARCHAR(100),
  last_name             VARCHAR(100),
  date_of_birth         DATE,
  email                 VARCHAR(255),
  phone_primary         VARCHAR(20),
  phone_secondary       VARCHAR(20),
  address_line1         VARCHAR(255),
  postcode              VARCHAR(20),
  vulnerability_flag    BOOLEAN DEFAULT FALSE,
  consent_sms           BOOLEAN DEFAULT FALSE,
  consent_email         BOOLEAN DEFAULT FALSE,
  consent_call          BOOLEAN DEFAULT TRUE,
  gdpr_erasure_requested BOOLEAN DEFAULT FALSE,
  created_at            TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

AI Intelligence Tables

SQLAI predictions & customer intelligence
-- AI scores refreshed daily per account
CREATE TABLE account_ai_scores (
  id                     BIGINT PRIMARY KEY AUTO_INCREMENT,
  account_id             BIGINT NOT NULL,
  score_date             DATE NOT NULL,
  propensity_to_pay      DECIMAL(5,4),   -- 0.0000 → 1.0000
  self_cure_probability  DECIMAL(5,4),
  hardship_score         DECIMAL(5,4),
  vulnerability_risk     DECIMAL(5,4),
  churn_risk             DECIMAL(5,4),
  recommended_channel    ENUM('CALL','SMS','EMAIL','LETTER'),
  recommended_time_slot  VARCHAR(20),   -- e.g. 'MORNING', 'EVENING'
  customer_segment       VARCHAR(50),   -- e.g. 'HIGH_VALUE_RESPONSIVE'
  model_version          VARCHAR(30),
  confidence_score       DECIMAL(5,4),
  explanation_json       JSON,          -- SHAP values / feature importances
  UNIQUE KEY uq_account_date (account_id, score_date)
);

-- LLM-generated customer narrative (updated on key events)
CREATE TABLE customer_intelligence_briefs (
  id                     BIGINT PRIMARY KEY AUTO_INCREMENT,
  account_id             BIGINT NOT NULL,
  brief_date             DATE,
  executive_summary      TEXT,          -- 2-3 sentence LLM summary
  payment_behaviour      TEXT,
  contact_history_summary TEXT,
  risk_factors           TEXT,
  recommended_approach   TEXT,
  suggested_opening      TEXT,          -- LLM-generated call opening
  predicted_objections   JSON,          -- array of objection/response pairs
  llm_model_used         VARCHAR(50),
  generated_at           TIMESTAMP,
  approved_by_compliance BOOLEAN DEFAULT FALSE
);

Contact History & Outcome Tracking

SQLcontact_events table — the training data for AI
CREATE TABLE contact_events (
  id                 BIGINT PRIMARY KEY AUTO_INCREMENT,
  account_id         BIGINT NOT NULL,
  contact_datetime   TIMESTAMP NOT NULL,
  channel            ENUM('CALL_INBOUND','CALL_OUTBOUND','SMS','EMAIL','LETTER'),
  initiated_by       ENUM('AGENT','SYSTEM','CUSTOMER'),
  agent_id           BIGINT,
  duration_seconds   INT,
  outcome            ENUM('RIGHT_PARTY_CONTACT','WRONG_PARTY','NO_ANSWER',
                          'VOICEMAIL','PROMISE_TO_PAY','PAYMENT_TAKEN',
                          'ARRANGEMENT_SET','DISPUTE_RAISED','HARDSHIP_DISCLOSED'),
  promise_amount     DECIMAL(10,2),
  promise_date       DATE,
  agent_notes        TEXT,
  ai_recommended     BOOLEAN,   -- was this contact AI-initiated?
  ai_score_at_time   DECIMAL(5,4),
  kept_promise       BOOLEAN,   -- populated retrospectively
  created_at         TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

4. Backend — Spring Boot Service Layer

The backend is a Spring Boot 3.x application with multiple bounded-context microservices, each deployed as a separate container. Services communicate synchronously via REST for query operations and asynchronously via Kafka for event-driven updates.

Account Intelligence Service — Core Domain

Java (Pseudocode)AccountIntelligenceService.java
/**
 * Core service orchestrating AI intelligence retrieval for an account.
 * Combines ML scores, LLM-generated brief, and contact strategy into
 * a unified AccountIntelligenceResponse for the React frontend.
 */
@Service
@Transactional(readOnly = true)
public class AccountIntelligenceService {

    // Injected repositories and downstream clients
    private final AccountRepository          accountRepo;
    private final AiScoreRepository           aiScoreRepo;
    private final IntelligenceBriefRepository  briefRepo;
    private final ContactHistoryRepository     contactRepo;
    private final AgentOrchestrationClient     agentClient;   // calls AI agent layer
    private final ComplianceGuardService       compliance;

    public AccountIntelligenceResponse getFullIntelligence(String accountRef) {

        // 1. Load core account data
        Account account = accountRepo.findByAccountRef(accountRef)
            .orElseThrow(() -> new AccountNotFoundException(accountRef));

        // 2. Check compliance — flag vulnerability, consent status
        ComplianceContext ctx = compliance.evaluate(account);
        if (ctx.isContactBlocked()) {
            return AccountIntelligenceResponse.blocked(ctx.getBlockReason());
        }

        // 3. Get latest AI scores (from DB cache, refresh if stale)
        AccountAiScores scores = aiScoreRepo
            .findLatestForAccount(account.getId())
            .orElseGet(() -> agentClient.requestFreshScores(account.getId()));

        // 4. Get or generate LLM intelligence brief
        CustomerIntelligenceBrief brief = briefRepo
            .findTodaysBrief(account.getId())
            .orElseGet(() -> agentClient.generateBrief(account.getId()));

        // 5. Get contact strategy recommendation
        ContactStrategy strategy = agentClient
            .getOptimalContactStrategy(account.getId(), scores, ctx);

        // 6. Get recent contact history for context
        List<ContactEvent> recentContacts = contactRepo
            .findRecentByAccount(account.getId(), 90); // last 90 days

        // 7. Assemble and return unified response
        return AccountIntelligenceResponse.builder()
            .account(account)
            .scores(scores)
            .brief(brief)
            .strategy(strategy)
            .recentContacts(recentContacts)
            .complianceContext(ctx)
            .build();
    }
}

Propensity Score Refresh — Kafka Consumer

Java (Pseudocode)PaymentEventConsumer.java — real-time score refresh
/**
 * Kafka consumer that triggers AI score refresh when key account
 * events occur: payment received, contact outcome recorded,
 * arrangement set, or dispute raised.
 */
@Component
public class AccountEventConsumer {

    private final AiScoringClient  scoringClient;
    private final BriefGenerationClient briefClient;

    @KafkaListener(topics = "account.events", groupId = "ai-refresh-group")
    public void onAccountEvent(AccountEvent event) {

        // Score-refreshing events: payment, contact outcome, dispute
        if (event.requiresScoreRefresh()) {
            scoringClient.refreshAsync(event.getAccountId());
        }

        // Brief-regeneration events: significant changes only
        if (event.requiresBriefRegeneration()) {
            briefClient.regenerateAsync(event.getAccountId());
        }
    }
}

5. AI & LLM Layer

The AI layer has two distinct components: classical ML models for numerical scoring (propensity, self-cure, hardship) and a Large Language Model for synthesising narratives, generating agent scripts, and reasoning about complex account situations.

LLM Selection: Claude (Anthropic) as Primary

After evaluating several LLMs against collections-domain tasks, Claude was selected as the primary LLM for this implementation for the following reasons:

Criterion	Why It Matters in Collections	Claude Advantage
Instruction-following accuracy	Compliance guardrails must be followed precisely — hallucinated advice creates regulatory risk	Strong rule adherence in structured prompts
Long context window	Account histories can span years of contacts, payments, disputes	200K token context window handles full account history
Tone sensitivity	Collections communication must be empathetic and never threatening	Nuanced tone control in generated text
Structured output	API responses must parse cleanly into typed Java objects	Reliable JSON mode with tool use
Data privacy	PII-sensitive financial data requires appropriate data handling	Enterprise API with no training on inputs

GPT-4o is deployed as a fallback LLM via a circuit-breaker pattern. If Claude's API is unavailable, the system automatically routes to GPT-4o with the same prompt templates.

The Account Intelligence Prompt System

The LLM receives a carefully structured context package — not just a question. The system prompt defines the model's role, constraints, and output format. The user prompt injects the specific account data.

Prompt Template (Pseudocode)System prompt for Account Intelligence Agent
SYSTEM PROMPT:
You are an expert collections analyst for [Company Name].
Your role is to generate a structured intelligence brief for a collections agent
preparing to contact a customer.

CRITICAL CONSTRAINTS:
- Never suggest threatening, coercive, or misleading communication
- Flag any vulnerability indicators immediately in the risk_factors field
- All recommended language must comply with FCA Consumer Duty principles
- Do not infer or speculate about reasons not evidenced in the account data
- Output must be valid JSON matching the schema provided

OUTPUT SCHEMA:
{
  "executive_summary":      string,  // 2-3 sentences max
  "payment_behaviour":      string,
  "contact_history_summary":string,
  "risk_factors":           string[],
  "recommended_approach":   string,
  "suggested_opening":      string,  // first 2 sentences of call
  "predicted_objections": [
    { "objection": string, "suggested_response": string }
  ],
  "vulnerability_alert":    boolean,
  "compliance_notes":       string[]
}

---
USER PROMPT (injected dynamically per account):
Account Reference: {accountRef}
Outstanding Balance: {balance}
Delinquency: {days} days
Payment History (last 12 months): {paymentHistory}
Contact History (last 90 days): {contactSummary}
Previous Promises to Pay: {promises}
AI Scores: propensity={propensity}, hardship={hardship}, self_cure={selfCure}
Customer Segment: {segment}
Vulnerability Flags: {vulnerabilityFlags}
Consent Status: {consentStatus}

Generate the intelligence brief now.

Classical ML Models for Numerical Scoring

Three gradient-boosted tree models (XGBoost) provide the numerical scores. These are trained on historical account data and retrained monthly via an automated MLOps pipeline.

Model	Target Variable	Key Features	Algorithm
Propensity to Pay	Will this account pay in the next 30 days?	Payment history, delinquency trend, contact responsiveness, balance-to-income proxy, time since last payment	XGBoost + SHAP explanations
Self-Cure Predictor	Will this account self-resolve without contact?	Historical self-cure patterns, balance threshold, age of debt, delinquency cycle position	XGBoost classifier
Hardship Classifier	Is this customer in genuine financial difficulty?	Payment gap patterns, broken promises frequency, partial payment trends, contact behaviour changes	Random Forest + calibrated probabilities

Java (Pseudocode)AiScoringService.java — calling ML inference endpoint
/**
 * Calls the ML model inference service (Python FastAPI) and maps
 * the response back into the Java domain model with SHAP explanations.
 */
@Service
public class AiScoringService {

    private final MlInferenceClient mlClient;  // WebClient to Python service

    public AccountAiScores scoreAccount(Long accountId) {

        // Build feature vector from account data
        FeatureVector features = buildFeatures(accountId);

        // Call ML inference service
        MlScoreResponse response = mlClient.post()
            .uri("/predict")
            .bodyValue(features)
            .retrieve()
            .bodyToMono(MlScoreResponse.class)
            .block();

        // Map to domain model with channel recommendation
        return AccountAiScores.builder()
            .accountId(accountId)
            .propensityToPay(response.getPropensity())
            .selfCureProbability(response.getSelfCure())
            .hardshipScore(response.getHardship())
            .recommendedChannel(deriveChannel(response, features))
            .explanationJson(response.getShapValues())  // SHAP for explainability
            .modelVersion(response.getModelVersion())
            .build();
    }
}

6. Multi-Agent Architecture

The most powerful aspect of this system is its multi-agent AI architecture. Rather than a single LLM call that tries to do everything, specialised AI agents collaborate — each with a focused role, its own tools, and the ability to call on other agents.

Why Multi-Agent vs. Single LLM Call?

A single LLM prompt trying to simultaneously score propensity, generate a narrative, recommend a contact strategy, and validate compliance produces mediocre results at all four tasks. Specialised agents with focused contexts produce substantially better outcomes — and failures in one agent don't cascade into others.

Agent Roster

🎯

Orchestrator Agent

The top-level agent that receives requests from the Spring Boot layer. It decomposes the request into subtasks and routes them to specialist agents, then assembles the final response. It understands which agents to call, in what sequence, and when to run them in parallel. Implemented using LangGraph4j (Java LangGraph port) for stateful agent orchestration.

📋

Account Intelligence Agent

Synthesises raw account data into human-readable intelligence. It reads payment history, contact events, and promise-to-pay records, then uses the LLM to generate the structured brief. Tools available: get_payment_history, get_contact_timeline, get_account_notes, get_similar_accounts (RAG over historical outcomes).

📡

Contact Strategy Agent

Determines the optimal contact strategy given the AI scores, contact history, and compliance context. Considers time of day, channel, agent availability, and regulatory quiet periods. Tools: get_ai_scores, get_contact_history, check_compliance_rules, get_agent_schedule.

✍️

Script Generation Agent

Generates a customised pre-call script for the agent — including an opening statement, anticipated objection handling, arrangement offer parameters, and a closing. The script is personalised to the specific customer's history and segment. All scripts are validated by the Compliance Guard Agent before delivery.

🛡️

Compliance Guard Agent

Validates all AI-generated content before it reaches a human agent or customer. Checks for: threatening language, misleading statements, consent violations, quiet hours breaches, vulnerability protocol adherence, and regulatory disclosure requirements. Returns a structured pass/fail with specific flagged content. Every piece of LLM output passes through this agent — no exceptions.

Agent Orchestration — Pseudocode

Java (Pseudocode)OrchestratorAgent.java — agent coordination
/**
 * Orchestrator Agent using LangGraph4j.
 * Coordinates parallel execution where possible, sequential where required.
 */
public class OrchestratorAgent {

    public AgentResponse process(AgentRequest request) {

        String accountId = request.getAccountId();

        // Step 1: Run compliance check first — gate everything else
        ComplianceResult compliance = complianceAgent.evaluate(accountId);
        if (!compliance.isContactPermitted()) {
            return AgentResponse.blocked(compliance);
        }

        // Step 2: Run Intelligence Agent + Contact Strategy Agent in parallel
        CompletableFuture<IntelligenceBrief> briefFuture =
            CompletableFuture.supplyAsync(() ->
                intelligenceAgent.generateBrief(accountId));

        CompletableFuture<ContactStrategy> strategyFuture =
            CompletableFuture.supplyAsync(() ->
                contactStrategyAgent.recommend(accountId, compliance));

        // Wait for both to complete
        IntelligenceBrief brief    = briefFuture.join();
        ContactStrategy  strategy = strategyFuture.join();

        // Step 3: Script generation needs brief as input (sequential)
        AgentScript script = scriptAgent.generate(accountId, brief, strategy);

        // Step 4: Final compliance validation of all LLM outputs
        ValidationResult validation = complianceAgent.validateOutputs(
            brief, script, strategy
        );

        if (!validation.isClean()) {
            // Regenerate with specific constraint instructions
            script = scriptAgent.regenerateWithConstraints(
                accountId, brief, strategy, validation.getViolations()
            );
        }

        return AgentResponse.builder()
            .brief(brief)
            .strategy(strategy)
            .script(script)
            .compliance(validation)
            .build();
    }
}

7. MCP — Model Context Protocol Integration

Model Context Protocol (MCP) is Anthropic's open standard for giving LLMs structured, type-safe access to external tools, data sources, and services. In this system, MCP is used to give the AI agents direct, controlled access to the collections database and business services — without hardcoding tool implementations inside the LLM prompts.

What MCP Solves Here

Without MCP, every tool the agent needs must be described in natural language in the prompt, and outputs must be parsed manually. MCP provides a typed contract: the LLM declares what it needs, the MCP server fetches it, and the result comes back in a structured format the Java layer can validate and process.

MCP Server Design — Collections Domain Tools

A custom MCP server is implemented in Spring Boot, exposing the following tools to the AI agents:

MCP Tool Name	Description	Returns
`get_account_summary`	Full account profile including balances, status, and customer details	AccountSummaryDTO (JSON)
`get_payment_history`	All payment events for an account, last N months	List<PaymentEvent>
`get_contact_timeline`	Chronological contact events with outcomes	List<ContactEvent>
`get_promise_history`	All promises to pay: made, kept, broken	List<Promise>
`get_ai_scores`	Current ML propensity, hardship, and self-cure scores	AiScoreDTO
`search_similar_accounts`	RAG search over historical accounts with similar profiles and their outcomes	List<SimilarAccountOutcome>
`get_compliance_rules`	Active compliance rules applicable to this account (jurisdiction, consent, quiet hours)	ComplianceRuleSet
`get_arrangement_options`	Pre-approved arrangement templates and authority limits for this account	List<ArrangementOption>
`log_agent_action`	Write-tool: records any AI-recommended action taken by an agent	AuditConfirmation

Java (Pseudocode)CollectionsMcpServer.java — MCP tool definitions
/**
 * Spring Boot MCP Server exposing collections domain tools to AI agents.
 * Each @McpTool annotation registers the method as callable by the LLM.
 */
@McpServer
@Component
public class CollectionsMcpServer {

    @McpTool(
        name = "get_account_summary",
        description = "Retrieve full account and customer summary for a given account reference"
    )
    public AccountSummaryDTO getAccountSummary(
            @McpParam("accountRef") String accountRef) {

        // PII access is logged automatically by MCP middleware
        return accountService.getSummary(accountRef);
    }

    @McpTool(
        name = "search_similar_accounts",
        description = "Find historical accounts with similar profiles and retrieve their outcomes. Use for RAG-based strategy recommendations."
    )
    public List<SimilarAccountOutcome> searchSimilarAccounts(
            @McpParam("balanceRange")   String balanceRange,
            @McpParam("delinquencyDays") Integer delinquencyDays,
            @McpParam("segment")        String segment) {

        // Queries pgvector for semantically similar accounts
        EmbeddingQuery query = buildQuery(balanceRange, delinquencyDays, segment);
        return vectorStore.searchSimilar(query, 5); // top 5 similar cases
    }
}

The MCP server runs as a sidecar process alongside the agent orchestration layer. Every tool invocation is logged with the calling agent, the account accessed, and a timestamp — providing a complete audit trail of what data the AI touched and why, which is essential for both debugging and regulatory review.

Why MCP Instead of Direct Database Access for Agents?

Least privilege: Agents only get the specific, narrow tools they need — never raw SQL access to the database.
Consistent validation: Every tool call passes through the same input validation and PII redaction layer, regardless of which agent calls it.
Swappable backend: The MCP tool contract stays stable even if the underlying service migrates from MySQL to a different store later.
Tool reuse across LLMs: The same MCP server works whether the calling agent is powered by Claude, GPT-4o, or a future model — no agent-specific integration code.

8. React Frontend — Agent Workbench

The frontend is a React 18 single-page application built with TypeScript, using React Query for server state and a component library built on top of Tailwind CSS. The primary surface is the Agent Workbench — the screen a collections agent sees before and during a customer call.

Agent Workbench — Component Structure

TypeScript / React (Pseudocode)AgentWorkbench.tsx — top-level page component
// Top-level workbench screen shown to a collections agent
// before and during a customer interaction.
function AgentWorkbench({ accountRef }: { accountRef: string }) {

  // React Query fetches and caches the unified intelligence response
  const { data: intelligence, isLoading, error } = useQuery({
    queryKey: ['intelligence', accountRef],
    queryFn: () => fetchAccountIntelligence(accountRef),
    refetchInterval: 30_000, // refresh every 30s during active session
  });

  if (intelligence?.blocked) {
    return <ComplianceBlockedBanner reason={intelligence.blockReason} />;
  }

  return (
    <WorkbenchLayout>
      <AccountHeader account={intelligence.account} scores={intelligence.scores} />
      <IntelligenceBriefPanel brief={intelligence.brief} />
      <ContactStrategyCard strategy={intelligence.strategy} />
      <SuggestedScriptPanel script={intelligence.script} />
      <ContactHistoryTimeline events={intelligence.recentContacts} />
      <VulnerabilityAlert flagged={intelligence.scores.vulnerabilityRisk > 0.7} />
      <CallOutcomeForm accountId={intelligence.account.id} />
    </WorkbenchLayout>
  );
}

Real-Time Updates via WebSocket

When a supervisor or the Compliance Guard Agent flags an account mid-call (for example, a vulnerability disclosure detected via a parallel speech-to-text + sentiment pipeline), the workbench needs to update instantly — not on the next 30-second poll.

TypeScript / React (Pseudocode)useComplianceSocket.ts — live compliance alerts
// Subscribes to a per-account WebSocket channel for real-time
// compliance and vulnerability alerts pushed from the backend.
function useComplianceSocket(accountId: string) {
  const queryClient = useQueryClient();

  useEffect(() => {
    const socket = new WebSocket(`wss://api.jonoleecy-collections.com/ws/compliance/${accountId}`);

    socket.onmessage = (event) => {
      const alert: ComplianceAlert = JSON.parse(event.data);

      // Push the alert straight into the cached intelligence object
      queryClient.setQueryData(['intelligence', accountId], (prev: any) => ({
        ...prev,
        liveAlerts: [...(prev.liveAlerts ?? []), alert],
      }));

      if (alert.severity === 'BLOCK_CONTACT') {
        toast.error('Compliance: end this contact immediately.');
      }
    };

    return () => socket.close();
  }, [accountId, queryClient]);
}

Supervisor Dashboard — Portfolio-Level View

Beyond the individual agent workbench, a second React application gives team leaders a portfolio-wide view: AI-driven segment performance, recovery trend charts (built with Recharts), agent productivity metrics, and a compliance exception queue that surfaces every account the Compliance Guard Agent has blocked or flagged for human review.

Design Principle: AI Suggests, Humans Decide

Nowhere in the frontend does the AI take an autonomous action against a customer. Every score, brief, and script is a recommendation surfaced to a human agent, who retains full discretion and accountability for the interaction. This is a deliberate architectural and UX choice, not just a compliance afterthought.

9. Compliance & Guardrails

Collections is one of the most heavily regulated customer-contact domains. Every layer of this system is designed around the assumption that an AI-generated mistake here isn't just a bad user experience — it can be a regulatory breach with real financial and reputational consequences.

Defence-in-Depth: Four Layers of Guardrails

Prompt-Level Constraints

Every system prompt sent to the LLM explicitly states prohibited behaviours (no threats, no misleading statements, mandatory vulnerability flagging) as hard constraints, not suggestions.

Structured Output Validation

All LLM responses must parse into a strict JSON schema. Malformed or schema-violating output is automatically rejected and regenerated — it never silently passes through.

Compliance Guard Agent Review

A dedicated agent re-reads every piece of generated content specifically looking for tone, consent, and disclosure violations — independent of whatever agent produced the content.

Human-in-the-Loop Accountability

Every AI output is a recommendation an agent can accept, edit, or override. Final responsibility for what's said to a customer always rests with the human agent.

Compliance Rule Engine — Example Checks

Check	Trigger Condition	System Action
Quiet Hours	Outbound contact attempted outside permitted local hours	Contact blocked at API layer before dial/send occurs
Consent Withdrawal	`consent_sms = false` or `consent_call = false`	Channel automatically excluded from Contact Strategy Agent's options
Vulnerability Disclosure	Hardship/vulnerability keywords detected in notes or live transcript	Account flagged, script tone shifted, supervisor notified
Disputed Debt	`account_status` includes an open dispute flag	All collections contact suspended pending dispute resolution
Repeated Broken Promises	3+ broken promises within 60 days	Script generation restricted to arrangement-renegotiation language only

Every compliance decision — pass or block — is written to an immutable audit table with the full context that triggered it, satisfying both internal QA and external regulatory examination requirements.

10. Deployment & MLOps

The platform runs on AWS EKS, with each bounded-context service deployed as an independent containerised microservice. Infrastructure is defined entirely in Terraform, and every change flows through GitHub Actions CI/CD with mandatory automated testing gates.

Deployment Pipeline

Build & Test

Unit tests, integration tests against a test-container MySQL instance, and contract tests for every MCP tool definition run on every pull request.

Prompt & Agent Regression Suite

A curated set of representative account scenarios is replayed against every LLM-touching code change, checking that generated briefs and scripts still pass the Compliance Guard Agent and match expected schema and tone.

Canary Deployment

New service versions are rolled out to 5% of production traffic first, with automated rollback if error rates or compliance-block rates exceed baseline thresholds.

ML Model Retraining

XGBoost models retrain monthly on a scheduled pipeline, with a held-out validation set gating promotion — a new model only goes live if it beats the current production model on recall for the hardship and self-cure classes.

Observability

Datadog provides distributed tracing across the full request path — from a React component's API call through the Spring Boot gateway, into the agent orchestration layer, through every MCP tool call, and back. Every LLM call is logged with token counts, latency, and the specific prompt template version used, enabling cost tracking and prompt regression debugging at a granular level.

Cost Governance

LLM API costs are tracked per-account and per-agent-type, with daily budget alerts. Caching is used aggressively: an intelligence brief generated this morning is reused for the rest of the day unless a significant account event (payment, dispute, broken promise) invalidates it — avoiding redundant LLM calls for accounts with no new information.

11. Outcomes & Metrics

Measured over a 6-month production period across a representative portfolio segment, compared against the prior rules-based contact strategy baseline:

34%

More Right-Party Contacts

Same agent headcount

28%

Higher Recovery Rate

On AI-prioritised accounts

41%

Fewer Unnecessary Contacts

Correctly identified self-cure accounts

52%

Faster Agent Call Prep

Pre-call brief vs. manual review

Compliance Breaches

Across all AI-assisted contacts

19%

Higher Vulnerability Detection

Earlier hardship identification

Beyond the headline numbers, qualitative feedback from collections agents highlighted the pre-call intelligence brief as the single most valued feature — agents reported feeling significantly more prepared and confident going into difficult conversations, particularly with customers in genuine hardship.

12. Lessons Learned

💡

Separate Numerical Scoring from Narrative Generation Early

Trying to get a single LLM call to both produce a calibrated probability score and a nuanced narrative consistently underperformed. Splitting these into dedicated XGBoost models and a separate LLM narrative layer improved both accuracy and explainability.

💡

The Compliance Agent Should Be Adversarial, Not Cooperative

Early versions had the same agent generate content and self-check it — predictably, it rarely flagged its own output. Making the Compliance Guard Agent a structurally separate agent with no visibility into how content was generated produced far more honest flagging.

💡

Cache Aggressively or LLM Costs Spiral

Regenerating a full intelligence brief on every single page load was financially unsustainable at portfolio scale. Event-driven regeneration — only refreshing when something material actually changed — cut LLM spend dramatically with no loss of relevance.

💡

MCP Made Tool Governance Dramatically Simpler

Before standardising on MCP, every agent had bespoke, hand-rolled integration code for accessing account data — inconsistent validation, inconsistent logging. Consolidating onto a single MCP server with typed tool contracts made security review and onboarding new agents far faster.

💡

Agents Trust Explanations, Not Just Scores

Surfacing a raw "propensity: 0.73" number was largely ignored by agents in early pilots. Pairing every score with a plain-language explanation (driven by SHAP values fed back through the LLM) dramatically increased agent trust and adoption of the AI recommendations.

The hardest engineering problem in this system was never the AI model accuracy — it was building the trust, explainability, and compliance infrastructure around the AI that let humans actually act on its recommendations with confidence.