By patrick

Royal Mail Email Pipeline

By patrick · Created Mar 9, 2026 public

How Royal Mail emails flow from inbox to claim resolution. Covers the full journey: email polling, LLM classification, Temporal workflow signaling, email parsing, and status transitions.

End-to-End Flow

graph TB
    subgraph Inbox["Microsoft 365 Inbox"]
        EMAIL[Incoming Email<br/>from Royal Mail]
    end

    subgraph Poller["Email Poller (every 15 min)"]
        FETCH[Fetch Unread Emails]
        DEDUP_CHECK{Dedup Check<br/>MSGraphId in DB?}
        LLM[LLM Classification<br/>Category + Confidence]
        TRACK[Extract Tracking<br/>LLM priority, regex fallback]
        RESOLVE[Resolve Claim<br/>TrackingNumber or CourierClaimRef]
        SIGNAL[Signal Temporal Workflow]
        DEDUP_SAVE[Save Dedup Record<br/>AIEmailCategorisationDecisions]
        EARLY_RETURN[Return Early<br/>Workflow owns the rest]
    end

    subgraph Temporal["Temporal Signal Service"]
        POLICY{AllowsEmail<br/>CreatedWorkflows?}
        SWS[SignalWithStart<br/>Create workflow if needed]
        SIG[Signal Only<br/>Workflow must exist]
    end

    subgraph Workflow["RoyalMailClaimWorkflow (per-claim)"]
        PHASE[Determine Starting Phase<br/>RoyalMailPhaseMapper]
        P1[Phase 1: Tracking Enrichment]
        P2[Phase 2: Validation<br/>13-day eligibility]
        P3[Phase 3: Web Form Submission]
        P4[Phase 4: Wait for Emails<br/>50-day timeout]
        QUEUE[Email Signal Queue]
        PROCESS[Process Email]
    end

    subgraph Activity["RoyalMailEmailResponseActivity"]
        PARSE[Parse Email HTML Table]
        IDEMP{Already in<br/>final state?}
        DETERMINE[Determine Email Type<br/>Credited / Rejected / Found]
        TRANSITION[Transition Claim Status]
        MOVE[Mark Read + Move Email]
    end

    EMAIL --> FETCH
    FETCH --> DEDUP_CHECK
    DEDUP_CHECK -->|Already processed| EARLY_RETURN
    DEDUP_CHECK -->|New email| LLM
    LLM --> TRACK
    TRACK --> RESOLVE
    RESOLVE --> SIGNAL
    SIGNAL --> DEDUP_SAVE
    DEDUP_SAVE --> EARLY_RETURN

    SIGNAL --> POLICY
    POLICY -->|Royal Mail: Yes| SWS
    POLICY -->|Other couriers: No| SIG

    SWS --> QUEUE
    SIG --> QUEUE

    PHASE --> P1
    P1 --> P2
    P2 --> P3
    P3 --> P4
    P4 --> QUEUE

    QUEUE --> PROCESS
    PROCESS --> PARSE
    PARSE --> IDEMP
    IDEMP -->|Yes, skip| MOVE
    IDEMP -->|No| DETERMINE
    DETERMINE --> TRANSITION
    TRANSITION --> MOVE
    MOVE -->|Stay open 50 days| P4

Stage 1: Email Poller

The email poller runs on a Temporal schedule every 15 minutes. For each unread email, it runs ProcessEmailInternalAsync in EmailPoller.cs.

Dedup Check

Before any LLM work, the poller checks if this email was already processed:

// EmailPoller.cs:429
var recentlyProcessed = await _db.AIEmailCategorisationDecisions
    .AnyAsync(d => d.MSGraphId == msg.Id
        && d.CreatedAt > DateTime.UtcNow.AddHours(-4), token);

If a record exists within 4 hours, the email is skipped entirely. This prevents burning LLM tokens on emails that couldn't be matched to a claim last time.

LLM Classification

Every email goes through LLM classification regardless of courier:

// EmailPoller.cs:489
llmPrediction = await _llmClassifier.ClassifyEmailAsync(
    fromAddress, toAddress, subject, body,
    courierDisplayName, guidancePrompt, ...)

The LLM returns:

PredictedCategory (e.g. "Credit", "Rejection", "Claim acknowledged")
Confidence score
TrackingNumbers extracted from the email body
MonetaryAmount if a credit is detected

Tracking Number Resolution

Tracking numbers are resolved in priority order:

graph LR
    subgraph Extraction["Number Extraction"]
        LLM_NUMS[LLM Tracking Numbers]
        REGEX_NUMS[Regex Tracking Numbers]
    end

    subgraph Resolution["Claim Resolution"]
        TRY_LLM[Try LLM numbers first]
        TRY_REGEX[Fall back to regex]
        MATCH_TRACK[Match by TrackingNumber<br/>fuzzy, 2-char tolerance]
        MATCH_REF[Match by CourierClaimReference<br/>exact match]
    end

    LLM_NUMS --> TRY_LLM
    REGEX_NUMS --> TRY_REGEX
    TRY_LLM --> MATCH_TRACK
    TRY_LLM --> MATCH_REF
    TRY_REGEX --> MATCH_TRACK
    TRY_REGEX --> MATCH_REF

For each tracking number, FindClaimByTrackingOrCourierRefAsync tries:

Exact match on Claim.TrackingNumber
Fuzzy match (up to 2 characters difference)
Exact match on Claim.CourierClaimReference

Royal Mail emails often contain the CourierClaimReference (e.g. "10640468") rather than the tracking number, so the reference match is critical.

Workflow Signal

If a claim is matched, the poller signals the Temporal workflow:

// EmailPoller.cs:869
await _operations.SignalTemporalWorkflowAsync(
    temporalClaim.Id, matchedTracking, courierDisplayName,
    courier.CourierId, client.Name,
    msg.Id, fromAddress, toAddress, subject, body,
    folderPath, msg.ReceivedDateTime?.UtcDateTime ?? DateTime.UtcNow,
    $"Email received for claim {temporalClaim.Id}");

Workflow-Handled Early Return

Because Royal Mail has EmailHandledByWorkflow = true, the poller returns early after signaling. It does NOT run any category handlers (credit, rejection, etc.) or mark the email as read. The workflow owns all of that.

Before returning, a dedup record is saved directly to the database:

// EmailPoller.cs:896 — uses _db.SaveChangesAsync, NOT _operations.SaveChangesAsync
// Must persist even in DryRun to prevent re-LLM'ing
_db.AIEmailCategorisationDecisions.Add(new AIEmailCategorisationDecisions
{
    CourierId = courierId.Value,
    MSGraphId = msg.Id,
    Confidence = (float)activeConfidence,
    InformationJson = JsonConvert.SerializeObject(new
    {
        TopPredictions = topPredictions?.Select(p => new { p.Label, p.Score }),
        WorkflowHandled = true  // Dashboard visibility flag
    }),
    ClaimId = matchedClaimId,
});
await _db.SaveChangesAsync(token);

The WorkflowHandled = true flag in InformationJson lets the dashboard distinguish workflow-processed emails from poller-processed ones.

Stage 2: Temporal Signal Service

TemporalSignalService.SignalEmailReceivedAsync() decides how to deliver the email signal to the workflow.

SignalWithStart vs Signal-Only

graph TB
    subgraph Decision["TemporalSignalService"]
        CHECK{CourierRegistry<br/>AllowsEmailCreatedWorkflows?}
    end

    subgraph SignalWithStart["SignalWithStart Path"]
        SWS_DESC[Create workflow if it<br/>does not exist, then signal]
        SWS_NEW[New workflow created<br/>IsBackfill=true<br/>ReconstructedState=null]
        SWS_EXIST[Existing workflow<br/>Signal delivered normally]
    end

    subgraph SignalOnly["Signal-Only Path"]
        SIG_DESC[Signal existing workflow]
        SIG_OK[Workflow found<br/>Signal delivered]
        SIG_FAIL[WorkflowNotFoundException<br/>Log WARNING, email dropped]
    end

    CHECK -->|"Royal Mail: true<br/>Asendia: true"| SWS_DESC
    CHECK -->|"DPD: false<br/>UPS: false"| SIG_DESC
    SWS_DESC --> SWS_NEW
    SWS_DESC --> SWS_EXIST
    SIG_DESC --> SIG_OK
    SIG_DESC --> SIG_FAIL

Royal Mail uses SignalWithStart (AllowsEmailCreatedWorkflows = true). This means:

If the claim already has a running workflow (created during submission or backfill), the email signal is delivered normally
If no workflow exists (pre-migration claims that were submitted before Temporal), a new workflow is created on-the-fly with the email as its first signal

Email-Created Workflow Safety

When SignalWithStart creates a workflow, TemporalSignalService sets IsBackfill = true but ReconstructedState = null (no DB lookup). The RoyalMailPhaseMapper handles this safely:

// RoyalMailPhaseMapper.cs:57
if (!claimStatusId.HasValue)
{
    // Email-created workflow — claim was already submitted pre-Temporal
    return WorkflowPhase.WaitingForEmail;
}

This skips all submission phases and goes straight to waiting for emails, which is correct because the claim was already submitted through the old system.

Stage 3: Royal Mail Claim Workflow

RoyalMailClaimWorkflow is a per-claim Temporal workflow that manages the entire lifecycle.

Phase Determination

stateDiagram-v2
    [*] --> PhaseMapper: DetermineStartingPhase()

    state PhaseMapper {
        state "Is Backfill?" as backfill_check
        state "Has StatusId?" as status_check

        backfill_check --> TrackingEnrichment: No (new workflow)
        backfill_check --> status_check: Yes
        status_check --> WaitingForEmail: No (email-created)
        status_check --> MapStatus: Yes
    }

    state MapStatus {
        state "New / Pending / MoreInfo" as pre_sub
        state "SubmittedSent / SubmittedAck / Accepted" as post_sub
        state "Credited / Rejected / Found" as terminal

        pre_sub --> TrackingEnrichment
        post_sub --> WaitingForEmail
        terminal --> Error: Should not be backfilled
    }

    state TrackingEnrichment
    state Validation
    state Submission
    state WaitingForEmail

    TrackingEnrichment --> Validation
    Validation --> Submission
    Submission --> WaitingForEmail

Scenario	Starting Phase	Why
New claim (submitted through Temporal)	TrackingEnrichment	Full lifecycle
Backfill, status = SubmittedAck	WaitingForEmail	Already submitted, just wait for response
Backfill, status = New	TrackingEnrichment	Needs enrichment + submission
Email-created (SignalWithStart, no status)	WaitingForEmail	Pre-Temporal claim, already submitted
Backfill, status = Credited	ERROR	Terminal state, should not be backfilled

Four Workflow Phases

Phase 1 - Tracking Enrichment: Query Royal Mail API for tracking data. Populates service code, delivery status, and timestamps.

Phase 2 - Validation: Check the 13-day eligibility window. If not yet eligible, the workflow sleeps until the eligible date (using Temporal durable timers).

Phase 3 - Submission: Submit the claim through the Royal Mail web form (5-page automated form fill). Includes retry logic for transient failures.

Phase 4 - Wait for Emails: The core of email processing. Runs a loop waiting for email signals.

Email Waiting Loop

sequenceDiagram
    participant WF as RoyalMailClaimWorkflow
    participant Q as Email Signal Queue
    participant ACT as RoyalMailEmailResponseActivity
    participant MOVE as MarkAsReadAndMoveActivity
    participant DB as Database

    loop Every email signal (up to 50 days)
        Note over WF: WaitConditionAsync<br/>(email in queue OR timeout)

        Q->>WF: Email signal received
        WF->>ACT: Parse email + determine type
        ACT->>DB: Lookup claim status
        ACT-->>WF: EmailType + Outcome + CreditAmount

        alt Credited
            WF->>DB: Transition to Credited
        else Rejected
            WF->>DB: Transition to Rejected
        else Found / Returned to Sender
            WF->>DB: Transition to Found
        else Claimed Too Soon
            WF->>DB: Transition to Pending
            WF->>WF: Retry submission (Phase 3)
        else Duplicate Claim
            Note over WF: Ignore, keep waiting<br/>(original claim stays valid)
        end

        WF->>MOVE: Mark as read + move to client folder

        Note over WF: Record conclusion but<br/>stay open for duplicates
    end

    Note over WF: 50-day timeout reached<br/>Workflow completes

The workflow stays open after processing a conclusion email. This handles duplicate emails from Royal Mail (which are common) — subsequent conclusion emails for the same claim are processed idempotently and moved to the correct folder.

Timeout Behavior

// RoyalMailClaimWorkflow.cs — patched timeout
var emailTimeout = Workflow.Patched("rm-keep-open-after-conclusion")
    ? TimeSpan.FromDays(50)    // New behavior
    : TimeSpan.FromDays(365 * 5);  // Legacy (5 years)

After 50 days with no new emails, the workflow completes. The Workflow.Patched() mechanism ensures existing running workflows keep their original timeout, while newly created workflows get the 50-day timeout.

Stage 4: Email Response Activity

RoyalMailEmailResponseActivity parses the email HTML and determines the outcome.

Email Parsing

Royal Mail conclusion emails have a structured HTML table with 7 cells:

Cell	Content
1	Customer Reference
2	Tracking Number
3	RMG Reference
4	Resolution
5	Amount Claimed
6	Amount Credited (empty if rejected)
7	Rejection Reason (empty if credited)

Idempotency Check

Before processing, the activity checks if the claim is already in a final state:

var finalStates = new[]
{
    ClaimStatuses.DatabaseIds.Credited,        // 6
    ClaimStatuses.DatabaseIds.RejectedAssumed, // 8
    ClaimStatuses.DatabaseIds.Rejected,        // 9
    ClaimStatuses.DatabaseIds.Found            // 12
};

if (finalStates.Contains(claim.ClaimStatusId))
    return success with "idempotent check";

This prevents duplicate status transitions when the same conclusion email is signaled multiple times.

Email Type Detection

graph TB
    subgraph Parse["Parse Email Table"]
        HAS_CREDIT{AmountCredited<br/>present?}
        HAS_REASON{RejectionReason<br/>present?}
    end

    subgraph Types["Email Types"]
        CREDITED[Credited<br/>Extract amount]
        FOUND[Found / Returned to Sender]
        CUSTOMS[Held in Customs]
        TOO_SOON[Claimed Too Soon<br/>Retry later]
        DUPLICATE[Duplicate Claim<br/>Ignore safely]
        REJECTED[Standard Rejection<br/>Extract reason ID]
        UNKNOWN[Indeterminate<br/>Error: cannot determine type]
    end

    subgraph Outcomes["Claim Outcomes"]
        O_CREDIT[Submitted -> Credited]
        O_REJECT[Submitted -> Rejected]
        O_FOUND[Submitted -> Found]
        O_PENDING[Submitted -> Pending<br/>then retry submission]
        O_IGNORE[No status change<br/>Keep workflow running]
    end

    HAS_CREDIT -->|Yes| CREDITED
    HAS_CREDIT -->|No| HAS_REASON
    HAS_REASON -->|"Item Returned to Sender"| FOUND
    HAS_REASON -->|"Held by overseas customs"| CUSTOMS
    HAS_REASON -->|"Claimed too soon"| TOO_SOON
    HAS_REASON -->|"Duplicate Claim"| DUPLICATE
    HAS_REASON -->|Other reason| REJECTED
    HAS_REASON -->|No| UNKNOWN

    CREDITED --> O_CREDIT
    FOUND --> O_FOUND
    CUSTOMS --> O_FOUND
    TOO_SOON --> O_PENDING
    DUPLICATE --> O_IGNORE
    REJECTED --> O_REJECT

Duplicate Claim is a special case. Royal Mail sends this when a claim was submitted twice. The activity does NOT reject the original claim — it returns Outcome = Submitted so the workflow keeps waiting for the real conclusion.

Three Layers of Dedup

The system prevents the same email from being processed multiple times at three independent layers:

graph TB
    subgraph Layer1["Layer 1: Database (4h sliding window)"]
        L1[AIEmailCategorisationDecisions<br/>MSGraphId + 4h window check<br/>Prevents re-LLM after poller completes]
    end

    subgraph Layer2["Layer 2: Temporal Workflow ID"]
        L2[ProcessEmailWorkflow<br/>SHA256 deterministic ID from MSGraphId<br/>Prevents concurrent processing]
    end

    subgraph Layer3["Layer 3: Dedup Record (permanent)"]
        L3[Saved before early return<br/>WorkflowHandled = true<br/>Prevents re-LLM after workflow completes]
    end

    subgraph Layer4["Layer 4: Activity Idempotency"]
        L4[RoyalMailEmailResponseActivity<br/>Final state check<br/>Prevents duplicate status transitions]
    end

    EMAIL[Incoming Email] --> Layer1
    Layer1 -->|New email| Layer2
    Layer2 -->|Not concurrent| Layer3
    Layer3 --> Layer4

Layer	Scope	Mechanism	Prevents
DB 4h window	Email poller entry	`AnyAsync(MSGraphId, CreatedAt > -4h)`	Re-LLM'ing recently processed emails
Temporal workflow ID	Per-email workflow	SHA256 of MSGraphId	Two pollers processing same email concurrently
Dedup record save	Workflow-handled return	`_db.SaveChangesAsync()` (bypasses DryRun)	Re-LLM'ing after workflow completes
Activity idempotency	Conclusion processing	Final state check in activity	Duplicate status transitions

CourierRegistry Configuration

Royal Mail's behavior is defined entirely in code (not config):

// CourierRegistry.cs
new CourierDefinition
{
    Name = "Royal Mail",
    WorkflowType = WorkflowType.RoyalMail,
    WorkflowName = "RoyalMailClaimWorkflow",
    Enabled = true,
    SupportsPerClaimWorkflows = true,
    AllowsEmailCreatedWorkflows = true,   // SignalWithStart
    AllowsPortalCreatedWorkflows = false,
    EmailHandledByWorkflow = true,        // Poller returns early
}

Flag	Value	Effect
`EmailHandledByWorkflow`	`true`	Poller skips all category handlers, returns after signaling
`AllowsEmailCreatedWorkflows`	`true`	SignalWithStart creates workflow if none exists
`SupportsPerClaimWorkflows`	`true`	One workflow per claim (not batch)
`AllowsPortalCreatedWorkflows`	`false`	Workflows not created from portal actions

Key Files

Component	File	Entry Point
Email Poller	`AI.ModelRunner/EmailPoller.cs`	`ProcessEmailInternalAsync()`
Signal Service	`AI.ModelRunner/Services/TemporalSignalService.cs`	`SignalEmailReceivedAsync()`
Courier Registry	`Claimit.Temporal.Contracts/Routing/CourierRegistry.cs`	`GetCourier()`
Phase Mapper	`EmailProcessing.Workflows/Couriers/RoyalMail/Helpers/RoyalMailPhaseMapper.cs`	`DetermineStartingPhase()`
Claim Workflow	`EmailProcessing.Workflows/Couriers/RoyalMail/RoyalMailClaimWorkflow.cs`	`RunAsync()`
Email Response Activity	`EmailProcessing.Activities/Couriers/RoyalMail/RoyalMailEmailResponseActivity.cs`	`ExecuteAsync()`
Process Email Workflow	`EmailProcessing.Workflows/EmailPoller/ProcessEmailWorkflow.cs`	`RunAsync()`