LLM Bias Detection and Enforcement in Production AI Systems

Deploying large language models in regulated decision-making contexts — credit underwriting, employment screening, healthcare triage, insurance pricing — creates bias risks that do not exist in traditional statistical models. LLMs encode correlations from their training data that map onto protected-class characteristics in ways that are difficult to detect and even more difficult to prevent after the fact. The difference between a compliant AI lending system and one that triggers a CFPB enforcement action often comes down to a single architectural question: does your system detect potential bias in the output, or does it enforce against it at the point of decision? Detection is necessary. Enforcement is what protects you.

Types of AI Bias in Production LLM Systems

Bias in LLM-based AI systems takes several distinct forms, each with different causal mechanisms and different regulatory implications. A complete bias management program must address all of them.

Why Bias Detection Alone Is Insufficient

Bias detection — analyzing model outputs for evidence of differential treatment across demographic groups — is a necessary part of an AI bias management program. It is not sufficient for regulatory compliance because it is retrospective. Detection tells you that your model produced biased outputs on your test set, or in a historical audit of production data. It does not prevent biased outputs from being delivered to users.

The compliance gap between detection and enforcement is not semantic. Consider the regulatory timeline:

• Your bias detection pipeline runs weekly against a sample of production decisions.
• Week 3, it identifies a new proxy variable pattern — loan officers using the AI assistant are asking about applicants' neighborhoods in ways that correlate with race.
• You investigate, confirm the pattern, and update the model or add a rule. This takes 4–6 weeks.
• During those 4–6 weeks, the biased pattern continued in production, affecting real applicants.
• When a CFPB examination asks for a control that prevented this outcome, you have a detection log, not an enforcement record.

Enforcement means the biased decision was blocked before delivery. The applicant never received the biased output. The audit record shows BLOCKED, not ALLOWED-THEN-DETECTED. This is the difference between a preventive control and a detective control — and regulators require both, with a clear preference for preventive controls in the highest-risk categories.

ECOA / Reg B Disparate Impact Requirements for AI Lending Decisions

The Equal Credit Opportunity Act and its implementing regulation, Regulation B, prohibit creditors from discriminating against applicants on the basis of race, color, religion, national origin, sex, marital status, age, or the fact that the applicant receives income from public assistance. These prohibitions apply to any lending decision — including decisions made by or with the assistance of an AI system.

There are two theories of liability under ECOA that apply to AI lending decisions:

Disparate Treatment

Disparate treatment occurs when a creditor treats applicants differently based on a protected characteristic — even if the basis for treatment is an LLM's internal representation rather than an explicit rule. An AI system that produces systematically harsher credit assessments for applicants with names associated with certain racial or ethnic groups, identical financial profiles, has engaged in disparate treatment even if no human actor intended discrimination.

Disparate Impact

Disparate impact occurs when a facially neutral policy or practice disproportionately harms a protected class without business necessity. Under ECOA, disparate impact claims are cognizable even without intent to discriminate. An AI lending model that uses zip code as a significant factor in credit decisions may produce disparate impact on protected-class applicants through the proxy variable mechanism, even if zip code is a statistically valid predictor of credit risk. Business necessity alone does not provide a complete defense — the creditor must also show there is no less discriminatory alternative with comparable predictive power.

How CoreGuard Detects and Blocks Biased Decisions at Policy Evaluation

CoreGuard's bias enforcement architecture operates on the structured action context submitted to the API — the fields, values, and relationships in the lending decision request — rather than on the LLM output text. This is a deliberate design choice. Output text analysis is probabilistic and can be evaded. Structured field evaluation against documented rules is deterministic and cannot be evaded without changing the input.

The lending_v1 policy pack implements bias detection in three layers:

• Direct protected class detection: Rules that block any decision where the action context includes direct reference to protected characteristics in evaluative fields. These are hard blocks — confidence 1.0, never overridable.
• Proxy variable detection: Rules that identify when known proxy variables appear as primary or heavily weighted factors. The proxy variable catalog in lending_v1 covers geographic variables (zip code, census tract, county), name-derived signals, educational institution, and occupational titles. When a proxy variable appears as a primary decision factor, the rule triggers a BLOCKED or MODIFIED decision requiring removal of the proxy variable from the decision basis.
• Inference detection: Rules that detect when the action description or reasoning contains language that suggests protected class inference from context — phrases indicating assessment of neighborhood character, inference from surname, or assessment of demographic profile from application characteristics.

Disparate Impact Testing Methodology

Runtime enforcement blocks known violations. Disparate impact testing identifies emerging patterns that do not yet have explicit enforcement rules — the feedback loop that keeps the policy pack current.

Audit Record Format for Fair Lending Compliance

Every CoreGuard evaluation generates an audit record that is designed to satisfy Reg B recordkeeping requirements. The key fields for fair lending compliance are:

{
  "audit_record": {
    "record_id": "aud_def456",          // Unique record ID for CFPB examination reference
    "timestamp": "2026-05-05T10:30:00.000Z",  // ISO 8601 UTC — exact decision time
    "request_hash": "sha256:abc...",      // Hash of the full request — proves exact inputs
    "decision_hash": "sha256:def...",      // Hash of the full response — proves exact output
    "certificate": "eyJ...",              // JWT-format signed certificate
    "hmac_signature": "hmac-sha256:xyz...", // Tamper-evident HMAC-SHA256 signature
    "policy_set": "lending_v1",           // Exact policy version — enables historical replay
    "evaluator_version": "1.3.0"          // Engine version — for regression test documentation
  },
  "policy_violations": [
    {
      "rule_id": "lending.ecoa.015",     // Maps directly to Reg B adverse action reason code
      "rule_description": "Zip code used as primary factor — known proxy for race",
      "severity": "CRITICAL",
      "field": "action.context.zip_code",   // Exact field that triggered the violation
      "regulatory_citation": "Reg B, 12 CFR 1002.6(b)(2)"
    }
  ]
}

The rule_id field serves as the adverse action reason code for Reg B compliance. Each rule in the lending_v1 policy pack maps to a specific regulatory citation, enabling a direct connection from the BLOCKED decision to the Reg B disclosure requirement. When this decision results in a denied application, the adverse action notice can cite "proxy variable usage (Reg B, 12 CFR 1002.6(b)(2))" as the specific reason — satisfying the specificity requirement that generic "automated system" notices do not.

Real-World Scenario: The Loan Officer AI and Biased Follow-Up Questions

Consider a regional lender that deploys an AI assistant to help loan officers document and evaluate applications. The system is designed to help officers ask relevant follow-up questions and identify missing information. Without a governance enforcement layer, the following interaction is possible:

In the uncontrolled scenario, the biased questions were generated, delivered to the loan officer, and potentially asked to the applicant before any detection system could flag the violation. The harm occurred at the moment of delivery. In the enforced scenario, the policy evaluation ran before the questions reached the loan officer, and the output was either rewritten or blocked before delivery. The audit record shows a compliant interaction from the first moment.