AI for Federal Decision-Making Training Course

Lesson 1.1: Foundations of Artificial Intelligence for Federal Decision-Making

• Evolution from rule-based systems to machine learning and foundation models
• Functionality of large language models: predictive text generation versus information retrieval
• Key terminology for officials: models, prompts, context windows, tokens, fine-tuning, retrieval-augmented generation, and autonomous agents
• Distinguishing AI from automation, dashboards, and rules engines
• Drivers of recent efficiency and speed improvements and their impact on agency operations

Lesson 1.2: Data as the Cornerstone of Federal Governance

• The federal data ecosystem: administrative, operational, sensor, mission, survey, and public record datasets
• The Federal Data Act, Federal Data Strategy, and the role of Chief Data Officers
• Fit for purpose: accuracy for specific uses versus potential for misinterpretation in other contexts
• Data quality dimensions: completeness, consistency, timeliness, accuracy, provenance, and representativeness
• Addressing missing data, inconsistent reporting, and the limitations of concluding based solely on recorded information

AI Lab 2, Disaster Response Data Analysis: Participants initiate the core case study using a synthetic multi-source disaster-response dataset containing incident records, resource allocations, timestamps, geographic data, and unstructured logs. They employ AI tools to profile the data, evaluate quality, identify operational trends, and detect anomalies. Deliverables include a documented data profile and a trend analysis memo.

Lesson 1.3: Value Creation of AI in Federal Decision Processes

• Application to repetitive patterns: document comprehension, search, summarization, classification, extraction, anomaly detection, and forecasting
• Enhancing situational awareness and identifying operational trends
• Differentiating between mission support and mission execution
• Understanding hallucinations and their implications for official records
• Distinguishing between model confidence and actual accuracy

AI Lab 1, Collaborating with an AI Analyst: Participants process a lengthy federal after-action report through core analytical operations: summarization, inference, transformation, and expansion. They then compare basic prompting against optimized prompting for the same task. The outcome is a personalized prompt library for analytic applications.

AI Lab 3, Identifying Analytic Failures: Participants examine AI-generated analytic outputs that appear correct but contain subtle errors, such as fabricated citations, trends derived from reporting artifacts, or inverted caveats. They identify these errors and determine if current review processes would have detected them. The deliverable is a defined trust threshold for three products within the participant’s area of responsibility.

Lesson 2.1: Ethical Considerations in Federal AI Data Analysis

• Differences between federal AI ethics and corporate AI ethics
• Core principles: accountability, transparency, fairness, contestability, privacy, security, and human oversight
• Bias and disparate impacts within operational datasets
• Data representativeness: inclusion and exclusion of population segments
• Explainability at the level required by human reviewers
• Mitigating over-reliance and automation bias
• Evaluating appropriate versus inappropriate uses based on reversibility, stakes, and personal accountability

AI Lab 4, Ethics and Compliance Review: Teams analyze three AI-supported federal use cases, including one from the disaster-response scenario, against a structured review framework. They assess decision impact, affected parties, high-impact status, disclosure requirements, oversight needs, and impermissible conditions. Deliverables include a completed review sheet for each case and a defended recommendation to proceed, modify, or reject the use case.

Lesson 2.2: Federal Regulatory and Policy Environment for AI

• Structure of federal AI policy: executive orders, Office of Management and Budget (OMB) memoranda, and agency-specific policies
• Interpreting executive direction as an evolving pattern rather than a static state
• OMB directives on usage and acquisition, Chief AI Officer roles, governance boards, and high-impact AI definitions
• Agency AI use case inventories and the implications of public listing
• The National Institute of Standards and Technology (NIST) AI Risk Management Framework and its Generative AI Profile
• The Government Accountability Office (GAO) AI Accountability Framework and audit expectations
• Related regulatory regimes: FISMA and Authority to Operate, FedRAMP, Section 508 accessibility, records management, Freedom of Information Act, and acquisition regulations

AI Lab 5, Navigating Internal Regulations: Participants review their own agency’s AI usage policies, data policies, and approved tool guidance, using course-provided examples where internal policies are unavailable. They produce a plain-language summary of permitted actions. A group comparison follows to identify areas of divergence and convergence across agencies. The deliverable is a personal compliance reference card.

Lesson 3: Data Privacy in AI Applications

• The Privacy Act, systems of records, E-Government Act privacy impact assessments, and OMB Circular A-130
• Personally Identifiable Information (PII) and sensitive categories: direct identifiers, quasi-identifiers, location data, and unstructured text
• Purpose limitation and challenges associated with compatible use
• Data minimization as a primary control mechanism
• Techniques for de-identification, aggregation, masking, and synthetic data generation
• Addressing aggregation and mosaic effects
• Data handling distinctions between consumer accounts and controlled tenancy environments
• Third-party and contractor data handling requirements and acquisition standards

AI Lab 6, Privacy-Protective Analysis: Teams analyze a raw disaster-response extract containing PII in structured fields and unstructured text. They identify sensitive data, determine the minimum necessary information for the analytic question, create a minimized and de-identified working dataset, and re-run the analysis. They compare results, documenting any loss of analytic value and its significance. Deliverables include a data-handling and privacy checklist and a preliminary Privacy Impact Assessment (PIA) summary.

Lesson 4: AI Models for Decision Support Systems

• Distinguishing decision support from decision automation
• A plain-language taxonomy: descriptive, diagnostic, classification, forecasting, anomaly detection, optimization, geospatial, and generative models
• Selecting the appropriate model type for specific decision contexts
• Retrieval and grounding using authoritative documents with verifiable citations
• Federal decision support system patterns in emergency management, search and rescue, logistics, inspection targeting, fraud detection, and benefits triage
• Evaluation metrics: accuracy, calibration, robustness, drift, and relevant operational indicators
• Managing uncertainty and confidence in presenting recommendations
• Avoiding the misapplication of language models to optimization problems
• Human-in-the-loop design: defining the human role and verification capabilities
• Operational failure modes: brittleness, degraded inputs during incidents, and lack of explainability under time pressure

AI Lab 7, Constructing a Grounded Decision-Support Assistant: Participants build an assistant grounded in policy, doctrine, standard operating procedures, and after-action reports that answers operational questions with citations. They then test the assistant’s limitations with out-of-scope, ambiguous, and unanswerable queries. Deliverables include a functional assistant and a documented list of prohibited query types.

AI Lab 8, Decision-Support Insights from Case Data: Teams translate data trends into specific decisions, such as resource allocation recommendations for the disaster-response scenario. They must state assumptions, uncertainty levels, equity considerations, and supporting evidence. Teams then evaluate peer recommendations using a structured rubric.

Lesson 5: Implementing AI-Enhanced Solutions

• Transitioning from use case to implementation: defining scope, ownership, success criteria, decision rights, and stop conditions
• End-to-end responsible analytics workflow: intake, quality assurance, privacy screening, analysis, review, documentation, approval, and archiving
• Strategic options: procurement, configuration, custom development, or no-code solutions
• Acquisition and vendor evaluation considerations, including FedRAMP requirements and current OMB directives
• Assessing the true cost of AI in a federal environment
• Designing pilots that lead to definitive decisions and avoiding prolonged pilot phases
• Governance structures: usage policies, approval pathways, inventories, logging, monitoring, and auditing
• Change management, workforce development, and assignment of accountability
• Assessing the current capabilities of no-code tools and signals indicating the need for engineering, authorization, or formal acquisition
• Sustainability practices: monitoring for drift, re-evaluation following policy changes, and criteria for retiring capabilities

AI Lab 9, Incident Tabletop Exercise: Teams respond to a scenario where an AI-assisted product contains a fabricated reference and an undetected data quality issue, and a decision has already been made. They execute the response protocol: detection, containment, notification, record correction, remediation, and prevention. Deliverables include three immediate controls each team proposes.

AI Lab 10, Implementation Action Plan: Each participant develops a one-page implementation plan for a real AI-supported capability in their organization. The plan must address the problem, approach, data sources, privacy and governance controls, oversight points, costs, risks, pilot design, measurement metrics, and a 30/60/90-day timeline. A peer review follows using a challenge checklist derived from the five course lessons.

Conclusion: Challenge and Commitment

• Presentation of implementation plans to a review board
• Review against a challenge checklist covering data, privacy, oversight, cost, and accountability
• Finalized, defended plan with a 30/60/90-day execution sequence