Hands reviewing a typed analysis page with passages underlined and circled, beside an open reference book, a fountain pen, and a yellow “Verify” sticky note.

Module 1

What AI Actually Is

AI can be useful without being reliable. Fluency is not understanding, and confidence is not correctness.

How AI Systems Work

AI systems generate outputs by learning patterns from vast amounts of training data. You provide a prompt — a question, a task, an instruction — and the system responds with text that matches the pattern of a correct-looking answer. These systems can produce genuinely useful output: language, code, summaries, classifications, and analysis. What they cannot do is understand what they are producing. A well-formed, polished answer is not evidence the system verified anything.

Hallucination: when an AI generates something that sounds correct but is factually wrong. It is not intent. It is not deception. It is a structural property of how these systems work. It cannot be eliminated, only managed.

In 2023, attorneys in Mata v. Avianca, Inc. (S.D.N.Y.) submitted a legal brief containing AI-generated case citations. The cited cases did not exist. The court described them as “bogus” and sanctioned the attorneys. The output was polished and looked authoritative. Hallucination does not announce itself.

Source: Mata v. Avianca, Inc., No. 1:22-cv-01461 (S.D.N.Y. 2023). Court record public.

The full case record and additional references are listed in the Appendix.

What AI Is Good For

UseWhat AI does well
DraftingProduces a first version quickly.
SummarizingCompresses large amounts of text.
RewritingChanges tone, structure, or clarity.
BrainstormingGenerates options and angles.
TransformationConverts styles, structures, and formats.
Pattern assistanceHelps surface possible themes or inconsistencies.

What AI Is Weak or Risky For

Risky UseWhy It Needs Control
Factual claimsMay fabricate or misstate, confidently.
Legal, medical, financial, or HR tasksStakes are high; context matters; AI cannot take responsibility.
Arithmetic and formal logicMay appear reasoned while making basic errors.
Security-sensitive tasksCan leak, mishandle, or be manipulated.
Autonomous actionCan act on bad assumptions at machine speed.
Long planning chainsCan drift, omit constraints, or contradict itself.

Is AI Appropriate for This Task?

Before using AI on any task, work through these questions:

  • Does this require factual accuracy? Verify independently before using the output.
  • Does it involve sensitive or private data? Do not paste until organizational policy is clear.
  • Could someone be harmed by a wrong answer? Require human review before action.
  • Could money, access, employment, safety, or legal status be affected? Treat as High or Critical risk.
  • Will AI take action automatically? Require formal controls before proceeding.
  • Would you be comfortable explaining this decision later? If no, stop.