Start with a question, not an acronym

You ask a workplace AI assistant: "I lost my work laptop. When should I report it?" It replies, "By the end of the day."

That answer sounds clear. But the current policy says to report it immediately. The assistant used an old document.

The problem is not its grammar. The problem is the evidence behind the answer. Today you will follow that evidence from its source, through a search, into an answer, and through the checks that keep a system useful.

All organizations, policies, records, and results in this lesson are fictional. They are teaching examples, not instructions for handling an actual incident at your school or employer.

SecAI+ connection: This topic develops CY0-001 Objective 1.2, data security, and Objective 1.3, security throughout the AI lifecycle. Learn to explain the examples first; the vocabulary names what you have just done.

What you need from the previous topic

A model is the learned structure used to produce an output. Training develops or adjusts that structure. Inference uses an existing model to make a prediction or generate an answer. A prompt supplies a request and possibly supporting information.

Giving a model a policy paragraph to use in an answer is not, by itself, training it. Keep that distinction in mind.

1. What counts as data?

Data is information a system can store or process. For AI, it might be examples used in training, a user's question, a retrieved paragraph, or the resulting answer. It is not just numbers in a spreadsheet.

Three ways information can be organized

Structured data follows a consistent set of fields. Each row below describes one login event using the same columns:

Semi-structured data has markers such as names and values, but records can contain different or nested fields. This JSON example has named fields and a list of methods:

{"event":"L31","account":"demo-a","methods":["password","security-key"]}

Unstructured data does not present its main content as a fixed table of fields. For example: "I lost my laptop while traveling. Please tell me how to report it." Photographs and spoken recordings are other examples.

These terms describe organization, not trustworthiness. A perfectly organized table can contain false information. A paragraph can contain someone's private address. A file may also mix types: an email has fields in its header and free-form text in its body. JSON can be constrained by a schema, and a PDF can contain a table; do not classify solely by file extension.

Try this: A record has named keys for device and status, with an optional nested location object. Which description fits the way it is organized?

Reason it through: Semi-structured. The named markers provide organization without requiring the same flat set of columns in every record. That tells us how to process the record, not whether its contents are safe.

2. Fix the data without inventing facts

Imagine an export contains these three rows:

Data cleansing means fixing or handling data-quality problems, such as a confirmed duplicate. Data verification means checking data against evidence or requirements to see whether it is accurate and usable.

Worked example: three rows do not mean three events

1. Both L31 rows refer to the same event ID. Check the original system to confirm that one is a duplicate export, not a distinct event.
2. Keep the raw export in its approved protected location. In the working copy, retain one L31 row and record the change. This is cleansing.
3. L32 has a missing count. A blank does not mean zero failed attempts. Check the original event record. This is verification.
4. If you cannot recover the value, mark it as missing and follow the project's handling rule. Do not make up a number merely to complete the table.

Why security teams care: Counting a duplicated failure twice could exaggerate suspicious activity. Turning missing values into zeros could hide it. Cleaning should improve evidence, not quietly rewrite it.

Try this: A cleaned copy says L32 had zero failures, but the original still has no count. Is the zero verified?

Reason it through: No. Neat formatting is not evidence. The team must resolve or preserve the uncertainty instead of presenting an invented value as a fact.

Integrity is a different question

Data integrity concerns whether data remains complete and protected against improper alteration. Suppose an approved policy says "report immediately," but an unauthorized edit changes it to "report next month." The file may still open and look professional; its integrity has been compromised.

Access restrictions help prevent unauthorized changes. Version records and change checks help detect and investigate them. A cryptographic hash is a content fingerprint: comparing it with a trusted reference can detect a change, but it does not prove the original statement was true. Authorized updates also change a hash.

3. Where did it come from? What happened to it?

Two related terms help you investigate an answer:

• Provenance emphasizes origin and source history: who produced the data, where it came from, and which version it is.
• Lineage emphasizes the route through processing: what was copied, filtered, transformed, or used downstream.

They overlap in real systems. Use these questions to understand the distinction, not as a claim that every tool defines them identically. A lineage record can trace data from its origin through its transformations. Microsoft Purview lineage overview

Worked example: follow a policy paragraph

IT publishes Policy P2, revision 2
              |
              v
Text extracted from the approved document
              |
              v
Split into short passages, keeping source/version
              |
              v
Passage stored in a searchable collection
              |
              v
Passage supplied to an AI answer

"IT published revision 2" records provenance. The documented extraction, splitting, storage, and use record lineage. The small pieces of descriptive information, such as document ID, revision, and allowed audience, are called metadata: data about the data.

Try this: A correct source document says "immediately," but the extracted passage says "within a month." What should you inspect?

Reason it through: Inspect the processing history between the source and the extracted passage. Lineage helps locate where the text changed. Knowing the original publisher alone does not explain the changed wording.

Why it matters: Without the trail, you may repair the original file while the assistant continues retrieving an incorrect copy.

4. Let the model consult a reference

A model's prior training does not guarantee it knows a private organization's current policy. One solution is to search approved documents and supply the relevant passages when answering.

Retrieval-augmented generation (RAG) means retrieving information and providing it as context for a model to generate a response. Think "look it up, then write with that evidence." RAG combines retrieval with generation; it is not simply a new name for training. Original RAG research

Work through the lost-laptop question

Our fictional collection contains:

Before answering, the application should restrict retrieval to documents the requester is allowed to use. Within that permitted collection, it looks for relevant, appropriate evidence. Current-version metadata helps exclude the superseded policy.

QUESTION: When do I report a lost work laptop?
    |
    v
RETRIEVE: Select relevant, permitted, current P2 passage
    |
    v
ADD CONTEXT: Question + P2 passage + response instructions
    |
    v
GENERATE: Draft an answer using the supplied passage
    |
    v
CHECK: Does the answer match P2? Is the source current?

A supported answer is: "Report it to the IT service desk immediately, according to Policy P2, revision 2."

The passage supports who to contact and when. It does not establish that the device has already been locked or that someone stole it. Adding either claim would exceed the evidence.

Try this: If the assistant instead retrieves P1 and faithfully repeats "by the end of the day," did accurate paraphrasing solve the problem?

Reason it through: No. It used an outdated source. Retrieval quality and generation quality are separate checks. A citation can help you inspect the evidence, but a citation alone is not proof that an answer is right.

RAG needs controlled retrieval, suitable context, and verification of the resulting answer. It does not automatically prevent hallucinations or enforce permissions. The application's data-access controls must do that work. Microsoft's RAG overview

5. How can search match different words?

The question might say "missing notebook computer" while a policy says "lost laptop." Exact-word matching could miss a useful passage. A common approach is to compare numerical representations of the text.

An embedding is a list of numbers representing an item, such as a passage, in a form suitable for comparison. An embedding model creates these numbers. For a text-similarity application, related meanings are intended to produce similar representations; results are not infallible.

Vector storage holds these numerical representations and supports similarity search. The stored entries are linked to passages and metadata. A chunk is one of the smaller passages created from a larger document.

Two processes, not one mysterious box

PREPARE THE COLLECTION
Approved document -> chunks -> embeddings -> searchable storage
                      Keep text, source, version, and permissions

ANSWER A QUESTION
Question -> compatible query embedding -> find similar passages
         -> supply permitted text to the model -> draft answer

The model that writes the answer generally receives the retrieved text, not just a mysterious array of numbers. The numerical representation helps find the text. Question and document representations must be comparable, typically by using the same embedding model. Keyword and vector search can also be combined; not every RAG system requires a vector database. Microsoft's vector-search overview

A deliberately tiny search example

Suppose a teaching search tool reports these illustrative similarity values, not confidence percentages:

The old policy ranks highest for similarity, but it is still outdated. Search similarity does not establish truth, freshness, or authorization. Metadata and access checks remain necessary. The exact values here are invented to make that distinction visible; they are not outputs of an embedding model.

Try this: "We converted private notes into embeddings, so anyone can access the vector store." What is wrong with the claim?

Reason it through: Numerical representation is not encryption or permission removal. Treat the representations and their linked text as protected data. Changing the format does not grant someone access.

6. Three more data tools, with clear limits

Augmentation: create useful variations

Data augmentation adds modified or synthetic training examples intended to improve coverage. For an authorized phishing-detection exercise, an existing example might be rewritten with different wording while preserving the underlying deceptive request. The new example's label must still be checked.

Original: "Reply with your password to keep access." Variation: "Send your account password so access can continue." Both illustrate the same fictional credential-request pattern. If a variation changes the meaning into a legitimate safety warning, blindly retaining the phishing label would introduce an error.

More examples are not automatically better examples. Keep generated variants out of held-out evaluation data when they are closely related to training examples.

Balancing: prevent one category from dominating learning

Data balancing adjusts how categories are represented or weighted during training. Imagine 90 legitimate messages and 10 phishing messages. A system that always predicts "legitimate" gets 90 out of 100 right but catches zero of the ten attacks.

Possible responses include collecting more varied attack examples, changing sampling, or weighting errors differently. Balancing is not a rule that every dataset must be exactly 50/50, and it does not guarantee fairness or reliable detection. Evaluate on realistic, separate data and inspect how each category performs.

Try this: Adding more legitimate examples to that dataset increases its size. Does that necessarily fix the missing attack coverage?

Reason it through: No. Size and category coverage are different. You must address the underrepresented or poorly learned class, not simply add more rows.

Watermarking: mark origin, not truth

Watermarking embeds a detectable marker in content. One use is helping identify content made by a particular AI generation system. A marker may be invisible to a person and require a compatible detector. Google DeepMind's watermarking example

A watermarked image can still depict a fictional event. A watermark does not verify every factual claim, encrypt the content, or guarantee detection after every edit. No detected watermark does not prove that content was human-created.

Try this: A detector recognizes a watermark in an image of a break-in. Does that prove a real break-in occurred?

Reason it through: No. Information about how content was produced is different from evidence that the depicted event happened.

7. Security does not end when the AI goes live

The AI lifecycle is the continuing process of deciding what a system should do, building or configuring it, testing it, using it, and improving it. Our policy helper is a small example: answer employees' questions using approved policies, without taking action on accounts or devices.

The following is a practical route through the stages named in SecAI+ Objective 1.3. Real projects revisit stages rather than moving through them only once.

Validation is not confined to one date: checks occur before release and as the environment changes. Monitoring observes ongoing behavior; maintenance makes needed updates. NIST treats AI risk management as work across the lifecycle, not simply a release-day check. NIST AI Risk Management Framework

Worked example: the policy changes after launch

The helper passed its original tests. Later, revision 3 replaces revision 2. The team must approve and ingest the new document, update its searchable passages and metadata, remove or exclude superseded entries, test retrieval and answers, and monitor the result. Retraining the language model is not automatically required just to update retrieved policy text.

Try this: A student reports that an answer cites a policy that has been withdrawn. Is "the system passed testing last month" an adequate response?

Reason it through: No. Investigate during monitoring and maintenance, correct the source or retrieval configuration, and revalidate. Past success does not establish current correctness.

8. Make the human's job explicit

"A person is involved" is too vague. What does that person actually do?

These responsibilities can overlap. Checking an answer before approving its use can be both validation and part of an approval loop. Our policy helper only drafts an answer; it does not lock a device. If a future system can act, explicitly define the approval step instead of assuming a person will notice a bad action afterward.

What you should be able to explain now

An AI answer has more than a model behind it. There are sources, transformations, search decisions, permissions, and people. You should be able to point to a specific failure and say what to check:

• Wrong value in a table? Verify the evidence; do not just tidy its appearance.
• Wrong extracted passage? Follow its lineage back to the source.
• Wrong policy version? Check metadata, retrieval, and maintenance.
• Unsupported sentence in an answer? Compare the claim with the supplied passage.
• Restricted information? Enforce authorization before it becomes answer context.
• Bad behavior after launch? Investigate, review the change, test, and keep monitoring.

Use the guided walkthrough and review below to practice. No external AI account, coding, or file download is needed.

Review the distinctions, not just the names

Explain each term with this lesson's example

Four mistakes to avoid

"Clean means correct." A tidy zero is still wrong if someone invented it to fill a blank.

"The closest match must be the right source." The old policy can match the topic extremely well. Similarity does not prove currency, truth, or permission.

"RAG is retraining." Supplying a retrieved passage changes the evidence for an answer, not automatically the model's learned parameters.

"Numbers are not sensitive." Embeddings are representations, not encryption. Protect their store and linked source content.

Remember the limits

• Augmentation can add bad examples if meaning or labels change unnoticed.
• Balancing is not a universal 50/50 requirement or a guarantee of fairness.
• Watermarks help with attribution; they do not prove the depicted event occurred.
• A citation is something to inspect, not permission to skip verification.
• Missing evidence calls for uncertainty or a request for better evidence, not invention.

Reconstruct the lifecycle

Start with the business use case. Collect and prepare appropriate data. Develop or select a model, evaluate it, deploy it, and validate it in the intended environment. Monitor and maintain it. Review feedback, make changes, and test again.

For each stage, name an actual check. "Be secure" is not a specific check; "test that an employee cannot retrieve restricted case records" is.

Explain the person's responsibility

• Human validation: check whether the answer matches evidence.
• Human-in-the-loop: require approval within the workflow before an action proceeds.
• Human oversight: assign ongoing responsibility and authority to intervene or stop the system.

The roles can overlap. Identify what the person does rather than assuming the terms are mutually exclusive job titles.

Before the Canvas quiz

Try explaining why P2 is appropriate, P1 is outdated, M1 is irrelevant, and R1 is not authorized. Then explain why a policy update still needs testing even when the underlying language model is unchanged.

The quiz uses only multiple-choice and true/false questions about concepts taught in this lesson. Practice here is ungraded and does not add another submission.