AI-103 Certification Practice Question #15

Question

You have a Microsoft Foundry project that contains a customer support agent grounded in internal documentation.
After a recent update, users report the following issues:
Some answers are unsupported by retrieved documents.
A small number of responses are flagged for policy violations.
You need to evaluate each issue.
Which observability signals should you use for each issue? To answer, drag the appropriate observability signals to the correct issues. Each observability signal may be used once, more than once, or not at all. You may need to drag the spit bar between panes or scroll to view content.
NOTE: Each correct selection is worth one point.

Accepted Answer

The two issues map cleanly onto two distinct Foundry observability signal families. "Answers unsupported by retrieved documents" is exactly the groundedness failure mode — the groundedness evaluator measures faithfulness to the retrieved context and flags unsupported/fabricated claims — so it pairs with Groundedness evaluation metrics. "Responses flagged for policy violations" is a content-safety/harmful-content concern, which the risk and safety evaluators measure, so it pairs with Risk and safety metrics. Latency breakdown traces and token usage analytics measure performance and cost, not answer faithfulness or safety, so they are not used. There are no community votes, but the mapping is strongly grounded in the built-in evaluator taxonomy, so confidence sits in the upper HOTSPOT/DRAG band.

More AI-103 practice questions