Generative AI and RAG Applications

Explain tokens, context, sampling and common failure modes.

Evidence: A short experiment connects settings to measured output differences.

Separate instructions, context and user input; specify output constraints.

Evidence: A prompt passes a fixed set of normal and ambiguous requests.

Validate generated JSON against an application schema.

Evidence: Malformed or missing fields trigger controlled handling.

Parse approved sources and preserve citation metadata.

Evidence: Each answerable passage retains a source ID and location.

Retrieve relevant chunks before generating an answer.

Evidence: A trace shows the question, retrieved evidence and final response.

Require supported claims and abstention when evidence is absent.

Evidence: An unsupported question produces a clear refusal or clarification.

Build answerable, unanswerable and adversarial test cases.

Evidence: A versioned set includes expected evidence and scoring guidance.

Measure correctness, citation support and relevance.

Evidence: Human-reviewed scores expose at least two distinct failure types.

Treat retrieved instructions as untrusted and redact sensitive fields.

Evidence: Injected source instructions cannot trigger prohibited actions.

Measure token use, retrieval time and total response time.

Evidence: A budget report identifies two tested optimisation options.

Add source links, feedback and visible uncertainty states.

Evidence: Keyboard users can inspect citations and submit feedback.

Publish a versioned prototype with rollback and evaluation records.

Evidence: A release checklist links code, prompts, data and measured quality.