Develop generative AI

apps in Azure
https://aka.ms/mslearn-generative-ai

© Copyright Microsoft Corporation. All rights reserved. August 2025

What is Artificial Intelligence?

Generative Agents and Natural Computer Information

AI automation Language Vision Extraction

𝑦 = 𝑓 ( 𝑥 ) Machine Learning

© Copyright Microsoft Corporation. All rights reserved.

Our focus today

Generative Agents and Natural Computer Information

AI automation Language Vision Extraction

𝑦 = 𝑓 ( 𝑥 ) Machine Learning

© Copyright Microsoft Corporation. All rights reserved.

Plan and prepare to
develop AI solutions on
Azure
https://aka.ms/mslearn-prepare-ai-dev

© Copyright Microsoft Corporation. All rights reserved.

What is Generative AI?
• Content-generation by a language model
• Large language models (LLMs)
• Huge volumes of training data
• Many millions of parameters (weights)
……
• Small language models (SLMs)
• Smaller volumes of training data
• Fewer parameters

• Input to the model takes the form of a

prompt …
<..>
• Output contains the generated response:
• Natural language
• Image
• Code
•…
© Copyright Microsoft Corporation. All rights reserved.
Prompts in a generative AI chat session

• System prompt: Constraints and

guidelines for model behavior
… +
…
…
…
• User prompt: Specific question or
instruction …
…
• Completion/Response: The
content generated by the model
• History: The prompts in each
iteration are often appended as …
… …
conversation history on subsequent …
+ …
iterations to provide additional
context for “follow up” prompts

© Copyright Microsoft Corporation. All rights reserved.

Generative AI development in Azure AI Foundry
Azure AI Foundry is the platform for developing generative AI solutions and custom
apps

Two project types are supported:

Azure AI Foundry project Azure AI hub based project

AI hub
Azure AI Foundry
Azure AI Foundry Agents Prompt Flow Compute Azure storage

Azure AI Foundry Models Azure AI Foundry

Azure AI Foundry Agents
Azure AI services
Azure AI Foundry Models
Azure key vault
Azure AI services

© Copyright Microsoft Corporation. All rights reserved.

 Azure AI Foundry Portal
Web-based portal in which you
can:
• Create and manage hubs and
projects
• Manage service connections
• Integrate AI services
• Choose and deploy models
• Test models in interactive
playgrounds
• Manage data and indexes
• Fine-tune models
• Create prompt flows
• Evaluate models and prompt flows
• Apply content safety
Create
• © Copyright AI agents
Microsoft Corporation. All rights reserved.
• Edit and test code
 Demonstration
Prepare for an AI
development project

In this demonstration, you’ll see how to:

• Open Azure AI Foundry portal
• Create a project
• Deploy and test a generative AI model

Try this demo for yourself after class at:

https://go.microsoft.com/fwlink/?linkid=226
0323
© Copyright Microsoft Corporation. All rights reserved.
Knowledge check

1 Which Azure resource provides language and vision services from a single endpoint?
⃣Azure AI Language
⃣Azure AI Vision
⃣Azure AI Services

You plan to create a simple chat app that uses a generative AI model. What kind of project should you
2 create?
⃣Azure AI Foundry project
⃣Azure AI hub based project
⃣Azure AI Custom Vision project

3 Which SDK enables you to connect to resources in a project?

⃣Azure AI Services SDK
⃣Semantic Kernel SDK
⃣Azure AI Foundry SDK

© Copyright Microsoft Corporation. All rights reserved.

Choose and deploy
models from the model
catalog
https://aka.ms/mslearn-ai-models

© Copyright Microsoft Corporation. All rights reserved.

What is the model catalog?
A central repository of
models:
• OpenAI models (like GPT
models)
• Microsoft models (like Phi
models)
• Third-party models (Mistral,
Model selection factors
Nvidia Meta, Cohere, and
others)
Vendor and ✓
license
Size and cost ✓
Performance ✓
metrics
Deployment ✓
options
Regional ✓
availability
© Copyright Microsoft Corporation. All rights reserved.
Compare benchmarks across models

Model benchmarks are used to assess and compare language models

Common metrics:
• Quality index
• Accuracy
• Cost
• Coherence
• Fluency
• GPTSimilarity
• Groundedness
• Relevance
• Latency

© Copyright Microsoft Corporation. All rights reserved.

 Model deployment options

To use a model, you must deploy it to an endpoint

Standard deployment Serverless compute Managed compute

Azure AI Foundry models Open models

Supported models Azure AI Foundry models
with serverless billing Custom models

Hosting resource Azure AI Foundry AI Project in a hub AI Project in a hub

Billing basis Token-based billing Token-based billing Compute-based billing

© Copyright Microsoft Corporation. All rights reserved.

Optimizing model performance

Optimization strategy choice depends on your requirements:

• Optimize for context when Retrieval Combined strategies

the model lacks contextual Augmented Prompt engineering
Generation (RAG) +
knowledge, and you want to RAG
Add contextual search
maximize response results to the prompt +
accuracy Fine-Tuning
Context
• Optimize the model when optimizatio Prompt engineering Fine-tuning
you want to improve the n System messages specify Additional model
response format, style, or What the tone, format, content training with example
model need expectations. prompts and responses
speech by maximizing to know Detailed, explicit user
consistency of behavior prompts for better
responses

Model optimization
© Copyright Microsoft Corporation. All rights reserved. How the model needs to
act
Exercise

Choose and deploy a language model

In this exercise, you’ll use the Azure AI Foundry portal
to:
• Create a hub and project
• Review model benchmarks
• Compare models Trainers:
• Deploy models Customize this slide to
•Start
Test models in the
the exercise at: Chat playground include the link to the
https://go.microsoft.com/fwlink/?linkid=227 hosted lab
7718 environment and any
custom instructions.
(then remove this
arrow)

© Copyright Microsoft Corporation. All rights reserved.

Knowledge check

1 Where can you test a deployed model in the Azure AI Foundry portal?
⃣Chat playground
⃣Sandbox
⃣Development toolbox

You want to specify the tone, format, and content for each interaction with your model in the
2 playground. What should you use to customize the model response?
⃣Benchmarks
⃣Grounding
⃣System message

3 What deployment option should you choose to host an OpenAI model in an Azure AI
Foundry resource?
⃣Standard deployment
⃣Serverless compute
⃣Managed compute

© Copyright Microsoft Corporation. All rights reserved.

Develop an AI app with
the Azure AI Foundry
SDK
https://aka.ms/mslearn-ai-foundry-sdk

© Copyright Microsoft Corporation. All rights reserved.

What is the Azure AI Foundry SDK?
A comprehensive toolchain for AI
development on Azure Azure AI Search

• Connect securely to a project with Azure OpenAI Service Azure AI Agent Service
the Azure AI Projects Client
library
• Use project connections to access
resources, and write code using Azure AI Services …
their own APIs: Project

• Azure AI Services
Entra authentication
• Azure OpenAI Service
• Azure AI Search
• Azure AI Agent Service
• Others
© Copyright Microsoft Corporation. All rights reserved.
Project connections
Project

Connections
Retrieve connections, including
credentials for access to connected Azure AI Services
services
• List of all connections in the project Azure OpenAI Service
• List of connections of a specific type
• The default connection for a specific Azure AI Search

type
Azure AI Agent Service
• A specific connection by ID

…
© Copyright Microsoft Corporation. All rights reserved.
Creating a chat client

Azure AI Foundry Models

1. Get an authenticated chat client
for a deployed model from the
project Project

2. Use the OpenAI API to chat with

the model

ChatClient
(Azure OpenAI)

© Copyright Microsoft Corporation. All rights reserved.

Exercise

Create a generative AI chat app

In this exercise, you’ll use the Azure AI Foundry portal
to:
• Deploy a model in an Azure AI Foundry project
• Create a chat app using the Azure AI Foundry SDK
Trainers:
Customize this slide to
Start the exercise at: include the link to the
https://go.microsoft.com/fwlink/?linkid=230 hosted lab
3427 environment and any
custom instructions.
(then remove this
arrow)

© Copyright Microsoft Corporation. All rights reserved.

Knowledge check

1 What class in the Azure AI Foundry SDK provides a proxy object for a project?
⃣ConnectionProperties
⃣AIProjectClient
⃣ChatCompletionsClient

What value is needed to instantiate a AIProjectClient object?

2
⃣The project endpoint
⃣The Azure OpenAI authorization key
⃣The Azure subscription ID

3 Which SDK should you use to chat with a model that is deployed in an Azure AI Foundry
resource?
⃣Azure OpenAI
⃣Azure Machine Learning
⃣Azure AI Language

© Copyright Microsoft Corporation. All rights reserved.

Develop a RAG-based
solution with your own
data using Azure AI
Foundry
https://aka.ms/mslearn-ai-foundry-rag

© Copyright Microsoft Corporation. All rights reserved.

 Grounding a language model with your own data
Language models create coherent answers to questions, but what are those answers
based on?

Grounding provides specific context to the model to provide accurate, relevant

responses.
Ungrounded Grounded
Original
training
data

What What
Product
product product
catalog
should I use should I use
to do X? A grammatically to do X? A
correct, but contextualized
uncontextualized response
response
© Copyright Microsoft Corporation. All rights reserved.
Retrieval Augmented Generation (RAG)
RAG gathers relevant data to include with a prompt for the language model to use for
grounding context.

A RAG pattern is an architectural design for RAG by including retrieved relevant

data in a prompt:

Retrieve grounding Generate a

data based on user contextualized
Model
input response
User Grounde
d Respons
input e
prompt
Vector-based index Augment the
Lorem -> [1,34,76, prompt with
…] grounding data
Ipsum -> [6,59,54,
…]
… All rights reserved.
© Copyright Microsoft Corporation.
Use RAG with the Azure OpenAI API

Azure OpenAI

1. Index data in your Azure AI

Foundry project using Azure AI
Project
Search Azure AI Search

2. Get Azure AI Search connection

and OpenAI ChatClient from the
project
3. Include Azure AI Search
connection details in OpenAI ChatClient
ChatClient calls

© Copyright Microsoft Corporation. All rights reserved.

Exercise

Create a generative AI app that uses your own

data
In this exercise, you’ll use the Azure AI Foundry portal
to:
• Create a hub and project
• Deploy a model
• Add data to your project
• Create an index for your data Trainers:
• Test the index in the playground Customize this slide to
• Create a RAG client app include the link to the
Start the exercise at: hosted lab
https://go.microsoft.com/fwlink/?linkid=226 environment and any
1362 custom instructions.
(then remove this
arrow)

© Copyright Microsoft Corporation. All rights reserved.

Knowledge check

1 What does groundedness refer to in the context of generative AI?

⃣The use of a locally deployed language model
⃣Using the lowest possible number of tokens in a prompt
⃣Using data to contextualize prompts and ensure relevant responses

2 What pattern can you use to ground prompts?

⃣Metadata Optimized Prompt (MOP)
⃣Retrieval Augmented Generation (RAG)
⃣Data Understanding Support Text (DUST)

3 How can you use the RAG pattern in a client app that uses the Azure OpenAI SDK?

⃣Add text files containing the grounding data to the app folder.
⃣You don't need to do anything. Azure AI Foundry automatically grounds all prompts using Bing
Search.
⃣Add index connection details to the OpenAI ChatClient configuration.

© Copyright Microsoft Corporation. All rights reserved.

Fine-tune a language
model with Azure AI
Foundry
https://aka.ms/mslearn-ai-fine-tune

© Copyright Microsoft Corporation. All rights reserved.

What is fine-tuning?

• Additional training for a foundation

model
• Provides the model with example + = *
prompts and responses
Foundation Training Fine-
• Used to maximize the consistency of
model data tuned
the model’s tone and style in model
responses (how it needs to act)

© Copyright Microsoft Corporation. All rights reserved.

Prepare your data to fine-tune a model
The data must be formatted as a JSON Lines (JSONL) document, for example:

{"messages": [

{"role": "system", "content": "You are an Xbox customer support agent whose primary goal is
to help users with issues they are experiencing with their Xbox devices. You are friendly and
concise. You only provide factual answers to queries, and do not provide answers that are not
related to Xbox."},

{"role": "user", "content": "Is Xbox better than PlayStation?"},

{"role": "assistant", "content": "I apologize, but I cannot provide personal opinions. My primary
job is to assist you with any issues related to your Xbox device. Do you have any Xbox-related
issues that need addressing?"},

]}

© Copyright Microsoft Corporation. All rights reserved.

Fine-tune a model

1. Select the base model

2. Select your training data
3. Optional: Select your validation data
4. Configure advanced options:

Name Description
Number of training examples used to train a single forward
batch_size
and backward pass
learning_rate_multipli Original learning rate used for pre-training multiplied by this
er value
The number of epochs to train the model for. An epoch refers
n_epochs
to one full cycle through the training dataset
seed Random seed used to control the reproducibility of the job

© Copyright Microsoft Corporation. All rights reserved.

 Demonstration
Fine-tune a language model

In this demonstration, you’ll see how to:

• Fine-tune a model in Azure AI Foundry portal

Try this demo for yourself after class at:

https://go.microsoft.com/fwlink/?linkid=227
7719
© Copyright Microsoft Corporation. All rights reserved.
Knowledge check

1 How must data be formatted for fine-tuning?

⃣JSONL
⃣YAML
⃣HTML

2 What does fine-tuning optimize in your model?

⃣What the model needs to know
⃣How the model needs to act
⃣Which words aren’t allowed

3 Which advanced option refers to one full cycle through the training dataset?

⃣seed
⃣batch_size
⃣n_epochs

© Copyright Microsoft Corporation. All rights reserved.

Implement a responsible
generative AI solution in
Azure AI Foundry
https://aka.ms/mslearn-responsible-ai

© Copyright Microsoft Corporation. All rights reserved.

Plan a responsible generative AI solution

User experience
1. Map potential harms that are relevant to your
planned solution
System message & grounding
2. Measure the presence of these harms in the
outputs generated by your solution
Safety system
3. Mitigate the harms at multiple layers in your
solution
4. Manage the solution responsibly by defining Model
and following a deployment and operational
readiness plan

© Copyright Microsoft Corporation. All rights reserved.

Content filters and prompt shields

Filter input (prompt) and output (response) content using thresholds for:
Language related to physical actions intended to cause physical harm or
Violence injury or that describes weapons and related entities
Content that attacks or uses discriminatory language with reference to a
Hate and unfairness person or Identity group based on certain differentiating attributes of these
groups

Sexual content Language related to sexual anatomy, romantic relationships and sexual acts

Language related to physical actions intended to purposely cause physical

Self-harm harm to oneself

Enable prompt shields to prevent “jailbreak” attacks by malicious actors seeking to

circumvent content safety settings

© Copyright Microsoft Corporation. All rights reserved.

Exercise
Apply content filters to prevent the output of
harmful content
In this exercise, you’ll use the Azure AI Foundry portal
to:
• Create a hub and project
• Deploy a model
• Chat with no content filter
• Apply a default content filter Trainers:
• Create and test a custom content filter Customize this slide to
include the link to the
Start the exercise at: hosted lab
https://go.microsoft.com/fwlink/?linkid=227 environment and any
3316 custom instructions.
(then remove this
arrow)

© Copyright Microsoft Corporation. All rights reserved.

Knowledge check

1 Why should you consider creating an AI Impact Assessment when designing a

generative AI solution?
⃣To make a legal case that indemnifies you from responsibility for harms caused by the
solution
⃣To document the purpose, expected use, and potential harms for the solution
⃣To evaluate the cost of cloud services required to implement your solution

2 What capability of Azure AI Foundry helps mitigate harmful content generation at the Safety
System level?
⃣DALL-E model support
⃣Fine-tuning
⃣Content filters

3 Why should you consider a phased delivery plan for your generative AI solution?

⃣To enable you to gather feedback and identify issues before releasing the solution more
broadly
⃣To eliminate the need to map, measure, mitigate, and manage potential harms
⃣To enable you to charge more for the solution

© Copyright Microsoft Corporation. All rights reserved.

Evaluate generative AI
performance in Azure AI
Foundry portal
https://aka.ms/mslearn-ai-evaluation

© Copyright Microsoft Corporation. All rights reserved.

Manual model evaluation

• Evaluate prompt responses

from a model
• Optionally, specify a context
data source
• Define a set of test prompts
with expected responses
• Compare the actual answers
returned by the model
• Score with a thumbs-up or
thumbs-down vote for each
response

© Copyright Microsoft Corporation. All rights reserved.

Automated evaluation

• Evaluate a model, a dataset,

or a prompt flow
• Optionally, use AI to
generate test data
• Review evaluations for:
• AI quality (AI assisted)
• AI quality (NLP metrics)
• Risk and safety

© Copyright Microsoft Corporation. All rights reserved.

Evaluators and metrics

Automated evaluation metrics are calculated by evaluators

AI Quality (AI assisted) AI Quality (NLP-based) Risk and safety

Language quality metrics NLP metrics that compare Metrics related to content
based on analysis by an LLM: responses to ground truth: safety:
• Coherence: logical flow and • F1-Score: ratio of words • Violent content: language
“understandability” of shared by response and pertaining to physical harm or
responses expected response use of weapons
• Fluency: grammar and • BLEU: Bilingual Evaluation • Sexual content: language of
readability of response Understudy metric an explicit or pornographic
• Similarity: semantic • METEOR: Metric for nature
alignment between response Evaluation of Translation with • Self-harm related content:
and expected response Explicit Ordering ROUGE: language pertaining to actions
• Groundedness: relevance Recall-Oriented Understudy for intended to hurt oneself
and alignment with context Gisting Evaluation
data
© Copyright Microsoft Corporation. All rights reserved.
Exercise

Evaluate generative AI model performance

In this exercise, you’ll use the Azure AI Foundry portal
to:
• Create a hub and project
• Deploy a model
• Manually evaluate a model Trainers:
• Use automated evaluation Customize this slide to
Start the exercise at: include the link to the
https://go.microsoft.com/fwlink/?linkid=227 hosted lab
7720 environment and any
custom instructions.
(then remove this
arrow)

© Copyright Microsoft Corporation. All rights reserved.

Knowledge check

1 Which evaluation technique can you use to apply your own judgement about the
quality of responses to a set of specific prompts?
⃣Model benchmarks
⃣Manual evaluations
⃣Automated evaluations

2 Which evaluator compares generated responses to ground truth based on standard

metrics?
⃣Coherence
⃣F1 Score
⃣Protected material

3 Which evaluator uses an AI model to judge the structure and logical flow of ideas in a response?

⃣Coherence
⃣F1 Score
⃣Protected material

© Copyright Microsoft Corporation. All rights reserved.