Comparing AI models using different tasks

Overview

Each AI model supported in Copilot Chat has different strengths. To help you decide which model to use, this article provides real-world examples of common developer tasks along with sample prompts, responses, and guidance on why a particular model is a good fit.

These examples show how models vary in their reasoning style, response depth, and ability to handle visual input. Use them to compare output quality and choose the right model for your specific workflow.

For a full list of supported models and side-by-side feature comparisons, see Choosing the right AI model for your task.

GPT-4o

对于受益于速度、响应能力和常规用途推理的常见开发任务，GPT-4o 是一个不错的选择。 在处理需要广泛知识、快速迭代或基础代码理解的任务时，可以使用 GPT-4o 模型。

Example scenario

Consider a scenario where you are writing helper functions for access control in a Python service. You want to add docstrings to clarify the purpose of the function.

Below is the helper function that grants editor access to a document:

def grant_editor_access(user_id, doc_id):
    access = AccessManager()
    access.assign_role(
        resource_type="document",
        resource_id=doc_id,
        role="editor",
        user_id=user_id
    )

Example prompt

Add a docstring to the function that explains what it does.

Example response

def grant_editor_access(user_id, doc_id):
    """
    Grants editor access to a specific document for a given user.

    This function uses the AccessManager to assign the "editor" role
    to the specified user for the provided document ID.

    Args:
        user_id (str): The ID of the user to grant access to.
        doc_id (str): The ID of the document to which access is being granted.
    """
    access = AccessManager()
    access.assign_role(
        resource_type="document",
        resource_id=doc_id,
        role="editor",
        user_id=user_id
    )

Why GPT-4o is a good fit

• The function is short and self-contained, making it ideal for quick docstring generation.
• GPT-4o can recognize the pattern and provide a clear, concise explanation.
• The task doesn't require deep reasoning or complex logic.

o3-mini

OpenAI o3-mini is a fast, cost-effective reasoning model designed to deliver coding performance while maintaining lower latency and resource usage. o3-mini outperforms o1 on coding benchmarks with response times that are comparable to o1-mini. Copilot is configured to use OpenAI's "medium" reasoning effort.

Example scenario

Consider a scenario where you are working on a software licensing system. You have a list of user records, each stored as a dictionary with fields like name, active, and signup_date. You want to find only the active users, sorted by their signup date, so that the newest users appear last.

Below is list of users you are working with:

users = [
    {"name": "Alice", "active": True, "signup_date": "2023-12-10"},
    {"name": "Bob", "active": False, "signup_date": "2023-11-05"},
    {"name": "Charlie", "active": True, "signup_date": "2024-01-15"}
]

Example prompt

Filter the list of users to only include active users, and sort them by signup_date.

Example response

# Filter to only include active users
active_users = [user for user in users if user["active"]]

# Sort active users by signup_date
active_users_sorted = sorted(active_users, key=lambda user: user["signup_date"])

print(active_users_sorted)

Why o3-mini is a good fit

• The task involves simple filtering and sorting, which doesn’t require deep reasoning.
• Its fast responses make it ideal for quick iteration during development.

Gemini 2.0 Flash

Gemini 2.0 Flash 支持图像输入，开发人员可以在执行 UI 检查、图表分析或布局调试等任务时引入视觉对象上下文。 这使得 Gemini 2.0 Flash 特别适用于需要图像增强问题解决的场景，例如请求 Copilot 分析 UI 截图中的辅助功能问题，或帮助理解布局中的视觉对象 Bug。

Example scenario

You’re designing a shopping cart system and have a UML class diagram that outlines the relationships between Cart, Product, User, and Order. Instead of writing out the system from scratch, you want to upload the image of the diagram and ask Copilot to generate the classes.

Example prompt

Use this UML diagram to generate the classes for a shopping cart system.

Example response

Copilot will generate a class for each of the classes in the diagram, including the relationships between them. Here’s the example code generated for the Cart class:

from cart_item import CartItem
from order import Order

class Cart:
    def __init__(self, cart_id, user_id):
        self.cart_id = cart_id
        self.user_id = user_id
        self.items = []

    def add_item(self, item):
        self.items.append(item)

    def remove_item(self, item):
        self.items = [i for i in self.items if i != item]

    def calculate_total(self):
        return sum(item.quantity * item.product.price for item in self.items)

    def place_order(self):
        # Implementation for placing an order
        return Order("", None, 0)

Why Gemini 2.0 Flash is a good fit

• It can interpret visual assets, such as UML diagrams, wireframes, or flowcharts, to generate code scaffolding or suggest architecture.
• It can be useful for reviewing screenshots of UI layouts or form designs and generating.

Claude Sonnet 3.5

Claude Sonnet 3.5 是获得日常编码支持的不错的选择，包括编写文档、回答特定于语言的问题或生成样板代码。 它提供有用的直接答案，不会过度复杂化任务。 如果你的工作存在成本约束，则建议使用 Claude Sonnet 3.5，因为它为与 Claude Sonnet 3.7 相同的许多任务提供可靠的性能，但资源使用显著降低。

Example scenario

Consider a scenario where you are implementing both unit tests and integration tests for an application. You want to ensure that the tests are comprehensive and cover any edge cases that you may and may not have thought of.

For a complete walkthrough of the scenario, see Writing tests with GitHub Copilot.

Why Claude Sonnet 3.5 is a good fit

• It performs well on everyday coding tasks like test generation, boilerplate scaffolding, and validation logic.
• The task leans into multi-step reasoning, but still stays within the confidence zone of a less advanced model because the logic isn’t too deep.

Claude Sonnet 3.7

Claude Sonnet 3.7 在软件开发生命周期的各个阶段（从初始设计、错误修复、维护到性能优化）均表现出色。 它特别适合用于多文件重构或架构规划等需要理解跨组件上下文的场景。

Example scenario

Consider a scenario where you're modernizing a legacy COBOL application by rewriting it in Node.js. The project involves understanding unfamiliar source code, converting logic across languages, iteratively building the replacement, and verifying correctness through a test suite.

For a complete walkthrough of the scenario, see Modernizing legacy code with GitHub Copilot.

Why Claude Sonnet 3.7 is a good fit

• Claude Sonnet 3.7 handles complex context well, making it suited for workflows that span multiple files or languages.
• Its hybrid reasoning architecture allows it to switch between quick answers and deeper, step-by-step problem-solving.

Further reading

• Choosing the right AI model for your task
• Copilot Chat Cookbook