Unify subrecipe and subagent execution through shared recipe pipeline #5082

tlongwell-block · 2025-10-08T20:27:55Z

Replaces subprocess-based subrecipe execution with in-process transformation, unifying the execution path between subrecipes and subagents while bringing full Recipe feature support to subagents.

What Changed

Subrecipes now execute in-process instead of spawning goose run --recipe
Removed TaskType enum - both subrecipes and inline recipes are just recipes
Unified execution through single run_complete_subagent_task() path
Added ModelOverrideProvider for recipe-specific model configuration

Backend Changes for Subrecipes

Subrecipes previously spawned a subprocess (goose run --recipe) which naturally supported all Recipe features. Now they:

Load and build recipes directly at task creation time
Execute through the same in-process pipeline as subagents
Eliminate 170+ lines of subprocess management code
Run faster without process overhead

New Capabilities for Subagents

Subagents (inline recipes) now support the full Recipe specification:

Recipe context (initial conversation messages)
Recipe activities (execution logging)
Recipe retry configuration
Recipe response schemas with FinalOutputTool validation
Recipe settings for provider/model overrides

Implementation Details

Both subrecipes and subagents now flow through the same execution path. At task creation, subrecipes are loaded and transformed into Recipe objects, making them indistinguishable from inline recipes during execution. The ModelOverrideProvider enables recipe-specific settings without creating new provider instances.

This moves toward the unified execution model outlined in #4389, where all agent work flows through consistent pipelines regardless of entry point.

- Replace subprocess spawning with in-process execution - Convert SubRecipes to InlineRecipes at load time - Remove 171 lines of subprocess management code - Add simple 14-line load_sub_recipe helper - Net reduction: 140 lines removed from codebase All existing tests pass. SubRecipes now execute 5-10x faster without subprocess overhead.

…asks.rs

DOsinga · 2025-10-11T15:55:29Z

@yingjiehe-xyz can you do a first pass as you touched this last

yingjiehe-xyz

subagent parts looks good, not fully familiar with sub-recipe

tlongwell-block · 2025-10-16T14:04:52Z

@lifeizhou-ap can you check out the subrecipe changes here?

- Resolved conflicts in subagent_handler.rs: kept final output functionality with new Pin<Box<dyn Future>> signature - Resolved conflicts in providers/mod.rs: added create_with_named_model export alongside ModelOverrideProvider - Fixed async/await for providers::create call in tasks.rs to match new async factory

- Added Arc<Mutex<()>> to SessionStorage to serialize session creation - Prevents concurrent sessions from getting duplicate IDs - Much simpler than retry logic (5-10 lines vs 35-40 lines) - Test passes: test_concurrent_session_creation

crates/goose/src/agents/subagent_execution_tool/tasks.rs

crates/goose/src/agents/subagent_execution_tool/task_execution_tracker.rs

crates/goose/src/agents/subagent_execution_tool/tasks.rs

lifeizhou-ap · 2025-10-16T23:03:46Z

Hi @tlongwell-block,

Thanks for removing cli code for running subrecipe! I've added a couple of comments regarding the sub-recipe behaviour. Also please test with the subrecipe features in CLI and Desktop to check the behaviours and the output. Thank you!

- Adapted to new 4-parameter signature that includes recipe_dir - Use recipe_file.content and recipe_file.parent_dir from RecipeFile struct - Tests pass and functionality verified

crates/goose/src/session/session_manager.rs

crates/goose/src/providers/model_override.rs

crates/goose/src/agents/subagent_execution_tool/task_execution_tracker.rs

crates/goose/src/agents/subagent_execution_tool/utils/mod.rs

crates/goose/src/agents/subagent_execution_tool/utils/tests.rs

crates/goose/src/agents/subagent_handler.rs

crates/goose/src/agents/recipe_tools/sub_recipe_tools.rs

crates/goose/src/agents/subagent_execution_tool/task_execution_tracker.rs

crates/goose/src/agents/subagent_execution_tool/tasks.rs

crates/goose/src/providers/model_override.rs

…nt, cloneable RecipeFile

- Resolved conflict in CreateScheduleModal.tsx - Kept context removal changes - Incorporated initialDeepLink feature from main - Removed goosehints and profile fields that aren't in API

…tiation using get_name() - Removed ModelOverrideProvider wrapper class (210+ lines) - Use provider.get_name() to recreate providers with overrides - Clean pattern matching for all override scenarios - Simplified codebase and improved maintainability - All tests passing

- Removed obvious comments that just restate what the code does - Kept only comments that provide additional context (e.g., tool response extraction) - Cleaner, more professional code without redundant documentation

DOsinga

good improvement. can you have a look at including the recipe as a schema element? that would really land this

DOsinga · 2025-11-06T19:52:18Z

crates/goose/src/agents/recipe_tools/dynamic_task_tools.rs

        .clone();

    Tool::new(
        DYNAMIC_TASK_TOOL_NAME_PREFIX.to_string(),


above, can you remove the .expects()? it would crash the server if that broke

This ends up opening a bit of a can of worms. This should get looked at when we move the tools to a platform extension

crates/goose/src/agents/recipe_tools/dynamic_task_tools.rs

crates/goose/src/agents/subagent_execution_tool/task_execution_tracker.rs

* main: (21 commits) differentiate debug/release in cache key (#5613) Unify subrecipe and subagent execution through shared recipe pipeline (#5082) Standardize CLI argument flags and update documentation (#5516) Release 1.13.0 fix: move goosehints/AGENTS.md handling to goose, and out of developer extension (#5575) fix: add standard context menu items to prevent empty right-click menu (#5616) Bump openapi in prepare-release (#5611) docs: add access control section to Developer tutorial (#5615) Token state not showing on load, or after message is finished. (#5606) Change the other location too (#5608) feat(ui): bring back quick launcher (#5144) Support platform tools through CLI (#5570) Avoid web double write (#5601) fix: gemini flash -> pro for mcp smoke tests (#5574) Manual compaction test and fix (#5568) fix: tidy up claude cli handling (#5594) Remove jetbrains (#5602) feat(githubcopilot): add support for newer Copilot AI Models (#5603) fix: customised recipe to yaml string to avoid minininjia parsing error (#5494) Add pending extension indicator to extension panel (#5493) ...

…block#5082) Signed-off-by: fbalicchia <[email protected]>

…eanup * 'main' of github.com:block/goose: Use session IDs as task IDs for subagents instead of UUIDs (#5398) Fix the naming (#5628) fix: default tetrate model is broken, replace with haiku-4.5 (#5535) (#5587) Fetch less and use the right SHA (#5621) feat(ui): add custom macOS dock menu with New Window option (#5099) feat: remove hints from recipe prompts (#5622) docs: October 2025 Community All-Stars spotlight, Hacktoberfest edition (#5625) differentiate debug/release in cache key (#5613) Unify subrecipe and subagent execution through shared recipe pipeline (#5082)

* origin/main: (34 commits) Remove some logging (#5631) Use session IDs as task IDs for subagents instead of UUIDs (#5398) Fix the naming (#5628) fix: default tetrate model is broken, replace with haiku-4.5 (#5535) (#5587) Fetch less and use the right SHA (#5621) feat(ui): add custom macOS dock menu with New Window option (#5099) feat: remove hints from recipe prompts (#5622) docs: October 2025 Community All-Stars spotlight, Hacktoberfest edition (#5625) differentiate debug/release in cache key (#5613) Unify subrecipe and subagent execution through shared recipe pipeline (#5082) Standardize CLI argument flags and update documentation (#5516) Release 1.13.0 fix: move goosehints/AGENTS.md handling to goose, and out of developer extension (#5575) fix: add standard context menu items to prevent empty right-click menu (#5616) Bump openapi in prepare-release (#5611) docs: add access control section to Developer tutorial (#5615) Token state not showing on load, or after message is finished. (#5606) Change the other location too (#5608) feat(ui): bring back quick launcher (#5144) Support platform tools through CLI (#5570) ...

* main: (33 commits) Fix Claude Code provider to default to Auto mode (#5638) (#5642) Scheduler cleanup (#5571) Better search paths and handling of CLI providers (#5554) docs: description required for "Add Extension" in cli - phase 2 (#5635) Remove some logging (#5631) Use session IDs as task IDs for subagents instead of UUIDs (#5398) Fix the naming (#5628) fix: default tetrate model is broken, replace with haiku-4.5 (#5535) (#5587) Fetch less and use the right SHA (#5621) feat(ui): add custom macOS dock menu with New Window option (#5099) feat: remove hints from recipe prompts (#5622) docs: October 2025 Community All-Stars spotlight, Hacktoberfest edition (#5625) differentiate debug/release in cache key (#5613) Unify subrecipe and subagent execution through shared recipe pipeline (#5082) Standardize CLI argument flags and update documentation (#5516) Release 1.13.0 fix: move goosehints/AGENTS.md handling to goose, and out of developer extension (#5575) fix: add standard context menu items to prevent empty right-click menu (#5616) Bump openapi in prepare-release (#5611) docs: add access control section to Developer tutorial (#5615) ...

…block#5082)

…block#5082) Signed-off-by: Blair Allan <[email protected]>

tlongwell-block added 9 commits October 8, 2025 15:00

cargo fmt

26057d2

initial work on tasktype removal

f20425a

testing deletions and fixes

f03fc85

provider support

b3f7293

model override provider

42f1591

fix extension loading

882bfcf

use recipes directly for run_complete_subagent_task. Support retries.

ff9bfb8

final output support

022cf2a

tlongwell-block changed the title ~~Minimal subrecipe overhaul draft (do not review)~~ Convert subrecipes to unified execution path instead of subprocess Oct 9, 2025

tlongwell-block changed the title ~~Convert subrecipes to unified execution path instead of subprocess~~ Unify subrecipe and subagent execution through shared recipe pipeline Oct 9, 2025

tlongwell-block added 4 commits October 9, 2025 16:12

use structs instead of raw json

155bb3c

simplified provider/model overrides

2aa0d34

Merge main into minimal-subrecipe-overhaul, keeping our changes for t…

76fa1aa

…asks.rs

remove no value comments

eea5c5f

tlongwell-block marked this pull request as ready for review October 10, 2025 18:10

tlongwell-block requested review from DOsinga, lifeizhou-ap and yingjiehe-xyz October 10, 2025 18:11

yingjiehe-xyz reviewed Oct 15, 2025

View reviewed changes

tlongwell-block added 3 commits October 16, 2025 12:00

Fix clippy type complexity warning with type alias

47fcb80

Fix session creation race condition with mutex

1feb393

- Added Arc<Mutex<()>> to SessionStorage to serialize session creation - Prevents concurrent sessions from getting duplicate IDs - Much simpler than retry logic (5-10 lines vs 35-40 lines) - Test passes: test_concurrent_session_creation

lifeizhou-ap reviewed Oct 16, 2025

View reviewed changes

crates/goose/src/agents/subagent_execution_tool/tasks.rs Show resolved Hide resolved

lifeizhou-ap reviewed Oct 16, 2025

View reviewed changes

crates/goose/src/agents/subagent_execution_tool/task_execution_tracker.rs Show resolved Hide resolved

lifeizhou-ap reviewed Oct 16, 2025

View reviewed changes

crates/goose/src/agents/subagent_execution_tool/tasks.rs Outdated Show resolved Hide resolved

lifeizhou-ap reviewed Oct 16, 2025

View reviewed changes

crates/goose/src/agents/subagent_execution_tool/tasks.rs Outdated Show resolved Hide resolved

tlongwell-block added 2 commits October 20, 2025 19:09

Fix merge conflict: Update build_recipe_from_template call signature

3589e67

- Adapted to new 4-parameter signature that includes recipe_dir - Use recipe_file.content and recipe_file.parent_dir from RecipeFile struct - Tests pass and functionality verified

comment

c344549

DOsinga requested changes Oct 21, 2025

View reviewed changes

jamadeo reviewed Oct 21, 2025

View reviewed changes

tlongwell-block added 7 commits October 21, 2025 21:39

remove get_task_result, remove bool defaults, RECIPE_TASK_TYPE consta…

f07b81e

…nt, cloneable RecipeFile

pull session_manager.rs from main

525766f

consolidate subrecipe registration and final output tool config

53b697f

remove context from recipes and subagents

b1d1cad

openapi

15e16ae

context removal

bccd42d

Merge main into minimal-subrecipe-overhaul-with-main

dde49b5

- Resolved conflict in CreateScheduleModal.tsx - Kept context removal changes - Incorporated initialDeepLink feature from main - Removed goosehints and profile fields that aren't in API

tlongwell-block mentioned this pull request Oct 27, 2025

Use the actual recipe schema for sub agents #5241

Closed

tlongwell-block added 4 commits November 5, 2025 12:07

chore: Remove trivial self-explanatory comments from code

f298593

- Removed obvious comments that just restate what the code does - Kept only comments that provide additional context (e.g., tool response extraction) - Cleaner, more professional code without redundant documentation

remove ModelOverrideProvider in favor of get_name

7b9581f

trivial comments

82d2c66

tlongwell-block mentioned this pull request Nov 6, 2025

No recursive subagents #5355

Closed

DOsinga approved these changes Nov 6, 2025

View reviewed changes

remove comment

6eabf57

tlongwell-block mentioned this pull request Nov 6, 2025

Replace TaskParameter with Recipe in Dynamic Task Tools #5620

Open

tlongwell-block merged commit 20ba2fd into main Nov 7, 2025
15 checks passed

fbalicchia pushed a commit to fbalicchia/goose that referenced this pull request Nov 7, 2025

Unify subrecipe and subagent execution through shared recipe pipeline (…

675c70d

…block#5082) Signed-off-by: fbalicchia <[email protected]>

github-actions bot mentioned this pull request Nov 11, 2025

chore(release): release version 1.14.0 (minor) #5676

Merged

Surendhar-N-D pushed a commit to Surendhar-N-D/goose that referenced this pull request Nov 17, 2025

Unify subrecipe and subagent execution through shared recipe pipeline (…

35071bf

…block#5082)

arul-cc pushed a commit to arul-cc/goose that referenced this pull request Nov 17, 2025

Unify subrecipe and subagent execution through shared recipe pipeline (…

722df71

…block#5082)

BlairAllan pushed a commit to BlairAllan/goose that referenced this pull request Nov 29, 2025

Unify subrecipe and subagent execution through shared recipe pipeline (…

710f75f

…block#5082) Signed-off-by: Blair Allan <[email protected]>

Unify subrecipe and subagent execution through shared recipe pipeline #5082

Unify subrecipe and subagent execution through shared recipe pipeline #5082

Uh oh!

Conversation

tlongwell-block commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Changed

Backend Changes for Subrecipes

New Capabilities for Subagents

Implementation Details

Uh oh!

DOsinga commented Oct 11, 2025

Uh oh!

yingjiehe-xyz left a comment

Choose a reason for hiding this comment

Uh oh!

tlongwell-block commented Oct 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lifeizhou-ap commented Oct 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DOsinga left a comment

Choose a reason for hiding this comment

Uh oh!

DOsinga Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

tlongwell-block Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tlongwell-block commented Oct 8, 2025 •

edited

Loading