Add aspire test command with pytest support and auxiliary backchannel infrastructure

[Copilot]

Description

Introduces the aspire test command that will launch the AppHost, run test resources to completion, and shutdown. This PR implements the command infrastructure, DI plumbing, execution logic with full auxiliary backchannel integration, and the supporting infrastructure for test resource management and AppHost lifecycle control.

Changes

CLI Command Infrastructure

• Feature flag: Added testCommandEnabled to KnownFeatures.cs (disabled by default)
• Command class: Created TestCommand with complete execution logic that:
  • Finds the AppHost project using IProjectLocator
  • Launches it via IDotNetCliRunner.RunAsync with backchannel connection
  • Connects to the auxiliary backchannel socket to communicate with AppHost
  • Calls GetTestResultsAsync() via auxiliary backchannel to wait for test resources to complete
  • Displays test results (success/failure) to the user
  • Calls StopAppHostAsync() via auxiliary backchannel for orderly AppHost shutdown
  • Returns appropriate exit code based on test results
• Command options: Added --project option to specify the AppHost project file path
• Resource strings: Added TestCommandStrings.resx with localization for description, status messages, and option descriptions
• DI registration: Registered in both production (Program.cs) and test (CliTestHelper.cs) containers
• RootCommand integration: Conditionally adds subcommand when feature flag is enabled
• Tests: Added TestCommandTests.cs with 4 tests covering help, feature flag visibility, and project discovery
• Error handling: Handles ProjectLocatorException, FailedToConnectBackchannelConnection, and general exceptions

Test Resource Support (Aspire.Hosting)

• TestResourceAnnotation: Added marker annotation to identify test resources that run to completion
• AddPytest Extension (Aspire.Hosting.Python):
  • Wraps AddPythonExecutable to run pytest from the virtual environment
  • Automatically adds TestResourceAnnotation to mark the resource as a test
  • Includes XML documentation with usage examples

Auxiliary Backchannel Enhancements

• New RPC Methods (AuxiliaryBackchannelRpcTarget):
  • GetTestResultsAsync: Scans for resources with TestResourceAnnotation, waits for them to reach finished state using ResourceNotificationService, and returns test results
  • StopAppHostAsync: Initiates orderly shutdown of the AppHost via IHostApplicationLifetime.StopApplication()
• IAuxiliaryBackchannel Interface (Aspire.Cli): Defines contract for all AppHost communication via auxiliary backchannel
• AuxiliaryBackchannel Implementation (Aspire.Cli): Concrete implementation wrapping JsonRpc calls - exceptions bubble up to callers for proper error handling
• Updated AuxiliaryBackchannelMonitor: Now creates and uses IAuxiliaryBackchannel instances instead of direct RPC calls
• Shared Data Types: Added TestResults and TestResourceResult classes to BackchannelDataTypes.cs
• Socket Connection: TestCommand establishes Unix domain socket connection to auxiliary backchannel using SHA256 hash of AppHost path

Usage

# Enable feature flag
aspire config set features:testCommandEnabled true

# View help
aspire test --help

# Run with auto-discovery
aspire test

# Run with specific project
aspire test --project path/to/AppHost.csproj

Example AppHost with pytest

var builder = DistributedApplication.CreateBuilder(args);

// Add pytest test resource
builder.AddPytest("api-tests", "../api")
       .WithArgs("-v", "tests/");

builder.Build().Run();

When executed via aspire test, the command will:

1. Start the AppHost
2. Connect to the auxiliary backchannel socket
3. Call GetTestResultsAsync() which waits for the pytest resource to complete
4. Display test results to the user
5. Call StopAppHostAsync() to shut down the AppHost
6. Return appropriate exit code

Fixes #(issue)

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
  • No
• Does the change require an update in our Aspire docs?
  • Yes
  • No

Original prompt

I want to introduce a new command into the Aspire CLI. The command is aspire test. The purpose of this command is to launch the apphost similarly to aspire run but instead of running continuously it will run specified test resources to completion and then shutdown the resources and exit.

This is just the first of many PRs and for now I want you to just create the new TestCommand and inject it into the DI infrastructure and create a basic test for it. I also want a testCommandEnabled feature flag added and I only want this command to be visble if that feature flag is enabled.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[github-actions]

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 13275

Or

• Run remotely in PowerShell:

iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 13275"

[mitchdenny]

@copilot Now that we have this basic plumbing in place. Create a minimal version of the logic in TestCommand based on logic in the aspire run command that finds the project then launches it with the backchannel connected via the IDotNetCliRunner and then waits for it to exit.

[Copilot]

@copilot Now that we have this basic plumbing in place. Create a minimal version of the logic in TestCommand based on logic in the aspire run command that finds the project then launches it with th...

Implemented in commit 89545b3. The TestCommand now:

1. Finds the AppHost project using IProjectLocator
2. Launches it via IDotNetCliRunner.RunAsync with backchannel connection
3. Waits for the AppHost to exit and returns its exit code

Added --project option to specify the AppHost project file path, and updated resource strings accordingly.

[mitchdenny]

@copilot introduce a new API to the Aspire.Hosting.Python package called AddPytest which is a wrapper around AddPythonExecutable(...) which specifies the last executable argument as pytest.

In addition to wrapping a call to AddPythonExecutable(...) it should add a new annotation called TestResourceAnnotation (defined in Aspire.Hosting). For now this is just a marker annotation which will be expanded on more later.

Then to the auxillary backchannel in the apphost add a method called GetTestResultsAsync(...) which will be a new RPC method. The GetTestResultsAsync call will scan through the apphost looking for all resources that have a TestResourceAnnotation and wait for them to enter a completed state (using the ResourceNotificationService).

Then add another RPC method to the auxillary backchannel which is StopAppHostAsync(...). This fires off a background task which waits for the RPC channel to disconnect and then calls IHostApplicationLifetime.StopApplication(...) for an orderly shutdown of the apphost.

Introduce a new AuxillaryBackchannel type which encapsulates all communication between the CLI and the apphost. Hold a reference to AuxillaryBackchannel in the list of connections that the auxillary backchannel monitor uses and use the AuxillaryBackchannel API instead of calling the RPC API directly (introduce an IAuxillaryBackchannel interface and use it for testability).

[Copilot]

@copilot introduce a new API to the Aspire.Hosting.Python package called AddPytest which is a wrapper around AddPythonExecutable(...) which specifies the last executable argument as pytest.

In ad...

Implemented in commit c5c4890. Changes include:

1. TestResourceAnnotation - Marker annotation in Aspire.Hosting to identify test resources

2. AddPytest - Extension method in Aspire.Hosting.Python:

builder.AddPytest("api-tests", "../api")
       .WithArgs("-v", "tests/");

Wraps AddPythonExecutable and adds TestResourceAnnotation

3. Auxiliary Backchannel RPC Methods:

• GetTestResultsAsync() - Scans for resources with TestResourceAnnotation, waits for them to reach Finished state using ResourceNotificationService
• StopAppHostAsync() - Fires background task that waits 500ms for RPC disconnect, then calls IHostApplicationLifetime.StopApplication()

4. IAuxiliaryBackchannel interface and AuxiliaryBackchannel implementation - Encapsulates all RPC communication between CLI and AppHost

5. Updated AuxiliaryBackchannelMonitor - Now uses IAuxiliaryBackchannel and stores it in AppHostConnection

All components build successfully and are ready for use by the test command.

[mitchdenny]

@copilot connect to the apphost using the auxillary backchannel and call the get test results RPC.

Also ... at the moment the AuxillaryBackchannel is swallowing exceptions. Allow the exceptions to bubble out for now.

[Copilot]

@copilot connect to the apphost using the auxillary backchannel and call the get test results RPC.

Also ... at the moment the AuxillaryBackchannel is swallowing exceptions. Allow the exceptions to ...

Implemented in commit 4498c67.

Changes:

1. Removed exception swallowing - AuxiliaryBackchannel methods now let exceptions bubble up to callers

2. TestCommand now connects to auxiliary backchannel:

   • Computes socket path using SHA256 hash of AppHost path (matches AppHost logic)
   • Waits up to 30 seconds for socket to be created
   • Creates JsonRpc connection over Unix domain socket
   • Calls GetTestResultsAsync() to wait for all test resources to complete
   • Displays test results (success/failure)
   • Calls StopAppHostAsync() for orderly shutdown

All tests pass.