-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Howie/sample-recording-7 #44536
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Howie/sample-recording-7 #44536
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request refactors the sample testing infrastructure for the Azure AI Projects SDK to improve modularity, add support for optional environment variables in recorded tests, and provide comprehensive documentation.
Key Changes
- Splits the monolithic
sample_executor_helpers.pyinto separate, focused modules:sample_executor.py(core executors and decorators), andtest_samples_helpers.py(shared test configuration) - Introduces the
@additionalSampleTestsdecorator to enable testing samples with optional environment variables through additional parameterized test cases - Adds comprehensive README.md documentation with usage examples for sync/async sample testing, including record/playback workflows
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/ai/azure-ai-projects/tests/samples/sample_executor_helpers.py | Deleted old monolithic helper file (206 lines removed) |
| sdk/ai/azure-ai-projects/tests/samples/sample_executor.py | New comprehensive module with sync/async executors, path discovery functions, and additionalSampleTests decorator for optional environment variables |
| sdk/ai/azure-ai-projects/tests/samples/test_samples_helpers.py | New shared configuration module containing LLM validation instructions and environment variable mappings |
| sdk/ai/azure-ai-projects/tests/samples/test_samples.py | Updated to use new imports and demonstrates usage of @additionalSampleTests for the computer_use sample |
| sdk/ai/azure-ai-projects/tests/samples/test_samples_async.py | Updated to use new imports and refactored async sample executor |
| sdk/ai/azure-ai-projects/tests/samples/README.md | New comprehensive documentation for sample testing patterns, including examples and best practices |
| sdk/ai/azure-ai-projects/assets.json | Updated asset tag reference for test recordings |
| test_id="sample_agent_computer_use_with_flag", | ||
| ), | ||
| ] | ||
| ) |
Copilot
AI
Jan 2, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Inconsistent naming: the decorator is referenced as @samplePathPasser() but should be @SamplePathPasser() (capital S and P) to match the class name used throughout the codebase and in the earlier examples.
|
|
||
| The decorator also appends env-var keys to the pytest id for the matching sample. | ||
| """ | ||
|
|
Copilot
AI
Jan 2, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The variable name env_var_sets_by_sample uses an unclear abbreviation 'env_var'. Consider renaming to environment_variable_sets_by_sample for better clarity, or at minimum env_variable_sets_by_sample to make it clear this refers to environment variables.
4cd48f0 to
2d72059
Compare
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines