FEAT: Target Registry and AIRT Targets Initializer #1320
Conversation
| ) | ||
|
|
||
| @staticmethod | ||
| def _compute_identifier_hash(target: "PromptTarget") -> str: |
There was a problem hiding this comment.
I like the idea of an identifier hash, but I think if we want to add it, it should be general. I also think we should have a minor refactor while it's still early.
- Substitute RegistryItemMetadata to a frozen
Identifierclass which is inmodelsmodule - Have that class include the hash
- As we add registries, swap out our dictionary identifiers for dataclass identifiers. These should be used everywhere
I can create a PR for the Identifiers refactor, and there is a story in the backlog to do ScorerIdentifier. But I think as part of this we should move TargetIdentifier to a dataclass also
There was a problem hiding this comment.
For you, I might make a separate (prereq) PR that just converts the TargetIdentifier to the dataclass and we can get that in first as I work on the other piece.
|
The GUI may need to know if it's possible to set the system prompt on a target. Can we provide that information in the registry? |
Do you mean PromptChatTarget vs ChatTarget? This can be an attribute of the TargetMetadata possibly? @romanlutz |
Exactly |
3 participants
Description
This PR adds a Target Registry feature for storing instances of targets to be re-used and an AIRT initializer that instantiates a wide range of commonly used targets using environment variables. These additions will be useful for the GUI as they enable us to load commonly used targets and have them ready to use.
TargetRegistryfollows the pattern of the existingScorerRegistryAIRTTargetInitializeris aPyRITInitializersubclass that scans environment variables and registers available targets into theTargetRegistry. No environment variables are strictly required. The supported targets includeOpenAIChatTarget,OpenAIResponseTarget,RealtimeTarget,OpenAIImageTarget,OpenAITTSTarget,OpenAIVideoTarget, andPromptShieldTargetTests and Documentation
Unit tests for
TargetRegistryaddedTODO
AIRTTargetInitializer2_instance_registry