You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Utilize test data from aas-core3.0-testgen rather than our proprietary set. #109
Issue:
We've been using our proprietary test set for validating the SDK against the metamodel. However, with every change in the metamodel that necessitates an update to our SDK, we find ourselves having to make corresponding adjustments to our test data. This iterative process is resource-intensive and can lead to potential inconsistencies and oversights.
Proposal:
To streamline this process and ensure a more robust testing framework, we recommend transitioning to the test data provided by aas-core3.0-testgen. Here are the key benefits of making this switch:
Comprehensive Coverage: The aas-core3.0-testgen provides an extensive set of test data, ensuring thorough validation of our SDK against a wide range of scenarios.
Automatic Generation: The test data from aas-core3.0-testgen is automatically generated, which means we'll always have the most up-to-date test cases without the manual effort of creating or updating them.
Reduced Maintenance: By relying on this external test set, we can significantly reduce the overhead associated with maintaining our own test data. This will allow our team to focus on enhancing the SDK and addressing other critical priorities.
Consistency with the Community: Adopting a widely-used test set ensures that our SDK is in alignment with community standards and best practices.
Next Steps:
We should initiate a transition plan to integrate aas-core3.0-testgen test data into our testing pipeline. This will involve:
Setting up a mechanism to regularly fetch the latest test data from aas-core3.0-testgen.
Adapting our testing infrastructure to utilize this new data source.
Conducting an initial comprehensive test run to identify any gaps or inconsistencies.
Decommissioning our proprietary test set once the transition is complete and verified.
@s-heppner We should definitely reconsider this. The aas-core-works/aas-core-testdatagen repository provides fixed fuzzy tests for v3.1 of the metamodel.
We already used this locally to discover some minor deviations in our implementation of the SDK.
We should discuss how we can implement this in our CI in order to increase the conformance with the standard and reduce the risk of missing changes in future versions of the metamodel.
I agree, though we should evaluate how much resources a whole run of all the test files from testdatagen takes.
Depending on the intensity, we can decide if this should run on every PR, or only on PRs to main.
Let's keep it out of scope of the current CI refactor, but take it on very soon.
Activity
hpoeche commentedon Jul 27, 2026
@s-heppner We should definitely reconsider this. The aas-core-works/aas-core-testdatagen repository provides fixed fuzzy tests for
v3.1of the metamodel.We already used this locally to discover some minor deviations in our implementation of the SDK.
We should discuss how we can implement this in our CI in order to increase the conformance with the standard and reduce the risk of missing changes in future versions of the metamodel.
s-heppner commentedon Jul 27, 2026
I agree, though we should evaluate how much resources a whole run of all the test files from testdatagen takes.
Depending on the intensity, we can decide if this should run on every PR, or only on PRs to
main.Let's keep it out of scope of the current CI refactor, but take it on very soon.