cc: @rbavery @weiji14
I looked at torchmetrics. And it seems since the metric there -- just like evalem.metrics -- also uses (predictions, references) type of argument, I think we can have new wrappers around torchmetrics at evalem.cv.metrics
For instance, the base metrics in the evalem has evalem._base.metrics.compute(...) method to be overridden by downstream implementation:
https://github.com/NASA-IMPACT/evalem/blob/main/evalem/_base/metrics.py#L38
One evalem.nlp.metrics.ExactMatchMetric has the implementation for example:
https://github.com/NASA-IMPACT/evalem/blob/main/evalem/nlp/metrics/basics.py#L16
cc: @rbavery @weiji14
I looked at torchmetrics. And it seems since the metric there -- just like
evalem.metrics-- also uses(predictions, references)type of argument, I think we can have new wrappers around torchmetrics atevalem.cv.metricsFor instance, the base metrics in the evalem has
evalem._base.metrics.compute(...)method to be overridden by downstream implementation:https://github.com/NASA-IMPACT/evalem/blob/main/evalem/_base/metrics.py#L38
One
evalem.nlp.metrics.ExactMatchMetrichas the implementation for example:https://github.com/NASA-IMPACT/evalem/blob/main/evalem/nlp/metrics/basics.py#L16