Metrics in Beta

Our goal is to provide high quality, mathematically sound and vetted metrics in the SDMetrics library, and we recognize that synthetic data is a new space undergoing active research. So to encourage discussion and collaboration, we've introduced a metrics in Beta section for anyone wanting to explore with us.
We envision many new metrics may start out in Beta before being validated and adopted by the wider community.

What can cause a metric to be in Beta?

A metric can be experimental for many reasons, including the ones below.
The mathematical concepts are too new. Synthetic data is an area of active research. The research might be so new that it would benefit from more validation through the open source community before wider adoption.
The metric isn't robust. Some metrics may not be reliable for every dataset. They may fluctuate widely based on built-in randomness or they may heavily depend on external algorithms that aren't optimized for every dataset.
The interpretation isn't clear. Metric scores should have a clear interpretation. Even if a metric uses a well-known mathematical method, it may lack clarity in the context of synthetic data. It may be possible to "trick" the metric or there may be multiple, conflicting interpretations for it.


What is the process upgrading these metrics from Beta?
Last modified 2mo ago