This guide walks you through the workflow for developing new metrics in the SDMetrics library.
Have you defined your metric? Before you begin development, we recommend you read the Defining your metric guide, file an issue and wait for our feedback. This will prevent any throwaway work and help us release the metric faster.
Get your machine for local development.
2. Install your local copy. We recommend using a virtual environment, such as virtualenv or conda.
(env)$ cd SDMetrics/
(env)$ make install-develop
3. Create a branch for local development
(env)$ git checkout -b issue-123-adding-your-new-metric
Branch naming scheme: Every feature branch should be associated with completing a single GitHub issue. Prefix your branch with
<x>is the GitHub issue number.
4. Now you can develop your metric locally on your branch.
Please thoroughly test and verify changes before making a pull request
Your new metric should include at least one associated test method. It's likely that you have multiple use cases or scenarios. Write a separate test method for each.
Each test should validate only the code that you wrote. If you are using another library, assume that it works properly when being passed the right values and use the
mockpackage to mock any random behavior. Do not read or write to any file system or database.
Write your unit tests inside the
testsfolder, which contains the same hierarchy as
sdmetrics. Begin the filename with
test_. For example, if your metric code is written inside:
Then the corresponding unit tests should be written in:
Unit tests should use only the
pytestmodules. Each test method should start with the
test_prefix and have descriptive names about the scenario covered.
Names such as
test_my_metric_timeoutare descriptive. Names such as
test_errorare not descriptive.
Your changes should also pass the existing tests and style checks in the library.
Check the style of your code using lint:
(env)$ make lint
Run the full test suite:
(env)$ make test
Tip! During development, you can also run a subset of tests for convenience.
(env)$ python -m pytest tests.test_my_metric_timeout
(env)$ python -m pytest -k 'my_metric'
make test-allto test all supported Python versions.
Get a full coverage report:
(env)$ make coverage
When you are done developing your metric and testing, you can make a pull request. This lets us know that your code is ready for review.
Are you ready to make a pull request? Go through the checklist below to verify that you are ready to make a pull request.
✓ The branch fully resolves a GitHub issue and the issue number is referenced in the branch name
✓ The branch includes new unit tests that cover the added functionality
✓ All existing tests and style checks pass
Make sure you've committed all changes to your branch and pushed it to GitHub
(env)$ git add .
(env)$ git commit -m "<Descriptive details about the new metric>"
(env)$ git push origin issue-123-your-new-metric-name
Then, you can make a pull request through the SDMetrics GitHub site.
Tip! You can automatically link your pull request to the GitHub issue. When making the pull request, add a comment:
Resolves #<x>, where
xis the number of your issue.
For example the following comment will link the pull request to issue 123:
Click the button below to make your pull request.
And that's it! A member of the core SDV team will review your changes and leave feedback on the pull request. Your code may require a few changes before it's ready to be merged.
The SDV open source team is a passionate but small group of maintainers. We appreciate your time and patience as we review your contributions! Feel free to reach out to us via your GItHub issue or Slack with any questions or concerns.