Development

This guide walks you through the workflow for developing new metrics in the SDMetrics library.

Have you defined your metric? Before you begin development, we recommend you read the Defining your metric guide, file an issue and wait for our feedback. This will prevent any throwaway work and help us release the metric faster.

Local Setup

Get your machine for local development.

1. Fork the SDMetrics repo on GitHub and then clone your fork locally

$ git clone [e-mail]:your_username/SDMetrics

2. Install your local copy. We recommend using a virtual environment, such as virtualenv or conda.

(env)$ cd SDMetrics/
(env)$ make install-develop

3. Create a branch for local development

(env)$ git checkout -b issue-123-adding-your-new-metric

Branch naming scheme: Every feature branch should be associated with completing a single GitHub issue. Prefix your branch with issue-<x> where <x> is the GitHub issue number.

4. Now you can develop your metric locally on your branch.

Testing

Please thoroughly test and verify changes before making a pull request

Add new unit tests

Your new metric should include at least one associated test method. It's likely that you have multiple use cases or scenarios. Write a separate test method for each.

Scope

Each test should validate only the code that you wrote. If you are using another library, assume that it works properly when being passed the right values and use the mock package to mock any random behavior. Do not read or write to any file system or database.

Structure

Write your unit tests inside the tests folder, which contains the same hierarchy as sdmetrics. Begin the filename with test_. For example, if your metric code is written inside:

sdmetrics/column_pairs/statistical/your_module.py

Then the corresponding unit tests should be written in:

tests/sdmetrics/column_pairs/statistical/test_your_module.py

Unit tests should use only the unittest and pytest modules. Each test method should start with the test_ prefix and have descriptive names about the scenario covered.

Names such as test_my_metric_value_error and test_my_metric_timeout are descriptive. Names such as test_1 or test_error are not descriptive.

Existing tests

Your changes should also pass the existing tests and style checks in the library.

Check the style of your code using lint:

(env)$ make lint

Run the full test suite:

(env)$ make test

Tip! During development, you can also run a subset of tests for convenience.

(env)$ python -m pytest tests.test_my_metric_timeout
(env)$ python -m pytest -k 'my_metric'

Run make test-all to test all supported Python versions.

Get a full coverage report:

(env)$ make coverage

Making a pull request

When you are done developing your metric and testing, you can make a pull request. This lets us know that your code is ready for review.

Are you ready to make a pull request? Go through the checklist below to verify that you are ready to make a pull request.

✓ The branch fully resolves a GitHub issue and the issue number is referenced in the branch name

✓ The branch includes new unit tests that cover the added functionality

✓ All existing tests and style checks pass

Make sure you've committed all changes to your branch and pushed it to GitHub

(env)$ git add .
(env)$ git commit -m "<Descriptive details about the new metric>"
(env)$ git push origin issue-123-your-new-metric-name

Then, you can make a pull request through the SDMetrics GitHub site.

Tip! You can automatically link your pull request to the GitHub issue. When making the pull request, add a comment: Resolves #<x>, where x is the number of your issue.

For example the following comment will link the pull request to issue 123:

Resolves #123 

Click the button below to make your pull request.

Compare · sdv-dev/SDMetricsGitHub

And that's it! A member of the core SDV team will review your changes and leave feedback on the pull request. Your code may require a few changes before it's ready to be merged.

The SDV open source team is a passionate but small group of maintainers. We appreciate your time and patience as we review your contributions! Feel free to reach out to us via your GItHub issue or Slack with any questions or concerns.