OpenBB/tests/README.md

# 1. Gamestonk Terminal : `Unit Testing`

This document is part of the `Gamestonk Terminal` library documentation.

It aims to provide necessary information in order :
  - Run `unit tests`
  - Build `unit tests`
  - Maintain `unit tests`

## 1.1. Why having unit tests ?

Here `Unit tests` purpose is to allow : update without fear.

Insuring with enough confidence that : addition, removal or update of a `module/function` won't break other parts of the code.

To do that we need `unit tests` to be :
- fast to write
- fast to run

**FAST TO WRITE**

The `unit tests` will only tests the most common cases.

**FAST TO RUN**

Slow part of the code like `network` or `database` access should be either `mocked` or `marked` as `slow`.

# 2. Run `unit tests`

In this section we will explain every think you need to run the `unit tests` on `Gamestonk Terminal`.

## 2.1. How to install tests dependencies ?

To run the tests you will need the `dev-dependencies`.

By default poetry installs the `dev-dependencies` when you run this command :
```bash
poetry install
```

If you dont want to install the `dev-dependencies` you will have to add the option `--no-dev` like this :
```bash
poetry install --no-dev
```

## 2.2. How to run `tests` : by `module` ?

You can run tests on a specific package/module by specifying the path of this package/module, like this :

```bash
pytest tests/gamestonk_terminal/some_package
pytest tests/gamestonk_terminal/some_package/some_module.py
```

## 2.3. How to run `tests` : by `name` ?

You can run tests by their name :

```
pytest -k "test_function1"
```


## 2.4. How to run `tests` : by `markers` ?

You can run tests only on specific markers, like this :

```
pytest -m slow
pytest -k "not slow"
pytest -k "not slow and not network"
```

You can list the available markers using this command :

```
pytest --markers
```

## 2.5. How to skip tests function ?

```python
import pytest

@pytest.skip
def test_some_function(mocker):
    pass

@pytest.skip("This time with a comment")
def test_another_function(mocker):
    pass
```

## 2.6. How to skip tests modules ?

```python
import pytest

pytest.skip(msg="Some optional comment.", allow_module_level=True)

def test_some_function(mocker):
    pass
```

# 3. Build `unit tests`

## 3.1. Where to add my `test module` ?

All the `unit tests` should be insides the `tests` folder.

There should be at most one `test module` for each `module` of `Gamestonk Terminal`.

Each `test module` should follow the same path than the `module` it is `testing`.

For instance to `test` the following module :
- `gamestonk_terminal/stocks/due_diligence/dd_controller.py`

A `test module` should be added here :
- `tests/gamestonk_terminal/stocks/due_diligence/test_dd_controller.py`

## 3.2. How to record network ?

Network exchange can be recording inside a `test function` using the `@pytest.mark.vcr` `marker`.

Here is an example :

```python
@pytest.mark.vcr
def test_single():
    assert requests.get("http://httpbin.org/get").text == '{"get": true}'
```

You can choose the `cassette` name using the `@pytest.mark.default_cassette` marker.
Here is an example :

```python
@pytest.mark.default_cassette("example.yaml")
@pytest.mark.vcr
def test_default():
    assert requests.get("http://httpbin.org/get").text == '{"get": true}'
```

## 3.2. Where to put data files ?

Each data file should be related exactly to one `test module`.

The purpose is to have each `test module` as self-contained as possible.

Data related to a `test module` should be stored within the same folder than this `test module`.

Example with `tests/gamestonk_terminal/some_package/some_module.py` :

- `tests/gamestonk_terminal/some_package`/cassettes/`some_module`/test_function1.yaml
- `tests/gamestonk_terminal/some_package`/txt/`some_module`/test_function1.csv
- `tests/gamestonk_terminal/some_package`/csv/`some_module`/test_function1.csv
- `tests/gamestonk_terminal/some_package`/json/`some_module`/test_function1.json

## 3.3. How to filter `api keys` on `cassettes` ?

The filtering of `api keys` from a `cassette` should be done directly inside the `test module` using this `cassette`.

The purpose is to have each `test module` as self-contained as possible.

To filter `api keys` from a `cassettes` you can define a `vcr_config` fixture inside your `test module`.

Here is an example of filtering :

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_query_parameters": [
            ("token", "MOCK_TOKEN"),
        ]
    }
```

More information about `vcr_config` fixture in the documentation of `pytest-recording` :
- https://github.com/kiwicom/pytest-recording

## 3.4. How to `mock` ?

**PYTEST-MOCK**

A `mocker` fixture is available through the package `pytest-mock`.

Here is an example on how to `mocker` fixture can be used :
```python
def test_something(mocker):
    mocker.patch("some_package.some_module")
```

More information about `pytest-mock` are available here :
- https://pypi.org/project/pytest-mock/

**MOCKEYPATCH**

There is also a `monkeypatch` fixture available by default inside `pytest`.

```python
def test_double(monkeypatch):
    monkeypatch.setattr("some_package.some_module", "Some value")
```

More information about `monkeypatch` are available here :
- https://docs.pytest.org/en/6.2.x/monkeypatch.html

## 3.5. Which `helpers` are available ?

You can find the available helpers inside the following package/module :
- `tests/helpers/`
- `tests/conftest.py`

See also the `pytest fixtures` which are autoloaded helpers.

## 3.6. Which `fixtures` are available ?

You can list all the available `pytest fixtures` using the following command :

```
pytest --fixtures
```

More on custom fixtures here :
 - [FIXTURES](FIXTURES.md)

## 3.7. Which `markers` are available ?

You can list the available markers using this command :

```
pytest --markers
```

More information on markers location are available in pytest documentation :
- https://docs.pytest.org/en/6.2.x/mark.html#mark


## 3.8. Known `issue` / `solution`

**YFINANCE**

If a method is using the `yfinance` library but do not let you pick the `start/end` dates it will pick the current date each time.

So your test might fail the next days.

To solve that you can filter the `start/end` date like this :

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_query_parameters": [
            ("period1", "1598220000"),
            ("period2", "1635980400"),
        ]
    }
```

You can also refactor this method to let access to `stard/end` dates.


**USER-AGENT**

Some function uses a random `User-Agent` on the `HTTP HEADER` when fetching data from an `API`.

Here is how to filter this random `User-Agent`.

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_headers": [("User-Agent", None)],
    }
```

**YFINANCE**

If you do something like this with `yfinance` library.

```python
import yfinance as yf

yf.download(tickers="VCYT QSI")
```

Chances are you requests will be multi-threaded.

Issues : as for now `vcrpy` seems to be incompatible with multi-threading.

The library `vcrpy` is used to record `cassettes` (`network` calls into `yaml` files).

Here is a solution to still combine `yfinance` and `vcrpy` :
```python
import pytest
import yfinance

yf_download = yf.download
def mock_yf_download(*args, **kwargs):
    kwargs["threads"] = False
    return yf_download(*args, **kwargs)

@pytest.mark.vcr
def test_ark_orders_view(kwargs_dict, mocker, use_color):
    mocker.patch("yfinance.download", side_effect=mock_yf_download)
    
    yf.download(tickers="VCYT QSI")
```

## 3.9. List of useful `vscode` tools for `unit tests`

**VSCODE TESTING**

The default testing tool from `vscode` should let you add breakpoints and run debug on a specific `test`.

It's a convenient way to see what's inside your `test` while running.

More information on this tool are available here :
- https://code.visualstudio.com/docs/python/testing

## 3.10. How to handle `dev-dependencies` ?

**UPDATE PYPROJECT**

If a library need to be added or removed from `dev-dependencies` this can be done directly in the `pyproject.toml` file.

All the `dev-dependencies` should be under the `tool.poetry.dev-dependencies` section, like in this example :

```toml
[tool.poetry.dev-dependencies]
pytest = "^6.2.2"
pylint = "^2.7.2"
flake8 = "^3.9.0"
...
```

**EXPORT REQUIREMENTS**

After updating the `pyproject.toml` you will have to export the `requirements` files using the following commands :
```bash
poetry export -f requirements.txt  -o requirements.txt --without-hashes --dev
poetry export -f requirements.txt  -o requirements-full.txt --extras prediction --without-hashes --dev
```

# 4. Maintain `unit tests`

## 4.1. What is the PR process ? (in progress)
Here are the steps to write proper tests for Gamestonk :

    A. Find right place
    B. Verify coverage is above 90%
    C. Set the right markers

**A. Find right place**

Put the code following the same module and package structure than `gamestonk_terminal` package.

**B. Verify coverage is above 90%**

Once you made your `PullRequest` an automation will let you know whether or not you have the proper amount of tests coverage.

You can also run the following command to check your coverage manually :
- `pytest --cov --cov-fail-under=90`

**C. Set the right markers**

If parts of your have specificities like :
- being `slow`
- requiring `network` connectivity
- requiring `authentication` to `APIs`

Then the proper markers should be sent on each test.


The rests of this document is there to provide a deeper comprehension of this steps.

A tests update might be asked whenever the answer to one of the following question is `yes` :

    A. Is this PR fixing bug which was not detected by the current tests ?
    B. Is this PR reducing code coverage ?
    C. Is this PR adding features ?
    D. Is this PR updating features ?

## 4.2. Which automations are available ?

**PULL REQUEST**

A `github action` should comment every `Pull Request` with the code `coverage`.

This automation will not enforce any rules regarding `tests`.

## 4.3. How to check code coverage ?

**PULL REQUEST AUTOMATION**

You should be able to see the code coverage on comment of your `Pull Requests`.

**MANUALLY**

This is how to manually check code coverage.
```bash
pytest --cov=gamestonk_terminal --cov=terminal --cov-report term-missing
```

You can also select a specific package/module with the option `--cov`.

Here is an example where we select only the package `stocks\due_diligence` :
```bash
pytest --cov=gamestonk_terminal\stocks\due_diligence --cov-report term-missing
```

**PRE-COMMIT**

Code `coverage` and `unit tests` are not run on `pre-commit` hooks since it will slow down each commit.