OpenBB/tests/README.md

# 1. Gamestonk Terminal : `Unit Testing`

This document is part of the `Gamestonk Terminal` library documentation.

It aims to provide necessary information in order :

- Run `unit tests`
- Build `unit tests`
- Maintain `unit tests`

## 1.1. Why having unit tests ?

Here `Unit tests` purpose is to allow : update without fear.

Insuring with enough confidence that : addition, removal or update of a `module/function` won't break other parts of the code.

To do that we need `unit tests` to be :

- fast to write
- fast to run

**FAST TO WRITE**

The `unit tests` will only tests the most common cases.

**FAST TO RUN**

Slow part of the code like `network` or `database` access should be either `mocked` or `marked` as `slow`.

# 2. Run `unit tests`

In this section we will explain everything you need to run the `unit tests` on `Gamestonk Terminal`.

## 2.1. How to install tests dependencies ?

To run the tests you will need the `dev-dependencies`.

By default poetry installs the `dev-dependencies` when you run this command :

```bash
poetry install
```

If you dont want to install the `dev-dependencies` you will have to add the option `--no-dev` like this :

```bash
poetry install --no-dev
```

## 2.2. How to run `tests` : by `module` ?

You can run tests on a specific package/module by specifying the path of this package/module, like this :

```bash
pytest tests/gamestonk_terminal/some_package
pytest tests/gamestonk_terminal/some_package/some_module.py
```

## 2.3. How to run `tests` : by `name` ?

You can run tests by their name :

```
pytest -k "test_function1"
```

## 2.4. How to run `tests` : by `markers` ?

You can run tests only on specific markers, like this :

```
pytest -m slow
pytest -k "not slow"
pytest -k "not slow and not network"
```

You can list the available markers using this command :

```
pytest --markers
```

## 2.5. How to skip tests function ?

```python
import pytest

@pytest.skip
def test_some_function(mocker):
    pass

@pytest.skip("This time with a comment")
def test_another_function(mocker):
    pass
```

## 2.6. How to skip tests modules ?

```python
import pytest

pytest.skip(msg="Some optional comment.", allow_module_level=True)

def test_some_function(mocker):
    pass
```

# 3. Build `unit tests`

## 3.1. Where to add my `test module` ?

All the `unit tests` should be insides the `tests` folder.

There should be at most one `test module` for each `module` of `Gamestonk Terminal`.

Each `test module` should follow the same path than the `module` it is `testing`.

For instance to `test` the following module :

- `gamestonk_terminal/stocks/due_diligence/dd_controller.py`

A `test module` should be added here :

- `tests/gamestonk_terminal/stocks/due_diligence/test_dd_controller.py`

## 3.2. How to record network ?

Network exchange can be recording inside a `test function` using the `@pytest.mark.vcr` `marker`.

Here is an example :

```python
@pytest.mark.vcr
def test_single():
    assert requests.get("http://httpbin.org/get").text == '{"get": true}'
```

You can choose the `cassette` name using the `@pytest.mark.default_cassette` marker.
Here is an example :

```python
@pytest.mark.default_cassette("example.yaml")
@pytest.mark.vcr
def test_default():
    assert requests.get("http://httpbin.org/get").text == '{"get": true}'
```

## 3.2. Where to put data files ?

Each data file should be related exactly to one `test module`.

The purpose is to have each `test module` as self-contained as possible.

Data related to a `test module` should be stored within the same folder than this `test module`.

Example with `tests/gamestonk_terminal/some_package/some_module.py` :

- `tests/gamestonk_terminal/some_package`/cassettes/`some_module`/test_function1.yaml
- `tests/gamestonk_terminal/some_package`/txt/`some_module`/test_function1.csv
- `tests/gamestonk_terminal/some_package`/csv/`some_module`/test_function1.csv
- `tests/gamestonk_terminal/some_package`/json/`some_module`/test_function1.json

## 3.3. How to filter `api keys` on `cassettes` ?

The filtering of `api keys` from a `cassette` should be done directly inside the `test module` using this `cassette`.

The purpose is to have each `test module` as self-contained as possible.

To filter `api keys` from a `cassettes` you can define a `vcr_config` fixture inside your `test module`.

Here is an example of filtering :

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_query_parameters": [
            ("token", "MOCK_TOKEN"),
        ]
    }
```

More information about `vcr_config` fixture in the documentation of `pytest-recording` :

- https://github.com/kiwicom/pytest-recording

## 3.4. How to `mock` ?

**PYTEST-MOCK**

A `mocker` fixture is available through the package `pytest-mock`.

Here is an example on how to `mocker` fixture can be used :

```python
def test_something(mocker):
    mocker.patch("some_package.some_module")
```

More information about `pytest-mock` are available here :

- https://pypi.org/project/pytest-mock/

**MOCKEYPATCH**

There is also a `monkeypatch` fixture available by default inside `pytest`.

```python
def test_double(monkeypatch):
    monkeypatch.setattr("some_package.some_module", "Some value")
```

More information about `monkeypatch` are available here :

- https://docs.pytest.org/en/6.2.x/monkeypatch.html

## 3.5. Which `helpers` are available ?

You can find the available helpers inside the following package/module :

- `tests/helpers/`
- `tests/conftest.py`

See also the `pytest fixtures` which are autoloaded helpers.

## 3.6. Which `fixtures` are available ?

You can list all the available `pytest fixtures` using the following command :

```
pytest --fixtures
```

More on custom fixtures here :

- [FIXTURES](FIXTURES.md)

## 3.7. Which `markers` are available ?

You can list the available markers using this command :

```
pytest --markers
```

More information on markers location are available in pytest documentation :

- https://docs.pytest.org/en/6.2.x/mark.html#mark

## 3.8. Known `issue` / `solution`

**ION Usage**

If you are testing for a `_view.py` module that uses `ION`, you would need to mock it. With Windows, the charts / graphs are unable to close the graph applications. This would make the tests failed when running with Windows.

You can use the following example below. Once you have mocked the graphs / charts, simply call the display function, which is now a mocked object.

```
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_display_defi_tvl(mocker):
    # MOCK CHARTS
    mocker.patch.object(target=llama_view.gtff, attribute="USE_ION", new=True)
    mocker.patch(target="gamestonk_terminal.stocks.options.yfinance_view.plt.ion")
    mocker.patch(target="gamestonk_terminal.stocks.options.yfinance_view.plt.show")
    llama_view.display_defi_tvl(20)
```

**YFINANCE**

If a method is using the `yfinance` library but do not let you pick the `start/end` dates it will pick the current date each time.

So your test might fail the next days.

To solve that you can filter the `start/end` date like this :

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_headers": [("User-Agent", None)],
        "filter_query_parameters": [
            ("period1", "MOCK_PERIOD_1"),
            ("period2", "MOCK_PERIOD_2"),
            ("date", "MOCK_DATE"),
        ],
    }
```

You can also refactor this method to let access to `stard/end` dates.

**USER-AGENT**

Some function uses a random `User-Agent` on the `HTTP HEADER` when fetching data from an `API`.

Here is how to filter this random `User-Agent`.

```python
@pytest.fixture(scope="module")
def vcr_config():
    return {
        "filter_headers": [("User-Agent", None)],
    }
```

**YFINANCE**

If you do something like this with `yfinance` library.

```python
import yfinance as yf

yf.download(tickers="VCYT QSI")
```

Chances are you requests will be multi-threaded.

Issues : as for now `vcrpy` seems to be incompatible with multi-threading.

The library `vcrpy` is used to record `cassettes` (`network` calls into `yaml` files).

Here is a solution to still combine `yfinance` and `vcrpy` :

```python
import pytest
import yfinance

yf_download = yf.download
def mock_yf_download(*args, **kwargs):
    kwargs["threads"] = False
    return yf_download(*args, **kwargs)

@pytest.mark.vcr
def test_ark_orders_view(kwargs_dict, mocker, use_color):
    mocker.patch("yfinance.download", side_effect=mock_yf_download)

    yf.download(tickers="VCYT QSI")
```

**BROTLI**

The library `requests` doesn't support : `brotli` comnpression.

Unless `brotli` library is installed in the environment.

Plus `brotli` is not in `GamestonkTerminal` requirements.

So if both of these conditions are fulfilled :
- one generates a cassette with `brotli` installed in his/her environment
- the server choose to send `brotli` compressed data

Then the `test` might work in local but crash during `PullRequest`.



**BEFORE_RECORD_RESPONSE**

The library `vcrpy` has a `before_record_response` which accept a filtering function that can be used to filter your cassette content :

```python
import pytest

def my_custom_filter(response):
    return response

@pytest.mark.vcr(before_record_response=my_custom_filter)
def test_simple(recorder):
    pass
```

For instance this can be used to reduce the size of a `cassette` if it's too heavy.

Or to filter sensitive data in the response.

The issue with this `before_record_response` : it isn't launched at the first run of the test.

More on this here : https://github.com/kevin1024/vcrpy/pull/594

A solution for now is to run this command while initializing the `cassettes` :
```bash
# THE SAME COMMAND NEEDS TO BE RUN TWICE
pytest tests/.../test_some_test_module.py --record-mode=once --rewrite-expected
pytest tests/.../test_some_test_module.py --record-mode=once --rewrite-expected
```

## 3.9. List of useful `vscode` tools for `unit tests`

**VSCODE TESTING**

The default testing tool from `vscode` should let you add breakpoints and run debug on a specific `test`.

It's a convenient way to see what's inside your `test` while running.

More information on this tool are available here :

- https://code.visualstudio.com/docs/python/testing

## 3.10. How to handle `dev-dependencies` ?

**UPDATE PYPROJECT**

If a library need to be added or removed from `dev-dependencies` this can be done directly in the `pyproject.toml` file.

All the `dev-dependencies` should be under the `tool.poetry.dev-dependencies` section, like in this example :

```toml
[tool.poetry.dev-dependencies]
pytest = "^6.2.2"
pylint = "^2.7.2"
flake8 = "^3.9.0"
...
```

**EXPORT REQUIREMENTS**

After updating the `pyproject.toml` you will have to export the `requirements` files using the following commands :

```bash
poetry export -f requirements.txt  -o requirements.txt --without-hashes --dev
poetry export -f requirements.txt  -o requirements-full.txt --extras prediction --without-hashes --dev
poetry export -f requirements.txt  -o discordbot/requirements.txt --extras discord --without-hashes --dev
```

# 4. Maintain `unit tests`

## 4.1. What is the PR process ? (in progress)

Here are the steps to write proper tests for Gamestonk :

    A. Find right place
    B. Verify coverage is above 90%
    C. Set the right markers

**A. Find right place**

Put the code following the same module and package structure than `gamestonk_terminal` package.

**B. Verify coverage is above 90%**

Once you made your `PullRequest` an automation will let you know whether or not you have the proper amount of tests coverage.

You can also run the following command to check your coverage manually :

- `pytest --cov --cov-fail-under=90`

**C. Set the right markers**

If parts of your have specificities like :

- being `slow`
- requiring `network` connectivity
- requiring `authentication` to `APIs`

Then the proper markers should be sent on each test.

The rests of this document is there to provide a deeper comprehension of this steps.

A tests update might be asked whenever the answer to one of the following question is `yes` :

    A. Is this PR fixing bug which was not detected by the current tests ?
    B. Is this PR reducing code coverage ?
    C. Is this PR adding features ?
    D. Is this PR updating features ?

## 4.2. Which automations are available ?

**PULL REQUEST**

A `github action` should comment every `Pull Request` with the code `coverage`.

This automation will not enforce any rules regarding `tests`.

## 4.3. How to check code coverage ?

**PULL REQUEST AUTOMATION**

You should be able to see the code coverage on comment of your `Pull Requests`.

**MANUALLY**

This is how to manually check code coverage.

```bash
pytest --cov=gamestonk_terminal --cov=terminal --cov-report term-missing
```

You can also select a specific package/module with the option `--cov`.

Here is an example where we select only the package `stocks\due_diligence` :

```bash
pytest --cov=gamestonk_terminal\stocks\due_diligence --cov-report term-missing
```

**PRE-COMMIT**

Code `coverage` and `unit tests` are not run on `pre-commit` hooks since it will slow down each commit.