jerry/OpenBB
1
0
Fork 0
mirror of https://github.com/OpenBB-finance/OpenBB.git synced 2026-10-27 04:31:13 +00:00
Code Issues Packages Projects Releases Wiki Activity

Feature/contributing guidelines (#4726)

Browse Source 
* updating readme

* updating unit tests readme

* integration tests readme

* minor to ref it on the comment

* chaging the example and removing unused files

* updating path

* changing the controller example to follow view and model

* obeying conventions

* mathcing the right controller

* matching examples

* include details on creating a new branch

* removing old references to stocks/dd

* adding sheet name to the export_data example

* circle back autocompletion to queue-and-pipeline

* fixing the list numeration

* Doc

* fixing the tests (were failing bc of changed function name)

* Remove tests/FIXTURES.md

* reverting

* small edit

* lint

* mardown linting

* fixing forecast dashboard

* fixing forecast dashboard

---------

Co-authored-by: Chavithra PARANA <chavithra@gmail.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
This commit is contained in:
Henrique Joaquim
2023-04-12 16:11:32 +01:00
committed by GitHub
parent 080d37e405
commit 495beee887
16 changed files with 520 additions and 979 deletions
Download Patch File Download Diff File Expand all files Collapse all files

308
CONTRIBUTING.md vendored
View File

@@ -10,7 +10,7 @@ Use your best judgment, and feel free to propose changes to this document in a p
- [Adding a new command](#adding-a-new-command)
- [Select Feature](#select-feature)
- [Model](#model)
- [Data source](#data-source)
- [Data sources](#data-sources)
- [View](#view)
- [Controller](#controller)
- [Add SDK endpoint](#add-sdk-endpoint)
@@ -42,11 +42,12 @@ Use your best judgment, and feel free to propose changes to this document in a p
- [Important functions and classes](#important-functions-and-classes)
- [Base controller class](#base-controller-class)
- [Default Data Sources](#default-data-sources)
- [Export Data](#export-data)
- [Queue and pipeline](#queue-and-pipeline)
- [Auto Completer](#auto-completer)
- [Logging](#logging)
- [Internationalization](#internationalization)
- [Export Data](#export-data)
- [Queue and pipeline](#queue-and-pipeline)
- [Auto Completer](#auto-completer)
- [Logging](#logging)
- [Internationalization](#internationalization)
- [Settings](#settings)
- [Write Code and Commit](#write-code-and-commit)
- [Pre Commit Hooks](#pre-commit-hooks)
- [Coding](#coding)
@@ -164,57 +165,64 @@ Note:
2. If the model requires an API key, make sure to handle the error and output relevant message.
3. If the data provider is not yet supported, you'll most likely need to do some extra steps in order to add it to the `keys` menu. See [this section](#external-api-keys) for more details.
In the example below, you can see that we explicitly handle 4 important error types:
Some of the most common error messages are:
- Error in the request (HTTP error)
- Invalid API Keys
- API Keys not authorized for Premium feature
- Empty return payload
- Invalid arguments (Optional)
It's not always possible to distinguish error types using status_code. So depending on the API provider, you can use either error messages or exception.
In the example below, you can see that we explicitly handle some of them.
It's not always possible to distinguish error types using `status_code`. So depending on the API provider, you can use either error messages or exceptions.
```python
def get_economy_calendar_events() -> pd.DataFrame:
"""Get economic calendar events
@log_start_end(log=logger)
@check_api_key(["API_NEWS_TOKEN"])
def get_news(
query: str,
limit: int = 10,
start_date: Optional[str] = None,
show_newest: bool = True,
sources: str = "",
) -> pd.DataFrame:
Returns
-------
pd.DataFrame
Get dataframe with economic calendar events
"""
current_user = get_current_user()
response = request(
f"https://finnhub.io/api/v1/calendar/economic?token={current_user.credentials.API_FINNHUB_KEY}"
)
...
df = pd.DataFrame()
link += f"&apiKey={get_current_user().credentials.API_NEWS_TOKEN}"
response = request(link)
articles = {}
if response.status_code == 200:
d_data = response.json()
if "economicCalendar" in d_data:
df = pd.DataFrame(d_data["economicCalendar"])
else:
console.print("No latest economy calendar events found\n")
response_json = response.json()
articles = (response_json["articles"] if show_newest else response_json["articles"][::-1])
elif response.status_code == 426:
console.print(f"Error in request: {response.json()['message']}", "\n")
elif response.status_code == 401:
console.print("[red]Invalid API Key[/red]\n")
elif response.status_code == 403:
console.print("[red]API Key not authorized for Premium Feature[/red]\n")
elif response.status_code == 429:
console.print("[red]Exceeded number of calls per minute[/red]\n")
else:
console.print(f"Error in request: {response.json()['error']}", "\n")
console.print(f"Error in request: {response.json()['message']}", "\n")
return df
...
```
### Data source
> Click [here](openbb_terminal/common/newsapi_model.py) to see the example in detail.
Now that we have added the model function getting, we need to specify that this is an available data source. To do so, we edit the `openbb_terminal/miscellaneous/data_sources_default.json` file. This file, described below, uses a dictionary structure to identify available sources. Since we are adding FMP to `stocks/fa/pt`, we find that entry and append it:
### Data sources
Now that we have added the model function getting, we need to specify that this is an available data source. To do so, we edit the `openbb_terminal/miscellaneous/sources/openbb_default.json` file. This file, described below, uses a dictionary structure to identify available sources. Since we are adding FMP to `stocks/fa/pt`, we find that entry and append it:
```json
"fa": {
"pt": ["BusinessInsider", "FinancialModelingPrep"],
```
If you are adding a new function with a new data source, make a new value in the file.
If you are adding a new function with a new data source, make a new value in the file. If the data source requires an
API key, please refer to the guide below for adding them. Instructions for obtaining the new api key
should be included in the file `OpenBBTerminal/website/content/terminal/usage/guides/api-keys.md`.
### View
@@ -297,15 +305,27 @@ Now that we have the model and views, it is time to add to the controller.
3. Add command and source to `print_help()`.
```python
def print_help(self):
"""Print help"""
mt = MenuText("stocks/dps/")
def print_help(self):
"""Print help."""
mt = MenuText("stocks/fa/")
mt.add_cmd("load")
mt.add_raw("\n")
mt.add_cmd("shorted")
mt.add_param("_ticker", self.ticker.upper())
mt.add_raw("\n")
mt.add_info("_company_overview")
mt.add_cmd("mktcap")
mt.add_cmd("overview")
mt.add_cmd("divs", not self.suffix)
...
mt.add_cmd("pt")
mt.add_cmd("dcf")
mt.add_cmd("dcfc")
console.print(text=mt.menu_text, menu="Stocks - Fundamental Analysis")
```
4. If there is a condition to display or not the command, this is something that can be leveraged through this `add_cmd` method, e.g. `mt.add_cmd("shorted", self.ticker_is_loaded)`.
4. If there is a condition to display or not the command, this is something that can be leveraged through the `add_cmd` method, e.g. `mt.add_cmd("divs", not self.suffix)`.
5. Add command description to file `i18n/en.yml`. Use the path and command name as key, e.g. `stocks/fa/pt` and the value as description. Please fill in other languages if this is something that you know.
@@ -356,7 +376,7 @@ Our new function will be:
self.custom_load_wrapper([self.ticker])
if ns_parser.source == "BusinessInsider":
business_insider_view.price_target_from_analysts(
business_insider_view.display_price_target_from_analysts(
symbol=self.ticker,
data=self.stock,
start_date=self.start,
@@ -430,7 +450,7 @@ def call_fa(self, _):
The **import only occurs inside this menu call**, this is so that the loading time only happens here and not at the terminal startup. This is to avoid slow loading times for users that are not interested in `stocks/fa` menu.
In addition, note the `self.load_class` which allows to not create a new DarkPoolShortsController instance but re-load the previous created one. Unless the arguments `self.ticker, self.start, self.stock` have changed since. The `self.queue` list of commands is passed around as it contains the commands that the terminal must perform.
In addition, note the `self.load_class` which allows to not create a new `FundamentalAnalysisController` instance but re-load the previous created one. Unless the arguments `self.ticker`, `self.start` or `self.stock` have changed since. The `self.queue` list of commands is passed around as it contains the commands that the terminal must perform.
### Add SDK endpoint
@@ -453,17 +473,31 @@ In order to add a command to the SDK, follow these steps:
trail,model,view
stocks.fa.analyst,stocks_fa_finviz_model.get_analyst_data,
stocks.fa.rot,stocks_fa_finnhub_model.get_rating_over_time,stocks_fa_finnhub_view.rating_over_time
...
```
In this file, the trail represents the path to the function to be called. The model represents the import alias we gave to the `_model` file. The view represents the import alias we gave to the `_view` file.
3. Add your new function to this structure. In our example of the `shorted` function, our trail would be `stocks.dps.shorted`.
The model is the import alias to the `_model` function that was written: `stocks_dps_yahoofinance_model.get_most_shorted`.
The view is the import alias to the `_view` function that was written: `stocks_dps_yahoofinance_view.display_most_shorted`.
The added line of the file should look like this:
3. Add your new function to this structure. In the below example of the `pt` function, our trail would be `stocks.fa.pt`.
Our naming convention is such that the data source should not be included in the trail. In this example, calling a new function `pt_fmp` would not be allowed.
For functions with multiple sources, there should be a single `pt` function that takes in the source as an argument.
In the following example, we will stick with showing how the business_insider was initially added to the sdk.
The model is the import alias to the `_model` function that was written:
- `stocks_fa_business_insider_model.get_price_target_from_analysts`
The view is the import alias to the `_view` function that was written:
- `stocks_fa_business_insider_view.display_price_target_from_analysts`
The added line of the file should look like this:
```csv
stocks.dps.shorted,stocks_dps_yahoofinance_model.get_most_shorted,stocks_dps_yahoofinance_view.display_most_shorted
stocks.fa.pt,stocks_fa_business_insider_model.get_price_target_from_analysts,stocks_fa_business_insider_view.display_price_target_from_analysts
```
4. Generate the SDK files by running `python generate_sdk.py` from the root of the project. This will automatically generate the SDK `openbb_terminal/sdk.py`, corresponding `openbb_terminal/core/sdk/controllers/` and `openbb_terminal/core/sdk/models/` class files.
@@ -484,15 +518,25 @@ A thorough introduction on the usage of unit tests and integration tests in Open
[Unit Test README](tests/README.md)
[Integration Test README](scripts/README.md)
[Integration Test README](openbb_terminal/miscellaneous/integration_tests_scripts/README.MD)
Any new features that do not contain unit tests will not be accepted.
### Open a Pull Request
For starters, you should ensure that you branch is up to date with the `develop` branch. To do that, one can run the following commands:
```bash
git fetch upstream
git checkout develop
git merge upstream/develop
```
After that, you can create a new branch for your feature. E.g. `git checkout -b feature/AmazingFeature`.
Once you're happy with what you have, push your branch to remote. E.g. `git push origin feature/AmazingFeature`.
> Note that we follow gitflow naming convention, so your branch name should be prefixed with `feature/` or `hotfix/` depending on the type of work you are doing.
> Note that we follow gitflow naming convention, so your branch name should be prefixed with `feature/` or `hotfix/` depending on the type of work you are doing. To learn more, please refer to [Branch Naming Conventions](#branch-naming-conventions).
A user may create a **Draft Pull Request** when there is the intention to discuss implementation with the team.
@@ -550,11 +594,6 @@ If there are sub-categories, the layout will be: `/<context>/<category>/<sub-cat
```text
openbb_terminal/stocks/stocks_controller.py
/stocks_helper.py
/due_diligence/dd_controller.py
/marketwatch_view.py
/marketwatch_model.py
/finviz_view.py
/finviz_model.py
/technical_analysis/ta_controller.py
/tradingview_view.py
/tradingview_model.py
@@ -574,11 +613,6 @@ With:
| :---------- | :-------------------- | :--------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `stocks/` | | `stocks_controller.py` | Manages **stocks** _context_ from a user perspective, i.e. routing _commands_ and arguments to output data, or, more importantly, redirecting to the selected _category_. |
| `stocks/` | | `stocks_helper.py` | Helper to `stocks` menu. This file is meant to hold generic purpose `stocks` functionalities. |
| `stocks/` | `due_diligence/` | `dd_controller.py` | Manages **due_diligence** _category_ from **stocks** _context_ from a user perspective, i.e. routing _commands_ and arguments to output data.
| `stocks/` | `due_diligence/` | `marketwatch_view.py` | This file contains functions that rely on **Market Watch** data. These functions represent _commands_ that belong to **due_diligence** _category_ from **stocks** _context_. These functions are called by `dd_controller.py` using the arguments given by the user and will output either a string, table or plot.
| `stocks/` | `due_diligence/` | `marketwatch_model.py` | This file contains functions that rely on **Market Watch** data. These functions represent _commands_ that belong to **due_diligence** _category_ from **stocks** _context_. These functions are called by `marketwatch_view.py` and will return data to be processed in either a string, dictionary or dataframe format. |
| `stocks/` | `due_diligence/` | `finviz_view.py` | This file contains functions that rely on **Finviz** data. These functions represent _commands_ that belong to **due_diligence** _category_ from **stocks** _context_. These functions are called by `dd_controller.py` using the arguments given by the user and will output either a string, table or plot. |
| `stocks/` | `due_diligence/` | `finviz_model.py` | This file contains functions that rely on **Finviz** data. These functions represent _commands_ that belong to **due_diligence** _category_ from **stocks** _context_. These functions are called by `finviz_view.py` and will return data to be processed in either a string, dictionary or dataframe format. |
| `stocks/` | `technical_analysis/` | `ta_controller.py` | Manages **technical_analysis** _category_ from **stocks** _context_ from a user perspective, i.e. routing _commands_ and arguments to output data.
| `stocks/` | `technical_analysis/` | `tradingview_view.py` | This file contains functions that rely on **TradingView** data. These functions represent _commands_ that belong to **technical_analysis** _category_ from **stocks** _context_. These functions are called by `ta_controller.py` using the arguments given by the user and will output either a string, table or plot. |
| `stocks/` | `technical_analysis/` | `tradingview_model.py` | This file contains functions that rely on **TradingView** data. These functions represent _commands_ that belong to **technical_analysis** _category_ from **stocks** _context_. These functions are called by `tradingview_view.py` and will return data to be processed in either a string, dictionary or dataframe format. |
@@ -1171,33 +1205,14 @@ It is important to keep a coherent UI/UX throughout the terminal. These are the
OpenBB Terminal currently has over 100 different data sources. Most of these require an API key that allows access to some free tier features from the data provider, but also paid ones.
When a new API data source is added to the platform, it must be added through [credentials_model.py](/openbb_terminal/credentials_model.py) under the section that resonates the most with its functionality, from: `Data providers`, `Socials` or `Brokers`.
When a new API data source is added to the platform, it must be added through [credentials_model.py](/openbb_terminal/core/models/credentials_model.py).
Example (a section from [credentials_model.py](/openbb_terminal/credentials_model.py)):
In order to do that, you'll simply need to choose from one the following files:
```python
# Data providers
API_DATABENTO_KEY = "REPLACE_ME"
API_KEY_ALPHAVANTAGE: str = "REPLACE_ME"
API_KEY_FINANCIALMODELINGPREP: str = "REPLACE_ME"
1. [local_credentials.json](openbb_terminal/miscellaneous/models/local_credentials.json) --> credentials that should only be stored locally and not pushed to the [OpenBB Hub](https://my.openbb.co/) like brokerage keys or other very sensitive or personal to the user.
2. [hub_credentials.json](openbb_terminal/miscellaneous/models/hub_credentials.json) --> credentials that should be stored in the [OpenBB Hub](https://my.openbb.co/) like API keys to access your favorite providers.
...
# Socials
API_GITHUB_KEY: str = "REPLACE_ME"
API_REDDIT_CLIENT_ID: str = "REPLACE_ME"
API_REDDIT_CLIENT_SECRET: str = "REPLACE_ME"
...
# Brokers or data providers with brokerage services
RH_USERNAME: str = "REPLACE_ME"
RH_PASSWORD: str = "REPLACE_ME"
DG_USERNAME: str = "REPLACE_ME"
...
```
> Note: By differentiating between local and hub credentials, we can ensure that the user's credentials are not pushed to the [OpenBB Hub](https://my.openbb.co/) and are only stored locally. This does not mean that the credentials are not secure in the OpenBB Hub, but rather that the user can choose to store them locally if they wish.
### Setting and checking API key
@@ -1324,40 +1339,31 @@ Worthy methods to mention are:
## Default Data Sources
The document [data_sources_default.json](/data_sources_default.json) contains all data sources that the terminal has access to and specifies the data source utilized by default for each command.
The document [openbb_default.json](openbb_terminal/miscellaneous/sources/openbb_default.json) contains all data sources that the terminal has access to and specifies the data source utilized by default for each command.
The convention is as follows:
```python
{
"stocks": {
"search": ["FinanceDatabase"],
"quote": ["YahooFinance"],
"candle": [],
"load": [
"YahooFinance",
"AlphaVantage",
"Polygon",
"EODHD"
],
"options": {
"unu": ["FDScanner"],
"calc": [],
"screen": {
"view": ["Syncretism"],
"set": [],
"scr": ["Syncretism"]
},
"load": [
"YahooFinance",
"Tradier",
"Nasdaq"
],
"exp": [
"YahooFinance",
"Tradier",
"Nasdaq"
],
"stocks": {
"search": [
"FinanceDatabase"
],
"quote": [
"FinancialModelingPrep"
],
"tob": [
"CBOE"
],
"candle": [],
"codes": [
"Polygon"
],
"news": [
"Feedparser",
"NewsApi",
"Ultima"
],
...
```
@@ -1370,7 +1376,9 @@ The way to interpret this file is by following the path to a data source, e.g.
- `stoks/options/unu` relies on `FDScanner`.
- `stocks/options/exp` relies on `YahooFinance` by default but `Tradier` and `Nasdaq` sources are allowed.
### Export Data
> Note: The default data sources can be changed directly in the [OpenBB Hub](https://my.openbb.co/) by the user and automatically synchronized with the terminal on login.
## Export Data
In the `_view.py` files it is common having at the end of each function `export_data` being called. This typically looks like:
@@ -1378,9 +1386,10 @@ In the `_view.py` files it is common having at the end of each function `export_
export_data(
export,
os.path.dirname(os.path.abspath(__file__)),
"contracts",
df_contracts,
figure=fig_contracts,
"pt",
df_analyst_data,
sheet_name,
fig,
)
```
@@ -1393,9 +1402,10 @@ Let's go into each of these arguments:
- `os.path.dirname(os.path.abspath(__file__))` corresponds to the directory path
- This is important when `export folder` selected is the default because the data gets stored based on where it is called.
- If this is called from a `common` folder, we can use `os.path.dirname(os.path.abspath(__file__)).replace("common", "stocks")` insteaad
- `"contracts"` corresponds to the name of the exported file (+ unique datetime) if the user doesn't provide one
- `df_contracts` corresponds to the dataframe with data.
- `figure=fig_contracts` corresponds to the figure to be exported as an image or pdf.
- `"pt"` corresponds to the name of the exported file (+ unique datetime) if the user doesn't provide one
- `df_analyst_data` corresponds to the dataframe with data.
- `sheet_name` corresponds to the name of the sheet in the excel file.
- `fig` corresponds to the figure to be exported as an image or pdf.
If `export_allowed=EXPORT_BOTH_RAW_DATA_AND_FIGURES` in `parse_known_args_and_warn`, valid examples are:
@@ -1408,7 +1418,7 @@ Note that these files are saved on a location based on the environment variable:
The default location is the `exports` folder and the data will be stored with the same organization of the terminal. But, if the user specifies the name of the file, then that will be dropped onto the folder as is with the datetime attached.
### Queue and pipeline
## Queue and pipeline
The variable `self.queue` contains a list of all actions to be run on the platform. That is the reason why this variable is always passed as an argument to a new controller class and received back.
@@ -1434,10 +1444,28 @@ And the user goes into the `stocks` menu and runs `load AAPL`. Then the queue is
At that point the user goes into the `dps` menu and runs the command `psi` with the argument `-l 90` therefore displaying price vs short interest of the past 90 days.
### Auto Completer
## Auto Completer
In order to help users with a powerful autocomplete, we have implemented our own (which can be found [here](/openbb_terminal/custom_prompt_toolkit.py)).
The queue, discussed in the previous section [Queue and pipeline](#queue-and-pipeline), is expected to link together with the autocompletion in order to provide the user with the available options for each command.
Here is an example of how it will look like:
```bash
2023 Apr 11, 11:41 (🦋) /stocks/dps/ $ psi
--nyse
--help
-h
--export
--raw
--limit
-l
--source
```
> Where `nyse`, `help`, `h`, `export`, `raw`, `limit`, `l` and `source` are the available options for the `psi` command.
> Those are selectable using the arrow keys and the `tab` key.
The list of options for each command is automatically generated, if you're interested take a look at its implementation [here](/openbb_terminal/core/completer/choices.py).
To leverage this functionality, you need to add the following line to the top of the desired controller:
@@ -1513,7 +1541,7 @@ This method should only be called when the user's state changes leads to the aut
In this case, this method is called as soon as the user successfully loads a new ticker since the options expiry dates vary based on the ticker. Note that the completer is recreated from it.
### Logging
## Logging
A logging system is used to help tracking errors inside the OpenBBTerminal.
@@ -1559,7 +1587,7 @@ def your_function() -> pd.DataFrame:
>
> **Disclaimer**: all the user paths, names, IPs, credentials and other sensitive information are anonymized, [take a look at how we do it](/openbb_terminal/core/log/generation/formatter_with_exceptions.py).
### Internationalization
## Internationalization
WORK IN PROGRESS - The menu can be internationalised BUT we do not support yet help commands`-h` internationalization.
@@ -1589,6 +1617,46 @@ This is the convention in use for creating a new key/value pair:
- `stocks/_ticker` - Under `stocks` context, `_ticker` is used as a key of a parameter, and the displayed parameter description is given as value
- `crypto/dd/_tokenomics_` - Under `crypto` context and under `dd` menu, `_tokenomics_` is used as a key of an additional information, and the displayed information is given as value
## Settings
The majority of the settings used in the OpenBB Terminal are handled using [Pydantic Dataclasses](https://docs.pydantic.dev/usage/dataclasses/).
Some examples are:
1. [SystemModel](openbb_terminal/core/models/system_model.py)
2. [UserModel](openbb_terminal/core/models/user_model.py)
3. [CredentialsModel](openbb_terminal/core/models/credentials_model.py)
4. ...
This means that the settings are pretty much validated and documented automatically, as well as centralized in a single place.
This allows us to develop faster, efficiantly and with predictability.
Depending on your use case you'll most likely need to interact with these dataclasses or expand them with new settings.
**Disclaimer**: avoid at all costs to use the `os.environ` or `os.getenv` methods to retrieve settings. Settings should be retrieved using the appropriate methods from the respective class.
Here is an example of **accessing** a setting:
```python
from openbb_terminal.core.session.current_system import get_current_system
system = get_current_system()
system = get_current_system()
print(system.VERSION)
# 3.0.0rc1
```
And here is an example of **changing** a setting:
```python
from openbb_terminal.core.session.current_system import get_current_system
set_system_variable("TEST_MODE", True)
```
## Write Code and Commit
At this stage it is assumed that you have already forked the project and are ready to start working.

4
openbb_terminal/core/sdk/models/stocks_sdk_model.py
View File

@@ -324,7 +324,9 @@ class StocksFundamentalAnalysis(Category):
self.overview = lib.stocks_fa_av_model.get_overview
self.profile = lib.stocks_fa_fmp_model.get_profile
self.pt = lib.stocks_fa_business_insider_model.get_price_target_from_analysts
self.pt_chart = lib.stocks_fa_business_insider_view.price_target_from_analysts
self.pt_chart = (
lib.stocks_fa_business_insider_view.display_price_target_from_analysts
)
self.rating = lib.stocks_fa_fmp_model.get_rating
self.ratios = lib.stocks_fa_fmp_model.get_key_ratios
self.revfc = lib.stocks_fa_seeking_alpha_model.get_estimates_rev

2
openbb_terminal/core/sdk/trail_map.csv vendored
View File

@@ -465,7 +465,7 @@ stocks.fa.mktcap,stocks_fa_yahoo_finance_model.get_mktcap,stocks_fa_yahoo_financ
stocks.fa.news,stocks_fa_finviz_model.get_news,
stocks.fa.overview,stocks_fa_av_model.get_overview,
stocks.fa.profile,stocks_fa_fmp_model.get_profile,
stocks.fa.pt,stocks_fa_business_insider_model.get_price_target_from_analysts,stocks_fa_business_insider_view.price_target_from_analysts
stocks.fa.pt,stocks_fa_business_insider_model.get_price_target_from_analysts,stocks_fa_business_insider_view.display_price_target_from_analysts
stocks.fa.rating,stocks_fa_fmp_model.get_rating,
stocks.fa.ratios,stocks_fa_fmp_model.get_key_ratios,
stocks.fa.revfc,stocks_fa_seeking_alpha_model.get_estimates_rev,
1 trail model view
465 stocks.fa.news stocks_fa_finviz_model.get_news
466 stocks.fa.overview stocks_fa_av_model.get_overview
467 stocks.fa.profile stocks_fa_fmp_model.get_profile
468 stocks.fa.pt stocks_fa_business_insider_model.get_price_target_from_analysts stocks_fa_business_insider_view.price_target_from_analysts stocks_fa_business_insider_view.display_price_target_from_analysts
469 stocks.fa.rating stocks_fa_fmp_model.get_rating
470 stocks.fa.ratios stocks_fa_fmp_model.get_key_ratios
471 stocks.fa.revfc stocks_fa_seeking_alpha_model.get_estimates_rev

43
openbb_terminal/economy/finnhub_model.py
View File

@@ -1,43 +0,0 @@
import logging
import pandas as pd
from openbb_terminal.core.session.current_user import get_current_user
from openbb_terminal.decorators import check_api_key, log_start_end
from openbb_terminal.helper_funcs import request
from openbb_terminal.rich_config import console
logger = logging.getLogger(__name__)
@log_start_end(log=logger)
@check_api_key(["API_FINNHUB_KEY"])
def get_economy_calendar_events() -> pd.DataFrame:
"""Get economic calendar events
Returns
-------
pd.DataFrame
Get dataframe with economic calendar events
"""
response = request(
f"https://finnhub.io/api/v1/calendar/economic?token={get_current_user().credentials.API_FINNHUB_KEY}"
)
df = pd.DataFrame()
if response.status_code == 200:
d_data = response.json()
if "economicCalendar" in d_data:
df = pd.DataFrame(d_data["economicCalendar"])
else:
console.print("No latest economy calendar events found\n")
elif response.status_code == 401:
console.print("[red]Invalid API Key[/red]\n")
elif response.status_code == 403:
console.print("[red]API Key not authorized for Premium Feature[/red]\n")
else:
console.print(f"Error in request: {response.json()['error']}", "\n")
return df

93
openbb_terminal/economy/finnhub_view.py
View File

@@ -1,93 +0,0 @@
import logging
import os
from typing import Optional
from openbb_terminal.decorators import check_api_key, log_start_end
from openbb_terminal.economy import finnhub_model
from openbb_terminal.helper_funcs import export_data, print_rich_table
from openbb_terminal.rich_config import console
logger = logging.getLogger(__name__)
@log_start_end(log=logger)
@check_api_key(["API_FINNHUB_KEY"])
def economy_calendar_events(
country: str,
limit: int,
impact: str,
export: str = "",
sheet_name: Optional[str] = None,
):
"""Output economy calendar impact events. [Source: Finnhub]
Parameters
----------
country : str
Country from where to get economy calendar impact events
limit : int
Number economy calendar impact events to display
impact : str
Impact of the economy event
export : str
Export dataframe data to csv,json,xlsx file
"""
df_events = finnhub_model.get_economy_calendar_events()
if df_events.empty:
return
df_econ_calendar = df_events[df_events["country"] == country].sort_values(
"time", ascending=True
)
if df_econ_calendar.empty:
console.print(
"No latest economy calendar events found in the specified country\n"
)
return
if impact != "all":
df_econ_calendar = df_econ_calendar[df_econ_calendar["impact"] == impact]
if df_econ_calendar.empty:
console.print(
"No latest economy calendar events found in the specified country with this impact\n"
)
return
df_econ_calendar = df_econ_calendar.fillna("").head(n=limit)
d_econ_calendar_map = {
"actual": "Actual release",
"prev": "Previous release",
"country": "Country",
"unit": "Unit",
"estimate": "Estimate",
"event": "Event",
"impact": "Impact Level",
"time": "Release time",
}
df_econ_calendar = df_econ_calendar[
["time", "event", "impact", "prev", "estimate", "actual", "unit"]
].rename(columns=d_econ_calendar_map)
df_econ_calendar.replace("", float("NaN"), inplace=True)
df_econ_calendar.dropna(how="all", axis=1, inplace=True)
print_rich_table(
df_econ_calendar,
headers=list(df_econ_calendar.columns),
show_index=False,
title="Economy Calendar",
export=bool(export),
)
export_data(
export,
os.path.dirname(os.path.abspath(__file__)),
"events",
df_econ_calendar,
sheet_name,
)

119
openbb_terminal/miscellaneous/integration_tests_scripts/README.MD vendored Normal file
View File

@@ -0,0 +1,119 @@
# Integration tests
## Table of Contents
- [Integration tests](#integration-tests)
- [Table of Contents](#table-of-contents)
- [What is it?](#what-is-it)
- [OpenBB Terminal integration tests](#openbb-terminal-integration-tests)
- [How to list the available tests](#how-to-list-the-available-tests)
- [How to run a test](#how-to-run-a-test)
- [Run all tests](#run-all-tests)
- [Run a specific test](#run-a-specific-test)
- [Custom flags](#custom-flags)
## What is it?
Integration tests are a type of software testing that is used to validate the interactions between different components or modules of an application. Unlike unit tests, which test individual pieces of code in isolation, integration tests verify that the system's different components work together correctly.
The purpose of integration tests is to identify problems that arise when the various parts of an application are integrated. Integration tests help to ensure that the software works as a whole and that different modules can communicate and exchange data correctly.
Integration tests are often used to test the following:
- Integration between different software modules or services
- Integration between an application and external dependencies such as databases, APIs, and other third-party services
- End-to-end functionality of the entire system
Integration tests can be automated or performed manually. They are typically more complex and take longer to run than unit tests, as they require the integration of multiple components. However, integration tests are an essential part of a comprehensive testing strategy and can help to identify issues that may be missed by unit tests.
## OpenBB Terminal integration tests
Those can be found and organized by menu as follows:
- `stocks` menu: `openbb_terminal/miscellaneous/integration_tests_scripts/stocks/`, tests within the folder:
- `openbb_terminal/miscellaneous/integration_tests_scripts/stocks/test_stocks_ba.openbb`
- `openbb_terminal/miscellaneous/integration_tests_scripts/stocks/test_stocks_qa.openbb`
- ...
- `economy` menu: `openbb_terminal/miscellaneous/integration_tests_scripts/economy/`, tests within the folder:
- `openbb_terminal/miscellaneous/integration_tests_scripts/economy/test_economy.openbb`
- ...
The structure of each test should be a combination of the commands found in the terminal, here is an example you can find [here](openbb_terminal/miscellaneous/integration_tests_scripts/stocks/test_stocks_options_payoff.openbb):
```bash
stocks
options
load aapl
exp 0
payoff
list
pick long
add 20
add 15
add 10 -p
sop
plot
exit
```
Output:
```bash
* Collected 69 script(s)...
* Skipping 0 script(s)...
Available scripts:
0. account/test_account.openbb
1. alternative/test_alt_covid.openbb
2. alternative/test_alt_oss.openbb
3. alternative/test_alt_realestate.openbb
4. cryptocurrency/test_crypto.openbb
5. cryptocurrency/test_crypto_dd.openbb
6. cryptocurrency/test_crypto_defi.openbb
7. cryptocurrency/test_crypto_disc.openbb
8. cryptocurrency/test_crypto_nft.openbb
9. cryptocurrency/test_crypto_onchain.openbb
10. cryptocurrency/test_crypto_ov.openbb
...
```
### How to list the available tests
```bash
python terminal.py -t -l
```
### How to run a test
#### Run all tests
```bash
python terminal.py -t
```
#### Run a specific test
```bash
python terminal.py -t 2
```
> Here the number `2` is the index of the test you want to run, you can find the index by running `python terminal.py -t -l`
#### Custom flags
We can add the following flags to either run all tests or a specific test:
- `subproc` to choose the number of subprocesses to run the tests in parallel.
```bash
python terminal.py -t 2 --subproc 2
```
- `verbose` to print the output of the tests inside the OpenBB Terminal.
```bash
python terminal.py -t 2 --subproc 0 --verbose
```

2
openbb_terminal/sdk.py
View File

@@ -36,7 +36,7 @@ class OpenBBSDK:
Attributes:
`login`: Login and load user info.\n
`logout`: Logout and clear session.\n
`news`: Get news for a given term and source. [Source: Feedparser]\n
`news`: Get news for a given term and source. [Source: Ultima Insights News Monitor]\n
`whoami`: Display user info.\n
"""

4
openbb_terminal/stocks/fundamental_analysis/business_insider_view.py
View File

@@ -68,7 +68,7 @@ def display_management(symbol: str, export: str = "", sheet_name: Optional[str]
# pylint: disable=R0913
@log_start_end(log=logger)
def price_target_from_analysts(
def display_price_target_from_analysts(
symbol: str,
data: Optional[DataFrame] = None,
start_date: Optional[str] = None,
@@ -192,7 +192,7 @@ def price_target_from_analysts(
@log_start_end(log=logger)
def estimates(
def display_estimates(
symbol: str, estimate: str, export: str = "", sheet_name: Optional[str] = None
):
"""Display analysts' estimates for a given ticker. [Source: Business Insider]

4
openbb_terminal/stocks/fundamental_analysis/fa_controller.py
View File

@@ -1806,7 +1806,7 @@ class FundamentalAnalysisController(StockBaseController):
return
if ns_parser.source == "BusinessInsider":
business_insider_view.price_target_from_analysts(
business_insider_view.display_price_target_from_analysts(
symbol=self.ticker,
data=self.stock,
start_date=self.start,
@@ -1863,7 +1863,7 @@ class FundamentalAnalysisController(StockBaseController):
console.print(no_ticker_message)
return
business_insider_view.estimates(
business_insider_view.display_estimates(
symbol=self.ticker,
estimate=ns_parser.estimate,
export=ns_parser.export,

387
tests/FIXTURES.md vendored
View File

@@ -1,387 +0,0 @@
# 1. OpenBB Terminal : `Unit Testing` - `Fixtures`
In this document we will list custom `pytest fixtures` built in OpenBBTerminal and explain how they works.
## 1.1. What is a `fixture` ?
This part is fully documented so you can take a look directly on `pytest` documentation right here :
https://docs.pytest.org/en/6.2.x/fixture.html
## 1.2. Which `fixtures` are available ?
You can list all the available `pytest fixtures` using the following command :
```
pytest --fixtures
```
This will include the :
- `fixtures` available by default in `pytest`
- `fixtures` defined by installed `pytest plugins`
- custom `fixtures` built specially for this OpenBBTerminal
## 1.3. Which are the custom `fixtures` ?
The fixtures built for OpenBBTerminal are here to : make sure contributors are putting `test data file` in the right location.
Here we call `test data file` the files that you might use to store data.
For instance if you want to test a `function` generating a `Pandas DataFrame`, you might want to :
- store expected `DataFrame` result inside a `csv`
- then compare this expected result with the result of that `function`
Here is a table listing the fixtures specifically built for this library :
|**Fixture**|**Purpose**|
|:-|:-|
|default_csv_path|Builds a default location to store a `csv` file related to a test.|
|default_txt_path|Builds a default location to store a `txt` file related to a test.|
|default_json_path|Builds a default location to store a `json` file related to a test.|
|record_stdout|Capture stdout in a test. And compare it to the last recording.|
|recorder|Allow capturing one or multiple variable(s) in a test. And compare this captured variable(s) to the last recording|
Note that you can't combine `record_stdout` and `recorder` fixtures in the same test.
# 2. Path building `fixtures`
## 2.1. What is the purpose of these `fixtures` ?
These `fixtures` will provide a location to store your `data file`.
The folder to store the `data file` will be generated automatically.
The file will be store :
- inside the same folder than the `test module`
- inside a folder with the data type : for instance `txt` if you are storing text data
- inside a file with the name of the `test function` and with the right extension : `.txt` if its a text file.
Suppose `XXX` is your data type, the generated path will look like this :
- module_path/`XXX`/test_module_name/test_function_name.`XXX`
This mimic the default location for `vcrpy cassettes`.
More on `vcrpy` here :
https://vcrpy.readthedocs.io/en/latest/
## 2.2. How to use `default_csv_path` ?
```python
import pandas as pd
from tested_module import function_generating_a_dataframe
def test_function(default_csv_path):
result_dataframe = function_generating_a_dataframe()
result_dataframe.to_csv(default_csv_path)
expected_dataframe = result_dataframe.read_csv(default_csv_path)
pd.testing.assert_frame_equal(result_dataframe, expected_dataframe)
```
You might want to comment the file saving part after the first run.
Or find a way to conditionally disable it.
## 2.3. How to use `default_txt_path` ?
```python
from tested_module import function_generating_a_str
def test_function(default_txt_path):
result_txt = function_generating_a_str()
with open(file=default_txt_path, mode="w", encoding="utf-8") as f:
f.write(result_txt)
with open(file=default_txt_path, encoding="utf-8") as f:
expected_txt = f.read()
assert result_txt == expected_txt
```
## 2.4. How to use `default_json_path` ?
```python
import json
from tested_module import function_generating_a_json
def test_function(default_json_path):
result_json = function_generating_a_json()
with open(file=default_txt_path, mode="w", encoding="utf-8") as f:
json.dump(result_json, f)
with open(file=default_json_path, encoding="utf-8") as f:
expected_json = json.load(f)
assert result_json == expected_json
```
# 3. Recording fixtures
## 3.1. How to use the `record_stdout fixture` ?
**USAGE**
Example of usage :
```python
import pytest
@pytest.mark.record_stdout
def test_function():
print("Something")
```
This will generate a text file to store the `printed` output.
The content of this file will be compared to the `test_function` output at each execution of the test.
**RECORD ONCE**
You can run this command to generate the text file :
```bash
pytest --record-mode=once
```
This works if :
- the text file was not generated yet
- you remove the text file
**RECORD REWRITE**
You can run this command to regenerate the text file :
```bash
pytest --record_mode=once --rewrite-expected
```
**REWRITE EXPECTED**
This will force `record_stdout` to rewrite any changed file (`txt`).
This will not rewrite the cassettes.
Example :
```
pytest --rewrite-expected
```
**VCR**
You can combine `record_stdout` and `vcr` fixtures, like this :
```python
import pytest
import requests
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_function():
response = request('https://api.github.com/user', auth=('user', 'pass'))
print(response.status_code)
print(response.text)
```
**ASSERT IN LIST**
You are not forced to save a text file, you can use a list of texts instead like this :
```python
import pytest
@pytest.mark.record_stdout(
assert_in_list=["Some text", "Another text"],
save_record=False,
)
def test_function():
print("""
This text contains :
- Some text
- Another text
""")
```
**STRIP**
If your `test_function` output a random number of blank before or after you can `strip` the captured data.
Example :
```python
import pytest
import random
@pytest.mark.record_stdout(strip=True)
def test_function():
some_text = "Some text output"
random_int = random.randint(0,10)
for _ in range(random_int):
text += " "
print(some_text)
```
**RECORD MODE**
It is possible to programmatically change the `record_mode` on a `test`.
Example :
```python
import pytest
@pytest.mark.record_stdout(record_mode="rewrite")
def test_function():
some_text = "Some text output"
print(some_text)
```
## 3.2. How to use the `recorder fixture` ?
**BEWARE**
You can't combine these two fixtures :
- record_stdout
- recorder
**USAGE**
Example of usage :
```python
import pandas as pd
import pytest
@pytest.mark.record_stdout
def test_function(recorder):
some_dict = {1, 2, 3}
some_list = [7, 8, 9 ]
some_tuple = (4, 5, 6)
some_string = "Some string"
some_complex_tuple = (pd.DataFrame(), some_dict)
recorder.capture(some_dict)
recorder.capture(some_list)
recorder.capture(some_string)
recorder.capture(some_tuple)
recorder.capture_list(some_complex_tuple)
```
This will generate one or multiple text file(s) to store the `captured` variables.
The content of the file(s) will be compared to the `test_function` output at each execution of the test.
**RECORD ONCE**
You can run this command to generate the text file :
```bash
pytest --record-mode=once
```
This works if :
- the text file was not generated yet
- you remove the text file
**RECORD REWRITE**
You can run this command to regenerate the text file :
```bash
pytest --record-mode=rewrite
```
**REWRITE EXPECTED**
This will force `recorder` to rewrite any changed file (`csv`, `json`, `txt`).
This will not rewrite the cassettes.
Example :
```
pytest --record_mode=once --rewrite-expected
```
**VCR**
You can combine `recorder` and `vcr` fixtures, like this :
```python
import requests
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_function():
response = request('https://api.github.com/user', auth=('user', 'pass'))
recorder.capture(response.status_code)
recorder.capture(response.text)
```
**STRIP**
If your `test_function` output a random number of blank before or after you can `strip` the captured data.
Example :
```python
import random
def test_function(recorder):
some_text = "Some text output"
random_int = random.randint(0,10)
for _ in range(random_int):
text += " "
recorder.capture(some_text, strip=True)
```
**RECORD MODE**
It is possible to programmatically change the `record_mode` on a `test`.
Example :
```python
def test_function(recorder):
some_text = "Some text output"
recorder.record_mode = "rewrite"
recorder.capture(some_text)
```
**EXTRA ARGUMENTS - PD.TO_CSV**
If you capture a DataFrame it will be exported as a CSV using the function `pandas.DataFrame.to_csv`.
This function contains arguments that can be useful to the test builder.
Everything you pass as extra `kwargs` to the `capture` function will be passed to this `to_csv` function.
Example :
```python
import random
def test_function(recorder):
table_of_floats = pd.DataFrame(
np.random.uniform(low=1, high=10, size=(5,)),
columns=["numbers"],
)
recorder.capture(table_of_floats, float_format="%.2f")
# This will truncate the numbers
```
**EXTRA ARGUMENTS - JSON.DUMPS**
If you capture a TUPLE, LIST, DICT... it will be exported as a json using the function `json.dumps`.
This function contains arguments that can be useful to the test builder.
Everything you pass as extra `kwargs` to the `capture` function will be passed to this `dumps` function.

336
tests/README.md vendored
View File

@@ -1,53 +1,129 @@
# 1. OpenBB Terminal : `Unit Testing`
# 1. OpenBB Terminal Unit Testing
This document is part of the `OpenBB Terminal` library documentation.
This document is an annex of the OpenBB Terminal Contributing Guidelines.
It aims to provide necessary information in order to:
It provide the necessary information to build, run and maintain Unit Tests for OpenBB Terminal.
- Run `unit tests`
- Build `unit tests`
- Fix failed existing `unit tests`
- Maintain `unit tests`
## 1.1. Unit Tests
## 1.1. Why having unit tests ?
Unit Tests are early filters, having them working doesn't mean the function works.
`Unit tests` purpose is to allow developers to update without fear.
But it means the function will work with the most common use cases : the tested ones.
Writing tests that are easy to run will help the person modifying the code to do so quick and with more confidence. Having tests available also decreases the chances that bugs will make it into production and break other parts of the code.
## 1.2. Rules
To do that we need `unit tests` to be :
Here are few rules to follow when you write Unit Tests for OpenBB Terminal.
- fast to write
- fast to run
These rules are these to avoiding having one Contributor work negatively impact another Contributor.
**FAST TO WRITE.**
Thus keeping a good Contributor experience.
The `unit tests` is used for testing a single functionality for its correctness of input and the corresponding output. Unit tests are about ensuring that low-level implementation details are rock solid.
**TIMEBOX**
**FAST TO RUN.**
Unit Tests mustn't be an impediment to feature building.
Tests can be slow at times, due to the need to connect with external services. This can be sending a HTTP request or connecting to a database. We can speed this up by using mocks. In case this is not possible, make sure to `marked` it as `slow`
The following elements should be timeboxed:
- Building time
- Execution time
# 2. Run `unit tests`
**BUILDING**
In this section we will explain everything you need to run the `unit tests` on `OpenBB Terminal`.
Try to cover:
- the most common cases function parameters.
- check if the return type is correct
## 2.1. How to install tests dependencies ?
Only if times allow and it's convenient enough, we can:
- go deeper on function parameters : testing more cases
- have a thorough check of the returned variable
To run the tests you will need first to install the `dev-dependencies`. By default poetry installs the `dev-dependencies` when you run this command :
**EXECUTION**
We must be able to execute Unit Tests often, so they need to be fast to run.
Slow Unit Tests will discourage Developers to run tests, thus negatively impact all the Unit Tests.
If an Unit Test if not fast enough, either:
- Refactor it
- Disable it by default and mark them as `slow`
- Remote it
## 1.3. Table of contents
- [1. OpenBB Terminal Unit Testing](#1-openbb-terminal-unit-testing)
- [1.1. Unit Tests](#11-unit-tests)
- [1.2. Rules](#12-rules)
- [1.3. Table of contents](#13-table-of-contents)
- [2. Run `unit tests`](#2-run-unit-tests)
- [2.1. How to install tests dependencies?](#21-how-to-install-tests-dependencies)
- [2.2. How to run `tests`:](#22-how-to-run-tests)
- [By `module`](#by-module)
- [By test `name`](#by-test-name)
- [By `markers`](#by-markers)
- [2.3. How to skip tests](#23-how-to-skip-tests)
- [Skipping a specific test](#skipping-a-specific-test)
- [Skipping the entire test module](#skipping-the-entire-test-module)
- [3. How to build `unit tests`](#3-how-to-build-unit-tests)
- [3.1. Model Tests](#31-model-tests)
- [3.2. View Tests](#32-view-tests)
- [3.3. Controller Tests](#33-controller-tests)
- [Test print help](#test-print-help)
- [Test call function](#test-call-function)
- [3.4. Fixing existing tests](#34-fixing-existing-tests)
- [Scenario 1: Update Fixtures](#scenario-1-update-fixtures)
- [Scenario 2: Update both Fixtures and Cassettes](#scenario-2-update-both-fixtures-and-cassettes)
- [3.5. How to `mock`?](#35-how-to-mock)
- [PYTEST-MOCK](#pytest-mock)
- [MONCKEYPATCH](#monckeypatch)
- [3.6. Which `helpers` are available?](#36-which-helpers-are-available)
- [3.7. Which `fixtures` are available ?](#37-which-fixtures-are-available-)
- [3.8. Which `markers` are available ?](#38-which-markers-are-available-)
- [3.9. Known `issue` / `solution`](#39-known-issue--solution)
- [ION Usage](#ion-usage)
- [YFINANCE - example 1](#yfinance---example-1)
- [YFINANCE - example 2](#yfinance---example-2)
- [USER-AGENT](#user-agent)
- [BROTLI](#brotli)
- [BEFORE\_RECORD\_RESPONSE](#before_record_response)
- [3.10. List of useful `vscode` tools for `unit tests`](#310-list-of-useful-vscode-tools-for-unit-tests)
- [Testing on VSCODE](#testing-on-vscode)
- [3.11. How to handle `dev-dependencies` ?](#311-how-to-handle-dev-dependencies-)
- [UPDATE PYPROJECT](#update-pyproject)
- [EXPORT REQUIREMENTS](#export-requirements)
- [4. Maintain `unit tests`](#4-maintain-unit-tests)
- [4.1. What's the PR process?](#41-whats-the-pr-process)
- [4.1.1 Find right place](#411-find-right-place)
- [4.1.2 Verify coverage is above 90%](#412-verify-coverage-is-above-90)
- [4.1.3 Set the right markers](#413-set-the-right-markers)
- [4.2. Which automations are available ?](#42-which-automations-are-available-)
- [PULL REQUEST](#pull-request)
- [4.3. How to check code coverage ?](#43-how-to-check-code-coverage-)
- [PULL REQUEST AUTOMATION](#pull-request-automation)
- [MANUALLY](#manually)
- [PRE-COMMIT](#pre-commit)
## 2. Run `unit tests`
In this section we will explain everything you need to run the `unit tests` on the `OpenBB Terminal`.
## 2.1. How to install tests dependencies?
To run the tests you will need first to install the `dev-dependencies`. By default poetry installs the `dev-dependencies` when you run this command:
```bash
poetry install
```
If you do not want to install the `dev-dependencies` you will have to add the option `--no-dev` like this :
If you do not want to install the `dev-dependencies` you will have to add the `--no-dev` option:
```bash
poetry install --no-dev
```
## 2.2. How to run `tests` :
### By `module`
### 2.2. How to run `tests`:
#### By `module`
You can run tests on a specific package/module by specifying the path of this package/module, as follows:
@@ -55,27 +131,28 @@ You can run tests on a specific package/module by specifying the path of this pa
pytest tests/openbb_terminal/some_package
pytest tests/openbb_terminal/some_package/some_module.py
```
### By test `name`
#### By test `name`
You can run tests by their name with the argument `-k`
```
pytest -k "test_function1"
```bash
pytest tests/openbb_terminal/some_package/some_module.py -k test_function1
```
### By `markers`
You can run tests only on specific markers, like this :
```
```bash
pytest -m slow
pytest -k "not slow"
pytest -k "not slow and not network"
pytest -m "not slow"
pytest -m "not slow and not network"
```
You can list the available markers using this command :
```
```bash
pytest --markers
```
@@ -108,34 +185,34 @@ def test_some_function(mocker):
pass
```
# 3. How to build `unit tests`
## 3. How to build `unit tests`
When you contribute a new feature to the Terminal, it's important that tests are added for this particular feature. It is a part of the Checklist for the PR to be approved.
When you contribute a new feature to the OpenBB Terminal, it's important that tests are added for this particular feature since it is a part of the Checklist for the PR to be approved.
All the `unit tests` should be insides the `tests` folder. There should be at most one `test module` for each `module` of `OpenBB Terminal`.
All the `unit tests` should be insides the `tests` folder. There should be at most one `test module` for each `module` of the `OpenBB Terminal`.
Each `test module` should follow the same path of the `module` that it is `testing`. For instance,
Each `test module` should follow the same path of the `module` that it is `testing`.For instance,
- in order to test the following module `openbb_terminal/stocks/due_diligence/dd_controller.py`
- a `test module` should be added here: `tests/openbb_terminal/stocks/due_diligence/test_dd_controller.py`
Now that you know where to add tests, let's go through 3 types of unit tests that we have:
Now that you know where to add tests, let's go through the different types of tests that you can add.
### 3.1. Model Tests
## 3.1. Model Test
As mentioned in the main repo's README, we are following the MVC pattern. The `_model.py`'s job to get the required data, e.g. from an external APIs or scraping.
As mentioned in the [main documentation](README.md), we are following the [MVC pattern](https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller). The `_model.py`'s job to get the required data, e.g. from an external APIs or scraping.
Therefore, in unit tests for `_model.py`, the function output is captured and saved when the test is first run. Then, the next time the test is run, the function's returned value is compared with the captured value in the first run, to ensure parity.
Depending on the data type of function output, we can generate the following files:
- csv : for pandas.DataFrame
- json : for python native data type (list, dict, tuple)
- txt : store str or printed output
Examples:
```
```python
@pytest.fixture(scope="module")
def vcr_config():
return {
@@ -158,16 +235,16 @@ def test_check_supported_ticker(ticker, recorder):
recorder.capture(df)
```
Several things worth noting here:
1. If the model utilizes an API key or token, make sure to filter it insider the `vcr_config` fixture inside your test module, similar to the example. This will hide your actual API key / token for security purposes.
1. If the model utilizes an API key or token, make sure to filter it insider the `vcr_config` fixture inside your test module, similar to the example. This will hide your actual API key/token for security purposes.
2. Record HTTP requests using the decorator `@pytest.mark.vcr`
1. When run the first time, the VCR will record the requests to `cassettes/test_*.yml`
2. Run it again, and VCR will replay the recorded response when the HTTP request is made. This test is now fast (no real HTTP requests are made anymore), deterministic (the test will continue to pass, even if you have no internet connection or that external APIs go down) and accurate (the response will contain the same headers and body you get from a real request).
3. Test out multiple input variables to ensure tests cover multiple edge cases
1. Use the decorator `@pytest.mark.parametrize`to specify the inputs for your tests.
1. Use the decorator `@pytest.mark.parametrize` to specify the inputs for your tests.
4. Capture the returned df using `recorder.capture(<return_value>)`
1. Next time the test run, return output will be compared against the captured file
1. Next time the test runs, the return output will be compared against the captured file
5. Each fixture / cassette file for a test module should be stored within the same folder. An example with `tests/openbb_terminal/some_package/some_module.py` :
- `tests/openbb_terminal/some_package`/cassettes/`some_module`/test_function1.yaml
- `tests/openbb_terminal/some_package`/txt/`some_module`/test_function1.csv
@@ -182,14 +259,15 @@ def test_default():
assert request("http://httpbin.org/get").text == '{"get": true}'
```
## 3.2. View Test
Different from the `model.py`, `view.py` module handles how data is displayed, e.g.as a chart or a table.
### 3.2. View Tests
The `view.py` has no return statement, and simply prints to stdout. Hence, our unit tests will capture stdout values, and compare against the recorded fixture in subsequent runs.
Different from the `_model.py`, `_view.py` modules handles how data is displayed, e.g.as a chart or a table.
The `_view.py` has no return statement, and simply prints to stdout. Hence, our unit tests will capture stdout values, and compare against the recorded fixtures in subsequent runs.
Example:
```
```python
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_display_historical(mocker):
@@ -201,23 +279,26 @@ def test_display_historical(mocker):
```
Important notes to go through:
1. Use `@pytest.mark.record_stdout`to capture `sdtout`.
1. Use `@pytest.mark.record_stdout` to capture `sdtout`.
1. Recorded stdout is saved as fixtures in a txt file
2. If your view function only plots a chart, and does not output anything in `sdtout`, you can skip unit testing your `view.py`
3. Make sure to mock `visualize_output` method.
3. Make sure to mock `visualize_output` whenever possible.
1. Inside `visualize_output`, we use `ion` to enable interactive mode. However, under Windows, having `ion` enabled would make tests fail to run. Hence, you should always mock `visualize_output`, and hence `ion`
## 3.3. Controller Test
### 3.3. Controller Tests
Unit test for `_controller.py` is the last component you would need to add. Most of the time you won't need to create a `_controller.py` from scratch. Hence, we will only cover how to add tests for a new command to an existing `test_*_controller.py` file
#### Test print help
This test in the `test_*_controller.py` captured the help text printed to stdout.
Example:
```
```python
@pytest.mark.vcr(record_mode="none")
@pytest.mark.record_stdout
def test_print_help():
@@ -235,7 +316,7 @@ This test is to check when `controller.py` calls a command, the corresponding fu
Example:
```
```python
@pytest.mark.vcr(record_mode="none")
@pytest.mark.parametrize(
"tested_func, other_args, mocked_func, called_args, called_kwargs",
@@ -284,39 +365,22 @@ def test_call_func(
```
When adding a new feature to an existing menu, there should already be a `test_*_controller.py` file with the `test_call_func()` function. You can simply add another tuple under `@pytest.mark.parametrize()`, which contains:
1. `call_<command>` from the controller.py file
2. Arguments you want to pass in the `call_<command>`
3. The corresponding view function, which will be mocked (Here we only want check if the view function is called at least once. We mock it such that the function won't execute.)
4. Arguments you want to pass to the view function. Note that the arguments here need to be the same as arguments in (2). This is to ensure that arguments passed from the controller.py file are passed correctly to function in `view.py`
## 3.4. Fixing existing tests
### 3.4. Fixing existing tests
It is not uncommon that your new feature or bug fixes break the existing code. Understanding a few concepts can help you navigate and fix most common cases.
Before continuing this section, please have a look at our [FIXTURES](FIXTURES.md) documentation.
Before continuing this section, please have a look at our [pytest-recorder-documentation](https://github.com/OpenBB-finance/pytest_recorder/blob/main/README.md).
TL;DR:
### 3.5. How to `mock`?
You can run unit tests under a few mode:
- `--rewrite-expected` will re-generate the fixtures (csv, json and txt files) if there's detected change.
- `--record-mode` will impact both the cassette (yaml) and fixtures (csv, json, txt files)
1. `--record-mode=none` : searches for existing cassette / fixtures. If there's none, an exception will be thrown.
2. `--record-mode=once` : creates new fixtures and cassette if they did not exist. If there are already fixtures or cassette, it will take the existing one.
3. `--record-mode=rewrite` : this rewrites both the cassette and fixtures, even if they already exist.
### Scenario 1: Update Fixtures
Let's say you just updated the code in `_model.py` file, which then altered the recorded output. You can update the fixture files by running the tests with `--rewrite-expected` argument. This would update the fixtures with latest changes.
Example: `pytest openbb_terminal.cryptocurrency.defi.defi_controller --rewrite-expected`
### Scenario 2: Update both Fixtures and Cassettes
If you want to update existing cassettes and fixtures, use the argument `--record-mode=rewrite` when running tests.
As existing cassettes and fixtures will be overwritten (if there's any), make sure that you check both, and that they yield expected output.
## 3.5. How to `mock` ?
**PYTEST-MOCK**
#### PYTEST-MOCK
A `mocker` fixture is available through the package `pytest-mock`.
@@ -329,9 +393,9 @@ def test_something(mocker):
More information about `pytest-mock` are available here :
- https://pypi.org/project/pytest-mock/
- [Pytest Mock](https://pypi.org/project/pytest-mock/)
**MOCKEYPATCH**
#### MONCKEYPATCH
There is also a `monkeypatch` fixture available by default inside `pytest`.
@@ -342,9 +406,9 @@ def test_double(monkeypatch):
More information about `monkeypatch` are available here :
- https://docs.pytest.org/en/6.2.x/monkeypatch.html
- [Monkeypatch](https://docs.pytest.org/en/6.2.x/monkeypatch.html)
## 3.6. Which `helpers` are available ?
### 3.6. Which `helpers` are available?
You can find the available helpers inside the following package/module :
@@ -353,11 +417,11 @@ You can find the available helpers inside the following package/module :
See also the `pytest fixtures` which are autoloaded helpers.
## 3.7. Which `fixtures` are available ?
### 3.7. Which `fixtures` are available ?
You can list all the available `pytest fixtures` using the following command :
```
```bash
pytest --fixtures
```
@@ -365,11 +429,11 @@ More on custom fixtures here :
- [FIXTURES](FIXTURES.md)
## 3.8. Which `markers` are available ?
### 3.8. Which `markers` are available ?
You can list the available markers using this command :
```
```bash
pytest --markers
```
@@ -377,15 +441,15 @@ More information on markers location are available in pytest documentation :
- https://docs.pytest.org/en/6.2.x/mark.html#mark
## 3.9. Known `issue` / `solution`
### 3.9. Known `issue` / `solution`
**ION Usage**
#### ION Usage
As mentioned earlier, if you are testing for a `_view.py` module that uses `ION`, you would need to mock it. With Windows, the charts / graphs are unable to close the graph applications. This would make the tests failed when running with Windows.
You can use the following example below. Once you have mocked the graphs / charts, simply call the display function, which is now a mocked object.
```
```python
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_display_defi_tvl(mocker):
@@ -398,7 +462,7 @@ def test_display_defi_tvl(mocker):
llama_view.display_defi_tvl(20)
```
**YFINANCE**
#### YFINANCE - example 1
If a method is using the `yfinance` library but do not let you pick the `start/end` dates it will pick the current date each time.
@@ -421,21 +485,7 @@ def vcr_config():
You can also refactor this method to let access to `stard/end` dates.
**USER-AGENT**
Some function uses a random `User-Agent` on the `HTTP HEADER` when fetching data from an `API`.
Here is how to filter this random `User-Agent`.
```python
@pytest.fixture(scope="module")
def vcr_config():
return {
"filter_headers": [("User-Agent", None)],
}
```
**YFINANCE**
#### YFINANCE - example 2
If you do something like this with `yfinance` library.
@@ -469,9 +519,23 @@ def test_ark_orders_view(kwargs_dict, mocker, use_color):
yf.download(tickers="VCYT QSI")
```
**BROTLI**
#### USER-AGENT
The library `requests` doesn't support : `brotli` comnpression.
Some function uses a random `User-Agent` on the `HTTP HEADER` when fetching data from an `API`.
Here is how to filter this random `User-Agent`.
```python
@pytest.fixture(scope="module")
def vcr_config():
return {
"filter_headers": [("User-Agent", None)],
}
```
#### BROTLI
The library `requests` doesn't support: `brotli` comnpression.
Unless `brotli` library is installed in the environment.
@@ -484,7 +548,7 @@ So if both of these conditions are fulfilled :
Then the `test` might work in local but crash during `PullRequest`.
**BEFORE_RECORD_RESPONSE**
#### BEFORE_RECORD_RESPONSE
The library `vcrpy` has a `before_record_response` which accept a filtering function that can be used to filter your cassette content :
@@ -505,7 +569,7 @@ Or to filter sensitive data in the response.
The issue with this `before_record_response` : it isn't launched at the first run of the test.
More on this here : https://github.com/kevin1024/vcrpy/pull/594
More on this [here](https://github.com/kevin1024/vcrpy/pull/594).
A solution for now is to run this command while initializing the `cassettes` :
@@ -515,25 +579,23 @@ pytest tests/.../test_some_test_module.py --record-mode=once --rewrite-expected
pytest tests/.../test_some_test_module.py --record-mode=once --rewrite-expected
```
## 3.10. List of useful `vscode` tools for `unit tests`
### 3.10. List of useful `vscode` tools for `unit tests`
**VSCODE TESTING**
#### Testing on VSCODE
The default testing tool from `vscode` should let you add breakpoints and run debug on a specific `test`.
It's a convenient way to see what's inside your `test` while running.
More information on this tool are available here :
More information on this tool are available [here](https://code.visualstudio.com/docs/python/testing).
- https://code.visualstudio.com/docs/python/testing
### 3.11. How to handle `dev-dependencies` ?
## 3.11. How to handle `dev-dependencies` ?
**UPDATE PYPROJECT**
#### UPDATE PYPROJECT
If a library need to be added or removed from `dev-dependencies` this can be done directly in the `pyproject.toml` file.
All the `dev-dependencies` should be under the `tool.poetry.dev-dependencies` section, like in this example :
All the `dev-dependencies` should be under the `tool.poetry.dev-dependencies` section, like in this example:
```toml
[tool.poetry.dev-dependencies]
@@ -543,30 +605,30 @@ flake8 = "^3.9.0"
...
```
**EXPORT REQUIREMENTS**
#### EXPORT REQUIREMENTS
After updating the `pyproject.toml` you will have to export the `requirements` files using the following commands :
After updating the `pyproject.toml` you will have to export the `requirements` files using the following commands:
```bash
poetry export -f requirements.txt -o requirements.txt --without-hashes --with dev
poetry export -f requirements.txt -o requirements-full.txt --extras forecast --extras optimization --without-hashes --with dev
```
# 4. Maintain `unit tests`
## 4. Maintain `unit tests`
## 4.1. What is the PR process ? (in progress)
### 4.1. What's the PR process?
Here are the steps to write proper tests for Gamestonk :
Here are the steps to write proper tests for the OpenBBTerminal:
A. Find right place
B. Verify coverage is above 90%
C. Set the right markers
1. Find right place
2. Verify coverage is above 90%
3. Set the right markers
**A. Find right place**
#### 4.1.1 Find right place
Put the code following the same module and package structure than `openbb_terminal` package.
**B. Verify coverage is above 90%**
#### 4.1.2 Verify coverage is above 90%
Once you made your `PullRequest` an automation will let you know whether or not you have the proper amount of tests coverage.
@@ -574,7 +636,7 @@ You can also run the following command to check your coverage manually :
- `pytest --cov --cov-fail-under=90`
**C. Set the right markers**
#### 4.1.3 Set the right markers
If parts of your have specificities like :
@@ -588,26 +650,26 @@ The rests of this document is there to provide a deeper comprehension of this st
A tests update might be asked whenever the answer to one of the following question is `yes` :
A. Is this PR fixing bug which was not detected by the current tests ?
B. Is this PR reducing code coverage ?
C. Is this PR adding features ?
D. Is this PR updating features ?
- Is this PR fixing bug which was not detected by the current tests ?
- Is this PR reducing code coverage ?
- Is this PR adding features ?
- Is this PR updating features ?
## 4.2. Which automations are available ?
### 4.2. Which automations are available ?
**PULL REQUEST**
#### PULL REQUEST
A `github action` should comment every `Pull Request` with the code `coverage`.
This automation will not enforce any rules regarding `tests`.
## 4.3. How to check code coverage ?
### 4.3. How to check code coverage ?
**PULL REQUEST AUTOMATION**
#### PULL REQUEST AUTOMATION
You should be able to see the code coverage on comment of your `Pull Requests`.
**MANUALLY**
#### MANUALLY
This is how to manually check code coverage.
@@ -623,6 +685,6 @@ Here is an example where we select only the package `stocks\due_diligence` :
pytest --cov=openbb_terminal\stocks\due_diligence --cov-report term-missing
```
**PRE-COMMIT**
#### PRE-COMMIT
Code `coverage` and `unit tests` are not run on `pre-commit` hooks since it will slow down each commit.

56
tests/openbb_terminal/economy/test_finnhub_model.py
View File

@@ -1,56 +0,0 @@
# IMPORTATION STANDARD
# IMPORTATION THIRDPARTY
import pandas as pd
import pytest
# IMPORTATION INTERNAL
from openbb_terminal.economy import finnhub_model
@pytest.fixture(scope="module")
def vcr_config():
return {
"filter_query_parameters": [("token", "MOCK_TOKEN")],
}
@pytest.mark.vcr(record_mode="none")
def test_get_economy_calendar_events(mocker):
# MOCK JSON
mock_json = pd.DataFrame()
mock_json["economicCalendar"] = ["MOCK_ROW_1", "MOCK_ROW_2"]
# MOCK GET
attrs = {
"status_code": 200,
"json.return_value": mock_json,
}
mock_response = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.helper_funcs.requests.get",
new=mocker.Mock(return_value=mock_response),
)
result_df = finnhub_model.get_economy_calendar_events()
assert not result_df.empty
@pytest.mark.vcr(record_mode="none")
def test_get_economy_calendar_events_no_response(mocker):
# MOCK GET
attrs = {
"json.return_value": {"error": "mock error message"},
}
mock_response = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.helper_funcs.requests.get",
new=mocker.Mock(return_value=mock_response),
)
result_df = finnhub_model.get_economy_calendar_events()
assert isinstance(result_df, pd.DataFrame)
assert result_df.empty

130
tests/openbb_terminal/economy/test_finnhub_view.py
View File

@@ -1,130 +0,0 @@
# IMPORTATION STANDARD
import json
# IMPORTATION THIRDPARTY
import pytest
# IMPORTATION INTERNAL
from openbb_terminal.economy import finnhub_view
@pytest.fixture(scope="module")
def vcr_config():
return {
"filter_query_parameters": [("token", "MOCK_TOKEN")],
}
MOCK_ECONOMY_CALENDAR_EVENTS_DICT = json.loads(
"""
{
"economicCalendar": [
{
"actual": 8.4,
"country": "AU",
"estimate": 6.9,
"event": "Australia - Current Account Balance",
"impact": "low",
"prev": 1,
"time": "2020-06-02 01:30:00",
"unit": "AUD"
},
{
"actual": 0.5,
"country": "AU",
"estimate": 0.4,
"event": "Australia- Net Exports",
"impact": "low",
"prev": -0.1,
"time": "2020-06-02 01:30:00",
"unit": "%"
}
]
}
"""
)
@pytest.mark.vcr(record_mode="none")
@pytest.mark.record_stdout
def test_economy_calendar_events_empty(mocker):
# MOCK GET_ECONOMY_CALENDAR_EVENTS
attrs = {"empty": True}
mock_empty_df = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.economy.finnhub_view.finnhub_model.get_economy_calendar_events",
new=mock_empty_df,
)
finnhub_view.economy_calendar_events(
country="MOCK_COUNTRY",
limit=1,
impact="MOCK_IMPACT",
export="",
)
@pytest.mark.vcr(record_mode="none")
@pytest.mark.record_stdout
def test_economy_calendar_events_no_country(mocker):
# MOCK GET
attrs = {
"status_code": 200,
"json.return_value": MOCK_ECONOMY_CALENDAR_EVENTS_DICT,
}
mock_response = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.helper_funcs.requests.get",
new=mocker.Mock(return_value=mock_response),
)
finnhub_view.economy_calendar_events(
country="MOCK_COUNTRY",
limit=1,
impact="MOCK_IMPACT",
export="",
)
@pytest.mark.vcr(record_mode="none")
@pytest.mark.record_stdout
def test_economy_calendar_events_no_impact(mocker):
# MOCK GET
attrs = {
"status_code": 200,
"json.return_value": MOCK_ECONOMY_CALENDAR_EVENTS_DICT,
}
mock_response = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.helper_funcs.requests.get",
new=mocker.Mock(return_value=mock_response),
)
finnhub_view.economy_calendar_events(
country="AU",
limit=1,
impact="MOCK_IMPACT",
export="",
)
@pytest.mark.vcr(record_mode="none")
@pytest.mark.record_stdout
def test_economy_calendar_events(mocker):
# MOCK GET
attrs = {
"status_code": 200,
"json.return_value": MOCK_ECONOMY_CALENDAR_EVENTS_DICT,
}
mock_response = mocker.Mock(**attrs)
mocker.patch(
target="openbb_terminal.helper_funcs.requests.get",
new=mocker.Mock(return_value=mock_response),
)
finnhub_view.economy_calendar_events(
country="AU",
limit=1,
impact="low",
export="",
)

6
tests/openbb_terminal/stocks/fundamental_analysis/test_business_insider_view.py
View File

@@ -50,7 +50,7 @@ def test_display_management_nodata():
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_price_target_from_analysts_raw():
business_insider_view.price_target_from_analysts(
business_insider_view.display_price_target_from_analysts(
symbol="TSLA",
start_date=None,
data=None,
@@ -68,7 +68,7 @@ def test_price_target_from_analysts_plt():
start = datetime.strptime("2021-12-05", "%Y-%m-%d")
stock = load(symbol=ticker, start_date=start, interval=interval)
business_insider_view.price_target_from_analysts(
business_insider_view.display_price_target_from_analysts(
symbol=ticker,
start_date=start,
data=stock,
@@ -82,6 +82,6 @@ def test_price_target_from_analysts_plt():
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_estimates():
business_insider_view.estimates(
business_insider_view.display_estimates(
symbol="TSLA", estimate="annualearnings", export=None
)

4
tests/openbb_terminal/stocks/fundamental_analysis/test_fa_controller.py
View File

@@ -560,7 +560,7 @@ def test_call_func_expect_queue(expected_queue, queue, func):
),
(
"call_pt",
"business_insider_view.price_target_from_analysts",
"business_insider_view.display_price_target_from_analysts",
["--limit=10"],
{
"symbol": "TSLA",
@@ -574,7 +574,7 @@ def test_call_func_expect_queue(expected_queue, queue, func):
),
(
"call_est",
"business_insider_view.estimates",
"business_insider_view.display_estimates",
[],
{
"symbol": "TSLA",

1
tests/openbb_terminal/stocks/fundamental_analysis/txt/test_fa_controller/test_print_help.txt vendored
View File

@@ -38,4 +38,3 @@ Future Expectations:
pt price targets over time [BusinessInsider, FinancialModelingPrep]
dcf advanced Excel customizable discounted cash flow [StockAnalysis]
dcfc determine the (historical) discounted cash flow [FinancialModelingPrep]