From 41f49a4e49bd71062f6beff11ac44afaa3e8973f Mon Sep 17 00:00:00 2001 From: Pratyush Shukla Date: Fri, 11 Aug 2023 19:21:36 +0530 Subject: [PATCH] polished the README.md ; co-authored by: @IgorWounds --- openbb_sdk/README.md | 52 +++++----- openbb_sdk/extensions/README.md | 22 ++--- openbb_sdk/extensions/charting/README.md | 23 +++-- openbb_sdk/extensions/stocks/README.md | 6 +- openbb_sdk/providers/README.md | 43 +++++---- openbb_sdk/sdk/core/README.md | 118 ++++++++--------------- openbb_sdk/sdk/provider/README.md | 39 +++++--- 7 files changed, 134 insertions(+), 169 deletions(-) diff --git a/openbb_sdk/README.md b/openbb_sdk/README.md index 1220341f618..c408bdfec1b 100644 --- a/openbb_sdk/README.md +++ b/openbb_sdk/README.md @@ -4,9 +4,9 @@ - [Pre-requisites](#pre-requisites) - [Installation](#installation) - [API keys](#api-keys) - - [From local file](#from-local-file) - - [From runtime](#from-runtime) - - [From OpenBB Hub](#from-openbb-hub) + - [1. From local file](#1-from-local-file) + - [2. From runtime](#2-from-runtime) + - [3. From OpenBB Hub](#3-from-openbb-hub) - [Python Usage](#python-usage) - [API Usage](#api-usage) - [Docker](#docker) @@ -14,13 +14,13 @@ ## Pre-requisites -To install and use you need: +To install and use the SDK you need: - A fresh virtual environment - Python 3.8 or higher - API keys for the APIs you want to use -To develop you need: +For development you need: - Poetry 1.5.1 or higher - ruff 0.0.256 @@ -29,27 +29,27 @@ To develop you need: ## Installation -To install the SDK in the virtual environment from the repo root run: +To install the SDK in a virtual environment from the repo root run: ```bash pip install ./openbb_sdk ``` -This will install the SDK and all the dependencies into the site-packages folder of the virtual environment. +This will install the SDK and all its dependencies into the site-packages directory of the virtual environment. ## API keys -To connect to APIs you need to provide the API keys. Here 3 options how you can provide them: +To use your API keys you need to configure them. Here are the 3 options on how to do it: 1. From local file 2. At runtime 3. From OpenBB Hub -### From local file +### 1. From local file -You can specify the keys from the `~/.openbb_sdk/user_settings.json` file. +You can specify the keys in the `~/.openbb_sdk/user_settings.json` file. -Create this file with the following template: +Create this file using the following template and replace the values with your keys: ```json { @@ -62,11 +62,9 @@ Create this file with the following template: } ``` -Then add your API keys there and it should work. +### 2. From runtime -### From runtime - -You can also specify the keys at runtime: +You can also specify your keys at runtime: ```python from openbb import obb @@ -74,7 +72,7 @@ obb.settings.credentials.fmp_api_key = "REPLACE_ME" obb.settings.credentials.polygon_api_key = "REPLACE_ME" ``` -### From OpenBB Hub +### 3. From OpenBB Hub You can also load your the keys from the OpenBB Hub. @@ -89,7 +87,7 @@ Import and basic usage: ```python from openbb import obb -aapl = obb.stocks.load(symbol="AAPL", start_date="2020-01-01", provider="fmp") +aapl = obb.stocks.load("AAPL") df = aapl.to_dataframe() df.head() ``` @@ -108,38 +106,36 @@ Authorize with the developer credentials: openbb/openbb ## Docker -You can use the API through docker. +You can use the API through Docker. -To build the image from the repo root run: +To build the image, from the repo root run: ```bash docker build -f build/docker/api.dockerfile -t openbb-sdk:latest . ``` -To run the image: +To run this newly-built image: ```bash docker run --rm -p8000:8000 -v ~/.openbb_sdk:/root/.openbb_sdk openbb-sdk:latest ``` -This will mount the local `~/.openbb_sdk` folder into the docker container so you can use the API keys from there and it will expose the API on port 8000. +This will mount the local `~/.openbb_sdk` directory into the Docker container so you can use the API keys from there and it will expose the API on port `8000`. ## Development -From the root of the OpenBB Terminal repo run: +From the root of the OpenBB Terminal repo, run: `poetry install -C ./openbb_sdk` -When developing a specific extension cd into the extension folder and run: +When developing a specific extension `cd` into the extension directory and run: `pip install -U -e .` -While we're still developing it is required to create the static python interface: +While we're still developing, it is required to create the static python interface: ```python -import openbb -openbb._rebuild_python_interface() -exit() +python -c "import openbb; openbb._rebuild_python_interface()" ``` -Right now you need to do this before the first launch and every time you install or uninstall a new extension. +Currently you need to do this before the first launch and every time you install or uninstall a new extension. diff --git a/openbb_sdk/extensions/README.md b/openbb_sdk/extensions/README.md index aa4e76c1963..35e7f339f33 100644 --- a/openbb_sdk/extensions/README.md +++ b/openbb_sdk/extensions/README.md @@ -2,14 +2,14 @@ All currently supported extensions are listed below. -| Extension | -| --- | -| [Charting](./charting/README.md) | -| [Crypto](./crypto/README.md) | -| [Economy](./economy/README.md) | -| [Fixed Income](./fixed_income/README.md) | -| [Forex](./forex/README.md) | -| [News](./news/README.md) | -| [Quantitative Analysis](./qa/README.md) | -| [Stocks](./stocks/README.md) | -| [Technical Analysis](./technical_analysis/README.md) | +| Extension | Description | +| --- | --- | +| [Charting](./charting/README.md) | Charting Manager extension for the SDK. | +| [Crypto](./crypto/README.md) | Access cryptocurrency financial data. | +| [Economy](./economy/README.md) | Access global macroeconomic data. | +| [Fixed Income](./fixed_income/README.md) | Access central bank decisions, yield curves, government bonds and corporate bonds data. | +| [Forex](./forex/README.md) | Access foreign exchanges, quotes, and forward rates for currency pairs. | +| [News](./news/README.md) | Access news articles. | +| [Quantitative Analysis](./qa/README.md) | Access to statistical and computational techniques for financial data. | +| [Stocks](./stocks/README.md) | Access historical pricing data, options, sector and industry, and overall due diligence. | +| [Technical Analysis](./ta/README.md) | Access tools and techniques for financial market analysis. | diff --git a/openbb_sdk/extensions/charting/README.md b/openbb_sdk/extensions/charting/README.md index 7b3cf5b5b72..f6c91179728 100644 --- a/openbb_sdk/extensions/charting/README.md +++ b/openbb_sdk/extensions/charting/README.md @@ -4,9 +4,9 @@ This extension provides a charting library for OpenBB SDK. The library includes: -- charting infrastructure based on Plotly and +- a charting infrastructure based on Plotly - a set of charting components -- prebuilt charts for a set of commands that are available in built-in OpenBB SDK extensions +- prebuilt charts for a set of commands that are built-in OpenBB SDK extensions ## Installation @@ -20,36 +20,35 @@ pip install . The PyWry dependency handles display of interactive charts and tables in a separate window. It is installed automatically with the OpenBB Charting extension. -When using linux distributions, the PyWry dependency requires certain dependencies to be installed first. +When using Linux distributions, the PyWry dependency requires certain dependencies to be installed first. -Debian-based / Ubuntu / Mint: +- Debian-based / Ubuntu / Mint: `sudo apt install libwebkit2gtk-4.0-dev` -Arch Linux / Manjaro: +- Arch Linux / Manjaro: `sudo pacman -S webkit2gtk` -Fedora: +- Fedora: `sudo dnf install gtk3-devel webkit2gtk3-devel` ## Usage -To use the extension is as simple as running any of the OpenBB SDK endpoints with the `chart` argument equals to `True`. +To use the extension, run any of the OpenBB SDK endpoints with the `chart` argument set to `True`. Here's an example how it would look like in a python interface: ```python from openbb import obb -stock_data = obb.stocks.load(symbol="TSLA", start_date="2023-01-01", provider="fmp", chart=True) - +stock_data = obb.stocks.load(symbol="TSLA", chart=True) ``` -Which would result in an `CommandOutput` object containing a `chart` attribute, which contains plotly json data. +This results in a `CommandOutput` object containing a `chart` attribute, which contains Plotly JSON data. -In order to actually display the chart, you need to call the `show()` method. +In order to display the chart, you need to call the `show()` method: ```python stock_data.show() ``` -> Note: The `show()` method is only going to work either in a Jupyter Notebook or in a standalone python script where a PyWry based backend is properly initialized. +> Note: The `show()` method currently works either in a Jupyter Notebook or in a standalone python script with a PyWry based backend properly initialized. diff --git a/openbb_sdk/extensions/stocks/README.md b/openbb_sdk/extensions/stocks/README.md index 61c4e6be6ae..cff3aa32630 100644 --- a/openbb_sdk/extensions/stocks/README.md +++ b/openbb_sdk/extensions/stocks/README.md @@ -5,12 +5,12 @@ This extension provides stock market data tools for the OpenBB SDK. Features of the Stocks extension include: - Access to various stock market data sources -- Submenus such as - - `fa` for fundamental analysis +- Sub-modules such as: + - `ca` for fundamental analysis - `dd` for in-depth due diligence - `disc` for stock discovery - `dps` for Dark Pool Shorts data - - `ca` for comparison analysis + - `fa` for comparison analysis - `gov` for government contracts data - `ins` for insiders trading data - `options` for options diff --git a/openbb_sdk/providers/README.md b/openbb_sdk/providers/README.md index 03a7665fe35..789e88935c3 100644 --- a/openbb_sdk/providers/README.md +++ b/openbb_sdk/providers/README.md @@ -4,30 +4,28 @@ All currently supported providers are listed below. -| Provider | -| --- | -| [FMP](./fmp/README.md) | -| [Polygon](./polygon/README.md) | -| [Benzinga](./benzinga/README.md) | -| [FRED](./fred/README.md) | +| Provider | URL | +| --- | --- | +| [FMP](./fmp/README.md) | https://financialmodelingprep.com/ | +| [Polygon](./polygon/README.md) | https://polygon.io/ | +| [Benzinga](./benzinga/README.md) | https://www.benzinga.com/ | +| [FRED](./fred/README.md) | https://fred.stlouisfed.org/ | ## Default Provider Configuration -To configure the default providers for each route, edit the `.openbb_sdk/user_settings.json` file in your home directory by adding the `defaults` section, a `routes` subsection and for every route, the default provider desired. A sample is shown below. +To configure the default providers for each route, edit the `.openbb_sdk/user_settings.json` file in your home directory using the following template: ```json { "defaults": { "routes": { - "/stocks/fa/balance": { - "provider": "polygon" - }, "/stocks/load": { "provider": "fmp" }, - "/stocks/news": { - "provider": "benzinga" - } + "/stocks/fa/balance": { + "provider": "polygon" + }, + ... } } } @@ -35,19 +33,24 @@ To configure the default providers for each route, edit the `.openbb_sdk/user_se ## Provider Information -Every provider is located within a folder, with the following structure: +Every provider is located within a directory, with the following structure: ```{.bash} openbb_sdk └───providers └─── | README.md - │ `pyproject.toml` - │ `poetry.lock` + │ pyproject.toml + │ poetry.lock + |───tests └───openbb_ - │ `init.py` - │ .py - │ ... + │ __init__.py + |───models + | |───.py + | └───... + └───utils + |───.py + └───... ``` -The `openbb_` folder contains the provider's source code. The `README.md` file contains the provider's documentation. The `pyproject.toml` and `poetry.lock` files contain the provider's dependencies. +The models define the data structures that are used to query the provider endpoints and store the response data. diff --git a/openbb_sdk/sdk/core/README.md b/openbb_sdk/sdk/core/README.md index 781854fc0fc..38d91d6ccd8 100644 --- a/openbb_sdk/sdk/core/README.md +++ b/openbb_sdk/sdk/core/README.md @@ -1,6 +1,6 @@ -# THIS README IS A WORK IN PROGRESS AND CAN BE VERY MUCH OUT OF DATE. REFRESH THE PAGE UNTIL THIS BANNER IS GONE +# THIS README IS A WORK IN PROGRESS -- [THIS README IS A WORK IN PROGRESS AND CAN BE VERY MUCH OUT OF DATE. REFRESH THE PAGE UNTIL THIS BANNER IS GONE](#this-readme-is-a-work-in-progress-and-can-be-very-much-out-of-date-refresh-the-page-until-this-banner-is-gone) +- [THIS README IS A WORK IN PROGRESS](#this-readme-is-a-work-in-progress) - [1. Introduction](#1-introduction) - [2. How to install?](#2-how-to-install) - [Git clone](#git-clone) @@ -18,7 +18,7 @@ - [Settings](#settings) - [System](#system) - [Coverage](#coverage) - - [4.1.3. OpenBB Hub account](#413-openbb-hub-account) + - [4.1.3. OpenBB Hub Account](#413-openbb-hub-account) - [4.2 Dynamic version](#42-dynamic-version) - [5. REST API](#5-rest-api) - [5.1 Test users](#51-test-users) @@ -26,13 +26,8 @@ ## 1. Introduction -This is a collection of functions that allow extracting, transforming and exporting financial data. +This directory contains the OpenBB SDK's core functionality. It allows you to create an [extension](../../extensions/README.md) or a [provider](../../providers/README.md) that will be automatically turned into REST API endpoint and allow sharing data between commands. -These functions are special: - -- They provide configuration -- They will be automatically turned into REST API endpoint -- They allow sharing data between commands ## 2. How to install? @@ -65,11 +60,9 @@ poetry new openbb-sdk-my_extension ### Command -Add a router and a command. +Add a router and a command in the `openbb_sdk/extensions///_router.py` ```python -# File my_extension/extension_router.py - from openbb_core.app.router import Router router = Router(prefix="/router_name") @@ -81,9 +74,9 @@ def some_command( pass ``` -If your command only makes use of a `openbb-provider` model, there is no need to repeat its structure in the parameters. Just pass the model name as an argument. +If your command only makes use of a standard model defined inside `openbb_provider/standard_models` directory, there is no need to repeat its structure in the parameters. Just pass the model name as an argument. -This is an example how we do it for `stocks.load` which only depends on `StockEOD` model defined in `openbb-provider`. +This is an example how we do it for `stocks.load` which only depends on `StockEOD` model defined in `openbb-provider`: ```python @router.command(model="StockEOD") @@ -102,7 +95,6 @@ def load( Add an entrypoint for the extension inside your `pyproject.toml` file. ```toml -# File pyproject.toml packages = [{include = "openbb_sdk_my_extension"}] ... [tool.poetry.extensions."openbb_extensions"] @@ -120,15 +112,15 @@ poetry install ## 4. Usage -Update the settings +Update your credentials and default providers by modifying the `.openbb_sdk/user_settings.json` inside your home directory: ```{json} -# FILE /.openbb_sdk/user_settings.json { "credentials": { "benzinga_api_key": null, "fmp_api_key": null, - "polygon_api_key": null + "polygon_api_key": null, + "fred_api_key": null }, "defaults": { "routes": { @@ -146,8 +138,10 @@ Update the settings } ``` +Update your system settings by modifying the `.openbb_sdk/system_settings.json` file inside your home directory: + + ```{json} -# FILE /.openbb_sdk/system_settings.json { "run_in_isolation": null, "dbms_uri": null @@ -159,9 +153,9 @@ Update the settings Run your command: ```python -from openbb import sdk +from openbb import obb -output = sdk.stocks.load( +output = obb.stocks.load( symbol="TSLA", start_date="2023-01-01", provider="fmp", @@ -220,8 +214,8 @@ date ```python >>> output.to_plotly_json() -{'data': - [ +{ + 'data':[ { 'close': [260.02, 262.9, 291.26], 'decreasing': {'line': {'width': 1.1}}, @@ -241,24 +235,24 @@ date These are your user settings, you can change them anytime and they will be applied. Don't forget to `sdk.account.save()` if you want these changes to persist. ```python -from openbb import sdk +from openbb import obb -sdk.settings.profile -sdk.settings.credentials -sdk.settings.preferences -sdk.settings.defaults +obb.settings.profile +obb.settings.credentials +obb.settings.preferences +obb.settings.defaults ``` #### System -Check your system settings. Most of the properties are read-only during runtime, so any changes there will be void. +Check your system settings. Most of the properties are read-only during runtime. -- `debug_mode`: here if you set `debug_mode = True` any exception that occurs during execution will be raised immediately. +- `debug_mode`: Setting it as `True` will immediately raise all occurring exceptions. ```python -from openbb import sdk +from openbb import obb -sdk.system +obb.system ``` #### Coverage @@ -266,7 +260,7 @@ sdk.system Obtain the coverage of providers and commands. ```python ->>> sdk.coverage.commands +>>> obb.coverage.commands { '/crypto/load': ['fmp', 'polygon'], '/economy/const': ['fmp'], @@ -276,7 +270,7 @@ Obtain the coverage of providers and commands. ``` ```python ->>> sdk.coverage.providers +>>> obb.coverage.providers { 'fmp': [ @@ -290,34 +284,34 @@ Obtain the coverage of providers and commands. } ``` -### 4.1.3. OpenBB Hub account +### 4.1.3. OpenBB Hub Account -You can login to your OpenBB Hub account and save your credentials there. This way you can access them from any device. +You can login to your OpenBB Hub account and save your credentials there to access them from any device. ```python -from openbb import sdk +from openbb import obb # Login with email, password or SDK token -sdk.account.login(email="your_email", password="your_password", remember_me=True) # pragma: allowlist secret +obb.account.login(email="your_email", password="your_password", remember_me=True) # pragma: allowlist secret # Change a credential -sdk.account.settings.credentials.polygon_api_key = "new_key" # pragma: allowlist secret +obb.account.settings.credentials.polygon_api_key = "new_key" # pragma: allowlist secret # Save account changes -sdk.account.save() +obb.account.save() # Refresh account with latest changes -sdk.account.refresh() +obb.account.refresh() # Logout -sdk.account.logout() +obb.account.logout() ``` ## 4.2 Dynamic version -You can also use the dynamic version to consume the api endpoints from Python itself. +You can also use the dynamic version to consume the API endpoints from Python itself. -In fact, the static version makes use of this feature to run each command. Take a look at the example below. +In fact, the static version makes use of this feature to run each command. Take a look at the example below: ```python >>> from openbb_core.app.command_runner import CommandRunnerSession @@ -359,41 +353,7 @@ chart: ... # Chart object. ## 5. REST API -OpenBB SDK comes with a ready to use Rest API built with FastAPI. - -```{json} -# FILE /.openbb_sdk/user_settings.json -{ - "credentials": { - "benzinga_api_key": null, - "fmp_api_key": null, - "polygon_api_key": null - }, - "defaults": { - "routes": { - "/stocks/fa/balance": { - "provider": "polygon" - }, - "/stocks/load": { - "provider": "fmp" - }, - "/stocks/news": { - "provider": "benzinga" - } - } - } -} -``` - -```{json} -# FILE /.openbb_sdk/system_settings.json -{ - "run_in_isolation": null, - "dbms_uri": null -} -``` - -Start the application: +OpenBB SDK comes with a ready to use Rest API built with FastAPI. Start the application using this command: ```bash uvicorn openbb_core.api.rest_api:app --reload diff --git a/openbb_sdk/sdk/provider/README.md b/openbb_sdk/sdk/provider/README.md index 31271f98346..1b1a4f47237 100644 --- a/openbb_sdk/sdk/provider/README.md +++ b/openbb_sdk/sdk/provider/README.md @@ -1,40 +1,47 @@ # Table of contents - [Table of contents](#table-of-contents) - - [provider](#provider) + - [Provider](#provider) - [What is Data Standardization?](#what-is-data-standardization) - [Key Elements](#key-elements) - [Installation](#installation) -## provider +## Provider The OpenBB Provider aims to give a coherent access to financial data providers by introducing standardization procedures. ## What is Data Standardization? -OpenBB Data Standardization is a process of transforming data into a common format to allow for efficient and accurate information processing, comparison, and analysis. +It's like teaching everyone to speak the same language with their data so we can understand and compare it easily. -Our data standardization can be considered as a sub-process of more general data normalization. There are some things that we do and don’t do when we apply OpenBB Data Standardization. +**Think of it as a part of**: -**We do**: -**Consistent Column Name Coding and Formats**: we ensure that data are consistently coded on a column name and column type level. +Data normalization, a bigger way of organizing data. -**Examples**: -Close for closing price column name in data from one provider might be coded as c in data from another provider Date and time can come in formats like YYYY-MM-DD, MM-DD-YYYY, and Unix timestamps in data from different providers. Standardization would bring these to a common format. +**What We Do**: -**We don’t do**: -**Data Cleaning**: we don’t remove errors and inconsistencies from data provider outputs. -**Data Transformation & Data Integration**: both transforming data by processing it and integrating data from multiple providers together to harmonize datasets are functions of the SDK and are not handled on a provider level. +- **Match Column Names and Formats**: Like making sure everyone calls a "closing price" the same thing. + +- **For instance**: Some might say "Close", others might say "c". We make sure everyone uses one term. +**Unify Date and Time Styles**: Like having everyone use the same calendar format. + +**Example**: Whether it's "YYYY-MM-DD" or "MM-DD-YYYY", we pick one style for everyone. + +**What We Don’t Do**: + +- **Cleaning Data**: We don't act like data detectives and remove mistakes from what providers give us. + +- **Change & Combine Data**: Tweaking data or mixing data from different places. ## Key Elements -- **QueryParams** : The input model for a particular query. To load stock market data, we would have StockQueryParams, which would have fields like symbol, start date, and end date. You can find the standard query params inside the `./models` folder. -- **Data** : The output model of a particular query. Stock market data would be StockPriceData and have fields such as Open, High, Low, Close, and Volume. You can find the standard data models inside the `./models` folder. -- **Fetcher** : Class containing a set of methods to receive query parameters, extract data and transform it, if necessary. +- **QueryParams** : The input model for a particular query. To load stock market data, we would have `StockQueryParams`, which would have fields like `symbol`, `start_date`, and `end_date`. You can find the standard query params inside the `standard_models` directory. +- **Data** : The output model of a particular query. Stock market data would be `StockPriceData` and have fields such as `Open`, `High`, `Low`, `Close`, and `Volume`. You can find the standard data models inside the `standard_models` directory. +- **Fetcher** : Class containing a set of methods to receive query parameters, extract data and transform it. This class is responsible for implementing the standardization procedures. - **Provider** : Entry point class for each provider extension. Contains information about the provider, it's required credentials and available fetchers. -- **RegistryLoader** : Loads the registry with the provider extensions installed. +- **RegistryLoader** : Loads the registry with the installed provider extensions. - **Registry** : Maintains a registry of provider extensions installed. -- **RegistryMap** : Stores the complete characterization of each provider. It centralizes information like required credentials, standardised and extra query parameters/data by provider. +- **RegistryMap** : Stores the complete characterization of each provider. It centralizes information like required credentials, standardized and extra query parameters/data by provider. - **QueryExecutor** : Executes a given query, routing it to the respective provider and fetcher. ## Installation