diff --git a/openbb_providers/README.md b/openbb_providers/README.md index 34c0f62814c..df3bf29e7b8 100644 --- a/openbb_providers/README.md +++ b/openbb_providers/README.md @@ -4,9 +4,9 @@ - [THIS README IS A WORK IN PROGRESS AND CAN BE VERY MUCH OUT OF DATE. REFRESH THE PAGE UNTIL THIS BANNER IS GONE](#this-readme-is-a-work-in-progress-and-can-be-very-much-out-of-date-refresh-the-page-until-this-banner-is-gone) - [Table of contents](#table-of-contents) -- [Temporary glossary](#temporary-glossary) - [openbb-provider](#openbb-provider) - - [Functional Requirements](#functional-requirements) + - [What is Data Standardization?](#what-is-data-standardization) +- [Key Elements](#key-elements) - [How to use it?](#how-to-use-it) - [Prerequisites](#prerequisites) - [1. Defining : OpenBB QueryParams](#1-defining--openbb-queryparams) @@ -22,31 +22,32 @@ - [2. Entrypoint](#2-entrypoint) - [3. Install the extension](#3-install-the-extension) -# Temporary glossary - -- **Provider** : A Provider is a Financial Data Provider such as FMP, Polygon, Benzinga, etc. -- **QueryParams** : A set of parameters that define a request to a Provider. It is defined by the user. -- **Provider QueryParams** : A set of parameters which are defined by the Provider. -- **Data** : A standardized set of data that is returned by a Provider and then standardized. -- **Provider Data** : Provider Data is a raw set of data that is returned by a Provider as is. - # openbb-provider -This library aim to give a coherent access to Financial Data Providers. +The openbb_providers aim to give a coherent access to Financial Data Providers by introducing standardization procedures. -It provides coherence on: +## What is Data Standardization? +OpenBB Data Standardization is a process of transforming data into a common format to allow for efficient and accurate information processing, comparison, and analysis. -- QueryParams : one can define QueryParams and use them on multiple Financial Data Providers. -- Data : one can process the data provided by multiple Financial Data Providers as if there were from the same one. +Our data standardization can be considered as a sub-process of more general data normalization. There are some things that we do and don’t do when we apply OpenBB Data Standardization. -## Functional Requirements +**We do**: +**Consistent Column Name Coding and Formats**: we ensure that data are consistently coded on a column name and column type level. -Functional requirements are the following: +**Examples**: +Close for closing price column name in data from one provider might be coded as c in data from another provider Date and time can come in formats like YYYY-MM-DD, MM-DD-YYYY, and Unix timestamps in data from different providers. Standardization would bring these to a common format. -- Provide a standardized access to similar Data from multiple Data Providers. -- Use the same QueryParams to query similar Data from multiple Providers. -- Get similar Data from multiple Provider and process it as if there were from the same Provider. +**We don’t do**: +**Data Cleaning**: we don’t remove errors and inconsistencies from data provider outputs. +**Data Transformation & Data Integration**: both transforming data by processing it and integrating data from multiple providers together to harmonize datasets are functions of the SDK and are not handled on a provider level. +# Key Elements + +- **QueryParams** : Are input parameters that are used for obtaining data. To load stock market data, we would have StockQueryParams, which would have fields like symbol, start date, and end date. +- **Data** : Are outputs. Stock market data would be StockPriceData and have fields such as Open, High, Low, Close, and Volume. +- **Fetcher** : A provider-specific object (FMPStockPriceFetcher) that holds executable logic that performs the standardization procedure going from QueryParams to the Data. +- **Provider** : An object that is specific to a data provider (i.e. PolygonProvider) which holds Provider attributes and unites all of the provider's fetchers into a list. +- **Provider Registry** : An aggregation of all the Provider objects and their API keys. It is the main point of usage and is responsible for dynamically serving the data by finding the correct Provider and Fetcher based on the given QueryParams. # How to use it? ### Prerequisites