* post-install clarification, fix links * more links * change word * toolkit cleanup * rename 'contributing' to 'development' * update extensions * some urls * Update sdk-platform and v4-develop in urls (#5730) * Update sdk-platform and v4-develop in urls * Seprate Intrinio financials data tags into a different command (#5723) * add `ebit` field in income statement standard model * only add required data tags * remove calculations data * use `name` key instead of `tag` key * set `period` as required in data * remove `period` and `cik` from provider models * add `equity/fundamental/search_financial_attributes` with tests * clean descriptions file * add `equity/fundamental/financial_attributes` with tests * set `value` as `Optional` * Fix threadpool test error (#5734) * Remove ultima from deps (#5736) * remove ultima from deps * skip ultima test * black * change skip location * try except statement to avoid error on the ci * new currency reference rates pr (#5724) * point files to linters (#5739) * remove news from equity (#5741) * Add tiingo provider (#5740) * first commit * things * Add news endpoints * CryptoHistorical * Add currency and trailing dividend yield * things working * deps * fix pre commit things * tests? * black on the tests * pratty told me to add it * integration tests * added helpers * crypto revamped * currency revamped * trailing dividend yield revamped * equity historical revamped * company news overhauled * global news overhauled * remove `tags` param from news models * crypto correct `exchanges` description * james smh * re-recorded fetcher tests * fixed integration tests * econometrics smh * james smh again --------- Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> * Add `derivatives` extension for `futures` and `options` (#5742) * removed options and futures * add derivatives extension * cleanup * bug #5649 (#5722) * bug #5649 * fixing --a upmom, downmom --------- Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com> * Rename the fmp search -> screener (#5744) * Rename the fmp search -> screener * misses some * i cant copy paste * docs/fix-urls: Updates the Terminal About URLs to match `/develop` (#5720) * update Terminal about URLs and some other broken links * ruff * intro * changing the heading levels * dependency management sections * structure * developer guides * structure * contributor guides * keywords and descriptions * broken urls * cool cards * Cool cards (#5727) * tidy up terminal/content * some more tidy up * fix styling for platform that was missing * more tidy up * improve color for dark and light mode * create new reference card for more basic cards * rename back to menus * implement all index.md for terminal, except reference * improve bot title * more refactoring * fix remaining index * header styling * more styling * heheheheheheeh * Revert "heheheheheheeh" This reverts commit7d095fcfd2. * Ccards (#5733) * links * more links * improved styling and chevron aligned to title * fix index.mdx urls for Terminal * update package.json * update index.mdx * fix generation * spelling * toolkit_extensions * update data extensions * couple of links * add tiingo to data extensions list --------- Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt> Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com> Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt> Co-authored-by: James Maslek <jmaslek11@gmail.com> * Stock leftover cleanup (#5745) * change to equity in `.md` files * `HistoricalStockSplits`->`HistoricalSplits` * `StockInsiderTrading`->`InsiderTrading` * sorry fmp * `EquityHistorical` cleanup * `EquityNBBO` cleanup * .md leftover * the last of stocks * post-install clarification, fix links * more links * change word * toolkit cleanup * rename 'contributing' to 'development' * Hotfix/urllib to dev group (#5732) * use the urllib3 dependency on the dev group instead * adapt the version script for the nightlies - if it's a nightly the package distribution name will have a different name * add dev dependencies to the dev installation bc of the ci * updating the import on validate_call according to: https://docs.pydantic.dev/latest/concepts/validation_decorator/ the way we were importing it breaks if pydantic=2.5 @montezdesousa * loosing urllib3 version specification on the provider and adding it to platform dev deps * rmeoving urllib from top level deps; and removing --with dev flag * changing to ^ instead * Revert "changing to ^ instead" This reverts commit7300904e9f. * unnecessary dep --------- Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com> Co-authored-by: James Maslek <jmaslek11@gmail.com> * feature/v4-bop - Add ECB Balance of Payments (Replaces #5716) (#5726) * add ecb balance of payments * missing __init__.py files * Nasdaq calendar earnings and fix FMP calendar earnings (#5718) Co-authored-by: James Maslek <jmaslek11@gmail.com> * feature/us-treasury-auctions: Adds Treasury Auctions endpoint and provider for US Government Data. (#5746) * add government_us provider and treasury auctions endpoint * black * add param for cusip * move logic for setting default start/end date * Update hub<>platform credentials map (#5749) * update hub<>platform credentials map * typo * Add API settings schema to system settings (#5743) Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com> * Skip checking creds (#5737) * skip checking creds * fix bug * ok mypy * add comment * rename required_credentials * fix * fix top_retail bug * don't require creds in some nasdaq fetchers * black * this was not here before * fix tests * update fetcher test * update contributing guidelines * Fix Quote (#5752) * Fix Quote * fixed return type and model validation --------- Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> * Wrap up renaming global news to world news on a model level (#5750) * Wrap up renaming global news to world news on a model level * cleanup --------- Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> * update extensions * some urls * Revert changes to v3 docs --------- Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt> Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com> Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com> Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com> Co-authored-by: James Maslek <jmaslek11@gmail.com> Co-authored-by: Ramazan <guvencramazan@yandex.com> Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com> Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt> * POST request example * update post example * find symbols * market calendars * codespell * codespell ftw * improve main page light mode --------- Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com> Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt> Co-authored-by: Pratyush Shukla <ps4534@nyu.edu> Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com> Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com> Co-authored-by: James Maslek <jmaslek11@gmail.com> Co-authored-by: Ramazan <guvencramazan@yandex.com> Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com> Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>
5.5 KiB
Vendored
title, sidebar_position, description, keywords
| title | sidebar_position | description | keywords | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Add a Data Point | 3 | This guide provides detailed instructions on how to add a new data point to the OpenBB Platform. It covers the process of creating a new provider, defining query parameters and data output models, and building a Fetcher class. |
|
import HeadTitle from '@site/src/components/General/HeadTitle.tsx';
In this section, we'll be adding a new data point to the OpenBB Platform. We will add a new provider with an existing standard data model.
Identify which type of data you want to add
In this example, we'll be adding OHLC stock data that is used by the obb.equity.price.historical command.
Note that, if no command exists for your data, we need to add one under the right router. Each router is categorized under different extensions (stocks, currency, crypto, etc.).
Check if the standard model exists
Given the fact that there's already an endpoint for OHLCV stock data, we can check if the standard exists.
In this case, it's EquityHistorical which can be found in /OpenBBTerminal/openbb_platform/platform/core/provider/openbb_provider/standard_models/equity_historical.
If the standard model doesn't exist:
- you won't need to inherit from it in the next steps.
- all your provider query parameters will be under the
**kwargsin the python interface. - it might not work out-of-the box with other extensions that follow standardization e.g. the
chartingextension
Create Query Parameters model
Query Parameters are the parameters that are passed to the API endpoint in order to make the request.
For the EquityHistorical example, this would look like the following:
class <ProviderName>EquityHistoricalQueryParams(EquityHistoricalQueryParams):
"""<ProviderName> Equity Historical Query.
Source: https://www.<provider_name>.co/documentation/
"""
# provider specific query parameters if any
Create Data Output model
The data output is the data that is returned by the API endpoint.
For the StockHistorical example, this would look like the following:
class <ProviderName>EquityHistoricalData(EquityHistoricalData):
"""<ProviderName> Equity Historical Data.
Source: https://www.<provider_name>.co/documentation/
"""
# provider specific data output fields if any
Note that, since
EquityHistoricalDatainherits from pydantic'sBaseModel, we can leverage validators to perform additional checks on the output model. A very good example of this, would be to transform a string date into a datetime object.
Build the Fetcher
The Fetcher class is responsible for making the request to the API endpoint and providing the output.
It will receive the Query Parameters, and it will return the output while leveraging the pydantic model schemas.
For the EquityHistorical example, this would look like the following:
class <ProviderName>EquityHistoricalFetcher(
Fetcher[
<ProviderName>EquityHistoricalQueryParams,
List[<ProviderName>EquityHistoricalData],
]
):
"""Transform the query, extract and transform the data."""
@staticmethod
def transform_query(params: Dict[str, Any]) -> <ProviderName>EquityHistoricalQueryParams:
"""Transform the query parameters."""
return <ProviderName>EquityHistoricalQueryParams(**transformed_params)
@staticmethod
def extract_data(
query: <ProviderName>EquityHistoricalQueryParams,
credentials: Optional[Dict[str, str]],
**kwargs: Any,
) -> dict:
"""Return the raw data from the endpoint."""
obtained_data = my_request(query, credentials, **kwargs)
return obtained_data
@staticmethod
def transform_data(
data: dict,
) -> List[<ProviderName>EquityHistoricalData]:
"""Transform the data to the standard format."""
return [<ProviderName>EquityHistoricalData.model_validate(d) for d in data]
Make sure that you're following the TET pattern when building a
Fetcher- Transform, Extract, Transform. See more on this here.
Make the provider visible
In order to make the new provider visible to the OpenBB Platform, you'll need to add it to the __init__.py file of the providers/<provider_name>/openbb_<provider_name>/ folder.
"""<Provider Name> Provider module."""
from openbb_provider.abstract.provider import Provider
from openbb_<provider_name>.models.equity_historical import <ProviderName>EquityHistoricalFetcher
<provider_name>_provider = Provider(
name="<provider_name>",
website="<URL to the provider website>",
description="Provider description goes here",
credentials=["api_key"],
fetcher_dict={
"EquityHistorical": <ProviderName>EquityHistoricalFetcher,
},
)
If the provider does not require any credentials, you can remove that parameter. On the other hand, if it requires more than 2 items to authenticate, you can add a list of all the required items to the credentials list.
After running pip install . on openbb_platform/providers/<provider_name> your provider should be ready for usage, both from the Python interface and the API.