Files
OpenBB/CONTRIBUTING.md
Pratyush Shukla a42c5604e3 [Release] 4.1.5 (#6190)
* Parameterize `obbject.to_df` with `index` and `sort_by` (#5877)

* Parameterize obbject.to_df with index and sort_by

* unit tests for obbject

* keep using the date as default to use on  basemode_to_df()

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Changes to the `obb.fixedincome.corporate.hqm()` output (#5876)

* changes to the hqm output

* typo

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Refactor method in RegistryMap (#5879)

* refactor extract_info

* add comment

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* remove Literal from providers in OBBject (#5882)

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Fix version (#5883)

* Fix version

* Fix linter on terminal

* Refactor package builder (#5880)

* breakdown builder + remove black

* rebuild

* rebuild

* rebuild

* rename method

* format built code without black

* we still lint if it's available

* fix existing tests

* add linter tests

* add black to dev deps

* auto_build as instance method

* reorder methods

* typing

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* small refactor for simplification (#5886)

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Fix bug. Add router test. (#5888)

* Fix bug. Add router test.

* Simplify

* separate pylint between platform + terminal (#5892)

* separate pylint b/w platform + terminal

* Explicitly set pylint check scope to files in the diff

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* sync descriptions (#5893)

* Add REST API docs page with advanced settings configuration (#5890)

* Don't mention docker in a list of python envs

* Create a REST API page

* Update docs to get api keys from the hub

* Update installation instructions

* Mention requirement of poetry and toml in source installation section

* move rest page up to the top. and add url to google colab example notebook

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* fix a v3 syntax example (#5891)

* Adds federal reserve data points (#5885)

* Allow optional input for 2-month treasury rates and add FederalReserve rates model

The commit includes two main changes. One being the modification to the standard treasury rate models, where the 2-month input field is now optional, and a default value is set to none. The other main change is the addition of a new module for the FederalReserve treasury rates model, which includes query parameters, a data model, and a data fetcher.

* Add Money Measures models and commands

This commit introduces new Money Measures models to handle money-related data, defines Money Measures commands in the economy router, and implements the Federal Reserve Money Measures model. The core model includes fields like start_date, end_date, and various components of the money supply. The new command added to the economy router allows the querying of these data.

* Add Federal Reserve Money Measures test file

This commit adds a test case file for the Federal Reserve Money Measures fetcher in the Federal Reserve module. The test case includes simulating a GET request and the corresponding expected response from the server.

* Cleanup

* Sneaky unformatted integration tests

* its a dawg life for me ...... ruff

* add the fed funds rate

* didnt mean to commmit this

* black

* this is why we should pre commit kids

* change install name

* add providers to int tests

* review comments + commit static

* replace nan with None

* fix float on None

* get rid of nans for nulls for json handling

* change `_` to `-` in readme pip install.

* Update __init__.py docstring

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Check if outdated (#5898)

* Add econometrics API tests (#5899)

* Docs/excel update (#5889)

* update xl docs

* update syntax line

* update again

* Fix pylint scope (#5895)

* Allow optional input for 2-month treasury rates and add FederalReserve rates model

The commit includes two main changes. One being the modification to the standard treasury rate models, where the 2-month input field is now optional, and a default value is set to none. The other main change is the addition of a new module for the FederalReserve treasury rates model, which includes query parameters, a data model, and a data fetcher.

* Add Money Measures models and commands

This commit introduces new Money Measures models to handle money-related data, defines Money Measures commands in the economy router, and implements the Federal Reserve Money Measures model. The core model includes fields like start_date, end_date, and various components of the money supply. The new command added to the economy router allows the querying of these data.

* Add Federal Reserve Money Measures test file

This commit adds a test case file for the Federal Reserve Money Measures fetcher in the Federal Reserve module. The test case includes simulating a GET request and the corresponding expected response from the server.

* Cleanup

* Sneaky unformatted integration tests

* its a dawg life for me ...... ruff

* add the fed funds rate

* didnt mean to commmit this

* black

* this is why we should pre commit kids

* change install name

* add providers to int tests

* review comments + commit static

* replace nan with None

* fix float on None

* get rid of nans for nulls for json handling

* change `_` to `-` in readme pip install.

* Update __init__.py docstring

* Debug pylint

* Adjust the fetch depth

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* fix `from openbb import obb` asyncio lock in python REPL (#5896)

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* feature/financials - PR for working on structure of financial statements. (#5794)

* balance sheet draft

* balance_sheet draft

* cash_flow draft

* income_statement draft

* intrinio financials

* description

* period field

* ruff

* income statement

* field

* label - gross profit ratio

* cash flow statement

* investing order

* add option for caching to cashflow

* add caching to balance and income

* typos and update order for financial institutions

* replace 0 values with None

* add financial ratios from intrinio

* black

* missing test params

* balance cassette

* query param description

* symbol list warning

* black

* handle BRK-A and symbols with a dash for Intrinio

* fix typo error

* add free cash flow to bottom of cash flow statement

* amortization_expense

* dev_install.py

* undo dev_install.. wrong branch

* cash flow order things

* standardized cash flow

* fields

* integration tests

* cash flow model validator

* more polygon cash

* black

* black

* Delete openbb_sdk/providers/polygon/openbb_polygon/utils/definitions.py

wrong file

* balance sheet

* some cleanup

* ruff

* income statement

* black

* test params

* financial ratios async update

* recapture fmp and polygon tests

* yFinance financials

* yfinance tests

* delete financial_statements standard model

* intrinio balance sheet order

* fiscal year standard param

* remove file accidentally committed

* add reported_financials end point

* fiscal_year param for reported_financials

* fix url

* fmp test cassette

* add safe-guard with warning for fiscal year before 2008

* periods can only be annual and quarter

* url in docstring correction

* add fiscal_year param to intrinio standardized financials and ratios

* fix fmp ratios

* black

* rename fields as per Minh

* rogue field

* black

* Fix typo in a docstring

* Resolve linting errors

* type ignore

* disable=unused-argument

* changing the arg name cc @minhhoang1023

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* docs/legacy-install: Add warning at top of page - Instructions on this page are for installing the Terminal (#5904)

* add warning at top of page - instructions are for installing the Terminal app

* pip install openbb-nightly

* hotfix/financials-field-maps: Updates A Couple FMP/Polygon Income Statement Fields (#5907)

* update income statement fields

* another polygon field

* ratio -> margin

* black

* explicit fmp fields

* add reported_currency for standardized intrinio

* # pylint: disable=unused-argument

* another unused-argument

* black..

* patch tests

* polygon income: eps -> basic earnings per share

* re-capture terminal portfolio menu test

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Fix test (#5916)

* Fix test

* Skip ultima integration tests

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* hotfix/fmp-price-performance-zeros: Replace returned zero with None and normalize percent values. (#5918)

* replace zeros with none and normalize percents

* pylint unused argument

* Update index.md (#5906)

* Release/4.1.0 (#5901)

* Hotfix/terminal tests (#5919)

* skip supressed arguments

* silent way of customly load on quote

* specify source for call_load

* docs/platform-faq: Start Platform FAQ page with bullet points on Platform vs SDK (#5914)

* platform vs sdk faq page

* add data and data providers faq section and some style cards

* typo

* makes changes as per reviews

* openbb-nightly as a way to install everything

* jupyter-lab instead of jupyter

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* hotfix/backend - Initialize OpenBBFigure (#5912)

* create backend

* openbb_figure

* black

* pylint

* pylint again

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* hotfix/yf-futures-historical: yfinance historical futures multi-ticker support and symbol parsing (#5923)

* multi-ticker support for yfinance futures historical

* raise-missing-from

* integration test params

* equity_historical adjusted kwargs

* pylint unused argument

* Docs/excel examples (#5920)

* regen excel md

* add syntax + basic example

* fix example

* create json with docs map

* update docs to latest functions.json + remove tiingo

* remove space from examples

* minor fix

* remove spaces

* remove spaces

* bug fix: remove folder before generating

* fix typo

* add link to appsource

* fix

* Update templates.md

* regenerate content

* change docs colors

* fix light mode

* add back tiingo

* update functions.json

* templates go later

* remove Library from reference

* Make the excel markdown being generated at runtime

* fix reference cards having commands under menu

* remove excel/reference folder since this is built at runtime by gh-pages action

* black formatting

* improve title

* add excel to the main page

* add excel image bg

* make excel sidebar show up next to pro

* improve main page

* small fix for platform page title

* update basics page

* add tip to autocomplete

---------

Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>

* Re-record tests and fix symbol bug (#5924)

* Re-record tests and fix symbol bug

* nordvpn us ftw

* @IgorWounds says "SPY >> IOO"

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Docs/bug fix (#5926)

* fix pro link

* fix wrong example

* Introduces a limit argument for the `equity.fundamental.trailing_dividend_yield` endpoint (#5925)

* introduces a limit argument for the equity.fundamental.trailing_dividend_yield endpoint

* add default args to test

* Add json_schema_extra to openapi.json (#5930)

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* feature/standardize-quote: Proposal to Standardize EquityQuote (#5922)

* standardize EquityQuote

* intrinio reference

* rogue comma

* pylint

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* hotfix/fmp-rev-segments: Adds Fiscal Period to busines/geographic segments revenue. (#5921)

* add fiscal_period to revenue by geography and business line

* data description and unused imports

* update test cassette

* Allow multiple symbols `equity.profile` (#5929)

* allow multiple symbols

* pylint

* pylint ignore position

* change callback to helper func

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Add filtering to AV historical price (#5928)

* add filtering to av historical price

* lints

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Fix vwap FMP error (#5936)

* Fix vwap FMP error

* Linting

* Painlint

* Add date filter + symbols to comp (#5931)

* add date filter + symbols to comp

* make sure year is in d

* tests fix

* Improving Platform's import time (#5894)

* missing decorator

* missing optional statements

* us-gov to optional and right position

* removing redundant call

* lazy load mappings

* docstrings

* introducing the extension loader class

* misleading docstring

* using the extension loader on the router

* using the extension loader on the credentials

* docstrings

* lazy load of entry points

* checkout dev_install

* using the extension loader to get the entry points instead of recalling the function

* improved auth service and charting service

* using the extension loader on the registry

* typos

* adding properties to extension loader for easy access

* using the extension loader on the package builder

* avoiding circular imports by using forward references

* fix charting service tests

* test for the extension loader

* Update openbb_platform/core/openbb_core/app/model/credentials.py

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* Update openbb_platform/core/openbb_core/app/router.py

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* Update openbb_platform/core/openbb_core/provider/registry.py

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* adjustments

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* Disable auto_build on test run (#5937)

* Disable auto_build on test run

* Move to conftest

* Update excel docs (#5940)

* update basics

* update reference generator

* Categorize release notes based on labels for OpenBB Platform (#5943)

* Categorize release notes based on labels for OpenBB Platform

* Add environment variable for GitHub token in labels-PR.yml

* Update labeler.yml to use "enhancement" label for feature branches

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Custom deprecation message in API endpoints and Python commands (#5942)

* add custom deprecation message feature in routers

* add custom deprecation message feature in package builder

* using an actual DeprecationWarning instead

* fix tests

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Hotfix/remove unused workflow (#5945)

* Disable auto_build on test run

* Remove unused workflow

* Revert

* Hotfix/fix typo (#5948)

* Disable auto_build on test run

* Fix typo

* Revert

* docs/data-models-url: Fix URLs on Development Page (#5944)

* fix url to the standard models

* github links

* `equity.fundamental.dividends`: allow filter by date (#5932)

* allow filter by date

* ruff

* pylint

* fix tests

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Create Excel templates page (#5947)

* create templates page

* Update index.mdx

* fix description

* update card

* remove clsx

* Update index.mdx

* Update index.mdx

* Update index.mdx

* Update index.mdx

* min height

* min height

* improve excel docs and point towards hub

---------

Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>

* hotfix/intrinio-dividends-limit: Fix Limit Param Alias (#5951)

* fix limit param

* pylint unused argument, recapture test cassette

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Docs/remove core refs (#5957)

* remove references to core: PathHandler + ProviderInterface

* remove unused function

* add comment

* regenerate

* fix a few examples

* add comment

* rename file

* Excel tutorials on the docs (#5956)

* update workflows name to tutorial

* update title

* get barebones to bring in Excel videos

* tutorial excel videos in

* hotfix/float-values: Convert Intrinio balance sheet values to float type (#5961)

* convert intrinio balance sheet values to float type

* unused import

* typo in alias dict

* Update to OpenBB Add-in for Excel (#5965)

* Log custom headers (#5966)

* add custom headers

* fix: send on startup and defaults

* pylint

* unittests

* move government-us to community in dev_install (#5963)

* fix: adds logic to check whther user is logged in on hub (#5941)

Co-authored-by: Luqman <luqazino@gmail.com>

* hotfix/fix-company-filings: Fixes too many mandatory fields. (#5909)

* Fixes too many mandatory fields.

* black

* black again

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* `equity.estimates.price_target` - Integrate with Benzinga (#5946)

* deprecation message

* fix lints

* adding equity.estimates.ratings and fmp as its provider

* unit tests for fmp

* removing unused file

* typos

* add benzinga to AnalystRatings

* revert changes to estimates router

* moving it to price target instead

* reverting changes

* fmp support for multiple ticker

* benzinga price target

* ruffy

* unused argument

* fix type hints

* tests for the new fetcher

* new integration tests

* fixing tests

* missing std arg

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [DEPRECATION] `index.market` renamed to `index.price.historical` (#5962)

* deprecate index.market and add index.price.historical

* add index.historical standard model

* add index.historical provider models

* index extension tests

* provider tests

* linting

* bug fix (#5969)

* Uppercase excel reference (#5958)

* uppercase reference

* fix name header

* add tab title

* remove reference excel folder

* add to gitignore excel reference files

---------

Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>

* move templates (#5970)

* Release 4.1.1. (#5964)

* update dev_install

* fix effr test

* Update dev_install.py

* fix fundamental balance test

* fix equity price historical test

* fix price target model

* black

* rebuild static files

* update publish.md

* fixed dev_install

* update publishing doc

* updated `openbb-core` to 1.1.1

* updated extensions

* updated providers

* updated `openbb` package

* updated static extension map

* updated publishing doc

* fixed toml for `openbb-finra`

* @jmaslek yfinance IS DEFAULT

* updated static

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Fix link to discord channel (#5910)

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Fixing Posthog handler for thread safety (#5968)

* fixing posthog handler for thread safety

* removing loggging init from the global scope - per sensei juan

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Update labeler.yml and labels-PR.yml (#5974)

* hotfix/ allow multiple `identify` calls (#5976)

* allow multiple `identify` calls

* Update posthog_handler.py

* regex out any invisible control character from sales_conditions field (#5977)

* docs/http-requests: Contributor Guide For Building HTTP Requests (#5973)

* contributor guide for http requests

* add a docstring

* add section on get_querystring

* extra space

* add note for people installing PyPI in a zsh terminal shell

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* feature/add-yfinance-functions: Add equity.profile for yFinance (#5978)

* equity profile for yFinance

* pylint

* equity quote

* unused imports

* remove yfinance from extras in toml (#5982)

* Release OpenBB Platform v`4.1.3` (#5983)

* fixing pyproject.toml

* adding pandas-ta to technical package for publishing

* bumping technical

* test for pyproject toml

* feature/cboe-async: Refactor Cboe for Async (#5900)

* refactor cboe for async

* codespell

* unused imports

* pylint

* more pylint

* platform metadata

* classmethod decorator

* test params

* intrinio quote

* test param names

* date in test needs to be a datetime.date not string

* make fmp unix timestamp tz-aware and normalize percent change

* different way of parsing datetime string

* recapture cboe equity historical test

* unused import

* index snapshot update

* black

* extension map

* pylint

* cboe pyproject.toml

* proposal to standardize equity_quote

* one more file

* removing changes on static assets

* index_historical

* index_historical again

* last_time -> last_timestamp

* obsolete router path

* fix tests

* test param

* restore market index deprecation that I forgot about

* fixing tests

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Fix typo in  website/README.md (#5980)

* Fix typo in  website/README.md

* Update linting.yml

* Update linting.yml

* Update linting.yml

* Change ref conditions

* Update linting.yml

---------

Co-authored-by: Luqman <luqman.bello@openbb.finance>
Co-authored-by: Luqman <luqazino@gmail.com>

* Hotfix/th etfs (#5959)

* added etf trading hours

* black formatter

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Luqman <luqazino@gmail.com>

* feature/finviz-provider: Adds Community Finviz Provider Extension (#5952)

* add finviz provider

* recapture tests

* recapture tests again?

* invalid test param

* static packages

* units_measurement -> unit_measurement

* static packages as core only

* openbb_core version bump in pyproject.toml

* try adding user-agent to test response

* fix tests

* linting

* provider model cosmetic changes

* fix install script and typo in openbb toml

* static

* fix: re-record unittests

* empty data error

* definitions as a file

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* Add support for multiple symbols on `obb.equity.price.historical` (#5984)

* av - support for multiple symbols

* tiingo - support for multiple symbols

* include symbol only if necessary

* update extension descriptions page (#5985)

* remove useless file (#5991)

* added symbol to headers (#5981)

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Fixing Docs Typos (#5955)

* fixed typo

* fixed stocka typo

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>

* fix nasdaq test cassettes (#5989)

* don't include `custom_headers` in openapi schema (#5994)

* hotfix/intrinio-index: Drop None Rows From intrinio Historical Index (#5987)

* drop None rows from intrinio historical indexes

* add empty data error

* static assets

* hotfix/insider-trading: Fix #5988 (#5990)

* fix validation errors

* empty strings

* validator

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>

* CI listing quick fix (#6002)

* BIGGGG LINTING

* fixing lints

* fixing lints

* black

* very ruff

* no export

* fix hedge_view again

* lints

* platform lints

* lints

* black

* black it @hjoaquim

* fix some more linting

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Release `openbb-finviz` `1.0.0` to the public (#6000)

* set version to `1.0.0`

* Set correct dep version in main package

* Add `finviz` to table

* Auto Labelling for OpenBB Excel Docs Changes (#5998)

* Update pull request labels

* Update labeler.yml with new label for Excel files

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Support for custom examples in router commands (#5993)

* Disable auto_build on test run

* Add support for custom router examples

* Revert odd file change

* Fix API pollution

* Refactor examples with @montezdesousa

* Check if model is inside the PI

* Fix

* feat: add & fix examples (#6001)

* feat: add & fix examples

* fix: ruff + comment

* feat: read parameter pool from file

* feat: typing + unit test

* Disable auto_build on test run

* Add field order to OBBject

* Revert

* lint and revert

* fix test

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* hotfix/intrinio-options-date: Fix comparing values between str and datetime (#5997)

* fix compare between str and datetime

* unused argument

* get_weekday

* lint

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Field order added to the OBBject extra as preference (#5999)

* Disable auto_build on test run

* Add field order to OBBject

* Improvements

* Remove noxfile

* Disable auto_build on test run

* Add field order to OBBject

* Revert

* lint

* Add docs

* Not sure how this happened

* lint

* Update `openbb-devtools` (#6008)

* update deps

* unpinned package versions from gh linting action

* codespell

* updated black dependency

* linting

* linting

* linting

* pin codespell version

* black

* black again

* install devtools in linting action

* format w new black (#6009)

* format w new black

* format w new black

* black again

* black the terminal

* ignoring unused arguments

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* remove reviewpad (#6010)

* Remove Ultima provider (#6012)

* Add OECD unemployment (+more) + data cache idea (#5917)

* Add OECD unemployment + data cache idea

* Docstring edits

* Tests

* Add composite leading indicator

* i truested ruff --fix too much

* Add short + long term interest rates

* pylint

* pylint2

* was not committing.  my b

* add csv cache + make it default.  remove pyarrow from poetry

* Clean up

* Didnt run on this file.  My bad

* Why was this ignored /shrug

* pylint

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Feature/improved pr template (#6018)

* format w new black

* Revert "format w new black"

This reverts commit 74b9619cc6.

* better pr templates

* developer template

* [Feature] - Custom deprecation (#6005)

* custom deprecation

* custom deprecation

* using the new deprecation

* custom deprecation on the package builder

* remove comment

* ruff

* black

* static assets

* tests

* using parametrization instead

* test for deprecated endpoints (#6014)

* Deprecation warning on the reference docs (#6015)

* typo/fix

* bring back methods needed for markdown generation

* add deprecation warning to docs

* contributor docs for deprecating endpoints - tks @deeleeramone

* small changes on publishing procedure per @the-praxs

* moving the deprecation summary class to deprecation file instead

* explanation on class variables

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update openbb_platform/openbb/package/index.py

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* deprecating on 4.3 instead @the-praxs

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* [BugFix] Fix package builder (#6022)

* fix: remove specific imports

* adjustments for correct deprecation

* static assets

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* [Feature] Add support for multiple tags in `historical_attributes` and `latest_attributes` (#6013)

* historical attributes standard model -
* add support multiple tags
* add `tag` field in data

* add support for multiple tags in historical attributes

* set correct return type in `callback` function

* latest attributes standard model -
* add support for multiple tags
* add `tag` field in data

* add support for multiple tags in latest attributes

* add test params for the attributes endpoints

* add support for multiple `symbol` and add `symbol` field in the data

* modified code to fetch data from multiple symbols and throw warning as per @deeleeramone

* add params for testing multiple symbols

* linting

* fix test params

* fixed condition to check if its a `Dict` type

* updated intrinio fetcher tests for the statements

* updated static

* [feature]: feature/overview-to-profile: Deprecate `equity.fundamental.overview()` and move to `equity.profile()` (#6011)

* fmp overview to equity profile

* deprecation warning

* urls

* wrangle fields

* black

* pylint unused argument

* insert additional securityID fields to standard model

* custom deprecation using OpenBBDeprecationWarning

* typo - need to have the deprecated=True flag

* static assets

* fix non related test

* minor adjustment - removing the need for the try/except block - using the get and pop removes the need for key existance checking. cc @deeleeramone

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* [BugFix] - Remove duplicate code (#6025)

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* remove `SectorNews` from router and standard models (#6026)

* feature/more-yfinance: Adds some more to openbb-yfinance (#5979)

* equity profile for yFinance

* pylint

* equity quote

* unused imports

* price target consensus

* add currency field

* add share_statistics

* don't append empty result to results

* key executives

* missing definition in key_executives

* pylint

* key metrics

* add currency to key metrics

* add etf_info

* rename trailing_annual_dividend -> dividend_rate_ttm

* add institutions count to share_statistics

* add historical dividends

* add logic for start-end dates to historical_dividends

* recapture the test..?

* funny test

* filter on tests?

* record test with clean cache..?

* year_price_change -> price_return_1y in key_metrics

* black

* make descriptions clear for beta and dividend_yield fields

* add description field to standard model

* updated static

* clear description of shares_implied_outstanding field

* updated static for equity

* clean descriptions

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* feature/commodities-lbma-fixing: Adds Commodity Extension and LBMA Gold/Silver daily fixing levels (#5760)

* add commodity extension and add lbma fixing endpoint

* dev_install.py

* readme url

* renovate the standard model

* makes a query_params model for nasdaq data link

* literal not optional

* unused imports

* missing test params

* test cassette

* DataLink query param description

* test file import

* actually fix import

* from openbb_core.provider.....

* ruff

* another import statement

* black

* async router

* move query_params

* update version

* version numbers

* extension_map

* satisfy pylint?

* pylint

* Update pyproject.toml

* black

* poetry lock

* Update extension_map.json

* Update __extensions__.py

* Update module_map.json

* clean descriptions

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Improve UX for `obb.news` (#5986)

* adding date,start_date and end_date to std params

* add date to special params which should've another type

* change types to dateType

* change helper to accomadate changes in types

* date filtering

* using the date filtering for all providers

* date params on the std model instead

* adding the right type

* filtering dates on world news

* integration tests

* fix descriptions

* static assets

* black

* Revert "black"

This reverts commit 7288b59f87.

* lint

* addressing @montezdesousa suggestion: removing qua 31 jan 2024 15:01:20 WET from standard models and using it only on benzing

* tests accordingly

* static assets

* black

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] - Improve Standardization - `IndexHistorical` (#6030)

* format w new black

* Revert "format w new black"

This reverts commit 74b9619cc6.

* small adjustment on the test

* index historical

* adjust tests accordingly

* keep the test skipped

* match tests w python

* fix tests

* keep the skip

* fix test

* [Feature] - Refactor Response Model (#6024)

* Start refactor with @montezdesousa

* feat: add response model inside openapi_extra

* feat: find+replace return types

* fix: ruff

* Feature/api response by provider (#6029)

* init

* Update router.py

* Update registry_map.py

* check dicts

* lint

* exclude provider on data model for package build

* `results_type_repr` logic for `Annotated`, ignore provider field in `_extract_data`

---------

Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>

* [BugFix] - Improve Standardization - `CotSearch` (#6031)

* format w new black

* Revert "format w new black"

This reverts commit 74b9619cc6.

* small adjustment on the test

* index historical

* adjust tests accordingly

* keep the test skipped

* match tests w python

* use cache as std field

* fix tests

* fix tests

* keep the skip

* fix test

* adding missing param

* fix tests accordingly

* Docs/byod (#5996)

* add byod to reference

* fix: rename add-in name

* feat: add data conectors page

* fix: additional notes

* tiny tweaks

* update functions.json and openapi.json

* fix: fix bug in hedge controller

* fix: another bug hedgecontroller

* black

---------

Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>

* Feature/ex-dividend: Renames "date" columns as "ex_dividend_date" for Dividend Calendar and Historical Dividends (#6006)

* date -> ex_dividend_date

* nasdaq historical dividends

* black

* pylint

* intrinio alias_dict

* dividend_currency -> currency

* yfinance field name

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] - Adds a timezone to the Unix timestamp conversion to correct localization errors. (#6023)

* fix: remove specific imports

* make polygon historical tz-aware

* undo including that commit

* that PR wasn't supposed to be there.

* pylint unused argument

* fix polygon crypto historical intervals

* pylint

* new cassette

* fix the rest of the polygon historical price query params

* black

* pylint..

* integration test param

* appease daylight savings handling

* paginate for crypto currency and index

* black

* raise EmptyDataError when no results instead of returning an empty list

* ruff

---------

Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* Hotfix/wrong pydantic validator (#6037)

* format w new black

* Revert "format w new black"

This reverts commit 74b9619cc6.

* Use field_validator instead

* fix openapi schema fields `to_snake` (#6036)

* Fix data connectors link typo (#6038)

* [BugFix] - Improve Standardization - `EquitySearch` (#6033)

* format w new black

* Revert "format w new black"

This reverts commit 74b9619cc6.

* small adjustment on the test

* index historical

* adjust tests accordingly

* keep the test skipped

* match tests w python

* use cache as std field

* fix tests

* fix tests

* use_cache as std

* adjust tests

* keep the skip

* re-record test

* black

* Docs / only display results for current platform (#6017)

* Update index.js

* Update index.js

* yfinance

* tzcache no longer exists

* fixxy

* ruff

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Andrew <andrew.kenreich@gmail.com>

* Improve the docstrings (#6040)

* Improve the docstrings (#6041)

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* dont run base tests with v4 label (#6046)

* Sync excel reference (#6039)

* sync excel reference

* fix: change delimiter to comma

* feat: update data connectors

* test skipping wf

* Update unit-test.yml

* Update unit-test.yml

* test skipping wf

* watch label changes

* Update unit-test.yml

* [Enhancement] - Improve `news` router docs and examples (#6047)

* better docs

* static assets

* Feature/platform standardization notebook (#6049)

* Add platform standardization overview notebook to examples

* Add py.typed marker to openbb-fred

* [Enhancement] - Improve `currency` router docs and examples (#6045)

* currency search

* reference rates

* historical price

* static assets

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* display warnings based on preference (#6051)

* [Enhancement] - Improve `crypto` router docs and examples (#6048)

* crypto search add validation for '-'-separated pairs

* set default query value to None

* improve /crypto description with custom examples

* improve /crypto/price/historical description with custom examples

* updated static for crypto menu

* ignore line too long

* linting

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Final Implementation of Changelog V2 (#6050)

* Updated the release drafter to process the released changelog for OpenBB Platform

* Update release drafter configuration

* Add OpenBB Documentation Changes and Create Release

* Update process_changelog.py path in draft-release.yml

* Update release-drafter version and save changelog

* Copy changelog with python

* Update draft-release.yml to save Changelog

* Update draft-release.yml

* Encode and save Changelog to Base64

* Update Changelog saving method

* Update Changelog format

* Update CHANGELOG.md with release notes

* Fix draft release workflow

* Add cat command to print CHANGELOG.md

* Add platform-drafter.yml and update release-drafter.yml and draft-release.yml

* Commented out config-name in release-drafter step

* Update branch name validation and process_changelog.py

* Refactor process_changelog.py to improve readability and maintainability

* Refactor process_changelog.py to handle file I/O errors and remove duplicate PR entries

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* improve discriminator logic, fix package return type docs (#6052)

* improve discriminator logic, fix package return type docs

* Update registry_map.py

* build package

* Update registry_map.py

* defaults

* Update registry_map.py

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Add LLM tools example notebook (#6053)

* Add LLM tools example notebook

* Update README, rename example notebook

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Remove the  preference (#6055)

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* [Feature] - Standardise multiple symbols input (#6056)

* feat: remove list of str

* fix: update cpi

* package builder + cpi

* build

* fix: update unittests

* fix: check_item

* fix: unittests

* Improve the docstrings (#6057)

* update economy router docstrings (#6058)

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] Fix OECD Long Term Interest Rate (#6060)

* fix long term interest rate

* pylint unused argument

* [Bugfix] - Fix economic calendar country (#6059)

* fix economic calendar country

* lint

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [HotFix] Index Router Docstrings & `sp500_multiples()` Cleanup (#6063)

* sp500 multiples cleanup

* remove dict from standard model

* static pacakge

* pylint

* [Feature] FMP Current Holdings (#6043)

* add fmp current holdings to etf holdings

* pylint unused argument

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Replace symbols in company/news (#6061)

* feat: replace symbols in /news

* fix: integration tests

* fix: integration tests api

* fix: typo

* Update test_intrinio_fetchers.py

* [Enhancement] Add function examples to router. (#6042)

* options router

* fixedincome router

* futures router

* missing comma

* static assets

* futures router

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Enhancement] Add function examples to Regulators Router (#6054)

* options router

* fixedincome router

* futures router

* missing comma

* static assets

* futures router

* sec router

* cftc router

* black

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] Fix World News (#6064)

* fix fmp

* fix intrinio

* fix biztoc

* actually commit intrinio fix

* fix tiingo

* unused import

* fix tests

* black

* ruff

* benzinga empty strings

* codespell

* fix test?

* more empty strings

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - remove unused validators from standard models (#6067)

* update standard models

* lint

* Update USD Liquidity Example Notebook for V4 (#5902)

* Update USD Liquidity Example Notebook for V4

* delete some v3 text in the copperToGold

* Update README.md

* findSymbols preview image

* Google Colab preview image

* financialStatements preview

* content tsx file

* ### title

* convert tsx to json

* indent size

* double quote keys.json

* trailing commas

* Update content.json

* patch

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Fix QA Examples (#6072)

* [Feature] - Ensure Bugfix branches can only do PRs to `develop` branches (#6073)

* bugfix branch name checks and few fixes on the linting not to run on pushes to branches outside PRs

* Update branch naming guidelines link

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Feature/charting obbject (#5938)

* fix circular import issues on the obbject extension framework

* settings available on the obbject

* extension init

* charting assets and code

* introduce to_chart functionality

* removes the to_chart method from obbject in detriment of the one from the obbject extension

* Update openbb_platform/obbject_extensions/charting/charting/__init__.py

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* fix on the show method

* adding the example to the to_chart docstring

* copying the exsting README from charting extension - needs to be revamped

* moving to openbb_charting instead

* update the pyproject.toml file

* pydocstyle

* versions

* better docstrings

* making accessors public

* removing unnecessary credentials variable cc @montezdesousa

* removing access to extra/metadata and use designated private attrs instead

* Fixes linting for the logging service (#5992)

* lints

* typing.Type instead

* deprecating ChartingService to use the new extension instead - adding function discovery capabilities using it

* tests for the obbject charting extension

* removing charting service from command runner

* filter functions not implemented in the module

* remove charting service tests

* removing charting service on the tests for int tests

* charting integration tests

* removing charting service file

* removing old charting extension code

* revamp builder

* fix obbject tests

* fix command_runner tests

* removing openbb_figure_table

* better website docs

* reviewed readme

* unsetting api key for posthog (pywry) windows

* added debug_mode to the system settings

* removing api key

* update nightly script

* fix tests

* black

* black

* Revert "black"

This reverts commit 21d76ad689.

* Revert "black"

This reverts commit 0904653fd7.

* black for henrique

* returning a list instead

* better docstrings

* removing unnecessary indicators docstring

* typo

* black

* intentional access to protected member

* changes to publish script

* filtering deleted files

* black

* adding render argument

* fixing the df_to_basemodel function: it should include the index in the columns either if flagged or if the index is named

* adding the hability to add you own data to to_chart method

* tests

* black

* disable protected access

* accept kwargs to eg. save image

* adjustment cc Igor

* add if statement for protection

* remove unused return typing

* fix technical charting views

* better error message cc @IgorWounds

* better structure and fix test

* Revert "better structure and fix test"

This reverts commit 612ea6117c.

* Revert "better error message cc @IgorWounds"

This reverts commit 3d7ee53878.

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Fix CAPM example (#6075)

* Start removing dead v3 commands for deprecation (#6070)

* Yeet tokenterminal

* Yeet crypto/onchain/hr and gwei

* yeet luna

* yeet coinglass

* yeet dps/ctb

* yeet reddit

* yeet covid

* actually not yeeting

hide yfinance future warning

* yeet options/hedge

* not yeeting.  i18n words

* something happened on accident

* still fixing

* git reset/git checkout -f 2516cfbc72 website/content

* still not sure whats going on

* Edit workflow to not look at deleted files + change bootup message

* Dont pin the workflow to my branch always lol

* Remove askobb

* [Feature]  - Comma separated string and list inputs (#6062)

* feat: balancesheet

* feat: income statement

* fix: cash flow statement

* feat: financial ratios

* feat: reported financials

* feat: revenue business line

* feat: revenue per geography

* fix: warnings

* Update balance_sheet.py

* fix: remove warning imports

* feat: balance sheet growth

* feat: calendar ipo

* feat: cashflow growth

* feat: __validators__

* fix: typing

* fix: validator msg

* feat: trailing dividend_yield

* feat: short volume

* feat: share statistics

* feat: price target + consensus

* fix: update validator

* feat: historical dividends

* feat: key executives

* fix: getattr directly

* fix: type hint validator

* fix: add default to getattr

* typo

* feat: otc

* feat: options chains

* fix: remove list validators

* feat: add validator dict to fetchers

* lint

* update standard models

* lint

* feat: rename validator

* fix: ownership

* feat: query params description

* fix: balance yfinance

* key execs, income statement

* remove change

* fix: revenue geographic

* fix: disable pylint

* fix: unusual + balance

* unit test validator

* minor fix

* explicit validators

* update test

* feat: multiple items allowed

* rebuild

* move check_single_value

* crypto historical

* order providers

* fix typing

* rename extra dict

* marking multiple items allowed

* Revert "marking multiple items allowed"

This reverts commit 7da04817e1.

* error msg

* fix: default provider runtime vs configuration

* minor fix

* docstring

* revert default changes

* get provider before executing cmd

* fix container bug

* clean filter

* add comment

* prevent bad defaults

* update docstring

* typing

* fix

* black

* fix docstring generation

* [Feature] - Mark multiple item fetchers (#6074)

* feat: price target

* bezinga company news

* equity quote

* marking multiple items allowed

* more multiple items

* black

* rebuild

* another batch

* rebuild

* another

* cpi

* move prop

* rebuild

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* rename var

* update msg

* rename func

* fix test

* mypy

* ruff

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* [Feature] Add Regional Data Endpoint from FRED (#6071)

* add regional data endpoint from FRED

* codespell

* ruff

* sort imports

* add provder='fred' to integration test

* thought I already sorted those imports..

* pylint..

* more test params....

* even more test params...

* even more test params....

* static assets

* reconfigure params

* ruff

* alias dict

* recapture cassette

* router doctstring

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] - Fix most integration tests (#6082)

* Fix most integration tests

* Lint

* Fix price_target column for Benzinga (#6083)

* Release/4.1.4 (#6081)

* static assets

* patch version increase

* core update

* bumping versions

* bump finviz version

* bump versions

* bump versions on package.extension

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Add dates to /quantitative as needed. (#6078)

* Add dates to /quantitative as needed.

Run new black

* Fix tests

* Pycharm is weird sometimes

* uncommit

* Add default window + error handling + mistake

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] Add end point: `etf.equity_exposure()` with FMP provider (#6079)

* add etf.equity_exposure from FMP

* empty data error

* recapture cassette

* pylint unused argument

* Fix router example typo

* fix test...?

* static file to fix test?

* fix test..?

* __json_schema_extra__

* black

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* [BugFix] Fix Equity Integration Tests (#6086)

* fix equity integration tests

* black

* pylint

* black

* black

* move df_apply to helpers

* unused import

* remove file that shouldn't have been added

* oops. didn't remove a couple lines

* [Enhancement] Add `analyst_search` to the Equity/Estimates Router (#6088)

* standardize price_target

* add analyst search

* black

* more black

* ruff

* linter, test parmas

* last missing param

* get rid of convert to upper

* ruff

* pylint

* [BugFix] - Fix POST docstring examples (#6095)

* Fix POST docstring examples.

* Move router examples to command decorator

* [BugFix] Fix `treasury_rates()` (#6092)

* fix treasury_rates

* drop rows with no data

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* clean up the remains from my yeeting (#6089)

* clean up the remains from my yeeting

* was not careful

* Add ETF examples (#6102)

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* [BugFix] Fix integration tests (#6100)

* fix int tests

* fix obbject int tests

* fix quant tests

* fix index tests

* use python 3.10 instead

* remove cache

* end_date < start_date

* [Enhancement]: Add TMX data provider extension. (#5913)

* dev install

* add TMX provider extension

* normalize ytm and coupopn_rate values

* black

* codespell

* ruff

* black again

* pylint

* ftw

* undo what Pylint thinks is best

* more black

* renovate insider_trading standard model

* fix tests

* typo

* tests

* empty string

* pylint unused argument

* nans for some reason

* black

* static files

* static files with openbb-tmx installed this time

* yfinance TypeError..?

* NA -> N/A

* update for version bump

* list exchange-calendars as tmx dependency

* eod chains date -> eod_date

* new line

* not that file

* more black

* one more black

* fix weird merge thing

* ruff

* index constituents params

* insider trading

* fixing things

* black

* black AGAIN

* news integration params

* the other missing param

* ruff

* historical dividends field update

* add json_schema_extra

* gainers

* symbols -> symbol

* company news symbol

* unit test param

* review items

* wrangle a rogue field

* index router example

* company news thing - symbols -> symbol

* field name map

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* [Feature] - OBB.GET (#6097)

* create OBB.GET and data slicer page

* remove quotes

* format description

* wording

* functions.json

* sidebar position

* remove if stamente

* [Docs] Remove Reference to `to_chart` in OBBject. (#6096)

* remove reference to to_chart in OBBject.

* put chart description back in OBBject properties

* [Feature] - Create openbb assets folder (#6101)

* create openbb assets folder

* fix unittest

* rebuild

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] Fixes to_df() where the date series contains multiple TZ-offsets. (#6099)

* allow mixed utc offset in a tz-aware date column

* enforce ascending from all providers for price.historical

* fix tests

* add comments and apply treatment to basemodel_from_df before JSON output

* yfinance consistency

* add unit test

* add unit test

* better test

* ruff

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Update the quantitative extension to make more sense (#6087)

* Split out a rolling submenu for the rolling functions

* Make a performance and a stats submenu.

* Test the statistics functions

* lint

* lint

* dupe test

* pylint

* ruff

* Try tests quick

* black magic signature funcs

* fix my custom tests

* Fix the existing imports/urls

* push the api update

* okay I figured out whats going on

* this should be all  of them

* Correct docstringing examples

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* [BugFix] Fix ECB Yield Curve (#6108)

* fix ecb yield curve

* standard model update

* Update unit-test workflow to check for changes to openbb_platform (#6109)

* [Bug fix] - Fix package mypy (#6110)

* fix: package mypy

* types

* black

* [Feature] - Equity router descriptions & examples (#6103)

* feat: equity/calendar

* feat: compare router

* fix descriptions

* darkpool + discovery

* fix paragraph issues

* fix descriptions

* fundamental part 1

* fundamental part2

* transcipts

* ownership

* fix

* price

* shorts

* Update darkpool_router.py

* minor fix

* shorts examples

* examples

* ruff

* black

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Add `exception_handler` decorator to static (#6112)

* add exception handler decorator

* add exception handler decorator code

* updated static

* rebuild to fix docstring indentation

---------

Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* update extensions list (#6114)

* add missing field descriptions to yfinance equity price historical (#6116)

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* remove charting extension env var (#6118)

* [BugFix] Fix Company News (#6111)

* fix company news

* update tests

* multiple symbols for polygon

* don't include pageSize in Benzinga querystring, fixed as 100

* remove extra space

* black

* unused imports

* make benzinga test cassette smaller

* missing params

* clear empty strings

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [BugFix] Allow Multiple Symbols in `equity.fundamental.metrics()` from `yfinance` (#6124)

* allow multiple symbols in key_metrics from yfinance

* field order thing

* remove unnecessary function (#6125)

* [Docs] Update OpenBB Copilot docs (#6106)

* WIP: Tweak description.

* WIP: Experimenting with react-player

* WIP: Add copilot docs.

* fixes videos width/height

* WIP: Refine videos and copy.

* WIP: Add last videos and sections.

* Add bring-your-own-copilot section.

* Tweak grammar.

* Add copilot link.

* Fix typos.

---------

Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>

* [Enhancement] Add Previous Close to DATA_DESCRIPTIONS (#6120)

* add data description for previous close

* ..just the one

* update the model tripped by this change

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* added australia as a country also (#6131)

* [Bug fix] - Fix regulator cik map standard model (#6129)

* Fix regulator cik map standard model

* fix description

* Update cik_map.py

* increase timeout to 30 (#6115)

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* clean up some v3 deps (#6105)

* clean up some v3 deps

* forgot we need to do the requirements txt files

* not sure it matters, but run ruff + black

* am addicted to bug fixes

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* [Bug fix] - explicit multiple items docstring (#6127)

* explicit multiple items docstring

* rebuiild

* rebuild again

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* [Feature] Pretty Pydantic `ValidationError` exceptions (#6130)

* fix validator in fmp IndexHistorical model

* make ValidationError pretty

* modify arg error

* linting fmp IndexHistorical

* black

* remove unused code

* tmx etf info (#6132)

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* [Feature] - Platform V4 Markdown Generator V2 (#6094)

* add recent `openapi.json`

* fix data types in models

* removed `openapi.json`

* set default date value as None in PolygonCurrencyPairs class

* add function to generate reference.json file

* reworked function to improve readability

* reword function to add seo metadata

* add functions for creating markdown sections

* add code to extract data card markdown title

* add additional functions for generating index and data models file

* extract first sentence of the description for cards

* code cleanup and documentation

* linting

* linting polygon models

* add openbb import statement in create_reference_markdown_examples function

* add POST method functions

* cleanup; reworked generate_reference_index_files function

* moved development section to 7th position in the sidebar

* fix POST function params default value

add standard flag to QueryParams and Data fields

cleanup

* add type expansion from package_builder.MethodDefinition

* sort data models cards alphabetically

make printing less verbose

* make MAX_CARDS global

display less content in cards in Commannds section

* cleanup

* Remove '_' from the cards under Commands section

* " to ' in econometrics/causality

* replace ' with " in ReferenceCard for reference dir index files

* remove extra . from the quantile function description

* shoutout to @deeleeramone for finding POST method description bug!

* set correct value for standard field

* handle BaseModel types in provider data fields

* unit tests for the platform markdown generator v2

* yeet 'Default' and 'Optional' columns in the 'Data' section

* last minute bug fix

* add info for multiple symbols

* make multiple items info same as platform static

* organize sections properly

* sort reference sub-directories alphabetically

* extra space in 'OBBject extra' description

* add type expansion for fields with multiple items

POST method cleanup

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Docs] Adds Example Notebook for the Implied Earnings Move (#6128)

* adds example notebook for the implied earnings move

* remove commented out line

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* [Feature] - Sync with new openapi.json (#6137)

* feat: create data tabs

* fix: uncomment

* fix: remove unecessary comment

* sort side bar

* comment

* uncomment

* use .dev functions.json + clean traverse

* fix futures example

* fix example

* point to .co

* Fix the v3 markdown generation workflow (#6138)

* fix workflow

* debug time

* see if the docs generate at least

* ahhhh i found it

* getting there

* put the test first

* some keys cleanup

* small change to error message

* update functions.json (#6140)

* fix intraday check in charting ta_class (#6119)

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* [BugFix] Display EconDB as source for macro (#6143)

* [BugFix] Disable Intrinio `InstitutionalOwnership` model (#6142)

* disable fetcher in init and fetcher tests

* disable provider test params from equity python and api tests

* updated static

* add more examples into findSymbols (#6139)

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* [BugFix] Fix SEC ETF Holdings Parsing Exception (#6121)

* fix etf holdings

* linting

* black

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Docs/update xl docs (#6145)

* remove print

* update obb.get docs

* example

* md lint

* fix get reference (#6147)

* [BugFix] Filter OECD data using `start_date` and `end_date` parameters (#6144)

* move constant dicts to constants.py

filter by start_date and end_date

* black

* black again again

* Merge 'main' into 'develop' (#6151)

* Release/3.2.4 (#5696)

* test

* ultima description (#5541)

* Updating showcase page (#5543)

* updating showcase page

* use sdk instead

* add new bot command autopost docs (#5530)

add new bot command autopost docs to docs.openbb.co

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Hotfix/ultima improvements (#5553)

* tweaks for monetization

* fix linting issue

* fix: Modify 'load' behavior to append data to existing sheet (#5503)

* fix: Modify 'load' behavior to append data to existing sheet

Details:
- Added logic to detect existing sheet and append data if present.
- There is no direct way to append on the sheet, so I am reading the sheet to get the last row number and then writing after that row number.

Resolves: #5472

* Fixed 'worksheet not found' error during export to an existing sheet

* Changes:
* Now if the sheet exists in the xlsx file it will ask the user if he
  wants to replace/append/create new sheet

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* added platform API integration test to Nightly build (#5571)

* Add columns to `stocks/search` (#5565)

* Add columns to `stocks/search`

isin, cusip, figi, composite_figi, shareclass_figi

* Update test_search[False].txt

* Update test_search[True].txt

* Delete tests/openbb_terminal/stocks/cassettes/test_stocks_helper/test_search.yaml

* Create test_search.yaml

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* updating ultima urls (#5587)

* fix: A value is trying to be set on a copy of a slice from a DataFrame #5582 (#5591)

* Update README.md (#5598)

* Hotfix/econometrics export filename fix (#5508)

* fix: Modify 'load' behavior to append data to existing sheet

Details:
- Added logic to detect existing sheet and append data if present.
- There is no direct way to append on the sheet, so I am reading the sheet to get the last row number and then writing after that row number.

Resolves: #5472

* Added --export option for exporting to a desired file in /econometrics menu

* Added a -f/--file option to provide the filename for export.
Made -t/--type and -f/--file mutually exclusive.
If -f is not given, -t is required and the filename will be generated based on the time and module.

* Revert `helper_funcs.py` to original state

* Made some arguments not required.

* Minor changes asked by @deeleeramone

* No sheet name

* black

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* FIX #5600 Updated the Twitter logo to X logo (#5601)

* Change Ultima paragraph (#5624)

* change ultima paragraph

* remove old paragraph

* Update CONTRIBUTING.md (#5623)

* mobile line-brake fixed (#5604)

In the mobile view (media max 400px ), the text line "Installation >>" appears broken into smaller segments.

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Docs/terminalpro and improvement (#5622)

* add first iteration of content for Terminal Pro

* Update index.md

* Update dashboards.md

* Update folders.md

* Update grouping.md

* Update report.md

* Update templates.md

* Update quick-start.md

* Update index.md

* Update index.md

* Update home.md

* Update news.md

* Update index.md

* Update index.md

* Update index.md

* Update data-provider.md

* Update chat-with-widget.md

* Update data-manipulation.md

* Update forecasting.md

* fix: images self closing tags

* fix: only pro visitors can see pro

* inter font

* improve SEO content and restructuring

* add new package for toggle on tutorials

* move folder around

* improve main page of each prod

* small updates and improvements

* fix logo above

* small improvement in headtitle for portfolio funcs

* fix typos from OpenAI's GPT-4

* fix _category_ json

* remove generic SEO words

* add script that generates SEO for documentation

* add missing file

* remove sdk warning message

* fix links for andrew

---------

Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>

* Add light blue banner, blue when clicking sidebar + fix link in docusaurus.config.js (#5627)

* typo

* pro docs tab coloring

* sidebar selection blue

* redirecting missing

* fix docs (#5646)

* Fix the redirect for intro to the correct url for docs (#5658)

* fix the redirect for intro to the correct url for docs

* fix code linting

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Improve Pro documentation further (#5641)

* small improvement

* Fix TutorialVideo for mobile view

* main page terminal pro docs

* jose whelp

* placeholder index file that needs to be removed

* fix: fixes sidebar, removes unused index.md

* update link to api keys

* update api keys link

* fix hyperlinks in terminal helper

* fix a few more links

* fix some more links

* linter

* black linter

* improve main docs page for each product

* small refactor

* header to take to main page of product

* run OpenAI script to generate SEO for pro content

---------

Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>

* Update folders.md

* Update integrate-your-own-backend.md

* Update index.md

* Update settings.md (#5665)

* fix front matter (#5667)

* #5633 adding comma every 1000 (#5652)

* #5633 adding comma every 1000

* pylint

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* Hotfix/reports model render report sanitize inputs (#5669)

* Adding more robust special character handling to create_output_path().Addresses https://github.com/OpenBB-finance/OpenBBTerminal/issues/5299

* Running black linter on code to conform to OBB style guides.

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* hotfix/fix-gov-histcont: Fix #5650 - /stocks/gov/histcont (#5676)

* fix histcont

* some cleanup

* remove sort line

* 12-31 not 12-12

* columns_keep_types

* remove space in regex (#5675)

* Pin openai (#5685)

* Clarified stocks/ta empty recom object message (#5468)

* Clarified stocks/ta empty recom object message

* black

* lint

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* improved market cap value extraction (#5169)

* improved market cap value extraction

* pylint

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* 3.2.3->3.2.4

* Changelogs

* new changelog

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Andrew <andrew.kenreich@gmail.com>
Co-authored-by: Adi Sai <adithya.sairam1@gmail.com>
Co-authored-by: Sandip Saha <sandipsmoto@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Luqman <luqazino@gmail.com>
Co-authored-by: Adwait <111136306+AdwaitSalankar@users.noreply.github.com>
Co-authored-by: Abhishek Amar <66067391+abhishek-amar@users.noreply.github.com>
Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>
Co-authored-by: professssor <62937125+professssor@users.noreply.github.com>
Co-authored-by: Farookh Zaheer Siddiqui <129654632+FarukhS52@users.noreply.github.com>
Co-authored-by: MAX SHESTOV <57421753+maxxsh@users.noreply.github.com>
Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Mohamad Fadil <fadil.parves@gmail.com>
Co-authored-by: Ramazan <guvencramazan@outlook.com>
Co-authored-by: joshuabuildsthings <joshuabuildsthings@users.noreply.github.com>
Co-authored-by: wojda <114031148+polooner@users.noreply.github.com>
Co-authored-by: Bim Phomthong <118195204+bimbolimbo@users.noreply.github.com>

* Release/4.0.1 (#5840)

* test

* ultima description (#5541)

* Updating showcase page (#5543)

* updating showcase page

* use sdk instead

* add new bot command autopost docs (#5530)

add new bot command autopost docs to docs.openbb.co

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Hotfix/ultima improvements (#5553)

* tweaks for monetization

* fix linting issue

* fix: Modify 'load' behavior to append data to existing sheet (#5503)

* fix: Modify 'load' behavior to append data to existing sheet

Details:
- Added logic to detect existing sheet and append data if present.
- There is no direct way to append on the sheet, so I am reading the sheet to get the last row number and then writing after that row number.

Resolves: #5472

* Fixed 'worksheet not found' error during export to an existing sheet

* Changes:
* Now if the sheet exists in the xlsx file it will ask the user if he
  wants to replace/append/create new sheet

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* added platform API integration test to Nightly build (#5571)

* Add columns to `stocks/search` (#5565)

* Add columns to `stocks/search`

isin, cusip, figi, composite_figi, shareclass_figi

* Update test_search[False].txt

* Update test_search[True].txt

* Delete tests/openbb_terminal/stocks/cassettes/test_stocks_helper/test_search.yaml

* Create test_search.yaml

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* updating ultima urls (#5587)

* fix: A value is trying to be set on a copy of a slice from a DataFrame #5582 (#5591)

* Update README.md (#5598)

* Hotfix/econometrics export filename fix (#5508)

* fix: Modify 'load' behavior to append data to existing sheet

Details:
- Added logic to detect existing sheet and append data if present.
- There is no direct way to append on the sheet, so I am reading the sheet to get the last row number and then writing after that row number.

Resolves: #5472

* Added --export option for exporting to a desired file in /econometrics menu

* Added a -f/--file option to provide the filename for export.
Made -t/--type and -f/--file mutually exclusive.
If -f is not given, -t is required and the filename will be generated based on the time and module.

* Revert `helper_funcs.py` to original state

* Made some arguments not required.

* Minor changes asked by @deeleeramone

* No sheet name

* black

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* FIX #5600 Updated the Twitter logo to X logo (#5601)

* Change Ultima paragraph (#5624)

* change ultima paragraph

* remove old paragraph

* Update CONTRIBUTING.md (#5623)

* mobile line-brake fixed (#5604)

In the mobile view (media max 400px ), the text line "Installation >>" appears broken into smaller segments.

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Docs/terminalpro and improvement (#5622)

* add first iteration of content for Terminal Pro

* Update index.md

* Update dashboards.md

* Update folders.md

* Update grouping.md

* Update report.md

* Update templates.md

* Update quick-start.md

* Update index.md

* Update index.md

* Update home.md

* Update news.md

* Update index.md

* Update index.md

* Update index.md

* Update data-provider.md

* Update chat-with-widget.md

* Update data-manipulation.md

* Update forecasting.md

* fix: images self closing tags

* fix: only pro visitors can see pro

* inter font

* improve SEO content and restructuring

* add new package for toggle on tutorials

* move folder around

* improve main page of each prod

* small updates and improvements

* fix logo above

* small improvement in headtitle for portfolio funcs

* fix typos from OpenAI's GPT-4

* fix _category_ json

* remove generic SEO words

* add script that generates SEO for documentation

* add missing file

* remove sdk warning message

* fix links for andrew

---------

Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>

* Add light blue banner, blue when clicking sidebar + fix link in docusaurus.config.js (#5627)

* typo

* pro docs tab coloring

* sidebar selection blue

* redirecting missing

* fix docs (#5646)

* Fix the redirect for intro to the correct url for docs (#5658)

* fix the redirect for intro to the correct url for docs

* fix code linting

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Improve Pro documentation further (#5641)

* small improvement

* Fix TutorialVideo for mobile view

* main page terminal pro docs

* jose whelp

* placeholder index file that needs to be removed

* fix: fixes sidebar, removes unused index.md

* update link to api keys

* update api keys link

* fix hyperlinks in terminal helper

* fix a few more links

* fix some more links

* linter

* black linter

* improve main docs page for each product

* small refactor

* header to take to main page of product

* run OpenAI script to generate SEO for pro content

---------

Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>

* Update folders.md

* Update integrate-your-own-backend.md

* Update index.md

* Update settings.md (#5665)

* fix front matter (#5667)

* #5633 adding comma every 1000 (#5652)

* #5633 adding comma every 1000

* pylint

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* Hotfix/reports model render report sanitize inputs (#5669)

* Adding more robust special character handling to create_output_path().Addresses https://github.com/OpenBB-finance/OpenBBTerminal/issues/5299

* Running black linter on code to conform to OBB style guides.

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* hotfix/fix-gov-histcont: Fix #5650 - /stocks/gov/histcont (#5676)

* fix histcont

* some cleanup

* remove sort line

* 12-31 not 12-12

* columns_keep_types

* remove space in regex (#5675)

* Pin openai (#5685)

* Clarified stocks/ta empty recom object message (#5468)

* Clarified stocks/ta empty recom object message

* black

* lint

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* improved market cap value extraction (#5169)

* improved market cap value extraction

* pylint

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Release/3.2.4 (#5704)

* 3.2.3->3.2.4

* Changelogs

* new changelog

* OpenBB Platform v4 (#5225)

* change location

* check something

* add cache

* fixed auth w/@IgorWounds and @montezdesousa

* fixes for econometrics

* include Darren auth suggestions

* some routers included if dev + chg hub backend var

* coverage router should not be here

* update docker readme

* forgot to check auth here

* Change extension imports

* update docstrings and readme

* headers function wasnt being called, just referenced

* relative import

* econometrics add headers everywhere

* Fix QA

* Fix TA

* Fix test

* Fix python interface QA and TA

* Lint

* Cleanup

* Change path

* make the changes and some etf models

* tests for command_runner

* ruff

* solving naming issues on pytest

* Update some docstrings, fix typos and lint

* solving econometrics instantiating wrong openbb

* chaging the way pytest runs

* messing around with the imports

* ruff

* Change template to add platform considerations (#5533)

* change template to add platform considerations

* @the-praxs suggestions

* solving __pycache__ issue

* removing typo

* ci yml

* making linters happy (i guess)

* Improve Omega Ratio and docstrings

* chaging pro validation hash

* ruff

* Fix OLS Summary and lint

* Add OECD Provider for economy extension (#5534)

* Add record to docs

* update a couple of cboe models

* Fix econometrics bugs. Make tests pass. Lint.

* black

* ruff

* pydantic v2: remove deprecated Extra

* pydantic v2: self.dict -> self.model_dump

* revert pydantic

* revert the revert

* try 1: pin urllib3 < 2

* Revert "try 1: pin urllib3 < 2"

This reverts commit 3736c311cb.

* Revert dep

* Fix

* re-lock

* Skip all tests when doing cspell in pre-commit

* Resolve linter errors

* Add OECD to install script

* Add OECD

* Resolve more linter errors

* Exclude static files of v4 python from ruff

* More linting

* Another attempt to resolve even more linting errors

* Change how Literal is used

* ticker. symbol. consistent. @piiq

* change ticker field in Stock Search to is_symbol

* Update Cboe model for stocks_search -> is_symbol parameter.

* Fix issue

* Lint

* Small QA to rename SDK to Platform (#5544)

* SDK -> Platform in docs

* renamed in core

* renamed in openbb `init`

* `sdk` variable renamed as `platform`

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* `validator` -> `field_validator` in standard models

* Standard convention for multi-ticker symbols  (#5549)

* standardize symbol input param in forex and crypto historical data

* static

* linting

* Fix stock_search integration test params

* Update dependencies and linting on the CI (#5550)

* Remove ruff and black from platform dependencies

* Update codespell configuration

* Lint python code

* Add toml and tomli to root dev dependencies

* Update requirements.txt files

* Fix buggy docstring in python package methods (#5548)

* fix buggy docstring for methods

* remove space

* fix typing

* avoid name collision with validator

* Revert "avoid name collision with validator"

This reverts commit 8a30a44555.

* typing

* rebuild

* python 3.8, ParamSpec not available

* lint

* ruff noqa

* Merge with develop (#5551)

* Charting extension integration tests (#5547)

* initial proposal for integration tests on the charting extension

* int tests for charting generation

* python generator

* ruff

* boilerplate templates

* python tests

* tests for the api

* using body instead so data don't get redefined

* unit tests for coverage

* ruff

* docstrings

* unnecessary access to keys()

* Add trading economics economic calendar

* dev install

* Data Models disposition on the website (#5538)

* pydantic v4

* flat structure

* md files

* Revert "md files"

This reverts commit bd2170e2ad.

* improve command descriptions

* nice title and description

* human readble data model names on left panel

* implementation details section

* content markdown files

* markdown versioned files

* showcase page

* updates

---------

Co-authored-by: tehcoderer <me@tehcoderer.com>

* everything except tests

* fix `datetime.date` import in fetcher test

* modified url to remove `api_key` placeholder

* record test

* api key is called c here

* fixed fetcher test for api key

* recorded tests

* refactor

* life is ruff

* codespell

* Make Intrinio options chains great again (#5540)

* add multithreading

* cleanup

* linting

* use dateutil.parser

* removed `TICKER_EXCEPTIONS`

* modified `date` and `type` fields in standard model

* set alias for `date` and `type` to match standards

* static

* static

* Hotfix/fix post tests (#5555)

* Fix econometrics

* Fix ta

* Fix qa. Remove intrinio to not be rate limited

* Lint

* Fix typo

* obbject tests (#5558)

* obbject tests

* ruff

* fix dev_install

* fix unittests

* black on nox

* black on nox

* nox

* fix fetcher test so it can compare dates

* fix intrinio tests

* ruff

* change to model_dum() instead (#5561)

* Passing tests (#5560)

* Passing tests

* fix options

* Fix most tests and bugs

* fix forex missing params

* make symbol upper in polygon fx pairs

* fix fixedincome integration tests (#5564)

* Fixing charting api integration tests (#5562)

* fixing charting api integration tests

* black

* fix stocks less five API fails

* fix the 5 failing stocks integration tests

* recapture fmp stock news unit test

* fix fmp forex api test

* couple more forex integration fixes

* changing the request to always use the legacy session. @jmaslek this is was broken for 3.8 and 3.9; with this change apparently all versions work - lemme know if you see any issue with this change

* ruff

* fix ta

* add missing param

* fix testers for the api

* fix globalnews to use images instead of image

* remove unncessary pandas import

* econcal int tests

* update gitignore

* fix get_querystring to support lists

* add missing parameters

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>

* Stock Historical standardization and UX improvements (#5457)

* polygon add interval standard param

* attempt at standardization

* Alpha Vantage params standardization

* rename `get_intervals` and fixed bug for extracting wrong value

* missed changing it in `extract_data`

* linting

* polygon cleanup

* modified `start_date` & `end_date` type to `datetime` from `dateType`

* standardized intrinio stock historical params

* fixed yfinance

* linting

* revert start and end date to dateType from datetime

* fix alpha vantage

* changed param

* fix intrinio infinite loop

* `validator` is `field_validator`

* revert time params and fixed intervals

* `outputsize` alias set in `Field`

* set interval description using in-build dict

* yfinance stock param standardization

* static

* intrinio cleanup

* Polygon add api params as `PrivateAttr`

* modified yfinance params to `PrivateAttr`

* intrinio remove overriding date params

* fixed description for AV adjusted param

* fix alpha vantage intraday not working

* re-record and pass unit tests

* pass integration tests

* Update test

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* Fix econcal endpoint (#5567)

* fix econcal for python interface

* fix econcal tests

* fix non supported json output

* Fixes FMP index endpoint to get 1 day interval data (#5535)

* fix FMP endpoint for getting index data for `1day` interval

* fix `open` type to prevent validation error

* modifed `validator` to `field_validator`

* type of fields is `StrictFloat`

* re-record and pass test

* linting

* `validator` -> `field_validator`, a3'd everything

* Fix regression in tests. Lint

* updating the integration test actions

* Fix typo

* Fix typo

* Fix import

* Publish versions `a3` to Pypi (#5568)

* bump versions and script to publish

* small improvements

* uncomment exception

* openbb

* publishing guidelines

* improve docs

* add providers to economy api

* make symbol required in stock_quote

* Fix keys

* Fix oecd

* add comma

* revert

* Add quotes

* Sleep

* Improve tests

* Improve

* Feature/query transform (#5566)

* inject query into transform_data

* swap query and data

* update missing fetchers

* last test failing

* sig test fetcher

* black

* av add `query` to `transform_data` call

* transform_data: missing refactoring

* fix uvicorn command in the doc

* Automated examples (#5576)

* Automated examples

* Fix bug

* Lint

* static assets built on 3.8

* Revert "static assets built on 3.8"

This reverts commit fb82ff7bd9.

* remove assets

* static assets on 3.8

* static assets on 3.8

* Wrap in quotes

* Improve wrap

* minor fix

* rebuild 3.8

* Fix bug

* Fix edge case

* Lint

* rebuild

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* use SecretStr for sensitive fields (#5578)

* use SecretStr for sensitive fields

* fix

* docstring

* Add tests for provider (#5583)

* Add tests for provider

* Test standard models

* Lint

* Fix standard model issues.

Co-authored-by: @the-praxs

* linting

* `symbol` -> `is_symbol` in `economy.index_search`

* static

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* feature/v4-biztoc-provider: Adds Biztoc provider and GlobalNews (#5546)

* add biztoc provider

* duplicate field

* black

* request headers

* recapture test

* move stuff around

* add integration tests

* duplicate OECD entry in dev_install

* duplicate entry for oecd in pyproject.toml

* Add Biztoc to workflow

* update readme: credentials

* Add Preferences to README

* improve tests

* fix biztoc

* obbject pylint

* Bug fix: serializ SecretStr before dumping to json

* export credentials as json (#5592)

* Check for wrong, changed, or outdated params. (#5597)

* Improving `economy.cpi` for the FRED data (#5602)

* remove multiindex using coding magic

* Update field descriptions

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Improve error message.

* Improve

* Test tests (#5607)

* Test tests

* Add test

* 4.0.0a4 PyPI publishing (#5608)

* changes to publish

* fix mini conflict

* typo

* cleaned `descriptions.py`

* fix description in `cpi.py`

* Add ETF extension to v4 (#5609)

* Add etf extension with search data model

* Add FMP ETF search

* Add fetcher tests

* Add ETF historical price from yfinance (#5610)

* Add etf extension with search data model

* Add FMP ETF search

* Add fetcher tests

* Add etf historical price standard model and route

* Add etf historical data from yfinance

* Add unit tests

* Fix linter error. Organize imports

* Update test data

* Revert common descriptions erased by CPI PR

* remove `symbol` from data for `stocks.ca.peers`

* Test docstrings (#5613)

* Test docstrings

* fix minor bugs w/@IgorWounds

* static only w/@IgorWounds

* fix `importance` param in `economy.econcal`

* right static only w/@IgorWounds

* Fix econcal and add tests to noxfile

* fix econcal static

* re-record

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Clean up the newly created ETF extension (#5611)

* Add ETF to dev_install bundle

* Add integration tests for openbb-etf

* Patch integration test generator scripts to produce lintable code

* Add py.typed markers for extensions

* user preferences descriptions (#5593)

* user preferences descriptions

* typo

* Feature/v4-sec-provider: Add SEC provider to V4 (#5556)

* coerce string type, add default=None

* add sec provider and FTD function

* update description

* ruff

* data model definitions in standard model already

* duplicate oecd entries in dev_install and pyproject

* actually commit

* integration test

* add stock search to SEC

* add company filings for SEC provider

* integration tests

* remove duplicated line

* ruff

* **kwargs in Transform Data

* some helper functions for later

* black

* add helper for mutual fund and etf lookup

* add some SEC lookup helpers to a new regulators extension

* black

* ruff

* fix test param

* take institution filter out of sec stock_search

* ruff

* remove URL that codespell doesn't like

* add cik-to-ticker map

* add litigations rss feed

* make openbb-regulators a dependency of openbb-sec

* add SIC search

* improve ftd and remove BeautifulSoup from SEC dependencies

* fix parsing the zip files pre-2009

* ruff

* update descriptions

* recapture ftd test

* tests params

* typo

* change limit param to 1 in test_sec_stock_ftd

* Add didiers to_records method (#5588)

* Add didiers to_records

* unused import

* Bring back polars + do what igor suggested

* Make the output consistent with what it was.

* Black ?

* spelling.  oops.  almost like we have a pre commit or something

* Feature/v4-price-performance: % returns for various horizons, comparable across multiple tickers. (#5618)

* coerce string type, add default=None

* add price_performance endpoint from FMP

* black

* extra period in docstring

* ruff

* forgot one file

* Feature/ipo-calendar: V4 IPO Calendar (#5605)

* coerce string type, add default=None

* add intrinio ipo calendar

* fix typo

* sort __init__

* fix description

* make requested changes

* update test params

* black

* test updates

* etf integration

* Add disc router and WSJ provider

* OBBject extensions (#5612)

* changes to core

* bug?

* docstring

* doc

* Update credentials.py

* fix model_dump

* create extensions

* fix docstring

* doc

* revert change query_exc

* doc

* fix container test

* redirect obbject test patch

* doc

* rename method

* move decorator to extension file

* rename method extend_obbject

* changes in creds model

* avoid credential racing

* doc

* rename prop

* doc

* doc

* remove comment

* comment some code

* free extension names

* docstring

* doc

* docs

* Hotfix/fix-v4-economic-calendar: Add FMP to Econ Calendar and fix some TE bugs (#5580)

* coerce string type, add default=None

* adds fmp to economic calendar and fixes some things with TE

* ruff

* etf integration test

* missing provider for test

* rename te cassette to economic_calendar

* fix test

* rename tests

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Add stocks disc endpoints (#5630)

* Add stocks disc endpoints

* Explicit

* Lint

* Add tests

* Bump timeout

* Feature/disc router (#5616)

* Add disc router and WSJ provider

* Lint

* Lint

* Move endpoints under ETF router

* Update etf_performance.py

* Add ETF info and sector exposure from FMP (#5620)

* Add ETF info from FMP

* Add integration tests

* Add unit tests

* Add etf sectors endpoint

* Add etf sectors fmp endpoint

* Omit nested sector list from fmp info data

* Update tests and test data

* Update test data

* Specify provider explicitly in integration tests

* Update tests

* Skip yfinance etf historical test

* Fix linter errors after the merge

* Fix more linter errors

* Fix failing and add missing tests

* Fix integration tests

* Fix calendar ipo tests

* Add ETF holdings and holding report dates from FMP (#5629)

* Add ETF info from FMP

* Add integration tests

* Add unit tests

* Add etf sectors endpoint

* Add etf sectors fmp endpoint

* Omit nested sector list from fmp info data

* Update tests and test data

* Update test data

* Specify provider explicitly in integration tests

* Update tests

* Skip yfinance etf historical test

* Add etf holdings and holdings date routes and standard models

* Add fmp holdings and holding dates fetchers

* Update holdings fetcher to properly handle date object inputs

* Add unit tests

* Add integration tests

* Black after merge

* Fix test input data format

* Disc router for equities. (#5640)

* Disc router for equities.

* add tests

* Bug fixes in financial statements (#5537)

* `validator` to `field_validator` in standard models

* Polygon add `ttm` to `period` param

* Revert "Polygon add `ttm` to `period` param"

* Polygon add `ttm` to param `period`

* `EarningsCallTranscript` standard model removed `quarter` param

* use `tag` instead of `name` for correct field name

* fix url for intrinio income statement

* modified intrinio cash flow `transform_data` function

* set optional data fields and validate dates

* fix insider trading not fetching recent data

* linting

* refactoring

* refactoring

* set param alias in fmp stock insider provider model

* get all quarter data for intrinio

* add `limit` param to `stocks.fa.ins`

* refactoring and linting

* removed additional comment

* linting

* re-recorded and fix tests

* cash flow standard model field type to `StrictFloat`

* Intrinio cash flow model fixed to get all quarters' data

* linting

* refactoring

* financial statement standard model field type to `StrictFloat`

* Intrinio financial statement model get all quarter data

* set `period` type to `Optional[str]`

* add `period` param to balance sheet provider models

* add `period` param to cash flow provider models

* use alias in model dump

* remove `ttm` from `period` param

* revamped intrinio financials

* fixed incorrect description for `limit` param

* modified tests for `balance`, `cash` and `income`

* re-recorded tests

* skipped intrinio financial statement fetcher tests

* Add ETF county weighting command with FMP data (#5645)

* Remove code that wrongfully infers the filing date as end of quarter

* Add etf countries command with fmp data

* Add countries command to etf router

* Update sector and country weighting fmp models

* Fix docstring typo

* Add unit tests and update mock data

* Add integration tests for etf.countries

* Bring back missing integration tests

* Fix economic calendar integration tests

* fix missing data in intrinio financials

* removed inutile params from Polygon financial statement models

* Intrinio IPO overhaul

* tests fixed and re-recorded

* Add ETF holdings performance with FMP data (#5659)

* Add etf holdings performance endpoint

* Add tests

* Fix failing polygon fetcher tests

* Split long holdings lists into chunks and update test data

* Change chunk length to 500 tickers

* Add stock search from FMP (#5642)

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Feature/v4 pypi (#5579)

* improvements to the publish script

* move scripts

* initial script that builds the package

* adjustments on the docs

* test pypi workflow

* add hour and minute to version

* black

* ruff

* duplicate file

* moving files instead

* build throught the script instead

* update nox file

* integration tests workflow

* change workflow

* fix dev_install

* fix in sed cmd

* testing the platform only

* commenting out the publishing

* fix run commands

* improvements to the publish script and doc

* fix publish script

* updating the script and the workflow

* feature/v4-pypi nightly (#5589)

* init

* Update nightly.py

* cleanup

* init file

* not referencing dev_utils

* updating obb platform key

* revert

* Increase warmup time

* project name on pyproject and adding the --pre flag on readme

* moving to the build folder

* fix noxfile

* reflecting the moves on the import statements

* fix pyproject path

* ruff

* fix paths

* fix super typo

* update workflows

* moving the pypi terminal files into its folder

---------

Co-authored-by: teh_coderer <me@tehcoderer.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* economy.calendar nasdaq (#5628)

* add quandl econcal

* unittest

* add integration tests

* replace &nbsp

* fix bug TE importance

* small fix

* remove stuff from quandl

* add nasdaq provider

* add nasdaq to install files

* Update test_economy_api.py

* test

* quandl leftovers

* remove quandl econcal test

* lint

* replace &nbsp

* ruff

* lint

* update nasdaq gmt

* validate countries

* here

* no need for this

* fix calendar tests

* static

* static

* remove error

* static

* readme + website

* readme

* remove changes TE

* typo

* fix unittest

* fix tests

* Update __init__.py

* Update economic_calendar.py

* revamped model and added helpers for date range

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Feature/v4-dividend-calendar: Adds Nasdaq dividend calendar and moves the command out of fundamental_analysis (#5672)

* add nasdaq dividend calendar and move out of fundamental_analysis

* black

* ruff

* add random-user-agent to pyproject.toml

* add nasdaq ipo calendar

* add nasdaq flag for SPO to IPO calendar

* add spo test param

* black

* dividend test

* fix transform_query - nasdaq_calendar_ipo

* bring back different headers for ipo calendar

* remove references to test

* Add disc endpoints (#5660)

* Add Quandl top retail

* Add tests

* Add Seeking Alpha

* Add Filings

* Change Provider

* Fix test

* Add dev_install

* static

* static correct

* y u forget me

* updated deps

* static

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* using dev install (#5673)

* Expanding the `fixedincome` menu (#5655)

* models and fetcher

* making the fetcher visible

* std model

* adding the command

* improvements to the models

* typo

* adding the command

* imrpovements to dwpcr

* standard model for ecb int rates

* adding the fetcher for fred

* standard model of ice bofa

* add ice_bofa command

* ruff

* ice bofa fetcher

* moody std model

* moody

* std model

* add cp command

* unused import

* cp

* spot

* ruff

* hqm

* tmc

* ffrmc

* tbffr

* ruff

* typing fixes

* buggy dates

* fred fetcher tests

* ruff

* adding ecb as a provider

* adding new provider related stuff

* adding the new command eu_ycvr

* models

* test for ecb fetcher

* Feature/v4 fixedincome int tests (#5661)

* integration tests

* happy ruff

* mini adjustment

* ruff

* typo

* verbose type

* fix tests

* docstrings everywhere and an happy pydocstyle

* fix tests

* Feature/user-cache-dir: Add User Preference for the Cache Directory (#5621)

* add user preference for storing cached data

* ruff

* fix tests?

* fix test key error..?

* etf integration test fix

* add cache dir to SEC

* black

* improve docs structure for V4 (#5586)

* improve docs structure for V4

* moving over changes

* Add Docs to the navbar

* add some, re-order sidebar

* fix sentence

* add docs install to source instructions

* insert a couple of links, fix a typo

* big terminal docs usage revamps

* improve toggle component for tutorials

* get rid of TOCs for all terminal docs for consistency and simplicity

* improve platform docs

* improve bot docs content structure

* remove index from bot docs

* improve ON THIS PAGE to only display TOC if at least 1 element

* make edit this page text and icon on same line

* add author and date to docs

* add basic syntax page to usage

* sidebar order for usage section

* move api start command to the top with the import python import

* some install page additions

* small improvement

* refactor routines section

* Update introduction-to-routines.md

* Update pipeline-of-commands.md

* Update introduction-to-routines.md

* Update routines-for-power-users.md

* Update community-routines.md

* update keyboard hotkey macros

* askobb page

* Update askobb-feature.md

* Update index.md

* Update index.md

* Update index.md

* Update index.md

* fix images

* remove HeadTitle from all files where it exist

* get rid of extra spacing in markdown

* small update

* small edit

* need index at the start because of main path when homepage of docs

* use openAI to create SEO for front matter on my behalf

* remove table of content from sdk page

* finish improving metadata SEO for SDK docs

* update duplicate header

* update HeadTitle to be more specific on product

* remove index from bot telegram reference

* delete duplicated title for platform reference

* another update

* Hotfix/econometrics export filename fix (#5508)

* fix: Modify 'load' behavior to append data to existing sheet

Details:
- Added logic to detect existing sheet and append data if present.
- There is no direct way to append on the sheet, so I am reading the sheet to get the last row number and then writing after that row number.

Resolves: #5472

* Added --export option for exporting to a desired file in /econometrics menu

* Added a -f/--file option to provide the filename for export.
Made -t/--type and -f/--file mutually exclusive.
If -f is not given, -t is required and the filename will be generated based on the time and module.

* Revert `helper_funcs.py` to original state

* Made some arguments not required.

* Minor changes asked by @deeleeramone

* No sheet name

* black

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* FIX #5600 Updated the Twitter logo to X logo (#5601)

* Change Ultima paragraph (#5624)

* change ultima paragraph

* remove old paragraph

* Update CONTRIBUTING.md (#5623)

* mobile line-brake fixed (#5604)

In the mobile view (media max 400px ), the text line "Installation >>" appears broken into smaller segments.

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* start PR so screenshots can get attached via GitHub website

* Update qa.md

* Docs/terminalpro and improvement (#5622)

* add first iteration of content for Terminal Pro

* Update index.md

* Update dashboards.md

* Update folders.md

* Update grouping.md

* Update report.md

* Update templates.md

* Update quick-start.md

* Update index.md

* Update index.md

* Update home.md

* Update news.md

* Update index.md

* Update index.md

* Update index.md

* Update data-provider.md

* Update chat-with-widget.md

* Update data-manipulation.md

* Update forecasting.md

* fix: images self closing tags

* fix: only pro visitors can see pro

* inter font

* improve SEO content and restructuring

* add new package for toggle on tutorials

* move folder around

* improve main page of each prod

* small updates and improvements

* fix logo above

* small improvement in headtitle for portfolio funcs

* fix typos from OpenAI's GPT-4

* fix _category_ json

* remove generic SEO words

* add script that generates SEO for documentation

* add missing file

* remove sdk warning message

* fix links for andrew

---------

Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>

* Add light blue banner, blue when clicking sidebar + fix link in docusaurus.config.js (#5627)

* typo

* pro docs tab coloring

* sidebar selection blue

* redirecting missing

* Update crypto introduction.md

* titles

* another title

* Update crypto dd.md

* Update defi.md

* Update disc.md

* Update nft.md

* Update onchain.md

* Update ov.md

* Update introduction.md

updates URL

* Update introduction.md

add HeadTitle

* Update ba.md

* Update comparison.md

* Update dark-pool-shorts.md

* Update disc.md

* fix docs (#5646)

* Update fa.md

* Update gov.md

* Update ins.md

* Update tradinghours.md

* Delete website/content/terminal/menus/stocks/options/screener.md

* Update Options introduction.md

* Update screener head title

* Update screener TOC

* Update econometrics.md

* Update economy head title

* Update etf.md

* codespell

* Fix the redirect for intro to the correct url for docs (#5658)

* fix the redirect for intro to the correct url for docs

* fix code linting

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Contributing guidelines (#5615)

* intro section

* what's expected sections

* improve sections

* improve sections and move how to add a data point

* moving qa under contributor

* better intro

* moving the sharing section

* better expectations

* small intro to the contributor guidelines

* moving things under the contributor section

* there is no pre commits on the platform yet

* dependency management

* best practices for extensions

* typo

* collapse options subfolder

* codespell

* Improve Pro documentation further (#5641)

* small improvement

* Fix TutorialVideo for mobile view

* main page terminal pro docs

* jose whelp

* placeholder index file that needs to be removed

* fix: fixes sidebar, removes unused index.md

* update link to api keys

* update api keys link

* fix hyperlinks in terminal helper

* fix a few more links

* fix some more links

* linter

* black linter

* improve main docs page for each product

* small refactor

* header to take to main page of product

* run OpenAI script to generate SEO for pro content

---------

Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>

* Update folders.md

* Update integrate-your-own-backend.md

* Update index.md

* Update fixedincome.md

* url

* Update forecast.md

* Update settings.md (#5665)

* Update forex.md

* fix front matter (#5667)

* Fix conflict markers that got into markdown during previous merges

* Fix front matter list formatting

* Fix typo

* small thing

* fix hyperlinks

* Update .codespell.ignore

* output_type options

* Architectural considerations

* Update sidebar to hide pro section by default

* Minor copy changes

* TET pattern

* import statements

* clarification on preferences

* Minor copy changes

* Lint markdown and python

* fix: open index on collapsible root elements

* remove old content

* options title

* Fix linting errors

* Refactor sidebar label to url mapping condition

---------

Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Sandip Saha <sandipsmoto@gmail.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: professssor <62937125+professssor@users.noreply.github.com>
Co-authored-by: Farookh Zaheer Siddiqui <129654632+FarukhS52@users.noreply.github.com>
Co-authored-by: MAX SHESTOV <57421753+maxxsh@users.noreply.github.com>
Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Mohamad Fadil <fadil.parves@gmail.com>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Add multi-ticker support to etf_countries (#5670)

* add multi-ticker support to etf_countries

* remove commented-out lines

* normalized percent points

* percent -> percentage

* Fix Nasdaq Econ Calendar Fetcher (#5668)

* fix nasdaq econ cal request

* more html tags

* add random-user-agent to pyproject

* another edge case - 24H -> 00:00

* Feature/v4-unusual-options: Adds unusual options endpoint. (#5677)

* add unusual options fro Intrinio

* test params

* move sorting

* Change type

* update FMP fetcher to use the V4 endpoint (#5681)

* Fix price performance not parsed correctly when there is only 1 ticker (#5678)

* Add crypto search standard model and FMP data for it (#5679)

* Apply renaming to news data (#5684)

* inegration complete

* reformatted for black

* fix ruff issues

* revised integration

* changes to versions in pyproject.toml and updated lock files

* Rename models

* Update tests

* Fix typos here and there

* Update routers

* Apply rename to ultima

* Properly mock api key

* Mock ultima api key

---------

Co-authored-by: Adi Sai <adithya.sairam1@gmail.com>

* Feature/v4-market-snapshots: Adds full market snapshots (#5473)

* add stocks market snapshots from fmp

* add fmp fetcher test

* unused imports

* add polygon to market_snapshots

* update fmp market_snapshots

* black

* pydantic2 things

* updates

* change validate_obj to model_validate

* polygon cleanup

* tests

* descriptions

* black

* data description

* minor adjustments, using less local variables, improving readability and removing pandas

* better structure

* minor adjustments

* lint

* fix int tests

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Add NBBO Quotes, with historical, to `obb.stocks.quote()` (#5617)

* coerce string type, add default=None

* add polygon stock quote

* limit cleanup

* add greater/less than to params

* limit param

* black

* tests

* fix tests

* test_etf

* rename model and function to NBBO

* add standard model for nbbo

* improve standardization

* not redifining builtin max

* fix input params

* integration test params

* removing unused import

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* revamped code

* re-recorded nbbo test

* added symbol validator

* added alias in field

* Stock news -> Company news

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* hotfix/update-quandl-to-nasdaq: Merge Quandl + Nasdaq (#5682)

* merge quandl nd nasdaq

* dev_install.py

* test param name

* economy python integration test

* Add more input types to data processing commands (#5453)

* adding more supported types to data processing commands

* ruff

* removing unused import

* docstrings

* data processing types covertion

* data description

* test filters

* ruff

* fix package builder tests

* ruff

* removing print

* Add stocks/dps module (#5638)

* short volume

* pyproject missed

* add sec short interest with days to cover

* Add FINRA OTC endopoint

* some tests

* rest of unit tests

* Integratin test scripts

* dev install

* linters caught by pre commit

* lines too long

* Last test files

* maybe it needs a lock file?

* thanks pratty

* Maybe ill actually listen to pratty next time

* sec test date freeze

* I cant hit buttons

* re record tests

* tests 3

* Stockgrid test

* regen int test

* more testing

* lint

* moved these fns

* dis killing me

* one more test

* last one

* Update integration tests

* Fix tests picking up wrong dps route

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Rename Forex to Currency (#5687)

* restore fields as optional (#5686)

* Rename Crypto (#5689)

* added `crypto.price.historical`

* static

* fix tests

* fix incorrect label

* Move model from sec to finra (#5692)

* Update toml

* fixed deps

* add openbb-etf lock file

* cleanup

* cleanup (#5694)

* Rename Fixed Income (#5695)

* Rename Fixed Income

* Add routers

* Rename stocks to equity (#5688)

* Rename stocks folder

* Rename openbb_stocks package

* Rename routers

* Reroute equity menu

* Update routes based on feedback

* StockNews -> EquityNews

* StockInfo -> EquityProfile

* Rename Equity calendars

* Equity peers rename

* Rename discovery

* Rename fundamental

* Rename price command models

* Rename shorts

* Update charting router

* Fix seeking alpha upcoming release date

* Fix linting errors

* Update tests

* Fix tests

* Skip econometrics api tests because it's python only

* Update integration tests and test data

* Historical analyst estimates

* Skip cboe options chain test. Unable to record vcr cassette

* re-record cboe cassettes

* Document CBOE unit test creation caveats

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* Feature/files-for-igor: Adding a few standard models for Igor (#5699)

* add some standard models for Igor

* codespell

* update descriptions

* Add devtools metapackage (#5703)

* Add openbb-devtools metapackage

* adding devtools to the dev_install script

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>

* Rename econometrics commands to be more verbose (#5702)

* Expose command to model map (#5706)

* Rename ta qa (#5708)

* Rename ta folder

* Rename ta to technical

* Rename qa folder

* Rename qa to quantitative

* Rename the actual routes in quantitative

* Equity cleanup (#5707)

* cleanup

* static

* renaming to equity cc @the-praxs

* Proof reading

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Don't enforce imperative for first word of the docstring

* Remove ultima from default extensions

* Rename `economy` and create `index` extension (#5705)

* renamed economy and re-recorded tests

* static

* market indices

* european indices

* fred indices

* index constituents

* tests

* correct package name

* oops

* static

* linting

* Move COT commands to regulators

* Rename economy.list to available_indices

* Rename GDP models and clean up

* Fix charting tests

* Drop _indices from available_indices command

* Fix obbject to_dict test

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Rename etf/disc (#5715)

* Docs/v4-working-docs: PR to Work on V4 Docs (#5693)

* Revert changes related to pydantic2 made into v3 code

* Revert fred model changes in v3

* Revert pydantic2 databento model changes in v2

* Update some names and v3 lock file

* Clean up openbb/package before the beta release

* Platform publishing procedure (#5701)

* update instructions

* introducing the version script

* publish message

* fix platform path

* reflecting version changes

* getting version from installed packages instead

* removing toml from dependencies

* Changes to get `a7` out (#5697)

* changes to get a7 out

* etf menu on the pyproject.toml

* poetry lock

* bump `openbb-etf` to `a5` to match version

* updated deps

* removing ultima

* removing references to a4 cc @the-praxs

* Bump index menu version

* poetry lock

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Update publishing instructions

* Bump core dependencies in extensions and providers

* Add changes to the publish script following the beta release

* Update the versions in static extension map and openbb metapackage

---------

Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: tehcoderer <me@tehcoderer.com>
Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>
Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>
Co-authored-by: Sandip Saha <sandipsmoto@gmail.com>
Co-authored-by: professssor <62937125+professssor@users.noreply.github.com>
Co-authored-by: Farookh Zaheer Siddiqui <129654632+FarukhS52@users.noreply.github.com>
Co-authored-by: MAX SHESTOV <57421753+maxxsh@users.noreply.github.com>
Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Mohamad Fadil <fadil.parves@gmail.com>
Co-authored-by: Adi Sai <adithya.sairam1@gmail.com>

* Hotfix/pypi openbb terminal (#5719)

* renaming the project

* documentation changes

* Finishing touches to the publishing procedure (#5728)

* finishing touches to the publishing procedure

* grammar improvements

* Run tests on develop (#5725)

* Run tests on develop

* Add a check that files changes in platform

* Seprate Intrinio financials data tags into a different command (#5723)

* add `ebit` field in income statement standard model

* only add required data tags

* remove calculations data

* use `name` key instead of `tag` key

* set `period` as required in data

* remove `period` and `cik` from provider models

* add `equity/fundamental/search_financial_attributes` with tests

* clean descriptions file

* add `equity/fundamental/financial_attributes` with tests

* set `value` as `Optional`

* Fix threadpool test error (#5734)

* Remove ultima from deps (#5736)

* remove ultima from deps

* skip ultima test

* black

* change skip location

* try except statement to avoid error on the ci

* new currency reference rates pr (#5724)

* point files to linters (#5739)

* remove news from equity (#5741)

* Add tiingo provider (#5740)

* first commit

* things

* Add news endpoints

* CryptoHistorical

* Add currency and trailing dividend yield

* things working

* deps

* fix pre commit things

* tests?

* black on the tests

* pratty told me to add it

* integration tests

* added helpers

* crypto revamped

* currency revamped

* trailing dividend yield revamped

* equity historical revamped

* company news overhauled

* global news overhauled

* remove `tags` param from news models

* crypto correct `exchanges` description

* james smh

* re-recorded fetcher tests

* fixed integration tests

* econometrics smh

* james smh again

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Add `derivatives` extension for `futures` and `options` (#5742)

* removed options and futures

* add derivatives extension

* cleanup

* bug #5649 (#5722)

* bug #5649

* fixing --a upmom, downmom

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* Rename the fmp search -> screener (#5744)

* Rename the fmp search -> screener

* misses some

* i cant copy paste

* docs/fix-urls: Updates the Terminal About URLs to match `/develop` (#5720)

* update Terminal about URLs and some other broken links

* ruff

* intro

* changing the heading levels

* dependency management sections

* structure

* developer guides

* structure

* contributor guides

* keywords and descriptions

* broken urls

* cool cards

* Cool cards (#5727)

* tidy up terminal/content

* some more tidy up

* fix styling for platform that was missing

* more tidy up

* improve color for dark and light mode

* create new reference card for more basic cards

* rename back to menus

* implement all index.md for terminal, except reference

* improve bot title

* more refactoring

* fix remaining index

* header styling

* more styling

* heheheheheheeh

* Revert "heheheheheheeh"

This reverts commit 7d095fcfd2.

* Ccards (#5733)

* links

* more links

* improved styling and chevron aligned to title

* fix index.mdx urls for Terminal

* update package.json

* update index.mdx

* fix generation

* spelling

* toolkit_extensions

* update data extensions

* couple of links

* add tiingo to data extensions list

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: andrewkenreich <andrew.kenreich@gmail.com>
Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* Stock leftover cleanup (#5745)

* change to equity in `.md` files

* `HistoricalStockSplits`->`HistoricalSplits`

* `StockInsiderTrading`->`InsiderTrading`

* sorry fmp

* `EquityHistorical` cleanup

* `EquityNBBO` cleanup

* .md leftover

* the last of stocks

* Hotfix/urllib to dev group (#5732)

* use the urllib3 dependency on the dev group instead

* adapt the version script for the nightlies - if it's a nightly the package distribution name will have a different name

* add dev dependencies to the dev installation bc of the ci

* updating the import on validate_call according to: https://docs.pydantic.dev/latest/concepts/validation_decorator/
the way we were importing it breaks if pydantic=2.5
@montezdesousa

* loosing urllib3 version specification on the provider and adding it to platform dev deps

* rmeoving urllib from top level deps; and removing --with dev flag

* changing to ^ instead

* Revert "changing to ^ instead"

This reverts commit 7300904e9f.

* unnecessary dep

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>

* feature/v4-bop - Add ECB Balance of Payments (Replaces #5716) (#5726)

* add ecb balance of payments

* missing __init__.py files

* Nasdaq calendar earnings and fix FMP calendar earnings (#5718)

Co-authored-by: James Maslek <jmaslek11@gmail.com>

* feature/us-treasury-auctions: Adds Treasury Auctions endpoint and provider for US Government Data. (#5746)

* add government_us provider and treasury auctions endpoint

* black

* add param for cusip

* move logic for setting default start/end date

* Update hub<>platform credentials map (#5749)

* update hub<>platform credentials map

* typo

* Add API settings schema to system settings (#5743)

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* Skip checking creds (#5737)

* skip checking creds

* fix bug

* ok mypy

* add comment

* rename required_credentials

* fix

* fix top_retail bug

* don't require creds in some nasdaq fetchers

* black

* this was not here before

* fix tests

* update fetcher test

* update contributing guidelines

* Fix Quote (#5752)

* Fix Quote

* fixed return type and model validation

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Wrap up renaming global news to world news on a model level (#5750)

* Wrap up renaming global news to world news on a model level

* cleanup

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Remove eod from docstrings (#5751)

* Wrap up renaming global news to world news on a model level

* Remove "end of day" from historical price model docstrings

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Massive docstring cleanup (#5755)

* sec router remove command

* standard models cleanup

* fmp cleanup

* av cleanup

* benzinga cleanup

* biztoc cleanup

* cboe cleanup

* ecb cleanup

* finra cleanup

* fred cleanup

* us cleanup

* intrinio cleanup

* nasdaq cleanup

* oecd cleanup

* polygon cleanup

* sec cleanup

* seeking alpha cleanup

* stockgrid cleanup

* tiingo cleanup

* tradingeconomics cleanup

* ultima cleanup

* wsj cleanup

* yfinance cleanup

* sec router lint

* fix date query param description

* Updates system and api settings (#5753)

* remove uncessary import on rest api

* remove FrozenField and add api.version

* allow setup from json

* remove id tag

* computed field is better

* formatting

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* Squeeze build time (#5756)

* squeeze build time

* bug fix

* fix test

* typo?

* rebuild

* keep tradingeconomics

* docs/working-docs: Working V4 Docs PR (#5747)

* post-install clarification, fix links

* more links

* change word

* toolkit cleanup

* rename 'contributing' to 'development'

* update extensions

* some urls

* Update sdk-platform and v4-develop in urls (#5730)

* Update sdk-platform and v4-develop in urls

* Seprate Intrinio financials data tags into a different command (#5723)

* add `ebit` field in income statement standard model

* only add required data tags

* remove calculations data

* use `name` key instead of `tag` key

* set `period` as required in data

* remove `period` and `cik` from provider models

* add `equity/fundamental/search_financial_attributes` with tests

* clean descriptions file

* add `equity/fundamental/financial_attributes` with tests

* set `value` as `Optional`

* Fix threadpool test error (#5734)

* Remove ultima from deps (#5736)

* remove ultima from deps

* skip ultima test

* black

* c…

* More v3 removals (#6141)

* remove ultima + althub

* move twitter keys

* economy kill bigmac + remove eval + plot+ stored data

* am losing track

* oanda is next

* brokers now

* fix sdk in case anyone wants it

* missed an sdk endpoint

* spec file

* small edits for installer

* stop the int tests on these

* always be fixin bugs

* some more removals

* [Enhancement] Show the source of error in exception traceback (#6153)

* show source of error

* restructure code for readability

* Improve the OECD requests (#6152)

* remove ultima + althub

* move twitter keys

* Enhance caching + more specific urls

* Update urls for stir

* LTIR

* cli

* one v3 file to fix

* Typing edits + gdp dates + gdp all countries

* Tests

* lint

* not sure why those didnt record

* handle the fact that we artifically add some start dates and end dates

* fix dates with actual solution not weird patch

* lint

* adds AlphaVantage to historical_eps (#6155)

* [BugFix] Fix FMP Market Snapshots (#6160)

* fix FMP market snapshots

* exception handling

* model type

* forgot one line

* fix test

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Excel help page (#6165)

* create help page

* rename file

* words

* words

* more issues

* less words

* minor fix

* email

* no bare urls

* unused attribute

* ruff

* [Feature] Add Currency Snapshots With Data From FMP (#6162)

* add obb.currency.snapshots() endpoint and create new standard model

* add standard model

* static assets

* add couple of comments

* codespell

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] Add `form_13f` Endpoint to `equity.ownership` (#6122)

* add form_13f endpoint

* review things

* option_type validator

* black

* codespell

* add some comments to the code

* remove duplicated process

* static file

* remove warning

* static why no one updates you :-(

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Bug fix] - Run charting tests again (#6167)

* run tests again

* forgot these 2

* [Enhancement] Slightly Change The Order of intrinio Cash Flow (#6168)

* fix order of intrinio cash flow

* apparently homebrew takes priority, deleted that black

* [Bug fix] - Fix integration tests (#6170)

* fix integration tests

* rebuild

* unittest

* [Bug fix] - Handle multiple items with arbitrary type (#6171)

* handle multiple items with arbitrary type

* minor fix

* ruff

* inequality

* integration tests

* test

* pylint

* fix tests

* fix category

* ruff

* [Feature] Custom choices (#6169)

* custom choices

* remove unnecessary choices from the extra info

* static assets

* simpler way to pop choices on extra

* Getting json_schema_extra without changing the original dict

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>

* exception in debug mode shows complete argument (#6174)

* [Bugfix] - Lower case validator in standard models (#6172)

* add to_lower validator

* change validator name

* docstrings

* fix interval

* [Enhancement] Standardize "X".price.historical Intervals and Params (#6146)

* FMP

* tiingo

* yfinance

* intrinio

* polygon

* fmp interval descriptions

* alpha_vantage

* update unit tests

* tests

* static files

* black

* black

* black again

* codespell

* codespell again

* futures historical

* ruff

* some more black

* pylint

* now black again

* test params

* skip deprecated

* fmp test cassette

* fix some tests

* fix test_query

* fix bad symbol in toolkit integration tests

* attempt at parameter deprecation warning

* Fix edge case

* test params

* more test params

* that's the ticket

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - refactor examples (#6154)

* remove auto examples and split python from api

* add missing exampels

* rebuild

* query params focstring

* black

* add scoped examples

* group model imports in the router

* unblock api

* fix

* classify examples

* fix package identation

* black sig

* fix indentation without linter

* fix python examples

* remove doc

* pylint

* again

* add model validators

* commodity, crypto, currency

* filter examples

* fix python examples

* fix filter and build

* undo toml change

* add comment

* oop

* comment

* typing

* lint

* Update router.py

* Update econometrics_router.py

* fix to_python

* to_python

* to_python

* black

* fix multiple line field descriptions

* remove circular import

* add missing import

* pass indent as object attr

* doc

* doc

* this looks better

* forgot this

* fix: new examples adapted

* fix prev commit

* avoid future problems

* Update ownership_router.py

* rebuild + remove todo

* make description mandatory for python examples

* simple indent

* append_examples func works for both website and docstring

* Add example unit tests

* Edit example unit tests

* fix validator

* this works

* fix to_python

* fix mandatory docstrings

* typing

* shorter error msgs

* economy router

* fix more examples

* more examples

* more examples

* more examples

* field desc

* minor fix

* fix etf examples

* handling types better

* lint

* rename method

* rename var

* docstring + move method

* typing

* prompt

* rebuild

* prompt fix in PythonEx

* fix examples without model

* example mock data

* add get_field_type func to get field data type

* panel data

* mock data

* fix

* fix

* docstring

* ohlcv

* pylint

* quantitaive router

* simplify mock data

* fix remaining quantitative routers

* a bit more flexibility for examples

* lint

* mock dat from sample

* docstring

* assertion

* bugfix

* minor fix

* technical

* fix integration tests

* fix docstring

* econometrics examples

* econometrics examples

* remove warning

* add technical examples

* fix bad examples

* fix more examples

* update test rules

* remoce old pydantic referenc

* fix bug in treasury prices

* fix quantitative examples

* discovery router

* equity shorts

* equity shorts

* fix currency pairs

* fix sec_router example

* fix panel data to support all funcs

* black

* more examples fixes

* simplify test

* rebuild

* test doc

* add provider to some ex

* ^

* last

* add provider

* rebuild

* working examples

* noqa

* pylint

* pylint

* clean

* minor fix

* fix test

* lint

* ruff

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Enhancement] Improve Intrinio Unusual Options (#6177)

* improve intrinio unusual options

* static files

* fix test

* really fix the test

* added code to get new examples as string (#6181)

* [Feature] Add Tradier Provider Extension (#6178)

* add tradier provider

* python 3-9 tz parsing

* unused argument

* add handling for cassette playback on GitHub

* sort equity historical nicely

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>

* [Feature] - Create static asset tests (#6182)

* Create static asset tests

* Lint

* Add labels for  `breaking_changes` tag (#6184)

* Add labels for  breaking changes***

***Update release-drafter configuration

* Update release-drafter.yml to include emoji for OpenBB Platform Breaking Changes

* [Bugfix] - Fix equity/price/historical w/ alphavantage (#6183)

* fix premium endpoint bug

* lint

* [Enhancement] Allow Multiple Symbols For Intrinio Key Metrics (#6186)

* improve intrinio key metrics

* type thing

* some thing

* ruff

* handle date type (#6185)

* Release bug fixes (#6187)

* release bug fixes

* rebuild

---------

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* [BugFix] Update `README` for Platform (#6189)

* update readme

* prevent oopsies

* beta versions

* print extensions name

* update toml with beta versions

* [BugFix] Raise Alpha Vantage API Error (#6188)

* raise API error

* handle more API error messages

* handle bad data

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

* bump versions for all packages

* update extensions

* update econ cal headers (#6191)

* fix for obbject extensions not added in extension map

* updated extension map

* update lock files

* fix maybe for SDK v3 docs

* Revert "Merge branch 'main' into release/4.1.5"

This reverts commit 5b333dbcd2, reversing
changes made to e4beecf5ef.

* Revert "fix for obbject extensions not added in extension map"

This reverts commit b6bd2b9d5e.

* Revert "updated extension map"

This reverts commit 9ed74fee19.

* don't get del files for terminal

* expiration choice u killing me

* bad mypy

* no linting for terminal v3

* add TODO

* disable pylint for sec files

---------

Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: James Maslek <jmaslek11@gmail.com>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>
Co-authored-by: DidierRLopes <dro.lopes@campus.fct.unl.pt>
Co-authored-by: Luqman <luqazino@gmail.com>
Co-authored-by: minhhoang1023 <40023817+minhhoang1023@users.noreply.github.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Ayrton Parkinson <ayrton.parkinson1@gmail.com>
Co-authored-by: northern-64bit <75195383+northern-64bit@users.noreply.github.com>
Co-authored-by: Luqman <luqman.bello@openbb.finance>
Co-authored-by: Eric Li <38477812+EricLi-Dev@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Andrew <andrew.kenreich@gmail.com>
Co-authored-by: Michael Struwig <mnicstruwig@gmail.com>
Co-authored-by: capital0 <72140796+capital0@users.noreply.github.com>
Co-authored-by: Ben Van Doren <benvandoren@users.noreply.github.com>
Co-authored-by: Adi Sai <adithya.sairam1@gmail.com>
Co-authored-by: Sandip Saha <sandipsmoto@gmail.com>
Co-authored-by: Adwait <111136306+AdwaitSalankar@users.noreply.github.com>
Co-authored-by: Abhishek Amar <66067391+abhishek-amar@users.noreply.github.com>
Co-authored-by: professssor <62937125+professssor@users.noreply.github.com>
Co-authored-by: Farookh Zaheer Siddiqui <129654632+FarukhS52@users.noreply.github.com>
Co-authored-by: MAX SHESTOV <57421753+maxxsh@users.noreply.github.com>
Co-authored-by: jose-donato <zmcdonato@gmail.com>
Co-authored-by: Mohamad Fadil <fadil.parves@gmail.com>
Co-authored-by: Ramazan <guvencramazan@outlook.com>
Co-authored-by: joshuabuildsthings <joshuabuildsthings@users.noreply.github.com>
Co-authored-by: wojda <114031148+polooner@users.noreply.github.com>
Co-authored-by: Bim Phomthong <118195204+bimbolimbo@users.noreply.github.com>
Co-authored-by: Ramazan <guvencramazan@yandex.com>
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com>
2024-03-12 08:16:42 -04:00

74 KiB
Vendored
Raw Blame History

CONTRIBUTING

First off, thanks for taking the time to contribute (or at least read the Contributing Guidelines)! 🚀

The following is a set of guidelines for contributing to OpenBB Terminal. These are mostly guidelines, not rules. Use your best judgment, and feel free to propose changes to this document in a pull request.

BASIC

Adding a new command

Before implementing a new command we highly recommend that you go through Understand Code Structure and Follow Coding Guidelines. This will allow you to get your PR merged faster and maintain consistency in our code base.

In the next sections we describe the process to add a new command. We will be adding a function to get price targets from the Financial Modeling Prep API. Note that there already exists a function to get price targets from the Business Insider website, stocks/fa/pt, so we will be adding a new function to get price targets from the Financial Modeling Prep API, and go through adding sources.

Select Feature

  • Pick a feature you want to implement or a bug you want to fix from our issues.
  • Feel free to discuss what you'll be working on either directly on the issue or on our Discord.
    • This ensures someone from the team can help you and there isn't duplicated work.

Before writing any code, it is good to understand what the data will look like. In this case, we will be getting the price targets from the Financial Modeling Prep API, and the data will look like this:

[
  {
    "symbol": "AAPL",
    "publishedDate": "2023-02-03T16:19:00.000Z",
    "newsURL": "https://pulse2.com/apple-stock-receives-a-195-price-target-aapl/",
    "newsTitle": "Apple Stock Receives A $195 Price Target (AAPL)",
    "analystName": "Cowen Cowen",
    "priceTarget": 195,
    "adjPriceTarget": 195,
    "priceWhenPosted": 154.5,
    "newsPublisher": "Pulse 2.0",
    "newsBaseURL": "pulse2.com",
    "analystCompany": "Cowen & Co."
  }

Model

  1. Create a file with the source of data as the name followed by _model if it doesn't exist. In this case, the file openbb_terminal/stocks/fundamental_analysis/fmp_model.py already exists, so we will add the function to that file.
  2. Add the documentation header
  3. Do the necessary imports to get the data
  4. Define a function starting with get_
  5. In this function:
    1. Use type hinting
    2. Write a descriptive description where at the end the source is specified.
    3. Utilize an official API, get and return the data.
""" Financial Modeling Prep Model """
__docformat__ = "numpy"

import logging

import pandas as pd

from openbb_terminal.core.session.current_user import get_current_user
from openbb_terminal.decorators import check_api_key, log_start_end
from openbb_terminal.helpers import request

logger = logging.getLogger(__name__)

@log_start_end(log=logger)
@check_api_key(["API_KEY_FINANCIALMODELINGPREP"])
def get_price_targets(cls, symbol: str) -> pd.DataFrame:
    """Get price targets for a company [Source: Financial Modeling Prep]

    Parameters
    ----------
    symbol : str
        Symbol to get data for

    Returns
    -------
    pd.DataFrame
        DataFrame of price targets
    """
    current_user = get_current_user()

    url = f"https://financialmodelingprep.com/api/v4/price-target?symbol={symbol}&apikey={current_user.credentials.API_KEY_FINANCIALMODELINGPREP}"
    response = request(url)

    # Check if the response is valid
    if response.status_code != 200 or "Error Message" in response.json():
        message = f"Error, Status Code: {response.status_code}."
        message = (
            message
            if "Error Message" not in response.json()
            else message + "\n" + response.json()["Error Message"] + ".\n"
        )
        console.print(message)
        return pd.DataFrame()

    return pd.DataFrame(response.json())

In this function:

  • We import the current user object and, consequently, preferences using the get_current_user function. API keys are stored in current_user.credentials
  • We use the @log_start_end decorator to add the function to our logs for debugging purposes.
  • We add the @check_api_key decorator to confirm the API key is valid.
  • We have type hinting and a docstring describing the function.
  • We use the openbb_terminal helper function request, which is an abstracted version of the requests library, which allows us to add user agents, timeouts, caches, etc. to any HTTP request in the terminal.
  • We check for different error messages. This will depend on the API provider and usually requires some trial and error. With the FMP API, if there is an invalid symbol, we get a response code of 200, but the json response has an error message field. Same with an invalid API key.
  • When an error is caught, we still return an empty dataframe.
  • We return the json response as a pandas dataframe. Most functions in the terminal should return a dataframe, but if not, make sure that the return type is specified.

Note:

  1. If the function is applicable to many asset classes, it is possible that this file needs to be created under common/ directory rather than stocks/, which means the function should be written in a generic way, i.e. not mentioning stocks or a specific context.
  2. If the model requires an API key, make sure to handle the error and output relevant message.
  3. If the data provider is not yet supported, you'll most likely need to do some extra steps in order to add it to the keys menu. See this section for more details.

Some of the most common error messages are:

  • Error in the request (HTTP error)
  • Invalid API Keys
  • API Keys not authorized for Premium feature
  • Empty return payload
  • Invalid arguments (Optional)

In the example below, you can see that we explicitly handle some of them. It's not always possible to distinguish error types using status_code. So depending on the API provider, you can use either error messages or exceptions.

@log_start_end(log=logger)
@check_api_key(["API_NEWS_TOKEN"])
def get_news(
    query: str,
    limit: int = 10,
    start_date: Optional[str] = None,
    show_newest: bool = True,
    sources: str = "",
) -> pd.DataFrame:

    ...

    link += f"&apiKey={get_current_user().credentials.API_NEWS_TOKEN}"
    response = request(link)
    articles = {}

    if response.status_code == 200:
        response_json = response.json()
        articles = (response_json["articles"] if show_newest else response_json["articles"][::-1])

    elif response.status_code == 426:
        console.print(f"Error in request: {response.json()['message']}", "\n")
    elif response.status_code == 401:
        console.print("[red]Invalid API Key[/red]\n")
    elif response.status_code == 429:
        console.print("[red]Exceeded number of calls per minute[/red]\n")
    else:
        console.print(f"Error in request: {response.json()['message']}", "\n")

    ...

Click here to see the example in detail.

Data sources

Now that we have added the model function getting, we need to specify that this is an available data source. To do so, we edit the openbb_terminal/miscellaneous/sources/openbb_default.json file. This file, described below, uses a dictionary structure to identify available sources. Since we are adding FMP to stocks/fa/pt, we find that entry and append it:

    "fa": {
      "pt": ["BusinessInsider", "FinancialModelingPrep"],

If you are adding a new function with a new data source, make a new value in the file. If the data source requires an API key, please refer to the guide below for adding them. Instructions for obtaining the new api key should be included in the file OpenBBTerminal/website/content/terminal/usage/data/api-keys.md.

View

  1. Create a file with the source of data as the name followed by _view if it doesn't exist, e.g. fmp_view
  2. Add the documentation header
  3. Do the necessary imports to display the data. One of these is the _model associated with this _view. I.e. from same data source.
  4. Define a function starting with display_
  5. In this function:
    • Use typing hints
    • Write a descriptive description where at the end the source is specified
    • Get the data from the _model and parse it to be output in a more meaningful way.
    • Do not degrade the main data dataframe coming from model if there's an export flag. This is so that the export can have all the data rather than the short amount of information we may show to the user. Thus, in order to do so df_data = df.copy() can be useful as if you change df_data, df remains intact.
  6. If the source requires an API Key or some sort of tokens, add check_api_key decorator on that specific view. This will throw a warning if users forget to set their API Keys
  7. Finally, call export_data where the variables are export variable, current filename, command name, and dataframe.
@log_start_end(log=logger)
@check_api_key(["API_KEY_FINANCIALMODELINGPREP"])
def display_price_targets(
    symbol: str, limit: int = 10, export: str = "", sheet_name: Optional[str] = None
):
    """Display price targets for a given ticker. [Source: Financial Modeling Prep]

    Parameters
    ----------
    symbol : str
        Symbol
    limit: int
        Number of last days ratings to display
    export: str
        Export dataframe data to csv,json,xlsx file
    sheet_name: str
        Optionally specify the name of the sheet the data is exported to.
    """
    columns_to_show = [
        "publishedDate",
        "analystCompany",
        "adjPriceTarget",
        "priceWhenPosted",
    ]
    price_targets = fmp_model.get_price_targets(symbol)
    if price_targets.empty:
        console.print(f"[red]No price targets found for {symbol}[/red]\n")
        return
    price_targets["publishedDate"] = price_targets["publishedDate"].apply(
        lambda x: datetime.strptime(x, "%Y-%m-%dT%H:%M:%S.%fZ").strftime("%Y-%m-%d %H:%M")
    )
    export_data(
        export,
        os.path.dirname(os.path.abspath(__file__)),
        "pt",
        price_targets,
        sheet_name,
    )

    print_rich_table(
        price_targets[columns_to_show].head(limit),
        headers=["Date", "Company", "Target", "Posted Price"],
        show_index=False,
        title=f"{symbol.upper()} Price Targets",
    )

In this function:

  • We use the same log and API decorators as in the model.
  • We define the columns we want to show to the user.
  • We get the data from the fmp_model function
  • We check if there is data. If something went wrong, we don't want to show it, so we print a message and return. Note that because we have error messages in both the model and view, there will be two print outs. If you wish to just show one, it is better to handle in the model.
  • We do some parsing of the data to make it more readable. In this case, the output from FMP is not very clear at quick glance, we we put it into something more readable.
  • We export the data. In this function, I decided to export after doing the manipulation. If we do any removing of columns, we should copy the dataframe before exporting.
  • We print the data in table form using our print_rich_table. This provides a nice console print using the rich library. Note that here I show the top limit rows of the dataframe. Care should be taken to make sure that things are sorted. If a sort is required, there is a reverse argument that can be added to sort in reverse order.

Controller

Now that we have the model and views, it is time to add to the controller.

  1. Import the associated _view function to the controller.

  2. Add command name to variable CHOICES_COMMANDS from FundamentalAnalysisController class.

  3. Add command and source to print_help().

     def print_help(self):
         """Print help."""
         mt = MenuText("stocks/fa/")
         mt.add_cmd("load")
         mt.add_raw("\n")
         mt.add_param("_ticker", self.ticker.upper())
         mt.add_raw("\n")
         mt.add_info("_company_overview")
         mt.add_cmd("mktcap")
         mt.add_cmd("overview")
         mt.add_cmd("divs", not self.suffix)
    
         ...
    
         mt.add_cmd("pt")
         mt.add_cmd("dcf")
         mt.add_cmd("dcfc")
         console.print(text=mt.menu_text, menu="Stocks - Fundamental Analysis")
    
  4. If there is a condition to display or not the command, this is something that can be leveraged through the add_cmd method, e.g. mt.add_cmd("divs", not self.suffix).

  5. Add command description to file i18n/en.yml. Use the path and command name as key, e.g. stocks/fa/pt and the value as description. Please fill in other languages if this is something that you know.

  6. Add a method to FundamentalAnalysisController class with name: call_ followed by command name.

    • This method must start defining a parser with arguments add_help=False and formatter_class=argparse.ArgumentDefaultsHelpFormatter. In addition prog must have the same name as the command, and description should be self-explanatory ending with a mention of the data source.

    • Add parser arguments after defining parser. One important argument to add is the export capability. All commands should be able to export data.

    • If there is a single or even a main argument, a block of code must be used to insert a fake argument on the list of args provided by the user. This makes the terminal usage being faster.

      if other_args and "-" not in other_args[0][0]:
            other_args.insert(0, "-l")
      
    • Parse known args from list of arguments and values provided by the user.

    • Call the function contained in a _view.py file with the arguments parsed by argparse.

Note that the function self.parse_known_args_and_warn() has some additional options we can add. If the function is showing a chart, but we want the option to show raw data, we can add the raw=True keyword and the resulting namespace will have the raw attribute. Same with limit, we can pass limit=10 to add the -l flag with default=10. Here we also specify the export, and whether it is data only, plots only or anything else. This function also adds the source attribute to the namespace. In our example, this is important because we added an additional source.

Our new function will be:

   @log_start_end(log=logger)
    def call_pt(self, other_args: List[str]):
        """Process pt command"""
        parser = argparse.ArgumentParser(
            add_help=False,
            prog="pt",
            description="""Prints price target from analysts. [Source: Business Insider and Financial Modeling Prep]""",
        )
        parser.add_argument(
            "-t",
            "--ticker",
            dest="ticker",
            help="Ticker to analyze",
            type=str,
            default=None,
        )
        if other_args and "-" not in other_args[0][0]:
            other_args.insert(0, "-t")
        ns_parser = self.parse_known_args_and_warn(
            parser, other_args, EXPORT_BOTH_RAW_DATA_AND_FIGURES, raw=True, limit=10
        )
        if ns_parser:
            if ns_parser.ticker:
                self.ticker = ns_parser.ticker
                self.custom_load_wrapper([self.ticker])

            if ns_parser.source == "BusinessInsider":
                business_insider_view.display_price_target_from_analysts(
                    symbol=self.ticker,
                    data=self.stock,
                    start_date=self.start,
                    limit=ns_parser.limit,
                    raw=ns_parser.raw,
                    export=ns_parser.export,
                    sheet_name=" ".join(ns_parser.sheet_name)
                    if ns_parser.sheet_name
                    else None,
                )
            elif ns_parser.source == "FinancialModelingPrep":
                fmp_view.display_price_targets(
                    symbol=self.ticker,
                    limit=ns_parser.limit,
                    export=ns_parser.export,
                    sheet_name=" ".join(ns_parser.sheet_name)
                    if ns_parser.sheet_name
                    else None,
                )

Here, we make the parser, add the arguments, and then parse the arguments. In order to use the fact that we had a new source, we add the logic to access the correct view function. In this specific menu, we also allow the user to specify the symbol with -t, which is what the first block is doing.

Note that in the fa submenu, we allow the function to be run by specifying a ticker, ie pt -t AAPL. In this submenu we do a load behind the scenes with the ticker selected so that other functions can be run without specifying the ticker.

Now from the terminal, this function can be run as desired:

2023 Mar 03, 11:37 (🦋) /stocks/fa/ $ pt -t aapl --source FinancialModelingPrep

                         AAPL Price Targets
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Date             ┃ Company               ┃ Target ┃ Posted Price ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━┩
│ 2023-02-03 16:19 │ Cowen & Co.           │ 195.00 │ 154.50       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 09:31 │ D.A. Davidson         │ 173.00 │ 157.09       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 08:30 │ Rosenblatt Securities │ 173.00 │ 150.82       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 08:29 │ Wedbush               │ 180.00 │ 150.82       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 07:21 │ Raymond James         │ 170.00 │ 150.82       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 07:05 │ Barclays              │ 145.00 │ 150.82       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-03 03:08 │ KeyBanc               │ 177.00 │ 150.82       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-02 02:08 │ Rosenblatt Securities │ 165.00 │ 145.43       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-02 02:08 │ Deutsche Bank         │ 160.00 │ 145.43       │
├──────────────────┼───────────────────────┼────────┼──────────────┤
│ 2023-02-02 02:08 │ J.P. Morgan           │ 180.00 │ 145.43       │
└──────────────────┴───────────────────────┴────────┴──────────────┘

When adding a new menu, the code looks like this:

@log_start_end(log=logger)
def call_fa(self, _):
    """Process fa command"""
    from openbb_terminal.stocks.fundamental_analysis.fa_controller import (
        FundamentalAnalysisController,
    )

    self.queue = self.load_class(
        FundamentalAnalysisController, self.ticker, self.start, self.stock, self.queue
    )

The import only occurs inside this menu call, this is so that the loading time only happens here and not at the terminal startup. This is to avoid slow loading times for users that are not interested in stocks/fa menu.

In addition, note the self.load_class which allows to not create a new FundamentalAnalysisController instance but re-load the previous created one. Unless the arguments self.ticker, self.start or self.stock have changed since. The self.queue list of commands is passed around as it contains the commands that the terminal must perform.

Add SDK endpoint (V3)

In order to add a command to the SDK, follow these steps:

  1. If you've created a new model or view file, add the import with an alias to openbb_terminal/core/sdk/sdk_init.py following this structure:

    # Stocks - Fundamental Analysis
    from openbb_terminal.stocks.fundamental_analysis import (
        finviz_model as stocks_fa_finviz_model,
        finnhub_model as stocks_fa_finnhub_model,
        finnhub_view as stocks_fa_finnhub_view,
    )
    
  2. Go to the trail_map.csv file located in openbb_terminal/core/sdk, which should look like this:

    trail,model,view
    stocks.fa.analyst,stocks_fa_finviz_model.get_analyst_data,
    stocks.fa.rot,stocks_fa_finnhub_model.get_rating_over_time,stocks_fa_finnhub_view.rating_over_time
    
    ...
    
    

    In this file, the trail represents the path to the function to be called. The model represents the import alias we gave to the _model file. The view represents the import alias we gave to the _view file.

  3. Add your new function to this structure. In the below example of the pt function, our trail would be stocks.fa.pt.

    Our naming convention is such that the data source should not be included in the trail. In this example, calling a new function pt_fmp would not be allowed. For functions with multiple sources, there should be a single pt function that takes in the source as an argument. In the following example, we will stick with showing how the business_insider was initially added to the sdk.

    The model is the import alias to the _model function that was written:

    • stocks_fa_business_insider_model.get_price_target_from_analysts

    The view is the import alias to the _view function that was written:

    • stocks_fa_business_insider_view.display_price_target_from_analysts

    The added line of the file should look like this:

    stocks.fa.pt,stocks_fa_business_insider_model.get_price_target_from_analysts,stocks_fa_business_insider_view.display_price_target_from_analysts
    
  4. Generate the SDK files by running python generate_sdk.py from the root of the project. This will automatically generate the SDK openbb_terminal/sdk.py, corresponding openbb_terminal/core/sdk/controllers/ and openbb_terminal/core/sdk/models/ class files.

    To sort the trail_map.csv file and generate the SDK files, run the following command

    python generate_sdk.py sort
    

Add OpenBB Platform endpoint (V4)

Refer to the documentation here.

Add Unit Tests

This is a vital part of the contribution process. We have a set of unit tests that are run on every Pull Request. These tests are located in the tests folder.

Unit tests minimize errors in code and quickly find errors when they do arise. Integration tests are standard usage examples, which are also used to identify errors.

A thorough introduction on the usage of unit tests and integration tests in OpenBBTerminal can be found on the following page:

Unit Test README

Integration Test README

Any new features that do not contain unit tests will not be accepted.

Open a Pull Request

For starters, you should ensure that your branch is up to date with the develop branch. To do that, one can run the following commands:

git fetch upstream
git checkout develop
git merge upstream/develop

After that, you can create a new branch for your feature. E.g. git checkout -b feature/AmazingFeature.

Once you're happy with what you have, push your branch to remote. E.g. git push origin feature/AmazingFeature.

Note that we follow gitflow naming convention, so your branch name should be prefixed with feature/ or hotfix/ depending on the type of work you are doing. To learn more, please refer to Branch Naming Conventions.

A user may create a Draft Pull Request when there is the intention to discuss implementation with the team.

Review Process

As soon as the Pull Request is opened, our repository has a specific set of github actions that will not only run linters on the branch just pushed, but also run pytest on it. This allows for another layer of safety on the code developed.

In addition, our team is known for performing diligent code reviews. This not only allows us to reduce the amount of iterations on that code and have it to be more future proof, but also allows the developer to learn/improve his coding skills.

Often in the past the reviewers have suggested better coding practices, e.g. using 1_000_000 instead of 1000000 for better visibility, or suggesting a speed optimization improvement.

Understand Code Structure

Backend

CLI 💻 → _controller.py 🤖 → _view.py 🎨     →      _model.py 🧠
                                                      chart=True                 chart=False
                                                                    ↑    sdk.py 🏭   ↑

File                            Role Description
_controller.py 🤖 The router/input validator The controller file should hold the least amount of logic possible. Its role is to be a stupid (no logic) router and redirect the command correctly while checking the input with argparser.
_view.py 🎨 The artist The view file should only output or visualize the data it gets from the _model file! The _view can limit the data coming from the _model, otherwise the data object should be identical in the _view and the _model files.
_model.py 🧠 The brain The model file is where everything fun happens. The data is gathered (external APIs), processed and returned here.
sdk.py 🏭 The SDK Factory The SDK file is where the callable functions are created for the SDK. There is only one SDK file in the openbb_terminal folder.

Frontend

Item Description Example
CONTEXT Specific instrument world to analyse. stocks, crypto, economy
CATEGORY Group of similar COMMANDS to do on the instrument
There are specialized categories, specific to each CONTEXT and there are common categories which are not specific to one CONTEXT.
due_diligence, technical_analysis, insider
COMMAND Operation on one or no instrument that retrieves data in form of string, table or plot. rating, supplier, sentiment

The following layout is expected: /<context>/<category>/<command_files>

If there are sub-categories, the layout will be: /<context>/<category>/<sub-category>/<command_files>

Example:

openbb_terminal/stocks/stocks_controller.py
                      /stocks_helper.py
                      /technical_analysis/ta_controller.py
                                         /tradingview_view.py
                                         /tradingview_model.py
                /common/technical_analysis/overlap_view.py
                                          /overlap_model.py
                /crypto/crypto_controller.py
                       /crypto_helper.py
                       /due_diligence/dd_controller.py
                                     /binance_view.py
                                     /binance_model.py
                       /technical_analysis/ta_controller.py

With:

Context Category File Description
stocks/ stocks_controller.py Manages stocks context from a user perspective, i.e. routing commands and arguments to output data, or, more importantly, redirecting to the selected category.
stocks/ stocks_helper.py Helper to stocks menu. This file is meant to hold generic purpose stocks functionalities.
stocks/ technical_analysis/ ta_controller.py Manages technical_analysis category from stocks context from a user perspective, i.e. routing commands and arguments to output data.
stocks/ technical_analysis/ tradingview_view.py This file contains functions that rely on TradingView data. These functions represent commands that belong to technical_analysis category from stocks context. These functions are called by ta_controller.py using the arguments given by the user and will output either a string, table or plot.
stocks/ technical_analysis/ tradingview_model.py This file contains functions that rely on TradingView data. These functions represent commands that belong to technical_analysis category from stocks context. These functions are called by tradingview_view.py and will return data to be processed in either a string, dictionary or dataframe format.
common/ technical_analysis/ overlap_view.py This file contains functions that rely on overlap data. In this case overlap is not a data source, but the type of technical analysis performed. These functions represent commands that belong to technical_analysis category from MULTIPLE contexts. These functions are called by ta_controller.py, from MULTIPLE contexts, using the arguments given by the user and will output either a string, table or plot. Due to the fact that this file is common to multiple contexts the functions need to be generic enough to accommodate for this. E.g. if we are providing a dataframe to these functions, we should make sure that stocks/ta_controller.py and crypto/ta_controller use the same formatting.
common/ technical_analysis/ overlap_model.py This file contains functions that rely on overlap data. In this case overlap is not a data source, but the type of technical analysis performed. These functions represent commands that belong to technical_analysis category from MULTIPLE contexts. These functions are called by overlap_view.py, and will return data to be processed in either a string, dictionary or dataframe format. Due to the fact that this file is common to multiple contexts the functions need to be generic enough to accommodate for this. E.g. if we are getting the sentiment of an instrument, we should ensure that these functions accept both a "GME" or a "BTC", for stocks and crypto, respectively.

Follow Coding Guidelines

General Code Requirements

  1. Each function should have default values for non critical kwargs

    • Why? It increases code readability and acts as an input example for the function's arguments. This increases the ease of use of the functions through the SDK, but also just generally.

    Watch out, add default values whenever possible, but take care for not adding mutable default arguments! More info


    Good code ✅ Bad code ❌
    def display_last_uni_swaps(
      top: int = 10,
      sortby: str = "timestamp",
      descend: bool = False,
      export: str = "",) -> None:
    
    def display_last_uni_swaps(
      top: int,
      sortby: str,
      descend: bool,
      export: str,) -> None:
    

  2. Simple and understandable input objects; avoid for example weird dictionaries packed with data: {“title”: DataFrame}

    • Why? Ease of use and often these complex formats are clumsy, prone to error and the formatting required for complex parameters is time consuming and unneeded.

    Good code ✅ Bad code ❌
    def get_coins(
      top: int = 250,
      category: str = "") -> pd.DataFrame:
    
    def load(
      file: str,
      file_types: list,
      data_files: Dict[Any, Any],
      data_examples: Dict[Any, Any],) -> pd.DataFrame:
    

  3. Each function needs to have a docstring explaining what it does, its parameters and what it returns.

    • Why? You can use the function without reading its source code. This improves the developing experience and SDK usage. The SDK factory also can’t handle functions without docstrings.

  4. Consistent and clear argument naming; not symbol in _view and then ticker in _file -> ticker everywhere; the name should be descriptive of what information it holds (see Style Guide section below)

    • Why? You can quickly understand what the input should be; example: tickers and stock names are fundamentally different, but they’re both strings so they should be named accordingly.

    Good code ✅ Bad code ❌
    data: pd.Series, dataset_name: str, y_label: str,
    
    data: pd.Series, dataset: str, column: str,
    

  5. Classes (for example the portfolio class) should hold the relevant data and perform no other calculations, these calculations should be done in an independent function.

    • Why? Two reasons:

    These calculations can then be used outside of the class with custom data; for example via the sdk or for tests.

    from openbb_terminal.portfolio.portfolio_helper import get_gaintopain_ratio
    
    # Direct function access
    get_gaintopain_ratio(historical_trade_data, benchmark_trades, benchmark_returns)
    

    The function can be loaded in SDK factory as an endpoint and user can get result by passing the class instance.

    from openbb_terminal.sdk import openbb
    from openbb_terminal.sdk import Portfolio
    
    transactions = Portfolio.read_orderbook("../../portfolio/holdings/example.csv")
    P = Portfolio(transactions)
    P.generate_portfolio_data()
    P.set_benchmark()
    
    # SDK endpoint access
    openbb.portfolio.gaintopain(P)
    
    Good code ✅ Bad code ❌
    def get_gaintopain_ratio(portfolio: PortfolioEngine) -> pd.DataFrame:
    
    """..."""
    
    gtp_period_df = portfolio_helper.get_gaintopain_ratio(
      portfolio.historical_trade_data,
      portfolio.benchmark_trades,
      portfolio.benchmark_returns)
    
    return gtp_period_df
    
    def get_gaintopain_ratio(self) -> pd.DataFrame:
    
    """..."""
    
    vals = list()
    
    for period in portfolio_helper.PERIODS:
           port_rets = portfolio_helper.filter_df_by_period(self.portfolio_returns, period)
           bench_rets =  portfolio_helper.filter_df_by_period(self.benchmark_returns, period)
    
    ...
    

  6. Naming among related model and view functions should be obvious; just different prefix if possible

    • Why? Eases SDK factory mapping and keeps code clean.

    Good code ✅ Bad code ❌
    # [fred_view.py]
    
    def display_yieldcurve(country: str):
    
          df = fred_model.get_yieldcurve(country)
    
          …
    
    # [fred_model.py]
    
    def get_yieldcurve(country: str) -> pd.Dataframe:
    
          …
    
    # [fred_view.py]
    
    def display_bondscrv(country: str):
    
          df = fred_model.get_yieldcurve(country)
    
          …
    
    # [fred_model.py]
    
    def get_yldcurve(country: str) -> pd.Dataframe:
    
          …
    

File Specific Requirements

  1. No data altering in the view file or controller file (view and model with same args)

    • Why? Consistency and good code structure. This also improves the SDK user experience. Thus follows that view and model files will have the same arguments (except for output options like raw, export, external_axes), since no data changes shall be done in the view file.

  2. Each model (get_) should almost always have its own view function (display_)

    • Why? To respect the principles laid out in Code Structure and the previous bullet point. If your code does not have this get_ → display_ map it’s likely that i. and/or ii. fail to hold.

      i. Data is processed in _model files and displayed in _view files

      ii. _view and _model files will have the same arguments (except for output options)


Coding Style

When in doubt, follow https://www.python.org/dev/peps/pep-0008/.

OpenBB Style Guide

The style guide is a reverse dictionary for argument names, where a brief definition is mapped to an OpenBB recommended argument name and type. When helpful a code snippet example is added below. Following this guide will help keep argument naming consistent and improve SDK users experience.

Style guide structure:

<definition> : <argument_name (argument_type)> e.g. <examples>

def func(..., argument_name: argument_type = default, ...):
    ...

Flags

Show raw data : raw (bool)

def display_data(..., raw: bool = False, ...):
    ...
    if raw:
        print_rich_table(...)

Sort in ascending order : ascend (bool)

def display_data(..., sortby: str = "", ascend: bool = False, ...):
    ...
    if sortby:
        data = data.sort_values(by=sortby, ascend=ascend)

Show plot : plot (bool)

def display_data(..., plot: bool = False, ...):
    ...
    if plot:
        ...
        fig.add_scatter(...)

Output format

Format to export data : export (str), e.g. csv, json, xlsx

def display_data(..., export: str = "", ...):
    ...
    export_data(export, os.path.dirname(os.path.abspath(__file__)), "func", data)

Whether to display plot or return figure (False: display, True: return) : external_axes (bool)

def display_data(..., external_axes: bool = False, ...):
    ...
    fig = OpenBBFigure()
    fig.add_scatter(...)
    return fig.show(external=external_axes)

Field by which to sort : sortby (str), e.g. "Volume"

def display_data(..., sortby: str = "col", ...):
    ...
    if sortby:
        data = data.sort_values(by=sortby)

Maximum limit number of output items : limit (int)

def display_data(..., limit = 10, ...):
    ...
    print_rich_table(
        data[:limit],
        ...
    )

Date from which data is fetched (YYYY-MM-DD) : start_date (str), e.g. 2022-01-01

Date up to which data is fetched (YYYY-MM-DD) : end_date (str), e.g. 2022-12-31

Note: We want to accept dates in string because it is easier to deal from user standpoint. Inside the function you can convert it to datetime and check its validity. Please specify date format in docstring.

def get_historical_data(..., start_date: str = "2022-01-01", end_date: str = "2022-12-31",):
    """
    ...
    Parameters
    ----------
    start_date: str
        Date from which data is fetched in format YYYY-MM-DD
    end_date: str
        Date up to which data is fetched in format YYYY-MM-DD
    ...
    """
    data = source_model.get_data(data_name, start_date, end_date, ...)

Year from which data is fetched (YYYY) : start_year (str), e.g. 2022

Year up to which data is fetched (YYYY) : end_year (str), e.g. 2023

def get_historical_data(..., start_year: str = "2022", end_year str = "2023", ...):
    ...
    data = source_model.get_data(data_name, start_year, end_year, ...)

Interval for data observations : interval (str), e.g. 60m, 90m, 1h

def get_prices(interval: str = "60m", ...):
    ...
    data = source.download(
        ...,
        interval=interval,
        ...
    )

Rolling window length : window (int/str), e.g. 252, 252d

def get_rolling_sum(returns: pd.Series, window: str = "252d"):
    rolling_sum = returns.rolling(window=window).sum()

Data selection and manipulation

Search term used to query : query (str)

Maximum limit of search items/periods in data source: limit (int)

Note: please specify limit application in docstring

def get_data_from_source(..., limit: int = 10, ...):
    """
    Parameters
    ----------
    ...
    limit: int
        Number of results to fetch from source
    ...
    """
    data = source.get_data(data_name, n_results=limit, ...)

Dictionary of input datasets : datasets (Dict[str, pd.DataFrame])

Note: Most occurrences are on the econometrics menu and might be refactored in near future

Input dataset : data (pd.DataFrame)

def process_data(..., data: pd.DataFrame, ...):
    """
    ...
    Parameters
    ----------
    ...
    data : pd.DataFrame
        Dataframe of ...
    ...
    """
    col_data = pd.DataFrame(data["Col"])

Dataset name : dataset_name (str)

Input series : data (pd.Series)

Dependent variable series : dependent_series (pd.Series)

Independent variable series : independent_series (pd.Series)

def get_econometric_test(dependent_series, independent_series, ...):
    ...
    dataset = pd.concat([dependent_series, independent_series], axis=1)
    result = econometric_test(dataset, ...)

Country name : country (str), e.g. United States, Portugal

Country initials or abbreviation : country_code (str) e.g. US, PT, USA, POR

Currency to convert data : currency (str) e.g. EUR, USD


Financial instrument characteristics

Instrument ticker, name or currency pair : symbol (str), e.g. AAPL, ethereum, ETH, ETH-USD

def get_prices(symbol: str = "AAPL", ...):
    ...
    data = source.download(
        tickers=symbol,
        ...
    )

Instrument name: name (str)

Note: If a function has both name and symbol as parameter, we should distinguish them and call it name

List of instrument tickers, names or currency pairs : symbols (List/List[str]), e.g. ["AAPL", "MSFT"]

Base currency under BASE-QUOTE → XXX-YYY convention : from_symbol (str), e.g. ETH in ETH-USD

Quote currency under BASE-QUOTE → XXX-YYY convention : to_symbol (str), e.g. USD in ETH-USD

def get_exchange_rate(from_symbol: str = "", to_symbol: str = "", ...):
    ...
    df = source.get_quotes(from_symbol, to_symbol, ...)

Instrument price : price (float)

Instrument implied volatility : implied_volatility (float)

Option strike price : strike_price (float)

Option days until expiration : time_to_expiration (float/str)

Risk free rate : risk_free_rate (float)

Options expiry date : expiry (str)


Naming Convention

  • The name of the variables must be descriptive of what they stand for. I.e. ticker is descriptive, aux is not.
  • Single character variables must be avoided. Except if they correspond to the iterator of a loop.

Docstrings

The docstring format used in numpy, an example is shown below:

def command_foo(var1: str, var2: List[int], var3: bool = False) -> Tuple[int, pd.DataFrame]:
"""Small description

[Optional: Longer description]

Parameters
----------
var1 : str
    var1 description
var2 : List[int]
    var2 description
var3 : bool, optional
    var3 description

Returns
-------
foo : int
    returned foo description
pd.DataFrame
    dataframe returned
"""

Linters

The following linters are used by our codebase:

Linter Description
bandit security analyzer
black code formatter
codespell spelling checker
ruff a fast python linter
mypy static typing checker
pylint static code analysis
markdownlint markdown linter

Command names

  • The command name should be as short as possible.

  • The command name should allow the user to know what the command refers to without needing to read description. (e.g. earn)

    • If this is not possible, then the command name should be an abbreviation of what the functionality corresponds to (e.g. ycrv for yield curve)
  • The command name should not have the data source explicit

UI and UX

Screenshot 2022-10-26 at 12 17 19

It is important to keep a coherent UI/UX throughout the terminal. These are the rules we must abide:

  • There is 1 single empty line between user input and start of the command output.
  • There is 1 single empty line between command output and the user input.
  • The menu help has 1 empty line above text and 1 empty line below. Both still within the rectangular panel.
  • From menu help rectangular panel there's no empty line below - this makes it more clear to the user that they are inside such menu.

External API Keys

Creating API key

OpenBB Terminal currently has over 100 different data sources. Most of these require an API key that allows access to some free tier features from the data provider, but also paid ones.

When a new API data source is added to the platform, it must be added through credentials_model.py.

In order to do that, you'll simply need to choose from one the following files:

  1. local_credentials.json --> credentials that should only be stored locally and not pushed to the OpenBB Hub like brokerage keys or other very sensitive or personal to the user.
  2. hub_credentials.json --> credentials that should be stored in the OpenBB Hub like API keys to access your favorite providers.

Then just update all_api_keys.json with the instructions to get the api key from the data source website. Make sure that this file has the correct .json format, otherwise the API keys page in the Hub will break (e.g. in json the last element key-value pair shouldn't be followed by a comma, and the last object in a list of dictionaries should also not be followed by a comma).

Note: By differentiating between local and hub credentials, we can ensure that the user's credentials are not pushed to the OpenBB Hub and are only stored locally. This does not mean that the credentials are not secure in the OpenBB Hub, but rather that the user can choose to store them locally if they wish.

Setting and checking API key

One of the first steps once adding a new data source that requires an API key is to add that key to our keys_controller.py. This menu allows the user to set API keys and check their validity.

The following code allows to check the validity of the Polygon API key.


def check_polygon_key(show_output: bool = False) -> str:
    """Check Polygon key

    Parameters
    ----------
    show_output: bool
        Display status string or not. By default, False.

    Returns
    -------
    str
        Status of key set
    """

    current_user = get_current_user()

    if current_user.credentials.API_POLYGON_KEY == "REPLACE_ME":
        logger.info("Polygon key not defined")
        status = KeyStatus.NOT_DEFINED
    else:
        r = request(
            "https://api.polygon.io/v2/aggs/ticker/AAPL/range/1/day/2020-06-01/2020-06-17"
            f"?apiKey={current_user.credentials.API_POLYGON_KEY}"
        )
        if r.status_code in [403, 401]:
            logger.warning("Polygon key defined, test failed")
            status = KeyStatus.DEFINED_TEST_FAILED
        elif r.status_code == 200:
            logger.info("Polygon key defined, test passed")
            status = KeyStatus.DEFINED_TEST_PASSED
        else:
            logger.warning("Polygon key defined, test inconclusive")
            status = KeyStatus.DEFINED_TEST_INCONCLUSIVE

    if show_output:
        console.print(status.colorize())

    return str(status)

Note that there are usually 3 states:

  • defined, test passed: The user has set their API key and it is valid.
  • defined, test failed: The user has set their API key but it is not valid.
  • not defined: The user has not defined any API key.

Note: Sometimes the user may have the correct API key but still not have access to a feature from that data source, and that may be because such feature required an API key of a higher level.

A function can then be created with the following format to allow the user to change its environment key directly from the terminal.

def call_polygon(self, other_args: List[str]):
    """Process polygon command"""
    parser = argparse.ArgumentParser(
        add_help=False,
        formatter_class=argparse.ArgumentDefaultsHelpFormatter,
        prog="polygon",
        description="Set Polygon API key.",
    )
    parser.add_argument(
        "-k",
        "--key",
        type=str,
        dest="key",
        help="key",
    )
    if not other_args:
        console.print("For your API Key, visit: https://polygon.io")
        return

    if other_args and "-" not in other_args[0][0]:
        other_args.insert(0, "-k")
    ns_parser = self.parse_simple_args(parser, other_args)
    if ns_parser:
        self.status_dict["polygon"] = keys_model.set_polygon_key(
            key=ns_parser.key, persist=True, show_output=True
        )

ADVANCED

Important functions and classes

Base controller class

This BaseController class is inherited by all controllers on the terminal.

This class contains both important variables and methods that are common across all terminal controllers.

CHOICES_COMMON: List of common commands across all controllers

  • cls: clear screen
  • home: go back to the main root
  • about: allows to open our documentation directly on the menu or command
  • h, ? and help: display the help menu the user is in
  • q, quit and ..: go back to one menu above
  • exit: exit the platform
  • r and reset: reset the platform (reading code and settings again but going into the same state)
  • support: create a support request ticket

All of these variables have a call_FUNCTION associated with them.

Worthy methods to mention are:

  • load_class: Checks for an existing instance of the controller before creating a new one to speed up access to that menu.
  • custom_reset: Should be used by controllers that rely on a state variable - meant to be overridden. They should add the commands necessary to have the same data loaded.
  • print_help: Meant to be overridden by each controller
  • parse_input: Processes the string the user inputs into a list of actionable commands
  • switch: Acts upon the command action received
  • parse_known_args_and_warn: Parses the command with the - and -- flags and variables. Some built-in flags are:
    • export_allowed: Which can be set to _NO_EXPORT_, _EXPORT_ONLY_RAW_DATA_ALLOWED_, _EXPORT_ONLY_FIGURES_ALLOWED_ and _EXPORT_BOTH_RAW_DATA_AND_FIGURES_
    • raw: Displaying the data raw
    • limit: Number of rows to display
  • menu: Most important method. When a menu is executed, the way to call it is through stocks_menu.menu()

Default Data Sources

The document openbb_default.json contains all data sources that the terminal has access to and specifies the data source utilized by default for each command.

The convention is as follows:

{
  "stocks": {
    "search": [
      "FinanceDatabase"
    ],
    "quote": [
      "FinancialModelingPrep"
    ],
    "tob": [
      "CBOE"
    ],
    "candle": [],
    "codes": [
      "Polygon"
    ],
    "news": [
      "Feedparser",
      "NewsApi",
    ],
            ...

The way to interpret this file is by following the path to a data source, e.g.

  • stocks/search relies on FinanceDatabase
  • stocks/candle does not rely on any data source. This means that it relies on data that has been loaded before.
  • stocks/load relies on YahooFinance, AlphaVantage, Polygon or EODHD.
    • The order is important as the first data source is the one utilized by default.
  • stocks/options/unu relies on FDScanner.
  • stocks/options/exp relies on YahooFinance by default but Tradier and Nasdaq sources are allowed.

Note: The default data sources can be changed directly in the OpenBB Hub by the user and automatically synchronized with the terminal on login.

Export Data

In the _view.py files it is common having at the end of each function export_data being called. This typically looks like:

    export_data(
        export,
        os.path.dirname(os.path.abspath(__file__)),
        "pt",
        df_analyst_data,
        sheet_name,
        fig,
    )

Let's go into each of these arguments:

  • export corresponds to the type of file we are exporting.
    • If the user doesn't have anything selected, then this function doesn't do anything.
    • The user can export multiple files and even name the files.
    • The allowed type of files json,csv,xlsx for raw data and jpg,pdf,png,svg for figures depends on the export_allowed variable defined in parse_known_args_and_warn.
  • os.path.dirname(os.path.abspath(__file__)) corresponds to the directory path
    • This is important when export folder selected is the default because the data gets stored based on where it is called.
    • If this is called from a common folder, we can use os.path.dirname(os.path.abspath(__file__)).replace("common", "stocks") instead
  • "pt" corresponds to the name of the exported file (+ unique datetime) if the user doesn't provide one
  • df_analyst_data corresponds to the dataframe with data.
  • sheet_name corresponds to the name of the sheet in the excel file.
  • fig corresponds to the figure to be exported as an image or pdf.

If export_allowed=EXPORT_BOTH_RAW_DATA_AND_FIGURES in parse_known_args_and_warn, valid examples are:

  • cmd --export csv
  • cmd --export csv,png,jpg
  • cmd --export mydata.csv
  • cmd --export mydata.txt,alsomydata.csv,alsoalsomydata.png

Note that these files are saved on a location based on the environment variable: EXPORT_FOLDER_PATH. Which can be set in settings/export.

The default location is the exports folder and the data will be stored with the same organization of the terminal. But, if the user specifies the name of the file, then that will be dropped onto the folder as is with the datetime attached.

Queue and pipeline

The variable self.queue contains a list of all actions to be run on the platform. That is the reason why this variable is always passed as an argument to a new controller class and received back.

  self.queue = self.load_class(
      DarkPoolShortsController, self.ticker, self.start, self.stock, self.queue
  )

Example:

If a user is in the root of the terminal and runs:

stocks/load AAPL/dps/psi -l 90

The queue created becomes: self.queue = ["stocks", "load AAPL", "dps", "psi -l 90"]

And the user goes into the stocks menu and runs load AAPL. Then the queue is updated to self.queue = ["dps", "psi -l 90"]

At that point the user goes into the dps menu and runs the command psi with the argument -l 90 therefore displaying price vs short interest of the past 90 days.

Auto Completer

In order to help users with a powerful autocomplete, we have implemented our own (which can be found here).

The queue, discussed in the previous section Queue and pipeline, is expected to link together with the autocompletion in order to provide the user with the available options for each command. Here is an example of how it will look like:

2023 Apr 11, 11:41 (🦋) /stocks/dps/ $ psi
                                            --nyse
                                            --help
                                            -h
                                            --export
                                            --raw
                                            --limit
                                            -l
                                            --source

Where nyse, help, h, export, raw, limit, l and source are the available options for the psi command. Those are selectable using the arrow keys and the tab key.

The list of options for each command is automatically generated, if you're interested take a look at its implementation here.

To leverage this functionality, you need to add the following line to the top of the desired controller:

CHOICES_GENERATION = True

Here's an example of how to use it, on the forex controller:

class ForexController(BaseController):
    """Forex Controller class."""

    CHOICES_COMMANDS = [
        "fwd",
        "candle",
        "load",
        "quote",
    ]
    CHOICES_MENUS = [
        "forecast",
        "qa",
        "ta",
    ]
    RESOLUTION = ["i", "d", "w", "m"]

    PATH = "/forex/"
    FILE_PATH = os.path.join(os.path.dirname(__file__), "README.md")
    CHOICES_GENERATION = True

    def __init__(self, queue: Optional[List[str]] = None):
        """Construct Data."""
        super().__init__(queue)

        self.fx_pair = ""
        self.from_symbol = ""
        self.to_symbol = ""
        self.source = get_ordered_list_sources(f"{self.PATH}load")[0]
        self.data = pd.DataFrame()

        if session and get_current_user().preferences.USE_PROMPT_TOOLKIT:
            choices: dict = self.choices_default
            choices["load"].update({c: {} for c in FX_TICKERS})

            self.completer = NestedCompleter.from_nested_dict(choices)


    ...

In case the user is interested in a DYNAMIC list of options which changes based on user's state, then a class method must be defined.

The example below shows an excerpt from update_runtime_choices method in the options controller.

def update_runtime_choices(self):
        """Update runtime choices"""
        if session and get_current_user().preferences.USE_PROMPT_TOOLKIT:
            if not self.chain.empty:
                strike = set(self.chain["strike"])

                self.choices["hist"]["--strike"] = {str(c): {} for c in strike}
                self.choices["grhist"]["-s"] = "--strike"
                self.choices["grhist"]["--strike"] = {str(c): {} for c in strike}
                self.choices["grhist"]["-s"] = "--strike"
                self.choices["binom"]["--strike"] = {str(c): {} for c in strike}
                self.choices["binom"]["-s"] = "--strike"

This method should only be called when the user's state changes leads to the auto-complete not being accurate.

In this case, this method is called as soon as the user successfully loads a new ticker since the options expiry dates vary based on the ticker. Note that the completer is recreated from it.

Logging

A logging system is used to help tracking errors inside the OpenBBTerminal.

This is storing every logged message inside the following location :

$HOME/OpenBBUserData/logs

Where $HOME is the user home directory, for instance:

  • C:\Users\foo if you are in Windows and your name is foo
  • /home/bar/ if you are in macOS or Linux and your name is bar

The user can override this location using the settings key OPENBB_USER_DATA_DIRECTORY.

If you want to log a particular message inside a function you can do like so:

import logging

logger = logging.getLogger(__name__)

def your_function() -> pd.DataFrame:
    logger.info("Some log message with the level INFO")
    logger.warning("Some log message with the level WARNING")
    logger.fatal("Some log message with the level FATAL")

You can also use the decorator @log_start_end to automatically record a message every time a function starts and ends, like this:

import logging

from openbb_terminal.decorators import log_start_end

logger = logging.getLogger(__name__)

@log_start_end(log=logger)
def your_function() -> pd.DataFrame:
    pass

Note

: if you don't want your logs to be collected, you can set the OPENBB_LOG_COLLECT environment variable on your .env file to False.

Disclaimer: all the user paths, names, IPs, credentials and other sensitive information are anonymized, take a look at how we do it.

Internationalization

WORK IN PROGRESS - The menu can be internationalised BUT we do not support yet help commands-h internationalization.

In order to add support for a new language, the best approach is to:

  1. Copy-paste i18n/en.yml
  2. Rename that file to a short version of language you are translating to, e.g. i18n/pt.yml for portuguese
  3. Then just update the text on the right. E.g.
  stocks/NEWS: latest news of the company

becomes

  stocks/NEWS: mais recentes notícias da empresa

Note: To speed up translation, the team developed a script that uses Google translator API to help translating the entire en.yml document to the language of choice. Then the output still needs to be reviewed, but this can be a useful bootstrap.

This is the convention in use for creating a new key/value pair:

  • stocks/search - Under stocks context, short command search description on the help menu
  • stocks/SEARCH - Under stocks context, long command search description, when search -h
  • stocks/SEARCH_query - Under stocks context, query description when inquiring about search command with search -h
  • stocks/_ticker - Under stocks context, _ticker is used as a key of a parameter, and the displayed parameter description is given as value
  • crypto/dd/_tokenomics_ - Under crypto context and under dd menu, _tokenomics_ is used as a key of an additional information, and the displayed information is given as value

Settings

The majority of the settings used in the OpenBB Terminal are handled using Pydantic Dataclasses. Some examples are:

  1. SystemModel
  2. UserModel
  3. CredentialsModel
  4. ...

This means that the settings are pretty much validated and documented automatically, as well as centralized in a single place. This allows us to develop faster, efficiantly and with predictability. Depending on your use case you'll most likely need to interact with these dataclasses or expand them with new settings.

Disclaimer: avoid at all costs to use the os.environ or os.getenv methods to retrieve settings. Settings should be retrieved using the appropriate methods from the respective class.

Here is an example of accessing a setting:


from openbb_terminal.core.session.current_system import get_current_system

system = get_current_system()
system = get_current_system()
print(system.VERSION)

# 3.0.0

And here is an example of changing a setting:


from openbb_terminal.core.session.current_system import get_current_system

set_system_variable("TEST_MODE", True)

Write Code and Commit

At this stage it is assumed that you have already forked the project and are ready to start working.

Pre Commit Hooks

Git hook scripts are useful for identifying simple issues before submission to code review. We run our hooks on every commit to automatically point out issues in code such as missing semicolons, trailing whitespace, and debug statements. By pointing these issues out before code review, this allows a code reviewer to focus on the architecture of a change while not wasting time with trivial style nitpicks.

Install the pre-commit hooks by running: pre-commit install.

Coding

Although the Coding Guidelines section has been already explained, it is worth mentioning that if you want to be faster at developing a new feature, you may implement it first on a jupyter notebook and then carry it across to the terminal. This is mostly useful when the feature relies on scraping data from a website, or implementing a Neural Network model.

Git Process

  1. Create your Feature Branch, e.g. git checkout -b feature/AmazingFeature
  2. Check the files you have touched using git status
  3. Stage the files you want to commit, e.g. git add openbb_terminal/stocks/stocks_controller.py openbb_terminal/stocks/stocks_helper.py. Note: DON'T add config_terminal.py or .env files with personal information, or even feature_flags.py which is user-dependent.
  4. Write a concise commit message under 50 characters, e.g. git commit -m "meaningful commit message". If your PR solves an issue raised by a user, you may specify such issue by adding #ISSUE_NUMBER to the commit message, so that these get linked. Note: If you installed pre-commit hooks and one of the formatters re-formats your code, you'll need to go back to step 3 to add these.

Branch Naming Conventions

The accepted branch naming conventions are:

  • feature/feature-name
  • hotfix/hotfix-name
  • release/2.1.0 or release/2.1.0rc0.
  • bugfix/bugfix-name

All feature/feature-name related branches can only have PRs pointing to develop branch. hotfix/hotfix-name and release/* branches can only have PRs pointing to main branch.

Installers

When implementing a new feature or fixing something within the codebase, it is necessary to ensure that it is working appropriately on the terminal. However, it is equally as important to ensure that new features or fixes work on the installer terminal too. This is because a large portion of users utilize the installer to use OpenBB Terminal. More information on how to build an installer can be found here.