Commit Graph

24 Commits

Author SHA1 Message Date
Danglewood
49cd447674 merge branch develop 2026-05-25 22:43:18 -07:00
Danglewood
3735afca3f backport OptionsChainsData serialize schema fix from v5 branch (#7511) 2026-05-26 05:33:14 +00:00
Danglewood
976a4fd098 fix test drift 2026-05-25 22:26:21 -07:00
Danglewood
172131ea5e [V5] openbb-core: 100% Test Coverage (#7484)
* increase test coverage to exercise 100% of the testable code path in openbb-core

* model_fields syntax cleanup

* set the environment variable for CI runs

* run codeql on v5 branch

* more CI env setup

* cleanup redundant files

* ruff

* dedupe workflow trigger

* some windows coverage missing

* ubuntu integration test job

* more windows coverage

* test_reference_generator

* test_docstring_generator

* remove line no references from test function docstrings

* clean useless comments from test files

* fix polars type check

* fix test_obbject

* fix test_obbject
2026-04-30 11:49:41 -07:00
Danglewood
7041362aa1 [Feature] OpenBB V5: Initial openbb-core Refactor (#7482)
* initial refactor

* ruff does not dictate style

* unresolved imports in test action

* fix general linting

* resolve pandas imports for actions run

* httpx in test workflow

* pytest-asyncio in test workflow

* pandas.Index(...)

* max version for windows CI is 3.14

* remove temp refactor files that shouldn't have been committed:

* use ruff format instead of black

* formatting and typing

* tab space helper

* test fixtures..

* ty check
2026-04-28 17:11:43 -07:00
John Seong
616c54d79e feat(platform): standardize exchange inputs with ISO 10383 MIC codes (#7351)
* feat(platform): standardize exchange inputs with ISO 10383 MIC codes

Add Exchange type in provider utils that standardizes exchange inputs across
providers using ISO 10383 Market Identifier Codes (MIC).

Similar to the Country type from PR #7333, this allows providers to accept:
- ISO 10383 MIC codes (e.g., 'XNAS', 'XNYS')
- Exchange acronyms (e.g., 'NASDAQ', 'NYSE')
- Full exchange names (e.g., 'New York Stock Exchange')
- lower_snake_case names (e.g., 'new_york_stock_exchange')

The Exchange class inherits from str (storing the MIC code) while providing
access to mic, acronym, name, country, and city properties.

Example usage in FMP equity screener demonstrates provider integration pattern:
- exchange: Exchange | None with validator for provider-specific validation
- Maps Exchange.acronym to FMP's expected lowercase exchange codes

Addresses part of #6969

* refactor(platform): use ExchangeData TypedDict for proper typing

- Add ExchangeData TypedDict with explicit field types (consistent with CountryData)
- Update function signatures and class attributes to use ExchangeData
- Add _last_updated metadata to exchange_data.json
- Add Wikipedia reference URL to docstring

Aligns Exchange implementation with Country pattern from PR #7352.

* fix: add missing major exchanges including Cboe (XCBO, XCBT)

* refactor: expand to full ISO 10383 MIC list (2,272 active MICs)

Address review feedback to include all active MICs, not just curated majors.

- Expand exchange_data.json from 66 to 2,272 active MICs (operating + segments)
- Remove country and city fields (out of scope for this PR)
- Add scripts/update_exchange_data.py to regenerate from official ISO 20022
  CSV source using stdlib only (no external dependencies)
- Use first-write-wins lookup with operating MICs ordered before segments
  to resolve acronym collisions (e.g. LSE -> XLON, not XPVT)
- Update tests to reflect simplified data model

* feat: add city, country, and website fields to exchange data

* refactor: move update_exchange_data.py to provider utils

* style: fix black formatting in update_exchange_data.py

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
2026-02-25 19:17:02 +00:00
John Seong
e6a3879e98 feat(platform): add membership groups to Country type (#7352)
* feat(platform): add membership groups to Country type

Add membership groups (G7, G20, EU, NATO, OECD, OPEC, BRICS) to the Country
type, as suggested by maintainer in PR #7333.

Changes:
- Updated country_data.json with 'groups' field for each country
- Added Country.groups property returning list of group memberships
- Added Country.is_member_of(group) method for membership checks
- Added comprehensive tests for group functionality

This enables filtering and analysis by economic/political groupings:
  c = Country('Germany')
  c.groups        # ['G7', 'G20', 'EU', 'NATO', 'OECD']
  c.is_member_of('EU')  # True

Follow-up to #7333

* fix(platform): add Indonesia to BRICS (joined Jan 2025)

* refactor(platform): use TypedDict for country data typing

- Replace dict[str, Any] with CountryData TypedDict for proper type inference
- Add official data sources to module docstring
- Add _sources metadata section to country_data.json with references

Sources:
- G7: g7italy.it
- G20: g20.org
- EU: european-union.europa.eu
- NATO: nato.int
- OECD: oecd.org
- OPEC: opec.org
- BRICS: brics-russia2024.ru

* fix(platform): update group membership reference URLs

Replace broken/inaccessible URLs with stable Wikipedia references:
- G7, G20, OECD, OPEC, BRICS → Wikipedia (stable, well-maintained)
- EU, NATO → Updated to current official URLs

* feat: add developer script to regenerate country_data.json from sources

* fix: resolve codespell lint errors in update_country_data.py

* fix: resolve black/ruff lint issues in update_country_data.py

* feat(core): normalize unicode country names for ASCII input matching

* fix(core): remove Angola from OPEC and Bangladesh from NATO

Angola withdrew from OPEC effective Jan 2024. The OPEC scraper falls
back to a hardcoded list which still included AO — removed it.

Bangladesh was incorrectly scraped as a NATO member due to the scraper
iterating all wikitables on the page. Now filters tables by their
nearest h2 section header to only process member state tables.

* fix(core): normalize country names to ASCII in update script

Strip diacritical marks at generation time so country_data.json
uses ASCII-only names (e.g., Turkiye, Curacao, Cote d'Ivoire).

* Update openbb_platform/core/openbb_core/provider/utils/update_country_data.py

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* fix imports

* fix: resolve mypy type narrowing for expected counts

* restore reference.json

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
2026-02-25 19:16:36 +00:00
John Seong
f0c3742729 feat(platform): standardize country inputs with ISO 3166 Country type (#7333)
* feat(platform): standardize country inputs using pydantic-extra-types

Add CountryParam type that accepts:
- ISO 3166-1 alpha-2 codes (e.g., 'US', 'us')
- Full country names (e.g., 'United States')
- lower_snake_case names (e.g., 'united_states')

Uses pydantic-extra-types for programmatic ISO validation instead of
hardcoded mappings. Serializes to lowercase alpha-2 code by default.

Example usage in FMP equity screener:
- country: CountryParam | None

Partially addresses #6969

* refactor: use Country class inheriting from str for type checker compatibility

- Replace Annotated CountryParam with Country(str) class per maintainer feedback
- Store full pycountry object for access to alpha_2, alpha_3, name, numeric
- Use pycountry directly instead of pydantic-extra-types
- Add support for alpha-3 codes (e.g., 'USA')
- Update tests to cover new functionality

* Replace pycountry with static JSON country data

- Add country_data.json with ISO 3166-1 data (20KB vs 6MB pycountry)
- Update Country class to use static JSON lookup
- Remove pycountry dependency from pyproject.toml
- No decompression needed, data loaded once at import

* fix(lint): D402 docstring should start with verb

* fix(lint): work around pydocstyle D402 false positive

When a method is named `name` and `.pydocstyle.ini` uses `match = .*\.py`,
pydocstyle incorrectly triggers D402 for any docstring containing the word
"name" anywhere - not just at the start as the rule intends.

Changed docstring from "Return the full country name..." to
"The country's official designation..." to avoid the false positive.

See: https://github.com/PyCQA/pydocstyle/issues/375

* fix(lint): resolve pylint warnings in country_utils

* fix(lint): use ternary for ruff SIM108 in country_utils

* fix(lint): format ternary for black line length

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
2026-02-09 21:11:11 +00:00
Danglewood
06041a8b3a [Feature] Add Svensson Nominal Yield Curve From Federal Reserve (#7338)
* add svensson yield curve, cleanup fomc documents endpoint, make provider a standalone extension, move apps from platform-api into extension, and update some models to add prefix/suffix in widget definitions

* black

* do what codeQL says

* black again

* use get_args instead of Literal.__args__

* linting

* test items

* grammar police

* test functions need to start with test_*

* test params

* add total factor productivity from sf fed

* openpyxl required for Excel file handling

* grammar police

* inflation expectations

* Update pr-description check action to properly handle quotes

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
2026-02-02 16:05:57 +00:00
Danglewood
e0868c73d1 [BugFix] Remove Polygon - No Longer Exists and Can't Support (#7337)
* remove polygon - no longer exists and can't support

* update metadata asset

* dev_install.py
2026-01-27 15:48:15 +00:00
Danglewood
9f0d592839 [Feature] Remove Python 3.9 (#7235)
* remove python 3.9 support and code

* black

* more cli lint

* more linting

* more lint

* fix for tests

* docstring grammar police

* add lock to to build function to avoid async import race conditions

* grammar police

* lots more linting

* relock
2025-10-10 23:16:16 +00:00
Danglewood
d75b22f819 [Feature] Add SSL Certificate Context And Setting For Async/Sync HTTP Requests (#6976)
* add SSL certificate context and setting

* codespell

* linting

* cleanup section

* allow defining uvicorn.run kwargs in system_settings

* refactor to return session objects

* hub service rename it as _request_session

* typing

* more linting

* pylint

* missing __init__ files

* fix test_hub_service

* missing init files..?

* shouldn't need an init file there..

* now for some random linting

* maybe need that __init__ after all...?

* fix test_make_request

* relative import in conftest..?

* issue can't be from missing init files..

* revert the inits

* small cleanup

* pass the kwargs to the async session object on init for use outside of amake_request

* try something...

* try something else..

* turn on verbosity for errors..

* try something else...

* undo test change

* Update lock files

* undo another test check thing

* Revert "Update lock files"

This reverts commit 714df64416.

* try with just the platform lock

* unexpected argument

* do a dummy check to return the session object if it is passed in the kwargs

* also dummy check for TCPConnector instance

* bit more cleanup

* no implementation of server_hostname

* Move setting session to PosthogHandler init

* add comments

* review items

* update PythonSettings model definitions and annotations

* fix latest yfinance breaking changes

* unused import

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
2025-01-21 01:39:32 +00:00
montezdesousa
01a71a8995 [Feature] Set provider fallback based on credentials (#6446)
* feat: allow provider fallback based on credentials

* fix: rename prop and raise error

* fix: raise if all providers fail

* fix: remove dead line

* fix: rename defaults field to commands

* rename key and update docs

* fix: rename some stuff

* fix bug

* fix: mypy

* fix: unittests

* docstring

* fix: provider field description

* fix: error messages

* fix: msg

* feat: update defaults class

* feat: update defaults class

* unit tests

* fix test

* msg

* fix: add website documentation

* fix: detailed error message

* update core ruff version

* fix test

* rebuild

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
2024-05-29 14:22:14 +00:00
montezdesousa
b8d1846d46 [BugFix] - Provider is added to every response item (#6305)
* Exclude provider and don't model_dump

* still missing docstrings

* undo package changes

* minor fix

* mypy

* minor fix

* cleaner

* private var

* docstring

* docstrings

* add package builder tests

* ruff

* rename

* rename

* update tests

* minor fix

* fix test

* handle docstring edge cases

* test
2024-04-12 12:08:55 +00:00
Igor Radovanovic
a255fa52a0 [Enhancement] - PyDocstyle Compliant (#6284)
* PyDocstyle Compliant batch 1

* Batch 2

* Batch 3

* Batch 4

* Batch 5

* Batch 6

* Batch 7

* Linting

* MyPy

* mypy things

* black

* more mypy

* black

* fix that

* black again

* undo that

* Fix error message

* pylint

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
2024-04-09 13:30:29 +00:00
montezdesousa
23135abf1b [BugFix] - Bring back mypy (#6242)
* first batch

* another

* another

* another batch unyped defs

* final untyped defs

* update yml

* pylint

* remove unecessary __init__.py files

* Revert "remove unecessary __init__.py files"

This reverts commit 5977eccfee.

* fix company_news typing

* rename func

* fix tests

* tests
2024-03-22 17:53:40 +00:00
Pratyush Shukla
13283fbfce CI listing quick fix (#6002)
* BIGGGG LINTING

* fixing lints

* fixing lints

* black

* very ruff

* no export

* fix hedge_view again

* lints

* platform lints

* lints

* black

* black it @hjoaquim

* fix some more linting

---------

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
2024-01-26 17:08:42 +00:00
teh_coderer
da7886d0a7 Make Pro Fetchers async by replacing ThreadPoolExecutor with asyncio.gather (#5822)
* init

* Update helpers.py

* filter URL on raise

* polygon, fmp done, update cassettes

* updates

* cleanup

* revert tests for now

* Update discovery_filings.py

* Update helpers.py

* add `async` overload to `Fetcher.extract_data`

* update merged fred models

* tests, improve fred series

* Update insider_trading.py

* insider

* tests

* intrinio

* Update equity_historical.py

* implement abstract async extract on Fetcher, with simple switch

* cleanup

* Update helpers.py

* Update helpers.py

* fix fmp historical deprecated urls

* Revert "fix fmp historical deprecated urls"

This reverts commit 1a760a2473.

* fix fmp `index.market` 1day url missing query params

* Update test_fmp_market_indices_fetcher.yaml

* field fixes

* `extract_data_async` -> `aextract_data`, `async_request` -> `amake_request`

* `async_requests` -> `amake_requests`

* Update company_news.py

* add sort to index `integration` params

* add `client.py` tests

* Update equity_search.py

* Add lost v in platform api version string

* Resolve linter errors here and there

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
2023-12-08 14:41:37 +00:00
Henrique Joaquim
ff00312ec4 Hotfix/unskip tests (#5851)
* support for nested structures

* unskip extension map test

* unskip fred tests

* stockgrid tests

* adding pytest-assyncio

* fix query executor test

* package builder tests

* test query test with the asyncio marker

* asyncio marker

* fix extension map test: tests run with all extensions installed

* adding missing tests on user service

* fix update default test

* skipping allows version for non required extensions
2023-12-04 15:45:46 +00:00
Pratyush Shukla
ce14ee612f Fix nasdaq top retail working without a key resulting in an error (#5811)
* raise error if no api key found

* check for empty strings

* black

* fix test

* less verbose

---------

Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
2023-11-29 14:16:31 +00:00
montezdesousa
ed2fc04814 replace provider error by openbb (#5812) 2023-11-28 19:19:21 +00:00
montezdesousa
bb5e76ce5c Feature/general fixes (#5786)
* fix mypy

* import helper from source

* improve missing api key msg

* fix

* remove dynamic default

* rename rest_api_settings file

* add comment async/sync

* fix bug

* Revert "fix bug"

This reverts commit 7d9a0d2749.

* fix bug
2023-11-23 16:16:56 +00:00
teh_coderer
bc2fa9b3e1 Feature/platform async (#5763)
* init

* updates

* Update helpers.py

* tests

* Update helpers.py

* Update command_runner.py

* Update helpers.py

* deps

* Update helpers.py

* only check `async` if model is specified, revert `qa`, `ta`, and `econometrics`

* merge issues

* Update command_runner.py

* remove typing and type: ignore

* fix helpers typing

* add aiohttp to core deps, remove from charting

* remove @overload

* ruff

* import helper from source

* helpers typing+pylint

* black

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
Co-authored-by: Henrique Joaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
2023-11-23 13:01:56 +00:00
Henrique Joaquim
4400cd9a8f Merge provider and core into core (#5774)
* small changes - moving provider inside the core

* uptade readme

* lint

* remove old provider readme

* leftovers

* refactor website examples

* ruff

* fix nox

* relock

* fix fails_to_deliver headers

* fix for tests that have renamed endpoints

* fix discovery

* removed test for renamed endpoint

* fix broken syntax - test_search_financial_attributes

* fix cash/income/balance

* fix derivatives tests

* fix etf tests

* fix news tests

* fix regulators

* black

* wrong test param

* sort equity gainers/losers by percent

* platform markdown generation import

* Hotfix/fix int tests core merge (#5779)

* removing tests on old endpoint

* removing visualization on old endpoint

* fixing equity tests

* fixing examples

* fix news tests

* fix index test

* example

* add comments

* example

---------

Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>

---------

Co-authored-by: montezdesousa <79287829+montezdesousa@users.noreply.github.com>
Co-authored-by: Diogo Sousa <montezdesousa@gmail.com>
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
2023-11-22 13:48:01 +00:00