mirror of
https://github.com/OpenBB-finance/OpenBB.git
synced 2026-10-30 14:02:06 +00:00
* [Feature] OpenBB V5: Initial `openbb-core` Refactor (#7482)
* initial refactor
* ruff does not dictate style
* unresolved imports in test action
* fix general linting
* resolve pandas imports for actions run
* httpx in test workflow
* pytest-asyncio in test workflow
* pandas.Index(...)
* max version for windows CI is 3.14
* remove temp refactor files that shouldn't have been committed:
* use ruff format instead of black
* formatting and typing
* tab space helper
* test fixtures..
* ty check
* also run integration test in CI
* remove that annoying labeler action
* [V5] `openbb-core`: 100% Test Coverage (#7484)
* increase test coverage to exercise 100% of the testable code path in openbb-core
* model_fields syntax cleanup
* set the environment variable for CI runs
* run codeql on v5 branch
* more CI env setup
* cleanup redundant files
* ruff
* dedupe workflow trigger
* some windows coverage missing
* ubuntu integration test job
* more windows coverage
* test_reference_generator
* test_docstring_generator
* remove line no references from test function docstrings
* clean useless comments from test files
* fix polars type check
* fix test_obbject
* fix test_obbject
* package builder fixes
* charting pyproject version
* fix docstring generator for nested models
* missing coverage
* [V5] `openbb-core`: Add Backwards-Compatible Layered Config System (openbb.toml) (#7492)
* add backwards-compatible layered config system
* fix path reference in windows test
* need python version-aware handling in docstring generator
* update for deprecated pydantic syntax
* ruff UP007
* test_system_settings
* remove v4 platform workflow
* [V5] Refactor `openbb-oecd` for full SDMX 2 API Exposure and V5 Packaging (#7413)
* stash some changes
* add more robust testing
* mypy
* point PR at V5
* introduce spec file
* codespell
* test fix
* fix workflow environment setup
* fix workflow environment setup
* fix workflow environment setup
* add pyyaml to dependencies
* split lint jobs
* fix workflow environment setup
* fix workflow environment setup
* workflow env setup
* workflow env setup
* clean up code comments
* add auth hook entrypoints
* codespell
* add codegen feature
* codespell
* move _unpack into dispatchers for consistency with codegen packages
* surface nested models in the response
* fix missing coverage in CI
* socrata updates
* test fix
* detect plotly output
* add --include and --exclude flags from generate-extension command
* refactor openbb-platform-api for V5 operations and spec file handling
* delete poetry.lock
* add test workflow
* add temp branch id for test workflow
* add httpx to dev dependencies
* add a conftest.py
* tweak conftest
* tweak conftest
* python 3.10 issues
* more python 3.10
* more python 3.10
* more python 3.10
* more python 3.10
* fix url template resolution
* multi-spec support
* fix a windows path issue
* merge branch develop
* table builder, integration tests, and more
* linting
* merge branch develop
* bit of cleanup
* some more updates, cleanup, and apps.json
* codespell
* black
* integration test params and linting
* mypy
* pylint
* black
* more pylint
* recapture teest cassettes
* black again
* more linting
* table presentation choices
* black
* fix merge conflict
* split up metadata module and reformat cache to use msgpack
* mypy
* regenerate lock file
* use json and include dependencies to make openbb-oecd a standalone install
* pylint
* more linting
* serve metadata as dependency instead of assign and initialize
* fix integration_tests_testers to not count dependency injections as paramters
* add assets to git ignore
* ruff
* v5 refactor and publish workflow
* test coverage and ty fixes
* codespell
* update lock file
* workflow setup
* ty check
* workflow setup
* Scope PR to OECD changes only
Restore non-OECD files in the diff against v5 to their v5 state. The
branch was originally created off feature/cli-2 and inherited 186 files
of unrelated changes (cli, agent_server, mcp_server, platform_api,
core/, .github/workflows/* not oecd). Those have nothing to do with the
openbb-oecd v5 refactor and were polluting the PR diff.
* scope cleanup: include leaked v5-resident files (general-linting workflow, platform_api test fixtures)
---------
Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>
* fix docstring generator and body param unwrapping
* project ty.toml file
* unpack POST parameters from params: Model
* remove comment chatter
* [V5] `openbb-oecd`: Rename `oecd_utils` as `oecd` (#7509)
* rename oecd_utils to oecd and populate all models there when openbb-economy is not installed
* linting
* bump dependencies
* fix test drift
* linter
* FieldInfo import
* [V5] `openbb-imf`: Refactor for V5 (#7508)
* refactor imf extension for V5
* codespell
* fix some python 3.10 items
* return self on copy of metadata object
* [V5] `openbb-news`: Refactor News for V5 and Add Configurable RSS Endpoint (#7507)
* refactor news for v5 and add rss endpoint
* remove poetry.lock
---------
Co-authored-by: deeleeramone <>
* [V5] `openbb-quantitative`: Refactor for V5 (#7506)
* refactor quantitative for v5
* add factors endpoint and add a few charts
---------
Co-authored-by: deeleeramone <>
* [V5] `openbb-technical`: Refactor for V5 & Add More Quick Win Endpoints (#7502)
* refactor openbb-technical for v5
* forward refs
* linting
* no python 3.15 in test matrix
* reame
* ty
* move all params to body
---------
Co-authored-by: deeleeramone <>
* remove stockgrid provider (#7486)
* [V5] remove wsj provider and related code (#7485)
* remove wsj provider and related code
* purge from metadata
* actions
* actions
* openbb-core publish workflow
* delete biztoc (#7523)
* [V5] Remove `openbb-alpha-vantage` (#7522)
* delete alpha vantage
* more files
---------
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
* [V5] Remove `openbb-benzinga` (#7521)
* delete benzinga
* more references
---------
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
* [V5] `openbb-bls`: Refactor for V5 + Add Many Endpoints (#7518)
* refactor bls for v5, add 100 widgets
* some CI fixes
* no python 3.15
* fix missing dev group in pyproject.toml
* more pyproject fallout
* action
* refactor econometrics for v5 (#7505)
Co-authored-by: deeleeramone <>
* [V5] `openbb-famafrench`: Refactor for V5 (#7503)
* refactor famafrench for v5
* codespell
* finishing touches
---------
Co-authored-by: deeleeramone <>
* [V5] CLI 2.0 - Pluggable Backends, Non-TTY Default, HTTP Dispatcher, Spec Files + Codegen (#7433)
* stash some changes
* add more robust testing
* mypy
* point PR at V5
* introduce spec file
* codespell
* test fix
* fix workflow environment setup
* fix workflow environment setup
* fix workflow environment setup
* add pyyaml to dependencies
* split lint jobs
* fix workflow environment setup
* fix workflow environment setup
* workflow env setup
* workflow env setup
* clean up code comments
* add auth hook entrypoints
* codespell
* add codegen feature
* codespell
* move _unpack into dispatchers for consistency with codegen packages
* surface nested models in the response
* fix missing coverage in CI
* socrata updates
* test fix
* detect plotly output
* add --include and --exclude flags from generate-extension command
* cap test matrix at python 3.14
* no useless comments
* platform controller command description split
* merge URL overloads from path params
* exclude none and unset from model dump
---------
Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>
* general-linting
* downgrade aiohttp in devtools for vcrpy
* some ty items and add requests-types stubs for ty in devtools
* markdown linting
* remove integration tests from extensions that were moved into the famafrench provider
* [V5] `openbb-congress-gov`: Refactor for V5 + Convert To Using Bulk Files Instead of `api.congress.gov`. (#7528)
* refactor congress-gov for v5 and use bulk files instead of API with credentials
* integration tests
* apps tab name
* convert caching to sql
* codespell
* [V5] [BugFix] Fix for Breaking Changes in FastAPI 0.137+ (#7537)
* fix route introspection for fastapi 0.137
* recapture oeccd gdp forecast test cassette
---------
Co-authored-by: deeleeramone <>
* [V5] `openbb-platform-api`: Refactor for V5 Spec, Config, Packaging, Test Coverage (#7494)
* stash some changes
* add more robust testing
* mypy
* point PR at V5
* introduce spec file
* codespell
* test fix
* fix workflow environment setup
* fix workflow environment setup
* fix workflow environment setup
* add pyyaml to dependencies
* split lint jobs
* fix workflow environment setup
* fix workflow environment setup
* workflow env setup
* workflow env setup
* clean up code comments
* add auth hook entrypoints
* codespell
* add codegen feature
* codespell
* move _unpack into dispatchers for consistency with codegen packages
* surface nested models in the response
* fix missing coverage in CI
* socrata updates
* test fix
* detect plotly output
* add --include and --exclude flags from generate-extension command
* refactor openbb-platform-api for V5 operations and spec file handling
* delete poetry.lock
* add test workflow
* add temp branch id for test workflow
* add httpx to dev dependencies
* add a conftest.py
* tweak conftest
* tweak conftest
* python 3.10 issues
* more python 3.10
* more python 3.10
* more python 3.10
* more python 3.10
* fix url template resolution
* multi-spec support
* fix a windows path issue
* cap test matrix at python 3.14
* no 3.15 runners for platform-api
* no useless comments
* platform controller command description split
* merge URL overloads from path params
* exclude none and unset from model dump
* markdown lint
* ignore get/post endpoints that are return SSE
* fix remaining fastapi 0.137 fallout
---------
Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>
* add full support for AsyncIterator response type for streaming output (#7530)
* [V5] Refactor Shared Pass Through Router Extensions For Hatchling (#7524)
* refactor shared pass-through routers for hatchling and ruff + ty
* add publish workflows
* [V5] `openbb-charting`: Refactor for V5 and Decouple (#7545)
* refactor charting for v5 patterns and decouple entirely
* coveragerc
* lint and tests
* fix tests
* configure dev group with openbb-technical
* ty
* fix cli test
* remove 3.15 from test matrix - dependencies are unstable
---------
Co-authored-by: deeleeramone <>
* fix merge widgets
* workflow cleanup"
* fixes for ty
* fogot that file
* general lint setup
* consolidate lint job
* remove seeking alpha provider (#7570)
* remove tradier provider (#7573)
* remove tiingo provider (#7574)
* remove intrinio provider (#7575)
Remove the openbb-intrinio provider extension from the repository.
Reason: proprietary vendor with no internal access to any endpoint;
cannot build, develop or maintain.
* remove yfinance provider from repo (#7576)
* fix(cli): resolve modular OpenAPI references (#7586)
Fixes #7585
* [V5] `openbb-cli`: Modular OpenAPI Refs (#7589)
* follow up to #7585
* add another codegen test
* unused ty ignore
* Update generated tests to get python parameter names instead of raw API-provider parameter names. (#7597)
* Update the record-vs-envelope heuristic in the payload processing code to not treat a record that happens to have a list datatype as an entire envelope. (#7598)
* Update class generation code to handle empty data that php-style serialization emits as []. (#7599)
* fix ty check failures exposed by newer version
* add drugs.com and brutalist as news rss feeds (#7602)
Co-authored-by: deeleeramone <>
* codespell
* missing test coverage that appeared in famafrench..
* plotly 7 handling for mapbox namespace change
* remove finviz (#7577)
* remove trading economics provider (#7578)
* remove openbb-multpl from repository (#7610)
* add httpx2 to devtools for starlette test client
* restore test coverage to 100% in openbb_core
* remove fmp provider extension from repository (#7665)
* unpin ruff
* fix charting conftest and missing coverage in 3.10
* add charting coverage
* fix new ty issues
* fix tag grouping in swagger docs
* [V5] `openbb-ecb`: Refactor for V5 and Expand Coverage (#7560)
* refactor openbb-ecb for v5 and expand coverage
* some updates
* tests
* workflow
* remove 3.15 from test matrix
* some more table schema updates
* remove some html chrome on embedded articles
---------
Co-authored-by: deeleeramone <>
* [V5] `openbb-federal-reserve`: Refactor for V5 + FFIEC & Regional Fed Routers (#7561)
* refactor for v5
* add workflows
* fix bhcpr schema
* fix pyproject.toml
* fix test
* codespell
* ty
* fix pyproject
* fix ci environment
* fix ci environment
* convert documents endpoints to use fedinprint service
* codeql fix
* fix ny publications and add ny nowcast
* [V5] `openbb-eia`: Refactor for V5 + Add Full API Implementation (#7584)
* eia refactor for v5
* add test and publish workflows
* data browser state
* fix workflows
* more workflow test items
* temp files in tests
* more fixes
* more browsers
* fix codeql issues
* fix mer state and ensure multi-user capability
* uv lock
* table bridge
* fix electricity browser state
* convert HTML widgets to iFrame
* fix recursion error in workspace mcp
* [V5] `openbb-government-us`: Refactor for V5 + Move `uscongress` (#7604)
* v5 government_us provider
* workflow files
* ty and add xlrd to pyproject
* shrink test cassettes
* dev dependencies
* wheel packaging and more tests
* python 3.10 thing..?
* fix test
* default state for multiselect dropdowns
* replace congress html widgets with iframes
* fallback to congress api for congress before 108
* ensure member passage voting count populates
* [V5] Feature: Add JODI World Oil & Gas Database (#7593)
* add the JODI oil and gas database as a provider/router extension
* lint fixes
* add grouping to apps.json
---------
Co-authored-by: deeleeramone <>
* [V5] `openbb-cftc`: Refactor for V5 + Add OIS, Forward Curve, and FX Trade Analysis (#7615)
* refactor openbb-cftc for v5
* add SHIBOR to fixings
---------
Co-authored-by: deeleeramone <>
* fix for plotly 7 breaking change to schemas
* [V5] `openbb-tmx`: Refactor extension for V5 (#7668)
* refactor tmx provider for v5
* fix tests..
* fix tests stack trace exposure
* fix workflow setup
* [V5] `openbb-nasdaq`: Refactor provider for V5. (#7670)
* refactor nasdaq provider for v5
* add workflows
* ty fix
* [V5] `openbb-fred`: Refactor as V5 extension. (#7669)
* refactor fred extension for v5
* fix markdown MD037 lint error in README
* codespell
* [V5] `openbb-cboe`: Refactor for V5 + Add Payoff Diagrams (#7617)
* refactor cboe for v5
* use zoneinfo instead
* fix test
* some fixes related to the cache access
* fix a couple of tests
* fix the test coverage
* [V5] `openbb-deribit`: Refactor provider for V5 (#7674)
* refactor openbb-deribit for v5
* restructure the optimizer around the published combos data and add integration test suites
* fix markdown lint errors
* remove openbb-econdb provider (#7675)
* codespell
* [V5] `openbb-sec`: Refactor for V5 (#7525)
* refactor openbb-sec for v5
* linting
* ty
* normalize compare_company_facts
* add workflows
* python 3.10 fix
* add more thorough url validation and tests
* ty check
* ty for python 3.10
* reconfigure with apps
* temp pyproject overrride
* pyproject
* pyproject
* pyproject
* codespell
* pyproject
* fix uk schema file links in schema_files
* test coverage
* test_cli
* port #7547 into this pr
* no hard coded prefix
* improve filing rendering capabilities
* fix xml text detection
* make codeql happy
* missed a file in the commit
* test coverage
* codespell
* fix flakey test
* some updates
* management_ownership --> management_profiles
* proxy statement markdown
* ruff
* fix test coverage
* more tests
* test coverage
* fix lint errors
* Add SEC adviser search, profile, and universe routes (#7559)
* Add SEC institutional data fetchers
* Add SEC filing index and Form D routes
* Add SEC investment adviser people data
* Expose SEC adviser and institutional routes
* fix(sec): address adviser review feedback
* refactor(sec): narrow adviser support to IAPD search
* feat(sec): add adviser firm and individual searches
* feat(sec): add adviser profile drill-down
* fix: repair pre-commit configuration
* feat(sec): add adviser universe route
* feat(sec): expose complete Form ADV data
---------
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
* fix test coverage
---------
Co-authored-by: deeleeramone <>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
* [V5] `openbb-finra`: Refactor for V5 (#7676)
* refactor finra provider for v5
* add workflows
* fix tests
* fix lint errors
* [V5] Update Repository License to Apache 2.0 (#7677)
* change license to Apache 2.0 for V5
* codespell
* bump devtools and repoint pyproject.toml
* Enterprise Flask Migration Toolkit (#7264)
* feat: Add Flask-to-OpenBB enterprise migration toolkit
- Implement Router.from_flask() for zero-touch legacy API conversion
- Enable 4-hour migration vs 6-month traditional rewrite (80% cost reduction)
- Generate FastAPI endpoints from Flask routes with automatic type validation
- Support enterprise consulting services and custom integration workflows
Demonstrates OpenBB's competitive advantage for institutional client migrations.
Complements existing Workspace applications like Open Portfolio suite.
* chore: Resolve .gitignore conflict and add common ignores
* refactor: implement Phase 1 Flask integration architecture- Move Flask conversion logic to openbb_core.app.utils.flask- Add Flask detection to extension_loader - Integrate with OpenBB's core extension system- Add Flask dependency validation with sys.modules checkingAddresses @deeleeramone's architectural feedback for native OpenBB integration.
* docs: add Flask direct entry point demos- demo_direct_flask_entry_point.py: shows direct Flask instance reference- example_pyproject_flask_entry.toml: demonstrates pyproject.toml configurationSupports new architecture eliminating wrapper extensions.
* Implement Phase 1 Flask adapter per code review feedback
- Minimal working implementation with Flask detection and router creation
- Removes complex code generation logic for focused Phase 1 delivery
- Establishes foundation for Phase 2 route conversion features
* Fix Flask integration: Replace missing FlaskToOpenBBAdapter with WSGIMiddleware
- Remove FlaskToOpenBBAdapter which was not implemented
- Use FastAPI's WSGIMiddleware to mount Flask apps directly
- Update extension_loader to use returned Router object
- Convert utils.py to utils/__init__.py to support flask submodule
- Addresses PR review comment about unhandled AttributeError
* Address PR review feedback: fix Flask imports and remove unused files
- Revert .gitignore to upstream/develop
- Remove AI-generated demo/example files and flask_adapter extension
- Move Flask/werkzeug imports inside conditional blocks to avoid ImportError
- Use lazy imports via __getattr__ in utils/flask/__init__.py
- Add Flask tests with @pytest.mark.skipif markers
* Force APIRouter.include_router to propagate Mount paths instead of dropping them. (#1)
This change ensures that mounted ASGI applications (such as Flask apps wrapped in WSGIMiddleware) are not dropped when `APIRouter.include_router` is called.
Previously, while the initial load in `ExtensionLoader` worked as expected, the `Mount` objects were ignored by FastAPI's default inclusion logic. This caused the mounted elements to be lost during the final application assembly, resulting in 404 errors for the Flask routes.
This fix overrides the inclusion logic to manually propagate `Mount` objects downstream.
* feat(flask): Add metadata layer for OpenAPI documentation
Enhance Flask-to-OpenBB converter with structured docstring parsing:
- Parse Google-style docstrings (summary, description, params)
- Extract parameter descriptions from Args section
- Pass metadata to Router.command() for rich OpenAPI docs
This addresses PR feedback to provide tangible benefit by generating
comprehensive API documentation for migrated Flask routes.
* feat(flask): Implement manual OpenAPI spec generation and attachment
- Add FlaskParam and FlaskRouteInfo TypedDict for type-safe metadata
- Implement OpenAPISpecGenerator for Flask→OpenAPI conversion
- Extract route metadata via FlaskIntrospector
- Attach generated spec to router.openbb_openapi_spec_ext
- Add unit tests (3/3 passing)
Addresses WSGI/mount boundary issue where Router.command metadata
cannot propagate through WSGIMiddleware. Spec attached to router
instance for downstream OpenAPI assembly.
Resolves maintainer feedback on PR #7264
* feat(flask): Implement FlaskMountRegistry for multiple mount support
- Add FlaskMountRegistry class for centralized OpenAPI spec management
- Replace router.openbb_openapi_spec_ext with registry pattern
- Support multiple Flask apps at different mount prefixes
- Automatic path rewriting in aggregated OpenAPI specs
- Add registry test case (4/4 tests passing)
Addresses @deeleeramone's feedback on PR #7264 regarding router
attribute fragility and multiple mount support. Registry provides
lifecycle-independent state management and enables enterprise
scenarios with multiple Flask apps.
Resolves: OpenBB-finance/OpenBB#7264 (partial)
* feat(core): load Flask apps as openbb_core_extension WSGI sub-apps
A Flask app referenced directly by an openbb_core_extension entry point is
mounted via WSGIMiddleware and its routes are introspected into openapi.json
(operationId/widgetId/mcp_tool, parameters, request bodies, security, widget_config
and HTML widgets) so the widget factory and MCP server can discover them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(core): 100% coverage for flask utils + flask dev dependency
Add branch-coverage tests for the Flask integration package (introspection,
AST analysis, docstrings, OpenAPI generation, registry, loader, detection) and
declare flask as a dev dependency so the suite runs in CI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flask): treat f-string returns as HTML responses
* fix(flask): mount via a2wsgi instead of deprecated starlette WSGIMiddleware
- add openbb-core[flask] extra (flask, a2wsgi); a2wsgi resolved through require_optional
- run the WSGI serve test through mount_flask_extensions; drop httpx importorskip
- core CI installs [pandas,flask] and httpx2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(flask): cover failed entry point loads; narrow optional types for ty
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* perf(core): vectorize the date conversion in `basemodel_to_df` (#7622)
* perf(core): vectorize the date conversion in basemodel_to_df
basemodel_to_df ran three per-element Python passes over the date column of
every payload it converted. Series.apply(to_datetime) calls pandas.to_datetime
once per row, and because each call receives a lone scalar with no format hint,
pandas also re-runs its datetime format guesser on every row.
Parse the column in one call instead, and use vectorized comparisons for the
all-midnight test and the reduction to datetime.date.
The element-wise parse is kept as a fallback for one case that the vectorized
path cannot represent: a column mixing UTC offsets, such as a series crossing a
daylight-savings boundary. Parsing per element lets each value keep its own
offset and yields an object column of tz-aware Timestamps, whereas a single
to_datetime call over that column raises "Mixed timezones detected". That
behaviour is covered by test_to_df_daylight_savings and is preserved exactly.
Measured on a payload built through EquityHistoricalData, median of 9 under an
exclusive CPU lock:
250 daily bars 8.92 ms -> 1.46 ms
1,000 daily bars 32.93 ms -> 3.13 ms
5,000 daily bars 162.25 ms -> 12.23 ms
* fix a broken test - unrelated to this PR...
---------
Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix lint errors
* fix oecd lint error
* [V5] `openbb-mcp-server`: Refactor for v5 - Spec, Config, Packaging, Test Coverage (#7495)
* stash some changes
* add more robust testing
* mypy
* point PR at V5
* introduce spec file
* codespell
* test fix
* fix workflow environment setup
* fix workflow environment setup
* fix workflow environment setup
* add pyyaml to dependencies
* split lint jobs
* fix workflow environment setup
* fix workflow environment setup
* workflow env setup
* workflow env setup
* clean up code comments
* add auth hook entrypoints
* codespell
* add codegen feature
* codespell
* move _unpack into dispatchers for consistency with codegen packages
* surface nested models in the response
* fix missing coverage in CI
* socrata updates
* test fix
* detect plotly output
* add --include and --exclude flags from generate-extension command
* refactor mcp server for v5
* test action
* fix tests
* tests
* test fixture name
* revert: drop unrelated provider/README changes from mcp-server refactor
Strip the changes to ``README.md`` and ``openbb_platform/providers/{imf,
tradingeconomics,yfinance}`` that snuck into 4ac3602 (refactor mcp server
for v5). Those files are out of scope for the mcp-server work and were
modified inadvertently — restore them to their pre-4ac3602 state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* tests and workflows
* fix workflow
* tests and workflows
* tests and workflows
* tests and workflows
* tests and workflows
* beartype shim for 3.15
* muck around with beartype for 3.15
* muck around with beartype for 3.15
* no 3.15 runners for mcp-server
* update skill files
* cap test matrix at python 3.14
* merge branch v5
* no useless comments
* platform controller command description split
* merge URL overloads from path params
* exclude none and unset from model dump
* refactor for fastmcp4
* wrap markdown lines that are too long
---------
Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* update pre-commit, noxfile, and uv locks
* fixes for docstring generation
* test coverage
* fix cli renderer helper with figure object JSON
* fix the figure font when rendered
* refactor cookiecutter
* bump cookiecutter packagge version
* fix sec test by adding fixture
* fix desktop environments test
* fix(cli): inherit path-item-level OpenAPI parameters (#7606)
* fix(cli): inherit path-item-level OpenAPI parameters
OpenAPI 3.x lets a Path Item Object declare `parameters` shared by every
operation under that path, with operation-level entries overriding them by
(name, in). The codegen read only `op.get("parameters")`, so inherited
parameters were dropped silently — the generated command kept the URL
template but had nothing registered to fill it.
Real-world impact: the Xero Accounting API declares its required
`xero-tenant-id` header once per path item across all 138 paths. Before this
change `build_command_spec` produced 78 commands and none carried the
header, so every generated command 403s. Nothing warned.
Adds `operation_parameters()`, which merges the two levels and dereferences
both, and routes the pre-computed spec path (`build_command_spec`) and the
interactive path (`build_command_index` -> `build_parser_from_operation`)
through it. Unresolvable refs, non-dict entries, and nameless parameters are
skipped as before. Specs without path-item parameters are unaffected.
* fix(cli): narrow path-item inheritance to path and query parameters
Per review: headers do not translate into a Python interface function, and
outbound headers are transport configuration
(SystemSettings.PythonSettings.http.headers, the CLI's -H) or a security-scheme
concern — not a per-command argument.
Path-item `in: header` and `in: cookie` entries are now dropped rather than
inherited. Operation-level parameters are untouched, so an operation that
declares its own header keeps today's behavior.
Scope is now only what cannot be supplied any other way: `in: path` and
`in: query` parameters declared on the Path Item Object.
* fix(cli): merge allOf compositions in response schemas
Response-schema $refs were already expanded by deref_schema, but a schema
whose top level is an allOf composition was left unflattened. The properties
live inside the members, so a consumer reading them off the composition finds
none. fetcher_gen._data_schema looks for `properties.results`, gets nothing,
and falls through to _unwrap_schema_envelopes, which handles oneOf and anyOf
but not allOf. The generated Data model ends up with no fields.
Codat's listBills response is `allOf: [{properties: {results}}, {properties:
{_links, pageNumber, pageSize, totalResults}}]`, which produced an empty model.
merge_allof collapses a composition of object subschemas into one object
schema: properties unioned with the first declaration winning on conflict,
required unioned, and sibling keywords on the allOf node outranking members.
A composition that is not a plain intersection of object subschemas (a member
carrying its own oneOf/anyOf, or a non-object type) is returned unchanged
rather than flattened into something the spec does not describe. Nested
compositions are merged too, so a $ref inside a merged member still resolves.
Applied to extract_response_schema and extract_response_schemas.
Measured across five specs that use the pattern, counting operations whose
response schema exposes readable properties: 244 of 588 before, 394 after
(Codat Accounting 36 to 88, Codat Banking 0 to 8, Codat Commerce 1 to 11,
Rebilly 62 to 104, Box 145 to 183).
* fix(cli): merge allOf compositions in request body schemas
Same gap as the response side. request_body_parameters needs `type: object`
or `properties` to flatten a body into parameters, and an allOf composition
carries neither, so a POST or PUT generated from one gets no body parameters
at all.
extract_request_body_schema now runs merge_allof after deref_schema, matching
extract_response_schema.
Across the same five specs, counting request bodies that produce parameters:
138 of 206 before, 177 after (Codat Accounting 1 to 26 of its 30 bodies,
Rebilly 50 to 63, Box 87 to 88).
* fix(cli): stop annotation keywords blocking an allOf merge
_mergeable_allof_members required every member's keys to come from a small
whitelist, so any unrecognized keyword disqualified the whole composition.
Codat's PagingInfo carries `definitions` next to its properties, which meant
the live Codat-Accounting.yaml never merged and its response models stayed
empty. The APIs.guru copy of the same API omits that key, which is why the
earlier measurements looked clean.
The check is now a blocklist. A member blocks the merge when it carries its
own combinator (oneOf/anyOf/allOf/not), an unresolved $ref left by a cycle,
a scalar or array constraint (enum/const/items), or a type other than object.
Annotation keywords that do not affect the intersection (title, description,
examples, definitions, additionalProperties, vendor x- keys) no longer block
it, and are dropped from the merged result along with the members.
On the live Codat spec, operations whose response schema exposes readable
properties go from 36 of 127 to 114. Across the five specs measured earlier:
244 of 588 before, 488 after (was 394 under the whitelist).
Corpus check over 2,816 response and request-body extractions in 6 specs:
567 changed, every one attributable to an allOf in the pre-merge output,
0 unexplained. Twilio declares no allOf compositions and is unchanged.
* fix(cli): emit a name once when it is both a parameter and a body property
_signature_params appended request-body properties and then operation
parameters without deduplicating, so a name declared in both places produced
two function parameters with the same name and the generated module raised
SyntaxError.
Codat's push endpoints hit this: /push/bankAccounts/{accountId}/bankTransactions
declares accountId as a path parameter and repeats it in the request body, so
connections.push.bankAccounts.bankTransactions generated `accountId: str` and
`accountId: str = None` in one signature.
The operation parameter wins, since it carries the path placeholder and its own
required flag. The request payload is unchanged: _render_body_block derives body
fields from request_body_schema rather than from this list, so accountId is
still sent in the body.
This predates the path-item and allOf work in this PR. On origin/v5 the same
spec produces the same duplicate; on this branch the Codat spec generates 91
commands with 0 duplicated signatures, down from 1.
* fix(cli): use the safe identifier for keyword-named body properties
_render_body_block already writes `{'from': from_}`, keying the payload by the
wire name and reading a safe identifier, but _signature_params declared the raw
name. A body property named after a Python keyword therefore emitted
`from: str = None` and the generated module raised SyntaxError.
This did not fire before the allOf work in this PR, because a composed request
body produced no body parameters at all. Codat's Transfer body is such a
composition and has a `from` property, so connections.push.transfers started
generating invalid syntax once the body flattened.
_signature_params now emits safe_field_name(name)[0], the same helper
fetcher_gen and pydantic_gen already use. The body/query split in
_render_body_block compares against the safe names too: it previously matched
signature names against wire names, so `from_` was not recognized as a body
field and leaked into the query string as a `from_` parameter.
End to end on Codat-Accounting.yaml, generating the full extension and byte
compiling every emitted module: 136 files, 0 failures. origin/v5 emits 1 file
that does not compile (duplicate argument 'accountId').
Known gap left alone: an operation parameter named after a keyword would still
break, since the query and path lines use the raw name as a local variable.
None of the five specs checked declares one.
* fix(cli): do not treat instance data as schema, and sanitize POST identifiers
Two defects found by generating extensions from real specs and byte compiling
every emitted module.
merge_allof and deref_schema walked every value in a schema dict, including
keywords that hold instance data rather than subschemas. A `default` of
`{"allOf": [{"a": 1}, {"b": 2}]}` was rewritten to `{"type": "object"}`, and a
`$ref` key inside an example was resolved as if it were a reference. Both now
copy `default`, `example`, `examples`, `enum`, `const` and vendor `x-` values
through untouched.
_signature_params emitted operation parameter names verbatim, so a wire name
that is not a Python identifier produced invalid syntax. Rebilly declares an
`Organization-Id` header and a `REB-APIKEY` security header, which emitted
`REB-APIKEY: str = None`. It now uses safe_field_name, matching fetcher_gen,
and _render_body_block keys the query dict by the wire name while reading the
safe identifier so the request is unchanged.
This surfaced through the allOf work rather than being caused by it: a composed
request body used to yield no properties, so `has_body` was false in
package_gen and the command was generated as a GET fetcher, where identifiers
were already sanitized. With bodies flattening, 14 of Codat's 24 POST endpoints
correctly move to the POST path, which had never been exercised by these specs.
Generating the full extension and compiling every module:
Codat-Accounting.yaml v5: 1 failure branch: 0 (136 files)
Rebilly 2.1 v5: 14 failures branch: 1 (97 files)
The one remaining Rebilly failure is `async def 3dsecure(` in the router, which
fails identically on v5. Sanitizing that is a naming decision about the command
surface, so it is left alone here.
Known gap: a non-credential header parameter is still placed in the query
string on the POST path. Pre-existing, and unchanged by this PR.
* fix(cli): resolve signature collisions on the emitted identifier
The previous dedupe compared wire names, but the failure it prevents depends on
the sanitized identifier. Two wire names can collapse onto one (`from` and
`from_`, `Organization-Id` and `Organization_Id`), and operation_parameters
keeps a name declared twice under different `in` locations by design, so both
cases could still emit a duplicate argument.
_signature_params now tracks the identifiers it has emitted and skips anything
that would repeat one. Emission order is unchanged, so existing signatures are
byte-identical; only genuinely colliding names are dropped, and operation
parameters still win over body properties.
Neither collision occurs in Codat-Accounting.yaml or Rebilly 2.1. This closes
the class rather than the instance, since the instances are what this PR has
been finding.
Generation unchanged: Codat 136 files 0 failures, Rebilly 97 files 1 failure
(`async def 3dsecure(`, identical on v5).
* test(cli): cover the instance-data guard, and drop a redundant branch
Mutation checking the new tests (revert each fix, confirm its test fails)
found two problems with the previous commits.
The guard that stops merge_allof and deref_schema from walking into `default`,
`example`, `enum`, `const` and vendor values shipped with no test at all. It is
now covered from both directions: a default holding an `allOf` key keeps its
value, and a `$ref` inside a default stays a literal while a real reference
beside it still resolves.
The body-property loop also carried a redundant check. It tested the raw wire
name against the operation parameter names before testing the sanitized one,
but `operation_param_names` holds both forms and `safe_field_name` is idempotent
on identifiers, so the first test could never fire alone. Reverting it left
every test passing, which is what surfaced it. Removed, and the sanitized name
is now computed once.
All nine mutations are caught by the suite: both data guards, the allOf
blocklist in each direction, the response merge wiring, body/parameter
precedence, the keyword identifier, the body/query split and the
emitted-identifier dedupe.
cli/tests at 1,954 passed against the same 80 environment failures as the base.
Generation unchanged: Codat 136 files 0 failures, Rebilly 97 files 1 failure.
* fix(cli): route required path-item headers/cookies through credential handling
* fix(cli): narrow Popen.stdin/stdout for ty in run_in_obb fixture
---------
Co-authored-by: deeleeramone <>
* fix warnings raised by federal_reserve on some xlsx workbook parsing
* fix publish workflows
* test publish pipeline with openbb-core-v2.0.0rc0 (#7681)
* fix publish workflows with missing tag job, some tests and bump fastapi
* fix python 3.10 test failure in core
* actually fix python 3.10 test failure in core
* deincrement openbb-core version to allow release pattern
* bump core version (#7682)
* remove uv.sources from pyproject.toml files and point to openbb-core-2.0.0 on PyPI
* remove oecd cache asset that wasn't supposed to be commuitted
* remove poetry.lock, update pyproject.tml and uv.lock for openbb-platform-api-v2.0.0 (#7684)
Co-authored-by: deeleeramone <>
* add the missing publish-openbb-platform-api.yml workflow file
* remove poetry.lock, update pyproject.tml and uv.lock for openbb-platform-api-v2.0.0 (#7685)
Co-authored-by: deeleeramone <>
* fix the tag generation after publish
* [Release] `openbb-charting-v4.0.0` (#7686)
* up openbb-charting to 4.0.0 and fill missing test coverage in chart_style
* test coverage and format markdown files
---------
Co-authored-by: deeleeramone <>
* remove scikit-learn from dependencies, update tests, and format markdown files (#7687)
Co-authored-by: deeleeramone <>
* [Release] `openbb-cli-v2.0.0` (#7692)
* fix missing test coverage in openbb_cli
* fix conftest
---------
Co-authored-by: deeleeramone <>
* add missing publish-openbb-cli.yml workflow
* [Release] `openbb-cli-v2.0.0` (#7693)
* fix missing test coverage in openbb_cli
* fix conftest
---------
Co-authored-by: deeleeramone <>
* test fixes for cli...
* [Release] `openbb-cli-v.2.0.0` (#7694)
* fix missing test coverage in openbb_cli
* fix conftest
---------
Co-authored-by: deeleeramone <>
* fix hanging mcp server test
* [Release] `openbb-mcp-server-v2.0.0` (#7695)
Co-authored-by: deeleeramone <>
* update uv.lock (#7696)
Co-authored-by: deeleeramone <>
* release openbb-commodity-v2.0.0 (#7697)
Co-authored-by: deeleeramone <>
* update uv.lock (#7698)
Co-authored-by: deeleeramone <>
* update uv.lock (#7699)
Co-authored-by: deeleeramone <>
* update uv.lock (#7700)
Co-authored-by: deeleeramone <>
* [Release] `openbb-econometrics-v2.0.0` (#7701)
* update uv.lock
* update pyproject.toml
---------
Co-authored-by: deeleeramone <>
* update uv.lock (#7703)
Co-authored-by: deeleeramone <>
* update uv.lock (#7702)
Co-authored-by: deeleeramone <>
* update uv.lock (#7704)
Co-authored-by: deeleeramone <>
* update uv.lock (#7705)
Co-authored-by: deeleeramone <>
* update uv.lock (#7706)
Co-authored-by: deeleeramone <>
* [V5] Fix a Number of V5 Bugs (#7707)
* fix a number of v5 bugs
* fix lint error
* more lint
* fixes for quantitative, technical, charting, famafrench for version bumps
---------
Co-authored-by: deeleeramone <>
* update uv.lock (#7709)
Co-authored-by: deeleeramone <>
* update uv.lock (#7708)
Co-authored-by: deeleeramone <>
* update uv.lock (#7710)
Co-authored-by: deeleeramone <>
* update uv.lock (#7711)
Co-authored-by: deeleeramone <>
* [Release] `openbb-cboe-v2.0.0` (#7713)
Co-authored-by: deeleeramone <>
* update uv.lock (#7714)
Co-authored-by: deeleeramone <>
* update uv.lock (#7717)
Co-authored-by: deeleeramone <>
* [Release] `openbb-ecb-v2.0.0` (#7715)
* update uv.lock
* update uv.lock
---------
Co-authored-by: deeleeramone <>
* update uv.lock (#7718)
Co-authored-by: deeleeramone <>
* update uv.lock (#7719)
Co-authored-by: deeleeramone <>
* update uv.lock (#7720)
Co-authored-by: deeleeramone <>
* add missing publish-openbb-fred.yml to repo
* update uv.lock (#7723)
Co-authored-by: deeleeramone <>
* update uv.lock (#7722)
Co-authored-by: deeleeramone <>
* update uv.lock (#7716)
Co-authored-by: deeleeramone <>
* update uv.lock (#7724)
Co-authored-by: deeleeramone <>
* update uv.lock (#7726)
Co-authored-by: deeleeramone <>
* [Release] `openbb-fred-v2.0.0` (#7725)
* update uv.lock
* fix tests and dependencies
* fix test_fred_fetchers
---------
Co-authored-by: deeleeramone <>
* [Release] `openbb-imf-v3.0.0` (#7727)
* update uv.lock
* fix tests and dependencies
* fix test_fred_fetchers
* bump openbb-imf version to 3.0.0
---------
Co-authored-by: deeleeramone <>
* update uv.lock (#7729)
Co-authored-by: deeleeramone <>
* update uv.lock (#7728)
Co-authored-by: deeleeramone <>
* update uv.lock (#7730)
Co-authored-by: deeleeramone <>
* update uv.lock (#7731)
Co-authored-by: deeleeramone <>
* update uv.lock (#7732)
Co-authored-by: deeleeramone <>
* update uv.lock (#7733)
Co-authored-by: deeleeramone <>
* cookiecutter pyproject and lock
* update pyproject.toml (#7734)
Co-authored-by: deeleeramone <>
* update pyproject.toml (#7735)
Co-authored-by: deeleeramone <>
* update uv.lock (#7736)
Co-authored-by: deeleeramone <>
* fix some sec issues, allegedly (#7737)
Co-authored-by: deeleeramone <>
* update uv.lock (#7738)
Co-authored-by: deeleeramone <>
* update main pyproject.toml
* lint fixes
* update assets (#7739)
Co-authored-by: deeleeramone <>
* [Release] `openbb-v5.0.0` (#7740)
* update assets
* add missing workflow
---------
Co-authored-by: deeleeramone <>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: Dex <i@dex.moe>
Co-authored-by: Monte Hoover <monte.b.hoover@gmail.com>
Co-authored-by: Boris Li <65254053+BorisQuanLi@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Lucas Zimmermann <lucaszmayer@gmail.com>
815 lines
45 KiB
Markdown
Vendored
815 lines
45 KiB
Markdown
Vendored
# CFTC Provider Extension
|
||
|
||
This package provides full access to the CFTC Commitment of Traders database.
|
||
Reports are fetched by CFTC Code (e.g. `13874+`) and returns the full history of the contract.
|
||
|
||
It also exposes the swap transaction and pricing data published by DTCC's Public Price
|
||
Dissemination service — the swap data repository for CFTC-jurisdiction trades — including
|
||
the CDS index tape, central bank overnight index swap curves, and FX forward points
|
||
constructed from executed transactions.
|
||
|
||
## Installation
|
||
|
||
Install into a Python environment (3.10 - 3.14) from PyPI with:
|
||
|
||
```sh
|
||
pip install openbb-cftc
|
||
```
|
||
|
||
Then build the Python static assets:
|
||
|
||
```sh
|
||
openbb-build
|
||
```
|
||
|
||
## Credentials
|
||
|
||
Credentials are not required, but your IP address may be subject to throttling limits on the
|
||
Commitment of Traders endpoints. The DTCC Public Price Dissemination endpoints are public and
|
||
require no credentials.
|
||
|
||
API requests made using an application token are not throttled.
|
||
|
||
Create a free account here: <https://evergreen.data.socrata.com/signup>
|
||
|
||
Then, generate the app_token by signing in with the credentials here: <https://publicreporting.cftc.gov/profile/edit/developer_settings>.
|
||
|
||
### Credentials Key
|
||
|
||
If adding a token, use `cftc_app_token` as the key in the `user_settings.json` file. The value expected value is the app_token and not the `secret` or `api_key`.
|
||
|
||
The token can be set as an environment variable:
|
||
|
||
```env
|
||
CFTC_APP_TOKEN
|
||
```
|
||
|
||
## Coverage
|
||
|
||
This extension provides a dedicated router module with eight endpoints, and one OpenBB Workspace application.
|
||
|
||
| Endpoint | Source | Description |
|
||
|:---------|:-------|:------------|
|
||
| `obb.cftc.cot` | CFTC Public Reporting | Commitment of Traders reports. |
|
||
| `obb.cftc.cot_search` | CFTC Public Reporting | Search COT report series. |
|
||
| `obb.cftc.swap_trades` | DTCC PPD slice | Swap transaction and pricing data. |
|
||
| `obb.cftc.cds_index_trades` | DTCC PPD slice | CDS index transactions, curated per print. |
|
||
| `obb.cftc.ois_curve` | DTCC PPD search | Central bank overnight index swap curve. |
|
||
| `obb.cftc.ois_curve_history` | DTCC PPD slice | OIS curve over time, by tenor. |
|
||
| `obb.cftc.ois_forward_curve` | DTCC PPD search | Forward par swap rate curve, by start date. |
|
||
| `obb.cftc.fx_forward_points` | DTCC PPD search | FX forward points for the major crosses. |
|
||
|
||
The DTCC endpoints draw on two different sources, deliberately:
|
||
|
||
- The **cumulative slice** is one zip per calendar day holding every transaction disseminated
|
||
that day, in a 110-column CSV. It has no record cap.
|
||
- The **search** endpoint queries a date range server-side and returns JSON. It filters by UPI,
|
||
currency and notional, but truncates at 10,000 records without saying so.
|
||
|
||
`swap_trades` stays on the slice. It is a raw feed of the published columns, its model is
|
||
CSV-shaped, and the search's schema is a different shape (116 camelCase fields that split some
|
||
of the CSV's merged columns) that would need a mapping layer for no gain — the slice already
|
||
returns a full day uncapped, which is what a raw feed should do. With no `date`, it walks back
|
||
from the newest published file to the most recent one holding transactions that match the
|
||
query, because a Saturday's file is published but nearly empty.
|
||
|
||
`ois_curve_history` also stays on the slice: it is explicitly a per-report-date series, so
|
||
per-node as-of dating would defeat its purpose.
|
||
|
||
`cds_index_trades` stays on the slice for the same reason `swap_trades` does, plus one of its
|
||
own: it is a **tape**, so it must not silently drop prints, and the search's 10,000-record cap
|
||
would do exactly that on a busy day. The slice returns a full day, uncapped.
|
||
|
||
## Methodology
|
||
|
||
A compact reference for the formulas and assumptions behind the calculated endpoints. The
|
||
discount factors `DF(t)` bootstrapped in the first block feed every block below.
|
||
|
||
### Rate curve bootstrap — `ois_curve`, `ois_curve_history`
|
||
|
||
- **Par rate** at a tenor: the median of the day's executed fixed rates (`aggregation` selects
|
||
median / mean / vwap).
|
||
- **Discount factors**, bootstrapped from par rate `c`, with annual fixed payments and accrual
|
||
`τ_i = days_i / b` on the currency's money-market basis `b` (360 or 365):
|
||
- tenor ≤ 1Y (one payment): `DF = 1 / (1 + c·τ)`;
|
||
- longer: `DF_n = (1 − c·Σ_{i<n} τ_i·DF_i) / (1 + c·τ_n)`, the `DF_i` read off the curve built
|
||
so far.
|
||
- **Zero rate** `z`, annually compounded ACT/365F over `t` years: `DF = (1 + z)^(−t)`.
|
||
- Intermediate DFs interpolate on `log DF` (`log_linear`, or monotone Hermite `log_cubic`).
|
||
- *Assumptions:* fixed leg pays annually; negative rates admitted, so `DF > 1` is allowed and the
|
||
curve is not required to be monotone; a node is dropped only if its bootstrapped `|z|` exceeds a
|
||
sanity bound.
|
||
|
||
### Forward par swap rate — `ois_forward_curve`
|
||
|
||
A Nelson-Siegel-Svensson curve is fit to the bootstrapped `log DF` (trade-count weighted). The
|
||
rate of a `T`-year swap starting `t0` forward is the par rate of that forward schedule:
|
||
|
||
`f(t0, T) = (DF(t0) − DF(t0 + T)) / Σ_i τ_i·DF(t0 + t_i)`
|
||
|
||
over the swap's annual points `t_i`. A tail past the last pillar uses the extended fit and is
|
||
flagged `extrapolated`.
|
||
|
||
### Policy path — `ois_policy_path`
|
||
|
||
The forward par curve *implies* forwards off the spot curve; this reads them **directly** from
|
||
the forward-starting OIS the spot curve excludes. A forward-starting OIS over `[E, E+τ]` fixes at
|
||
the expected average compounded overnight rate over that window, so its fixed rate is a market
|
||
observation of the forward policy rate. Trades are anchored to the curve date (a start that has
|
||
rolled to spot drops out), bucketed to whole-month `(forward start, tenor)`, medianed on the most
|
||
recent day each bucket traded in depth, and sorted by start — the priced central-bank path. Its
|
||
short, meeting-dated tenors are the policy segments; `max_tenor` bounds how far up the term a
|
||
forward-starting swap is still treated as a policy point rather than a forward par rate.
|
||
|
||
### FX forward points — `fx_forward_points`
|
||
|
||
- **Rate** of a trade is quote-per-base from the leg-notional ratio (not the reported basis
|
||
field, which flips orientation).
|
||
- **Spot** `S` is the median of the shortest-dated trades: deliverable pairs use trades settling
|
||
≤ 2 days; a non-deliverable pair has no spot leg, so its two shortest tenor buckets are linearly
|
||
extrapolated back to T+2.
|
||
- **Executed points** `= (F − S)·pip`, with `F` the tenor's median forward rate and `pip` the
|
||
quote's smallest increment (`10^4`, or `10^2` for a JPY / two-decimal EM quote).
|
||
- **Covered-interest-parity (theoretical) points:** `F_cip = S·(1 + r_q·T/b_q)/(1 + r_b·T/b_b)`.
|
||
Each leg's simple rate is `r = (1/DF − 1)·b/days`, so `(1 + r·T/b) = 1/DF` and `F_cip` collapses
|
||
to the factor ratio `S·DF_base/DF_quote` — basis-independent. `basis_points = executed − theoretical`
|
||
is the cross-currency / NDF basis to onshore rates.
|
||
- An EM leg with no OIS builds `DF_quote` from its fixed-float IRS, then its non-deliverable cross
|
||
swap — deepest source first.
|
||
|
||
### FX implied volatility — `fx_implied_vol`
|
||
|
||
- Each option is inverted with **Garman-Kohlhagen** (two-rate Black-Scholes), forward
|
||
`F = S·DF_for/DF_dom`, implied vol solved by **bisection** on the price (out-of-the-money options
|
||
only, since an ITM one is redundant by put-call parity).
|
||
- A **non-deliverable pair** has no domestic (EM) OIS, so its domestic DF is implied from the
|
||
observed NDF forward: `DF_dom(t) = DF_for(t)·S / F_ndf(t)`, which makes `F` equal the traded NDF
|
||
forward rather than a covered-parity one.
|
||
- **Empirical** surface: median vol per (grid strike, expiry). **Theoretical:** a lognormal-SABR
|
||
smile per expiry (`β = 1`, `α` pinned to the ATM vol, `ρ`/`ν` by grid-and-refine least squares),
|
||
evaluated on the grid.
|
||
- Strike grid: pip offsets from spot for the majors; **basis-point moneyness**,
|
||
`K = S·(1 + offset/10^4)`, for the wider-trading NDF pairs.
|
||
|
||
### Swap valuation — `swap_valuation`
|
||
|
||
Off the bootstrapped `DF` and an annual fixed schedule to maturity `T` with annuity
|
||
`A = Σ_i τ_i·DF(t_i)`:
|
||
|
||
- **Par rate** `c* = (1 − DF(T)) / A`.
|
||
- **Fixed leg** PV `= N·c·A`; **floating leg** PV `= N·(1 − DF(T))` — the sum of the period
|
||
forwards `f_i = (DF_{i−1}/DF_i − 1)/τ_i`, discounted, telescopes to it.
|
||
- **NPV** (pay-fixed) `= PV_float − PV_fixed`; receive-fixed is its negative. A contract at
|
||
`c = c*` prices to zero.
|
||
- **DV01** `= N·A·10^−4`, the present value of one basis point on the fixed leg.
|
||
|
||
### Swap payoff — `swap_payoff`
|
||
|
||
The valuation is one point; the payoff is the whole curve. Holding the contract rate `c`, the
|
||
discount curve is shifted in parallel by `δ` (basis points), `DF_δ(t) = DF(t)·e^{−δ·t}`, and the
|
||
swap re-valued at each shift. The result is the mark-to-market P&L profile against rates: zero at
|
||
the entry par rate, rising for the side long rates (pay-fixed gains as rates rise), and **concave
|
||
for pay-fixed** — it is short duration, so a symmetric shift loses more than it gains. Re-pricing
|
||
off the shifted curve, rather than a straight `DV01·δ` line, is what shows that curvature.
|
||
|
||
## Usage
|
||
|
||
The package can be used as a Python module, or a REST API.
|
||
|
||
### REST API
|
||
|
||
Start the server over localhost with:
|
||
|
||
```sh
|
||
openbb-api
|
||
```
|
||
|
||
### Python
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
search_results = obb.cftc.cot_search(query="gold")
|
||
|
||
print(search_results.to_df())
|
||
```
|
||
|
||
| code | name | category | subcategory | commodity_name | contract_units |
|
||
|:------------|:-------------------|:------------------|:----------------|:-----------------|:-------------------------------|
|
||
| CFTC_088691 | GOLD | NATURAL RESOURCES | PRECIOUS METALS | GOLD | (CONTRACTS OF 100 TROY OUNCES) |
|
||
| CFTC_088LM1 | GOLD -1 TROY OUNCE | NATURAL RESOURCES | PRECIOUS METALS | GOLD | (1 troy ounce x $2,300) |
|
||
| CFTC_088695 | MICRO GOLD | NATURAL RESOURCES | PRECIOUS METALS | GOLD | (CONTRACTS OF 10 TROY OUNCES) |
|
||
|
||
```python
|
||
report = obb.cftc.cot(code="CFTC_088695", measure="percent_of_oi", limit=4)
|
||
print(report.to_df().T)
|
||
```
|
||
|
||
| | 2026-03-03 | 2026-03-10 | 2026-03-17 | 2026-03-24 |
|
||
|:--------------------------------------------|-------------:|-------------:|-------------:|-------------:|
|
||
| open_interest_all | 60933 | 58119 | 64124 | 71998 |
|
||
| open_interest_pct_all | 1 | 1 | 1 | 1 |
|
||
| open_interest_pct_non_commercial_long_all | 0.186 | 0.18 | 0.168 | 0.285 |
|
||
| open_interest_pct_non_commercial_short_all | 0.527 | 0.546 | 0.555 | 0.515 |
|
||
| open_interest_pct_non_commercial_spread | 0.058 | 0.06 | 0.078 | 0.124 |
|
||
| open_interest_pct_commercial_long_all | 0.088 | 0.095 | 0.091 | 0.083 |
|
||
| open_interest_pct_commercial_short_all | 0 | 0 | 0 | 0 |
|
||
| open_interest_pct_total_reportable_long_all | 0.333 | 0.335 | 0.336 | 0.492 |
|
||
| open_interest_pct_total_reportable_short | 0.585 | 0.606 | 0.632 | 0.64 |
|
||
| open_interest_pct_non_reportable_long_all | 0.667 | 0.665 | 0.664 | 0.508 |
|
||
| open_interest_pct_non_reportable_short_all | 0.415 | 0.394 | 0.368 | 0.36 |
|
||
|
||
## Overnight Index Swap Curves
|
||
|
||
`obb.cftc.ois_curve` builds a curve from the overnight index swaps reported to the CFTC.
|
||
Par rates are the median of the executed fixed rates at each tenor; discount factors are
|
||
bootstrapped from the par rates, and the zero rate is derived from the bootstrapped discount
|
||
curve. The forward rate curve is a separate endpoint, `ois_forward_curve` (see below).
|
||
|
||
Each currency references its own central bank's overnight benchmark. Currencies whose OIS is
|
||
too thinly reported to bootstrap a curve (AUD, NZD, KRW, PHP, TRY) are not offered — their
|
||
liquidity sits in fixed-float swaps against a term benchmark instead.
|
||
|
||
### Depth against staleness, and how this endpoint resolves it
|
||
|
||
Not every tenor trades every day, and a file is published every calendar day including the
|
||
ones where nothing trades. A curve built from a single day is therefore full of holes: CHF
|
||
OIS yields three nodes from one day, and a Saturday yields none at all.
|
||
|
||
Searching a **window** of dissemination days fixes the holes but introduces a worse problem
|
||
if it is done naively. A 30-day median is not today's curve — it is a 30-day-average curve,
|
||
and rates move inside the window. Buying depth with silent staleness is its own kind of lie.
|
||
|
||
So each tenor is priced from **the most recent day it actually traded in depth**, and reports
|
||
that day as `as_of_date` along with `staleness_days`. Deep tenors stay fresh; thin ones show
|
||
their age instead of hiding it in an average.
|
||
|
||
Because the search filters on the product's FISN (`NA/Swap OIS {ccy}`), not just its currency,
|
||
it returns only that currency's OIS — so even a full year of prints stays well under the
|
||
source's 10,000-record response cap. A thinly traded currency can therefore raise `lookback_days`
|
||
to reach the whole **one-year search horizon** and still price each tenor from its freshest print;
|
||
the search chains 180-day sub-windows to cover a span longer than one query allows. Widen the
|
||
window for a currency that never fills its curve from a short one — a THB, ILS or BRL — and leave
|
||
it short for USD or EUR, whose curve is dense from a week. A wide window resurrects tenors that
|
||
last traded months ago, so pair it with `max_staleness_days` to keep the curve current — covered
|
||
next.
|
||
|
||
```python
|
||
curve = obb.cftc.ois_curve(currency="CHF", lookback_days=30)
|
||
```
|
||
|
||
| tenor | as_of_date | staleness_days | num_trades | par_rate | discount_factor |
|
||
|:------|:-----------|---------------:|-----------:|---------:|----------------:|
|
||
| 1M | 2026-07-13 | 3 | 1 | −0.000490 | 1.000041 |
|
||
| 9M | 2026-07-02 | 14 | 1 | −0.000182 | 1.000138 |
|
||
| 1Y | 2026-07-16 | 0 | 2 | 0.000709 | 0.999282 |
|
||
| 10Y | 2026-07-16 | 0 | 5 | 0.006800 | 0.932928 |
|
||
| 30Y | 2026-07-16 | 0 | 1 | 0.009400 | 0.748409 |
|
||
|
||
The 10Y is today's; the 9M is a fortnight old and says so.
|
||
|
||
### Capping staleness on a wide window
|
||
|
||
Reaching back a year fills a thin currency's curve, but not for free: a tenor that last traded
|
||
eight months ago is priced against a market that has since moved, and splicing it beside a
|
||
fresh node is a curve of two different dates pretending to be one. NZD is the clearest case — a
|
||
full-year search finds a 1M node 250 days stale sitting under a 3M node priced yesterday, and the
|
||
short end zig-zags because the two are from different regimes.
|
||
|
||
`max_staleness_days` drops any node whose freshest print is older than the cutoff, **before** the
|
||
bootstrap, so a stale off-market rate never becomes a pillar the rest of the curve discounts
|
||
through. It defaults to `None` — every node is kept and its age reported in `staleness_days`, the
|
||
same judge-for-yourself default as `min_trades` — but on a wide window it is how you keep the
|
||
result a curve rather than a mosaic:
|
||
|
||
```python
|
||
curve = obb.cftc.ois_curve(currency="NZD", lookback_days=365, max_staleness_days=30)
|
||
```
|
||
|
||
Widen `lookback_days` for reach, then tighten `max_staleness_days` for freshness; the two knobs
|
||
trade the curve's completeness against its currency, and the balance differs by how often the
|
||
currency trades.
|
||
|
||
### `min_trades` counts one day, not the window
|
||
|
||
This is the subtlety worth understanding before raising it.
|
||
|
||
`min_trades` is the number of trades a tenor must have **on a single dissemination day** to
|
||
be dated to that day. It is not a count across the window. Widening `lookback_days` gives a
|
||
tenor *more chances* to clear the bar; it does not pool trades to help it clear.
|
||
|
||
It defaults to 1: one executed trade is still a price, and dropping it would discard real
|
||
transactional data in favour of no data at all. `num_trades`, `min_rate` and `max_rate` report
|
||
how thin a node is, so the caller can judge rather than have the choice made for them.
|
||
|
||
Raising it costs whole tenors on a thin currency, because the bar is per-day:
|
||
|
||
| currency | nodes at `min_trades=1` | nodes at `min_trades=3` |
|
||
|:--|--:|--:|
|
||
| USD | 26 | 25 |
|
||
| EUR | 26 | 24 |
|
||
| GBP | 26 | 24 |
|
||
| JPY | 24 | 21 |
|
||
| CAD | 24 | 14 |
|
||
| CHF | 19 | **9** |
|
||
| CLP | 14 | 5 |
|
||
|
||
CHF short-dated OIS trades **at most once on any given day**. No day will ever carry three
|
||
1M prints, so at `min_trades=3` the CHF short end does not exist at any `lookback_days` — the
|
||
curve simply starts at 1Y. There is no setting that gives you both a dated short end and
|
||
three prints behind it, because those trades were never reported.
|
||
|
||
### Negative rates
|
||
|
||
Negative rates are real — SARON, TONA and ESTR have all printed below zero — so the bootstrap
|
||
admits them. A negative par rate discounts to *more* than 1.0, because the lender pays; CHF's
|
||
short end carries `discount_factor` above 1 and a negative `zero_rate`.
|
||
|
||
Discount factors are therefore **not** guaranteed to decrease across the curve. A monotonically
|
||
decreasing discount curve is not a no-arbitrage condition: it is equivalent to assuming every
|
||
forward rate is positive, which is untrue of CHF and JPY. The condition the bootstrap enforces
|
||
is a positive discount factor, plus a bound on the implied zero rate that rejects a node whose
|
||
par rate is so far off-market that the bootstrap degenerates.
|
||
|
||
### Sources
|
||
|
||
`source="search"` (default) queries a window of dissemination days, as above.
|
||
`source="slice"` uses a single day's cumulative file: no per-node dating, sparse tenors
|
||
missing. A file is published every calendar day, but a weekend or holiday file carries no
|
||
swaps, so `slice` walks `date` back to the most recent day that actually priced the
|
||
currency rather than returning an empty curve. It is kept as a fallback and for reproducing
|
||
a specific published file exactly.
|
||
|
||
| Currency | Index | Administrator |
|
||
|:--|:--|:--|
|
||
| USD | SOFR | Federal Reserve |
|
||
| EUR | ESTR | European Central Bank |
|
||
| GBP | SONIA | Bank of England |
|
||
| JPY | TONA | Bank of Japan |
|
||
| CAD | CORRA | Bank of Canada |
|
||
| CHF | SARON | Swiss National Bank |
|
||
| MXN | TIIE | Banco de Mexico |
|
||
| SGD | SORA | Monetary Authority of Singapore |
|
||
| INR | MIBOR | Reserve Bank of India |
|
||
| COP | IBR | Banco de la Republica |
|
||
| ZAR | ZARONIA | South African Reserve Bank |
|
||
| CLP | ICP | Banco Central de Chile |
|
||
| THB | THOR | Bank of Thailand |
|
||
| ILS | SHIR | Bank of Israel |
|
||
| BRL | CDI | Banco Central do Brasil |
|
||
| AUD | AONIA | Reserve Bank of Australia |
|
||
| NZD | NZIONA | Reserve Bank of New Zealand |
|
||
|
||
AONIA and NZIONA trade only a handful of times a day, so their curves need a long
|
||
`lookback_days` and, to stay current, a `max_staleness_days` cap (see below).
|
||
|
||
BRL's DI maturities fall on fixed calendar dates rather than rolling tenors, so they do not
|
||
snap to benchmark nodes; use `granularity="observed"` for a full BRL curve. CDI accrues on
|
||
BUS/252, recovered to the same year fraction on calendar days.
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
curve = obb.cftc.ois_curve(currency="USD", date="2026-07-15")
|
||
print(curve.to_df())
|
||
```
|
||
|
||
The `rate` column carries the selected `curve_type` (par or zero) **in percent** — 4.32 is
|
||
4.32%, not a decimal — so the chart axis reads as a rate (the grid's percent formatter is not
|
||
applied to the chart). The par and zero values below are shown together for illustration; the
|
||
endpoint returns whichever one `curve_type` selects.
|
||
|
||
| tenor | num_trades | par (%) | zero (%) | discount_factor |
|
||
|:------|-----------:|--------:|---------:|----------------:|
|
||
| 1M | 16 | 3.6668 | 3.7120 | 0.996954 |
|
||
| 1Y | 90 | 4.0135 | 3.9886 | 0.960899 |
|
||
| 2Y | 193 | 4.0203 | 3.9954 | 0.923201 |
|
||
| 10Y | 368 | 4.1378 | 4.1281 | 0.661635 |
|
||
| 30Y | 195 | 4.3379 | 4.3111 | 0.274126 |
|
||
|
||
### Granularity
|
||
|
||
`granularity="benchmark"` (default) snaps trades to standard nodes. `granularity="observed"`
|
||
emits a node for every distinct tenor traded, including broken dates — roughly 180 nodes on a
|
||
typical day, against 24 benchmark nodes.
|
||
|
||
### Interpolation
|
||
|
||
`interpolation` sets how discount factors are read between pillars, on the log of the discount
|
||
factor. It applies to `ois_curve`, `ois_forward_curve` and `ois_curve_history`.
|
||
|
||
- `interpolation` governs how the **bootstrap** interpolates intermediate discount factors,
|
||
which shapes the par and zero curves (the forward curve is fitted separately, see below).
|
||
`log_linear` (default) is piecewise linear on the log discount factors — the standard, most
|
||
transparent choice, and the one a par-swap bootstrap is self-consistent with. `log_cubic`
|
||
fits a monotone Hermite cubic (Fritsch–Butland slopes) to the log discount factors, smoother
|
||
through the pillars without overshooting into a spurious negative forward.
|
||
|
||
The par rates are the curve's input and never move with `interpolation`; only the derived zero
|
||
rates and discount factors do. Tenors of one year or less are a single payment with no
|
||
interpolation, so they are identical under both schemes; longer nodes differ marginally.
|
||
|
||
### Conventions and caveats
|
||
|
||
- Par rates are aggregated with the **median** by default. Notionals above the reporting cap are
|
||
disseminated at the cap (and flagged by `is_capped`), which biases notional-weighted averages.
|
||
- The fixed leg accrues on the day count the trades report - ACT/360 for USD/EUR/CHF/MXN/COP/CLP,
|
||
ACT/365F for GBP/JPY/CAD/SGD/INR/ZAR/THB/ILS, and BRL's BUS/252 recovered to the same year
|
||
fraction on calendar days - with annual payments and a single payment for tenors of one year or
|
||
less. Zero rates are annually compounded ACT/365F, so on an upward-sloping curve the zero rate
|
||
sits above the par rate.
|
||
- `min_trades` (default 1) keeps every node, because one executed trade is still a price. It
|
||
counts one dissemination day's trades, not the window's — see above before raising it.
|
||
- Each node's `maturity_date` rolls off its own `as_of_date`, not the curve's date.
|
||
- `spot_only` (default True) restricts the curve to spot-starting trades, judged against the
|
||
day each trade was disseminated rather than the end of the window.
|
||
- Discount factors are positive but not necessarily decreasing: a negative par rate discounts
|
||
above 1.0. Derived values are withheld only where the bootstrap degenerates; the observed par
|
||
rate is still reported.
|
||
- The search endpoint truncates at 10,000 records **without signalling it**. A window that comes
|
||
back at exactly that count is treated as truncated and bisected until every sub-window clears
|
||
it; a single day that still hits the cap raises rather than building a curve from an arbitrary
|
||
subset.
|
||
- Closed dissemination days are immutable, so each is cached on disk as its own partition and
|
||
only the current day is re-queried. Daily slice files are revalidated against the source
|
||
`ETag`. Slice files are retained for 366 days; the search endpoint serves a rolling one year,
|
||
which a `lookback_days` beyond 180 reaches by chaining 180-day sub-windows.
|
||
|
||
## Forward Rate Curve
|
||
|
||
`obb.cftc.ois_forward_curve` prices the forward par swap rate curve off the OIS curve - the
|
||
object Bloomberg's Forward Curve (FWCV) shows. Each point is the rate of a `forward_tenor` swap
|
||
starting that far forward, read off a **Nelson-Siegel-Svensson** (Svensson) fit to the
|
||
bootstrapped discount factors and weighted by trade count, so a thin long-end node cannot kink
|
||
it and a specific point like 5Y1Y or 1Y5Y is smooth and stable.
|
||
|
||
The swap length and the spacing between start dates are **independent**: `forward_tenor` is how
|
||
long each forward swap runs, `forward_step` is how far apart the start dates sit. So
|
||
`forward_tenor="5Y", forward_step="1Y"` plots the 5Y forward at yearly starts, matching how
|
||
Bluegamma spaces its forward curve at the tenor. `forward_count` caps how many start dates are
|
||
drawn (the next N steps); left unset the curve runs out to its last pillar. A swap whose tail
|
||
runs past the last pillar is priced off the extended fit and flagged `extrapolated` rather than
|
||
withheld, so the final start is still a real point.
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
# the 5Y forward at yearly start dates
|
||
fwd = obb.cftc.ois_forward_curve(currency="USD", forward_tenor="5Y", forward_step="1Y")
|
||
print(fwd.to_df())
|
||
```
|
||
|
||
| index | tenor | start_years | forward_rate |
|
||
|:-----------|:-----------|------------:|-------------:|
|
||
| 2026-07-15 | 5Y @ 0.00Y | 0.00 | 4.0473% |
|
||
| 2027-07-15 | 5Y @ 1.00Y | 1.00 | 4.0221% |
|
||
| 2028-07-15 | 5Y @ 2.00Y | 2.00 | 4.0864% |
|
||
| 2031-07-15 | 5Y @ 5.00Y | 5.00 | 4.2861% |
|
||
|
||
The row `index` is the forward start date, so the chart reads across the dates each forward
|
||
begins; `tenor` labels the swap and its start (`5Y @ 5.00Y` is the 5Y swap five years out).
|
||
|
||
## FX Forward Points
|
||
|
||
`obb.cftc.fx_forward_points` builds forward points from the deliverable FX forwards and FX
|
||
swaps reported to the CFTC. Deliverable FX forwards and swaps are excluded from the swap
|
||
definition but remain reportable to swap data repositories, so they appear in the same data.
|
||
|
||
Spot FX is *not* reportable, so there is no spot fix in the data. Spot is proxied by the
|
||
median of trades settling within two days — overnight FX swap legs, in practice.
|
||
|
||
Tenors are dated per-tenor exactly as the OIS curve's nodes are, with one addition: **each
|
||
tenor is priced against the spot of its own as-of day**, not the curve's freshest spot. A
|
||
tenor that last traded six days ago is differenced against that day's spot, so its points
|
||
stay internally consistent instead of absorbing an unrelated spot move. `spot_rate` therefore
|
||
varies row to row whenever `staleness_days` does.
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
points = obb.cftc.fx_forward_points(pair="USDJPY", date="2026-07-15")
|
||
print(points.to_df())
|
||
```
|
||
|
||
| tenor | num_trades | forward_rate | forward_points |
|
||
|:------|-----------:|-------------:|---------------:|
|
||
| SPOT | 4 | 162.141038 | 0.00 |
|
||
| 1W | 12 | 162.133000 | −0.80 |
|
||
| 2W | 2 | 162.053933 | −8.71 |
|
||
| 1M | 12 | 161.990078 | −15.10 |
|
||
| 2M | 7 | 161.354029 | −78.70 |
|
||
| 3M | 4 | 160.805970 | −133.51 |
|
||
| 6M | 2 | 159.128884 | −301.22 |
|
||
|
||
### FX forward conventions and caveats
|
||
|
||
- Rates are derived from each trade's **leg notionals**, not the reported
|
||
`Exchange rate basis`, which flips between `USD/EUR` and `EUR/USD` on the same instrument.
|
||
- **`min_notional` (default 1,000,000 base units) is load-bearing.** Sub-institutional prints
|
||
— trades of order 10² base units — execute percent away from the interbank market. Left in,
|
||
they dominate a tenor's median: the EURUSD 3M point moves from ~46 pips to ~292 pips, a
|
||
level that is economically impossible for a 3M forward.
|
||
- Points are quoted in pips: 1/10,000 of the quote currency, or 1/100 against JPY.
|
||
- **Depth varies sharply by pair.** USDJPY is the best supported and is reliably monotonic;
|
||
USDCAD and EURUSD are deep but not always monotonic — an individual bucket can print away
|
||
from its neighbours on a given day, which is a property of the reported trades rather than
|
||
of the construction. In the extreme, a single day's bucket dominated by off-market prints (or
|
||
a funding squeeze, e.g. offshore CNH) can cross to the **wrong side of spot** and flip that
|
||
tenor's sign; the median faithfully reports what traded. `num_trades` and the `min_rate` /
|
||
`max_rate` spread — surfaced on the widget — flag such a bucket. GBPUSD and NZDUSD are thin and
|
||
noisy. Check them before relying on a tenor.
|
||
- A tenor's `spot_rate` is its own as-of day's spot, so rows with different `staleness_days`
|
||
are differenced against different spots. Points across such rows are each internally
|
||
consistent but are not a single instant's curve.
|
||
|
||
### Theoretical (covered interest parity) fair value
|
||
|
||
For a pair whose two legs both have an OIS curve — USD, EUR, GBP, JPY, CHF and CAD, i.e. all
|
||
the USD majors — each tenor also carries the **covered-interest-parity** forward points
|
||
alongside the executed ones. This is the forward that neutralizes the interest-rate
|
||
differential (Lehman Brothers, *FX Training Manual*): `F = S·(1 + r_quote·T/b) / (1 + r_base·T/b)`,
|
||
where each leg's money-market rate `r` is the simple rate implied by its OIS **discount
|
||
factor** — so the forward is the factor ratio `S·DF_base/DF_quote`, independent of how the
|
||
curve compounds its zero — accruing on its own basis (365 days for GBP, JPY and CAD, 360 for
|
||
USD, EUR and CHF). The base is at a forward **premium** (points added to spot) when the quote
|
||
out-yields it, a **discount** otherwise.
|
||
|
||
- `theoretical_points` — the CIP fair value, in pips.
|
||
- `basis_points` — executed less theoretical: **positive where the market pays over CIP fair
|
||
value, negative under**. This is the cross-currency basis the executed forwards are pricing.
|
||
- `quotation` — `premium` or `discount`, from the sign of the executed points.
|
||
- `carry` — whether a **long base-currency forward earns or pays** the points to the forward
|
||
date: it **earns** at a discount (the base out-yields the quote, so it is bought forward below
|
||
spot) and **pays** at a premium; the short-base side is the opposite (Lehman Brothers,
|
||
*FX Training Manual*: pay or earn the points). `quotation` and `carry` are set for every pair.
|
||
|
||
AUDUSD and NZDUSD have no OIS curve for the foreign leg, so they carry the executed points
|
||
(and `quotation`/`carry`) but no `theoretical_points`.
|
||
|
||
## FX Implied Volatility
|
||
|
||
`obb.cftc.fx_implied_vol` inverts the **vanilla FX options** (`NA/O Van Call|Put`) reported to
|
||
the CFTC to an implied volatility, using **Garman-Kohlhagen** — the two-rate FX form of
|
||
Black-Scholes (Lehman Brothers, *FX Training Manual*: implied volatility is extracted from the
|
||
Black-Scholes formula). Each option contributes its spot, strike, tenor, and premium; the two
|
||
legs' OIS **discount factors** supply the forward and the discounting — so the pricing is
|
||
consistent with the curve rather than a re-compounded zero — and spot is the same
|
||
concurrent-forward proxy the forward points use.
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
vol = obb.cftc.fx_implied_vol(pair="EURUSD")
|
||
print(vol.to_df())
|
||
```
|
||
|
||
The output is the **volatility surface**: one row per **strike**, one column per **expiry**. A
|
||
strike is an **actual price of the pair** — the strikes lie on a grid of **point (pip) offsets
|
||
from spot**, so spot (offset zero) is the at-the-money reference and every column shares the same
|
||
strike axis. Reading **down a column** traces that expiry's whole smile; reading **across the
|
||
spot strike** is the near-the-money term structure (values in percent, e.g. `6.15` = 6.15%).
|
||
Rendered as a table, each cell is colour-coded by its vol (green low → amber → red high), so the
|
||
smile and the term structure read at a glance — a proper heatmap rather than overlapping lines.
|
||
|
||
| Strike (USD/JPY) | pts | 1W | 1M | 3M | 6M | 1Y |
|
||
|:-----------------|----:|---:|---:|---:|---:|---:|
|
||
| 157.31 | −500 | 8.95 | 7.99 | 7.50 | 8.02 | 8.59 |
|
||
| 159.81 | −250 | 6.77 | 6.87 | 6.98 | 7.67 | 8.56 |
|
||
| 161.31 | −100 | 5.47 | 6.30 | 6.77 | 7.52 | 8.55 |
|
||
| **162.31** | **0** | 4.76 | 6.01 | 6.68 | 7.44 | 8.54 |
|
||
| 163.31 | +100 | 4.49 | 5.82 | 6.63 | 7.39 | 8.53 |
|
||
| 164.81 | +250 | 4.99 | 5.78 | 6.65 | 7.37 | 8.51 |
|
||
|
||
*(USD/JPY, theoretical basis — the downside strikes carry the higher vol, the market's standing
|
||
bid for USD/JPY downside; a negative SABR correlation captures the skew.)*
|
||
|
||
- **`basis`** picks how the surface is built:
|
||
- **`empirical`** (default) medians the traded options at each grid strike — what actually
|
||
traded, with **gaps** where nothing did. The vol is solved by **bisection** on the option
|
||
price, which recovers the out-of-the-money wings a Newton step abandons where its vega
|
||
vanishes.
|
||
- **`theoretical`** fits each expiry's out-of-the-money prints to a **lognormal-SABR** smile
|
||
(β = 1, `α` pinned to the observed at-the-money vol, `ρ`/`ν` found by a grid-and-refine
|
||
least-squares) and evaluates it on the **same strike grid** — a **smooth, gap-free** smile,
|
||
the per-expiry `(α, ρ, ν)` returned in the metadata. A fit needs strikes on both sides of the
|
||
forward, else that expiry is skipped.
|
||
- Only out-of-the-money options are priced — an in-the-money one is redundant by put-call parity
|
||
and its thin extrinsic value inverts to an unstable vol; the surviving puts and calls span the
|
||
smile below and above the forward.
|
||
- Each option is only used if it **executed on the dissemination day** — an option struck days
|
||
earlier priced against a different spot, so inverting it with today's spot would give a wrong
|
||
vol.
|
||
- Each cell is trimmed to a plausibility band around that expiry's own median vol, so a lone
|
||
off-market print cannot corrupt a strike.
|
||
|
||
The **spot** the whole surface hangs off is the median of the pair's shortest-dated
|
||
**deliverable** forwards (`NA/Fwd` / `NA/Swaps`). Non-deliverable forwards (`NA/Fwd NDF …`, a
|
||
scatter of off-spot strikes reported even for deliverable pairs) and vol/variance swaps are
|
||
excluded — left in, they bias spot by tens of pips and manufacture a false skew. A **cross** with
|
||
no deliverable forward of its own (e.g. GBP/JPY) is **triangulated through the dollar** from each
|
||
leg's USD pair; the result reprices the cross's own options to within a few pips of their
|
||
put-call-parity forward. Each option's **strike** is taken in whichever orientation sits closer to
|
||
spot, so a record that mislabels its strike currency pair cannot invert a near-the-money strike
|
||
into a bogus wing.
|
||
|
||
Offered for EURUSD, GBPUSD, USDJPY, USDCHF, USDCAD and EURGBP — the pairs whose options are
|
||
reported to the CFTC in enough depth to build a surface each day; the thinner crosses are left
|
||
off rather than returned empty. The two-rate pricing needs an OIS curve for each leg, priced off
|
||
the rates slice contemporaneous with the options (EURGBP triangulates its spot through the
|
||
dollar). Direction is read from the **Call and Put currency and amount** - the currency bought
|
||
and sold - not the FISN, whose `Call`/`Put` label contradicts the economic legs on a fraction of
|
||
trades. Depth is a single day's options, so on the `empirical` basis short expiries fill only
|
||
near the money and longer ones reach the wings — the `num_options` column shows the depth behind
|
||
each strike row.
|
||
|
||
## CDS Index Trades
|
||
|
||
`obb.cftc.cds_index_trades` is the credit default swap index tape: one row per disseminated
|
||
print, curated and fully typed from the credits file's 110 raw columns.
|
||
|
||
It is deliberately **not** a curve. There is no term structure to build — on 2026-07-15,
|
||
1,266 of the day's 1,338 index prints were the 5Y, against 7 at the 10Y and 7 at the 3Y. The
|
||
5Y *is* the market, and a bootstrap over the rest would be inventing a curve out of a handful
|
||
of prints.
|
||
|
||
```python
|
||
from openbb import obb
|
||
|
||
tape = obb.cftc.cds_index_trades(index="CDX.NA.HY", tenor="5Y", date="2026-07-15")
|
||
print(tape.to_df())
|
||
```
|
||
|
||
| index | maturity_date | tenor | coupon | upfront_amount | upfront_currency | notional_amount | index_factor | venue |
|
||
|:--|:--|:--|--:|--:|:--|--:|--:|:--|
|
||
| CDX.NA.HY | 2031-06-20 | 5Y | 0.05 | 1,241,583.75 | USD | 15,000,000 | 0.99 | BBSF |
|
||
|
||
A 500bp coupon, 1.24mm paid upfront on 15mm of protection, and an index factor of 0.99 — 1% of
|
||
the notional already written off by a default. That is the trade, priced without a spread.
|
||
|
||
### A CDS index is priced by coupon plus upfront
|
||
|
||
This is the whole point of the endpoint, and it comes straight from the technical
|
||
specification rather than from convention or inference.
|
||
|
||
For the credit asset class, `Other payment type` (#81) carries the validation rule:
|
||
|
||
> C, at least one is required: ([Fixed rate] or [Spread] or [Other payment type] = 'UFRO').
|
||
|
||
and `UFRO` is defined as:
|
||
|
||
> Upfront Payment, i.e., the initial payment made by one of the counterparties either to bring
|
||
> a transaction to fair value or for any other reason that may be the cause of an off-market
|
||
> transaction
|
||
|
||
The two price-carrying fields are conditional *on each other*. For credit, `Fixed rate` (#91)
|
||
is required only:
|
||
|
||
> if [Spread] is not populated and [Other payment type] ≠ 'UFRO' … else {blank}
|
||
|
||
and `Spread` (#100), symmetrically:
|
||
|
||
> if [Fixed rate] is not populated and [Other payment type] ≠ 'UFRO' … else {blank}
|
||
|
||
The specification is explicit that the generic `Price` element does not apply here. `Price`
|
||
(#93) states it is not applicable to:
|
||
|
||
> Credit default swaps and credit total return swaps, as it is understood that the information
|
||
> included in the data elements Fixed rate, Spread and Upfront payment (Other payment type:
|
||
> Upfront payment) may be interpreted as the price of the transaction.
|
||
|
||
The data agrees: `Price` is populated on **0 of 1,338** index prints. The three fields the spec
|
||
names are what price the trade, and they combine as the rule above dictates:
|
||
|
||
| Fixed rate | Spread | Upfront (UFRO) | Prints |
|
||
|:--:|:--:|:--:|--:|
|
||
| ✓ | ✓ | ✓ | 887 |
|
||
| ✓ | | ✓ | 212 |
|
||
| ✓ | | | 91 |
|
||
| ✓ | ✓ | | 79 |
|
||
| | ✓ | | 61 |
|
||
| | | ✓ | 8 |
|
||
|
||
Every print carries at least one of the three, exactly as the rule requires — and 1,269 of
|
||
1,338 carry the standardised coupon. The coupons are the index conventions and nothing else:
|
||
CDX.NA.HY and ITRAXX EUROPE CROSSOVER at `0.05`, CDX.NA.IG and ITRAXX EUROPE at `0.01`,
|
||
CMBX.NA.BBB- at `0.03`, CMBX.NA.AAA at `0.005`. So `coupon` alone never distinguishes two
|
||
trades in the same index; `upfront_amount` is what carries the price.
|
||
|
||
### Why `spread` is not the traded level
|
||
|
||
`spread` is disseminated, so it is surfaced — raw, as reported. But reading it as "the CDS
|
||
spread" is wrong twice over.
|
||
|
||
**First, the spec makes it a conditional alternative,** not a price: it is required only when
|
||
there is no fixed rate and no upfront. It is populated on 77% of index prints, against 95% for
|
||
the coupon.
|
||
|
||
**Second, the field does not always contain a spread at all.** Per the spec, notation `3` means
|
||
"value expressed as decimal (e.g., 0.0257 instead of 2.57%)" and notation `1` means a monetary
|
||
amount. On the price-quoted indices the reported number is the **price in points of par**, and
|
||
the notation merely rescales it:
|
||
|
||
| Index | Notation 3 (decimal) | Notation 1 (monetary) | What it actually is |
|
||
|:--|:--|:--|:--|
|
||
| CDX.NA.HY | 0.010788 – 0.010810 | 107.93 – 108.03 | price, ~108 points of par |
|
||
| CDX.EM | 0.009819 – 0.009825 | 98.21 – 98.25 | price, ~98 points of par |
|
||
| CDX.NA.IG | 0.001676 – 0.009121 | *(not reported)* | spread, 17 – 91bp |
|
||
| ITRAXX EUROPE CROSSOVER | 0.013345 – 0.027500 | *(not reported)* | spread, 133 – 275bp |
|
||
|
||
The two notations hold the same number, scaled by 10,000. **This is where "CDX.NA.HY trades at
|
||
108bp" comes from, and it is a misreading**: 108 is the price. High yield trades on upfront
|
||
against a 500bp coupon, and a 108bp spread against a 500bp coupon is not a thing the market
|
||
could produce.
|
||
|
||
So: read `spread` together with `spread_notation`, per index, or use `coupon` and
|
||
`upfront_amount`, which mean the same thing on every row.
|
||
|
||
### The index name has no series — maturity pins it
|
||
|
||
`index` comes from `UPI Underlier Name`, which names the **family only**: `CDX.NA.IG`, never
|
||
`CDX.NA.IG.45`. The elements that might have carried a series are not disseminated here —
|
||
`Underlying Asset Name` and `Underlier ID-Leg 1` are populated on **0 of 1,338** index prints.
|
||
|
||
`maturity_date` (the spec's `Expiration date`, #142) is therefore load-bearing: it is the only
|
||
field separating one series from another, and it is surfaced for exactly that reason.
|
||
|
||
```python
|
||
obb.cftc.cds_index_trades(index="CDX.NA.IG", tenor="5Y", date="2026-07-15")
|
||
```
|
||
|
||
returns both the on-the-run 5Y maturing 2031-06-20 and the previous series maturing
|
||
2030-12-20. Both are "the 5Y"; only the maturity tells them apart.
|
||
|
||
`tenor` is **derived, not disseminated**: it is the years from `execution_timestamp` to
|
||
`maturity_date`, snapped to the nearest standard CDS index tenor within one year. Two
|
||
consequences worth knowing:
|
||
|
||
- An off-the-run series keeps the tenor it is quoted at. The 2030-12-20 print above is 4.43
|
||
years from execution and is labelled `5Y`, because that is what it is.
|
||
- `Execution timestamp` "remains unchanged throughout the life of the UTI", so on a correction
|
||
or termination the tenor describes the **original** trade, not time remaining today.
|
||
- CMBX and other long-dated underliers fall outside the standard grid and keep a rounded year
|
||
count instead (`31Y`, `39Y`), rather than being forced onto a CDS tenor they do not trade.
|
||
|
||
### Scope and caveats
|
||
|
||
- **Included:** the index FISNs `NA/CDS Corp Idx`, `NA/CDS Corp Idx Tra` (tranches) and
|
||
`NA/CDS Sov Idx`. **Excluded:** single names (`NA/CDS Corp SN`), baskets, and index
|
||
swaptions (`NA/CDS Idx Swt`) — a swaption prices off a strike and premium rather than a
|
||
coupon, and its `UPI Underlier Name` carries the FISN string instead of an index name, so it
|
||
cannot be filtered by index at all. Check `upi_fisn` to separate tranche prints; the
|
||
`CDS index attachment point` and `detachment point` elements are **not** publicly
|
||
disseminated (P43 Reported = N), so a tranche cannot be identified by its attachment level.
|
||
- **There is no per-message dissemination timestamp in this data.** The spec defines one
|
||
(`Dissemination timestamp`, #D3, "to the nearest second"), but the cumulative slice does not
|
||
carry the column — the file's report date *is* the dissemination day. The endpoint therefore
|
||
reports `dissemination_date`, alongside the spec's `event_timestamp` and
|
||
`execution_timestamp`, which are per-print and exact.
|
||
- **`cleared` is `I` or `N`, never `Y`** (1,184 / 153 on the sample day). Part 43 disseminates
|
||
the original swap, and `I` is
|
||
"Intent to clear, for alpha transactions that are planned to be submitted to clearing". This
|
||
is why `venue` is well populated: `Platform identifier` is "C if [Cleared] = 'N' or 'I'; NR
|
||
if [Cleared] = 'Y'".
|
||
- Notionals above the reporting cap are disseminated at the cap and flagged by `is_capped`
|
||
(178 of 1,338 prints on the sample day). When the notional is capped the SDR must
|
||
proportionally scale the other disseminated amounts, so a capped print's `upfront_amount` is
|
||
scaled too — do not read it as dollars against the true notional.
|
||
- **`min_notional` therefore has a trap.** It compares against the *disseminated* notional, and
|
||
a capped notional is only a floor on the real one. The caps vary by index, tenor and currency
|
||
— on the sample day USD prints capped at 50mm, 76mm, 250mm and 400mm — so a threshold above a
|
||
cap silently drops capped prints that may be far larger than it. Filter on `is_capped` to see
|
||
them.
|
||
- `index` matching is **exact** and case-insensitive, not a substring match: `ITRAXX EUROPE` is
|
||
a prefix of `ITRAXX EUROPE CROSSOVER` and `ITRAXX EUROPE SENIOR FINANCIALS`, and a substring
|
||
filter would silently conflate three different indices.
|
||
- The `index` dropdown lists the families observed in a sampled day's file and is a convenience,
|
||
not a whitelist — the parameter is free text, so an index absent from the list still filters.
|
||
- The tape includes every action type (`NEWT`, `MODI`, `CORR`, `EROR`, `TERM`), because all of
|
||
them are disseminated prints — a tape that silently dropped corrections would not be the
|
||
tape. There is no `action_type` parameter; `action_type` and `original_dissemination_identifier`
|
||
are columns, so filtering to new trades is `df[df.action_type == "NEWT"]`.
|
||
- `index_factor` is below 1 once a constituent has defaulted; multiplied by `notional_amount`
|
||
it yields the notional actually covered by the seller of protection.
|
||
|
||
## Report Types
|
||
|
||
Reports can be filtered by measure, or include all reported fields. Report types come as 'Futures Only', or 'Combined' (futures and options combined positions), selectable by setting the `futures_only` boolean parameter.
|
||
|
||
### Legacy
|
||
|
||
The Legacy report is broken down by exchange with reported open interest further broken down into three trader classifications: commercial, non-commercial and non-reportable.
|
||
|
||
### Disaggregated
|
||
|
||
The Disaggregated reports are broken down by Agriculture and Natural Resource contracts. The Disaggregated reports break down reportable open interest positions into four classifications: Producer/Merchant, Swap Dealers, Managed Money and Other Reportables.
|
||
|
||
### Financial
|
||
|
||
The Traders in Financial Futures (TFF) report includes financial contracts. The TFF report breaks down the reported open interest into five classifications: Dealer, Asset Manager, Leveraged Money, Other Reportables and Non-Reportables.
|
||
|
||
### Supplemental
|
||
|
||
The Supplemental report includes 13 select agricultural commodity contracts for combined futures and options positions. Supplemental reports break down the reportable open interest positions into three trader classifications: non-commercial, commercial, and index traders.
|
||
|
||
Visit the CFTC [website](https://www.cftc.gov/MarketReports/CommitmentsofTraders/index.htm) for detailed descriptions of reports and classification methodology.
|