jerry/OpenBB
1
0
Fork 0
mirror of https://github.com/OpenBB-finance/OpenBB.git synced 2026-11-12 12:22:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

V5 (#7489)

Browse Source 
* [Feature] OpenBB V5: Initial `openbb-core` Refactor (#7482)

* initial refactor

* ruff does not dictate style

* unresolved imports in test action

* fix general linting

* resolve pandas imports for actions run

* httpx in test workflow

* pytest-asyncio in test workflow

* pandas.Index(...)

* max version for windows CI is 3.14

* remove temp refactor files that shouldn't have been committed:

* use ruff format instead of black

* formatting and typing

* tab space helper

* test fixtures..

* ty check

* also run integration test in CI

* remove that annoying labeler action

* [V5] `openbb-core`: 100% Test Coverage (#7484)

* increase test coverage to exercise 100% of the testable code path in openbb-core

* model_fields syntax cleanup

* set the environment variable for CI runs

* run codeql on v5 branch

* more CI env setup

* cleanup redundant files

* ruff

* dedupe workflow trigger

* some windows coverage missing

* ubuntu integration test job

* more windows coverage

* test_reference_generator

* test_docstring_generator

* remove line no references from test function docstrings

* clean useless comments from test files

* fix polars type check

* fix test_obbject

* fix test_obbject

* package builder fixes

* charting pyproject version

* fix docstring generator for nested models

* missing coverage

* [V5] `openbb-core`: Add Backwards-Compatible Layered Config System (openbb.toml) (#7492)

* add backwards-compatible layered config system

* fix path reference in windows test

* need python version-aware handling in docstring generator

* update for deprecated pydantic syntax

* ruff UP007

* test_system_settings

* remove v4 platform workflow

* [V5] Refactor `openbb-oecd` for full SDMX 2 API Exposure and V5 Packaging (#7413)

* stash some changes

* add more robust testing

* mypy

* point PR at V5

* introduce spec file

* codespell

* test fix

* fix workflow environment setup

* fix workflow environment setup

* fix workflow environment setup

* add pyyaml to dependencies

* split lint jobs

* fix workflow environment setup

* fix workflow environment setup

* workflow env setup

* workflow env setup

* clean up code comments

* add auth hook entrypoints

* codespell

* add codegen feature

* codespell

* move _unpack into dispatchers for consistency with codegen packages

* surface nested models in the response

* fix missing coverage in CI

* socrata updates

* test fix

* detect plotly output

* add --include and --exclude flags from generate-extension command

* refactor openbb-platform-api for V5 operations and spec file handling

* delete poetry.lock

* add test workflow

* add temp branch id for test workflow

* add httpx to dev dependencies

* add a conftest.py

* tweak conftest

* tweak conftest

* python 3.10 issues

* more python 3.10

* more python 3.10

* more python 3.10

* more python 3.10

* fix url template resolution

* multi-spec support

* fix a windows path issue

* merge branch develop

* table builder, integration tests, and more

* linting

* merge branch develop

* bit of cleanup

* some more updates, cleanup, and apps.json

* codespell

* black

* integration test params and linting

* mypy

* pylint

* black

* more pylint

* recapture teest cassettes

* black again

* more linting

* table presentation choices

* black

* fix merge conflict

* split up metadata module and reformat cache to use msgpack

* mypy

* regenerate lock file

* use json and include dependencies to make openbb-oecd a standalone install

* pylint

* more linting

* serve metadata as dependency instead of assign and initialize

* fix integration_tests_testers to not count dependency injections as paramters

* add assets to git ignore

* ruff

* v5 refactor and publish workflow

* test coverage and ty fixes

* codespell

* update lock file

* workflow setup

* ty check

* workflow setup

* Scope PR to OECD changes only

Restore non-OECD files in the diff against v5 to their v5 state. The
branch was originally created off feature/cli-2 and inherited 186 files
of unrelated changes (cli, agent_server, mcp_server, platform_api,
core/, .github/workflows/* not oecd). Those have nothing to do with the
openbb-oecd v5 refactor and were polluting the PR diff.

* scope cleanup: include leaked v5-resident files (general-linting workflow, platform_api test fixtures)

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>

* fix docstring generator and body param unwrapping

* project ty.toml file

* unpack POST parameters from params: Model

* remove comment chatter

* [V5] `openbb-oecd`: Rename `oecd_utils` as `oecd` (#7509)

* rename oecd_utils to oecd and populate all models there when openbb-economy is not installed

* linting

* bump dependencies

* fix test drift

* linter

* FieldInfo import

* [V5] `openbb-imf`: Refactor for V5 (#7508)

* refactor imf extension for V5

* codespell

* fix some python 3.10 items

* return self on copy of metadata object

* [V5] `openbb-news`: Refactor News for V5 and Add Configurable RSS Endpoint (#7507)

* refactor news for v5 and add rss endpoint

* remove poetry.lock

---------

Co-authored-by: deeleeramone <>

* [V5] `openbb-quantitative`: Refactor for V5 (#7506)

* refactor quantitative for v5

* add factors endpoint and add a few charts

---------

Co-authored-by: deeleeramone <>

* [V5] `openbb-technical`:  Refactor for V5 & Add More Quick Win Endpoints (#7502)

* refactor openbb-technical for v5

* forward refs

* linting

* no python 3.15 in test matrix

* reame

* ty

* move all params to body

---------

Co-authored-by: deeleeramone <>

* remove stockgrid provider (#7486)

* [V5] remove wsj provider and related code (#7485)

* remove wsj provider and related code

* purge from metadata

* actions

* actions

* openbb-core publish workflow

* delete biztoc (#7523)

* [V5] Remove `openbb-alpha-vantage` (#7522)

* delete alpha vantage

* more files

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* [V5] Remove `openbb-benzinga` (#7521)

* delete benzinga

* more references

---------

Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* [V5] `openbb-bls`: Refactor for V5 + Add Many Endpoints (#7518)

* refactor bls for v5, add 100 widgets

* some CI fixes

* no python 3.15

* fix missing dev group in pyproject.toml

* more pyproject fallout

* action

* refactor econometrics for v5 (#7505)

Co-authored-by: deeleeramone <>

* [V5] `openbb-famafrench`: Refactor for V5 (#7503)

* refactor famafrench for v5

* codespell

* finishing touches

---------

Co-authored-by: deeleeramone <>

* [V5] CLI 2.0 - Pluggable Backends, Non-TTY Default, HTTP Dispatcher, Spec Files + Codegen (#7433)

* stash some changes

* add more robust testing

* mypy

* point PR at V5

* introduce spec file

* codespell

* test fix

* fix workflow environment setup

* fix workflow environment setup

* fix workflow environment setup

* add pyyaml to dependencies

* split lint jobs

* fix workflow environment setup

* fix workflow environment setup

* workflow env setup

* workflow env setup

* clean up code comments

* add auth hook entrypoints

* codespell

* add codegen feature

* codespell

* move _unpack into dispatchers for consistency with codegen packages

* surface nested models in the response

* fix missing coverage in CI

* socrata updates

* test fix

* detect plotly output

* add --include and --exclude flags from generate-extension command

* cap test matrix at python 3.14

* no useless comments

* platform controller command description split

* merge URL overloads from path params

* exclude none and unset from model dump

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>

* general-linting

* downgrade aiohttp in devtools for vcrpy

* some ty items and add requests-types stubs for ty in devtools

* markdown linting

* remove integration tests from extensions that were moved into the famafrench provider

* [V5] `openbb-congress-gov`: Refactor for V5 + Convert To Using Bulk Files Instead of `api.congress.gov`. (#7528)

* refactor congress-gov for v5 and use bulk files instead of API with credentials

* integration tests

* apps tab name

* convert caching to sql

* codespell

* [V5] [BugFix] Fix for Breaking Changes in FastAPI 0.137+ (#7537)

* fix route introspection for fastapi 0.137

* recapture oeccd gdp forecast test cassette

---------

Co-authored-by: deeleeramone <>

* [V5] `openbb-platform-api`: Refactor for V5 Spec, Config, Packaging, Test Coverage (#7494)

* stash some changes

* add more robust testing

* mypy

* point PR at V5

* introduce spec file

* codespell

* test fix

* fix workflow environment setup

* fix workflow environment setup

* fix workflow environment setup

* add pyyaml to dependencies

* split lint jobs

* fix workflow environment setup

* fix workflow environment setup

* workflow env setup

* workflow env setup

* clean up code comments

* add auth hook entrypoints

* codespell

* add codegen feature

* codespell

* move _unpack into dispatchers for consistency with codegen packages

* surface nested models in the response

* fix missing coverage in CI

* socrata updates

* test fix

* detect plotly output

* add --include and --exclude flags from generate-extension command

* refactor openbb-platform-api for V5 operations and spec file handling

* delete poetry.lock

* add test workflow

* add temp branch id for test workflow

* add httpx to dev dependencies

* add a conftest.py

* tweak conftest

* tweak conftest

* python 3.10 issues

* more python 3.10

* more python 3.10

* more python 3.10

* more python 3.10

* fix url template resolution

* multi-spec support

* fix a windows path issue

* cap test matrix at python 3.14

* no 3.15 runners for platform-api

* no useless comments

* platform controller command description split

* merge URL overloads from path params

* exclude none and unset from model dump

* markdown lint

* ignore get/post endpoints that are return SSE

* fix remaining fastapi 0.137 fallout

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>

* add full support for AsyncIterator response type for streaming output (#7530)

* [V5] Refactor Shared Pass Through Router Extensions For Hatchling (#7524)

* refactor shared pass-through routers for hatchling and ruff + ty

* add publish workflows

* [V5] `openbb-charting`: Refactor for V5 and Decouple  (#7545)

* refactor charting for v5 patterns and decouple entirely

* coveragerc

* lint and tests

* fix tests

* configure dev group with openbb-technical

* ty

* fix cli test

* remove 3.15 from test matrix - dependencies are unstable

---------

Co-authored-by: deeleeramone <>

* fix merge widgets

* workflow cleanup"

* fixes for ty

* fogot that file

* general lint setup

* consolidate lint job

* remove seeking alpha provider (#7570)

* remove tradier provider (#7573)

* remove tiingo provider (#7574)

* remove intrinio provider (#7575)

Remove the openbb-intrinio provider extension from the repository.
Reason: proprietary vendor with no internal access to any endpoint;
cannot build, develop or maintain.

* remove yfinance provider from repo (#7576)

* fix(cli): resolve modular OpenAPI references (#7586)

Fixes #7585

* [V5] `openbb-cli`: Modular OpenAPI Refs (#7589)

* follow up to #7585

* add another codegen test

* unused ty ignore

* Update generated tests to get python parameter names instead of raw API-provider parameter names. (#7597)

* Update the record-vs-envelope heuristic in the payload processing code to not treat a record that happens to have a list datatype as an entire envelope. (#7598)

* Update class generation code to handle empty data that php-style serialization emits as []. (#7599)

* fix ty check failures exposed by newer version

* add drugs.com and brutalist as news rss feeds (#7602)

Co-authored-by: deeleeramone <>

* codespell

* missing test coverage that appeared in famafrench..

* plotly 7 handling for mapbox namespace change

* remove finviz (#7577)

* remove trading economics provider (#7578)

* remove openbb-multpl from repository (#7610)

* add httpx2 to devtools for starlette test client

* restore test coverage to 100% in openbb_core

* remove fmp provider extension from repository (#7665)

* unpin ruff

* fix charting conftest and missing coverage in 3.10

* add charting coverage

* fix new ty issues

* fix tag grouping in swagger docs

* [V5] `openbb-ecb`: Refactor for V5 and Expand Coverage (#7560)

* refactor openbb-ecb for v5 and expand coverage

* some updates

* tests

* workflow

* remove 3.15 from test matrix

* some more table schema updates

* remove some html chrome on embedded articles

---------

Co-authored-by: deeleeramone <>

* [V5] `openbb-federal-reserve`: Refactor for V5 + FFIEC & Regional Fed Routers (#7561)

* refactor for v5

* add workflows

* fix bhcpr schema

* fix pyproject.toml

* fix test

* codespell

* ty

* fix pyproject

* fix ci environment

* fix ci environment

* convert documents endpoints to use fedinprint service

* codeql fix

* fix ny publications and add ny nowcast

* [V5] `openbb-eia`: Refactor for V5 + Add Full API Implementation (#7584)

* eia refactor for v5

* add test and publish workflows

* data browser state

* fix workflows

* more workflow test items

* temp files in tests

* more fixes

* more browsers

* fix codeql issues

* fix mer state and ensure multi-user capability

* uv lock

* table bridge

* fix electricity browser state

* convert HTML widgets to iFrame

* fix recursion error in workspace mcp

* [V5] `openbb-government-us`: Refactor for V5 + Move `uscongress` (#7604)

* v5 government_us provider

* workflow files

* ty and add xlrd to pyproject

* shrink test cassettes

* dev dependencies

* wheel packaging and more tests

* python 3.10 thing..?

* fix test

* default state for multiselect dropdowns

* replace congress html widgets with iframes

* fallback to congress api for congress before 108

* ensure member passage voting count populates

* [V5] Feature: Add JODI World Oil & Gas Database (#7593)

* add the JODI oil and gas database as a provider/router extension

* lint fixes

* add grouping to apps.json

---------

Co-authored-by: deeleeramone <>

* [V5] `openbb-cftc`: Refactor for V5 + Add OIS, Forward Curve, and FX Trade Analysis (#7615)

* refactor openbb-cftc for v5

* add SHIBOR to fixings

---------

Co-authored-by: deeleeramone <>

* fix for plotly 7 breaking change to schemas

* [V5] `openbb-tmx`: Refactor extension for V5 (#7668)

* refactor tmx provider for v5

* fix tests..

* fix tests stack trace exposure

* fix workflow setup

* [V5] `openbb-nasdaq`: Refactor provider for V5. (#7670)

* refactor nasdaq provider for v5

* add workflows

* ty fix

* [V5] `openbb-fred`: Refactor as V5 extension. (#7669)

* refactor fred extension for v5

* fix markdown MD037 lint error in README

* codespell

* [V5] `openbb-cboe`: Refactor for V5 + Add Payoff Diagrams (#7617)

* refactor cboe for v5

* use zoneinfo instead

* fix test

* some fixes related to the cache access

* fix a couple of tests

* fix the test coverage

* [V5] `openbb-deribit`: Refactor provider for V5 (#7674)

* refactor openbb-deribit for v5

* restructure the optimizer around the published combos data and add integration test suites

* fix markdown lint errors

* remove openbb-econdb provider (#7675)

* codespell

* [V5] `openbb-sec`: Refactor for V5 (#7525)

* refactor openbb-sec for v5

* linting

* ty

* normalize compare_company_facts

* add workflows

* python 3.10 fix

* add more thorough url validation and tests

* ty check

* ty for python 3.10

* reconfigure with apps

* temp pyproject overrride

* pyproject

* pyproject

* pyproject

* codespell

* pyproject

* fix uk schema file links in schema_files

* test coverage

* test_cli

* port #7547 into this pr

* no hard coded prefix

* improve filing rendering capabilities

* fix xml text detection

* make codeql happy

* missed a file in the commit

* test coverage

* codespell

* fix flakey test

* some updates

* management_ownership --> management_profiles

* proxy statement markdown

* ruff

* fix test coverage

* more tests

* test coverage

* fix lint errors

* Add SEC adviser search, profile, and universe routes (#7559)

* Add SEC institutional data fetchers

* Add SEC filing index and Form D routes

* Add SEC investment adviser people data

* Expose SEC adviser and institutional routes

* fix(sec): address adviser review feedback

* refactor(sec): narrow adviser support to IAPD search

* feat(sec): add adviser firm and individual searches

* feat(sec): add adviser profile drill-down

* fix: repair pre-commit configuration

* feat(sec): add adviser universe route

* feat(sec): expose complete Form ADV data

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>

* fix test coverage

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>

* [V5] `openbb-finra`:  Refactor for V5 (#7676)

* refactor finra provider for v5

* add workflows

* fix tests

* fix lint errors

* [V5] Update Repository License to Apache 2.0 (#7677)

* change license to Apache 2.0 for V5

* codespell

* bump devtools and repoint pyproject.toml

* Enterprise Flask Migration Toolkit (#7264)

* feat: Add Flask-to-OpenBB enterprise migration toolkit

- Implement Router.from_flask() for zero-touch legacy API conversion
- Enable 4-hour migration vs 6-month traditional rewrite (80% cost reduction)
- Generate FastAPI endpoints from Flask routes with automatic type validation
- Support enterprise consulting services and custom integration workflows

Demonstrates OpenBB's competitive advantage for institutional client migrations.
Complements existing Workspace applications like Open Portfolio suite.

* chore: Resolve .gitignore conflict and add common ignores

* refactor: implement Phase 1 Flask integration architecture- Move Flask conversion logic to openbb_core.app.utils.flask- Add Flask detection to extension_loader - Integrate with OpenBB's core extension system- Add Flask dependency validation with sys.modules checkingAddresses @deeleeramone's architectural feedback for native OpenBB integration.

* docs: add Flask direct entry point demos- demo_direct_flask_entry_point.py: shows direct Flask instance reference- example_pyproject_flask_entry.toml: demonstrates pyproject.toml configurationSupports new architecture eliminating wrapper extensions.

* Implement Phase 1 Flask adapter per code review feedback

- Minimal working implementation with Flask detection and router creation
- Removes complex code generation logic for focused Phase 1 delivery
- Establishes foundation for Phase 2 route conversion features

* Fix Flask integration: Replace missing FlaskToOpenBBAdapter with WSGIMiddleware

- Remove FlaskToOpenBBAdapter which was not implemented
- Use FastAPI's WSGIMiddleware to mount Flask apps directly
- Update extension_loader to use returned Router object
- Convert utils.py to utils/__init__.py to support flask submodule
- Addresses PR review comment about unhandled AttributeError

* Address PR review feedback: fix Flask imports and remove unused files

- Revert .gitignore to upstream/develop
- Remove AI-generated demo/example files and flask_adapter extension
- Move Flask/werkzeug imports inside conditional blocks to avoid ImportError
- Use lazy imports via __getattr__ in utils/flask/__init__.py
- Add Flask tests with @pytest.mark.skipif markers

* Force APIRouter.include_router to propagate Mount paths instead of dropping them. (#1)

This change ensures that mounted ASGI applications (such as Flask apps wrapped in WSGIMiddleware) are not dropped when `APIRouter.include_router` is called.

Previously, while the initial load in `ExtensionLoader` worked as expected, the `Mount` objects were ignored by FastAPI's default inclusion logic. This caused the mounted elements to be lost during the final application assembly, resulting in 404 errors for the Flask routes.

This fix overrides the inclusion logic to manually propagate `Mount` objects downstream.

* feat(flask): Add metadata layer for OpenAPI documentation

Enhance Flask-to-OpenBB converter with structured docstring parsing:
- Parse Google-style docstrings (summary, description, params)
- Extract parameter descriptions from Args section
- Pass metadata to Router.command() for rich OpenAPI docs

This addresses PR feedback to provide tangible benefit by generating
comprehensive API documentation for migrated Flask routes.

* feat(flask): Implement manual OpenAPI spec generation and attachment

- Add FlaskParam and FlaskRouteInfo TypedDict for type-safe metadata
- Implement OpenAPISpecGenerator for Flask→OpenAPI conversion
- Extract route metadata via FlaskIntrospector
- Attach generated spec to router.openbb_openapi_spec_ext
- Add unit tests (3/3 passing)

Addresses WSGI/mount boundary issue where Router.command metadata
cannot propagate through WSGIMiddleware. Spec attached to router
instance for downstream OpenAPI assembly.

Resolves maintainer feedback on PR #7264

* feat(flask): Implement FlaskMountRegistry for multiple mount support

- Add FlaskMountRegistry class for centralized OpenAPI spec management
- Replace router.openbb_openapi_spec_ext with registry pattern
- Support multiple Flask apps at different mount prefixes
- Automatic path rewriting in aggregated OpenAPI specs
- Add registry test case (4/4 tests passing)

Addresses @deeleeramone's feedback on PR #7264 regarding router
attribute fragility and multiple mount support. Registry provides
lifecycle-independent state management and enables enterprise
scenarios with multiple Flask apps.

Resolves: OpenBB-finance/OpenBB#7264 (partial)

* feat(core): load Flask apps as openbb_core_extension WSGI sub-apps

A Flask app referenced directly by an openbb_core_extension entry point is
mounted via WSGIMiddleware and its routes are introspected into openapi.json
(operationId/widgetId/mcp_tool, parameters, request bodies, security, widget_config
and HTML widgets) so the widget factory and MCP server can discover them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(core): 100% coverage for flask utils + flask dev dependency

Add branch-coverage tests for the Flask integration package (introspection,
AST analysis, docstrings, OpenAPI generation, registry, loader, detection) and
declare flask as a dev dependency so the suite runs in CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flask): treat f-string returns as HTML responses

* fix(flask): mount via a2wsgi instead of deprecated starlette WSGIMiddleware

- add openbb-core[flask] extra (flask, a2wsgi); a2wsgi resolved through require_optional
- run the WSGI serve test through mount_flask_extensions; drop httpx importorskip
- core CI installs [pandas,flask] and httpx2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(flask): cover failed entry point loads; narrow optional types for ty

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

* perf(core): vectorize the date conversion in `basemodel_to_df` (#7622)

* perf(core): vectorize the date conversion in basemodel_to_df

basemodel_to_df ran three per-element Python passes over the date column of
every payload it converted. Series.apply(to_datetime) calls pandas.to_datetime
once per row, and because each call receives a lone scalar with no format hint,
pandas also re-runs its datetime format guesser on every row.

Parse the column in one call instead, and use vectorized comparisons for the
all-midnight test and the reduction to datetime.date.

The element-wise parse is kept as a fallback for one case that the vectorized
path cannot represent: a column mixing UTC offsets, such as a series crossing a
daylight-savings boundary. Parsing per element lets each value keep its own
offset and yields an object column of tz-aware Timestamps, whereas a single
to_datetime call over that column raises "Mixed timezones detected". That
behaviour is covered by test_to_df_daylight_savings and is preserved exactly.

Measured on a payload built through EquityHistoricalData, median of 9 under an
exclusive CPU lock:

  250 daily bars     8.92 ms -> 1.46 ms
  1,000 daily bars  32.93 ms -> 3.13 ms
  5,000 daily bars 162.25 ms -> 12.23 ms

* fix a broken test - unrelated to this PR...

---------

Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* fix lint errors

* fix oecd lint error

* [V5] `openbb-mcp-server`: Refactor for v5 - Spec, Config, Packaging, Test Coverage (#7495)

* stash some changes

* add more robust testing

* mypy

* point PR at V5

* introduce spec file

* codespell

* test fix

* fix workflow environment setup

* fix workflow environment setup

* fix workflow environment setup

* add pyyaml to dependencies

* split lint jobs

* fix workflow environment setup

* fix workflow environment setup

* workflow env setup

* workflow env setup

* clean up code comments

* add auth hook entrypoints

* codespell

* add codegen feature

* codespell

* move _unpack into dispatchers for consistency with codegen packages

* surface nested models in the response

* fix missing coverage in CI

* socrata updates

* test fix

* detect plotly output

* add --include and --exclude flags from generate-extension command

* refactor mcp server for v5

* test action

* fix tests

* tests

* test fixture name

* revert: drop unrelated provider/README changes from mcp-server refactor

Strip the changes to ``README.md`` and ``openbb_platform/providers/{imf,
tradingeconomics,yfinance}`` that snuck into 4ac3602 (refactor mcp server
for v5). Those files are out of scope for the mcp-server work and were
modified inadvertently — restore them to their pre-4ac3602 state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* tests and workflows

* fix workflow

* tests and workflows

* tests and workflows

* tests and workflows

* tests and workflows

* beartype shim for 3.15

* muck around with beartype for 3.15

* muck around with beartype for 3.15

* no 3.15 runners for mcp-server

* update skill files

* cap test matrix at python 3.14

* merge branch v5

* no useless comments

* platform controller command description split

* merge URL overloads from path params

* exclude none and unset from model dump

* refactor for fastmcp4

* wrap markdown lines that are too long

---------

Co-authored-by: deeleeramone <>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* update pre-commit, noxfile, and uv locks

* fixes for docstring generation

* test coverage

* fix cli renderer helper with figure object JSON

* fix the figure font when rendered

* refactor cookiecutter

* bump cookiecutter packagge version

* fix sec test by adding fixture

* fix desktop environments test

* fix(cli): inherit path-item-level OpenAPI parameters (#7606)

* fix(cli): inherit path-item-level OpenAPI parameters

OpenAPI 3.x lets a Path Item Object declare `parameters` shared by every
operation under that path, with operation-level entries overriding them by
(name, in). The codegen read only `op.get("parameters")`, so inherited
parameters were dropped silently — the generated command kept the URL
template but had nothing registered to fill it.

Real-world impact: the Xero Accounting API declares its required
`xero-tenant-id` header once per path item across all 138 paths. Before this
change `build_command_spec` produced 78 commands and none carried the
header, so every generated command 403s. Nothing warned.

Adds `operation_parameters()`, which merges the two levels and dereferences
both, and routes the pre-computed spec path (`build_command_spec`) and the
interactive path (`build_command_index` -> `build_parser_from_operation`)
through it. Unresolvable refs, non-dict entries, and nameless parameters are
skipped as before. Specs without path-item parameters are unaffected.

* fix(cli): narrow path-item inheritance to path and query parameters

Per review: headers do not translate into a Python interface function, and
outbound headers are transport configuration
(SystemSettings.PythonSettings.http.headers, the CLI's -H) or a security-scheme
concern — not a per-command argument.

Path-item `in: header` and `in: cookie` entries are now dropped rather than
inherited. Operation-level parameters are untouched, so an operation that
declares its own header keeps today's behavior.

Scope is now only what cannot be supplied any other way: `in: path` and
`in: query` parameters declared on the Path Item Object.

* fix(cli): merge allOf compositions in response schemas

Response-schema $refs were already expanded by deref_schema, but a schema
whose top level is an allOf composition was left unflattened. The properties
live inside the members, so a consumer reading them off the composition finds
none. fetcher_gen._data_schema looks for `properties.results`, gets nothing,
and falls through to _unwrap_schema_envelopes, which handles oneOf and anyOf
but not allOf. The generated Data model ends up with no fields.

Codat's listBills response is `allOf: [{properties: {results}}, {properties:
{_links, pageNumber, pageSize, totalResults}}]`, which produced an empty model.

merge_allof collapses a composition of object subschemas into one object
schema: properties unioned with the first declaration winning on conflict,
required unioned, and sibling keywords on the allOf node outranking members.
A composition that is not a plain intersection of object subschemas (a member
carrying its own oneOf/anyOf, or a non-object type) is returned unchanged
rather than flattened into something the spec does not describe. Nested
compositions are merged too, so a $ref inside a merged member still resolves.

Applied to extract_response_schema and extract_response_schemas.

Measured across five specs that use the pattern, counting operations whose
response schema exposes readable properties: 244 of 588 before, 394 after
(Codat Accounting 36 to 88, Codat Banking 0 to 8, Codat Commerce 1 to 11,
Rebilly 62 to 104, Box 145 to 183).

* fix(cli): merge allOf compositions in request body schemas

Same gap as the response side. request_body_parameters needs `type: object`
or `properties` to flatten a body into parameters, and an allOf composition
carries neither, so a POST or PUT generated from one gets no body parameters
at all.

extract_request_body_schema now runs merge_allof after deref_schema, matching
extract_response_schema.

Across the same five specs, counting request bodies that produce parameters:
138 of 206 before, 177 after (Codat Accounting 1 to 26 of its 30 bodies,
Rebilly 50 to 63, Box 87 to 88).

* fix(cli): stop annotation keywords blocking an allOf merge

_mergeable_allof_members required every member's keys to come from a small
whitelist, so any unrecognized keyword disqualified the whole composition.
Codat's PagingInfo carries `definitions` next to its properties, which meant
the live Codat-Accounting.yaml never merged and its response models stayed
empty. The APIs.guru copy of the same API omits that key, which is why the
earlier measurements looked clean.

The check is now a blocklist. A member blocks the merge when it carries its
own combinator (oneOf/anyOf/allOf/not), an unresolved $ref left by a cycle,
a scalar or array constraint (enum/const/items), or a type other than object.
Annotation keywords that do not affect the intersection (title, description,
examples, definitions, additionalProperties, vendor x- keys) no longer block
it, and are dropped from the merged result along with the members.

On the live Codat spec, operations whose response schema exposes readable
properties go from 36 of 127 to 114. Across the five specs measured earlier:
244 of 588 before, 488 after (was 394 under the whitelist).

Corpus check over 2,816 response and request-body extractions in 6 specs:
567 changed, every one attributable to an allOf in the pre-merge output,
0 unexplained. Twilio declares no allOf compositions and is unchanged.

* fix(cli): emit a name once when it is both a parameter and a body property

_signature_params appended request-body properties and then operation
parameters without deduplicating, so a name declared in both places produced
two function parameters with the same name and the generated module raised
SyntaxError.

Codat's push endpoints hit this: /push/bankAccounts/{accountId}/bankTransactions
declares accountId as a path parameter and repeats it in the request body, so
connections.push.bankAccounts.bankTransactions generated `accountId: str` and
`accountId: str = None` in one signature.

The operation parameter wins, since it carries the path placeholder and its own
required flag. The request payload is unchanged: _render_body_block derives body
fields from request_body_schema rather than from this list, so accountId is
still sent in the body.

This predates the path-item and allOf work in this PR. On origin/v5 the same
spec produces the same duplicate; on this branch the Codat spec generates 91
commands with 0 duplicated signatures, down from 1.

* fix(cli): use the safe identifier for keyword-named body properties

_render_body_block already writes `{'from': from_}`, keying the payload by the
wire name and reading a safe identifier, but _signature_params declared the raw
name. A body property named after a Python keyword therefore emitted
`from: str = None` and the generated module raised SyntaxError.

This did not fire before the allOf work in this PR, because a composed request
body produced no body parameters at all. Codat's Transfer body is such a
composition and has a `from` property, so connections.push.transfers started
generating invalid syntax once the body flattened.

_signature_params now emits safe_field_name(name)[0], the same helper
fetcher_gen and pydantic_gen already use. The body/query split in
_render_body_block compares against the safe names too: it previously matched
signature names against wire names, so `from_` was not recognized as a body
field and leaked into the query string as a `from_` parameter.

End to end on Codat-Accounting.yaml, generating the full extension and byte
compiling every emitted module: 136 files, 0 failures. origin/v5 emits 1 file
that does not compile (duplicate argument 'accountId').

Known gap left alone: an operation parameter named after a keyword would still
break, since the query and path lines use the raw name as a local variable.
None of the five specs checked declares one.

* fix(cli): do not treat instance data as schema, and sanitize POST identifiers

Two defects found by generating extensions from real specs and byte compiling
every emitted module.

merge_allof and deref_schema walked every value in a schema dict, including
keywords that hold instance data rather than subschemas. A `default` of
`{"allOf": [{"a": 1}, {"b": 2}]}` was rewritten to `{"type": "object"}`, and a
`$ref` key inside an example was resolved as if it were a reference. Both now
copy `default`, `example`, `examples`, `enum`, `const` and vendor `x-` values
through untouched.

_signature_params emitted operation parameter names verbatim, so a wire name
that is not a Python identifier produced invalid syntax. Rebilly declares an
`Organization-Id` header and a `REB-APIKEY` security header, which emitted
`REB-APIKEY: str = None`. It now uses safe_field_name, matching fetcher_gen,
and _render_body_block keys the query dict by the wire name while reading the
safe identifier so the request is unchanged.

This surfaced through the allOf work rather than being caused by it: a composed
request body used to yield no properties, so `has_body` was false in
package_gen and the command was generated as a GET fetcher, where identifiers
were already sanitized. With bodies flattening, 14 of Codat's 24 POST endpoints
correctly move to the POST path, which had never been exercised by these specs.

Generating the full extension and compiling every module:

  Codat-Accounting.yaml   v5: 1 failure    branch: 0   (136 files)
  Rebilly 2.1             v5: 14 failures  branch: 1   (97 files)

The one remaining Rebilly failure is `async def 3dsecure(` in the router, which
fails identically on v5. Sanitizing that is a naming decision about the command
surface, so it is left alone here.

Known gap: a non-credential header parameter is still placed in the query
string on the POST path. Pre-existing, and unchanged by this PR.

* fix(cli): resolve signature collisions on the emitted identifier

The previous dedupe compared wire names, but the failure it prevents depends on
the sanitized identifier. Two wire names can collapse onto one (`from` and
`from_`, `Organization-Id` and `Organization_Id`), and operation_parameters
keeps a name declared twice under different `in` locations by design, so both
cases could still emit a duplicate argument.

_signature_params now tracks the identifiers it has emitted and skips anything
that would repeat one. Emission order is unchanged, so existing signatures are
byte-identical; only genuinely colliding names are dropped, and operation
parameters still win over body properties.

Neither collision occurs in Codat-Accounting.yaml or Rebilly 2.1. This closes
the class rather than the instance, since the instances are what this PR has
been finding.

Generation unchanged: Codat 136 files 0 failures, Rebilly 97 files 1 failure
(`async def 3dsecure(`, identical on v5).

* test(cli): cover the instance-data guard, and drop a redundant branch

Mutation checking the new tests (revert each fix, confirm its test fails)
found two problems with the previous commits.

The guard that stops merge_allof and deref_schema from walking into `default`,
`example`, `enum`, `const` and vendor values shipped with no test at all. It is
now covered from both directions: a default holding an `allOf` key keeps its
value, and a `$ref` inside a default stays a literal while a real reference
beside it still resolves.

The body-property loop also carried a redundant check. It tested the raw wire
name against the operation parameter names before testing the sanitized one,
but `operation_param_names` holds both forms and `safe_field_name` is idempotent
on identifiers, so the first test could never fire alone. Reverting it left
every test passing, which is what surfaced it. Removed, and the sanitized name
is now computed once.

All nine mutations are caught by the suite: both data guards, the allOf
blocklist in each direction, the response merge wiring, body/parameter
precedence, the keyword identifier, the body/query split and the
emitted-identifier dedupe.

cli/tests at 1,954 passed against the same 80 environment failures as the base.
Generation unchanged: Codat 136 files 0 failures, Rebilly 97 files 1 failure.

* fix(cli): route required path-item headers/cookies through credential handling

* fix(cli): narrow Popen.stdin/stdout for ty in run_in_obb fixture

---------

Co-authored-by: deeleeramone <>

* fix warnings raised by federal_reserve on some xlsx workbook parsing

* fix publish workflows

* test publish pipeline with openbb-core-v2.0.0rc0 (#7681)

* fix publish workflows with missing tag job, some tests and bump fastapi

* fix python 3.10 test failure in core

* actually fix python 3.10 test failure in core

* deincrement openbb-core version to allow release pattern

* bump core version (#7682)

* remove uv.sources from pyproject.toml files and point to openbb-core-2.0.0 on PyPI

* remove oecd cache asset that wasn't supposed to be commuitted

* remove poetry.lock, update pyproject.tml and uv.lock for openbb-platform-api-v2.0.0 (#7684)

Co-authored-by: deeleeramone <>

* add the missing publish-openbb-platform-api.yml workflow file

* remove poetry.lock, update pyproject.tml and uv.lock for openbb-platform-api-v2.0.0 (#7685)

Co-authored-by: deeleeramone <>

* fix the tag generation after publish

* [Release] `openbb-charting-v4.0.0` (#7686)

* up openbb-charting to 4.0.0 and fill missing test coverage in chart_style

* test coverage and format markdown files

---------

Co-authored-by: deeleeramone <>

* remove scikit-learn from dependencies, update tests, and format markdown files (#7687)

Co-authored-by: deeleeramone <>

* [Release] `openbb-cli-v2.0.0` (#7692)

* fix missing test coverage in openbb_cli

* fix conftest

---------

Co-authored-by: deeleeramone <>

* add missing publish-openbb-cli.yml workflow

* [Release] `openbb-cli-v2.0.0` (#7693)

* fix missing test coverage in openbb_cli

* fix conftest

---------

Co-authored-by: deeleeramone <>

* test fixes for cli...

* [Release] `openbb-cli-v.2.0.0` (#7694)

* fix missing test coverage in openbb_cli

* fix conftest

---------

Co-authored-by: deeleeramone <>

* fix hanging mcp server test

* [Release] `openbb-mcp-server-v2.0.0` (#7695)

Co-authored-by: deeleeramone <>

* update uv.lock (#7696)

Co-authored-by: deeleeramone <>

* release openbb-commodity-v2.0.0 (#7697)

Co-authored-by: deeleeramone <>

* update uv.lock (#7698)

Co-authored-by: deeleeramone <>

* update uv.lock (#7699)

Co-authored-by: deeleeramone <>

* update uv.lock (#7700)

Co-authored-by: deeleeramone <>

* [Release] `openbb-econometrics-v2.0.0` (#7701)

* update uv.lock

* update pyproject.toml

---------

Co-authored-by: deeleeramone <>

* update uv.lock (#7703)

Co-authored-by: deeleeramone <>

* update uv.lock (#7702)

Co-authored-by: deeleeramone <>

* update uv.lock (#7704)

Co-authored-by: deeleeramone <>

* update uv.lock (#7705)

Co-authored-by: deeleeramone <>

* update uv.lock (#7706)

Co-authored-by: deeleeramone <>

* [V5] Fix a Number of V5 Bugs (#7707)

* fix a number of v5 bugs

* fix lint error

* more lint

* fixes for quantitative, technical, charting, famafrench for version bumps

---------

Co-authored-by: deeleeramone <>

* update uv.lock (#7709)

Co-authored-by: deeleeramone <>

* update uv.lock (#7708)

Co-authored-by: deeleeramone <>

* update uv.lock (#7710)

Co-authored-by: deeleeramone <>

* update uv.lock (#7711)

Co-authored-by: deeleeramone <>

* [Release] `openbb-cboe-v2.0.0` (#7713)

Co-authored-by: deeleeramone <>

* update uv.lock (#7714)

Co-authored-by: deeleeramone <>

* update uv.lock (#7717)

Co-authored-by: deeleeramone <>

* [Release] `openbb-ecb-v2.0.0` (#7715)

* update uv.lock

* update uv.lock

---------

Co-authored-by: deeleeramone <>

* update uv.lock (#7718)

Co-authored-by: deeleeramone <>

* update uv.lock (#7719)

Co-authored-by: deeleeramone <>

* update uv.lock (#7720)

Co-authored-by: deeleeramone <>

* add missing publish-openbb-fred.yml to repo

* update uv.lock (#7723)

Co-authored-by: deeleeramone <>

* update uv.lock (#7722)

Co-authored-by: deeleeramone <>

* update uv.lock (#7716)

Co-authored-by: deeleeramone <>

* update uv.lock (#7724)

Co-authored-by: deeleeramone <>

* update uv.lock (#7726)

Co-authored-by: deeleeramone <>

* [Release] `openbb-fred-v2.0.0` (#7725)

* update uv.lock

* fix tests and dependencies

* fix test_fred_fetchers

---------

Co-authored-by: deeleeramone <>

* [Release] `openbb-imf-v3.0.0` (#7727)

* update uv.lock

* fix tests and dependencies

* fix test_fred_fetchers

* bump openbb-imf version to 3.0.0

---------

Co-authored-by: deeleeramone <>

* update uv.lock (#7729)

Co-authored-by: deeleeramone <>

* update uv.lock (#7728)

Co-authored-by: deeleeramone <>

* update uv.lock (#7730)

Co-authored-by: deeleeramone <>

* update uv.lock (#7731)

Co-authored-by: deeleeramone <>

* update uv.lock (#7732)

Co-authored-by: deeleeramone <>

* update uv.lock (#7733)

Co-authored-by: deeleeramone <>

* cookiecutter pyproject and lock

* update pyproject.toml (#7734)

Co-authored-by: deeleeramone <>

* update pyproject.toml (#7735)

Co-authored-by: deeleeramone <>

* update uv.lock (#7736)

Co-authored-by: deeleeramone <>

* fix some sec issues, allegedly (#7737)

Co-authored-by: deeleeramone <>

* update uv.lock (#7738)

Co-authored-by: deeleeramone <>

* update main pyproject.toml

* lint fixes

* update assets (#7739)

Co-authored-by: deeleeramone <>

* [Release] `openbb-v5.0.0` (#7740)

* update assets

* add missing workflow

---------

Co-authored-by: deeleeramone <>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: deeleeramone <>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: Dex <i@dex.moe>
Co-authored-by: Monte Hoover <monte.b.hoover@gmail.com>
Co-authored-by: Boris Li <65254053+BorisQuanLi@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Lucas Zimmermann <lucaszmayer@gmail.com>
This commit is contained in:
Danglewood
2026-09-29 14:45:53 +00:00
committed by GitHub
parent 3e071fcc2c
commit bbf1ab2020
4360 changed files with 1340229 additions and 1396500 deletions
Download Patch File Download Diff File Expand all files Collapse all files

601
cli/openbb_cli/controllers/base_controller.py
View File

@@ -11,6 +11,9 @@ from pathlib import Path
from typing import Any, Literal
import pandas as pd
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from openbb_cli.config.completer import NestedCompleter
from openbb_cli.config.constants import SCRIPT_TAGS
from openbb_cli.controllers.choices import build_controller_choice_map
@@ -20,28 +23,20 @@ from openbb_cli.controllers.utils import (
get_flair_and_username,
handle_obbject_display,
parse_unknown_args_to_dict,
print_rich_table,
system_clear,
validate_register_key,
)
from openbb_cli.session import Session
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
# pylint: disable=C0301,C0302,R0902,global-statement,too-many-boolean-expressions
# pylint: disable=R0912
controllers: dict[str, Any] = {}
session = Session()
# TODO: We should try to avoid these global variables
RECORD_SESSION = False
SESSION_RECORDED = list()
SESSION_RECORDED_NAME = ""
SESSION_RECORDED_DESCRIPTION = ""
SESSION_RECORDED_TAGS = ""
SESSION_RECORDED_PUBLIC = False
class BaseController(metaclass=ABCMeta):
@@ -62,6 +57,7 @@ class BaseController(metaclass=ABCMeta):
"reset",
"stop",
"results",
"load",
]
CHOICES_COMMANDS: list[str] = []
@@ -87,11 +83,10 @@ class BaseController(metaclass=ABCMeta):
def __init__(self, queue: list[str] | None = None) -> None:
"""Create the base class for any controller in the codebase.
Used to simplify the creation of menus.
queue: List[str]
The current queue of jobs to process separated by "/"
E.g. /stocks/load gme/dps/sidtc/../exit
Parameters
----------
queue : list[str] | None
The current queue of jobs to process separated by "/".
"""
self.check_path()
self.path = [x for x in self.PATH.split("/") if x != ""]
@@ -113,12 +108,61 @@ class BaseController(metaclass=ABCMeta):
add_help=False,
prog=self.path[-1] if self.PATH != "/" else "cli",
)
self.parser.exit_on_error = False # type: ignore
self.parser.exit_on_error = False
self.parser.add_argument("cmd", choices=self.controller_choices)
def update_completer(self, choices) -> None:
"""Update the completer with new choices."""
if session.prompt_session and session.settings.USE_PROMPT_TOOLKIT:
from openbb_cli.controllers.utils import get_data_files_for_completion
if "load" not in choices:
choices["load"] = {}
data_files = get_data_files_for_completion()
if data_files:
file_completions = {file: None for file in data_files}
choices["load"]["--file"] = file_completions
choices["load"]["-f"] = file_completions
choices["load"]["--sheet-name"] = None
choices["load"]["--register_key"] = None
choices["load"]["--help"] = None
choices["load"]["-h"] = "--help"
if "results" not in choices:
choices["results"] = {}
registry_all = session.obbject_registry.all
index_completions = {str(idx): None for idx in registry_all}
key_completions = {
(data.get("extra") or {}).get("register_key"): None
for data in registry_all.values()
if (data.get("extra") or {}).get("register_key")
}
choices["results"]["--index"] = (
index_completions if index_completions else None
)
choices["results"]["-i"] = "--index"
choices["results"]["--key"] = key_completions if key_completions else None
choices["results"]["-k"] = "--key"
choices["results"]["--chart"] = None
choices["results"]["-c"] = "--chart"
choices["results"]["--export"] = {
"csv": None,
"json": None,
"xlsx": None,
"png": None,
"jpg": None,
"db": None,
"sqlite": None,
"sqlite3": None,
}
choices["results"]["-e"] = "--export"
choices["results"]["--sheet-name"] = None
choices["results"]["--help"] = None
choices["results"]["-h"] = "--help"
self.completer = NestedCompleter.from_nested_dict(choices)
def check_path(self) -> None:
@@ -128,9 +172,10 @@ class BaseController(metaclass=ABCMeta):
raise ValueError("Path must begin with a '/' character.")
if path[-1] != "/":
raise ValueError("Path must end with a '/' character.")
if not re.match("^[a-z/]*$", path):
if not re.match(r"^[a-z0-9_\-/]*$", path):
raise ValueError(
"Path must only contain lowercase letters and '/' characters."
"Path must only contain lowercase letters, digits, '_', '-', "
"and '/' characters."
)
def load_class(self, class_ins, *args, **kwargs):
@@ -141,6 +186,7 @@ class BaseController(metaclass=ABCMeta):
if class_ins.PATH in controllers and arguments == 1:
old_class = controllers[class_ins.PATH]
old_class.queue = self.queue
old_class.update_completer(old_class.choices_default)
return old_class.menu()
return class_ins(*args, **kwargs).menu()
@@ -149,10 +195,7 @@ class BaseController(metaclass=ABCMeta):
controllers[self.PATH] = self
def custom_reset(self) -> list[str]:
"""Implement custom reset.
This will be replaced by any children with custom_reset functions.
"""
"""Implement custom reset."""
return []
@abstractmethod
@@ -162,43 +205,68 @@ class BaseController(metaclass=ABCMeta):
def parse_input(self, an_input: str) -> list:
"""Parse controller input."""
# The original regex has been improved to handle quoted strings.
# It now splits by '/' only when it's not enclosed in single or double quotes.
# This allows commands like: exe --file "folder with spaces/file.openbb"
# or exe --file 'folder with spaces/file.openbb'
file_flag = r"(\ -f |\ --file )"
up_to = r".*?"
known_extensions = (
r"(\.(xlsx|csv|xls|tsv|json|yaml|ini|openbb|ipynb|db|sqlite|sqlite3))"
)
optional_args = r"(?:\ [^/]+)*?"
file_path_pattern = f"({file_flag}{up_to}{known_extensions}{optional_args})"
placeholders: dict[str, str] = {}
placeholder_count = 0
while True:
match = re.search(pattern=file_path_pattern, string=an_input)
if match is None:
break
placeholder = f"{{placeholder{placeholder_count}}}"
placeholders[placeholder] = an_input[match.span()[0] : match.span()[1]]
an_input = (
an_input[: match.span()[0]] + placeholder + an_input[match.span()[1] :]
)
placeholder_count += 1
commands = re.split(r"/(?=(?:[^\"']*[\"'][^\"']*[\"'])*[^\"']*$)", an_input)
# Remove empty strings from the list of commands
return [cmd.strip() for cmd in commands if cmd.strip()]
result = []
for cmd in commands:
cleaned = cmd.strip()
if cleaned:
for placeholder, original in placeholders.items():
cleaned = cleaned.replace(placeholder, original)
result.append(cleaned)
if an_input.startswith("/") and (not result or result[0] != "home"):
result.insert(0, "home")
return result
def switch(self, an_input: str) -> list[str]:
"""Process and dispatch input.
Returns
----------
List[str]
list of commands in the queue to execute
-------
list[str]
List of commands in the queue to execute.
"""
actions = self.parse_input(an_input)
if an_input and an_input != "reset":
session.console.print()
# Empty command
if len(actions) == 0:
pass
# Navigation slash is being used first split commands
elif len(actions) > 1:
# Absolute path is specified
if not actions[0]:
if not actions[0]: # pragma: no cover
actions[0] = "home"
# Add all instructions to the queue
for cmd in actions[::-1]:
if cmd:
self.queue.insert(0, cmd)
# Single command fed, process
else:
try:
known_args, other_args = self.parser.parse_known_args(
@@ -210,7 +278,6 @@ class BaseController(metaclass=ABCMeta):
if RECORD_SESSION:
SESSION_RECORDED.append(an_input)
# Redirect commands to their correct functions
if known_args.cmd:
if known_args.cmd in ("..", "q"):
known_args.cmd = "quit"
@@ -260,18 +327,13 @@ class BaseController(metaclass=ABCMeta):
self.queue.insert(0, "quit")
def call_exit(self, _) -> None:
# Not sure how to handle controller loading here
"""Process exit cli command."""
self.save_class()
for _ in range(self.PATH.count("/")):
self.queue.insert(0, "quit")
def call_reset(self, _) -> None:
"""Process reset command.
If you would like to have customization in the reset process define a method
`custom_reset` in the child class.
"""
"""Process reset command."""
self.save_class()
if self.PATH != "/":
if self.custom_reset():
@@ -334,15 +396,6 @@ class BaseController(metaclass=ABCMeta):
default="",
nargs="+",
)
parser.add_argument(
"-p",
"--public",
dest="public",
action="store_true",
help="Whether the routine should be public or not",
default=False,
)
if other_args and "-" not in other_args[0][0]:
other_args.insert(0, "-n")
@@ -388,7 +441,6 @@ class BaseController(metaclass=ABCMeta):
)
return
# Check if title has a valid format
title = " ".join(ns_parser.name) if ns_parser.name else ""
pattern = re.compile(r"^[a-zA-Z0-9\s]+$")
if not pattern.match(title):
@@ -401,7 +453,6 @@ class BaseController(metaclass=ABCMeta):
global SESSION_RECORDED_NAME # noqa: PLW0603
global SESSION_RECORDED_DESCRIPTION # noqa: PLW0603
global SESSION_RECORDED_TAGS # noqa: PLW0603
global SESSION_RECORDED_PUBLIC # noqa: PLW0603
RECORD_SESSION = True
SESSION_RECORDED_NAME = title
@@ -414,8 +465,6 @@ class BaseController(metaclass=ABCMeta):
SESSION_RECORDED_TAGS += "," + tag2 if tag2 else ""
SESSION_RECORDED_TAGS += "," + tag3 if tag3 else ""
SESSION_RECORDED_PUBLIC = ns_parser.public
session.console.print(
f"[green]The routine '{title}' is successfully being recorded.[/green]"
)
@@ -431,7 +480,6 @@ class BaseController(metaclass=ABCMeta):
prog="stop",
description="Stop recording session into .openbb routine file",
)
# This is only for auto-completion purposes
_, _ = self.parse_simple_args(parser, other_args)
if "-h" not in other_args and "--help" not in other_args:
@@ -457,7 +505,6 @@ class BaseController(metaclass=ABCMeta):
title_for_local_storage,
)
# If file already exists, add a timestamp to the name
if os.path.isfile(routine_file):
i = session.console.input(
"A local routine with the same name already exists, do you want to override it? (y/n): "
@@ -481,7 +528,6 @@ class BaseController(metaclass=ABCMeta):
f"[yellow]The routine name has been updated to '{new_name}'[/yellow]\n"
)
# Writing to file
Path(os.path.dirname(routine_file)).mkdir(parents=True, exist_ok=True)
with open(routine_file, "w") as file1:
@@ -495,14 +541,12 @@ class BaseController(metaclass=ABCMeta):
"\n\n",
]
lines += [c + "\n" for c in SESSION_RECORDED[:-1]]
# Writing data to a file
file1.writelines(lines)
session.console.print(
f"[green]Your routine has been recorded and saved here: {routine_file}[/green]\n"
)
# Clear session to be recorded again
RECORD_SESSION = False
SESSION_RECORDED = list()
@@ -516,17 +560,19 @@ class BaseController(metaclass=ABCMeta):
"'OBBjects' where all execution results are stored. "
"It is organized as a stack, with the most recent result at index 0.",
)
parser.add_argument("--index", dest="index", help="Index of the result.")
parser.add_argument("--key", dest="key", help="Key of the result.")
parser.add_argument("-i", "--index", dest="index", help="Index of the result.")
parser.add_argument("-k", "--key", dest="key", help="Key of the result.")
parser.add_argument(
"--chart", action="store_true", dest="chart", help="Display chart."
)
parser.add_argument(
"--export",
default="",
type=check_file_type_saved(["csv", "json", "xlsx", "png", "jpg"]),
type=check_file_type_saved(
["csv", "json", "xlsx", "png", "jpg", "db", "sqlite", "sqlite3"]
),
dest="export",
help="Export raw data into csv, json, xlsx and figure into png or jpg.",
help="Export raw data into csv, json, xlsx, db/sqlite and figure into png or jpg.",
nargs="+",
)
parser.add_argument(
@@ -547,34 +593,35 @@ class BaseController(metaclass=ABCMeta):
results = session.obbject_registry.all
if results:
df = pd.DataFrame.from_dict(results, orient="index")
print_rich_table(
df,
show_index=True,
index_name="stack index",
session.output_adapter.display(
data=df,
title="OBBject Results",
export=False,
chart=False,
)
else:
session.console.print("[info]No results found.[/info]")
elif ns_parser.index:
try:
index = int(ns_parser.index)
obbject = session.obbject_registry.get(index)
if obbject:
handle_obbject_display(
obbject=obbject,
chart=ns_parser.chart,
export=ns_parser.export,
sheet_name=ns_parser.sheet_name,
**kwargs,
)
else:
session.console.print(
f"[info]No result found at index {index}.[/info]"
)
except ValueError:
session.console.print(
f"[red]Index must be an integer, not '{ns_parser.index}'.[/red]"
)
return
obbject = session.obbject_registry.get(index)
if obbject:
handle_obbject_display(
obbject=obbject,
chart=ns_parser.chart,
export=ns_parser.export,
sheet_name=ns_parser.sheet_name,
**kwargs,
)
else:
session.console.print(
f"[info]No result found at index {index}.[/info]"
)
elif ns_parser.key:
obbject = session.obbject_registry.get(ns_parser.key)
if obbject:
@@ -590,6 +637,295 @@ class BaseController(metaclass=ABCMeta):
f"[info]No result found with key '{ns_parser.key}'.[/info]"
)
def call_load(self, other_args: list[str]): # noqa: PLR0912
"""Load data from CSV, JSON, or Excel file."""
parser = argparse.ArgumentParser(
add_help=False,
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog="load",
description="Load data from a CSV, JSON, or Excel file in OpenBBUserData folder. "
"The loaded data will be added to the results cache.",
)
parser.add_argument(
"-f",
"--file",
dest="file",
type=str,
required=True,
help="Path to file relative to OpenBBUserData folder.",
)
parser.add_argument(
"--sheet-name",
dest="sheet_name",
type=str,
default=None,
help="Name of excel sheet to load. Only for .xlsx files. If not provided, loads first sheet.",
)
parser.add_argument(
"--register_key",
dest="register_key",
default="",
help="Key to reference data in the OBBject registry.",
type=validate_register_key,
)
ns_parser = self.parse_simple_args(parser, other_args)[0]
if ns_parser and ns_parser.file:
from openbb_core.app.model.obbject import OBBject
file_path_str = ns_parser.file
user_data_dir = Path(session.user.preferences.data_directory)
file_path = user_data_dir / file_path_str
if not file_path.exists():
session.console.print(f"[red]File not found: {file_path}[/red]")
return
try:
file_ext = file_path.suffix.lower()
if file_ext == ".csv":
df = pd.read_csv(file_path, index_col=None)
df = df.loc[:, ~df.columns.str.startswith("Unnamed:")]
obbject = OBBject(results=df)
command = f"/load -f {file_path_str}"
obbject.extra["command"] = command
if ns_parser.register_key:
if (
ns_parser.register_key
not in session.obbject_registry.obbject_keys
):
obbject.extra["register_key"] = ns_parser.register_key
else:
session.console.print(
f"[yellow]Key `{ns_parser.register_key}` already exists in the registry. "
"The `OBBject` was kept without the key.[/yellow]"
)
if session.max_obbjects_exceeded():
session.obbject_registry.remove()
session.console.print(
"[yellow]Maximum number of OBBjects reached. The oldest entry was removed.[/yellow]"
)
if session.obbject_registry.register(obbject):
session.console.print(
f"[green]Successfully loaded {len(df)} rows from {file_path.name}[/green]"
)
if hasattr(self, "_link_obbject_to_data_processing_commands"):
self._link_obbject_to_data_processing_commands() # ty: ignore[call-non-callable]
self.update_completer(self.choices_default)
session.output_adapter.display(
data=obbject,
title=f"Loaded: {file_path_str}",
export=False,
chart=False,
)
else:
session.console.print(
"[yellow]Failed to register OBBject in registry.[/yellow]"
)
elif file_ext == ".json":
df = pd.read_json(file_path)
obbject = OBBject(results=df)
command = f"/load -f {file_path_str}"
obbject.extra["command"] = command
if ns_parser.register_key:
if (
ns_parser.register_key
not in session.obbject_registry.obbject_keys
):
obbject.extra["register_key"] = ns_parser.register_key
else:
session.console.print(
f"[yellow]Key `{ns_parser.register_key}` already exists in the registry. "
"The `OBBject` was kept without the key.[/yellow]"
)
if session.max_obbjects_exceeded():
session.obbject_registry.remove()
session.console.print(
"[yellow]Maximum number of OBBjects reached. The oldest entry was removed.[/yellow]"
)
if session.obbject_registry.register(obbject):
session.console.print(
f"[green]Successfully loaded {len(df)} rows from {file_path.name}[/green]"
)
if hasattr(self, "_link_obbject_to_data_processing_commands"):
self._link_obbject_to_data_processing_commands() # ty: ignore[call-non-callable]
self.update_completer(self.choices_default)
session.output_adapter.display(
data=obbject,
title=f"Loaded: {file_path_str}",
export=False,
chart=False,
)
else:
session.console.print(
"[yellow]Failed to register OBBject in registry.[/yellow]"
)
elif file_ext in [".xlsx", ".xls"]:
sheet_name = ns_parser.sheet_name if ns_parser.sheet_name else 0
df = pd.read_excel(file_path, sheet_name=sheet_name, index_col=None)
df = df.loc[:, ~df.columns.str.startswith("Unnamed:")]
obbject = OBBject(results=df)
command = f"/load -f {file_path_str}"
if ns_parser.sheet_name:
command += f" --sheet-name {ns_parser.sheet_name}"
obbject.extra["command"] = command
if ns_parser.register_key:
if (
ns_parser.register_key
not in session.obbject_registry.obbject_keys
):
obbject.extra["register_key"] = ns_parser.register_key
else:
session.console.print(
f"[yellow]Key `{ns_parser.register_key}` already exists in the registry. "
"The `OBBject` was kept without the key.[/yellow]"
)
if session.max_obbjects_exceeded():
session.obbject_registry.remove()
session.console.print(
"[yellow]Maximum number of OBBjects reached. The oldest entry was removed.[/yellow]"
)
if session.obbject_registry.register(obbject):
session.console.print(
f"[green]Successfully loaded {len(df)} rows from {file_path.name}[/green]"
)
if hasattr(self, "_link_obbject_to_data_processing_commands"):
self._link_obbject_to_data_processing_commands() # ty: ignore[call-non-callable]
self.update_completer(self.choices_default)
session.output_adapter.display(
data=obbject,
title=f"Loaded: {file_path_str}",
export=False,
chart=False,
)
else:
session.console.print(
"[yellow]Failed to register OBBject in registry.[/yellow]"
)
elif file_ext in [".db", ".sqlite", ".sqlite3"]:
import sqlite3
from openbb_cli.controllers.utils import SQLiteTable
conn = sqlite3.connect(file_path)
try:
cursor = conn.cursor()
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' ORDER BY name"
)
tables = [row[0] for row in cursor.fetchall()]
if not tables:
session.console.print(
f"[yellow]No tables found in database: {file_path.name}[/yellow]"
)
return
loaded_count = 0
for table_name in tables:
quoted = '"' + table_name.replace('"', '""') + '"'
cursor.execute(f"SELECT COUNT(*) FROM {quoted}") # noqa: S608
row_count = cursor.fetchone()[0]
sqlite_table = SQLiteTable(
db_path=str(file_path),
table_name=table_name,
row_count=row_count,
)
obbject = OBBject(results=sqlite_table)
command = f"/load -f {file_path_str} --table {table_name}"
obbject.extra["command"] = command
base_name = file_path.stem
auto_key = f"{base_name}_{table_name}"
if ns_parser.register_key and len(tables) == 1:
if (
ns_parser.register_key
not in session.obbject_registry.obbject_keys
):
obbject.extra["register_key"] = (
ns_parser.register_key
)
else:
session.console.print(
f"[yellow]Key `{ns_parser.register_key}` already exists. "
f"Using auto-generated key: {auto_key}[/yellow]"
)
obbject.extra["register_key"] = auto_key
elif auto_key not in session.obbject_registry.obbject_keys:
obbject.extra["register_key"] = auto_key
else:
session.console.print(
f"[yellow]Key `{auto_key}` already exists in the registry. "
"The `OBBject` was kept without the key.[/yellow]"
)
if session.max_obbjects_exceeded():
session.obbject_registry.remove()
session.console.print(
"[yellow]Maximum number of OBBjects reached. The oldest entry was removed.[/yellow]"
)
if session.obbject_registry.register(obbject):
loaded_count += 1
cursor.execute(f"PRAGMA table_info({table_name})")
columns = [col[1] for col in cursor.fetchall()]
session.console.print(
f"[green]Loaded table '{table_name}': {row_count} rows, "
f"{len(columns)} columns (lazy)[/green]"
)
if loaded_count > 0:
session.console.print(
f"[green]Successfully loaded {loaded_count} table(s) from {file_path.name}[/green]"
)
if hasattr(
self, "_link_obbject_to_data_processing_commands"
):
self._link_obbject_to_data_processing_commands() # ty: ignore[call-non-callable]
self.update_completer(self.choices_default)
finally:
conn.close()
else:
session.console.print(
f"[red]Unsupported file type: {file_ext}. "
"Supported: .csv, .json, .xlsx, .xls, .db, .sqlite, .sqlite3[/red]"
)
return
except Exception as e:
session.console.print(f"[red]Error loading file: {e}[/red]")
@staticmethod
def parse_simple_args(
parser: argparse.ArgumentParser,
@@ -600,19 +936,19 @@ class BaseController(metaclass=ABCMeta):
Parameters
----------
parser: argparse.ArgumentParser
Parser with predefined arguments
other_args: List[str]
List of arguments to parse
unknown_args: bool
Flag to indicate if unknown arguments should be returned
parser : argparse.ArgumentParser
Parser with predefined arguments.
other_args : list[str]
List of arguments to parse.
unknown_args : bool
Flag to indicate if unknown arguments should be returned.
Returns
-------
ns_parser: argparse.Namespace
Namespace with parsed arguments
l_unknown_args: List[str]
List of unknown arguments
ns_parser : argparse.Namespace
Namespace with parsed arguments.
l_unknown_args : list[str]
List of unknown arguments.
"""
parser.add_argument(
"-h", "--help", action="store_true", help="show this help message"
@@ -624,7 +960,6 @@ class BaseController(metaclass=ABCMeta):
try:
ns_parser, l_unknown_args = parser.parse_known_args(other_args)
except SystemExit:
# In case the command has required argument that isn't specified
session.console.print("\n")
return None, None
@@ -640,7 +975,7 @@ class BaseController(metaclass=ABCMeta):
return ns_parser, l_unknown_args
@classmethod
def parse_known_args_and_warn( # pylint: disable=R0917
def parse_known_args_and_warn(
cls,
parser: argparse.ArgumentParser,
other_args: list[str],
@@ -654,21 +989,21 @@ class BaseController(metaclass=ABCMeta):
Parameters
----------
parser: argparse.ArgumentParser
Parser with predefined arguments
other_args: List[str]
list of arguments to parse
export_allowed: Literal["no_export", "raw_data_only", "figures_only", "raw_data_and_figures"]
Export options
raw: bool
Add the --raw flag
limit: int
Add a --limit flag with this number default
parser : argparse.ArgumentParser
Parser with predefined arguments.
other_args : list[str]
List of arguments to parse.
export_allowed : Literal["no_export", "raw_data_only", "figures_only", "raw_data_and_figures"]
Export options.
raw : bool
Add the --raw flag.
limit : int
Add a --limit flag with this number default.
Returns
----------
ns_parser:
Namespace with parsed arguments
-------
ns_parser
Namespace with parsed arguments.
"""
parser.add_argument(
"-h", "--help", action="store_true", help="show this help message"
@@ -685,10 +1020,17 @@ class BaseController(metaclass=ABCMeta):
choices_export = ["png", "jpg"]
help_export = "Export figure into png or jpg."
else:
choices_export = ["csv", "json", "xlsx", "png", "jpg"]
help_export = (
"Export raw data into csv, json, xlsx and figure into png or jpg."
)
choices_export = [
"csv",
"json",
"xlsx",
"png",
"jpg",
"db",
"sqlite",
"sqlite3",
]
help_export = "Export raw data into csv, json, xlsx, db/sqlite and figure into png or jpg."
parser.add_argument(
"--export",
@@ -699,7 +1041,6 @@ class BaseController(metaclass=ABCMeta):
nargs="+",
)
# If excel is an option, add the sheet name
if export_allowed in [
"raw_data_only",
"raw_data_and_figures",
@@ -754,24 +1095,15 @@ class BaseController(metaclass=ABCMeta):
return None
try:
# Determine the index of the routine arguments
routine_args_index = next(
(
i + 1
for i, arg in enumerate(other_args)
if arg in ("-i", "--input")
and "routine_args"
in [
action.dest
for action in parser._actions # pylint: disable=protected-access
]
and "routine_args" in [action.dest for action in parser._actions]
),
-1,
)
# Collect indices whose values should NOT be comma-split because
# the provider may accept a comma-separated string (e.g. --symbol
# AAPL,MSFT). Handles --flag value, --flag=value, -f value, and
# multi-value flags (nargs="+" / nargs="*" / nargs=N).
no_split_indices: set[int] = set()
if 0 <= routine_args_index < len(other_args):
no_split_indices.add(routine_args_index)
@@ -779,38 +1111,31 @@ class BaseController(metaclass=ABCMeta):
for i, arg in enumerate(other_args):
if not arg.startswith("-"):
continue
# Handle --flag=value by extracting the flag portion.
flag_part = arg.split("=", 1)[0] if "=" in arg else arg
for action in parser._actions: # pylint: disable=protected-access
for action in parser._actions:
if flag_part in action.option_strings and action.nargs != 0:
if "=" in arg:
# Value is embedded in the same token.
no_split_indices.add(i)
elif action.nargs in ("+", "*") or (
isinstance(action.nargs, int) and action.nargs > 1
):
# Multi-value flag: protect all consecutive
# non-flag tokens after the flag.
j = i + 1
while j < len(other_args) and not other_args[j].startswith(
"-"
):
no_split_indices.add(j)
j += 1
# Single-value flag: protect the next token.
elif i + 1 < len(other_args):
no_split_indices.add(i + 1)
break
# Split comma-separated arguments only for positional / unflagged values.
other_args = [
part
for index, arg in enumerate(other_args)
for part in ([arg] if index in no_split_indices else arg.split(","))
]
# Check if the action has optional choices, if yes, remove them
for action in parser._actions: # pylint: disable=protected-access
for action in parser._actions:
if getattr(action, "optional_choices", None):
action.choices = None
@@ -825,8 +1150,6 @@ class BaseController(metaclass=ABCMeta):
)
except SystemExit:
# In case the command has required argument that isn't specified
return None
if l_unknown_args:
@@ -841,12 +1164,9 @@ class BaseController(metaclass=ABCMeta):
an_input = "HELP_ME"
while True:
# There is a command in the queue
if self.queue and len(self.queue) > 0:
if self.queue[0] in ("q", "..", "quit"):
self.save_class()
# Go back to the root in order to go to the right directory because
# there was a jump between indirect menus
if custom_path_menu_above:
self.queue.insert(1, custom_path_menu_above)
@@ -857,11 +1177,9 @@ class BaseController(metaclass=ABCMeta):
return ["help"]
return []
# Consume 1 element from the queue
an_input = self.queue[0]
self.queue = self.queue[1:]
# Print location because this was an instruction and we want user to know the action
if (
an_input
and an_input not in ("home", "help")
@@ -871,16 +1189,13 @@ class BaseController(metaclass=ABCMeta):
f"{get_flair_and_username()} {self.PATH} $ {an_input}"
)
# Get input command from user
else:
# Display help menu when entering on this menu from a level above
if an_input == "HELP_ME":
self.print_help()
try:
prompt_session = session.prompt_session
if prompt_session and settings.USE_PROMPT_TOOLKIT:
# Check if toolbar hint was enabled
if settings.TOOLBAR_HINT:
an_input = prompt_session.prompt(
f"{get_flair_and_username()} {self.PATH} $ ",
@@ -904,19 +1219,15 @@ class BaseController(metaclass=ABCMeta):
completer=self.completer,
search_ignore_case=True,
)
# Get input from user without auto-completion
else:
an_input = input(f"{get_flair_and_username()} {self.PATH} $ ")
except (KeyboardInterrupt, EOFError):
# Exit in case of keyboard interrupt
an_input = "exit"
try:
# Allow user to go back to root
an_input = "home" if an_input == "/" else an_input
# Process the input command
self.queue = self.switch(an_input)
except SystemExit:
@@ -934,7 +1245,7 @@ class BaseController(metaclass=ABCMeta):
candidate_input = (
f"{similar_cmd[0]} {' '.join(an_input.split(' ')[1:])}"
)
if candidate_input == an_input:
if candidate_input == an_input: # pragma: no cover
an_input = ""
self.queue = []
session.console.print("\n")

272
cli/openbb_cli/controllers/base_platform_controller.py
View File

@@ -1,24 +1,94 @@
"""Platform Equity Controller."""
import os
from functools import partial, update_wrapper
from functools import lru_cache, partial, update_wrapper
from types import MethodType
from typing import Any
import pandas as pd
from openbb import obb
from openbb_charting.core.openbb_figure import OpenBBFigure
from openbb_cli.argparse_translator.argparse_class_processor import (
ArgparseClassProcessor,
)
from openbb_cli.config.menu_text import MenuText
from openbb_cli.controllers.base_controller import BaseController
from openbb_cli.controllers.utils import export_data, print_rich_table
from openbb_cli.controllers.utils import (
export_data,
extract_dataframe,
print_rich_table,
)
from openbb_cli.outputs.figure import is_plotly_figure
from openbb_cli.session import Session
from openbb_core.app.model.obbject import OBBject
session = Session()
def _is_empty_result(result: object) -> bool:
"""Return whether a command result is ``None`` or an empty collection.
Parameters
----------
result : object
The command result.
Returns
-------
bool
True for ``None`` and for empty lists, tuples, and dicts.
"""
return result is None or (isinstance(result, (list, tuple, dict)) and not result)
def _display_dict_result(result: dict, title: str, export: bool) -> pd.DataFrame:
"""Display a dict command result, rendering Plotly figure JSON as a chart.
Parameters
----------
result : dict
The command result.
title : str
The command location used as the display title.
export : bool
Whether the result is being exported instead of displayed.
Returns
-------
pd.DataFrame
The tabular form of the result for export, empty for figures and
results that cannot be tabulated.
"""
if is_plotly_figure(result):
if export:
session.console.print(
"[yellow]This command returns a chart. "
"Add --raw to export its data.[/yellow]"
)
else:
session.output_adapter.display(data=result, title=title, chart=True)
return pd.DataFrame()
try:
df = pd.DataFrame.from_dict(result, orient="columns")
except ValueError:
if not export:
session.console.print(result)
return pd.DataFrame()
print_rich_table(df=df, show_index=True, title=title, export=export)
return df
@lru_cache(maxsize=1)
def _OBBject() -> type:
"""Return ``OBBject`` lazily."""
from openbb_core.app.model.obbject import OBBject
return OBBject
@lru_cache(maxsize=1)
def _OBBStream() -> type:
"""Return ``OBBStream`` lazily."""
from openbb_core.app.model.stream import OBBStream
return OBBStream
class DummyTranslation:
"""Dummy Translation for testing."""
@@ -33,36 +103,62 @@ class PlatformController(BaseController):
CHOICES_GENERATION = True
def __init__( # pylint: disable=too-many-positional-arguments
_factory_backend: Any = None
_factory_translators: dict[str, Any] | None = None
_factory_paths: dict[str, str] | None = None
def __init__(
self,
name: str,
parent_path: list[str],
platform_target: type | None = None,
queue: list[str] | None = None,
translators: dict | None = None,
paths: dict[str, str] | None = None,
):
"""Construct a Platform based Controller."""
self.PATH = f"/{'/'.join(parent_path)}/{name}/" if parent_path else f"/{name}/"
super().__init__(queue)
self._name = name
if not (platform_target or translators):
raise ValueError("Either platform_target or translators must be provided.")
if (
translators is None
and platform_target is None
and self._factory_translators is not None
):
translators = self._factory_translators
if paths is None and self._factory_paths is not None:
paths = self._factory_paths
self._translated_target = DummyTranslation()
elif platform_target is not None:
from openbb import obb
self._translated_target = (
ArgparseClassProcessor(
target_class=platform_target,
reference=obb.reference["paths"], # type: ignore
from openbb_cli.argparse_translator.argparse_class_processor import (
ArgparseClassProcessor,
)
if platform_target
else DummyTranslation()
)
self._translated_target = ArgparseClassProcessor(
target_class=platform_target,
reference=obb.reference["paths"], # ty: ignore[not-subscriptable]
)
elif translators is not None:
self._translated_target = DummyTranslation()
else:
raise ValueError(
"PlatformController needs one of: factory-injected translators, "
"platform_target, or translators=...; got none."
)
self.translators = (
translators
if translators is not None
else getattr(self._translated_target, "translators", {})
)
self.paths = getattr(self._translated_target, "paths", {})
self.paths = (
paths
if paths is not None
else getattr(self._translated_target, "paths", {})
)
if self.translators:
self._link_obbject_to_data_processing_commands()
@@ -73,9 +169,8 @@ class PlatformController(BaseController):
def _link_obbject_to_data_processing_commands(self):
"""Link data processing commands to OBBject registry."""
for _, trl in self.translators.items():
for action in trl._parser._actions: # pylint: disable=protected-access
for action in trl._parser._actions:
if action.dest == "data":
# Generate choices by combining indexed and key-based choices
action.choices = [
"OBB" + str(i)
for i in range(len(session.obbject_registry.obbjects))
@@ -98,7 +193,7 @@ class PlatformController(BaseController):
ns_parser.data in session.obbject_registry.obbject_keys
):
obbject = session.obbject_registry.get(ns_parser.data)
if obbject and isinstance(obbject, OBBject):
if obbject and isinstance(obbject, _OBBject()):
setattr(ns_parser, "data", obbject.results)
return ns_parser
@@ -111,25 +206,25 @@ class PlatformController(BaseController):
sub_menu_translators = {}
choices_commands = []
sub_prefix = f"{self._name}_{path}_"
for translator_name, translator in self.translators.items():
if f"{self._name}_{path}" in translator_name:
new_name = translator_name.replace(f"{self._name}_{path}_", "")
if translator_name.startswith(sub_prefix):
new_name = translator_name[len(sub_prefix) :]
sub_menu_translators[new_name] = translator
choices_commands.append(new_name)
if translator_name in self.CHOICES_COMMANDS:
self.CHOICES_COMMANDS.remove(translator_name)
# Create the sub controller as a new class
class_name = f"{self._name.capitalize()}{path.capitalize()}Controller"
SubController = type(
class_name,
(PlatformController,),
{
"CHOICES_GENERATION": True,
# "CHOICES_MENUS": [],
"CHOICES_COMMANDS": choices_commands,
"_factory_backend": self._factory_backend,
},
)
@@ -143,7 +238,6 @@ class PlatformController(BaseController):
def _generate_commands(self):
"""Generate commands."""
for name, translator in self.translators.items():
# Prepare the translator name to create a command call in the controller
new_name = name.replace(f"{self._name}_", "")
self._generate_command_call(name=new_name, translator=translator)
@@ -170,14 +264,18 @@ class PlatformController(BaseController):
obbject = translator.execute_func(parsed_args=ns_parser)
df: pd.DataFrame = pd.DataFrame()
fig: OpenBBFigure | None = None
title = f"{self.PATH}{translator.func.__name__}"
if obbject:
if isinstance(obbject, list):
obbject = OBBject(results=obbject)
obbject = _OBBject()(results=obbject)
if isinstance(obbject, _OBBStream()):
self._follow_stream(obbject)
elif isinstance(obbject, _OBBject()):
df = extract_dataframe(obbject)
if isinstance(obbject, OBBject):
if (
session.max_obbjects_exceeded()
and obbject.results
@@ -188,9 +286,7 @@ class PlatformController(BaseController):
"[yellow]Maximum number of OBBjects reached. The oldest entry was removed.[yellow]"
)
# use the obbject to store the command so we can display it later on results
obbject.extra["command"] = f"{title} {' '.join(other_args)}"
# if there is a registry key in the parser, store to the obbject
if (
hasattr(ns_parser, "register_key")
and ns_parser.register_key
@@ -209,15 +305,11 @@ class PlatformController(BaseController):
)
if store_obbject:
# store the obbject in the registry
register_result = session.obbject_registry.register(
obbject
)
# we need to force to re-link so that the new obbject
# is immediately available for data processing commands
self._link_obbject_to_data_processing_commands()
# also update the completer
self.update_completer(self.choices_default)
if (
@@ -228,37 +320,47 @@ class PlatformController(BaseController):
"Added `OBBject` to cached results."
)
# making the dataframe available either for printing or exporting
df = obbject.to_dataframe()
chart = hasattr(ns_parser, "chart") and ns_parser.chart
if hasattr(ns_parser, "chart") and ns_parser.chart:
fig = obbject.chart.fig if obbject.chart else None
if not export:
obbject.show()
elif session.settings.USE_INTERACTIVE_DF and not export:
obbject.charting.table() # type: ignore[attr-defined]
else:
if isinstance(df.columns, pd.RangeIndex):
df.columns = [str(i) for i in df.columns]
print_rich_table(
df=df, show_index=True, title=title, export=export
)
if not export:
try:
session.output_adapter.display(
data=obbject,
title=title,
export=export,
chart=chart,
)
except Exception as e:
session.console.print(
f"[red]Display error: {e}[/red]"
)
if hasattr(obbject, "model_dump"):
results = obbject.model_dump(
exclude_unset=True, exclude_none=True
).get("results")
session.console.print(results)
elif isinstance(obbject, dict):
df = pd.DataFrame.from_dict(obbject, orient="columns")
print_rich_table(
df=df, show_index=True, title=title, export=export
)
df = _display_dict_result(obbject, title, export)
elif not isinstance(obbject, OBBject):
elif not isinstance(obbject, _OBBject()):
session.console.print(obbject)
elif not export and _is_empty_result(obbject):
session.console.print("[yellow]No results.[/yellow]")
if export and not df.empty:
sheet_name = getattr(ns_parser, "sheet_name", None)
if sheet_name and isinstance(sheet_name, list):
sheet_name = sheet_name[0]
fig = None
if hasattr(ns_parser, "chart") and ns_parser.chart:
from contextlib import suppress
with suppress(Exception):
fig = obbject.chart.fig if obbject.chart else None
export_data(
export_type=",".join(ns_parser.export),
dir_path=os.path.dirname(os.path.abspath(__file__)),
@@ -274,13 +376,31 @@ class PlatformController(BaseController):
session.console.print(f"[red]{e}[/]\n")
return
# Bind the method to the class
bound_method = MethodType(method, self)
# Update the wrapper and set the attribute
bound_method = update_wrapper(partial(bound_method, translator=translator), method) # type: ignore
bound_method = update_wrapper(
partial(bound_method, translator=translator), method
)
setattr(self, f"call_{name}", bound_method)
def _follow_stream(self, stream) -> None:
"""Follow a streaming (e.g. SSE) result in the foreground.
Streams chunks to the console on a background thread and blocks until the
stream ends or the user presses Ctrl+C, then stops it cleanly.
"""
session.console.print(
f"[cyan]Streaming {stream.media_type or 'data'} — press Ctrl+C to stop.[/cyan]"
)
stream.start()
try:
stream.wait()
except KeyboardInterrupt:
session.console.print("\n[yellow]Stream interrupted.[/yellow]")
finally:
stream.stop()
session.console.print("[cyan]Stream stopped.[/cyan]")
def _generate_controller_call(self, controller, name, parent_path, translators):
"""Generate controller call."""
@@ -294,11 +414,9 @@ class PlatformController(BaseController):
queue=self.queue,
)
# Bind the method to the class
bound_method = MethodType(method, self)
# Update the wrapper and set the attribute
bound_method = update_wrapper( # type: ignore
bound_method = update_wrapper(
partial(
bound_method,
name=name,
@@ -310,10 +428,28 @@ class PlatformController(BaseController):
)
setattr(self, f"call_{name}", bound_method)
def _get_reference_paths(self) -> dict[str, dict[str, Any]]:
"""Return the reference path-description dict."""
if self._factory_backend is not None:
return self._factory_backend.reference_paths
from openbb_cli.backend import LocalBackend
return LocalBackend().reference_paths
def _get_reference_routers(self) -> dict[str, dict[str, Any]]:
"""Return the reference router-description dict."""
if self._factory_backend is not None:
return self._factory_backend.reference_routers
from openbb_cli.backend import LocalBackend
return LocalBackend().reference_routers
def _get_command_description(self, command: str) -> str:
"""Get command description."""
command_description = (
obb.reference["paths"].get(f"{self.PATH}{command}", {}).get("description", "") # type: ignore
self._get_reference_paths()
.get(f"{self.PATH}{command}", {})
.get("description", "")
)
if not command_description:
@@ -323,7 +459,7 @@ class PlatformController(BaseController):
if trl and hasattr(trl, "parser"):
command_description = trl.parser.description
return command_description.split(".")[0].lower()
return (command_description or "").split(".")[0].lower()
def _get_menu_description(self, menu: str) -> str:
"""Get menu description."""
@@ -338,12 +474,13 @@ class PlatformController(BaseController):
return commands
menu_description = (
obb.reference["routers"].get(f"{self.PATH}{menu}", {}).get("description", "") # type: ignore
self._get_reference_routers()
.get(f"{self.PATH}{menu}", {})
.get("description", "")
) or ""
if menu_description:
return menu_description.split(".")[0].lower()
# If no description is found, return the sub menu commands
return ", ".join(_get_sub_menu_commands())
def print_help(self):
@@ -371,8 +508,9 @@ class PlatformController(BaseController):
for key, value in list(session.obbject_registry.all.items())[
: session.settings.N_TO_DISPLAY_OBBJECT_REGISTRY
]:
command = (value.get("extra") or {}).get("command", "")
mt.add_raw(
f"[yellow]OBB{key}[/yellow]: {value['command']}",
f"[yellow]OBB{key}[/yellow]: {command}",
left_spacing=True,
)

100
cli/openbb_cli/controllers/choices.py
View File

@@ -1,4 +1,4 @@
"""This module contains functions to build the choice map for the controllers."""
"""Functions to build the choice map for the controllers."""
from argparse import SUPPRESS, ArgumentParser
from collections.abc import Callable
@@ -18,8 +18,8 @@ from openbb_cli.session import Session
session = Session()
def __mock_parse_known_args_and_warn( # pylint: disable=R0917
controller, # pylint: disable=unused-argument
def __mock_parse_known_args_and_warn(
controller,
parser: ArgumentParser,
other_args: list[str],
export_allowed: Literal[
@@ -30,21 +30,18 @@ def __mock_parse_known_args_and_warn( # pylint: disable=R0917
) -> None:
"""Add arguments.
Add the arguments that would have normally added by :
- openbb_cli.base_controller.BaseController.parse_known_args_and_warn
Parameters
----------
parser: argparse.ArgumentParser
Parser with predefined arguments
other_args: List[str]
list of arguments to parse
export_allowed: Literal["no_export", "raw_data_only", "figures_only", "raw_data_and_figures"]
Export options
raw: bool
Add the --raw flag
limit: int
Add a --limit flag with this number default
parser : argparse.ArgumentParser
Parser with predefined arguments.
other_args : list[str]
List of arguments to parse.
export_allowed : Literal["no_export", "raw_data_only", "figures_only", "raw_data_and_figures"]
Export options.
raw : bool
Add the --raw flag.
limit : int
Add a --limit flag with this number default.
"""
_ = other_args
@@ -114,15 +111,12 @@ def __mock_parse_known_args_and_warn( # pylint: disable=R0917
def __mock_parse_simple_args(parser: ArgumentParser, other_args: list[str]) -> tuple:
"""Add arguments.
Add the arguments that would have normally added by:
- openbb_cli.parent_classes.BaseController.parse_simple_args
Parameters
----------
parser: argparse.ArgumentParser
Parser with predefined arguments
other_args: List[str]
List of arguments to parse
parser : argparse.ArgumentParser
Parser with predefined arguments.
other_args : list[str]
List of arguments to parse.
"""
parser.add_argument(
"-h", "--help", action="store_true", help="show this help message"
@@ -136,14 +130,15 @@ def __get_command_func(controller, command: str):
Parameters
----------
controller: BaseController
controller : BaseController
Instance of the CLI Controller.
command: str
A name from controller.CHOICES_COMMANDS
command : str
A name from controller.CHOICES_COMMANDS.
Returns
-------
Callable: Command function.
Callable
Command function.
"""
if command not in controller.CHOICES_COMMANDS:
raise AttributeError(
@@ -163,20 +158,17 @@ def __get_command_func(controller, command: str):
def contains_functions_to_patch(command_func: Callable) -> bool:
"""Check command function.
Check if a `command_func` actually contains the functions we want to mock, i.e.:
- parse_simple_args
- parse_known_args_and_warn
Parameters
----------
command_func: Callable
command_func : Callable
Function to check.
Returns
-------
bool: Whether or not `command_func` contains the mocked functions.
bool
Whether or not `command_func` contains the mocked functions.
"""
co_names = command_func.__code__.co_names
co_names = command_func.__code__.co_names # ty: ignore[unresolved-attribute]
return bool(
"parse_simple_args" in co_names or "parse_known_args_and_warn" in co_names
@@ -187,21 +179,15 @@ def contains_functions_to_patch(command_func: Callable) -> bool:
def __patch_controller_functions(controller):
"""Patch controller functions.
Patch the following function from a BaseController instance:
- parse_simple_args
- parse_known_args_and_warn
These functions take an 'argparse.ArgumentParser' object as parameter.
We want to intercept this 'argparse.ArgumentParser' object.
Parameters
----------
controller: BaseController
controller : BaseController
BaseController object that needs to be patched.
Returns
-------
List[Callable]: List of mocked functions.
list[Callable]
List of mocked functions.
"""
bound_mock_parse_known_args_and_warn = MethodType(
__mock_parse_known_args_and_warn,
@@ -234,12 +220,13 @@ def __patch_controller_functions(controller):
for patcher in patcher_list:
patched_function_list.append(patcher.start())
yield patched_function_list
if not session.settings.DEBUG_MODE:
rich.stop()
for patcher in patcher_list:
patcher.stop()
try:
yield patched_function_list
finally:
if not session.settings.DEBUG_MODE:
rich.stop()
for patcher in patcher_list:
patcher.stop()
def _get_argument_parser(
@@ -248,21 +235,17 @@ def _get_argument_parser(
) -> ArgumentParser:
"""Intercept the ArgumentParser instance from the command function.
A command function being a function starting with `call_`, like:
- call_help
- call_overview
- call_load
Parameters
----------
controller: BaseController
controller : BaseController
Instance of the CLI Controller.
command: str
command : str
A name from `controller.CHOICES_COMMANDS`.
Returns
-------
ArgumentParser: ArgumentParser instance from the command function.
ArgumentParser
ArgumentParser instance from the command function.
"""
command_func: Callable = __get_command_func(controller=controller, command=command)
@@ -292,14 +275,13 @@ def _get_argument_parser(
" - parse_known_args_and_warn\n"
)
# pylint: disable=possibly-used-before-assignment
return argument_parser
def _build_command_choice_map(argument_parser: ArgumentParser) -> dict:
"""Build the choice map for a command."""
choice_map: dict = {}
for action in argument_parser._actions: # pylint: disable=protected-access
for action in argument_parser._actions:
if action.help == SUPPRESS:
continue
if len(action.option_strings) == 1:

561
cli/openbb_cli/controllers/cli_controller.py
View File

@@ -1,16 +1,11 @@
#!/usr/bin/env python
"""Main CLI Module."""
# pylint: disable=too-many-public-methods,import-outside-toplevel, too-many-function-args
# pylint: disable=too-many-branches,no-member,C0302,too-many-return-statements, inconsistent-return-statements
import argparse
import contextlib
import difflib
import os
import re
import sys
import time
import webbrowser
from datetime import datetime
from functools import partial, update_wrapper
@@ -19,8 +14,10 @@ from types import MethodType
from typing import Any
import pandas as pd
import requests
from openbb import obb
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from openbb_cli.backend import Backend, LocalBackend
from openbb_cli.config import constants
from openbb_cli.config.constants import (
ASSETS_DIRECTORY,
@@ -46,45 +43,82 @@ from openbb_cli.controllers.utils import (
welcome_message,
)
from openbb_cli.session import Session
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from pydantic import BaseModel
PLATFORM_ROUTERS = {
d: "menu" if not isinstance(getattr(obb, d), BaseModel) else "command"
for d in dir(obb)
if "_" not in d
}
NON_DATA_ROUTERS = ["coverage", "reference", "system", "user"]
DATA_PROCESSING_ROUTERS = ["technical", "quantitative", "econometrics"]
env_file = str(ENV_FILE_SETTINGS)
session = Session()
def _result_to_obbject(result: Any, router: str, other_args: list[str]) -> Any:
"""Lift a spec-mode dispatcher result to an ``OBBject`` for registry insert."""
try:
from openbb_core.app.model.obbject import OBBject
except ImportError:
return None
if isinstance(result, OBBject):
obbject = result
elif isinstance(result, (list, dict)):
try:
obbject = OBBject(results=result)
obbject._route = "/" + router.replace(".", "/").strip("/")
except Exception: # noqa: BLE001
return None
else:
return None
obbject.extra["command"] = f"/{router} {' '.join(other_args)}".strip()
return obbject
def _params_to_completions(parameters: list[dict[str, Any]]) -> dict[str, Any]:
"""Translate normalized spec parameters into a NestedCompleter dict."""
out: dict[str, Any] = {}
for p in parameters or []:
name = p.get("name")
if not name:
continue
flag = f"--{name}"
choices_list = p.get("choices") or []
if choices_list:
out[flag] = {str(c): None for c in choices_list}
else:
out[flag] = None
out["--help"] = None
out["-h"] = "--help"
return out
class CLIController(BaseController):
"""CLI Controller class."""
CHOICES_COMMANDS = ["record", "stop", "exe", "results"]
CHOICES_MENUS = [
"settings",
]
for router, value in PLATFORM_ROUTERS.items():
if value == "menu":
CHOICES_MENUS.append(router)
else:
CHOICES_COMMANDS.append(router)
CHOICES_COMMANDS_BUILTIN = ["record", "stop", "exe", "results"]
CHOICES_MENUS_BUILTIN = ["settings", "user", "feature"]
PATH = "/"
CHOICES_GENERATION = False
def __init__(self, jobs_cmds: list[str] | None = None):
def __init__(
self,
jobs_cmds: list[str] | None = None,
*,
backend: Backend | None = None,
):
"""Construct CLI controller."""
self.ROUTINE_FILES: dict[str, Path] = dict()
self.ROUTINE_DEFAULT_FILES: dict[str, Path] = dict()
self.ROUTINE_PERSONAL_FILES: dict[str, Path] = dict()
self.ROUTINE_CHOICES: dict[str, Any] = dict()
self._backend: Backend = backend if backend is not None else LocalBackend()
platform_routers = self._backend.routers
menus = list(self.CHOICES_MENUS_BUILTIN)
commands = list(self.CHOICES_COMMANDS_BUILTIN)
for router, kind in platform_routers.items():
if router == "user":
continue
(menus if kind == "menu" else commands).append(router)
self.CHOICES_MENUS = menus
self.CHOICES_COMMANDS = commands
super().__init__(jobs_cmds)
self.queue: list[str] = list()
@@ -101,77 +135,175 @@ class CLIController(BaseController):
self.update_runtime_choices()
def _generate_platform_commands(self):
"""Generate Platform based commands/menus."""
"""Generate Platform based commands/menus from the configured backend."""
backend = self._backend
def method_call_class(self, _, controller, name, parent_path, target):
self.queue = self.load_class(
controller, name, parent_path, target, self.queue
)
# pylint: disable=unused-argument
def method_call_command(self, _, router: str):
"""Call command."""
mdl = getattr(obb, router)
def method_call_command_legacy(self, _, router: str):
"""Display the static command-target metadata for in-process ``obb``."""
mdl = backend.get_command_target(router)
df = pd.DataFrame.from_dict(mdl.model_dump(), orient="index")
if isinstance(df.columns, pd.RangeIndex):
df.columns = [str(i) for i in df.columns]
return print_rich_table(df, show_index=True)
for router, value in PLATFORM_ROUTERS.items():
target = getattr(obb, router)
def method_call_command_spec(self, other_args: list[str], router: str):
"""Dispatch a top-level command-typed router via SpecBackend."""
from openbb_cli.backend import SpecTranslator
spec_doc: dict[str, Any] = getattr(backend, "_spec", {})
cmd_spec = spec_doc.get("commands", {}).get(router)
if cmd_spec is None:
session.console.print(f"[red]Command not found in spec: {router}[/red]")
return
translator = SpecTranslator(
router, cmd_spec, getattr(backend, "_dispatcher", None)
)
ns_parser = self.parse_known_args_and_warn(
parser=translator.parser,
other_args=other_args,
export_allowed="raw_data_only",
)
if not ns_parser:
return
try:
result = translator.execute_func(ns_parser)
except Exception as exc: # noqa: BLE001
session.console.print(f"[red]error: {exc}[/red]")
return
obbject = _result_to_obbject(result, router, other_args)
if obbject is not None and obbject.results is not None:
if session.max_obbjects_exceeded():
session.obbject_registry.remove()
session.console.print(
"[yellow]Maximum number of OBBjects reached. "
"The oldest entry was removed.[/yellow]"
)
if session.obbject_registry.register(obbject) and (
session.settings.SHOW_MSG_OBBJECT_REGISTRY
):
session.console.print("Added `OBBject` to cached results.")
payload: Any
if obbject is not None:
payload = obbject.results
elif isinstance(result, dict):
payload = result.get("results", result)
else:
payload = result
if isinstance(payload, dict):
df = pd.DataFrame([payload])
elif isinstance(payload, list):
df = pd.DataFrame(payload)
else:
df = pd.DataFrame({"value": [payload]})
session.output_adapter.display(df, title=f"/{router}")
is_spec_backend = hasattr(backend, "_spec") and hasattr(backend, "_dispatcher")
method_call_command: Any = (
method_call_command_spec if is_spec_backend else method_call_command_legacy
)
spec_commands: dict[str, Any] = (
getattr(backend, "_spec", {}).get("commands", {}) if is_spec_backend else {}
)
def method_call_menu_or_leaf(
self,
other_args: list[str],
controller,
name: str,
parent_path: list[str],
target,
router: str,
):
"""Hybrid for routers that have both a leaf path and nested commands."""
if other_args:
method_call_command_spec(self, other_args, router)
return
self.queue = self.load_class(
controller, name, parent_path, target, self.queue
)
for router, value in backend.routers.items():
if router == "user":
continue
has_leaf = router in spec_commands
if value == "menu":
pcf = PlatformControllerFactory(
target,
reference=obb.reference["paths"], # type: ignore
)
pcf = PlatformControllerFactory(backend=backend, router_name=router)
DynamicController = pcf.create()
# Bind the method to the class
bound_method = MethodType(method_call_class, self)
# Update the wrapper and set the attribute
bound_method = update_wrapper( # type: ignore
partial(
bound_method,
controller=DynamicController,
name=router,
target=target,
parent_path=self.path,
),
method_call_class,
)
if has_leaf:
bound_method = MethodType(method_call_menu_or_leaf, self)
bound_method = update_wrapper(
partial(
bound_method,
controller=DynamicController,
name=router,
target=None,
parent_path=self.path,
router=router,
),
method_call_menu_or_leaf,
)
else:
bound_method = MethodType(method_call_class, self)
bound_method = update_wrapper(
partial(
bound_method,
controller=DynamicController,
name=router,
target=None,
parent_path=self.path,
),
method_call_class,
)
else:
bound_method = MethodType(method_call_command, self)
bound_method = update_wrapper( # type: ignore
bound_method = update_wrapper(
partial(bound_method, router=router),
method_call_command,
)
setattr(self, f"call_{router}", bound_method)
def _spec_command_completions(self) -> dict[str, dict[str, Any]]:
"""Build per-command completion choices from the SpecBackend's metadata."""
out: dict[str, dict[str, Any]] = {}
backend = getattr(self, "_backend", None)
if backend is None:
return out
spec_doc = getattr(backend, "_spec", None)
if not isinstance(spec_doc, dict):
return out
commands = spec_doc.get("commands", {})
for router in backend.routers:
if router not in self.controller_choices:
continue
cmd_spec = commands.get(router)
if not cmd_spec:
continue
out[router] = _params_to_completions(cmd_spec.get("parameters", []))
return out
def update_runtime_choices(self):
"""Update runtime choices."""
routines_directory = Path(session.user.preferences.export_directory, "routines")
if session.prompt_session and session.settings.USE_PROMPT_TOOLKIT:
# choices: dict = self.choices_default
choices: dict = {c: {} for c in self.controller_choices} # type: ignore
choices: dict = {c: {} for c in self.controller_choices}
choices.update(self._spec_command_completions())
self.ROUTINE_FILES = {
filepath.name: filepath for filepath in routines_directory.rglob("*.openbb") # type: ignore
}
self.ROUTINE_DEFAULT_FILES = {
filepath.name: filepath # type: ignore
for filepath in Path(routines_directory / "hub" / "default").rglob(
"*.openbb"
)
}
self.ROUTINE_PERSONAL_FILES = {
filepath.name: filepath # type: ignore
for filepath in Path(routines_directory / "hub" / "personal").rglob(
"*.openbb"
)
filepath.name: filepath
for filepath in routines_directory.rglob("*.openbb")
}
choices["exe"] = {
@@ -183,7 +315,6 @@ class CLIController(BaseController):
"-e": "--example",
"--input": None,
"-i": "--input",
"--url": None,
"--help": None,
"-h": "--help",
}
@@ -192,8 +323,6 @@ class CLIController(BaseController):
"-n": "--name",
"--description": None,
"-d": "--description",
"--public": None,
"-p": "--public",
"--tag1": {c: None for c in constants.SCRIPT_TAGS},
"--tag2": {c: None for c in constants.SCRIPT_TAGS},
"--tag3": {c: None for c in constants.SCRIPT_TAGS},
@@ -201,14 +330,47 @@ class CLIController(BaseController):
"-h": "--help",
}
choices["stop"] = {"--help": None, "-h": "--help"}
registry_all = session.obbject_registry.all
index_completions = {str(idx): None for idx in registry_all}
key_completions = {
(data.get("extra") or {}).get("register_key"): None
for data in registry_all.values()
if (data.get("extra") or {}).get("register_key")
}
choices["results"] = {
"--index": index_completions if index_completions else None,
"-i": "--index",
"--key": key_completions if key_completions else None,
"-k": "--key",
"--chart": None,
"-c": "--chart",
"--export": {
c: None
for c in [
"csv",
"json",
"xlsx",
"png",
"jpg",
"db",
"sqlite",
"sqlite3",
]
},
"-e": "--export",
"--sheet-name": None,
"--help": None,
"-h": "--help",
}
choices["load"] = {
"--file": None,
"-f": "--file",
"--sheet-name": None,
"--register_key": None,
"--help": None,
"-h": "--help",
"--export": {c: None for c in ["csv", "json", "xlsx", "png", "jpg"]},
"--index": None,
"--key": None,
"--chart": None,
"--sheet_name": None,
}
self.update_completer(choices)
@@ -221,6 +383,10 @@ class CLIController(BaseController):
"settings",
description="enable and disable feature flags, preferences and settings",
)
mt.add_menu(
"user",
description="view and set platform user preferences for the session",
)
mt.add_raw("\n")
mt.add_info("Record and execute your own .openbb routine scripts")
mt.add_cmd("record", description="start recording current session")
@@ -234,46 +400,58 @@ class CLIController(BaseController):
mt.add_raw("\n")
mt.add_info("Retrieve data from different asset classes and providers")
for router, value in PLATFORM_ROUTERS.items():
platform_routers = self._backend.routers
reference_routers = self._backend.reference_routers
reference_paths = self._backend.reference_paths
def _menu_description(router: str) -> str:
"""Return the menu's reference description (tolerant of trailing slash)."""
for key in (f"{self.PATH}{router}/", f"{self.PATH}{router}"):
desc = reference_routers.get(key, {}).get("description") or ""
if desc:
return desc.split(".")[0].lower()
return ""
def _command_description(router: str) -> str:
"""Return the operation description for top-level command-typed routers."""
for key in (f"{self.PATH}{router}", f"{self.PATH}{router}/"):
desc = reference_paths.get(key, {}).get("description") or ""
if desc:
return desc.split(".")[0].lower()
return ""
for router, value in platform_routers.items():
if router in NON_DATA_ROUTERS or router in DATA_PROCESSING_ROUTERS:
continue
if value == "menu":
menu_description = (
obb.reference["routers"].get(f"{self.PATH}{router}", {}).get("description") # type: ignore
) or ""
mt.add_menu(
name=router,
description=menu_description.split(".")[0].lower(),
)
mt.add_menu(name=router, description=_menu_description(router))
else:
mt.add_cmd(router)
mt.add_cmd(name=router, description=_command_description(router))
if any(router in PLATFORM_ROUTERS for router in DATA_PROCESSING_ROUTERS):
if any(router in platform_routers for router in DATA_PROCESSING_ROUTERS):
mt.add_info("\nAnalyze and process previously obtained data")
for router, value in PLATFORM_ROUTERS.items():
for router, value in platform_routers.items():
if router not in DATA_PROCESSING_ROUTERS:
continue
if value == "menu":
menu_description = (
obb.reference["routers"].get(f"{self.PATH}{router}", {}).get("description") # type: ignore
) or ""
mt.add_menu(
name=router,
description=menu_description.split(".")[0].lower(),
)
mt.add_menu(name=router, description=_menu_description(router))
else:
mt.add_cmd(router)
mt.add_cmd(name=router, description=_command_description(router))
mt.add_raw("\n")
mt.add_info("Data manipulation and feature engineering")
mt.add_menu("feature", description="feature engineering and table operations")
mt.add_cmd("load", description="load data from file in OpenBBUserData")
mt.add_cmd("results")
if session.obbject_registry.obbjects:
mt.add_info("\nCached Results")
for key, value in list(session.obbject_registry.all.items())[ # type: ignore
for key, value in list(session.obbject_registry.all.items())[
: session.settings.N_TO_DISPLAY_OBBJECT_REGISTRY
]:
command = (value.get("extra") or {}).get("command", "")
mt.add_raw(
f"[yellow]OBB{key}[/yellow]: {value['command']}", # type: ignore[index]
f"[yellow]OBB{key}[/yellow]: {command}",
left_spacing=True,
)
@@ -288,9 +466,22 @@ class CLIController(BaseController):
self.queue = self.load_class(SettingsController, self.queue)
def call_user(self, _):
"""Process user command."""
from openbb_cli.controllers.user_controller import UserController
self.queue = self.load_class(UserController, self.queue)
def call_feature(self, _):
"""Process feature engineering command."""
from openbb_cli.controllers.feature_controller import (
FeatureController,
)
self.queue = self.load_class(FeatureController, self.queue)
def call_exe(self, other_args: list[str]):
"""Process exe command."""
# Merge rest of string path to other_args and remove queue since it is a dir
other_args += self.queue
if not other_args:
@@ -313,9 +504,7 @@ class CLIController(BaseController):
required="-h" not in other_args
and "--help" not in other_args
and "-e" not in other_args
and "--example" not in other_args
and "--url" not in other_args
and "my.openbb" not in other_args[0],
and "--example" not in other_args,
type=str,
nargs="+",
)
@@ -334,64 +523,18 @@ class CLIController(BaseController):
action="store_true",
default=False,
)
parser.add_argument(
"--url", help="URL to run openbb script from.", dest="url", type=str
)
if other_args and "-" not in other_args[0][0]:
if other_args[0].startswith("my.") or other_args[0].startswith("http"):
other_args.insert(0, "--url")
else:
other_args.insert(0, "--file")
other_args.insert(0, "--file")
ns_parser = self.parse_known_args_and_warn(parser, other_args)
if ns_parser:
if ns_parser.example:
routine_path = ASSETS_DIRECTORY / "routines" / "routine_example.openbb"
session.console.print( # TODO: Point to docs when ready
session.console.print(
"[info]Executing an example, please visit our docs to learn how to create your own script.[/info]\n"
)
time.sleep(3)
elif ns_parser.url:
if not ns_parser.url.startswith(
"https"
) and not ns_parser.url.startswith("http:"):
url = "https://" + ns_parser.url
elif ns_parser.url.startswith("http://"):
url = ns_parser.url.replace("http://", "https://")
else:
url = ns_parser.url
username = url.split("/")[-3]
script_name = url.split("/")[-1]
file_name = f"{username}_{script_name}.openbb"
final_url = f"{url}?raw=true"
response = requests.get(final_url, timeout=10)
if response.status_code != 200:
session.console.print(
"[red]Could not find the requested script.[/red]"
)
return
routine_text = response.json()["script"]
file_path = Path(session.user.preferences.export_directory, "routines")
routine_path = file_path / file_name
with open(routine_path, "w") as file:
file.write(routine_text)
self.update_runtime_choices()
elif ns_parser.file:
file_path = " ".join(ns_parser.file) # type: ignore
# if string is not in this format "default/file.openbb" then check for files in ROUTINE_FILES
full_path = file_path
hub_routine = file_path.split("/") # type: ignore
# Change with: my.openbb.co
if hub_routine[0] == "default":
routine_path = Path(
self.ROUTINE_DEFAULT_FILES.get(hub_routine[1], full_path)
)
elif hub_routine[0] == "personal":
routine_path = Path(
self.ROUTINE_PERSONAL_FILES.get(hub_routine[1], full_path)
)
else:
routine_path = Path(self.ROUTINE_FILES.get(file_path, full_path)) # type: ignore
file_path = " ".join(ns_parser.file)
routine_path = Path(self.ROUTINE_FILES.get(file_path, file_path))
else:
return
@@ -400,7 +543,6 @@ class CLIController(BaseController):
raw_lines = list(fp)
script_inputs = []
# Capture ARGV either as list if args separated by commas or as single value
if routine_args := ns_parser.routine_args:
pattern = r"\[(.*?)\]"
matches = re.findall(pattern, routine_args)
@@ -417,9 +559,6 @@ class CLIController(BaseController):
raw_lines=raw_lines, script_inputs=script_inputs
)
# If there err output is not an empty string then it means there was an
# issue in parsing the routine and therefore we don't want to feed it
# to the terminal
if err:
session.console.print(err)
return
@@ -434,7 +573,6 @@ class CLIController(BaseController):
if "export" in self.queue[0]:
export_path = self.queue[0].split(" ")[1]
# If the path selected does not start from the user root, give relative location from root
if export_path[0] == "~":
export_path = export_path.replace(
"~", HOME_DIRECTORY.as_posix()
@@ -444,7 +582,6 @@ class CLIController(BaseController):
os.path.dirname(os.path.abspath(__file__)), export_path
)
# Check if the directory exists
if os.path.isdir(export_path):
session.console.print(
f"Export data to be saved in the selected folder: '{export_path}'"
@@ -481,7 +618,6 @@ def handle_job_cmds(jobs_cmds: list[str] | None) -> list[str] | None:
os.path.dirname(os.path.abspath(__file__)), export_path
)
# Check if the directory exists
if os.path.isdir(export_path):
session.console.print(
f"Export data to be saved in the selected folder: '{export_path}'"
@@ -494,11 +630,15 @@ def handle_job_cmds(jobs_cmds: list[str] | None) -> list[str] | None:
return jobs_cmds
# pylint: disable=unused-argument
def run_cli(jobs_cmds: list[str] | None = None, test_mode=False):
"""Run the CLI menu."""
def run_cli(
jobs_cmds: list[str] | None = None,
test_mode=False,
*,
backend: Backend | None = None,
):
"""Run the CLI menu, optionally backed by a non-default ``Backend``."""
ret_code = 1
t_controller = CLIController(jobs_cmds)
t_controller = CLIController(jobs_cmds, backend=backend)
an_input = ""
jobs_cmds = handle_job_cmds(jobs_cmds)
@@ -514,29 +654,22 @@ def run_cli(jobs_cmds: list[str] | None = None, test_mode=False):
t_controller.print_help()
while ret_code:
# There is a command in the queue
if t_controller.queue and len(t_controller.queue) > 0:
# If the command is quitting the menu we want to return in here
if t_controller.queue[0] in ("q", "..", "quit"):
print_goodbye()
break
# Consume 1 element from the queue
an_input = t_controller.queue[0]
t_controller.queue = t_controller.queue[1:]
# Print the current location because this was an instruction and we want user to know what was the action
if an_input and an_input.split(" ")[0] in t_controller.CHOICES_COMMANDS:
session.console.print(f"{get_flair_and_username()} / $ {an_input}")
# Get input command from user
else:
try:
# Get input from user using auto-completion
if session.prompt_session and session.settings.USE_PROMPT_TOOLKIT:
# Check if toolbar hint was enabled
if session.settings.TOOLBAR_HINT:
an_input = session.prompt_session.prompt( # type: ignore[union-attr]
an_input = session.prompt_session.prompt(
f"{get_flair_and_username()} / $ ",
completer=t_controller.completer,
search_ignore_case=True,
@@ -554,13 +687,12 @@ def run_cli(jobs_cmds: list[str] | None = None, test_mode=False):
),
)
else:
an_input = session.prompt_session.prompt( # type: ignore[union-attr]
an_input = session.prompt_session.prompt(
f"{get_flair_and_username()} / $ ",
completer=t_controller.completer,
search_ignore_case=True,
)
# Get input from user without auto-completion
else:
an_input = input(f"{get_flair_and_username()} / $ ")
@@ -569,14 +701,12 @@ def run_cli(jobs_cmds: list[str] | None = None, test_mode=False):
break
try:
# Process the input command
t_controller.queue = t_controller.switch(an_input)
if an_input in ("q", "quit", "..", "exit", "e"):
print_goodbye()
break
# Check if the user wants to reset application
if an_input in ("r", "reset") or t_controller.update_success:
reset(t_controller.queue if t_controller.queue else [])
break
@@ -593,7 +723,7 @@ def run_cli(jobs_cmds: list[str] | None = None, test_mode=False):
)
if similar_cmd:
an_input = similar_cmd[0]
if " " in an_input:
if " " in an_input: # pragma: no cover
candidate_input = (
f"{similar_cmd[0]} {' '.join(an_input.split(' ')[1:])}"
)
@@ -617,7 +747,7 @@ def insert_start_slash(cmds: list[str]) -> list[str]:
return cmds
def run_scripts( # pylint: disable=R0917
def run_scripts(
path: Path,
test_mode: bool = False,
verbose: bool = False,
@@ -629,19 +759,18 @@ def run_scripts( # pylint: disable=R0917
Parameters
----------
path : str
The location of the .openbb file
path : Path
The location of the .openbb file.
test_mode : bool
Whether the CLI is in test mode
Whether the CLI is in test mode.
verbose : bool
Whether to run tests in verbose mode
routines_args : List[str]
Whether to run tests in verbose mode.
routines_args : list[str] | None
One or multiple inputs to be replaced in the routine and separated by commas.
E.g. GME,AMC,BTC-USD
special_arguments: Optional[Dict[str, str]]
Replace `${key=default}` with `value` for every key in the dictionary
output: bool
Whether to log tests to txt files
special_arguments : dict[str, str] | None
Replace `${key=default}` with `value` for every key in the dictionary.
output : bool
Whether to log tests to txt files.
"""
if not path.exists():
session.console.print(f"File '{path}' doesn't exist. Launching base CLI.\n")
@@ -661,13 +790,12 @@ def run_scripts( # pylint: disable=R0917
for i, arg in enumerate(routines_args):
templine = templine.replace(f"$ARGV[{i}]", arg)
lines.append(templine)
# Handle new testing arguments:
elif special_arguments:
lines = []
for line in raw_lines:
new_line = re.sub(
r"\${[^{]+=[^{]+}",
lambda x: replace_dynamic(x, special_arguments), # type: ignore
lambda x: replace_dynamic(x, special_arguments),
line,
)
lines.append(new_line)
@@ -678,7 +806,6 @@ def run_scripts( # pylint: disable=R0917
if test_mode and "exit" not in lines[-1]:
lines.append("exit")
# Deals with the export with a path with "/" in it
export_folder = ""
if "export" in lines[0]:
export_folder = lines[0].split("export ")[1].rstrip()
@@ -720,15 +847,15 @@ def replace_dynamic(match: re.Match, special_arguments: dict[str, str]) -> str:
Parameters
----------
match: re.Match[str]
The match object
special_arguments: Dict[str, str]
The key value pairs to replace in the scripts
match : re.Match
The match object.
special_arguments : dict[str, str]
The key value pairs to replace in the scripts.
Returns
----------
-------
str
The new string
The new string.
"""
cleaned = match[0].replace("{", "").replace("}", "").replace("$", "")
key, default = cleaned.split("=")
@@ -759,12 +886,13 @@ def run_routine(file: str, routines_args: str | None = None):
)
# pylint: disable=unused-argument
def main(
debug: bool,
dev: bool,
path_list: list[str],
routines_args: list[str] | None = None,
*,
backend: Backend | None = None,
**kwargs,
):
"""Run the CLI with various options.
@@ -772,28 +900,19 @@ def main(
Parameters
----------
debug : bool
Whether to run the CLI in debug mode
dev:
Points backend towards development environment instead of production
test : bool
Whether to run the CLI in integrated test mode
filtert : str
Filter test files with given string in name
paths : List[str]
The paths to run for scripts or to test
verbose : bool
Whether to show output from tests
routines_args : List[str]
Whether to run the CLI in debug mode.
dev : bool
Points backend towards development environment instead of production.
path_list : list[str]
The paths to run for scripts or to test.
routines_args : list[str] | None
One or multiple inputs to be replaced in the routine and separated by commas.
E.g. GME,AMC,BTC-USD
"""
if debug:
session.settings.DEBUG_MODE = True
if dev:
session.settings.DEV_BACKEND = True
session.settings.BASE_URL = "https://payments.openbb.dev/"
session.settings.HUB_URL = "https://my.openbb.dev"
if isinstance(path_list, list) and path_list[0].endswith(".openbb"):
run_routine(
@@ -803,9 +922,9 @@ def main(
elif path_list:
argv_cmds = list([" ".join(path_list).replace(" /", "/home/")])
argv_cmds = insert_start_slash(argv_cmds) if argv_cmds else argv_cmds
run_cli(argv_cmds)
run_cli(argv_cmds, backend=backend)
else:
run_cli()
run_cli(backend=backend)
def parse_args_and_run():
@@ -856,8 +975,6 @@ def parse_args_and_run():
"Run the CLI in testing mode. Also run this option and '-h' to see testing argument options."
),
)
# The args -m, -f and --HistoryManager.hist_file are used only in reports menu
# by papermill and that's why they have suppress help.
parser.add_argument(
"-m",
help=argparse.SUPPRESS,
@@ -883,8 +1000,6 @@ def parse_args_and_run():
sys.argv.insert(1, "--file")
ns_parser, unknown = parser.parse_known_args()
# This ensures that if cli.py receives unknown args it will not start.
# Use -d flag if you want to see the unknown args.
if unknown:
if ns_parser.debug:
session.console.print(unknown)
@@ -903,11 +1018,15 @@ def parse_args_and_run():
def launch(
debug: bool = False, dev: bool = False, queue: list[str] | None = None
debug: bool = False,
dev: bool = False,
queue: list[str] | None = None,
*,
backend: Backend | None = None,
) -> None:
"""Launch CLI."""
"""Launch CLI, optionally with a non-default ``Backend``."""
if queue:
main(debug, dev, queue, module="")
main(debug, dev, queue, module="", backend=backend)
else:
parse_args_and_run()

118
cli/openbb_cli/controllers/credentials_controller.py Normal file
View File

@@ -0,0 +1,118 @@
"""Credentials Controller Module."""
import argparse
from functools import partial, update_wrapper
from types import MethodType
from typing import Any, cast
from openbb import obb as _obb_container
from pydantic import SecretStr
obb = cast(Any, _obb_container)
from openbb_cli.controllers.base_controller import BaseController
from openbb_cli.session import Session
session = Session()
def _credential_status(raw: object) -> str:
"""Return a display string for a credential value."""
if isinstance(raw, SecretStr):
secret = raw.get_secret_value()
if secret:
return f"[green]Set ({secret[:4]}****)[/green]"
return "[red]Not set[/red]"
class CredentialsController(BaseController):
"""Credentials Controller class."""
PATH = "/user/credentials/"
CHOICES_GENERATION = True
_CRED_COMMANDS: dict[str, dict[str, Any]] = {}
for _fname, _finfo in sorted(obb.user.credentials.__class__.model_fields.items()):
_provider = _finfo.description or ""
if _provider == _fname:
_provider = (
_fname.rsplit("_api_key", 1)[0]
.rsplit("_token", 1)[0]
.rsplit("_key", 1)[0]
)
_CRED_COMMANDS[_fname] = {
"command": _fname,
"field_name": _fname,
"provider": _provider,
}
CHOICES_COMMANDS: list[str] = list(_CRED_COMMANDS.keys())
CHOICES_MENUS: list[str] = []
def __init__(self, queue: list[str] | None = None):
"""Initialize the Constructor."""
super().__init__(queue)
for cmd, field in self._CRED_COMMANDS.items():
self._generate_credential_command(cmd, field)
self.update_completer(self.choices_default)
def print_help(self):
"""Print help."""
spacing = 4
max_name = max((len(c) for c in self._CRED_COMMANDS), default=20)
col_width = max_name + spacing
lines: list[str] = []
lines.append("[info]Session-level credentials (not persisted):[/info]")
lines.append("[info]Use: <credential> -v <value>[/info]")
lines.append("")
for cmd, f in self._CRED_COMMANDS.items():
raw = getattr(obb.user.credentials, f["field_name"], None)
status = _credential_status(raw)
provider = f["provider"]
pad = " " * (col_width - len(cmd))
lines.append(
f"[cmds]{' ' * spacing}{cmd}{pad}{provider:<20s}[/cmds] {status}"
)
menu_text = "\n".join(lines) + "\n"
session.console.print(text=menu_text, menu="Credentials")
def _generate_credential_command(self, cmd_name: str, field: dict):
"""Generate a set command bound to obb.user.credentials."""
def _set_credential(self, other_args: list[str], field=field) -> None:
"""Set a credential value."""
field_name = field["field_name"]
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog=field["command"],
description=f"Set {field_name} credential",
add_help=False,
)
parser.add_argument(
"-v",
"--value",
dest="value",
action="store",
required=False,
type=str,
)
ns_parser, _ = self.parse_simple_args(parser, other_args)
if ns_parser:
if ns_parser.value is not None:
setattr(obb.user.credentials, field_name, ns_parser.value)
session.console.print(
f"[info]{field_name}:[/info] [green]Set[/green]"
)
elif not other_args:
raw = getattr(obb.user.credentials, field_name, None)
status = _credential_status(raw)
session.console.print(f"[info]{field_name}:[/info] {status}")
bound_method = update_wrapper(
wrapper=partial(MethodType(_set_credential, self), field=field),
wrapped=_set_credential,
)
setattr(self, f"call_{cmd_name}", bound_method)

1335
cli/openbb_cli/controllers/feature_controller.py Normal file
View File

File diff suppressed because it is too large Load Diff

102
cli/openbb_cli/controllers/platform_controller_factory.py
View File

@@ -1,56 +1,82 @@
"""Platform controller factory to create a platform controller."""
"""Platform controller factory."""
from openbb_cli.argparse_translator.argparse_class_processor import (
ArgparseClassProcessor,
)
from __future__ import annotations
from typing import Any
from openbb_cli.backend import Backend, LocalBackend
from openbb_cli.controllers.base_platform_controller import PlatformController
class PlatformControllerFactory:
"""Factory to create a platform controller."""
"""Factory to create a platform controller from a ``Backend``."""
def __init__(self, platform_router: type, **kwargs):
"""Create the controller name."""
self.platform_router = platform_router
self._translated_target = ArgparseClassProcessor(
target_class=self.platform_router, reference=kwargs.get("reference", {})
def __init__(
self,
platform_router: type | None = None,
*,
backend: Backend | None = None,
router_name: str | None = None,
reference: dict[str, Any] | None = None,
) -> None:
if backend is None and platform_router is None:
raise ValueError("Either ``backend`` or ``platform_router`` is required.")
if backend is not None:
if router_name is None:
raise ValueError("``router_name`` is required when ``backend`` is set.")
self._backend: Backend = backend
self._router_name: str = router_name
else:
del reference
assert platform_router is not None # noqa: S101
self._backend = LocalBackend()
self._router_name = _derive_router_name(platform_router)
self._translators, self._paths = self._backend.get_translators_for_path(
self._router_name
)
self.router_name = (
str(type(self.platform_router))
.rsplit(".", maxsplit=1)[-1]
.replace("'>", "")
.replace("ROUTER_", "")
.lower()
)
self.controller_name = f"{self.router_name.capitalize()}Controller"
@property
def router_name(self) -> str:
return self._router_name
@property
def controller_name(self) -> str:
return f"{self._router_name.capitalize()}Controller"
def create(self) -> type:
"""Create the platform controller."""
ClassName = self.controller_name
Parents = (PlatformController,)
Attributes: dict[str, bool | list[str]] = {"CHOICES_GENERATION": True}
# Menu and Command choices generation
"""Create the platform controller class for this router."""
choices_menus: list[str] = []
choices_commands: list[str] = []
translators = self._translated_target.translators
paths = self._translated_target.paths
# menus
for key, value in paths.items():
for key, value in self._paths.items():
if value == "path":
continue
choices_menus.append(key)
# commands
for name, _ in translators.items():
if any(f"{self.router_name}_{path}" in name for path in paths):
for name in self._translators:
if any(
f"{self._router_name}_{path}_" in f"{name}_" for path in self._paths
):
continue
new_name = name.replace(f"{self.router_name}_", "")
choices_commands.append(new_name)
choices_commands.append(name.replace(f"{self._router_name}_", ""))
Attributes["CHOICES_MENUS"] = choices_menus
Attributes["CHOICES_COMMANDS"] = choices_commands
attributes: dict[str, Any] = {
"CHOICES_GENERATION": True,
"CHOICES_MENUS": choices_menus,
"CHOICES_COMMANDS": choices_commands,
"_factory_backend": self._backend,
"_factory_translators": self._translators,
"_factory_paths": self._paths,
}
return type(self.controller_name, (PlatformController,), attributes)
# Use type to create the class
DynamicClass = type(ClassName, Parents, Attributes)
return DynamicClass
def _derive_router_name(platform_router: type) -> str:
"""Replicate the legacy class-name → router-name derivation."""
return (
str(type(platform_router))
.rsplit(".", maxsplit=1)[-1]
.replace("'>", "")
.replace("ROUTER_", "")
.lower()
)

99
cli/openbb_cli/controllers/script_parser.py
View File

@@ -5,14 +5,12 @@ from datetime import datetime, timedelta
from re import Match
from dateutil.relativedelta import relativedelta
from openbb_cli.session import Session
session = Session()
# pylint: disable=too-many-statements,consider-iterating-dictionary
# pylint: disable=too-many-branches,too-many-return-statements
# Necessary for OpenBB keywords
MONTHS_VALUE = {
"JANUARY": 1,
"FEBRUARY": 2,
@@ -45,12 +43,12 @@ def is_reset(command: str) -> bool:
Parameters
----------
command : str
The command to test
The command to test.
Returns
-------
answer : bool
Whether the command is a reset command
bool
Whether the command is a reset command.
"""
if "reset" in command:
return True
@@ -63,11 +61,12 @@ def match_and_return_openbb_keyword_date(keyword: str) -> str: # noqa: PLR0911
Parameters
----------
keyword : str
String with potential OpenBB keyword (e.g. 1MONTHAGO,LASTFRIDAY,3YEARSFROMNOW,NEXTTUESDAY)
String with potential OpenBB keyword.
Returns
----------
str: Date with format YYYY-MM-DD
-------
str
Date with format YYYY-MM-DD.
"""
now = datetime.now()
for i, regex in enumerate([r"^\$(\d+)([A-Z]+)AGO$", r"^\$(\d+)([A-Z]+)FROMNOW$"]):
@@ -79,22 +78,18 @@ def match_and_return_openbb_keyword_date(keyword: str) -> str: # noqa: PLR0911
if "DAYS" in clean_time or "MONTHS" in clean_time or "YEARS" in clean_time:
kwargs = {time_unit.lower(): integer_value}
if i == 0:
return (now - relativedelta(**kwargs)).strftime("%Y-%m-%d") # type: ignore
return (now + relativedelta(**kwargs)).strftime("%Y-%m-%d") # type: ignore
return (now - relativedelta(**kwargs)).strftime("%Y-%m-%d") # ty: ignore[invalid-argument-type]
return (now + relativedelta(**kwargs)).strftime("%Y-%m-%d") # ty: ignore[invalid-argument-type]
match = re.search(r"\$LAST(\w+)", keyword)
if match:
time_unit = match.group(1)
# Check if it corresponds to a month
if time_unit in list(MONTHS_VALUE.keys()):
the_year = now.year
# Calculate the year and month for last month date
if now.month <= MONTHS_VALUE[time_unit]:
# If the current month is greater than the last date month, it means it is this year
the_year = now.year - 1
return datetime(the_year, MONTHS_VALUE[time_unit], 1).strftime("%Y-%m-%d")
# Check if it corresponds to a week day
if time_unit in list(WEEKDAY_VALUE.keys()):
if datetime.weekday(now) > WEEKDAY_VALUE[time_unit]:
return (
@@ -112,11 +107,8 @@ def match_and_return_openbb_keyword_date(keyword: str) -> str: # noqa: PLR0911
match = re.search(r"\$NEXT(\w+)", keyword)
if match:
time_unit = match.group(1)
# Check if it corresponds to a month
if time_unit in list(MONTHS_VALUE.keys()):
# Calculate the year and month for next month date
if now.month < MONTHS_VALUE[time_unit]:
# If the current month is greater than the last date month, it means it is this year
return datetime(now.year, MONTHS_VALUE[time_unit], 1).strftime(
"%Y-%m-%d"
)
@@ -125,7 +117,6 @@ def match_and_return_openbb_keyword_date(keyword: str) -> str: # noqa: PLR0911
"%Y-%m-%d"
)
# Check if it corresponds to a week day
if time_unit in list(WEEKDAY_VALUE.keys()):
if datetime.weekday(now) < WEEKDAY_VALUE[time_unit]:
return (
@@ -151,35 +142,28 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
Parameters
----------
raw_lines : List[str]
Lines from .openbb script
script_inputs: str, optional
Inputs to the script that come externally
raw_lines : list[str]
Lines from .openbb script.
script_inputs : list[str] | None
Inputs to the script that come externally.
Returns
-------
str
Error that occurred - if empty means no error
str
Processed string from .openbb script that can be run by the OpenBB Platform CLI
tuple[str, str]
Error that occurred (empty means no error) and the processed script string.
"""
ROUTINE_VARS: dict[str, str | list[str]] = dict()
if script_inputs:
ROUTINE_VARS["$ARGV"] = script_inputs
## PRE PROCESSING
# Remove reset commands, comments, empty lines and trailing/leading whitespaces
raw_lines = [
x.strip()
for x in raw_lines
if (not is_reset(x)) and ("#" not in x) and x.strip()
]
## LOOK FOR NEW VARIABLES BEING DECLARED FROM USERS
lines_without_declarations = list()
for line in raw_lines:
# Check if this line has a variable attribution
# This currently allows user to override ARGV parameter
if "$" in line and "=" in line:
match = re.search(r"\$(\w+)\s*=\s*([\w\d,-.\s]+)", line)
if match:
@@ -189,7 +173,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
VAR_VALUES if "," not in VAR_VALUES else VAR_VALUES.split(",")
)
# Just throw a warning when user uses wrong convention
numdollars = len(re.findall(r"\$", line))
if numdollars > 1:
session.console.print(
@@ -203,19 +186,12 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
else:
lines_without_declarations.append(line)
# At this stage our ROUTINE_VARS should be completed coming from external AND from internal
# Now we want to replace the ROUTINE_VARS to where applicable throughout the .openbb script
# Due to this implementation, a variable declared at the end will still be effective
lines_with_vars_replaced = list()
foreach_loop_found = False
for line in lines_without_declarations:
# Save temporary line to ensure that all vars get replaced by correct vars
templine = line
# Found 'end' keyword which means that a loop has terminated
if re.match(r"^\s*end\s*$", line, re.IGNORECASE):
# Check whether the foreach loop has started or not
if not foreach_loop_found:
return (
"[red]The script has a foreach loop that terminates before it gets started. "
@@ -225,14 +201,11 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
foreach_loop_found = False
else:
# Found 'foreach' keyword which means there needs to be a matching 'end'
if re.search(r"foreach", line, re.IGNORECASE):
foreach_loop_found = True
# Regular expression pattern to match variables starting with $
pattern = r"(?<!\$)(\$(\w+)(\[[^\]]*\])?)(?=(?:[^\]]*\]*))"
# Find all matches of the pattern in the line
matches: list[Match[str]] | None = re.findall(pattern, line)
if matches:
@@ -241,11 +214,7 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
VAR_NAME = "$" + match[1]
VAR_SLICE = match[2][1:-1] if match[2] else ""
# Within a list refers to a single element
if VAR_SLICE.isdigit():
# This is an edge case for when the user has a variable such as $DATE = 2022-01-01
# We want the user to be able to access it with $DATE or $DATE[0] and the latest
# in python will only take the first '2'
if VAR_SLICE == "0":
if VAR_NAME in ROUTINE_VARS:
values = ROUTINE_VARS[VAR_NAME]
@@ -262,14 +231,12 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
"",
)
# Only enters here when any other index from 0 is used
elif VAR_NAME in ROUTINE_VARS:
variable = ROUTINE_VARS[VAR_NAME]
length_variable = (
len(variable) if isinstance(variable, list) else 1
)
# We use <= because we are using 0 index based lists
if length_variable <= int(VAR_SLICE):
return (
f"[red]Variable {VAR_NAME} only has "
@@ -287,7 +254,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
"",
)
# Involves slicing which is a bit more tricky to use eval on
elif (
":" in VAR_SLICE
and len(VAR_SLICE.split(":")) == 2
@@ -311,7 +277,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
)
vars_to_loop = ROUTINE_VARS[VAR_NAME][slice(start, stop)]
# Check whether the slicing was successful or not
if vars_to_loop:
templine = templine.replace(
match[0],
@@ -323,10 +288,8 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
"",
)
# Just replace value without slicing or list
else:
if VAR_SLICE:
# Check if the string starts with a minus sign
if VAR_SLICE.startswith("-"):
if not VAR_SLICE[1:].isdigit():
return (
@@ -347,7 +310,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
if VAR_NAME in ROUTINE_VARS:
value = ROUTINE_VARS[VAR_NAME]
# If the value is a list, we want to replace it with the whole list
if isinstance(value, list):
templine = templine.replace(
match[0],
@@ -357,9 +319,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
templine = templine.replace(match[0], value)
else:
# Check if this is an OpenBB keyword variable like
# 1MONTHAGO,LASTFRIDAY,3YEARSFROMNOW,NEXTTUESDAY
# and decode it into the right date if it exists
potential_date_match = (
match_and_return_openbb_keyword_date(VAR_NAME)
)
@@ -375,7 +334,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
lines_with_vars_replaced.append(templine)
# If this flags ends in True it means that the script routine has a foreach loop that never terminates
if foreach_loop_found:
return (
"[red]The script has a foreach loop that doesn't terminate. "
@@ -383,11 +341,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
"",
)
# Finally the only remaining thing to address are the foreach loops. For that we'll go through
# those lines and unroll the arguments that will be iterated by.
# Note that the fact that we checked before that the amount of foreach and end matches allow us
# to be confident that the script has no clear issues.
within_foreach = False
foreach_lines_loop: list[str] = list()
@@ -396,7 +349,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
varname = "VAR"
varused_inside = False
for line in lines_with_vars_replaced:
# Found 'foreach' header associated with loop
match = re.search(
r"foreach \$\$([A-Za-z\_]+) in ([A-Za-z0-9,-.]+)", line, re.IGNORECASE
)
@@ -405,16 +357,9 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
foreach_loop = match.group(2).split(",")
within_foreach = True
# We are inside a loop and this is a line that we will want to replicate,
# so we need to temporarily store it until we reach the end
elif within_foreach:
# Found 'end' keyword which means that the foreach loop has reached the end
if re.match(r"^\s*end\s*$", line, re.IGNORECASE):
# Now we want to process what we were waiting for before
# Iterate through main foreach header
for var in foreach_loop:
# Iterate through all lines within foreach and end loop
for foreach_line_loop in foreach_lines_loop:
if f"$${varname}" in foreach_line_loop:
final_lines.append(
@@ -437,7 +382,6 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
)
varused_inside = False
# Since this has been processed we reset the foreach loop lines
within_foreach = False
foreach_lines_loop = list()
@@ -447,25 +391,16 @@ def parse_openbb_script( # noqa: PLR0911,PLR0912
else:
final_lines.append(line)
# If the list is non null, then we want to convert this into a parsed string that is
# recognized by the OpenBB Platform CLI
if final_lines:
parsed_script = f"{'/'.join([line.rstrip() for line in final_lines])}".replace(
"//", "/home/"
)
if parsed_script[0] == "/":
# If the user had added a / at the beginning, then it was converted to //home/
# and we need to remove it
if parsed_script.startswith("//home"):
if parsed_script.startswith("//home"): # pragma: no cover
parsed_script = parsed_script[6:]
else:
# We want the script to start from the home menu, hence we add it if the user
# didn't add it
parsed_script = "/" + parsed_script
# If the script finishes with // it means that we converted it to /home/
# This means that we are expecting a command to follow, but since this is
# the end of the script, we need to remove the trailing /
if parsed_script.endswith("/home/"):
parsed_script = parsed_script[:-1]

6
cli/openbb_cli/controllers/settings_controller.py
View File

@@ -31,7 +31,7 @@ class SettingsController(BaseController):
"field_name": k,
}
for k, v in sorted(
session.settings.model_fields.items(),
session.settings.__class__.model_fields.items(),
key=lambda item: _get_extra(item[1]).get("command", ""),
)
if v.json_schema_extra
@@ -102,7 +102,6 @@ class SettingsController(BaseController):
if get_origin(annotation) is Literal:
choices = annotation.__args__
elif command == "console_style":
# To have updated choices for console style
choices = session.style.available_styles
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
@@ -116,13 +115,12 @@ class SettingsController(BaseController):
dest="value",
action="store",
required=False,
type=type_, # type: ignore[arg-type]
type=type_,
choices=choices,
)
ns_parser, _ = self.parse_simple_args(parser, other_args)
if ns_parser:
if ns_parser.value:
# Console style is applied immediately
if command == "console_style":
session.style.apply(ns_parser.value)
session.settings.set_item(field_name, ns_parser.value)

164
cli/openbb_cli/controllers/user_controller.py Normal file
View File

@@ -0,0 +1,164 @@
"""User Controller Module."""
import argparse
from functools import partial, update_wrapper
from types import MethodType
from typing import Annotated, Any, Literal, cast, get_args, get_origin
from openbb import obb as _obb_container
obb = cast(Any, _obb_container)
from openbb_cli.config.menu_text import MenuText
from openbb_cli.controllers.base_controller import BaseController
from openbb_cli.session import Session
session = Session()
class UserController(BaseController):
"""User Controller class."""
PATH = "/user/"
CHOICES_GENERATION = True
_PREF_COMMANDS: dict[str, dict[str, Any]] = {}
for _fname, _finfo in sorted(obb.user.preferences.__class__.model_fields.items()):
_annotation = _finfo.annotation
_is_bool = _annotation is bool
_PREF_COMMANDS[_fname] = {
"command": _fname,
"field_name": _fname,
"description": _finfo.description or _fname.replace("_", " "),
"annotation": _annotation,
"action": "toggle" if _is_bool else "set",
}
CHOICES_COMMANDS: list[str] = list(_PREF_COMMANDS.keys())
CHOICES_MENUS: list[str] = ["credentials"]
def __init__(self, queue: list[str] | None = None):
"""Initialize the Constructor."""
super().__init__(queue)
for cmd, field in self._PREF_COMMANDS.items():
self._generate_command(cmd, field, field["action"])
self.update_completer(self.choices_default)
def print_help(self):
"""Print help."""
mt = MenuText("user/")
mt.add_info("Session-level platform user settings (not persisted)")
mt.add_raw("\n")
mt.add_info("Toggles")
for cmd, f in self._PREF_COMMANDS.items():
if f["action"] == "toggle":
mt.add_setting(
name=cmd,
status=getattr(obb.user.preferences, f["field_name"]),
description=f["description"],
)
mt.add_raw("\n")
mt.add_info("Preferences")
for cmd, f in self._PREF_COMMANDS.items():
if f["action"] == "set":
current = getattr(obb.user.preferences, f["field_name"])
mt.add_cmd(
name=cmd,
description=f"{f['description']} [dim](current: {current})[/dim]",
)
mt.add_raw("\n")
mt.add_menu(
"credentials",
description="view and set API keys and tokens for the session",
)
session.console.print(text=mt.menu_text, menu="User")
def call_credentials(self, _):
"""Process credentials command."""
from openbb_cli.controllers.credentials_controller import (
CredentialsController,
)
self.queue = self.load_class(CredentialsController, self.queue)
def _generate_command(
self, cmd_name: str, field: dict, action_type: Literal["toggle", "set"]
):
"""Generate a toggle or set command bound to obb.user.preferences."""
def _toggle(self, other_args: list[str], field=field) -> None:
"""Toggle a boolean preference."""
field_name = field["field_name"]
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog=field["command"],
description=field["description"],
add_help=False,
)
ns_parser, _ = self.parse_simple_args(parser, other_args)
if ns_parser:
current = getattr(obb.user.preferences, field_name)
setattr(obb.user.preferences, field_name, not current)
session.console.print(f"[info]{field_name}:[/info] {not current}")
def _set(self, other_args: list[str], field=field) -> None:
"""Set a preference value."""
field_name = field["field_name"]
annotation = field["annotation"]
command = field["command"]
type_: type = str
choices = None
if get_origin(annotation) is Literal:
choices = list(get_args(annotation))
type_ = str
elif annotation is int or (
get_origin(annotation) is Annotated
and get_args(annotation)
and get_args(annotation)[0] is int
):
type_ = int
elif annotation is not str:
type_ = str
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog=command,
description=field["description"],
add_help=False,
)
parser.add_argument(
"-v",
"--value",
dest="value",
action="store",
required=False,
type=type_,
choices=choices,
)
ns_parser, _ = self.parse_simple_args(parser, other_args)
if ns_parser:
if ns_parser.value is not None:
setattr(obb.user.preferences, field_name, ns_parser.value)
session.console.print(
f"[info]{field_name}:[/info] {getattr(obb.user.preferences, field_name)}"
)
elif not other_args:
current = getattr(obb.user.preferences, field_name)
session.console.print(f"[info]{field_name}:[/info] {current}")
if action_type == "toggle":
action = _toggle
elif action_type == "set":
action = _set
else:
raise ValueError(f"Action type '{action_type}' not allowed.")
bound_method = update_wrapper(
wrapper=partial(MethodType(action, self), field=field), wrapped=action
)
setattr(self, f"call_{cmd_name}", bound_method)

594
cli/openbb_cli/controllers/utils.py
View File

@@ -2,34 +2,161 @@
import argparse
import ast
import io
import os
import random
import re
import shutil
import sqlite3
import subprocess
import sys
from contextlib import contextmanager
from datetime import (
datetime,
)
from pathlib import Path
from typing import TYPE_CHECKING, Optional
from typing import Any
from zoneinfo import ZoneInfo, available_timezones
import numpy as np
import pandas as pd
import requests
from openbb_cli.config.constants import AVAILABLE_FLAIRS, ENV_FILE_SETTINGS
from openbb_cli.session import Session
from openbb_core.app.model.obbject import OBBject
from pytz import all_timezones, timezone
all_timezones = available_timezones()
from rich.table import Table
if TYPE_CHECKING:
from openbb_charting.core.openbb_figure import OpenBBFigure
# pylint: disable=R1702,R0912
from openbb_cli.config.constants import AVAILABLE_FLAIRS, ENV_FILE_SETTINGS
from openbb_cli.session import Session
# pylint: disable=too-many-statements,no-member,too-many-branches,C0302
class SQLiteTable:
"""Lazy-loading wrapper for SQLite tables."""
def __init__(self, db_path: str, table_name: str, row_count: int = 0):
"""Initialize SQLite table wrapper.
Parameters
----------
db_path : str
Path to SQLite database file.
table_name : str
Name of the table.
row_count : int
Number of rows (from metadata).
"""
self.db_path = db_path
self.table_name = table_name
self.row_count = row_count
self._cached_df: pd.DataFrame | None = None
@property
def _quoted_name(self) -> str:
"""Return the table name quoted for safe SQL interpolation."""
return '"' + self.table_name.replace('"', '""') + '"'
def to_dataframe(self, use_cache: bool = True) -> pd.DataFrame:
"""Load table data from SQLite database.
Parameters
----------
use_cache : bool
If True, return cached DataFrame if available.
Returns
-------
pd.DataFrame
DataFrame containing table data.
"""
if use_cache and self._cached_df is not None:
return self._cached_df
conn = sqlite3.connect(self.db_path)
try:
sql = f"SELECT * FROM {self._quoted_name}" # noqa: S608
df = pd.read_sql_query(sql, conn)
if use_cache:
self._cached_df = df
return df
finally:
conn.close()
def get_schema(self) -> list[tuple]:
"""Get table schema (column names and types).
Returns
-------
list[tuple]
List of (column_name, type, notnull, default, pk) tuples.
"""
conn = sqlite3.connect(self.db_path)
try:
cursor = conn.cursor()
cursor.execute(f"PRAGMA table_info({self._quoted_name})")
return cursor.fetchall()
finally:
conn.close()
def query(self, where: str = "", limit: int | None = None) -> pd.DataFrame:
"""Execute SQL query with optional filters.
Parameters
----------
where : str
SQL WHERE clause (without WHERE keyword).
limit : int | None
Maximum number of rows to return.
Returns
-------
pd.DataFrame
DataFrame with query results.
"""
sql = f"SELECT * FROM {self._quoted_name}" # noqa: S608
if where:
sql += f" WHERE {where}"
if limit:
sql += f" LIMIT {limit}"
conn = sqlite3.connect(self.db_path)
try:
return pd.read_sql_query(sql, conn)
finally:
conn.close()
def extract_dataframe(obbject) -> pd.DataFrame:
"""Extract DataFrame from OBBject without using to_dataframe().
Parameters
----------
obbject
OBBject instance or other data.
Returns
-------
pd.DataFrame
DataFrame extracted from results.
"""
results = (
obbject.model_dump(exclude_unset=True, exclude_none=True).get("results")
if hasattr(obbject, "model_dump")
else obbject
)
if results is None:
return pd.DataFrame()
elif isinstance(results, SQLiteTable):
return results.to_dataframe()
elif isinstance(results, pd.DataFrame):
return results
elif isinstance(results, list):
return pd.DataFrame(results)
elif isinstance(results, dict):
return pd.DataFrame([results])
else:
return pd.DataFrame({"value": [results]})
session = Session()
@@ -47,7 +174,6 @@ def remove_file(path: Path) -> bool:
bool
The status of the removal.
"""
# TODO: Check why module level import leads to circular import.
try:
if os.path.isfile(path):
os.remove(path)
@@ -79,52 +205,52 @@ Please feel free to check out our other products:
def bootup():
"""Bootup the cli."""
if sys.platform == "win32":
# Enable VT100 Escape Sequence for WINDOWS 10 Ver. 1607
os.system("") # nosec # noqa: S605,S607
if sys.platform == "win32": # pragma: no cover
import ctypes
std_output_handle = -11
enable_virtual_terminal_processing = 0x0004
kernel32 = ctypes.windll.kernel32
handle = kernel32.GetStdHandle(std_output_handle)
mode = ctypes.c_ulong()
if kernel32.GetConsoleMode(handle, ctypes.byref(mode)):
kernel32.SetConsoleMode(
handle, mode.value | enable_virtual_terminal_processing
)
try:
if os.name == "nt":
# pylint: disable=E1101
sys.stdin.reconfigure(encoding="utf-8") # type: ignore
# pylint: disable=E1101
sys.stdout.reconfigure(encoding="utf-8") # type: ignore
except Exception as e:
if os.name == "nt": # pragma: no cover
for stream in (sys.stdin, sys.stdout):
if isinstance(stream, io.TextIOWrapper):
stream.reconfigure(encoding="utf-8")
except Exception as e: # pragma: no cover
session.console.print(e, "\n")
def welcome_message():
"""Print the welcome message.
Prints first welcome message, help and a notification if updates are available.
"""
"""Print the welcome message."""
session.console.print(
f"\nWelcome to OpenBB Platform CLI v{session.settings.VERSION}"
)
def reset(queue: list[str] | None = None):
"""Reset the CLI.
Allows for checking code without quitting.
"""
"""Reset the CLI."""
session.console.print("resetting...")
debug = session.settings.DEBUG_MODE
dev = session.settings.DEV_BACKEND
try:
# we clear all openbb_cli modules from sys.modules
for module in list(sys.modules.keys()):
parts = module.split(".")
if parts[0] == "openbb_cli":
del sys.modules[module]
queue_list = ["/".join(queue) if len(queue) > 0 else ""] # type: ignore
queue_list = ["/".join(queue) if len(queue) > 0 else ""] # ty: ignore[invalid-argument-type, no-matching-overload]
# pylint: disable=import-outside-toplevel
from openbb_cli.controllers.cli_controller import main
main(debug, dev, queue_list, module="") # type: ignore
main(debug, dev, queue_list, module="")
except Exception as e:
session.console.print(f"Unfortunately, resetting wasn't possible: {e}\n")
@@ -149,13 +275,10 @@ def suppress_stdout():
def first_time_user() -> bool:
"""Check whether a user is a first time user.
A first time user is someone with an empty .env file.
If this is true, it also adds an env variable to make sure this does not run again.
Returns
-------
bool
Whether or not the user is a first time user
Whether or not the user is a first time user.
"""
if ENV_FILE_SETTINGS.stat().st_size == 0:
session.settings.set_item("PREVIOUS_USE", True)
@@ -166,33 +289,29 @@ def first_time_user() -> bool:
def parse_and_split_input(an_input: str, custom_filters: list) -> list[str]:
"""Filter and split the input queue.
Uses regex to filters command arguments that have forward slashes so that it doesn't
break the execution of the command queue.
Currently handles unix paths and sorting settings for screener menus.
Parameters
----------
an_input : str
User input as string
custom_filters : List
Additional regular expressions to match
User input as string.
custom_filters : list
Additional regular expressions to match.
Returns
-------
List[str]
Command queue as list
list[str]
Command queue as list.
"""
# Make sure that the user can go back to the root when doing "/"
if an_input and an_input == "/":
an_input = "home"
# everything from ` -f ` to the next known extension
file_flag = r"(\ -f |\ --file )"
up_to = r".*?"
known_extensions = r"(\.(xlsx|csv|xls|tsv|json|yaml|ini|openbb|ipynb))"
unix_path_arg_exp = f"({file_flag}{up_to}{known_extensions})"
known_extensions = (
r"(\.(xlsx|csv|xls|tsv|json|yaml|ini|openbb|ipynb|db|sqlite|sqlite3))"
)
optional_args = r"(?:\ [^/]+)*?"
unix_path_arg_exp = f"({file_flag}{up_to}{known_extensions}{optional_args})"
# Add custom expressions to handle edge cases of individual controllers
custom_filter = ""
for exp in custom_filters:
if exp is not None:
@@ -207,9 +326,7 @@ def parse_and_split_input(an_input: str, custom_filters: list) -> list[str]:
match = re.search(pattern=slash_filter_exp, string=an_input)
if match is not None:
placeholder = f"{{placeholder{len(placeholders) + 1}}}"
placeholders[placeholder] = an_input[
match.span()[0] : match.span()[1]
] # noqa:E203
placeholders[placeholder] = an_input[match.span()[0] : match.span()[1]] # noqa:E203
an_input = (
an_input[: match.span()[0]] + placeholder + an_input[match.span()[1] :]
) # noqa:E203
@@ -225,28 +342,24 @@ def parse_and_split_input(an_input: str, custom_filters: list) -> list[str]:
if len(matching_placeholders) > 0:
for tag in matching_placeholders:
commands[command_num] = command.replace(tag, placeholders[tag])
return commands
return list(filter(None, commands))
def return_colored_value(value: str):
"""Return the string value based on condition.
Return it with green, yellow, red or white color based on
whether the number is positive, negative, zero or other, respectively.
Parameters
----------
value: str
string to be checked
value : str
String to be checked.
Returns
-------
value: str
string with color based on value of number if it exists
str
String with color based on value of number if it exists.
"""
values = re.findall(r"[-+]?(?:\d*\.\d+|\d+)", value)
# Finds exactly 1 number in the string
if len(values) == 1:
if float(values[0]) > 0:
return f"[green]{value}[/green]"
@@ -260,7 +373,6 @@ def return_colored_value(value: str):
return f"{value}"
# pylint: disable=too-many-arguments,too-many-positional-arguments
def print_rich_table( # noqa: PLR0912
df: pd.DataFrame,
show_index: bool = False,
@@ -281,45 +393,39 @@ def print_rich_table( # noqa: PLR0912
Parameters
----------
df: pd.DataFrame
Dataframe to turn into table
show_index: bool
Whether to include index
title: str
Title for table
df : pd.DataFrame
Dataframe to turn into table.
show_index : bool
Whether to include index.
title : str
Title for table.
index_name : str
Title for index column
headers: List[str]
Titles for columns
floatfmt: Union[str, List[str]]
Float number formatting specs as string or list of strings. Defaults to ".2f"
show_header: bool
Title for index column.
headers : list[str] | pd.Index | None
Titles for columns.
floatfmt : str | list[str]
Float number formatting specs as string or list of strings.
show_header : bool
Whether to show the header row.
automatic_coloring: bool
Automatically color a table based on positive and negative values
columns_to_auto_color: List[str]
Columns to automatically color
rows_to_auto_color: List[str]
Rows to automatically color
export: bool
Whether we are exporting the table to a file. If so, we don't want to print it.
limit: Optional[int]
automatic_coloring : bool
Automatically color a table based on positive and negative values.
columns_to_auto_color : list[str] | None
Columns to automatically color.
rows_to_auto_color : list[str] | None
Rows to automatically color.
export : bool
Whether we are exporting the table to a file.
limit : int | None
Limit the number of rows to show.
columns_keep_types: Optional[List[str]]
Columns to keep their types, i.e. not convert to numeric
columns_keep_types : list[str] | None
Columns to keep their types, i.e. not convert to numeric.
"""
if export:
return
MAX_COLS = session.settings.ALLOWED_NUMBER_OF_COLUMNS
MAX_ROWS = session.settings.ALLOWED_NUMBER_OF_ROWS
# Make a copy of the dataframe to avoid SettingWithCopyWarning
df = df.copy()
show_index = not isinstance(df.index, pd.RangeIndex) and show_index
# convert non-str that are not timestamp or int into str
# eg) praw.models.reddit.subreddit.Subreddit
for col in df.columns:
if columns_keep_types is not None and col in columns_keep_types:
continue
@@ -339,17 +445,14 @@ def print_rich_table( # noqa: PLR0912
output = list(_headers)
if len(output) != len(df.columns):
raise ValueError("Length of headers does not match length of DataFrame.")
return output # type: ignore
return output # ty: ignore[invalid-return-type]
if session.settings.USE_INTERACTIVE_DF:
if session.settings.USE_INTERACTIVE_DF and session.backend is not None:
df_outgoing = df.copy()
# If headers are provided, use them
if headers is not None:
# We check if headers are valid
df_outgoing.columns = _get_headers(headers)
if show_index and index_name not in df_outgoing.columns:
# If index name is provided, we use it
df_outgoing.index.name = index_name or "Index"
df_outgoing = df_outgoing.reset_index()
@@ -357,47 +460,31 @@ def print_rich_table( # noqa: PLR0912
if col == "":
df_outgoing = df_outgoing.rename(columns={col: " "})
session._backend.send_table( # type: ignore # pylint: disable=protected-access
df_table=df_outgoing,
title=title,
theme=session.user.preferences.table_style,
)
return
try:
session.backend.send_table(
df_table=df_outgoing,
title=title,
theme=session.user.preferences.table_style,
)
return
except Exception: # noqa: S110
pass
df = df.copy() if not limit else df.copy().iloc[:limit]
if automatic_coloring:
if columns_to_auto_color:
for col in columns_to_auto_color:
# checks whether column exists
if col in df.columns:
df[col] = df[col].apply(lambda x: return_colored_value(str(x)))
if rows_to_auto_color:
for row in rows_to_auto_color:
# checks whether row exists
if row in df.index:
df.loc[row] = df.loc[row].apply(
lambda x: return_colored_value(str(x))
)
if columns_to_auto_color is None and rows_to_auto_color is None:
df = df.map(lambda x: return_colored_value(str(x))) # type: ignore
exceeds_allowed_columns = len(df.columns) > MAX_COLS
exceeds_allowed_rows = len(df) > MAX_ROWS
if exceeds_allowed_columns:
original_columns = df.columns.tolist()
trimmed_columns = df.columns.tolist()[:MAX_COLS]
df = df[trimmed_columns]
trimmed_columns = [
col for col in original_columns if col not in trimmed_columns
]
if exceeds_allowed_rows:
n_rows = len(df.index)
max_rows = MAX_ROWS
df = df[:max_rows]
trimmed_rows_count = n_rows - max_rows
df = df.map(lambda x: return_colored_value(str(x)))
if use_tabulate_df:
table = Table(title=title, show_lines=True, show_header=show_header)
@@ -423,7 +510,6 @@ def print_rich_table( # noqa: PLR0912
floatfmt = [floatfmt for _ in range(len(df.columns))]
for idx, values in zip(df.index.tolist(), df.values.tolist()):
# remove hour/min/sec from timestamp index - Format: YYYY-MM-DD # make better
row_idx = [str(idx)] if show_index else []
row_idx += [
(
@@ -446,23 +532,6 @@ def print_rich_table( # noqa: PLR0912
else:
session.console.print(df.to_string(col_space=0))
if exceeds_allowed_columns:
session.console.print(
f"[yellow]\nAllowed number of columns exceeded ({session.settings.ALLOWED_NUMBER_OF_COLUMNS}).\n"
f"The following columns were removed from the output: {', '.join(trimmed_columns)}.\n[/yellow]"
)
if exceeds_allowed_rows:
session.console.print(
f"[yellow]\nAllowed number of rows exceeded ({session.settings.ALLOWED_NUMBER_OF_ROWS}).\n"
f"{trimmed_rows_count} rows were removed from the output.\n[/yellow]"
)
if exceeds_allowed_columns or exceeds_allowed_rows:
session.console.print(
"Use the `--export` flag to analyse the full output on a file."
)
def check_non_negative(value) -> int:
"""Argparse type to check non negative int."""
@@ -489,6 +558,41 @@ def validate_register_key(value: str) -> str:
return str(value)
def get_user_data_directory() -> Path:
"""Get the OpenBBUserData directory path."""
return Path(session.user.preferences.data_directory)
def get_data_files_for_completion() -> list[str]:
"""Get list of data files in OpenBBUserData for tab completion.
Returns list of file paths relative to OpenBBUserData directory.
Includes CSV, JSON, and Excel files.
"""
try:
user_data_dir = get_user_data_directory()
if not user_data_dir.exists():
return []
files = []
for file_path in user_data_dir.rglob("*"):
if file_path.is_file() and file_path.suffix.lower() in [
".csv",
".json",
".xlsx",
".xls",
".db",
".sqlite",
".sqlite3",
]:
rel_path = file_path.relative_to(user_data_dir)
files.append(str(rel_path).replace("\\", "/"))
return sorted(files)
except Exception:
return []
def get_user_agent() -> str:
"""Get a not very random user agent."""
user_agent_strings = [
@@ -501,7 +605,7 @@ def get_user_agent() -> str:
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:84.0) Gecko/20100101 Firefox/84.0",
]
return random.choice(user_agent_strings) # nosec # noqa: S311
return random.choice(user_agent_strings) # noqa: S311
def get_flair() -> str:
@@ -515,7 +619,7 @@ def get_dtime() -> str:
"""Get a datetime string."""
dtime = ""
if session.settings.USE_DATETIME and get_user_timezone_or_invalid() != "INVALID":
dtime = datetime.now(timezone(get_user_timezone())).strftime("%Y %b %d, %H:%M")
dtime = datetime.now(ZoneInfo(get_user_timezone())).strftime("%Y %b %d, %H:%M")
return dtime
@@ -533,13 +637,13 @@ def is_timezone_valid(user_tz: str) -> bool:
Parameters
----------
user_tz: str
Timezone to check for validity
user_tz : str
Timezone to check for validity.
Returns
-------
bool
True if timezone provided is valid
True if timezone provided is valid.
"""
return user_tz in all_timezones
@@ -550,7 +654,7 @@ def get_user_timezone() -> str:
Returns
-------
str
user timezone based on .env file
User timezone based on .env file.
"""
return session.settings.TIMEZONE
@@ -561,7 +665,7 @@ def get_user_timezone_or_invalid() -> str:
Returns
-------
str
user timezone based on timezone.openbb file or INVALID
User timezone based on timezone.openbb file or INVALID.
"""
user_tz = get_user_timezone()
if is_timezone_valid(user_tz):
@@ -574,13 +678,13 @@ def check_file_type_saved(valid_types: list[str] | None = None):
Parameters
----------
valid_types: List[str]
List of valid types to export data
valid_types : list[str] | None
List of valid types to export data.
Returns
-------
check_filenames: Optional[List[str]]
Function that returns list of filenames to export data
Callable
Function that returns list of filenames to export data.
"""
def check_filenames(filenames: str = "") -> str:
@@ -588,13 +692,13 @@ def check_file_type_saved(valid_types: list[str] | None = None):
Parameters
----------
filenames: str
filenames to be saved separated with comma
filenames : str
Filenames to be saved separated with comma.
Returns
----------
-------
str
valid filenames separated with comma
Valid filenames separated with comma.
"""
if not filenames or not valid_types:
return ""
@@ -618,21 +722,20 @@ def remove_timezone_from_dataframe(df: pd.DataFrame) -> pd.DataFrame:
Parameters
----------
df : pd.DataFrame
The dataframe to remove timezone information from
The dataframe to remove timezone information from.
Returns
-------
pd.DataFrame
The dataframe with timezone information removed
The dataframe with timezone information removed.
"""
date_cols = []
index_is_date = False
# Find columns and index containing date data
if (
df.index.dtype.kind == "M"
and hasattr(df.index.dtype, "tz")
and df.index.dtype.tz is not None # type: ignore
and df.index.dtype.tz is not None
):
index_is_date = True
@@ -640,13 +743,12 @@ def remove_timezone_from_dataframe(df: pd.DataFrame) -> pd.DataFrame:
if dtype.kind == "M" and hasattr(df.index.dtype, "tz") and dtype.tz is not None:
date_cols.append(col)
# Remove the timezone information
for col in date_cols:
df[col] = df[col].dt.date
if index_is_date:
index_name = df.index.name
df.index = df.index.date # type: ignore
df.index = df.index.date # ty: ignore[unresolved-attribute]
df.index.name = index_name
return df
@@ -655,25 +757,20 @@ def remove_timezone_from_dataframe(df: pd.DataFrame) -> pd.DataFrame:
def compose_export_path(func_name: str, dir_path: str) -> Path:
"""Compose export path for data from the terminal.
Creates a path to a folder and a filename based on conditions.
Parameters
----------
func_name : str
Name of the command that invokes this function
Name of the command that invokes this function.
dir_path : str
Path of directory from where this function is called
Path of directory from where this function is called.
Returns
-------
Path
Path variable containing the path of the exported file
Path variable containing the path of the exported file.
"""
now = datetime.now()
# Resolving all symlinks and also normalizing path.
resolve_path = Path(dir_path).resolve()
# Getting the directory names from the path. Instead of using split/replace (Windows doesn't like that)
# check if this is done in a main context to avoid saving with openbb_cli
if resolve_path.parts[-2] == "openbb_cli":
path_cmd = f"{resolve_path.parts[-1]}"
else:
@@ -689,8 +786,10 @@ def compose_export_path(func_name: str, dir_path: str) -> Path:
def ask_file_overwrite(file_path: Path) -> tuple[bool, bool]:
"""Provide a prompt for overwriting existing files.
Returns two values, the first is a boolean indicating if the file exists and the
second is a boolean indicating if the user wants to overwrite the file.
Returns
-------
tuple[bool, bool]
Whether the file exists and whether the user wants to overwrite it.
"""
if session.settings.FILE_OVERWRITE:
return False, True
@@ -700,26 +799,28 @@ def ask_file_overwrite(file_path: Path) -> tuple[bool, bool]:
overwrite = input("\nFile already exists. Overwrite? [y/n]: ").lower()
if overwrite == "y":
file_path.unlink(missing_ok=True)
# File exists and user wants to overwrite
return True, True
# File exists and user does not want to overwrite
return True, False
# File does not exist
return False, True
# This is a false positive on pylint and being tracked in pylint #3060
# pylint: disable=abstract-class-instantiated,too-many-positional-arguments
def save_to_excel(df, saved_path, sheet_name, start_row=0, index=True, header=True):
"""Save a Pandas DataFrame to an Excel file.
Args:
df: A Pandas DataFrame.
saved_path: The path to the Excel file to save to.
sheet_name: The name of the sheet to save the DataFrame to.
start_row: The row number to start writing the DataFrame at.
index: Whether to write the DataFrame index to the Excel file.
header: Whether to write the DataFrame header to the Excel file.
Parameters
----------
df
A Pandas DataFrame.
saved_path
The path to the Excel file to save to.
sheet_name
The name of the sheet to save the DataFrame to.
start_row
The row number to start writing the DataFrame at.
index
Whether to write the DataFrame index to the Excel file.
header
Whether to write the DataFrame header to the Excel file.
"""
overwrite_options = {
"o": "replace",
@@ -746,7 +847,7 @@ def save_to_excel(df, saved_path, sheet_name, start_row=0, index=True, header=Tr
with pd.ExcelWriter(
saved_path,
mode="a",
if_sheet_exists=overwrite_options[overwrite_option], # type: ignore
if_sheet_exists=overwrite_options[overwrite_option], # ty: ignore[invalid-argument-type]
engine="openpyxl",
) as writer:
df.to_excel(
@@ -758,15 +859,13 @@ def save_to_excel(df, saved_path, sheet_name, start_row=0, index=True, header=Tr
)
# This is a false positive on pylint and being tracked in pylint #3060
# pylint: disable=abstract-class-instantiated,too-many-positional-arguments
def export_data(
def export_data( # noqa: PLR0912
export_type: str,
dir_path: str,
func_name: str,
df: pd.DataFrame = pd.DataFrame(),
sheet_name: str | None = None,
figure: Optional["OpenBBFigure"] = None,
figure: Any = None,
margin: bool = True,
) -> None:
"""Export data to a file.
@@ -774,17 +873,17 @@ def export_data(
Parameters
----------
export_type : str
Type of export between: csv,json,xlsx,xls
Type of export between: csv, json, xlsx, xls.
dir_path : str
Path of directory from where this function is called
Path of directory from where this function is called.
func_name : str
Name of the command that invokes this function
df : pd.Dataframe
Dataframe of data to save
sheet_name : str
If provided. The name of the sheet to save in excel file
figure : Optional[OpenBBFigure]
Figure object to save as image file
Name of the command that invokes this function.
df : pd.DataFrame
Dataframe of data to save.
sheet_name : str | None
The name of the sheet to save in excel file.
figure : Any
Figure object to save as image file.
margin : bool
Automatically adjust subplot parameters to give specified padding.
"""
@@ -792,10 +891,8 @@ def export_data(
saved_path = compose_export_path(func_name, dir_path).resolve()
saved_path.parent.mkdir(parents=True, exist_ok=True)
for exp_type in export_type.split(","):
# In this scenario the path was provided, e.g. --export pt.csv, pt.jpg
if "." in exp_type:
saved_path = saved_path.with_name(exp_type)
# In this scenario we use the default filename
else:
if ".OpenBB_openbb_cli" in saved_path.name:
saved_path = saved_path.with_name(
@@ -832,7 +929,6 @@ def export_data(
df.reset_index(drop=True, inplace=True)
df.to_json(saved_path)
elif exp_type.endswith("xlsx"):
# since xlsx does not support datetimes with timezones we need to remove it
df = remove_timezone_from_dataframe(df)
if sheet_name is None: # noqa: SIM223
@@ -849,6 +945,51 @@ def export_data(
session.console.print("No plot to export.")
continue
figure.show(export_image=saved_path, margin=margin)
elif saved_path.suffix in [".db", ".sqlite", ".sqlite3"]:
import sqlite3
table_name = sheet_name if sheet_name else "data"
conn = sqlite3.connect(saved_path)
try:
cursor = conn.cursor()
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name=?",
(table_name,),
)
table_exists = cursor.fetchone() is not None
if table_exists:
choice = input(
f"\nTable '{table_name}' exists. Overwrite/Append/New? [o/a/n]: "
).lower()
if choice == "o":
df.to_sql(
table_name, conn, if_exists="replace", index=False
)
elif choice == "a":
df.to_sql(table_name, conn, if_exists="append", index=False)
elif choice == "n":
i = 1
new_name = f"{table_name}_{i}"
while True:
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name=?",
(new_name,),
)
if cursor.fetchone() is None:
break
i += 1
new_name = f"{table_name}_{i}"
df.to_sql(new_name, conn, if_exists="fail", index=False)
table_name = new_name
else:
session.console.print("Invalid choice. Skipping.")
continue
else:
df.to_sql(table_name, conn, if_exists="fail", index=False)
finally:
conn.close()
else:
session.console.print("Wrong export file specified.")
continue
@@ -859,15 +1000,16 @@ def export_data(
session.console.print(f"Failed to save file: {saved_path}")
if figure is not None:
figure._exported = True # pylint: disable=protected-access
figure._exported = True
def system_clear():
"""Clear screen."""
os.system("cls||clear") # nosec # noqa: S605,S607
subprocess.run( # noqa: S602
"cls" if os.name == "nt" else "clear", shell=True, check=False
)
# Write an abstract helper to make requests from a url with potential headers and params
def request(
url: str, method: str = "get", timeout: int = 0, **kwargs
) -> requests.Response:
@@ -876,29 +1018,25 @@ def request(
Parameters
----------
url : str
Url to make the request to
Url to make the request to.
method : str
HTTP method to use. Choose from:
delete, get, head, patch, post, put, by default "get"
HTTP method to use: delete, get, head, patch, post, put.
timeout : int
How many seconds to wait for the server to send data
How many seconds to wait for the server to send data.
Returns
-------
requests.Response
Request response object
Request response object.
Raises
------
ValueError
If invalid method is passed
If invalid method is passed.
"""
method = method.lower()
if method not in ["delete", "get", "head", "patch", "post", "put"]:
raise ValueError(f"Invalid method: {method}")
# We want to add a user agent to the request, so check if there are any headers
# If there are headers, check if there is a user agent, if not add one.
# Some requests seem to work only with a specific user agent, so we want to be able to override it.
headers = kwargs.pop("headers", {})
timeout = timeout or session.user.preferences.request_timeout
@@ -942,27 +1080,41 @@ def handle_obbject_display(
):
"""Handle the display of an OBBject."""
df: pd.DataFrame = pd.DataFrame()
fig: OpenBBFigure | None = None
fig: Any = None
if isinstance(getattr(obbject, "results", None), SQLiteTable):
sqlite_tbl: SQLiteTable = obbject.results # ty: ignore[invalid-assignment]
obbject.results = sqlite_tbl.to_dataframe()
if chart:
try:
if obbject.chart:
obbject.show(**kwargs)
else:
obbject.charting.to_chart(**kwargs) # type: ignore
obbject.charting.to_chart(**kwargs) # ty: ignore[unresolved-attribute]
if export:
fig = obbject.chart.fig # type: ignore
df = obbject.to_dataframe()
fig = obbject.chart.fig # ty: ignore[unresolved-attribute]
df = extract_dataframe(obbject)
except Exception as e:
session.console.print(f"Failed to display chart: {e}")
elif session.settings.USE_INTERACTIVE_DF:
obbject.charting.table() # type: ignore
try:
obbject.charting.table() # ty: ignore[unresolved-attribute]
except AttributeError:
df = extract_dataframe(obbject)
session.output_adapter.display(
data=df,
title=obbject.extra.get("command", ""),
export=bool(export),
chart=False,
)
else:
df = obbject.to_dataframe()
print_rich_table(
df=df,
show_index=True,
df = extract_dataframe(obbject)
session.output_adapter.display(
data=df,
title=obbject.extra.get("command", ""),
export=bool(export),
chart=False,
)
if export and not df.empty:
if sheet_name and isinstance(sheet_name, list):