jerry/OpenBB
1
0
Fork 0
mirror of https://github.com/OpenBB-finance/OpenBB.git synced 2026-10-27 12:40:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

[Feature] - Custom deprecation (#6005)

Browse Source 
* custom deprecation

* custom deprecation

* using the new deprecation

* custom deprecation on the package builder

* remove comment

* ruff

* black

* static assets

* tests

* using parametrization instead

* test for deprecated endpoints (#6014)

* Deprecation warning on the reference docs (#6015)

* typo/fix

* bring back methods needed for markdown generation

* add deprecation warning to docs

* contributor docs for deprecating endpoints - tks @deeleeramone

* small changes on publishing procedure per @the-praxs

* moving the deprecation summary class to deprecation file instead

* explanation on class variables

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update openbb_platform/openbb/package/index.py

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* Update website/content/platform/development/contributor-guidelines/deprecating_endpoints.md

Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>

* deprecating on 4.3 instead @the-praxs

---------

Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com>
Co-authored-by: Pratyush Shukla <ps4534@nyu.edu>
This commit is contained in:
Henrique Joaquim
2024-02-01 10:00:47 +00:00
committed by GitHub
parent 5fd11beaee
commit d4baba0284
13 changed files with 353 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
build/pypi/openbb_platform/PUBLISH.md vendored
View File

@@ -8,7 +8,7 @@ Publishing checklist:
2. Ensure all integration tests pass: `pytest openbb_platform -m integration`
3. Run `python -c "import openbb; openbb.build()"` to build the static assets. Make sure that only required extensions are installed.
> **Note** Run `python -c "import openbb"` after building the static to check that no additional static is being built.
> **Note** Run `python -c "import openbb"` after building the static to check that no additional static is being built.
4. Run the following commands for publishing the packages to PyPI:
@@ -17,11 +17,13 @@ Publishing checklist:
1. For the core package run: `python build/pypi/openbb_platform/publish.py --core`
2. For the extension and provider packages run: `python build/pypi/openbb_platform/publish.py --extensions`
3. For the `openbb` package, do the following
3. For the `openbb` package - **which requires manual publishing** - do the following
- Bump the dependency package versions
- Re-build the static assets that are bundled with the package
- Run unit tests to validate the existence of deprecated endpoints
> Note that, in order for packages to pick up the latest versions of dependencies, it might be necessary to clear the local cache of the dependencies:
> [!TIP]
> Note that, in order for packages to pick up the latest versions of dependencies, it is advised to clear the local cache of the dependencies:
>
> We can do that with `pip cache purge` and `poetry cache clear pypi --all`
>

62
openbb_platform/core/openbb_core/app/deprecation.py Normal file
View File

@@ -0,0 +1,62 @@
"""
OpenBB-specific deprecation warnings.
This implementation was inspired from Pydantic's specific warnings and modified to suit OpenBB's needs.
"""
from typing import Optional, Tuple
class DeprecationSummary(str):
"""A string subclass that can be used to store deprecation metadata."""
def __new__(cls, value, metadata):
"""Create a new instance of the class."""
obj = str.__new__(cls, value)
obj.metadata = metadata
return obj
class OpenBBDeprecationWarning(DeprecationWarning):
"""
A OpenBB specific deprecation warning.
This warning is raised when using deprecated functionality in OpenBB. It provides information on when the
deprecation was introduced and the expected version in which the corresponding functionality will be removed.
Attributes
----------
message: Description of the warning.
since: Version in what the deprecation was introduced.
expected_removal: Version in what the corresponding functionality expected to be removed.
"""
# The choice to use class variables is based on the potential for extending the class in future developments.
# Example: launching Platform V5 and decide to create a subclimagine we areass named OpenBBDeprecatedSinceV4,
# which inherits from OpenBBDeprecationWarning. In this subclass, we would set since=4.X and expected_removal=5.0.
# It's important for these values to be defined at the class level, rather than just at the instance level,
# to ensure consistency and clarity in our deprecation warnings across the platform.
message: str
since: Tuple[int, int]
expected_removal: Tuple[int, int]
def __init__(
self,
message: str,
*args: object,
since: Tuple[int, int],
expected_removal: Optional[Tuple[int, int]] = None,
) -> None:
super().__init__(message, *args)
self.message = message.rstrip(".")
self.since = since
self.expected_removal = expected_removal or (since[0] + 1, 0)
self.long_message = (
f"{self.message}. Deprecated in OpenBB Platform V{self.since[0]}.{self.since[1]}"
f" to be removed in V{self.expected_removal[0]}.{self.expected_removal[1]}."
)
def __str__(self) -> str:
"""Return the warning message."""
return self.long_message

15
openbb_platform/core/openbb_core/app/router.py
View File

@@ -24,6 +24,7 @@ from pydantic import BaseModel
from pydantic.v1.validators import find_validators
from typing_extensions import Annotated, ParamSpec, _AnnotatedAlias
from openbb_core.app.deprecation import DeprecationSummary, OpenBBDeprecationWarning
from openbb_core.app.example_generator import ExampleGenerator
from openbb_core.app.extension_loader import ExtensionLoader
from openbb_core.app.model.abstract.warning import OpenBBWarning
@@ -232,7 +233,6 @@ class Router:
api_router = self._api_router
model = kwargs.pop("model", "")
deprecation_message = kwargs.pop("deprecation_message", None)
examples = kwargs.pop("examples", [])
exclude_auto_examples = kwargs.pop("exclude_auto_examples", False)
@@ -279,14 +279,13 @@ class Router:
},
)
# For custom deprecation messages
# For custom deprecation
if kwargs.get("deprecated", False):
if deprecation_message:
kwargs["summary"] = deprecation_message
else:
kwargs["summary"] = (
"This functionality will be deprecated in the future releases."
)
deprecation: OpenBBDeprecationWarning = kwargs.pop("deprecation")
kwargs["summary"] = DeprecationSummary(
deprecation.long_message, deprecation
)
api_router.add_api_route(**kwargs)

146
openbb_platform/core/openbb_core/app/static/package_builder.py
View File

@@ -9,9 +9,11 @@ from inspect import Parameter, _empty, isclass, signature
from json import dumps, load
from pathlib import Path
from typing import (
Any,
Callable,
Dict,
List,
Literal,
Optional,
OrderedDict,
Set,
@@ -20,6 +22,7 @@ from typing import (
TypeVar,
Union,
get_args,
get_origin,
get_type_hints,
)
@@ -35,7 +38,7 @@ from openbb_core.app.charting_service import ChartingService
from openbb_core.app.extension_loader import ExtensionLoader, OpenBBGroups
from openbb_core.app.model.custom_parameter import OpenBBCustomParameter
from openbb_core.app.provider_interface import ProviderInterface
from openbb_core.app.router import RouterLoader
from openbb_core.app.router import CommandMap, RouterLoader
from openbb_core.app.static.utils.console import Console
from openbb_core.app.static.utils.linters import Linters
from openbb_core.env import Env
@@ -302,6 +305,8 @@ class ImportDefinition:
for child_path in child_path_list:
route = PathHandler.get_route(path=child_path, route_map=route_map)
if route:
if route.deprecated:
hint_type_list.append(type(route.summary.metadata))
function_hint_type_list = cls.get_function_hint_type_list(func=route.endpoint) # type: ignore
hint_type_list.extend(function_hint_type_list)
@@ -331,10 +336,11 @@ class ImportDefinition:
code += "\nimport typing"
code += "\nfrom typing import List, Dict, Union, Optional, Literal"
code += "\nfrom annotated_types import Ge, Le, Gt, Lt"
code += "\nfrom warnings import warn, simplefilter"
if sys.version_info < (3, 9):
code += "\nimport typing_extensions"
else:
code += "\nfrom typing_extensions import Annotated"
code += "\nfrom typing_extensions import Annotated, deprecated"
code += "\nfrom openbb_core.app.utils import df_to_basemodel"
code += "\nfrom openbb_core.app.static.utils.decorators import validate\n"
code += "\nfrom openbb_core.app.static.utils.filters import filter_inputs\n"
@@ -347,6 +353,15 @@ class ImportDefinition:
module_list = list(set(module_list))
module_list.sort()
specific_imports = {
hint_type.__module__: hint_type.__name__
for hint_type in hint_type_list
if getattr(hint_type, "__name__", None) is not None
}
code += "\n"
for module, name in specific_imports.items():
code += f"from {module} import {name}\n"
code += "\n"
for module in module_list:
code += f"import {module}\n"
@@ -637,6 +652,7 @@ class MethodDefinition:
func_name: str,
formatted_params: OrderedDict[str, Parameter],
return_type: type,
path: str,
model_name: Optional[str] = None,
) -> str:
"""Build the command method signature."""
@@ -651,7 +667,20 @@ class MethodDefinition:
if "pandas.DataFrame" in func_params
else ""
)
code = f"\n @validate{args}"
msg = ""
if MethodDefinition.is_deprecated_function(path):
deprecation_message = MethodDefinition.get_deprecation_message(path)
deprecation_type_class = type(
deprecation_message.metadata # type: ignore
).__name__
msg = "\n @deprecated("
msg += f'\n "{deprecation_message}",'
msg += f"\n category={deprecation_type_class},"
msg += "\n )"
code = f"\n @validate{args}{msg}"
code += f"\n def {func_name}("
code += f"\n self,\n {func_params}\n ) -> {func_returns}:\n"
@@ -698,7 +727,7 @@ class MethodDefinition:
if MethodDefinition.is_deprecated_function(path):
deprecation_message = MethodDefinition.get_deprecation_message(path)
code += " from warnings import warn, simplefilter; simplefilter('always', DeprecationWarning)\n"
code += " simplefilter('always', DeprecationWarning)\n"
code += f""" warn("{deprecation_message}", category=DeprecationWarning, stacklevel=2)\n\n"""
code += " return self._run(\n"
@@ -748,6 +777,7 @@ class MethodDefinition:
func_name=func_name,
formatted_params=formatted_params,
return_type=sig.return_annotation,
path=path,
model_name=model_name,
)
code += cls.build_command_method_doc(
@@ -957,6 +987,114 @@ class DocstringGenerator:
return doc
return doc
@staticmethod
def get_model_standard_params(param_fields: Dict[str, FieldInfo]) -> Dict[str, Any]:
"""Get the test params for the fetcher based on the required standard params."""
test_params: Dict[str, Any] = {}
for field_name, field in param_fields.items():
if field.default and field.default is not PydanticUndefined:
test_params[field_name] = field.default
elif field.default and field.default is PydanticUndefined:
example_dict = {
"symbol": "AAPL",
"symbols": "AAPL,MSFT",
"start_date": "2023-01-01",
"end_date": "2023-06-06",
"country": "Portugal",
"date": "2023-01-01",
"countries": ["portugal", "spain"],
}
if field_name in example_dict:
test_params[field_name] = example_dict[field_name]
elif field.annotation == str:
test_params[field_name] = "TEST_STRING"
elif field.annotation == int:
test_params[field_name] = 1
elif field.annotation == float:
test_params[field_name] = 1.0
elif field.annotation == bool:
test_params[field_name] = True
elif get_origin(field.annotation) is Literal: # type: ignore
option = field.annotation.__args__[0] # type: ignore
if isinstance(option, str):
test_params[field_name] = f'"{option}"'
else:
test_params[field_name] = option
return test_params
@staticmethod
def get_full_command_name(route: str) -> str:
"""Get the full command name."""
cmd_parts = route.split("/")
del cmd_parts[0]
menu = cmd_parts[0]
command = cmd_parts[-1]
sub_menus = cmd_parts[1:-1]
sub_menu_str_cmd = f".{'.'.join(sub_menus)}" if sub_menus else ""
full_command = f"{menu}{sub_menu_str_cmd}.{command}"
return full_command
@classmethod
def generate_example(
cls,
model_name: str,
standard_params: Dict[str, FieldInfo],
) -> str:
"""Generate the example for the command."""
# find the model router here
cm = CommandMap()
commands_model = cm.commands_model
route = [k for k, v in commands_model.items() if v == model_name]
if not route:
return ""
full_command_name = cls.get_full_command_name(route=route[0])
example_params = cls.get_model_standard_params(param_fields=standard_params)
# Edge cases (might find more)
if "crypto" in route[0] and "symbol" in example_params:
example_params["symbol"] = "BTCUSD"
elif "currency" in route[0] and "symbol" in example_params:
example_params["symbol"] = "EURUSD"
elif (
"index" in route[0]
and "european" not in route[0]
and "symbol" in example_params
):
example_params["symbol"] = "SPX"
elif (
"index" in route[0]
and "european" in route[0]
and "symbol" in example_params
):
example_params["symbol"] = "BUKBUS"
elif (
"futures" in route[0] and "curve" in route[0] and "symbol" in example_params
):
example_params["symbol"] = "VX"
elif "futures" in route[0] and "symbol" in example_params:
example_params["symbol"] = "ES"
example = "\n Example\n -------\n"
example += " >>> from openbb import obb\n"
example += f" >>> obb.{full_command_name}("
for param_name, param_value in example_params.items():
if isinstance(param_value, str):
param_value = f'"{param_value}"' # noqa: PLW2901
example += f"{param_name}={param_value}, "
if example_params:
example = example[:-2] + ")\n"
else:
example += ")\n"
return example
class PathHandler:
"""Handle the paths for the Platform."""

25
openbb_platform/core/tests/app/static/test_package_builder.py
View File

@@ -261,8 +261,10 @@ def test_build_func_returns(method_definition, return_type, expected_output):
assert output == expected_output
def test_build_command_method_signature(method_definition):
@patch("openbb_core.app.static.package_builder.MethodDefinition")
def test_build_command_method_signature(mock_method_definitions, method_definition):
"""Test build command method signature."""
mock_method_definitions.is_deprecated_function.return_value = False
formatted_params = {
"param1": Parameter("NoneType", kind=Parameter.POSITIONAL_OR_KEYWORD),
"param2": Parameter("int", kind=Parameter.POSITIONAL_OR_KEYWORD),
@@ -272,10 +274,31 @@ def test_build_command_method_signature(method_definition):
func_name="test_func",
formatted_params=formatted_params,
return_type=return_type,
path="test_path",
)
assert output
@patch("openbb_core.app.static.package_builder.MethodDefinition")
def test_build_command_method_signature_deprecated(
mock_method_definitions, method_definition
):
"""Test build command method signature."""
mock_method_definitions.is_deprecated_function.return_value = True
formatted_params = {
"param1": Parameter("NoneType", kind=Parameter.POSITIONAL_OR_KEYWORD),
"param2": Parameter("int", kind=Parameter.POSITIONAL_OR_KEYWORD),
}
return_type = int
output = method_definition.build_command_method_signature(
func_name="test_func",
formatted_params=formatted_params,
return_type=return_type,
path="test_path",
)
assert "@deprecated" in output
def test_build_command_method_doc(method_definition):
"""Test build command method doc."""

28
openbb_platform/core/tests/app/test_deprecation.py Normal file
View File

@@ -0,0 +1,28 @@
import unittest
from openbb_core.app.static.package_builder import PathHandler
from openbb_core.app.version import VERSION
def get_major_minor(version):
parts = version.split(".")
return (int(parts[0]), int(parts[1]))
class DeprecatedCommandsTest(unittest.TestCase):
"""Test deprecated commands."""
def test_deprecated_commands(self):
"""Test deprecated commands."""
current_major_minor = get_major_minor(VERSION)
route_map = PathHandler.build_route_map()
for path, route in route_map.items():
with self.subTest(i=path):
if getattr(route, "deprecated", False):
deprecation_message = getattr(route, "summary", "")
obb_deprecation_warning = deprecation_message.metadata
assert (
obb_deprecation_warning.expected_removal != current_major_minor
), f"The expected removal version of `{path}` matches the current version, please remove it."

7
openbb_platform/extensions/index/openbb_index/index_router.py
View File

@@ -1,5 +1,6 @@
"""Index Router."""
from openbb_core.app.deprecation import OpenBBDeprecationWarning
from openbb_core.app.model.command_context import CommandContext
from openbb_core.app.model.obbject import OBBject
from openbb_core.app.provider_interface import (
@@ -22,7 +23,11 @@ router.include_router(price_router)
@router.command(
model="MarketIndices",
deprecated=True,
deprecation_message="This endpoint will be deprecated in the future releases. Use '/index/price/historical' instead.",
deprecation=OpenBBDeprecationWarning(
message="This endpoint is deprecated; use `/index/price/historical` instead.",
since=(4, 1),
expected_removal=(4, 3),
),
)
async def market(
cc: CommandContext,

2
openbb_platform/openbb/package/extension_map.json vendored
View File

@@ -24,4 +24,4 @@
"tradingeconomics@1.1.1",
"yfinance@1.1.1"
]
}
}

12
openbb_platform/openbb/package/index.py
View File

@@ -2,13 +2,15 @@
import datetime
from typing import List, Literal, Optional, Union
from warnings import simplefilter, warn
from openbb_core.app.deprecation import OpenBBDeprecatedSince41
from openbb_core.app.model.custom_parameter import OpenBBCustomParameter
from openbb_core.app.model.obbject import OBBject
from openbb_core.app.static.container import Container
from openbb_core.app.static.utils.decorators import validate
from openbb_core.app.static.utils.filters import filter_inputs
from typing_extensions import Annotated
from typing_extensions import Annotated, deprecated
class ROUTER_index(Container):
@@ -155,6 +157,10 @@ class ROUTER_index(Container):
)
@validate
@deprecated(
"This endpoint is deprecated; use `/index/price/historical` instead. Deprecated in OpenBB Platform V4.1 to be removed in V4.5.",
category=OpenBBDeprecatedSince41,
)
def market(
self,
symbol: Annotated[
@@ -266,11 +272,9 @@ class ROUTER_index(Container):
>>> obb.index.market(symbol="SPX")
""" # noqa: E501
from warnings import simplefilter, warn
simplefilter("always", DeprecationWarning)
warn(
"This endpoint will be deprecated in the future releases. Use '/index/price/historical' instead.",
"This endpoint is deprecated since v4.1 and will be removed in v4.3; Use `/index/price/historical` instead.",
category=DeprecationWarning,
stacklevel=2,
)

2
openbb_platform/openbb/package/module_map.json vendored
View File

@@ -140,4 +140,4 @@
"regulators_sec_schema_files": "/regulators/sec/schema_files",
"regulators_sec_sic_search": "/regulators/sec/sic_search",
"regulators_sec_symbol_map": "/regulators/sec/symbol_map"
}
}

57
website/content/platform/development/contributor-guidelines/deprecating_endpoints.md vendored Normal file
View File

@@ -0,0 +1,57 @@
---
title: Deprecating Endpoints
sidebar_position: 5
description: This guide provides detailed instructions on how to deprecate an endpoint in the OpenBB Platform.
keywords:
- OpenBB community
- OpenBB Platform
- Custom commands
- API
- Python Interface
- Deprecation
- Deprecated
---
import HeadTitle from '@site/src/components/General/HeadTitle.tsx';
<HeadTitle title="Deprecating Endpoints - Contributor Guidelines - Development | OpenBB Platform Docs" />
Deprecating commands is essential to maintaining the OpenBB Platform. This guide outlines the process for deprecating an endpoint.
## Deprecating an endpoint
1. Add the new endpoint that will replace the deprecated one.
2. Add the deprecation warning
Navigate to the **router** where the endpoint to be deprecated exists. Set the `deprecated` flag to `True` and add `deprecation=OpenBBDeprecationWarning(…)` argument to the decorator. Refer to the example below:
```python
...
from openbb_core.app.deprecation import OpenBBDeprecationWarning
...
@router.command(
model="MarketIndices",
deprecated=True,
deprecation=OpenBBDeprecationWarning(
message="This endpoint is deprecated; use `/index/price/historical` instead.",
since=(4, 1),
expected_removal=(4, 5),
),
)
async def market(
cc: CommandContext,
provider_choices: ProviderChoices,
standard_params: StandardParams,
extra_params: ExtraParams,
) -> OBBject[BaseModel]:
"""Historical Market Indices."""
return await OBBject.from_query(Query(**locals()))
```
3. Get approval from a OpenBB Platform maintainer: We will help you determine the appropriate version for the deprecation warning, and communicate the change(s) to the relevant personnel that might depend on the endpoint you are deprecating.
4. Remove as we say - the endpoint will be removed in the version specified in the deprecation warning.

2
website/content/platform/development/contributor-guidelines/write_code_commit.md vendored
View File

@@ -1,6 +1,6 @@
---
title: Write Code and Commit
sidebar_position: 5
sidebar_position: 6
description: This guide provides detailed instructions on how to write code and commit changes for the OpenBB Platform. It covers the process of creating a PR, branch naming conventions, and important guidelines to follow when committing changes.
keywords:
- OpenBB code writing

14
website/generate_platform_v4_markdown.py
View File

@@ -327,6 +327,8 @@ def get_command_meta(path: str, route_map: Dict[str, Any]) -> Dict[str, Any]:
"returns": {},
"schema": {},
"model": model_name,
"deprecated": route.deprecated or False,
"deprecation_message": route.summary if route.deprecated else "",
}
# Extract the full path from the 'path' variable, excluding the method name
@@ -346,7 +348,8 @@ def get_command_meta(path: str, route_map: Dict[str, Any]) -> Dict[str, Any]:
]
meta_command["description"] += "\n\n" + DocstringGenerator.generate_example(
model_name, obb_query_fields
model_name=model_name,
standard_params=obb_query_fields,
)
available_fields = list(obb_query_fields.keys())
@@ -435,8 +438,8 @@ def get_command_meta(path: str, route_map: Dict[str, Any]) -> Dict[str, Any]:
chart : Optional[Chart]
Chart object.
metadata: Optional[Metadata]
Metadata info about the command execution."""
extra: Dict[str, Any]
Extra info."""
meta_command["returns"] = {
"type": return_type.__name__,
@@ -513,6 +516,11 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx';
<HeadTitle title="{'/'.join(title)} - Reference | OpenBB Platform Docs" />\n\n"""
if meta_command["deprecated"]:
meta_command[
"header"
] += f":::caution Deprecated\n{meta_command['deprecation_message']}\n:::\n\n"
return meta_command