Add logging w decorators (#1173)

* Example of adding logging

* make log possible with decorators

* remove lineno since we'll have logging at start and end

* rename log_decorator to log_execution

* add lineno and let it be 0 for log_start_end

* Decrease logging level (#1150)

* reset different coin from dd controller on crypto (#1118)

* Fix to dcf command (#1115)

* Fix to dcf command

* Forgot to take out a comment

* Dider change

* user data patch for now

* revert DD report with pickle dataframe being stored (#1120)

* Discordbot/expiry reactions (#1114)

* added load extension for options menu

* options menu: oi, iv, unusual options

* Update opt_menu.py

* Update unu.py

if no num requested set default to 10

* Update opt_menu.py

+ !oi ticker expiry request added without inputting date format. accepts 0-9 up to 10 weeks
+ added option volume plot (works same as oi)

* Create vol.py

* Create reaction_helper.py

* Cleaned up, added historical options

Cleaned up
+ added historical options data
+ added expiry reaction helper function

* Create hist.py

* cleanup

* added min/max strike price

* cleanup

* cleanup, added min/max strike price

* Update opt_menu.py

* added yfinance if no tradier token

* Update vol.py

* Update opt_menu.py

* Create oi_tradier.py

* Update oi.py

* Update opt_menu.py

* Update oi.py

* Update opt_menu.py

* Update oi.py

* Update iv.py

* Update unu.py

* Update oi_tradier.py

* Update hist.py

* Update opt_menu.py

* Update reaction_helper.py

* Create options_menu.py

* Update run_discordbot.py

* Update vol.py

* Rename discordbot/options_menu.py to discordbot/stocks/options/options_menu.py

* Delete opt_menu.py

* Update oi.py

+ min strike/max strike

* Delete oi_tradier.py

* Update options_menu.py

* Update hist.py

* Update vol.py

* Update oi.py

* Update options_menu.py

* Update options_menu.py

* Update options_menu.py

* Linting/Pre-Commit Fixes

* linting

* Linting

* linting

* pylint fixes

* Update iv.py

* Update options_menu.py

* Update reaction_helper.py

* Improved Code

- removed 148 lines same function

* Update run_discordbot.py

* Update reaction_helper.py

* Updated Options Menu

+ reaction expiry lookup
+ added unu, iv, oi, vol ,hist

* mypy fix

* msg delete delay

* Update options_menu.py

* fix

seems bot doesn't like when last emoji is empty.
added an X when strike c/p empty

* Update opt_cmd.py

* Update options_menu.py

* better handling

* Update options_menu.py

* Update opt_cmd.py

* fix stocks.opt indention

* hist.py token/error handling

* Update hist.py

* Update options_menu.py

* Update hist.py

* Add whatif scenario feature (#1122)

* Add whatif scenario feature

* fix tests

* Mutual funds (#1106)

* Add mutual fund menu based primarily on investpy (investment.com)

* Add start/end date for loading + some bug fixes

* hugo

* poetry add investpy rich dnspython  then poetry update

* fix reset (need to load country) and dim yfinance commands when country not US

* Fix search and add better error handling

* Catch yfinance commands when country not US.  Improve loading to change country if symbol found but not in matching country.  Changed main menu printing.

* Fix poetry / reqs merge conflict

* review comments

* Once again update poetry

* Clean up poetry to match conda (numpy was culprit)

* cvxpy down

* Think i goofed on generating reqs full

* update poetry metadata

* cleanup after review 2

* move space

* another deps try

* bump pygmnets in yaml

* try something different

* rename console -> t_console (terminal_console) for mypy

* Handle KeyboardInterrupt on user input read (#1124)

Co-authored-by: David Khouya <david.khouya@leddartech.com>

* Improve hugo docs (#1113)

* added gitignore to website, fix portfolio toggle and website path

* updating styles and homepage

* added index page, removed dark mode, updated styling

* fixed colors

* updated urls (removed /GamestonkTerminal after url)

* added new hr feature (#1128)

* fix assets on hugo server and revert url (#1129)

* fix darkmode issues (#1130)

* Tests : Stocks > Options (#1125)

* Update tests : conftest

* Updating tests : stocks/options

* Updating tests : fix typing

* Updating tests : black

* Updating tests : pyupgrade

* Updating tests : black

* Updating tests : mock dates in cassettes

* Updating tests : conftest

* Updating tests : black

* Updating tests : force single threading

* Updating tests : skip

* Updating tests : black

* Updating tests : conftest

* Update tests : skip stocks/options/controller

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : fixing issue

* Updating tests : add init

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : conftest

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Updating tests : skip

* Tests : Stocks > Research + Screener (#1131)

* Updating tests : stocks/research

* Updating tests : stocks/screener

* Updating tests : stocks/screener

* improve usage of timezone in terminal (#1126)

* improve usage of timezone in terminal

* lint

* update dependencies

* address James review comments

* remove seconds from time on cmd line

* skip test

* Add insider presets (#1127)

* Add prediction to economy/fred (#1133)

* Fixing tradier (#1135)

* improve additional info of stocks (#1137)

* improve additional info of stocks

* unnecessary code

* Qa update (#1139)

* Add line option to common - qa

* rich to controller + lower case for pick

* Change qa_view to rich + hugo

* Update _index.md

* fix test

* fix test pt 2

* change menu print

* Refactored Reports (#1123)

* First working program

* added Voila support

* Attempt to fix requirements

* Revert depen

* Another dependency test

* Fixed errors

* Small changes

* Fixed help

* Added port warning

* Fixed continued terminal use

* Last change, we good fam

* Added placeholder so we can have stored

* Didier changes

* Fixed exit

* Fixed issue

* Update stocks : ta > tradingview (#1144)

* Tests : Stocks > Option + Technical Analysis + Screener (#1145)

* Updating tests : stocks/ta

* Updating tests : conftest

* Updating tests : stocks/screener/finviz

* Updating tests : stocks/options

* interactive line chart and annotation (#1140)

* interactive line chart and annotation

* add drawing flag

* New alternative data context + first sub context (Covid) (#1142)

* Crypto menu refactor (#1119)

* enabled some crypto commands in dd to be called independent of source loaded

* support for coin_map_df in all dd functions + load ta and plot chart refactor

* updated tests and removed coingecko scrapping where possible

* removed ref of command from hugo

* updated pycoingecko version

* refactoring load

* refactored load to fetch prices; pred can run independent of source now

* load by default usd on cp/cg and usdt on cb/bin

* updated to rich for formatting and updated dependencies

* fixed changes requested

* update docs

* revert discord requirements

* removed absolute from calculate change for price

* fixing pr issues

* fix loading issue when similar coins exist, move coins to home, fill n/a

* update docs for coins

* adds load to ta and pred menu

* Fix library loggers always setting DEBUG verbosity

* Set default log level to INFO

Co-authored-by: Didier Rodrigues Lopes <dro.lopes@campus.fct.unl.pt>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>
Co-authored-by: jmaslek <jmaslek11@gmail.com>
Co-authored-by: David Khouya <dakhouya@gmail.com>
Co-authored-by: David Khouya <david.khouya@leddartech.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Chavithra <chavithra@gmail.com>

* Fix linting errors

* Move log decorator to the decorators.py file

* Revert test assertion so that windows tests don't hang up

* Fix pylint exception

* Bump pylint version in CI

* Skip test that hangs on windows

Co-authored-by: Artem Veremey <artem@veremey.net>
Co-authored-by: Theodore Aptekarev <aptekarev@gmail.com>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
Co-authored-by: teh_coderer <me@tehcoderer.com>
Co-authored-by: jmaslek <jmaslek11@gmail.com>
Co-authored-by: David Khouya <dakhouya@gmail.com>
Co-authored-by: David Khouya <david.khouya@leddartech.com>
Co-authored-by: jose-donato <43375532+jose-donato@users.noreply.github.com>
Co-authored-by: Chavithra <chavithra@gmail.com>
This commit is contained in:
didierlopes.eth
2022-01-11 16:01:46 +00:00
committed by GitHub
parent 9d36f84ac6
commit f1ecd1fe90
11 changed files with 191 additions and 28 deletions

View File

@@ -26,7 +26,7 @@ jobs:
key: ${{ runner.os }}-linting
restore-keys: ${{ runner.os }}-linting
- run: pip install bandit black codespell flake8 mypy pyupgrade safety pylint==2.11.1
- run: pip install bandit black codespell flake8 mypy pyupgrade safety pylint==2.12.2
- run: pip install types-pytz types-requests types-termcolor types-tabulate types-PyYAML types-python-dateutil
- run: bandit -x ./tests -r . || true
- run: black --check .
@@ -249,7 +249,7 @@ jobs:
- name: Run linter (Bash)
shell: bash -l {0}
run: |
pip install pylint==2.11.1
pip install pylint==2.12.2
pylint discordbot gamestonk_terminal terminal.py tests
- name: Run MyPy (Bash)

View File

@@ -18,7 +18,7 @@ PAPERMILL_NOTEBOOK_REPORT_PORT = "8888"
# 1 - DEBUG for terminal, INFO for libraries
# 2 - DEBUG for terminal, DEBUG for libraries
LOGGING_VERBOSITY = 2
LOGGING_VERBOSITY = 0
if tmp_verbosity := os.getenv("GT_LOGGING_VERBOSITY"):
print(f"Setting verbosity to {tmp_verbosity}")

View File

@@ -28,3 +28,62 @@ def try_except(f):
return []
return inner
def log_start_end(func=None, log=None):
"""Wrap function to add a log entry at execution start and end.
Parameters
----------
func : optional
Function, by default None
log : optional
Logger, by default None
Returns
-------
Wrapped function
"""
assert callable(func) or func is None # nosec
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
args_passed_in_function = [repr(a) for a in args]
# view files have parameters that are usually small given they are input by the user
if log.name[-5:] == "_view":
kwargs_passed_in_function = [f"{k}={v!r}" for k, v in kwargs.items()]
# other files can have as parameters big variables, therefore adds logic to only add small ones
else:
kwargs_passed_in_function = list()
for k, v in kwargs.items():
if type(v) in (int, float):
kwargs_passed_in_function.append(f"{k}={v!r}")
elif k == "export":
kwargs_passed_in_function.append(f"{k}={v!r}")
else:
kwargs_passed_in_function.append(f"{k}={type(v)!r}")
if args_passed_in_function or kwargs_passed_in_function:
formatted_arguments = ", ".join(
args_passed_in_function + kwargs_passed_in_function
)
log.info(
f"START params: {formatted_arguments}",
extra={"func_name_override": func.__name__},
)
else:
log.info("START", extra={"func_name_override": func.__name__})
try:
value = func(*args, **kwargs)
log.info("END", extra={"func_name_override": func.__name__})
return value
except Exception:
log.exception("Exception", extra={"func_name_override": func.__name__})
return None
return wrapper
return decorator(func) if callable(func) else decorator

View File

@@ -1,5 +1,6 @@
"""Helper functions"""
__docformat__ = "numpy"
# pylint: disable=too-many-lines
import argparse
import logging
from typing import List
@@ -426,7 +427,7 @@ def divide_chunks(data, n):
"""Split into chunks"""
# looping till length of data
for i in range(0, len(data), n):
yield data[i : i + n]
yield data[i : i + n] # noqa: E203
def get_next_stock_market_days(last_stock_day, n_next_days) -> list:
@@ -857,7 +858,7 @@ def get_last_time_market_was_open(dt):
def export_data(
export_type: str, dir_path: str, func_name: str, df: pd.DataFrame = pd.DataFrame()
):
) -> None:
"""Export data to a file.
Parameters
@@ -930,23 +931,13 @@ def get_rf() -> float:
class LineAnnotateDrawer:
"""Line drawing class."""
def __init__(self, ax: matplotlib.axes = None):
self.ax = ax
def draw_lines_and_annotate(self):
# ymin, _ = self.ax.get_ylim()
# xmin, _ = self.ax.get_xlim()
# self.ax.plot(
# [xmin, xmin],
# [ymin, ymin],
# lw=0,
# color="white",
# label="X - leave interactive mode\nClick twice for annotation",
# )
# self.ax.legend(handlelength=0, handletextpad=0, fancybox=True, loc=2)
# self.ax.figure.canvas.draw()
"""Draw lines."""
print("Click twice for annotation.\nClose window to keep using terminal.\n")
while True:

View File

@@ -38,7 +38,6 @@ def library_loggers(verbosity: int = 0) -> None:
def setup_file_logger() -> None:
"""Setup File Logger"""
file_path = Path(__file__)
logger.debug("Parent dir: %s", file_path.parent.parent.absolute())
log_dir = file_path.parent.parent.absolute().joinpath("logs")
@@ -114,6 +113,9 @@ class CustomFormatterWithExceptions(logging.Formatter):
str
Formatted log message
"""
if hasattr(record, "func_name_override"):
record.funcName = record.func_name_override # type: ignore
record.lineno = 0
s = super().format(record)
if record.levelname:
prefix = record.levelname[0]
@@ -176,7 +178,7 @@ def setup_logging() -> None:
else:
logger.debug("Unknown loghandler")
library_loggers(verbosity)
library_loggers(int(cfg.LOGGING_VERBOSITY))
logger.info("Logging configuration finished")
logger.info("Logging set to %s", cfg.LOGGING_HANDLERS)

View File

@@ -2,12 +2,17 @@
__docformat__ = "numpy"
import os
import logging
from tabulate import tabulate
from gamestonk_terminal.decorators import log_start_end
from gamestonk_terminal.helper_funcs import export_data
from gamestonk_terminal.stocks.discovery import yahoofinance_model
logger = logging.getLogger(__name__)
def display_gainers(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_gainers(num_stocks: int, export: str) -> None:
"""Display gainers. [Source: Yahoo Finance]
Parameters
@@ -17,6 +22,7 @@ def display_gainers(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df_gainers = yahoofinance_model.get_gainers()
df_gainers.dropna(how="all", axis=1, inplace=True)
df_gainers = df_gainers.replace(float("NaN"), "")
@@ -43,7 +49,8 @@ def display_gainers(num_stocks: int, export: str):
)
def display_losers(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_losers(num_stocks: int, export: str) -> None:
"""Display losers. [Source: Yahoo Finance]
Parameters
@@ -53,6 +60,7 @@ def display_losers(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df_losers = yahoofinance_model.get_losers()
df_losers.dropna(how="all", axis=1, inplace=True)
df_losers = df_losers.replace(float("NaN"), "")
@@ -79,7 +87,8 @@ def display_losers(num_stocks: int, export: str):
)
def display_ugs(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_ugs(num_stocks: int, export: str) -> None:
"""Display most undervalued growth stock. [Source: Yahoo Finance]
Parameters
@@ -89,6 +98,7 @@ def display_ugs(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df = yahoofinance_model.get_ugs()
df.dropna(how="all", axis=1, inplace=True)
df = df.replace(float("NaN"), "")
@@ -115,7 +125,8 @@ def display_ugs(num_stocks: int, export: str):
)
def display_gtech(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_gtech(num_stocks: int, export: str) -> None:
"""Display growth technology stocks. [Source: Yahoo Finance]
Parameters
@@ -125,6 +136,7 @@ def display_gtech(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df = yahoofinance_model.get_gtech()
df.dropna(how="all", axis=1, inplace=True)
df = df.replace(float("NaN"), "")
@@ -151,7 +163,8 @@ def display_gtech(num_stocks: int, export: str):
)
def display_active(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_active(num_stocks: int, export: str) -> None:
"""Display most active stocks. [Source: Yahoo Finance]
Parameters
@@ -161,6 +174,7 @@ def display_active(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df = yahoofinance_model.get_active()
df.dropna(how="all", axis=1, inplace=True)
df = df.replace(float("NaN"), "")
@@ -187,7 +201,8 @@ def display_active(num_stocks: int, export: str):
)
def display_ulc(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_ulc(num_stocks: int, export: str) -> None:
"""Display potentially undervalued large cap stocks. [Source: Yahoo Finance]
Parameters
@@ -197,6 +212,7 @@ def display_ulc(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df = yahoofinance_model.get_ulc()
df.dropna(how="all", axis=1, inplace=True)
df = df.replace(float("NaN"), "")
@@ -223,7 +239,8 @@ def display_ulc(num_stocks: int, export: str):
)
def display_asc(num_stocks: int, export: str):
@log_start_end(log=logger)
def display_asc(num_stocks: int, export: str) -> None:
"""Display small cap stocks with earnings growth rates better than 25%. [Source: Yahoo Finance]
Parameters
@@ -233,6 +250,7 @@ def display_asc(num_stocks: int, export: str):
export : str
Export dataframe data to csv,json,xlsx file
"""
df = yahoofinance_model.get_asc()
df.dropna(how="all", axis=1, inplace=True)
df = df.replace(float("NaN"), "")

View File

@@ -13,6 +13,7 @@ def vcr_config():
}
@pytest.mark.skip("Hangs on Windows")
@pytest.mark.vcr
@pytest.mark.record_stdout
def test_display_btc_rainbow():

View File

@@ -69,7 +69,7 @@ def test_screener_no_data(data, mocker):
export="",
)
assert result == []
assert result == [] # pylint: disable=use-implicit-booleaness-not-comparison
@pytest.mark.vcr

View File

@@ -0,0 +1,61 @@
interactions:
- request:
body: null
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate
Connection:
- keep-alive
User-Agent:
- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML,
like Gecko) Chrome/39.0.2171.95 Safari/537.36
method: GET
uri: https://query2.finance.yahoo.com/v8/finance/chart/TSLA?period1=1639695600&period2=1639782000&interval=1d&includePrePost=False&events=div%2Csplits
response:
body:
string: '{"chart":{"result":[{"meta":{"currency":"USD","symbol":"TSLA","exchangeName":"NMS","instrumentType":"EQUITY","firstTradeDate":1277818200,"regularMarketTime":1641330002,"gmtoffset":-18000,"timezone":"EST","exchangeTimezoneName":"America/New_York","regularMarketPrice":1149.59,"chartPreviousClose":926.92,"priceHint":2,"currentTradingPeriod":{"pre":{"timezone":"EST","start":1641373200,"end":1641393000,"gmtoffset":-18000},"regular":{"timezone":"EST","start":1641393000,"end":1641416400,"gmtoffset":-18000},"post":{"timezone":"EST","start":1641416400,"end":1641430800,"gmtoffset":-18000}},"dataGranularity":"1d","range":"","validRanges":["1d","5d","1mo","3mo","6mo","1y","2y","5y","10y","ytd","max"]},"timestamp":[1639665000,1639751400],"indicators":{"quote":[{"volume":[27590500,33479100],"open":[994.5,914.77001953125],"close":[926.9199829101562,932.5700073242188],"high":[994.97998046875,960.6599731445312],"low":[921.8499755859375,909.0399780273438]}],"adjclose":[{"adjclose":[926.9199829101562,932.5700073242188]}]}}],"error":null}}'
headers:
Age:
- '2'
Connection:
- keep-alive
Expect-CT:
- max-age=31536000, report-uri="http://csp.yahoo.com/beacon/csp?src=yahoocom-expect-ct-report-only"
Referrer-Policy:
- no-referrer-when-downgrade
Strict-Transport-Security:
- max-age=15552000
X-Content-Type-Options:
- nosniff
X-Frame-Options:
- SAMEORIGIN
X-XSS-Protection:
- 1; mode=block
cache-control:
- public, max-age=10, stale-while-revalidate=20
content-length:
- '1035'
content-type:
- application/json;charset=utf-8
date:
- Wed, 05 Jan 2022 11:45:24 GMT
server:
- ATS
vary:
- Origin,Accept-Encoding
x-envoy-decorator-operation:
- finance-chart-api--mtls-production-ir2.finance-k8s.svc.yahoo.local:4080/*
x-envoy-upstream-service-time:
- '3'
x-request-id:
- ccc14cc5-af7c-40ac-ac28-44d081fd5beb
x-yahoo-request-id:
- 0gsv07dgtb16k
y-rid:
- 0gsv07dgtb16k
status:
code: 200
message: OK
version: 1

View File

@@ -0,0 +1,20 @@
╒════════════╤══════════════════╤═══════╤════════╤═══════════╕
│ INTERVAL │ RECOMMENDATION │ BUY │ SELL │ NEUTRAL │
╞════════════╪══════════════════╪═══════╪════════╪═══════════╡
│ 1 month │ BUY │ 14 │ 5 │ 7 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 1 week │ BUY │ 14 │ 4 │ 8 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 1 day │ NEUTRAL │ 9 │ 7 │ 10 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 4 hours │ SELL │ 6 │ 11 │ 9 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 1 hour │ SELL │ 2 │ 14 │ 10 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 15 min │ STRONG_SELL │ 0 │ 16 │ 10 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 5 min │ STRONG_SELL │ 0 │ 16 │ 10 │
├────────────┼──────────────────┼───────┼────────┼───────────┤
│ 1 min │ STRONG_SELL │ 1 │ 16 │ 9 │
╘════════════╧══════════════════╧═══════╧════════╧═══════════╛

View File

@@ -1,6 +1,17 @@
import functools
import sys
import io
from typing import Union
def calc_change(current: Union[float, int], previous: Union[float, int]):
"""Calculates change between two different values"""
if current == previous:
return 0
try:
return ((current - previous) / previous) * 100.0
except ZeroDivisionError:
return float("inf")
def check_print(assert_in: str = "", length: int = -1):