Add Sentiment Investor API integration (#562)

* Add SentimentInvestor core and social metrics

* Added `metrics` and `social` commands to behaviour analysis menu
* `metrics` fetches the core metrics for a given ticker from
  SentimentInvestor: AHI, RHI, SGP and sentiment score.
* `social` fetches the aggregated data from several social media
  networks, i.e. the number of mentions and the sentiment score
  associated with a specific ticker on each network

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Format code to conform to PEP8 style guide

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Use `argparse` to handle command-line arguments

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Use `multiprocessing` to reduce wait time

* Reduce `metrics` from 7.2s to 2.9s
* Reduce `social` from 13.5s to 5.6s
* NB: These times could be reduced further but that would require
  changes to `sentipy` itself

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add `historical` command to plot past week of data

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add three new commands to discovery mode

* `popular` for stocks with the highest AHI
* `emerging` for stocks with the highest RHI
* `positive` for stocks with the highest sentiment scores

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Aggregate data for past week when showing table

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Fix `-t` flag not working for `social` command

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Handle lacking or incomplete historical data

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Improve error handling for unsupported stocks

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Remove `positive` command

* Sorting SentimentInvestor data by sentiment does not provide much
  value, seeing as it simply returns stocks that have very few comments,
  giving it a sentiment score of 1.0, which is inaccurate and misleading

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Reformat code per project guidelines

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Remove `positive` from list of valid commands

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* `poetry add sentiment-investor`

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add SentimentInvestor to "About us" section

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Use `typing.List` annotation instead of `builtins.list`

* This is to support Python versions older than 3.9

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add `sentiment-investor` to `requirements-full.txt`

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add SentimentInvestor API key check

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Show correct program name in help message

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add README documentation for newly added commands

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Make changes as requested by code review

* Coloring applied to the whole row in `metrics` and `social` commands
* Convert `--ticker` flag into an optional positional argument for
  single-argument commands such as `metrics` and `social`
* Make help menus and command descriptions far more detailed
* Explain what AHI, RHI, SGP and sentiment score are and how they are
  calculated by Sentiment.
* `historical` command is now sorted with the most recent date first
* `historical` command output table can be sorted using new `--sort` and
  `--direction` arguments
* Non-fatal warning and error messages have been suppressed

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Wrap command methods in try-except

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Add SentimentInvestor API key help

* Add to the top-level README
* Adjust environment variable name to conform with standard set by
  others, start it with GT_

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>

* Comment `coinpaprika` erroring line

* https://github.com/GamestonkTerminal/GamestonkTerminal/pull/562#issuecomment-887842888

Signed-off-by: Timothy Langer <zeevox.dev@gmail.com>
This commit is contained in:
Timothy Langer
2021-07-28 23:00:20 +01:00
committed by GitHub
parent a9986d3b90
commit 06957c27f3
14 changed files with 967 additions and 4 deletions

2
README.md vendored
View File

@@ -239,6 +239,7 @@ These are the ones where a key is necessary:
* Polygon: https://polygon.io
* Quandl: https://www.quandl.com/tools/api
* Reddit: https://www.reddit.com/prefs/apps
* SentimentInvestor: https://sentimentinvestor.com
* Tradier: https://developer.tradier.com/getting_started
* Twitter: https://developer.twitter.com
@@ -260,6 +261,7 @@ Alternatively, you can also set them to the following environment variables:
| [Polygon](https://polygon.io) | GT_API_POLYGON_KEY |
| [Quandl](https://www.quandl.com) | GT_API_KEY_QUANDL |
| [Reddit](https://www.reddit.com) | GT_API_REDDIT_CLIENT_ID <br> GT_API_REDDIT_CLIENT_SECRET <br/> GT_API_REDDIT_USERNAME <br/> GT_API_REDDIT_USER_AGENT <br/> GT_API_REDDIT_PASSWORD|
| [SentimentInvestor](https://sentimentinvestor.com) | GT_API_SENTIMENTINVESTOR_TOKEN <br> GT_API_SENTIMENTINVESTOR_KEY |
| [Tradier](https://developer.tradier.com) | GT_TRADIER_TOKEN |
| [Twitter](https://developer.twitter.com) | GT_API_TWITTER_KEY <br/> GT_API_TWITTER_SECRET_KEY <br/> GT_API_TWITTER_BEARER_TOKEN |

View File

@@ -45,6 +45,13 @@ This menu aims to extrapolate behavioural analysis regarding a pre-loaded ticker
* [rise](#rise)
- top rising related queries with stock
[SENTIMENT INVESTOR](#SENTIMENTINVESTOR)
* [metrics](#metrics)
- core social sentiment metrics for this stock
* [social](#social)
- social media figures for stock popularity
* [historical](#historical)
- plot the past week of data for a selected metric
## finbrain <a name="finbrain"></a>
```
@@ -251,3 +258,106 @@ Print top rising related queries with this stock's query. [Source: Google]
* -n : number of top rising related queries to print. Default 10.
<img width="934" alt="Captura de ecrã 2021-02-22, às 22 21 21" src="https://user-images.githubusercontent.com/25267873/108777814-4f21fa80-755c-11eb-96da-0327c9a0da57.png">
# SENTIMENTINVESTOR
> **[Sentiment Investor](https://sentimentinvestor.com)** analyzes data from four major social media platforms to
> generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
> sentiment metrics powered by proprietary NLP models.
## Metric definitions
### AHI (Absolute Hype Index) <a name="AHI"></a>
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
### RHI (Relative Hype Index) <a name="RHI"></a>
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
### Sentiment Score <a name="sentiment"></a>
Sentiment score is the percentage of people talking positively about the stock.
For each social network the number of positive posts/comments is divided by the
total number of both positive and negative posts/comments.
### SGP (Standard General Perception) <a name="SGP"></a>
SGP is a measure of whether people are more or less positive about a stock than
usual. It is calculated by averaging the past day of sentiment values and then
dividing it by the average of the past week of sentiment values.
## metrics <a name="metrics"></a>
```
usage: metrics [-h] [ticker]
```
The `metrics` command presents the four core metrics provided by Sentiment Investor,
including AHI, RHI, sentiment and SGP, as described above.
Optional arguments:
- [optional, positional] ticker
ticker to use instead of the loaded one
- `-h`, `--help`
show this help message
Example output:
![`metrics NFLX` command output](https://user-images.githubusercontent.com/8385172/127154566-1af3c274-c521-426b-8580-2ae714c2c1ca.png)
## social <a name="social"></a>
```
usage: social [-h] [ticker]
```
The `social` command prints the raw data for a given stock, including the number
of mentions it has received on social media in the last hour and the sentiment
score of those comments.
Optional arguments:
- [optional, positional] ticker
ticker to use instead of the loaded one
- `-h`, `--help`
show this help message
Example output:
![`social AAPL` command output](https://user-images.githubusercontent.com/8385172/127154775-13b1a81d-f5d2-4768-bf7e-bca8886f6d24.png)
## historical <a name="historical"></a>
```
usage: historical [-t TICKER] [-s [{date,value}]] [-d [{asc,desc}]] [-h] [{sentiment,AHI,RHI,SGP}]
```
The `historical` command plots the past week of data for a selected
SentimentInvestor core metric, one of:
[AHI](#ahi), [RHI](#ahi), [sentiment](#sentiment) or [SGP](#SGP).
It also outputs a sorted table of the mean value for that metric for each day.
Positional arguments:
- one of `sentiment`, `AHI`, `RHI` or `SGP`
(optional) the metric to plot (default: `sentiment`)
Optional arguments:
- `-t`, `--ticker TICKER`
ticker for which to fetch data
- `-s`, `--sort [{date,value}]`
the parameter to sort output table by
- `-d`, `--direction [{asc,desc}]`
the direction to sort the output table
- `-h`, `--help`
show this help message
Example output:
![`historical -t GME RHI` plot output](https://user-images.githubusercontent.com/8385172/126782509-bfc21dbb-2fe6-4cd4-a384-018eb41ac8f3.png)
![`historical -t NFLX -s value -d asc RHI` command output](https://user-images.githubusercontent.com/8385172/127157236-7818a635-4394-4372-9673-2834b32811af.png)

View File

@@ -16,6 +16,7 @@ from gamestonk_terminal.behavioural_analysis import (
stocktwits_view,
finbrain_view,
finnhub_view,
sentimentinvestor_view,
)
@@ -46,6 +47,9 @@ class BehaviouralAnalysisController:
"rise",
"finbrain",
"stats",
"metrics",
"social",
"historical",
]
def __init__(self, ticker: str, start: datetime):
@@ -102,6 +106,11 @@ class BehaviouralAnalysisController:
print(" queries top related queries with this stock")
print(" rise top rising related queries with stock")
print("")
print("SentimentInvestor:")
print(" metrics core social sentiment metrics for this stock")
print(" social social media figures for stock popularity")
print(" historical plot the past week of data for a selected metric")
print("")
def switch(self, an_input: str):
"""Process and dispatch input
@@ -246,6 +255,18 @@ class BehaviouralAnalysisController:
"""Process stats command"""
finnhub_view.sentiment_stats(other_args, self.ticker)
def call_metrics(self, other_args: List[str]):
"""Process metrics command"""
sentimentinvestor_view.metrics(self.ticker, other_args)
def call_social(self, other_args: List[str]):
"""Process social command"""
sentimentinvestor_view.socials(self.ticker, other_args)
def call_historical(self, other_args: List[str]):
"""Process historical command"""
sentimentinvestor_view.historical(self.ticker, other_args)
def menu(ticker: str, start: datetime, context: str = ""):
"""Behavioural Analysis Menu"""

View File

@@ -0,0 +1,611 @@
import argparse
import dataclasses
import datetime
import logging
import multiprocessing
import os
import statistics
import textwrap
import time
from typing import Union, Optional, List, Tuple
import matplotlib.dates as mdates
import pandas as pd
import seaborn as sns
from colorama import Fore, Style
from matplotlib import pyplot as plt
from sentipy.sentipy import Sentipy
import tabulate
from gamestonk_terminal import config_terminal as cfg
from gamestonk_terminal.config_plot import PLOT_DPI
from gamestonk_terminal.helper_funcs import (
parse_known_args_and_warn,
plot_autoscale,
)
sentipy: Sentipy = Sentipy(
token=cfg.API_SENTIMENTINVESTOR_TOKEN, key=cfg.API_SENTIMENTINVESTOR_KEY
)
"""Initialise SentiPy with the user's API token and key"""
pd.plotting.register_matplotlib_converters()
# suppress warning messages for a clean interface
logging.getLogger().setLevel(logging.CRITICAL)
@dataclasses.dataclass
class _Boundary:
"""Represents a strong or weak bounding inequality for categorising a value"""
min: Union[float, int]
"Minimum value that this metric could take"
max: Union[float, int]
"Maximum value that this metric could take"
strong: bool = False
"Whether this a strongly bounded inequality"
def categorise(self, num: Union[float, int]) -> Tuple[str, str]:
"""
Categorise a given number using this bounding inequality
Parameters
----------
num: the number to bucket
Returns
-------
A color string and a string with low / medium / high as appropriate
"""
if num is None or self.min is None or self.max is None:
return Style.DIM + Fore.WHITE, "N/A"
boundaries = [self.min + (self.max - self.min) / 5 * i for i in range(1, 5)]
if (num <= self.min or num >= self.max) and self.strong:
return Fore.WHITE, "Extreme (anomaly?)"
if num < boundaries[0]:
return Style.BRIGHT + Fore.RED, "Much Lower"
if num < boundaries[1]:
return Fore.RED, "Lower"
if num < boundaries[2]:
return Fore.YELLOW, "Same"
if num < boundaries[3]:
return Fore.GREEN, "Higher"
return Style.BRIGHT + Fore.GREEN, "Much Higher"
def _get_past_week_average(ticker: str, metric: str) -> float:
historical_data = [
dp
for dp in sentipy.historical(
ticker, metric, int(time.time() - 60 * 60 * 24 * 7), int(time.time())
).values()
if dp is not None
]
return statistics.mean(historical_data) if historical_data else None
@dataclasses.dataclass
class _MetricInfo:
name: str
"""Metric attribute name"""
title: str
"""Human-friendly attribute name"""
description: str
"""Metric attribute description"""
format: str
"""Format string for presenting the value"""
class _Metric(_MetricInfo):
boundary: _Boundary
"""A contextual range to for this metric"""
ticker: str
"""Ticker symbol for which this metric relates"""
value: Union[float, int]
"""Value"""
def __init__(
self,
metric_info: _MetricInfo,
ticker: str,
value: Union[float, int],
boundary: Optional[_Boundary] = None,
):
super().__init__(**dataclasses.asdict(metric_info))
weekly_mean = _get_past_week_average(ticker, self.name)
self.boundary = (
_Boundary(0, None if weekly_mean is None else 2 * weekly_mean)
if boundary is None
else boundary
)
self.ticker = ticker
self.value = value
def visualise(self) -> Tuple[str, str, str, str]:
color, category = self.boundary.categorise(self.value)
return (
color + self.title,
category,
"N/A" if self.value is None else f"{self.value:{self.format}}",
self.description + Style.RESET_ALL,
)
core_metrics = [
_MetricInfo(
"sentiment",
"Sentiment Score",
"the percentage of people that are talking positively about a stock",
".0%",
),
_MetricInfo(
"AHI",
"Average Hype Index",
"how much people are talking about a stock on social media",
".3f",
),
_MetricInfo(
"RHI",
"Relative Hype Index",
"whether people are talking about a stock more or less than usual",
".3f",
),
_MetricInfo(
"SGP",
"Standard General Perception",
"whether people are more or less positive about a stock than usual",
".3f",
),
]
social_metrics = [
_MetricInfo(
"reddit_comment_mentions",
"Reddit Comment Mentions",
"the number of mentions this stock has received in Reddit comments",
"n",
),
_MetricInfo(
"reddit_comment_sentiment",
"Reddit Comment Sentiment",
"percentage of Reddit comments mentioning this stock in a positive light",
".0%",
),
_MetricInfo(
"reddit_post_mentions",
"Reddit Post Mentions",
"the number of mentions this stock has received in Reddit posts",
"n",
),
_MetricInfo(
"reddit_post_sentiment",
"Reddit Post Sentiment",
"percentage of Reddit posts mentioning this stock in a positive light",
".0%",
),
_MetricInfo(
"tweet_mentions",
"Twitter Mentions",
"the number of mentions this stock has received on Twitter",
"n",
),
_MetricInfo(
"tweet_sentiment",
"Twitter Sentiment",
"percentage of tweets mentioning this stock in a positive light",
".0%",
),
_MetricInfo(
"stocktwits_post_mentions",
"Stocktwits Mentions",
"the number of mentions this stock has received in Stocktwits posts",
"n",
),
_MetricInfo(
"stocktwits_post_sentiment",
"Stocktwits Sentiment",
"percentage of Stocktwits posts mentioning this stock in a positive light",
".0%",
),
_MetricInfo(
"yahoo_finance_comment_mentions",
"Yahoo! Finance Comment Mentions",
"the number of mentions this stock has received in Yahoo! Finance comments",
"n",
),
_MetricInfo(
"yahoo_finance_comment_sentiment",
"Yahoo! Finance Comment Sentiment",
"percentage of Yahoo! Finance comments mentioning this stock in a positive light",
".0%",
),
]
def _contextualise_metrics(
data: object, ticker: str, metric_infos: List[_MetricInfo]
) -> Optional[List[_Metric]]:
arguments = []
for metric_info in metric_infos:
if not hasattr(data, metric_info.name):
logging.error(
"data for %s stock is incomplete. If you believe this is an error, "
"please contact hello@sentimentinvestor.com",
ticker,
)
return None
arguments.append((metric_info, ticker, data.__getattribute__(metric_info.name)))
with multiprocessing.Pool(os.cpu_count()) as pool:
return pool.starmap(_Metric, arguments)
def _tabulate_metrics(ticker: str, metrics_list: List[_Metric]):
table_data = []
table_headers = [
f"{Style.BRIGHT}{ticker}{Style.RESET_ALL} Metrics",
"vs Past 7 Days",
"Value",
"Description",
]
for metric in metrics_list:
table_data.append(metric.visualise())
return tabulate.tabulate(table_data, table_headers, tablefmt="grid")
def _customise_plot() -> None:
sns.set(
font="Arial",
style="darkgrid",
rc={
"axes.axisbelow": False,
"axes.edgecolor": "lightgrey",
"axes.facecolor": "None",
"axes.grid": False,
"axes.labelcolor": "dimgrey",
"axes.spines.right": False,
"axes.spines.top": False,
"figure.facecolor": "white",
"lines.solid_capstyle": "round",
"patch.edgecolor": "w",
"patch.force_edgecolor": True,
"text.color": "dimgrey",
"xtick.bottom": False,
"xtick.color": "dimgrey",
"xtick.direction": "out",
"xtick.top": False,
"ytick.color": "dimgrey",
"ytick.direction": "out",
"ytick.left": False,
"ytick.right": False,
},
)
sns.despine(left=True, bottom=True)
sns.color_palette("pastel")
plt.legend(frameon=False)
def _parse_args_for_ticker(other_args: List[str], ticker: str, command: str, desc: str):
parser = argparse.ArgumentParser(
add_help=False,
prog=command,
formatter_class=argparse.RawDescriptionHelpFormatter,
description=desc,
)
parser.add_argument(
"ticker",
action="store",
type=str,
nargs="?",
default=ticker,
help="ticker to use instead of the loaded one",
)
return parse_known_args_and_warn(parser, other_args)
def metrics(ticker: str, other_args: List[str]) -> None:
command_description = f"""
{Style.BRIGHT}Sentiment Investor{Style.RESET_ALL} analyzes data from four major social media platforms to
generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
sentiment metrics powered by proprietary NLP models.
The {Style.BRIGHT}metrics{Style.RESET_ALL} command prints the following realtime metrics:
{Style.BRIGHT}AHI (Absolute Hype Index){Style.RESET_ALL}
---
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
{Style.BRIGHT}RHI (Relative Hype Index){Style.RESET_ALL}
---
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
{Style.BRIGHT}Sentiment Score{Style.RESET_ALL}
---
Sentiment score is the percentage of people talking positively about the stock.
For each social network the number of positive posts/comments is divided by the
total number of both positive and negative posts/comments.
{Style.BRIGHT}SGP (Standard General Perception){Style.RESET_ALL}
---
SGP is a measure of whether people are more or less positive about a stock than
usual. It is calculated by averaging the past day of sentiment values and then
dividing it by the average of the past week of sentiment values.
"""
ns_parser = _parse_args_for_ticker(
other_args=other_args,
ticker=ticker,
command="metrics",
desc=textwrap.dedent(command_description),
)
try:
if not ns_parser:
logging.error("There was an error in parsing the command-line arguments.")
return
if not sentipy.supported(ns_parser.ticker):
print("This stock is not supported by the SentimentInvestor API.")
return
data = sentipy.parsed(ns_parser.ticker)
metric_values = _contextualise_metrics(data, ns_parser.ticker, core_metrics)
if not metric_values:
logging.error("No data available or an error occurred.")
return
print(_tabulate_metrics(ns_parser.ticker, metric_values))
print()
except Exception as e:
logging.error(e)
print(e, "\n")
def socials(ticker: str, other_args: List[str]) -> None:
command_description = f"""
{Style.BRIGHT}Sentiment Investor{Style.RESET_ALL} analyzes data from four major social media platforms to
generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
sentiment metrics powered by proprietary NLP models.
The {Style.BRIGHT}social{Style.RESET_ALL} command prints the raw data for a given stock, including the number
of mentions it has received on social media in the last hour and the sentiment
score of those comments.
"""
try:
ns_parser = _parse_args_for_ticker(
other_args=other_args,
ticker=ticker,
command="social",
desc=textwrap.dedent(command_description),
)
if not ns_parser:
logging.error("There was an error in parsing the command-line arguments.")
return
if not sentipy.supported(ns_parser.ticker):
print("This stock is not supported by the SentimentInvestor API.")
return
data = sentipy.raw(ns_parser.ticker)
metric_values = _contextualise_metrics(data, ns_parser.ticker, social_metrics)
if not metric_values:
logging.error("No data available or an error occurred.")
return
print(_tabulate_metrics(ns_parser.ticker, metric_values))
print()
except Exception as e:
logging.error(e)
print(e, "\n")
def historical(ticker: str, other_args: List[str]) -> None:
command_description = f"""
{Style.BRIGHT}Sentiment Investor{Style.RESET_ALL} analyzes data from four major social media platforms to
generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
sentiment metrics powered by proprietary NLP models.
The {Style.BRIGHT}historical{Style.RESET_ALL} command plots the past week of data for a selected metric, one of:
{Style.BRIGHT}AHI (Absolute Hype Index){Style.RESET_ALL}
---
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
{Style.BRIGHT}RHI (Relative Hype Index){Style.RESET_ALL}
---
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
{Style.BRIGHT}Sentiment Score{Style.RESET_ALL}
---
Sentiment score is the percentage of people talking positively about the stock.
For each social network the number of positive posts/comments is divided by the
total number of both positive and negative posts/comments.
{Style.BRIGHT}SGP (Standard General Perception){Style.RESET_ALL}
---
SGP is a measure of whether people are more or less positive about a stock than
usual. It is calculated by averaging the past day of sentiment values and then
dividing it by the average of the past week of sentiment values.
"""
parser = argparse.ArgumentParser(
add_help=False,
prog="historical",
formatter_class=argparse.RawDescriptionHelpFormatter,
description=textwrap.dedent(command_description),
)
parser.add_argument(
"-t",
"--ticker",
action="store",
dest="ticker",
type=str,
default=ticker,
help="ticker for which to fetch data",
)
parser.add_argument(
"-s",
"--sort",
action="store",
type=str,
default="date",
help="the parameter to sort output table by",
dest="sort_param",
nargs="?",
choices=["date", "value"],
)
parser.add_argument(
"-d",
"--direction",
action="store",
type=str,
default="desc",
help="the direction to sort the output table",
dest="sort_dir",
nargs="?",
choices=["asc", "desc"],
)
parser.add_argument(
"metric",
type=str,
action="store",
default="sentiment",
nargs="?",
choices=["sentiment", "AHI", "RHI", "SGP"],
help="the metric to plot",
)
try:
ns_parser = parse_known_args_and_warn(parser, other_args)
if not ns_parser:
logging.error("There was an error in parsing the command-line arguments.")
return
if not sentipy.supported(ns_parser.ticker):
print("This stock is not supported by the SentimentInvestor API.")
return
data = sentipy.historical(
ns_parser.ticker,
ns_parser.metric,
int(time.time() - 60 * 60 * 24 * 7),
int(time.time()),
)
df = pd.DataFrame.from_dict(
{
"date": map(datetime.datetime.utcfromtimestamp, data.keys()),
ns_parser.metric: data.values(),
}
)
df.sort_index(ascending=True, inplace=True)
if df.empty:
logging.error("The dataset is empty, something must have gone wrong")
return
_customise_plot()
# use seaborn to lineplot
ax = sns.lineplot(
data=df,
x="date",
y=ns_parser.metric,
legend=False,
label=[ns_parser.metric],
)
# always show zero on the y-axis
plt.ylim(bottom=0)
# set the x-axis date formatting
ax.xaxis.set_major_formatter(mdates.DateFormatter("%x"))
# scale the plot appropriately
plt.gcf().set_size_inches(plot_autoscale())
plt.gcf().set_dpi(PLOT_DPI)
plt.gcf().autofmt_xdate()
# fill below the line
plt.fill_between(df.date, df[ns_parser.metric], alpha=0.3)
# add title e.g. AAPL sentiment since 22/07/21
plt.title(
f"{ns_parser.ticker} {ns_parser.metric} since {min(df.date).strftime('%x')}"
)
plt.show()
###
boundary = _Boundary(
0, max(df[ns_parser.metric].max(), 2 * df[ns_parser.metric].mean())
)
# average for each day
aggregated = df.resample("D", on="date").mean()
# reverse the ordering if requested
aggregated.sort_values(
ns_parser.metric
if ns_parser.sort_param == "value"
else ns_parser.sort_param,
axis=0,
ascending=ns_parser.sort_dir == "asc",
inplace=True,
)
# format the date according to user's locale
aggregated.index = aggregated.index.strftime("%x")
# apply coloring to every value
aggregated[ns_parser.metric] = [
boundary.categorise(value)[0] + str(value) + Style.RESET_ALL
for value in aggregated[ns_parser.metric]
]
print(
tabulate.tabulate(
aggregated,
headers=["Day", f"average {ns_parser.metric}"],
tablefmt="psql",
floatfmt=".3f",
)
)
print()
except Exception as e:
logging.error(e)
print(e, "\n")

View File

@@ -70,3 +70,9 @@ API_FINNHUB_KEY = os.getenv("GT_API_FINNHUB_KEY") or "REPLACE_ME"
# https://iexcloud.io
API_IEX_TOKEN = os.getenv("GT_API_IEX_KEY") or "REPLACE_ME"
# https://www.sentimentinvestor.com
API_SENTIMENTINVESTOR_KEY = os.getenv("GT_API_SENTIMENTINVESTOR_KEY") or "REPLACE_ME"
API_SENTIMENTINVESTOR_TOKEN = (
os.getenv("GT_API_SENTIMENTINVESTOR_TOKEN") or "REPLACE_ME"
)

View File

@@ -70,7 +70,8 @@ CURRENCIES = [
PLATFORMS = paprika.get_all_contract_platforms()["platform_id"].tolist()
COINS = paprika.get_list_of_coins()
COINS_DCT = dict(zip(COINS.id, COINS.symbol))
EXCHANGES = paprika.get_list_of_exchanges()
# see https://github.com/GamestonkTerminal/GamestonkTerminal/pull/562#issuecomment-887842888
# EXCHANGES = paprika.get_list_of_exchanges()
def global_market(other_args: List[str]):

View File

@@ -46,6 +46,10 @@ This menu aims to discover new stocks, and the usage of the following commands a
* dark pool short position [Stockgrid]
* [shortvol](#shortvol)
* short interest and days to cover [Stockgrid]
* [popular](#popular)
* most popular stocks on social media right now [SentimentInvestor.com]
* [emerging](#emerging)
* stocks that are being talked about more than usual [SentimentInvestor.com]
## ipo <a name="ipo"></a>
@@ -327,3 +331,55 @@ Print short interest and days to cover. [Source: Stockgrid]
<img width="949" alt="shortvol" src="https://user-images.githubusercontent.com/25267873/122323861-af7be400-cf1f-11eb-9de2-5c7f2debddf0.png">
## popular <a name="popular"></a>
```
usage: popular [-l LIMIT] [-h]
```
The `popular` command prints the top `n` (default=10) stocks with the highest Average Hype Index right now.
### AHI (Absolute Hype Index)
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
> **[Sentiment Investor](https://sentimentinvestor.com)** analyzes data from four major social media platforms to
> generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
> sentiment metrics powered by proprietary NLP models.
Optional arguments:
- `-l`, `--limit LIMIT`
the maximum number of stocks to retrieve
- `-h`, `--help`
show this help message
![`popular` command output](https://user-images.githubusercontent.com/8385172/126782851-5ede26c2-ab70-480d-a56d-867ba2c8ab73.png)
## emerging <a name="emerging"></a>
```
usage: emerging [-l LIMIT] [-h]
```
The `emerging` command prints the top `n` (default=10) stocks with the highest Relative Hype Index right now.
### RHI (Relative Hype Index)
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
> **[Sentiment Investor](https://sentimentinvestor.com)** analyzes data from four major social media platforms to
> generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
> sentiment metrics powered by proprietary NLP models.
Optional arguments:
- `-l`, `--limit LIMIT`
the maximum number of stocks to retrieve
- `-h`, `--help`
show this help message
![`emerging` command output](https://user-images.githubusercontent.com/8385172/126782979-33f2ad1e-e69d-4c52-91c3-438dedea2084.png)

View File

@@ -24,6 +24,7 @@ from gamestonk_terminal.discovery import (
finra_ats_view,
finnhub_view,
stockgrid_view,
sentimentinvestor_view,
)
@@ -59,6 +60,8 @@ class DiscoveryController:
"darkpool",
"darkshort",
"shortvol",
"popular",
"emerging",
]
def __init__(self):
@@ -112,6 +115,12 @@ class DiscoveryController:
)
print(" darkshort dark pool short position [Stockgrid.io]")
print(" shortvol short interest and days to cover [Stockgrid.io]")
print(
" popular show most popular stocks on social media right now [SentimentInvestor.com]"
)
print(
" emerging show stocks that are being talked about more than usual [SentimentInvestor.com]"
)
print("")
def switch(self, an_input: str):
@@ -255,6 +264,14 @@ class DiscoveryController:
"""Process shortvol command"""
stockgrid_view.shortvol(other_args)
def call_popular(self, other_args: List[str]):
"""Process popular command"""
sentimentinvestor_view.sort("AHI", other_args, "popular")
def call_emerging(self, other_args: List[str]):
"""Process popular command"""
sentimentinvestor_view.sort("RHI", other_args, "emerging")
def menu():
"""Discovery Menu"""

View File

@@ -0,0 +1,91 @@
import argparse
import logging
import textwrap
from typing import Any, List
from colorama import Style
from sentipy.sentipy import Sentipy
from tabulate import tabulate
from gamestonk_terminal import config_terminal as cfg
from gamestonk_terminal.helper_funcs import parse_known_args_and_warn
sentipy: Sentipy = Sentipy(
token=cfg.API_SENTIMENTINVESTOR_TOKEN, key=cfg.API_SENTIMENTINVESTOR_KEY
)
"""Initialise SentiPy with the user's API token and key"""
__command_descriptions = {
"popular": f"""
The {Style.BRIGHT}popular{Style.RESET_ALL} command prints the stocks with highest Average Hype Index right now.
{Style.BRIGHT}AHI (Absolute Hype Index){Style.RESET_ALL}
---
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
===
{Style.BRIGHT}Sentiment Investor{Style.RESET_ALL} analyzes data from four major social media platforms to
generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
sentiment metrics powered by proprietary NLP models.
""",
"emerging": f"""
The {Style.BRIGHT}emerging{Style.RESET_ALL} command prints the stocks with highest Relative Hype Index right now.
{Style.BRIGHT}RHI (Relative Hype Index){Style.RESET_ALL}
---
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
===
{Style.BRIGHT}Sentiment Investor{Style.RESET_ALL} analyzes data from four major social media platforms to
generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
sentiment metrics powered by proprietary NLP models.
""",
}
def sort(metric: str, other_args: List[str], command_name: str) -> None:
parser = argparse.ArgumentParser(
add_help=False,
prog=command_name,
formatter_class=argparse.RawDescriptionHelpFormatter,
description=textwrap.dedent(__command_descriptions[command_name]),
)
parser.add_argument(
"-l",
"--limit",
action="store",
dest="limit",
type=int,
default=10,
help="the maximum number of stocks to retrieve",
)
try:
ns_parser = parse_known_args_and_warn(parser, other_args)
if not ns_parser:
return
data = sentipy.sort(metric, ns_parser.limit)
table: List[List[Any]] = []
for index, stock in enumerate(data):
if not hasattr(stock, "symbol") or not hasattr(stock, metric):
logging.warning("data for stock %s is incomplete, ignoring", index + 1)
table.append([])
else:
table.append([index + 1, stock.symbol, stock.__getattribute__(metric)])
print(tabulate(table, headers=["Rank", "Ticker", metric], floatfmt=".3f"))
print()
except Exception as e:
logging.error(e)
print(e, "\n")

View File

@@ -867,6 +867,20 @@ def check_api_keys():
else:
key_dict["BINANCE"] = "defined, not tested"
# SentimentInvestor keys
si_keys = [cfg.API_SENTIMENTINVESTOR_KEY, cfg.API_SENTIMENTINVESTOR_TOKEN]
if "REPLACE_ME" in si_keys:
key_dict["SENTIMENT_INVESTOR"] = "Not defined"
else:
account = requests.get(
f"https://api.sentimentinvestor.com/v4/account"
f"?token={cfg.API_SENTIMENTINVESTOR_TOKEN}&key={cfg.API_SENTIMENTINVESTOR_KEY}"
)
if account.ok and account.json().get("success", False):
key_dict["SENTIMENT_INVESTOR"] = "Defined, test passed"
else:
key_dict["SENTIMENT_INVESTOR"] = "Defined, test unsuccessful"
print(
tabulate(
pd.DataFrame(key_dict.items()),
@@ -966,6 +980,7 @@ def about_us():
f"{Fore.CYAN}FinBrain: {Style.RESET_ALL}https://finbrain.tech\n"
f"{Fore.CYAN}Quiver Quantitative: {Style.RESET_ALL}https://www.quiverquant.com\n"
f"{Fore.CYAN}Ops.Syncretism: {Style.RESET_ALL}https://ops.syncretism.io/api.html\n"
f"{Fore.CYAN}SentimentInvestor: {Style.RESET_ALL}https://sentimentinvestor.com\n"
f"\n{Fore.RED}"
"DISCLAIMER: Trading in financial instruments involves high risks including the risk of losing some, "
"or all, of your investment amount, and may not be suitable for all investors. Before deciding to trade in "

36
poetry.lock generated vendored
View File

@@ -2380,6 +2380,28 @@ python-versions = "*"
[package.extras]
win32 = ["pywin32"]
[[package]]
name = "sentiment-investor"
version = "2.0.0"
description = "Access the Sentiment Investor API through Python"
category = "main"
optional = false
python-versions = "*"
[package.dependencies]
sentipy = "*"
[[package]]
name = "sentipy"
version = "0.2.0"
description = "Python implementations of Sentinel 2 Toolbox L2 products & spectral indices"
category = "main"
optional = false
python-versions = ">=3.6,<4.0"
[package.dependencies]
numpy = ">=1.18.4,<2.0.0"
[[package]]
name = "six"
version = "1.15.0"
@@ -3030,7 +3052,7 @@ prediction = ["tensorflow", "pmdarima"]
[metadata]
lock-version = "1.1"
python-versions = "^3.6.8"
content-hash = "47ea04f7495d8f1e468fcf36dc988d67aacd436a44149261bd77c631b5fc4a72"
content-hash = "f0f4648cb473e92ab7e49380ac8a8b7f0ec9ec05e8cf1c8c8bc2ffb3a390e599"
[metadata.files]
absl-py = [
@@ -3247,8 +3269,8 @@ chardet = [
{file = "chardet-4.0.0.tar.gz", hash = "sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa"},
]
charset-normalizer = [
{file = "charset-normalizer-2.0.3.tar.gz", hash = "sha256:c46c3ace2d744cfbdebceaa3c19ae691f53ae621b39fd7570f59d14fb7f2fd12"},
{file = "charset_normalizer-2.0.3-py3-none-any.whl", hash = "sha256:88fce3fa5b1a84fdcb3f603d889f723d1dd89b26059d0123ca435570e848d5e1"},
{file = "charset-normalizer-2.0.1.tar.gz", hash = "sha256:ad0da505736fc7e716a8da15bf19a985db21ac6415c26b34d2fafd3beb3d927e"},
{file = "charset_normalizer-2.0.1-py3-none-any.whl", hash = "sha256:b68b38179052975093d71c1b5361bf64afd80484697c1f27056e50593e695ceb"},
]
click = [
{file = "click-8.0.1-py3-none-any.whl", hash = "sha256:fba402a4a47334742d782209a7c79bc448911afe1149d07bdabdf480b3e2f4b6"},
@@ -4698,6 +4720,14 @@ send2trash = [
{file = "Send2Trash-1.7.1-py3-none-any.whl", hash = "sha256:c20fee8c09378231b3907df9c215ec9766a84ee20053d99fbad854fe8bd42159"},
{file = "Send2Trash-1.7.1.tar.gz", hash = "sha256:17730aa0a33ab82ed6ca76be3bb25f0433d0014f1ccf63c979bab13a5b9db2b2"},
]
sentiment-investor = [
{file = "sentiment-investor-2.0.0.tar.gz", hash = "sha256:ff97986fbd414e8166f92e4503b3b9cb8c583bff860aaba1dc9a3536666b57a4"},
{file = "sentiment_investor-2.0.0-py3-none-any.whl", hash = "sha256:e5744dd6e591d52d44d2ea51b1c5483e434affc598ea1a4b0cabab02f0d455ec"},
]
sentipy = [
{file = "sentipy-0.2.0-py3-none-any.whl", hash = "sha256:ca80a3cc284c08de8c7df0ee9bccff27fb2f4b07e8aa7f0dc52683e765855287"},
{file = "sentipy-0.2.0.tar.gz", hash = "sha256:dfcb4d5caeee6606714bc6e00966692ebf637616ce9215c69768583ad757efac"},
]
six = [
{file = "six-1.15.0-py2.py3-none-any.whl", hash = "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced"},
{file = "six-1.15.0.tar.gz", hash = "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259"},

1
pyproject.toml vendored
View File

@@ -76,6 +76,7 @@ GitPython = "^3.1.17"
fear-greed-index = "^0.1.3"
pyEX = "^0.5.0"
charset-normalizer = "2.0.1"
sentiment-investor = "^2.0.0"
[tool.poetry.dev-dependencies]
pytest = "^6.2.2"

View File

@@ -178,6 +178,7 @@ scs==2.1.4; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and
seaborn==0.11.1; python_version >= "3.6"
selenium==3.141.0
send2trash==1.7.1; python_version >= "3.6"
sentiment-investor==2.0.0
six==1.15.0; python_full_version >= "3.6.1" and python_version >= "3.5" and (python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0") and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6") and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.5") and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6") and (python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.6.0")
smmap==4.0.0; python_version >= "3.6"
snowballstemmer==2.1.0; python_full_version >= "3.6.8" and python_full_version < "4.0.0" and python_version >= "3.6"

1
requirements.txt vendored
View File

@@ -163,6 +163,7 @@ scs==2.1.4; python_full_version >= "3.6.1" and python_full_version < "4.0.0" and
seaborn==0.11.1; python_version >= "3.6"
selenium==3.141.0
send2trash==1.7.1; python_version >= "3.6"
sentiment-investor==2.0.0
six==1.15.0; python_full_version >= "3.6.1" and python_version >= "3.5" and (python_version >= "2.7" and python_full_version < "3.0.0" or python_full_version >= "3.3.0") and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.6") and (python_version >= "3.5" and python_full_version < "3.0.0" or python_full_version >= "3.3.0" and python_version >= "3.5") and (python_version >= "3.6" and python_full_version < "3.0.0" or python_full_version >= "3.5.0" and python_version >= "3.6")
smmap==4.0.0; python_version >= "3.6"
snowballstemmer==2.1.0; python_full_version >= "3.6.8" and python_full_version < "4.0.0" and python_version >= "3.6"