Remove latest news and add Customised news (#923)

* remove disc/latest feature from seeking alpha

* Add cnews to stocks/discovery

* Improve cnews command

* update hugo server

* remove old READMEs

* fix typo

* Improve code to remove d_ and l_

* add *.env to gitignore
This commit is contained in:
DidierRLopes
2021-11-06 12:08:22 +00:00
committed by GitHub
parent 3b9c8eccce
commit faf412de98
11 changed files with 197 additions and 1053 deletions

2
.gitignore vendored
View File

@@ -1,7 +1,7 @@
__pycache__/
*.pyc
.DS_Store
.env
*.env
.venv
.vscode
*.ipynb

View File

@@ -1,419 +0,0 @@
# BEHAVIOURAL ANALYSIS
This menu aims to extrapolate behavioural analysis regarding a pre-loaded ticker, and the usage of the following
commands along with an example will be exploited below.
* [finbrain](#finbrain)
* sentiment from 15+ major news headlines
* [stats](#stats)
* sentiment stats including comparison with sector
[REDDIT](#REDDIT)
* [wsb](#wsb)
* show what WSB gang is up to in subreddit wallstreetbets
* [watchlist](#watchlist)
* show other users watchlist
* [popular](#popular)
* show popular tickers
* [spac_c](#spac_c)
* show other users spacs announcements from subreddit SPACs community
* [spac](#spac)
* show other users spacs announcements from other subs
[STOCKTWITS](#STOCKTWITS)
* [bullbear](#bullbear)
* estimate quick sentiment from last 30 messages on board
* [messages](#messages)
* output up to the 30 last messages on the board
* [trending](#trending)
* trending stocks
* [stalker](#stalker)
* stalk stocktwits user's last messages
[TWITTER](#TWITTER)
* [infer](#infer)
* infer about stock's sentiment from latest tweets
* [sentiment](#sentiment)
* in-depth sentiment prediction from tweets over time
[GOOGLE](#GOOGLE)
* [mentions](#mentions)
* interest over time based on stock's mentions
* [regions](#regions)
* regions that show highest interest in stock
* [queries](#queries)
* top related queries with this stock
* [rise](#rise)
* top rising related queries with stock
[SENTIMENT INVESTOR](#SENTIMENTINVESTOR)
* [metrics](#metrics)
* core social sentiment metrics for this stock
* [social](#social)
* social media figures for stock popularity
* [historical](#historical)
* plot the past week of data for a selected metric
## finbrain <a name="finbrain"></a>
```text
usage: finbrain
```
FinBrain collects the news headlines from 15+ major financial news sources on a daily basis and analyzes them to
generate sentiment scores for more than 4500 US stocks. FinBrain Technologies develops deep learning algorithms for
financial analysis and prediction, which currently serves traders from more than 150 countries all around the world.
[Source: See <https://finbrain.tech>]
![finbrain2](https://user-images.githubusercontent.com/25267873/111629515-49c97000-87e9-11eb-92a6-a9eebb4b4bd9.png)
<img width="986" alt="Finbrain" src="https://user-images.githubusercontent.com/25267873/111629542-4fbf5100-87e9-11eb-8d5d-96faa8d3fca4.png">
## stats <a name="stats"></a>
```text
usage: stats
```
Sentiment stats which displays buzz, news score, articles last week, articles weekly average, bullish vs bearish
percentages, sector average bullish percentage, and sector average news score. [Source: <https://finnhub.io>]
<img width="1003" alt="Stats" src="https://user-images.githubusercontent.com/25267873/116888922-e9b94c80-ac23-11eb-959e-be24c79488a3.png">
# REDDIT <a name="REDDIT"></a>
## wsb <a name="wsb"></a>
```text
usage: wsb [-l N_LIMIT] [-n]
```
Print what WSB gang are up to in subreddit wallstreetbets. [Source: Reddit]
* -l : limit of posts to print. Default 10.
* -n : new flag, if true the posts retrieved are based on being more recent rather than their score. Default False.
<img width="947" alt="wsb" src="https://user-images.githubusercontent.com/25267873/108612074-d5afce00-73dc-11eb-80ca-8ecfad12020c.png">
## watchlist <a name="watchlist"></a>
```text
usage: watchlist [-l N_LIMIT]
```
Print other users watchlist. [Source: Reddit]
* -l : limit of posts to print. Default 5.
<img width="941" alt="watchlist" src="https://user-images.githubusercontent.com/25267873/108920576-caeb7800-762c-11eb-937c-94ecbb65d119.png">
## popular <a name="popular"></a>
```text
usage: popular [-l N_LIMIT] [-s S_SUBREDDIT] [-d N_DAYS]
```
Print latest popular tickers. [Source: Reddit]
* -l : limit of posts retrieved per sub reddit. Default 50.
* -s : subreddits to look for tickers, e.g. pennystocks,stocks. Default: pennystocks, RobinHoodPennyStocks, Daytrading,
StockMarket, stocks, investing, wallstreetbets.
<img width="948" alt="popular" src="https://user-images.githubusercontent.com/25267873/108917846-4dbe0400-7628-11eb-821e-9fda97a6d9cd.png">
## spac_c <a name="spac_c"></a>
```text
usage: spac_c [-l N_LIMIT] [-p]
```
Print other users SPACs announcement under subreddit 'SPACs'. [Source: Reddit]
* -l : limit of posts with SPACs retrieved. Default 10.
* -p : popular flag, if true the posts retrieved are based on score rather than time. Default False.
<img width="936" alt="spac_c" src="https://user-images.githubusercontent.com/25267873/108920571-c8891e00-762c-11eb-8da8-e2b775bc7109.png">
## spac <a name="spac"></a>
```text
usage: spac [-h] [-l N_LIMIT] [-d N_DAYS]
```
Print other users SPACs announcement under subreddit 'SPACs'. [Source: Reddit]
* -l : limit of posts with SPACs retrieved. Default 5.
<img width="939" alt="spac" src="https://user-images.githubusercontent.com/25267873/108920696-04bc7e80-762d-11eb-896b-1b1c19903584.png">
# STOCKTWITS <a name="STOCKTWITS"></a>
## bullbear <a name="bullbear"></a>
```text
usage: bullbear [-t S_TICKER]
```
Print bullbear sentiment based on last 30 messages on the board. Also prints the watchlist_count. [Source: Stocktwits]
* -t : ticker to gather sentiment from.
<img width="934" alt="sentiment" src="https://user-images.githubusercontent.com/25267873/108612307-42c46300-73df-11eb-9cec-253c8fb6d62f.png">
## messages <a name="messages"></a>
```text
usage: messages [-t S_TICKER] [-l N_LIM]
```
Print up to 30 of the last messages on the board. [Source: Stocktwits]
* -t : get board messages from this ticker. Default pre-loaded.
* -l : limit messages shown. Default 30.
<img width="958" alt="messages" src="https://user-images.githubusercontent.com/25267873/108612310-448e2680-73df-11eb-8fa8-b619c6269742.png">
## trending <a name="trending"></a>
```text
usage: trending
```
Stocks trending. [Source: Stocktwits]
<img width="956" alt="trending" src="https://user-images.githubusercontent.com/25267873/108612311-4526bd00-73df-11eb-840b-18829fcecdd3.png">
## stalker <a name="stalker"></a>
```text
usage: stalker [-u S_USER] [-l N_LIM]
```
Print up to the last 30 messages of a user. [Source: Stocktwits]
* -u : username. Default newsfilter.
* -l : limit messages shown. Default 30.
<img width="943" alt="stalker" src="https://user-images.githubusercontent.com/25267873/108612309-435cf980-73df-11eb-92c9-e9f15f966d8e.png">
# TWITTER <a name="TWITTER"></a>
Uses VADER sentiment. VADER (Valence Aware Dictionary and sEntiment Reasoner) is a lexicon and rule-based sentiment
analysis tool that is specifically attuned to sentiments expressed in social media.
The output of VADER analysis :
```text
The compound score is computed by summing the valence scores of each word in the lexicon, adjusted according to the
rules, and then normalized to be between -1 (most extreme negative) and +1 (most extreme positive). This is the most
useful metric if you want a single unidimensional measure of sentiment for a given sentence. Calling it a 'normalized,
weighted composite score' is accurate.
```
A sample tweet is shown below along with what the VADER polarity scores are.
```text
Have some $SENS 🚀🏆\n👉FDA approval coming very soon, 👉News is out, amazing test results. \n👉Very high short
interest rate. \n👉Major short squeeze coming. \n\n$amc $gme $nakd $ctrm $tsla $aapl $oeg $aht $bbby $bb $fsr $spce
https://t.co/B0cPhufpA2'
```
This gives the following scores:
```python
created_at Thu Jun 03 20:37:54
text Have some $SENS 🚀🏆\n👉FDA approval coming very...
sentiment 0.872
positive 0.177
negative 0
neutral 0.823
```
Note that both functions use twitters v2 endpoint, which only gives 7 days of historical data. It also requires at
least 10 tweets be pulled.
## infer <a name="infer"></a>
```text
usage: infer [-n N_NUM]
```
Print quick sentiment inference from last tweets that contain the ticker. This model splits the text into
character-level tokens and uses theVADER model to make predictions.
* -n : num of latest tweets to infer from. Default 100.
## sentiment <a name="sentiment"></a>
```text
usage: sentiment [-n N_NUM] [-d N_DAYS_PAST]
```
Plot in-depth sentiment extracted from tweets from last days that contain pre-defined ticker. This model splits the
text into character-level tokens and uses the VADER model to make predictions.
* -n : num of tweets to extract per hour. Default 15.
* -d : num of days in the past to extract tweets. Default 6. Max 6
# GOOGLE
## mentions <a name="mentions"></a>
```text
usage: mentions [-s S_START]
```
Plot weekly bars of stock's interest over time. other users watchlist. [Source: Google]
* -s : starting date (format YYYY-MM-DD) from when we are interested in stock's mentions. Default: the one provided
in main menu.
![gme](https://user-images.githubusercontent.com/25267873/108776894-e9813e80-755a-11eb-8b7e-654124c0ef8f.png)
## regions <a name="regions"></a>
```text
usage: regions [-n N_NUM]
```
Plot bars of regions based on stock's interest. [Source: Google]
* -n : number of regions to plot that show highest interest. Default 10.
![regions](https://user-images.githubusercontent.com/25267873/108776889-e8e8a800-755a-11eb-8bcc-fcf7b6156f50.png)
## queries <a name="queries"></a>
```text
usage: queries [-n N_NUM]
```
Print top related queries with this stock's query. [Source: Google]
* -n : number of top related queries to print. Default 10.
<img width="937" alt="Queries" src="https://user-images.githubusercontent.com/25267873/108777341-91970780-755b-11eb-985e-819285688eea.png">
## rise <a name="rise"></a>
```text
usage: rise [-n N_NUM]
```
Print top rising related queries with this stock's query. [Source: Google]
* -n : number of top rising related queries to print. Default 10.
<img width="934" alt="Rise" src="https://user-images.githubusercontent.com/25267873/108777814-4f21fa80-755c-11eb-96da-0327c9a0da57.png">
# SENTIMENTINVESTOR
> **[Sentiment Investor](https://sentimentinvestor.com)** analyzes data from four major social media platforms to
> generate hourly metrics on over 2,000 stocks. Sentiment provides volume and
> sentiment metrics powered by proprietary NLP models.
## Metric definitions
### AHI (Absolute Hype Index) <a name="AHI"></a>
AHI is a measure of how much people are talking about a stock on social media.
It is calculated by dividing the total number of mentions for the chosen stock
on a social network by the mean number of mentions any stock receives on that
social medium.
### RHI (Relative Hype Index) <a name="RHI"></a>
RHI is a measure of whether people are talking about a stock more or less than
usual, calculated by dividing the mean AHI for the past day by the mean AHI for
for the past week for that stock.
### Sentiment Score <a name="sentiment"></a>
Sentiment score is the percentage of people talking positively about the stock.
For each social network the number of positive posts/comments is divided by the
total number of both positive and negative posts/comments.
### SGP (Standard General Perception) <a name="SGP"></a>
SGP is a measure of whether people are more or less positive about a stock than
usual. It is calculated by averaging the past day of sentiment values and then
dividing it by the average of the past week of sentiment values.
## metrics <a name="metrics"></a>
```text
usage: metrics [-h] [ticker]
```
The `metrics` command presents the four core metrics provided by Sentiment Investor,
including AHI, RHI, sentiment and SGP, as described above.
Optional arguments:
* [optional, positional] ticker
ticker to use instead of the loaded one
* `-h`, `--help`
show this help message
Example output:
![`metrics NFLX` command output](https://user-images.githubusercontent.com/8385172/127154566-1af3c274-c521-426b-8580-2ae714c2c1ca.png)
## social <a name="social"></a>
```text
usage: social [-h] [ticker]
```
The `social` command prints the raw data for a given stock, including the number
of mentions it has received on social media in the last hour and the sentiment
score of those comments.
Optional arguments:
* [optional, positional] ticker
ticker to use instead of the loaded one
* `-h`, `--help`
show this help message
Example output:
![`social AAPL` command output](https://user-images.githubusercontent.com/8385172/127154775-13b1a81d-f5d2-4768-bf7e-bca8886f6d24.png)
## historical <a name="historical"></a>
```text
usage: historical [-t TICKER] [-s [{date,value}]] [-d [{asc,desc}]] [-h] [{sentiment,AHI,RHI,SGP}]
```
The `historical` command plots the past week of data for a selected
SentimentInvestor core metric, one of:
[AHI](#ahi), [RHI](#ahi), [sentiment](#sentiment) or [SGP](#SGP).
It also outputs a sorted table of the mean value for that metric for each day.
Positional arguments:
* one of `sentiment`, `AHI`, `RHI` or `SGP`
(optional) the metric to plot (default: `sentiment`)
Optional arguments:
* `-t`, `--ticker TICKER`
ticker for which to fetch data
* `-s`, `--sort [{date,value}]`
the parameter to sort output table by
* `-d`, `--direction [{asc,desc}]`
the direction to sort the output table
* `-h`, `--help`
show this help message
Example output:
![`historical -t GME RHI` plot output](https://user-images.githubusercontent.com/8385172/126782509-bfc21dbb-2fe6-4cd4-a384-018eb41ac8f3.png)
![`historical -t NFLX -s value -d asc RHI` command output](https://user-images.githubusercontent.com/8385172/127157236-7818a635-4394-4372-9673-2834b32811af.png)

View File

@@ -1,308 +0,0 @@
# PREDICTION TECHNIQUES
This menu aims to predict the share price of a pre-loaded stock, and the usage of the following commands along with an
example will be exploited below. See [How to fune-tuning a NN model](FUNE-TUNING-NN-models.md).
* [ets](#ets)
* Exponential Smoothing (e.g. Holt-Winters)
* [knn](#knn)
* k-Nearest Neighbors
* [linear](#linear)
* linear regression (polynomial 1)
* [quadratic](#quadratic)
* quadratic regression (polynomial 2)
* [cubic](#cubic)
* cubic regression (polynomial 3)
* [regression](#regression)
* regression (other polynomial)
* [arima](#arima)
* autoregressive integrated moving average
* [mlp](#mlp)
* MultiLayer Perceptron
* [rnn](#rnn)
* Recurrent Neural Network
* Contains a [looping example](#looping)
* [lstm](#lstm)
* Long-Short Term Memory
* Contains a [backtesting example](#backtesting)
* [conv1d](#conv1d)
* 1D Convolutional Neural Net
**Note:** _Use this at your own discretion. All of these prediciton techniques rely solely on the closing price of the
stock. This means that there are several factors that the models aren't aware of at the time of prediction, and may -
drastically - move the price up or down. Examples are: news, analyst price targets, reddit post, tweets from Elon Musk,
and so on._
**Note 2:** _[Enabling GPU acceleration for TensorFlow requires CUDA setup](README-gpu-accel.md) and will probably not
provide any speedup unless you are building large custom models._
## ets <a name="ets"></a>
```text
usage: ets [-t TREND] [-s SEASONAL] [-p SEASONAL_PERIODS] [-d N_DAYS]
```
Exponential Smoothing (based on trend+seasonality, see <https://otexts.com/fpp2/taxonomy.html>):
* -t : trend component: N: None, A: Additive, Ad: Additive Damped. Default N.
* -s : seasonality component: N: None, A: Additive, M: Multiplicative. Default N.
* -p : seasonal periods. Default 5.
* -d : prediction days. Default 5.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![ets_pltr](https://user-images.githubusercontent.com/25267873/110266847-97a6d280-7fb6-11eb-997e-0b598abc713b.png)
## knn <a name="knn"></a>
```text
usage: knn [-i N_INPUTS] [-d N_DAYS] [-n N_NEIGHBORS]
```
k-Nearest Neighbors:
* -i : number of days to use for prediction. Default 40.
* -d : prediction days. Default 5.
* -n : number of neighbors to use on the algorithm. Default 20.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![knn](https://user-images.githubusercontent.com/25267873/108604942-d169bd80-73a8-11eb-9021-6f787cbd41e3.png)
## linear <a name="linear"></a>
```text
usage: linear [-i N_INPUTS] [-d N_DAYS]
usage: regression -p 1 [-i N_INPUTS] [-d N_DAYS]
```
Linear Regression (p=1):
* -i : number of days to use for prediction. Default 40.
* -d : prediction days. Default 5.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![linear](https://user-images.githubusercontent.com/25267873/108604948-d3cc1780-73a8-11eb-860f-49274a34038b.png)
## quadratic <a name="quadratic"></a>
```text
usage: quadratic [-i N_INPUTS] [-d N_DAYS]
usage: regression -p 2 [-i N_INPUTS] [-d N_DAYS]
```
Quadratic Regression (p=2):
* -i : number of days to use for prediction. Default 40.
* -d : prediction days. Default 5.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![quadratic](https://user-images.githubusercontent.com/25267873/108604935-cca50980-73a8-11eb-9af1-bba807203cc6.png)
## cubic <a name="cubic"></a>
```text
usage: cubic [-i N_INPUTS] [-d N_DAYS]
usage: regression -p 3 [-i N_INPUTS] [-d N_DAYS]
```
Cubic Regression (p=3):
* -i : number of days to use for prediction. Default 40.
* -d : prediction days. Default 5.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![cubic](https://user-images.githubusercontent.com/25267873/108604941-d169bd80-73a8-11eb-9220-84a7013e1283.png)
## regression <a name="regression"></a>
```text
usage: regression -p N_POLYNOMIAL [-i N_INPUTS] [-d N_DAYS]
```
Regression:
* -i : number of days to use for prediction. Default 40.
* -d : prediction days. Default 5.
* -p : polynomial associated with regression. Required.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![regression](https://user-images.githubusercontent.com/25267873/108604946-d3338100-73a8-11eb-9e99-fa526fb56672.png)
## arima <a name="arima"></a>
```text
usage: arima [-d N_DAYS] [-i {aic,aicc,bic,hqic,oob}] [-s] [-r] [-o S_ORDER]
```
Auto-Regressive Integrated Moving Average:
* -d : prediciton days. Default 5.
* -i : information criteria - used if auto_arima library is invoked. Default aic.
* -s : weekly seasonality flag. Default False.
* -r : results about ARIMA summary flag. Default False.
* -o : arima model order. If the model order is defined, auto_arima is not invoked, deeming information criteria useless.
<br />Example: `-o 5,1,4` where:
* p = 5 : order (number of time lags) of the autoregressive model.
* d = 1 : degree of differencing (the number of times the data have had past values subtracted).
* q = 4 : order of the moving-average model.
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
![arima](https://user-images.githubusercontent.com/25267873/108604947-d3cc1780-73a8-11eb-9dbb-53b959ae7947.png)
## mlp <a name="mlp"></a>
```text
usage: mlp [-d N_DAYS] [-i N_INPUTS] [--epochs N_EPOCHS] [-e S_END_DATE] [--loops N_LOOPS] [-v VALID]
[--lr LEARNING_RATE] [--no_shuffle]
```
MulitLayer Perceptron:
* -d : prediciton days. Default 5.
* -i : number of days to use for prediction. Default 40.
* --epochs : number of training epochs. Default 200.
* -v : validation split. Default 0.1
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
* --xla_cpu: if present, will enable XLA for CPU (overrides environment variables during run).
* --xla_gpu: if present, will enable XLA for GPU (overrides environment variables during run).
* --force_allow_gpu_growth: if true, will force TensorFlow to allow GPU memory usage to grow as needed. Otherwise will
allocate 100% of available GPU memory when CUDA is set up. Default true.
* --batch_size: batch size for model training, should not be used unless advanced user. Default None.
* --lr : learning rate for optimizer
* --loops: number of loops to iterate and train models. Default 1.
* --no_shuffle : split validation data in time_ordered way instead of random.
Due to the complexity of defining a model through command line, one can define it in:
[config_neural_network_models.txt](/config_neural_network_models.py)
```python
MultiLayer_Perceptron \
= [ {'Dense':
{'units':50, 'activation':'relu'} },
{'Dense':
{'units':100, 'activation':'relu'} },
{'Dense':
{'units':80, 'activation':'relu'} },
{'Dense':
{'units':30, 'activation':'relu'} },
{'Dense':
{'activation':'relu'} }]
```
![mlp](https://user-images.githubusercontent.com/25267873/108604944-d2025400-73a8-11eb-9ab6-52972160cd2a.png)
## rnn <a name="rnn"></a>
```text
usage: rnn [-d N_DAYS] [-i N_INPUTS] [--epochs N_EPOCHS] [-p {normalization,standardization,none}]
[-l {mae,mape,mse,msle}] [-e S_END_DATE] [--loops N_LOOPS] [-v VALID] [--lr LEARNING_RATE] [--no_shuffle]
```
Recurrent Neural Network:
* -d : prediciton days. Default 5.
* -i : number of days to use for prediction. Default 40.
* --epochs : number of training epochs. Default 200.
* -v : validation split. Default 0.1
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
* --xla_cpu: if present, will enable XLA for CPU (overrides environment variables during run).
* --xla_gpu: if present, will enable XLA for GPU (overrides environment variables during run).
* --force_allow_gpu_growth: if true, will force TensorFlow to allow GPU memory usage to grow as needed.
Otherwise will allocate 100% of available GPU memory when CUDA is set up. Default true.
* --batch_size: batch size for model training, should not be used unless advanced user. Default None.
* --loops: number of loops to iterate and train models. Default 1.
* --lr : learning rate for optimizer
* --no_shuffle : split validation data in time_ordered way instead of random.
Due to the complexity of defining a model through command line, one can define it in:
[config_neural_network_models.txt](/config_neural_network_models.py)
```python
Recurrent_Neural_Network \
= [ {'SimpleRNN':
{'units':100, 'activation':'linear', 'return_sequences':True} },
{'SimpleRNN':
{'units':50, 'activation':'linear', 'return_sequences':True} },
{'Dropout':
{'rate':0.2} },
{'SimpleRNN':
{'units':21, 'activation':'linear', 'return_sequences':False} },
{'Dense':
{'activation':'relu'} }]
```
![rnn](https://user-images.githubusercontent.com/25267873/108604940-d0d12700-73a8-11eb-837e-a5aa128942d9.png)
### Looping Example <a name="looping"></a>
![loops](https://user-images.githubusercontent.com/25267873/111932423-479b3600-8ab5-11eb-9d0b-7210d5f02e83.png)
<img width="1006" alt="Captura de ecrã 2021-03-22, às 02 14 38" src="https://user-images.githubusercontent.com/25267873/111932420-436f1880-8ab5-11eb-831c-ebb88f1c5473.png">
## lstm <a name="lstm"></a>
```text
usage: lstm [-d N_DAYS] [-i N_INPUTS] [--epochs N_EPOCHS] [-e S_END_DATE] [--loops N_LOOPS] [-v VALID]
[--lr LEARNING_RATE] [--no_shuffle]
```
Long-Short Term Memory:
* -d : prediction days. Default 5.
* -i : number of days to use for prediction. Default 40.
* --epochs : number of training epochs. Default 200.
* -v : validation split. Default 0.1
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
* --xla_cpu: if present, will enable XLA for CPU (overrides environment variables during run).
* --xla_gpu: if present, will enable XLA for GPU (overrides environment variables during run).
* --force_allow_gpu_growth: if true, will force TensorFlow to allow GPU memory usage to grow as needed. Otherwise will
allocate 100% of available GPU memory when CUDA is set up. Default true.
* --batch_size: batch size for model training, should not be used unless advanced user. Default None.
* --loops: number of loops to iterate and train models. Default 1
* --lr : learning rate for optimizer
* --no_shuffle : split validation data in time_ordered way instead of random.
Due to the complexity of defining a model through command line, one can define it in:
[config_neural_network_models.py](/config_neural_network_models.py)
```python
Long_Short_Term_Memory \
= [ {'LSTM':
{'units':25, 'activation':'tanh', 'return_sequences':True} },
{'LSTM':
{'units':15, 'activation':'tanh', 'return_sequences':False} },
{'Dense':
{'activation':'relu'} }]
```
![lstm](https://user-images.githubusercontent.com/25267873/108604943-d2025400-73a8-11eb-83c5-edb4a2121cba.png)
### Backtesting Example <a name="backtesting"></a>
![appl_pred](https://user-images.githubusercontent.com/25267873/111053156-4173dc80-8459-11eb-9fcb-e81211961743.png)
![apple_error](https://user-images.githubusercontent.com/25267873/111053157-420c7300-8459-11eb-8c37-fb1b5208d635.png)
<img width="992" alt="Captura de ecrã 2021-03-14, às 00 06 51" src="https://user-images.githubusercontent.com/25267873/111053158-420c7300-8459-11eb-993b-6d9c26f98af9.png">
## conv1d <a name="conv1d"></a>
```text
usage: lstm [-d N_DAYS] [-i N_INPUTS] [--epochs N_EPOCHS] [-e S_END_DATE] [--loops N_LOOPS] [-v VALID]
[--lr LEARNING_RATE] [--no_shuffle]
```
1D Convolutional Neural Net:
* -d : prediction days. Default 5.
* -i : number of days to use for prediction. Default 40.
* --epochs : number of training epochs. Default 50.
* -v : validation split. Default 0.1
* -e : end date (format YYYY-MM-DD) of the stock - Backtesting. Default None.
* --xla_cpu: if present, will enable XLA for CPU (overrides environment variables during run).
* --xla_gpu: if present, will enable XLA for GPU (overrides environment variables during run).
* --force_allow_gpu_growth: if true, will force TensorFlow to allow GPU memory usage to grow as needed. Otherwise will
allocate 100% of available GPU memory when CUDA is set up. Default true.
* --batch_size: batch size for model training, should not be used unless advanced user. Default None.
* --loops: number of loops to iterate and train models. Default 1.
* --lr : learning rate for optimizer
* --no_shuffle : split validation data in time_ordered way instead of random.

View File

@@ -1,240 +0,0 @@
# TECHNICAL ANALYSIS
This menu aims to perform a technical analysis on a pre-loaded ticker chart, and the usage of the following commands
along with an example will be exploited below.
* [view](#view)
* view historical data and trendlines [Finviz]
* [summary](#summary)
* technical summary report [FinBrain API]
* [recom](#recom)
* recommendation based on Technical Indicators [Tradingview API]
* [pr](#pr)
* pattern recognition [Finnhub]
[OVERLAP](#OVERLAP)
* [ema](#ema)
* exponential moving average
* [sma](#sma)
* simple moving average
* [vwap](#vwap)
* volume weighted average price
[MOMENTUM](#MOMENTUM)
* [cci](#cci)
* commodity channel index
* [macd](#macd)
* moving average convergence/divergence
* [rsi](#rsi)
* relative strength index
* [stoch](#stoch)
* stochastic oscillator
[TREND](#TREND)
* [adx](#adx)
* average directional movement index
* [aroon](#aroon)
* aroon indicator
[VOLATILITY](#VOLATILITY)
* [bbands](#bbands)
* bollinger bands
[VOLUME](#VOLUME)
* [ad](#ad)
* Accumulation/distribution line values
* [obv](#obv)
* on balance volume
[CUSTOM](#CUSTOM)
* [fib](#fib)
* Shows fibonocci levels
**S/O to <https://github.com/twopirllc/pandas-ta>** _Owing to this library, it is fairly easy to add other technical
indicators. So, let us know if there's any other that you would like._
## view <a name="view"></a>
```text
usage: view
```
View historical price with trendlines. [Source: Finviz]
![aapl](https://user-images.githubusercontent.com/25267873/113757843-02107700-970b-11eb-99ab-eb9b1312547f.png)
## summary <a name="summary"></a>
```text
usage: summary
```
Technical summary report provided by FinBrain's API. FinBrain Technologies develops deep learning algorithms for
financial analysis and prediction, which currently serves traders from more than 150 countries all around the world.
[Source: <https://finbrain.tech>]
<img width="976" alt="Captura de ecrã 2021-03-29, às 00 31 29" src="https://user-images.githubusercontent.com/25267873/112772089-db758080-9026-11eb-93d5-9fd7a4b40380.png">
## recom <a name="recom"></a>
```text
usage: recom [-s {crypto,forex,cfd}] [-e EXCHANGE] [-i {1M,1W,1d,4h,1h,15m,5m,1m}]
```
Print tradingview recommendation based on technical indicators. [Source: <https://pypi.org/project/tradingview-ta/>]
* -s : screener. Default america.
* -e : exchange, e.g. NASDAQ, NYSE. Default uses Alpha Vantage to get this information from ticker
* -i : interval. Default is showing all of different intervals.
<img width="977" alt="Recom" src="https://user-images.githubusercontent.com/25267873/113069531-76ea2b00-91b8-11eb-8934-9f693d3b4ffa.png">
## pr <a name="pr"></a>
```text
usage: pr [-r {1,5,15,30,60,D,W,M}]
```
Display pattern recognition signals on the data. [Source: <https://finnhub.io>]
* -r : Plot resolution to look for pattern signals. Default 'D'.
![pr](https://user-images.githubusercontent.com/25267873/116812520-06338700-ab47-11eb-8db4-8f5e940507ed.png)
<img width="985" alt="PR" src="https://user-images.githubusercontent.com/25267873/116812505-f6b43e00-ab46-11eb-9776-fa469dc3d3f0.png">
# OVERLAP <a name="OVERLAP"></a>
## ema <a name="ema"></a>
```text
usage: ema [-l N_LENGTH] [-o N_OFFSET]
```
![ema](https://user-images.githubusercontent.com/25267873/108602300-33bac200-7399-11eb-971a-f23f7097aba3.png)
## sma <a name="sma"></a>
```text
usage: sma [-l L_LENGTH] [-o N_OFFSET]
```
![sma](https://user-images.githubusercontent.com/25267873/108602304-36b5b280-7399-11eb-86fe-d490fb32aaff.png)
## vwap <a name="vwap"></a>
VWAP is available for intraday data. This plots the most recent day only.
```text
usage: vwap [-o N_OFFSET]
```
![vwap](https://user-images.githubusercontent.com/18151143/127217340-0f38bafd-fc16-4be4-a80a-6b89913559f5.png)
# MOMENTUM <a name="MOMENTUM"></a>
## cci <a name="cci"></a>
```text
usage: cci [-l N_LENGTH] [-s N_SCALAR] [-o N_OFFSET]
```
![cci](https://user-images.githubusercontent.com/25267873/108602745-96ad5880-739b-11eb-9e86-a1706c3ecb1f.png)
## macd <a name="macd"></a>
```text
usage: macd [-f N_FAST] [-s N_SLOW] [--signal N_SIGNAL] [-o N_OFFSET]
```
![macd](https://user-images.githubusercontent.com/25267873/108602739-92813b00-739b-11eb-8a4e-8fa7ed66b2b6.png)
## rsi <a name="rsi"></a>
```text
usage: rsi [-l N_LENGTH] [-s N_SCALAR] [-d N_DRIFT] [-o N_OFFSET]
```
![rsi](https://user-images.githubusercontent.com/25267873/108602743-957c2b80-739b-11eb-9d0d-6f530b3abb91.png)
## stoch <a name="stoch"></a>
```text
usage: stoch [-k N_FASTKPERIOD] [-d N_SLOWDPERIOD] [--slowkperiod N_SLOWKPERIOD] [-o N_OFFSET]
```
![stoch](https://user-images.githubusercontent.com/25267873/108602744-9614c200-739b-11eb-8e4d-2cdc9981cb67.png)
# TREND <a name="TREND"></a>
## adx <a name="adx"></a>
```text
usage: adx [-l N_LENGTH] [-s N_SCALAR] [-d N_DRIFT] [-o N_OFFSET]
```
![adx](https://user-images.githubusercontent.com/25267873/108602991-2bfd1c80-739d-11eb-973f-b836c1b839e6.png)
## aroon <a name="aroon"></a>
```text
usage: aroon [-l N_LENGTH] [-s N_SCALAR] [-o N_OFFSET]
```
![aroon](https://user-images.githubusercontent.com/25267873/108602987-2b648600-739d-11eb-9a90-154145190595.png)
# VOLATILITY <a name="VOLATILITY"></a>
## bbands <a name="bbands"></a>
```text
usage: bbands [-h] [-l N_LENGTH] [-s N_STD] [-m S_MAMODE] [-o N_OFFSET]
```
![bbands](https://user-images.githubusercontent.com/25267873/108602984-28699580-739d-11eb-9b82-2683a9840145.png)
# VOLUME <a name="VOLUME"></a>
## ad <a name="ad"></a>
```text
usage: ad [-o N_OFFSET] [--open]
```
![ad](https://user-images.githubusercontent.com/18151143/127217265-73415705-3321-478f-b5a0-3ae1487c74ac.png)
## obv <a name="obv"></a>
```text
usage: obv [-o N_OFFSET]
```
![obv](https://user-images.githubusercontent.com/18151143/127217286-d73c1079-1b84-4da9-a5d4-39f4eb2d6b1b.png)
# CUSTOM <a name="CUSTOM"></a>
## fib <a name="fib"></a>
Shows fibonocci levels
```python
usage: fib [-p PERIOD] [--start] [--end]
```
* -p/--period: Number of days to lookback for retracement. Default = 120.
* --start: Allows for custom selection of min point.
* --end : Allows for custom selection of max point.
The fibonocci retracement works by looking at a minimum and maximum point. Between these values, there is typically
support/resistance at 23.5 %, 38.2%, 50% and 61.8%. Those values follow the golden ratio from the fibonocci sequence.
The `--start` and `--end` flags allow the user to select dates for the min and max points as your retracement levels.
Both flags are required if using.
![Fib1](https://user-images.githubusercontent.com/18151143/127217240-0b4cd88b-8fd3-484f-941e-b02a0efe08dd.png)

View File

@@ -15,8 +15,8 @@ Command|Description|Source
`ford` |orders by Fidelity Customers |[Fidelity](https://www.fidelity.com/)
`arkord` |orders by ARK Investment Management LLC | [Cathiesark](https://www.cathiesark.com)
`upcoming` |upcoming earnings release dates |[Seeking Alpha](https://seekingalpha.com/)
`latest` |latest news |[Seeking Alpha](https://seekingalpha.com/)
`trending` |trending news |[Seeking Alpha](https://seekingalpha.com/)
`cnews` |customized news (buybacks, ipos, spacs, healthcare, politics) |[Seeking Alpha](https://seekingalpha.com/)
`low_float` |show low float stocks under 10M shares float |[Low Float](https://www.lowfloat.com/)
`hotpenny` |today's hot penny stocks |[hotpenny](https://pennystockflow.com)
`fds` |advanced Equities search based on country, sector, industry, name and/or description |[FinanceDatabase](https://github.com/JerBouma/FinanceDatabase)

View File

@@ -58,12 +58,12 @@ class DiscoveryController:
"ford",
"arkord",
"upcoming",
"latest",
"trending",
"lowfloat",
"hotpenny",
"rtearn",
"fds",
"cnews",
]
CHOICES += CHOICES_COMMANDS
@@ -105,8 +105,8 @@ cathiesark.com:
arkord orders by ARK Investment Management LLC
Seeking Alpha:
upcoming upcoming earnings release dates
latest latest news
trending trending news
cnews customized news (buybacks, ipos, spacs, healthcare, politics)
shortinterest.com
lowfloat low float stocks under 10M shares float
pennystockflow.com
@@ -683,66 +683,6 @@ Finance Database:
export=ns_parser.export,
)
@try_except
def call_latest(self, other_args: List[str]):
"""Process latest command"""
parser = argparse.ArgumentParser(
add_help=False,
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog="latest",
description="""Latest news articles. [Source: Seeking Alpha]""",
)
parser.add_argument(
"-i",
"--id",
action="store",
dest="n_id",
type=check_positive,
default=-1,
help="article ID",
)
parser.add_argument(
"-n",
"--num",
action="store",
dest="n_num",
type=check_positive,
default=5,
help="number of articles being printed",
)
parser.add_argument(
"-d",
"--date",
action="store",
dest="s_date",
type=valid_date,
default=datetime.now().strftime("%Y-%m-%d"),
help="starting date of articles",
)
parser.add_argument(
"--export",
choices=["csv", "json", "xlsx"],
default="",
type=str,
dest="export",
help="Export dataframe data to csv,json,xlsx file",
)
if other_args:
if "-" not in other_args[0]:
other_args.insert(0, "-i")
ns_parser = parse_known_args_and_warn(parser, other_args)
if not ns_parser:
return
seeking_alpha_view.news(
news_type="latest",
article_id=ns_parser.n_id,
num=ns_parser.n_num,
start_date=ns_parser.s_date,
export=ns_parser.export,
)
@try_except
def call_trending(self, other_args: List[str]):
"""Process trending command"""
@@ -844,6 +784,78 @@ Finance Database:
export=ns_parser.export,
)
@try_except
def call_cnews(self, other_args: List[str]):
"""Process cnews command"""
parser = argparse.ArgumentParser(
add_help=False,
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
prog="cnews",
description="""Customized news. [Source: Seeking Alpha]""",
)
l_news_type = [
"Top-News",
"On-The-Move",
"Market-Pulse",
"Notable-Calls",
"Buybacks",
"Commodities",
"Crypto",
"Issuance",
"Global",
"Guidance",
"IPOs",
"SPACs",
"Politics",
"M-A",
"Consumer",
"Energy",
"Financials",
"Healthcare",
"MLPs",
"REITs",
"Technology",
]
parser.add_argument(
"-t",
"--type",
action="store",
dest="s_type",
choices=[tnews.lower() for tnews in l_news_type],
default="Top-News",
help="number of news to display",
)
parser.add_argument(
"-n",
"--num",
action="store",
dest="n_num",
type=check_positive,
default=5,
help="number of news to display",
)
parser.add_argument(
"--export",
choices=["csv", "json", "xlsx"],
default="",
type=str,
dest="export",
help="Export dataframe data to csv,json,xlsx file",
)
if other_args:
if "-" not in other_args[0]:
other_args.insert(0, "-t")
ns_parser = parse_known_args_and_warn(parser, other_args)
if not ns_parser:
return
seeking_alpha_view.display_news(
news_type=ns_parser.s_type,
num=ns_parser.n_num,
export=ns_parser.export,
)
@try_except
def call_hotpenny(self, other_args: List[str]):
"""Process hotpenny command"""

View File

@@ -1,7 +1,7 @@
""" Seeking Alpha Model """
__docformat__ = "numpy"
from typing import List
from typing import List, Dict
from datetime import datetime
import requests
from bs4 import BeautifulSoup
@@ -210,3 +210,66 @@ def get_article_data(article_id: int) -> dict:
}
return article
def get_news_html(news_type: str = "Top-News") -> dict:
"""Gets news. [Source: SeekingAlpha]
Parameters
----------
news_type : str
From: Top-News, On-The-Move, Market-Pulse, Notable-Calls, Buybacks, Commodities, Crypto, Issuance, Global,
Guidance, IPOs, SPACs, Politics, M-A, Consumer, Energy, Financials, Healthcare, MLPs, REITs, Technology
Returns
-------
dict
HTML page of articles
"""
sa_url = (
f"http://seekingalpha.com/api/v3/news?filter%5Bcategory%5D=market-news%3A%3A{news_type}"
"&filter%5Bsince%5D=0&filter%5Buntil%5D=0&include=author%2CprimaryTickers%2CsecondaryTickers"
"&isMounting=true&page%5Bsize%5D=25&page%5Bnumber%5D=1"
)
articles_html = requests.get(
sa_url, headers={"User-Agent": get_user_agent()}
).json()
return articles_html
def get_news(news_type: str = "Top-News", num: int = 5) -> List:
"""Gets news. [Source: SeekingAlpha]
Parameters
----------
news_type : str
From: Top-News, On-The-Move, Market-Pulse, Notable-Calls, Buybacks, Commodities, Crypto, Issuance, Global,
Guidance, IPOs, SPACs, Politics, M-A, Consumer, Energy, Financials, Healthcare, MLPs, REITs, Technology
num : int
Number of news to display
Returns
-------
List[dict]
List of dict news
"""
news_articles: Dict = get_news_html(news_type)
news_to_display = list()
if news_articles:
for idx, news in enumerate(news_articles["data"]):
if idx > num:
break
news_to_display.append(
{
"publishOn": news["attributes"]["publishOn"].replace("T", " ")[:-6],
"id": news["id"],
"title": news["attributes"]["title"],
"url": news["links"]["canonical"],
}
)
return news_to_display

View File

@@ -1,6 +1,7 @@
""" Seeking Alpha View """
__docformat__ = "numpy"
from typing import List
import os
from datetime import datetime
from tabulate import tabulate
@@ -149,3 +150,41 @@ def news(news_type: str, article_id: int, num: int, start_date: datetime, export
news_type,
df_articles,
)
def display_news(news_type: str = "Top-News", num: int = 5, export: str = ""):
"""Display news. [Source: SeekingAlpha]
Parameters
----------
news_type : str
From: Top-News, On-The-Move, Market-Pulse, Notable-Calls, Buybacks, Commodities, Crypto, Issuance, Global,
Guidance, IPOs, SPACs, Politics, M-A, Consumer, Energy, Financials, Healthcare, MLPs, REITs, Technology
num : int
Number of news to display
export : str
Export dataframe data to csv,json,xlsx file
"""
news_to_display: List = seeking_alpha_model.get_news(news_type, num)
if not news:
print("No news found.", "\n")
else:
for news_element in news_to_display:
print(
news_element["publishOn"]
+ " - "
+ news_element["id"]
+ " - "
+ news_element["title"]
)
print(news_element["url"])
print("")
export_data(
export,
os.path.dirname(os.path.abspath(__file__)),
"cnews : " + news_type,
pd.DataFrame(news_to_display),
)

View File

@@ -0,0 +1,16 @@
```
usage: cnews [-t {top-news,on-the-move,market-pulse,notable-calls,buybacks,commodities,crypto,issuance,global,guidance,ipos,spacs,politics,m-a,consumer,energy,financials,healthcare,mlps,reits,technology}] [-n N_NUM] [--export {csv,json,xlsx}] [-h]
```
Customized news. [Source: Seeking Alpha]
```
optional arguments:
-t {top-news,on-the-move,market-pulse,notable-calls,buybacks,commodities,crypto,issuance,global,guidance,ipos,spacs,politics,m-a,consumer,energy,financials,healthcare,mlps,reits,technology}, --type {top-news,on-the-move,market-pulse,notable-calls,buybacks,commodities,crypto,issuance,global,guidance,ipos,spacs,politics,m-a,consumer,energy,financials,healthcare,mlps,reits,technology}
number of news to display (default: Top-News)
-n N_NUM, --num N_NUM
number of news to display (default: 5)
--export {csv,json,xlsx}
Export dataframe data to csv,json,xlsx file (default: )
-h, --help show this help message (default: False)
```

View File

@@ -1,19 +0,0 @@
```
usage: latest [-i N_ID] [-n N_NUM] [-d S_DATE] [--export {csv,json,xlsx}] [-h]
```
Latest news articles. [Source: Seeking Alpha]
```
optional arguments:
-i N_ID, --id N_ID article ID (default: -1)
-n N_NUM, --num N_NUM
number of articles being printed (default: 5)
-d S_DATE, --date S_DATE
starting date of articles (default: 2021-09-16)
--export {csv,json,xlsx}
Export dataframe data to csv,json,xlsx file (default: )
-h, --help show this help message (default: False)
```
<img width="1208" alt="latest" src="https://user-images.githubusercontent.com/25267873/115089633-926c6a00-9f0a-11eb-9d0e-1eedfd8ba7ce.png">

View File

@@ -40,8 +40,8 @@ main:
ref: "/stocks/discovery/arkord"
- name: upcoming
ref: "/stocks/discovery/upcoming"
- name: latest
ref: "/stocks/discovery/latest"
- name: cnews
ref: "/stocks/discovery/cnews"
- name: trending
ref: "/stocks/discovery/trending"
- name: lowfloat