diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index d08515fc06f..f3015b652c2 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -32,7 +32,7 @@ jobs: - run: pip install types-pytz types-requests types-termcolor types-tabulate types-PyYAML types-python-dateutil types-setuptools types-six - run: bandit -x ./tests -r . || true - run: black --diff --check . - - run: codespell --ignore-words-list=commun,statics,ro,zar,zlot,ba,buil,coo,ether,hist,hsi,mape,navagation,operatio,pres,ser,yeld,shold,ist,varian,datas,ake,creat,vie,hel,ket,toke,certi,buidl --quiet-level=2 --skip=./tests,.git,*.css,*.csv,*.html,*.ini,*.ipynb,*.js,*.json,*.lock,*.scss,*.txt,*.yaml,./build/pyinstaller,./website/config.toml + - run: codespell --ignore-words-list=commun,statics,ro,zar,zlot,jewl,ba,buil,coo,ether,hist,hsi,mape,navagation,operatio,pres,ser,yeld,shold,ist,varian,datas,ake,creat,vie,hel,ket,toke,certi,buidl --quiet-level=2 --skip=./tests,.git,*.css,*.csv,*.html,*.ini,*.ipynb,*.js,*.json,*.lock,*.scss,*.txt,*.yaml,./build/pyinstaller,./website/config.toml - run: flake8 . --count --ignore=E203,W503 --max-line-length=122 --show-source --statistics --exclude ./build/pyinstaller,./website/content - run: mypy --ignore-missing-imports openbb_terminal - run: shopt -s globstar && pyupgrade --py38-plus **/*.py diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index b20981bda31..7a0333e584d 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -25,7 +25,7 @@ repos: entry: codespell args: [ - "--ignore-words-list=commun,ro,zar,vie,hel,zlot,ba,buil,coo,ether,hist,hsi,mape,navagation,operatio,pres,ser,yeld,shold,ist,varian,datas,ake,creat,statics,ket,toke,certi,buidl", + "--ignore-words-list=commun,ro,zar,vie,hel,jewl,zlot,ba,buil,coo,ether,hist,hsi,mape,navagation,operatio,pres,ser,yeld,shold,ist,varian,datas,ake,creat,statics,ket,toke,certi,buidl", "--quiet-level=2", "--skip=./tests,.git,*.css,*.csv,*.html,*.ini,*.ipynb,*.js,*.json,*.lock,*.scss,*.txt,*.yaml,build/pyinstaller/*,./website/config.toml", ] diff --git a/README.md b/README.md index d44dbbd9c44..8433c9c75c4 100644 --- a/README.md +++ b/README.md @@ -45,9 +45,9 @@

Investment research for everyone.
- ≪ GETTING STARTED + ≪ GETTING STARTED   ·   CONTRIBUTING   ·   - + SEE FEATURES »

@@ -98,14 +98,14 @@ If you wish to install the OpenBB Terminal or the OpenBB SDK, please use one of |**OpenBB Terminal**|**Usage**| |:-|:-| -|[Installer](https://openbb-finance.github.io/OpenBBTerminal/#accessing-the-openbb-terminal)|Recommended way if you just want to use the OpenBB Terminal| -|[Docker](openbb_terminal/DOCKER_README.md)|An alternative way if you just want to use the OpenBB Terminal| -|[Python](openbb_terminal/README.md#anaconda--python)|If you wish to contribute to the development of the OpenBB Terminal| +|[Installer](https://docs.openbb.co/terminal/quickstart/installation)|Recommended way if you just want to use the OpenBB Terminal| +|[Docker](https://docs.openbb.co/terminal/quickstart/installation)|An alternative way if you just want to use the OpenBB Terminal| +|[Python](https://docs.openbb.co/terminal/quickstart/installation)|If you wish to contribute to the development of the OpenBB Terminal| |**OpenBB SDK**         |**Usage**| |:-|:-| -|[PyPi](https://openbb-finance.github.io/OpenBBTerminal/SDK/)|Recommended way if you just want to use the OpenBB SDK| -|[Python](openbb_terminal/README.md#anaconda--python)|If you wish to contribute to the development of the OpenBB SDK        | +|[PyPi](https://docs.openbb.co/sdk/quickstart/installation)|Recommended way if you just want to use the OpenBB SDK| +|[Python](https://docs.openbb.co/sdk/quickstart/installation)|If you wish to contribute to the development of the OpenBB SDK        | ## 3. Contributing diff --git a/openbb_terminal/DOCKER_README.md b/openbb_terminal/DOCKER_README.md deleted file mode 100644 index 178a8c66643..00000000000 --- a/openbb_terminal/DOCKER_README.md +++ /dev/null @@ -1,183 +0,0 @@ -# OpenBBTerminal with Docker - -Here are the steps to get OpenBBTerminal using the Docker containers that we provide: - -1. Installing `Docker` and `Docker Compose` -2. Pulling and running `OpenBBTerminal Docker Container` -3. Configuring your X-server to show plots - -Each of this steps need to be followed to have a working version of OpenBBTerminal. - -We will detail these steps in the rest of this document. - -## 1. Installing `Docker` and `Docker Compose` - -**INSTALL DOCKER** - -Installing Docker Desktop is the easiest way for most people: - -It has a `Graphical User Interface` and comes with `Docke Compose` in it. - -You can find `Docker Desktop` installation file for your `OS` here: - -[Download Docker Desktop](https://www.docker.com/products/docker-desktop) - -**START DOCKER** - -Once you have `Docker` installed and running, you can use the following command to check the state: - -```bash -docker info -``` - -It should output a text like this one: - -```text -Client: - Context: default - Debug Mode: false - -Server: - Containers: 14 - Running: 2 - Paused: 1 - Stopped: 10 -... -``` - -If you see a message like the following, it most likely means you need to start `Docker`. - -Open the docker desktop app in this case. - -```text -Server: -ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. -Is the docker daemon running? -``` - -## 2. Pulling and running `OpenBBTerminal Docker Container` - -**DOCKER COMPOSE** - -Here are the commands to use `Docker Compose` to pull and run the `OpenBBTerminal Docker Container`: - -```bash -curl -o docker-compose.yaml https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml - -docker compose run poetry -``` - -The command line with `curl` is downloading this file : [`docker-compose.yaml`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml). - -The `docker-compose.yaml` file is a configuration file telling `Docker Compose`: - -- where to find `OpenBBTerminal Docker Container` -- how to run this container. - -The second command runs `Docker Compose` on the service `poetry` defined in this `docker-compose.yaml` file. - -This second command must be run in the same folder in which `docker-compose.yaml` file is. - -**DOCKER** - -If you don't have `Docker Compose` you can also use `Docker` directly to run the `OpenBBTerminal Docker Container`. - -Here is the commands to run: - -```bash -docker pull ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z - -docker run -v ~/.openbb_terminal/:/root/.openbb_terminal -v ~/OpenBBUserData:/root/OpenBBUserData -it --rm ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z -``` - -Be sure to replace `X.Y.Z` with the version you want to pull and run. - -Note for windows: - -```text - Ignore this message if you are using Powershell or a more evolved interpreters. - If you are using the builtin Windows interpreter. - Replace `~` by `%USERPROFILE%` in the command above. -``` - -## 3. Configuring your X-server to show plots - -In order to display plots in the docker container, we need to configure the XServer on the host machine. -Without this configuration the interactive charts will not be displayed. - -### On Windows - -Download and install : [VcXsrv](https://sourceforge.net/projects/vcxsrv/) - -When running the program is important to check "Disable access control" - -Run `Docker Compose` like this: - -```bash -curl -o docker-compose.yaml https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml - -docker compose run poetry -``` - -Or run `Docker` directly: - -```bash -docker run -v ~/.openbb_terminal:/root/.openbb_terminal -v ~/OpenBBUserData:/root/OpenBBUserData -it --rm --env DISPLAY=host.docker.internal:0.0 ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z -``` - -### X-Server on macOS - -Users familiar with Docker and X-Server can set the `DISPLAY` variable in the file [setenv](/docker/setenv) described above. If you use this approach remember to add `:0` at the end of your inet address. E.g. `DISPLAY=192.168.1.155:0`. - -For help setting up the X-Server continue reading: - -#### Setting up X Quartz/X11 - -On macOS the X11 client of choice is [XQuartz](https://www.xquartz.org/). On Windows it's [Xming](http://www.straightrunning.com/XmingNotes/). XQuartz will be used as an example further on. - -0. Install X Quartz from -1. With X Quartz open: go to Preferences -> Security and make sure both options are enabled. - ![Screen Shot 2021-09-08 at 12 21 48 PM](https://user-images.githubusercontent.com/18151143/132548605-235d774b-9aa6-4a45-afcf-58fb775d376a.png) - -#### Adding the display for Docker - -From the command prompt or terminal, run the following to add your local configuration to the list of allowed access control: - -```bash -IP=$(ifconfig | grep inet | grep -v -e "127.0.0.1" | awk '$1=="inet" {print $2}') -xhost + $IP -``` - -Now we can run the docker container, adding the display to the environment: - -```bash -docker run -v ~/.openbb_terminal/:/root/.openbb_terminal -v ~/OpenBBUserData:/root/OpenBBUserData -it --rm --env-file=path/to/setenv --env DISPLAY=$IP:0 ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z -``` - -This container will be able to display all the same plots as the terminal interface. - -### X-Server on Linux Desktop - -X-Server is default in Linux distribution. There is no need to install any clients. - -#### Local docker container - -We can use IPC socket to connect Desktop. - -Add this setting to your `.env` file. - -```bash -OPENBB_BACKEND=Qt5Agg -``` - -And run the following commands. - -```bash -xhost +local: -docker run -it --rm --name openbb --env-file=./.env -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z -xhost -local: -``` - -If you're using remote docker host, you can connect with "ssh -X ". - -Then run the previous docker command. diff --git a/openbb_terminal/README.md b/openbb_terminal/README.md deleted file mode 100644 index d20ded93fcd..00000000000 --- a/openbb_terminal/README.md +++ /dev/null @@ -1,336 +0,0 @@ -# OpenBB Terminal - - - - -

-

Table of Contents

 -
    -
  1. Anaconda & Python Installation
    2. -
  2. Docker Installation
    3. -
  3. API Keys
    4. -
-
- ---- - -There are currently four options to install the terminal: - -- [Using the Installer](https://openbb-finance.github.io/OpenBBTerminal/#accessing-the-openbb-terminal) (recommended if you just want to use the terminal) -- [Using Python](#anaconda--python) (recommended if you want to develop new features) -- [Using Docker](#docker-installation) (alternative option to the installer if preferred) - -First step in all options is to star the project - -OpenBB Terminal GitHub Stars - -## Anaconda & Python - -This installation type supports both Windows and Unix systems (Linux + MacOS). - -**NOTE for Windows users:** Some _not all_ Windows users would prefer to use an environment -similar to what Linux and macOS users use. In this case it is easier to use Windows Subsystem -for Linux (WSL). WSL emulates a Linux machine inside your Windows system. If this is the case - -jump to the Installing WSL (Only for Windows users) -section before proceeding. - -### Installing the terminal - -These steps are common in all operating systems (Windows with or without WSL, MacOS or Linux). - -This project supports Python 3.8 and 3.9. By default, the newly created virtual environment will use Python 3.9.13 - -Our current recommendation is to use this project with Anaconda's Python distribution - either full -[**Anaconda3 Latest**](https://www.anaconda.com/products/distribution) or -[**Miniconda3 Latest**](https://docs.conda.io/en/latest/miniconda.html) (recommended). -Several features in this project utilize Machine Learning. Machine Learning Python dependencies are optional. For MacOS systems, the "Miniconda3 MacOSX 64-bit" version that works on both Intel and M1 -macs is recommended. - -**NOTE:** We recommend using `conda` and `poetry` because it just works. You can use other python -distributions and use raw `pip` instead of `poetry` but you will very likely bump into installation -issues. - -#### 1. [Install Miniconda](https://docs.conda.io/en/latest/miniconda.html) - -Miniconda is a python environment and package manager. It is required if you want to -have the dependencies working straight away. - -- Follow the [link to the page with the latest installers for all platforms](https://docs.conda.io/en/latest/miniconda.html#latest-miniconda-installer-links) or click direct links to installer packages based on your operating system: - - If you are using macOS click [Miniconda for MacOS](https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh) - - If you are using WSL or Linux click [Miniconda for Linux](https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh) - - If you are using a Raspberry PI click [Miniconda for Raspberry PI](https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-aarch64.sh) - - If you are using Windows click [Miniconda for Windows](https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe). - - **ONLY REQUIRED ON WINDOWS IF NOT USING WSL**, Install/update Microsoft C++ Build Tools from here: - - **NOTE for macOS users:** The link above gets you the Intel version of miniconda meaning if you're on an - Apple Silicon powered machine you will be using the terminal through Apple's rosetta2 layer. We recommend - sticking to this distribution for better compatibility until the dependency developers fully catch up with - Apple's transition to Apple Silicon. - -- After following the steps, confirm that you have it by opening a terminal and running: `conda -V`. The output should be something along the lines of: `conda 22.9.0` - -#### 2. Install CMake - -CMake is required by several python modules. - -**On Linux or Raspberry Pi:** - -```bash -sudo apt update -sudo apt install -y gcc cmake -``` - -**On macOS:** - -Check if you have homebrew installed by running `brew --version` - -If you don't have homebrew installed run: - -```bash -/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -brew install cmake -``` - -If you have homebrew installed run: - -```bash -brew install cmake -``` - -**On Windows:** - -If you have followed the instructions in step 1 of this guide CMake was installed as a -part of you Microsoft C++ Build Tools - -#### 3. Install git - -```bash -conda install -c anaconda git -``` - -#### 4. Clone the Project - -- Via HTTPS: `git clone https://github.com/OpenBB-finance/OpenBBTerminal.git` -- via SSH: `git clone git@github.com:OpenBB-finance/OpenBBTerminal.git` - -#### 5. Navigate into the project's folder - -```bash -cd OpenBBTerminal/ -``` - -#### 6. Create Environment - -You can name the environment whatever you want. Although you could use names such as: -`welikethestock`, `thisistheway` or `diamondhands`, we recommend something simple and -intuitive like `obb`. This is because this name will be used from now onwards. - -Please note, the following setup has been confirmed to work for all OS (including M1) -with the standard miniconda distribution. If you are using a different distribution, -you will need to install it manually before proceeding. - -```bash -conda env create -n obb --file build/conda/conda-3-9-env.yaml -``` - -Or, to include machine learning type: - -```bash -conda env create -n obb --file build/conda/conda-3-9-env-full.yaml -``` - -Note: Using python 3.10+ can lead to undesirable functionality for certain commands. - -#### 7. Activate the virtual environment - -```bash -conda activate obb -``` - -Note: At the end, you can deactivate it with: `conda deactivate`. - -#### 8. Install dependencies with poetry - -Install the main dependencies with - -```bash -poetry install -``` - -You are good to go with the core of the OpenBB Terminal. To install additional toolkits -proceed with the following commands: - -To install the Portfolio Optimization Toolkit run: - -```bash -poetry install -E optimization -``` - -To install the Machine Learning Toolkit run: - -```bash -poetry install -E prediction -``` - -To install both the Portfolio Optimization and the Machine Learning Toolkit run: - -```bash -poetry install -E all -``` - - -#### 9. You're ready to use the terminal! - -```bash -openbb -``` - -Or if you are old-fashioned run: - -```bash -python terminal.py -``` - -**NOTE:** When you close the terminal and re-open it, the only command you need to re-call -is `conda activate obb` before you call `openbb` again. - -**TROUBLESHOOT:** If you are having troubles to install, check out the -[troubleshoot page](https://github.com/OpenBB-finance/OpenBBTerminal/blob/master/TROUBLESHOOT.md). - -You can also reach for help on our [discord](https://discord.gg/Up2QGbMKHY). - -## Advanced User Install - Custom installation procedures - -By default we advice using `conda` and `poetry` for environment setup and dependency management. -Conda ships binaries for packages like `numpy` so these dependencies are not built from source locally by `pip`. -Poetry solves the dependency tree in a way that the dependencies of dependencies of dependencies -use versions that are compatible with each other. - -If you are using a conda environment the `build/conda` folder contains multiple `.yaml` configuration -files that you can choose from. - -If you are using other python distributions we highly recommend that you use some virtual -environment like `virtualenv` or `pyenv` for installing the terminal dependency libraries. - -Requirements files that you can find in the project root: - -- `requirements.txt` list all the dependencies without Machine Learning libraries -- `requirements-full.txt` list all the dependencies without Machine Learning libraries - -You can install them with with pip - - ```bash - pip install -r requirements.txt - ``` - -The dependency tree is solved by poetry. - -Note: The libraries specified in the requirements files have been tested and work for -the purpose of this project, however, these may be older versions. Hence, it is recommended -for the user to set up a virtual python environment prior to installing these. This allows -to keep dependencies required by different projects in separate places. - -### Installing WSL (Only for Windows users) - -If you are using Windows you first you need to install WSL. The process is simple and a tutorial can be found [here](https://www.sitepoint.com/wsl2/). -Once you reach the section **Update Linux** on that tutorial, you should have a linux machine installed and can proceed -to the next steps. - -Since WSL installation is headless by default (i.e., you have only access to a terminal running a linux distribution) -you need some extra steps to be able to visualize the charts produced by the terminal (more detailed tutorial [here](https://medium.com/@shaoyenyu/make-matplotlib-works-correctly-with-x-server-in-wsl2-9d9928b4e36a)): - -1. Dynamically export the DISPLAY environment variable in WSL2: - - ```bash - # add to the end of ~/.bashrc file - export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):0 - # source the file - source ~/.bashrc - ``` - -2. Download and install [VcXsrv](https://sourceforge.net/projects/vcxsrv/) -3. When running the program is important to check "Disable access control" - -After this, `VcXsrv` should be running successfully and we can proceed to terminal installation. - -## Update Terminal - -The terminal is constantly being updated with new features and bug fixes, hence, for your terminal to be update, -you can run: - -```bash -git pull -``` - -to get the latest changes. - -If this fails due to the fact that you had modified some python files, and there's a conflict with the updates, you can use: - -```bash -git stash -``` - -Then, re-run `poetry install` to get any new dependencies. - -Once installation is finished, you're ready to openbb. - -If you `stashed` your changes previously, you can un-stash them with: - -```bash -git stash pop -``` - -**NOTE:** When you close the terminal and re-open it, the only command you need to re-call is `conda activate gst` -before you call `openbb` again. - -### API Keys - -The project is build around several different API calls, whether it is to access historical data or financials. -The table below shows the ones where a key is necessary. The easiest way is of updating the keys is by using the -terminal, see [this guide](https://openbb-finance.github.io/OpenBBTerminal/terminal/#accessing-other-sources-of-data-via-api-keys). - -You can also use the environment variable to set your API Keys directly instead of using the Terminal, for the variable -name in the code one just needs to remove the "GT\_", this can be found in [config_terminal.py](/openbb_terminal/config_terminal.py). - -| Website | Environment Variables | -| :------ | :-------------------- | -| [Alpha Vantage](https://www.alphavantage.co) | OPENBB_API_KEY_ALPHAVANTAGE | -| [Binance](https://binance.com) | OPENBB_API_BINANCE_KEY
OPENBB_API_BINANCE_SECRET | -| [BitQuery](https://bitquery.io/pricing) | OPENBB_API_BITQUERY_KEY | -| [Coinbase](https://docs.pro.coinbase.com/) | OPENBB_API_COINBASE_KEY
OPENBB_API_COINBASE_SECRET
OPENBB_API_COINBASE_PASS_PHRASE | -| [Coinglass](https://coinglass.github.io/API-Reference/#api-key) | OPENBB_API_COINGLASS_KEY | -| [CoinMarketCap](https://coinmarketcap.com) | OPENBB_API_CMC_KEY
| -| [Cryptopanic](https://cryptopanic.com/developers/api/) | OPENBB_API_CRYPTO_PANIC_KEY | -| [DEGIRO](https://www.degiro.fr) | OPENBB_DG_USERNAME
OPENBB_DG_PASSWORD
OPENBB_DG_TOTP_SECRET | -| [Ethplorer](https://github.com/EverexIO/Ethplorer/wiki/Ethplorer-API) | OPENBB_API_ETHPLORER_KEY | -| [Financial Modeling Prep](https://financialmodelingprep.com) | OPENBB_API_KEY_FINANCIALMODELINGPREP | -| [Finnhub](https://finnhub.io) | OPENBB_API_FINNHUB_KEY | -| [FRED](https://fred.stlouisfed.org) | OPENBB_API_FRED_KEY | -| [GitHub](https://docs.github.com/en/rest/guides/getting-started-with-the-rest-api) | OPENBB_API_GITHUB_KEY | -| [Glassnode](https://docs.glassnode.com/basic-api/api-key#how-to-get-an-api-key) | OPENBB_API_GLASSNODE_KEY | -| [News](https://newsapi.org) | OPENBB_API_NEWS_TOKEN | -| [Oanda](https://developer.oanda.com) | OPENBB_OANDA_TOKEN
OPENBB_OANDA_ACCOUNT | -| [Polygon](https://polygon.io) | OPENBB_API_POLYGON_KEY | -| [Quandl](https://www.quandl.com) | OPENBB_API_KEY_QUANDL | -| [Reddit](https://www.reddit.com) | OPENBB_API_REDDIT_CLIENT_ID
OPENBB_API_REDDIT_CLIENT_SECRET
OPENBB_API_REDDIT_USERNAME
OPENBB_API_REDDIT_USER_AGENT
OPENBB_API_REDDIT_PASSWORD | -| [SentimentInvestor](https://sentimentinvestor.com) | OPENBB_API_SENTIMENTINVESTOR_TOKEN
OPENBB_API_SENTIMENTINVESTOR_KEY | -| [Tradier](https://developer.tradier.com) | OPENBB_TRADIER_TOKEN | -| [Twitter](https://developer.twitter.com) | OPENBB_API_TWITTER_KEY
OPENBB_API_TWITTER_SECRET_KEY
OPENBB_API_TWITTER_BEARER_TOKEN | -| [Whale Alert](https://docs.whale-alert.io/) | OPENBB_API_WHALE_ALERT_KEY | - -Example: - -```bash -export OPENBB_API_REDDIT_USERNAME=SexyYear -``` - -Environment variables can also be set in a `.env` file at the top of the repo. This file is ignored by git so your API -keys will stay secret. The above example stored in `.env` would be: - -```bash -OPENBB_API_REDDIT_USERNAME=SexyYear -``` - -Note that the `OPENBB_API_REDDIT_USER_AGENT` is the name of the script that you set when obtained the Reddit API key. -Note that it is not necessary to have a valid Alpha Vantage key to get daily OHLC values. diff --git a/website/content/sdk/guides/intros/forecast.md b/website/content/sdk/guides/intros/forecast.md index b76cc56f86b..51f6dbf9419 100644 --- a/website/content/sdk/guides/intros/forecast.md +++ b/website/content/sdk/guides/intros/forecast.md @@ -70,7 +70,7 @@ Shows a correlation matrix between columns in dataset ​ ```python df = pd.read_csv(ANDREW_REPLACE) -openbb.forecast.corr_view(df) +openbb.forecast.corr_chart(df) ``` ![Corr View](https://user-images.githubusercontent.com/72827203/202424217-b549b6e7-b121-4273-a7d9-b478e89cd65a.png) @@ -81,7 +81,7 @@ This command allows you to see seasonality patterns in your data ```python df = pd.read_csv(ANDREW_REPLACE) -openbb.forecast.season(df, "Close") +openbb.forecast.season_chart(df) ``` ​ @@ -93,7 +93,7 @@ Predicts the future value of time series data using exponential smoothing ​ ```python df = pd.read_csv(ANDREW_REPLACE) -openbb.forecast.expo(df, "Close") +openbb.forecast.expo_chart(df) ``` ​ Here was can now see a chart and table with the expected values, and historic data. The chart also tells us how our backtesing performed, so that we can know the accuracy of our prediction. diff --git a/website/content/sdk/guides/intros/portfolio.md b/website/content/sdk/guides/intros/portfolio/index.md similarity index 100% rename from website/content/sdk/guides/intros/portfolio.md rename to website/content/sdk/guides/intros/portfolio/index.md diff --git a/website/content/sdk/guides/intros/portfolio/po.md b/website/content/sdk/guides/intros/portfolio/po.md new file mode 100644 index 00000000000..414d91ddd6f --- /dev/null +++ b/website/content/sdk/guides/intros/portfolio/po.md @@ -0,0 +1,212 @@ +--- +title: Portfolio Optimization +keywords: ["portfolio", "optimization", "mean variance", "risk parity", "black litterman", "mean risk", +"hierarchical clustering models"] +excerpt: "The Introduction to Portfolio Optimization within the Portfolio menu explains how to use various portfolio +optimization techniques and provides a brief description of its sub-menus" +geekdocCollapseSection: true +--- +The Portfolio Optimization menu allows the user to apply advanced optimization techniques to a portfolio of any type +and of any size. It does so by introducing a multitude of optimization techniques ranging from mean-variance optimization +to risk parity models and +hierarchical clustering models. +By providing Excel templates, the user can make sense of the vast array of parameters that each command has. E.g. think +of the historic period you wish to use or which of the more than 10 risk measures and covariance methods should be used? +These are questions the templates make easier to answer. + +The capabilities of the [Portfolio Optiminzation menu](/terminal/guides/portfolio/po) from the OpenBB Terminal are wrapped into a powerful SDK, enabling users to work with the data in a flexible environment that can be fully customized to meet the needs of any user. These functionalities allow the user to apply advanced optimization techniques to a portfolio of any type and of any size. It does so by introducing a multitude of optimization techniques ranging from mean-variance optimization to risk parity models and hierarchical clustering models. +By providing Excel templates, the user can make sense of the vast array of parameters that each command has. E.g. think of the historic period you wish to use or which of the more than 10 risk measures and covariance methods should be used? These are questions the templates make easier to answer. + +## How to use +Start a Python script or Notebook file by importing the module: + +```python +from openbb_terminal.sdk import openbb +``` + +This menu requires the usage of the Excel templates to work properly. As there is a lot of complexity involved around +these techniques, these templates allow the user to understand what values for each parameter are actually used and +allow for an easy way to define the allocation. + +> For this there are two templates that need to be set, the **OpenBB Parameters Template** and the **OpenBB Portfolio Template**. These files can be found in the +`OpenBBUserData` folder. Please see this guide for how to find this folder. + +### OpenBB Parameters Template +This template provides the user with the ability to set define values for each parameter based on the optimization +technique that is deployed. E.g. if you select `riskparity` for the `technique` parameter, you will notice that some +parameters turn grey. This means that the parameter is irrelevant for the selected +method. + +The OpenBB Terminal does, however, allow the user to run any model despite the `technique` you selected. Therefore, if +you are interested in running multiple models, consider removing the value for `technique`. Do note that this makes it +more difficult to understand which values are used for which model. + +OpenBB Parameters Template + +Once you have defined the parameters, save the template and load it inside the terminal by using the the following: + +```python +# Define your orderbook path here +order_book_path = "60_40_Portfolio.xlsx" + +# Read in the file +order_book = pd.read_excel(order_book_path) + +# Adjust the columns accordingly +order_book_cols = ['Ticker', 'Asset Class', 'Sector', 'Industry', 'Country', + 'Current Invested Amount', 'Currency'] + +order_book = order_book[order_book_cols] +``` + +### OpenBB Portfolio Template +This template hands the user a format to work with it to define the portfolio. Here, categorization is applied +based on asset class, sector, industry, country and currency. By using the dropdown menus within this Excel, you +are able to apply the proper categorization. This is based on the same methodology as found in other areas of the +terminal. + +OpenBB Portfolio Template + +You can load in the portfolio template with the following code: + +```python +import pandas as pd + +# Define your own orderbook path here, current value won't work +order_book_path = "60_40_Portfolio.xlsx" + +# Read in the file +order_book = pd.read_excel(order_book_path) + +# Adjust the columns accordingly +order_book_cols = ['Ticker', 'Asset Class', 'Sector', 'Industry', 'Country', + 'Current Invested Amount', 'Currency'] + +order_book = order_book[order_book_cols] + +# Load in the portfolio +tickers, categories = openbb.portfolio.po.load(order_book_path) +``` + +### Performing optimization +Based on the parameters and allocation the user has set, the optimization process begins. What optimization +technique is ideal depends entirely on the user's risk profile and objectives. As an illustration, +Risk Parity +is presented below: + +```python +# Perform calculations +weights_riskparity, data_returns_riskparity = openbb.portfolio.po.riskparity(tickers) + +# Convert to a DataFrame +pd.DataFrame.from_dict(weights_riskparity, orient='index', columns=["Risk Parity"]) +``` + +Which returns: + +| | Risk Parity | +|:------|--------------:| +| AAPL | 0.0462 | +| AMZN | 0.04545 | +| APTV | 0.03177 | +| ASML | 0.03328 | +| BABA | 0.04192 | +| GOOGL | 0.04758 | +| HYG | 0.13093 | +| NKE | 0.04909 | +| TIP | 0.30826 | +| TLT | 0.21927 | +| TSM | 0.04626 | + +It is possible to use the commands without loading in the parameters template or by using the parameters template but +changing some arguments directly into the terminal. For example, using the same method as described above, the risk +measure is changed to Conditional Value at Risk (CVaR) +and the used historic period is increased to 10 years (keeping all other parameters unchanged): + +```python +weights_riskparity, data_returns_riskparity = openbb.portfolio.po.riskparity(tickers, interval="10y", risk_measure="CVaR") +``` + +Which returns: + +| | Risk Parity | +|:------|--------------:| +| AAPL | 0.04843 | +| AMZN | 0.04302 | +| APTV | 0.03907 | +| ASML | 0.03717 | +| GOOGL | 0.04877 | +| HYG | 0.14643 | +| NKE | 0.05275 | +| TIP | 0.33264 | +| TLT | 0.20371 | +| TSM | 0.04802 | + + + +## Examples +To demonstrate the capabilities of the Portfolio Optimization menu, the entire S&P 500 index (as of 30th of May 2022) +is used and optimized and analysed in a variety of ways. Starting by loading in the dataset with the following: + +```python +import pandas as pd + +# Define your own orderbook path here, current value won't work +order_book_path = "SP_500_Portfolio.xlsx" + +# Read in the file +order_book = pd.read_excel(order_book_path) + +# Adjust the columns accordingly +order_book_cols = ['Ticker', 'Asset Class', 'Sector', 'Industry', 'Country', + 'Current Invested Amount', 'Currency'] + +order_book = order_book[order_book_cols] + +# Load in the portfolio +tickers, categories = openbb.portfolio.po.load(order_book_path) +``` + +Then, the Hierarchical Risk Parity technique is applied by using the following: + +```python +weights_hrp, data_returns_hrp = openbb.portfolio.po.hrp( + tickers, + interval="5y", + risk_measure="cVaR", + risk_aversion=0.8 +) + +pd.DataFrame.from_dict(weights_hrp, orient='index', columns=["Hierarchical Risk Parity"]) +``` + +This results in the following (the result is edited, as it would show 500 tickers, to prevent flooding this page): + +| | Hierarchical Risk Parity | +|:------|---------------------------:| +| A | 0.00199 | +| AAL | 0.00104 | +| AAP | 0.00185 | +| AAPL | 0.00184 | +| ABBV | 0.0028 | +| ... | ... | +| NOC | 0.00228 | +| XOM | 0.00167 | +| ZBRA | 0.00156 | +| ZION | 0.0014 | +| ZTS | 0.00225 | + +It is possible to delve further into these findings with the `plot` functionality for example done by looking at the portfolio's returns histogram +which also includes a variety of risk measures as well as the portfolio's drawdowns. + +```` +openbb.portfolio.po.plot(data=data_returns_hrp, weights=weights_hrp, risk_measure='cVaR', hist=True) +```` + +Portfokio Returns Histogram + +```` +openbb.portfolio.po.plot(data=data_returns_hrp, weights=weights_hrp, risk_measure='cVaR', dd=True) +```` + +Portfolio Drawdowns \ No newline at end of file diff --git a/website/content/sdk/guides/intros/stocks/behavioral-analysis.md b/website/content/sdk/guides/intros/stocks/behavioral-analysis.md index 8de5c897b0f..beb5aa2dc59 100644 --- a/website/content/sdk/guides/intros/stocks/behavioral-analysis.md +++ b/website/content/sdk/guides/intros/stocks/behavioral-analysis.md @@ -4,5 +4,140 @@ keywords: [ "stocks", "stock", "options", "option", "call", "put" ] -excerpt: "This guide introduces the Stocks menu in the context of the OpenBB SDK." +excerpt: "This guide introduces the BA menu in the context of the OpenBB SDK." --- + +The BA module provides programmatic access to the commands from within the OpenBB Terminal. Import the OpenBB SDK module, and then access the functions similarly to how the Terminal menus are navigated. The code completion will be activated upon entering `.`, after, `openbb.ba` +​ + +## How to Use + +​ +The examples provided below will assume that the following import block is included at the beginning of the Python script or Notebook file: +​ + +```python +from openbb_terminal.sdk import openbb +``` + +​ +A brief description below highlights the main Functions and Modules available in the BA SDK + +| Path | Type | Description | +| :------------------------- | :--------: | -------------------------------------------: | +| openbb.ba.bullbear | Function | Estimate Quick Sentiment | +| openbb.ba.headlines | Function | Sentiment from 15+ News Headlines | +| openbb.ba.spacc | Function | Shows SPAC announcements from SPAC subreddit | +| openbb.ba.watchlist | Function | Show Other User's Watchlists | +| openbb.ba.messages | Function | Output last 30 messages on the board | +| openbb.ba.mentions | Function | Interest Over Time Based on Mentions | +| openbb.ba.hist | Function | Plot Historical RHI and AHI | +| openbb.ba.snews | Function | Stock Price Plotted Over Sentiment | +| openbb.ba.redditsent | Function | Search for Tickers and Find Sentiment | +| openbb.ba.popular | Function | Show Popular Tickers | +| openbb.ba.spac | Function | Shows Other User's SPAC Announcements | +| openbb.ba.getdd | Function | Get Due Diligence | +| openbb.ba.regions | Function | Regions Showing the Highest Interest | +| openbb.ba.trending | Function | Trending Stocks | +| openbb.ba.wsb | Function | Highlights from WSB Subreddit | +| openbb.ba.queries | Function | Top Related Queries | +| openbb.ba.sentiment | Function | Stock Sentiment Prediction from Last Tweets | +| openbb.ba.infer | Function | Stock Sentiment from Last Tweets | +| openbb.ba.stalker | Function | Stocktwit's last messages | +| openbb.ba.rise | Function | Top Rising Related Queries | + +Alteratively you can print the contents of the BA SDK with: + +```python +help(openbb.ba) +``` + +## Examples + +### headlines + +Sentiment from over 15 news headlines for each day. This data can be used for further analysis of the stock + +```python +openbb.stocks.ba.headlines("TSLA") +``` + +| date | Sentiment Analysis | +|:--------------------|---------------------:| +| 2022-11-09 00:00:00 | 0.209 | +| 2022-11-10 00:00:00 | -0.025 | +| 2022-11-11 00:00:00 | 0.027 | +| 2022-11-14 00:00:00 | -0.402 | +| 2022-11-15 00:00:00 | -0.122 | +| 2022-11-16 00:00:00 | -0.35 | +| 2022-11-17 00:00:00 | 0.156 | +| 2022-11-18 00:00:00 | 0.216 | +| 2022-11-21 00:00:00 | -0.071 | +| 2022-11-22 00:00:00 | -0.003 | + +### regions + +See the top regions where a stock is searched + +```python +openbb.stocks.ba.regions("TSLA") +``` + +| geoName | TSLA | +|:---------------------------------------|-------:| +| Canada | 100 | +| United States | 95 | +| Singapore | 94 | +| Estonia | 51 | +| Taiwan | 50 | +| Slovenia | 40 | +| Switzerland | 36 | +| Denmark | 36 | +| New Zealand | 33 | +| Sweden | 31 | +| Finland | 24 | +| Australia | 24 | +| Belgium | 22 | +| Czechia | 20 | +| Bulgaria | 18 | +| United Arab Emirates | 17 | +| United Kingdom | 15 | +| Portugal | 12 | +| Austria | 11 | +| Romania | 10 | +| South Africa | 7 | +| Saudi Arabia | 7 | +| Germany | 7 | +| Argentina | 6 | +| Spain | 6 | +| France | 4 | +| Colombia | 4 | +| Philippines | 4 | +| Brazil | 4 | +| Thailand | 4 | +| Poland | 3 | +| Vietnam | 2 | +| Russia | 2 | +| Turkey | 2 | +| Pakistan | 0 | + +### rise + +See which search queries are popular right now + +```python +openbb.stocks.ba.rise("AAPL") +``` + +| | query | value | +|---:|:----------------|--------:| +| 0 | nio | 220300 | +| 1 | nio stock | 106650 | +| 2 | pltr stock | 64750 | +| 3 | mrna stock | 59750 | +| 4 | nio stock price | 53450 | +| 5 | zm stock | 49500 | +| 6 | bynd | 44850 | +| 7 | pltr | 41000 | +| 8 | spce | 37350 | +| 9 | zm | 31650 | \ No newline at end of file diff --git a/website/content/sdk/guides/intros/stocks/discovery.md b/website/content/sdk/guides/intros/stocks/discovery.md new file mode 100644 index 00000000000..557ea71e979 --- /dev/null +++ b/website/content/sdk/guides/intros/stocks/discovery.md @@ -0,0 +1,148 @@ +--- +title: Discovery +keywords: + [ + "stocks", "stock", "options", "option", "comparison", "analysis", "tickers", "stocks", "insight" + ] +excerpt: "This guide introduces the Disc SDK in the context of the OpenBB SDK." +--- + +The Disc module provides programmatic access to the commands from within the OpenBB Terminal. Import the OpenBB SDK module, and then access the functions similarly to how the Terminal menus are navigated. The code completion will be activated upon entering `.`, after, `openbb.disc` +​ + +## How to Use + +​ +The examples provided below will assume that the following import block is included at the beginning of the Python script or Notebook file: +​ + +```python +from openbb_terminal.sdk import openbb +``` + +​ +A brief description below highlights the main Functions and Modules available in the ETF SDK + +| Path | Type | Description | +| :------------------------- | :--------: | -------------------------------------------: | +| openbb.disc.pipo | Function | Past IPO Dates | +| openbb.disc.active | Function | Stocks with Highest Trade Volumes | +| openbb.disc.gainers | Function | Latest Top Gainers | +| openbb.disc.asc | Function | Small Caps with Earnings Growth Above 25% | +| openbb.disc.lowfloat | Function | Stocks with Under 10M Shares Float | +| openbb.disc.fipo | Function | Future IPO Dates | +| openbb.disc.upcoming | Function | Upcoming Earnings Release Dates | +| openbb.disc.trending | Function | Trending News | +| openbb.disc.ulc | Function | Potentially Undervalued Large Cap Stocks | +| openbb.disc.arkord | Function | Orders of ARK Investment Management LLC | +| openbb.disc.hotpenny | Function | Hot Penny Stocks | +| openbb.disc.gtech | Function | Tech Stocks with Earnings Growth Over 25% | +| openbb.disc.losers | Function | Show Top Losers | +| openbb.disc.ugs | Function | Undervalueds with Earnings Growth Over 25% | +| openbb.disc.rtat | Function | Top 10 Retail Traded Stocks | +| openbb.disc.dividends | Function | Screener Overview | + +Alteratively you can print the contents of the Disc SDK with: + +```python +help(openbb.disc) +``` + +## Examples + +### pipo + +Gives information on companies that recently had an IPO (Initial Public Offering) + +```python +openbb.stocks.disc.pipo() +``` + +| | Past | Exchange | Name | Number of Shares | Price | Status | symbol | Total Shares Value | +|---:|:-----------|:---------------|:-------------------------------|:-------------------|:----------|:----------|:---------|:---------------------| +| 0 | 2022-11-22 | NASDAQ Capital | Adamas One Corp. | 7165904.0 | 4.50-5.00 | expected | JEWL | 38192020.0 | +| 1 | 2022-11-21 | | Solta Medical Corp | | | withdrawn | | | +| 2 | 2022-11-18 | | MGO Global Inc. | | | filed | MGOL | 8625000.0 | +| 3 | 2022-11-18 | | CytoMed Therapeutics Pte. Ltd. | | | filed | GDTC | 15000000.0 | +| 4 | 2022-11-18 | | BKV Corp | | | filed | BKV | 100000000.0 | +| 5 | 2022-11-18 | | Coya Therapeutics, Inc. | | | filed | COYA | 17250000.0 | + +### gainers + +Shows stocks with the biggest percent change in daily growth + +```python +openbb.stocks.disc.gainers() +``` + +| | Symbol | Name | Price (Intraday) | Change | % Change | Volume | Avg Vol (3 month) | Market Cap | PE Ratio (TTM) | +|---:|:---------|:------------------------------------------------|-------------------:|---------:|:-----------|:---------|:--------------------|:-------------|:-----------------| +| 0 | MPCFF | Metro Pacific Investments Corporation | 0.1021 | 0.0273 | +36.50% | 20000 | 24674 | 2.963B | 10.21 | +| 1 | JBARF | Julius Bär Gruppe AG | 56.66 | 11.5 | +25.47% | 47922 | 276 | 14.48B | 12.26 | +| 2 | BURL | Burlington Stores, Inc. | 189.96 | 32.32 | +20.50% | 5.369M | 1.431M | 12.451B | 75.38 | +| 3 | PSNY | Polestar Automotive Holding UK PLC | 7.82 | 1.33 | +20.49% | 8.878M | 2.611M | 16.495B | 4.25 | +| 4 | AEO | American Eagle Outfitters, Inc. | 15.36 | 2.36 | +18.15% | 18.426M | 6.448M | 2.877B | 16.52 | +| 5 | MMS | Maximus, Inc. | 70.34 | 9.8 | +16.19% | 1.016M | 380221 | 4.259B | 23.45 | +| 6 | MRTX | Mirati Therapeutics, Inc. | 86.11 | 11.57 | +15.52% | 5.41M | 857355 | 4.959B | | +| 7 | WMG | Warner Music Group Corp. | 31.09 | 4.11 | +15.23% | 3.496M | 1.209M | 16.007B | 37.91 | +| 8 | TIAOF | Telecom Italia S.p.A. | 0.2477 | 0.0317 | +14.68% | 20000 | 37835 | 5.578B | | +| 9 | MANU | Manchester United plc | 14.94 | 1.91 | +14.66% | 5.253M | 500920 | 2.436B | | +| 10 | BBY | Best Buy Co., Inc. | 79.88 | 9.05 | +12.78% | 17.487M | 3.047M | 17.983B | 10.69 | +| 11 | SGIOY | Shionogi & Co., Ltd. | 13.8 | 1.55 | +12.65% | 217546 | 253257 | 16.545B | 20.0 | +| 12 | DKS | DICK'S Sporting Goods, Inc. | 117.76 | 10.82 | +10.12% | 6.403M | 1.735M | 9.326B | 10.14 | +| 13 | BCDRF | Banco Santander, S.A. | 2.85 | 0.25 | +9.62% | 1.001M | 230746 | 49.706B | 5.09 | +| 14 | EE | Excelerate Energy, Inc. | 30 | 2.63 | +9.61% | 842088 | 466600 | 3.248B | 16.39 | +| 15 | URBN | Urban Outfitters, Inc. | 27.8 | 2.27 | +8.89% | 7.168M | 2.121M | 2.562B | 11.26 | +| 16 | WHITF | Whitehaven Coal Limited | 6.2 | 0.5 | +8.77% | 39639 | 62464 | 5.762B | 4.37 | +| 17 | LINRF | Liontown Resources Limited | 1.5 | 0.12 | +8.70% | 36150 | 4637 | 3.295B | 150.0 | +| 18 | MBLY | Mobileye Global Inc. | 29.55 | 2.33 | +8.56% | 1.327M | 1.848M | 23.837B | | +| 19 | A | Agilent Technologies, Inc. | 156.86 | 11.72 | +8.07% | 3.468M | 1.406M | 46.437B | 35.81 | +| 20 | FUTU | Futu Holdings Limited | 54.39 | 3.99 | +7.92% | 4.094M | 2.489M | 7.995B | 27.61 | +| 21 | SYM | Symbotic Inc. | 10.77 | 0.79 | +7.92% | 442139 | 215487 | 5.983B | | +| 22 | KAEPF | The Kansai Electric Power Company, Incorporated | 8.36 | 0.6 | +7.73% | 21794 | 5106 | 7.93B | 17.79 | +| 23 | HL | Hecla Mining Company | 5.17 | 0.37 | +7.71% | 11.242M | 8.056M | 3.134B | 172.33 | +| 24 | RKLIF | Rentokil Initial plc | 6.52 | 0.47 | +7.68% | 29704 | 3911 | 16.434B | 36.23 | + +​ + +### rtat + +Shows information on how retail investors are trading a particular ticker + +```python +openbb.stocks.disc.rtat() +``` + +| | Date | Ticker | Activity | Sentiment | +|-----:|:-----------|:---------|-----------:|------------:| +| 0 | 2022-11-22 | TSLA | 0.0681 | 1 | +| 1 | 2022-11-22 | TQQQ | 0.0159 | -2 | +| 2 | 2022-11-22 | SQQQ | 0.0219 | 0 | +| 3 | 2022-11-22 | SPY | 0.0387 | 1 | +| 4 | 2022-11-22 | QQQ | 0.019 | -1 | +| 5 | 2022-11-22 | NVDA | 0.0427 | -3 | +| 6 | 2022-11-22 | META | 0.0137 | -1 | +| 7 | 2022-11-22 | AMZN | 0.0333 | -1 | +| 8 | 2022-11-22 | AMD | 0.0317 | -3 | +| 9 | 2022-11-22 | AAPL | 0.0307 | -5 | +| 10 | 2022-11-21 | TSLA | 0.0763 | 0 | +| 11 | 2022-11-21 | TQQQ | 0.0169 | 1 | +| 12 | 2022-11-21 | SQQQ | 0.0249 | -3 | +| 13 | 2022-11-21 | SPY | 0.0354 | 2 | +| 14 | 2022-11-21 | QQQ | 0.0203 | 2 | +| 15 | 2022-11-21 | NVDA | 0.0312 | -3 | +| 16 | 2022-11-21 | DIS | 0.0367 | 1 | +| 17 | 2022-11-21 | AMZN | 0.035 | 0 | +| 18 | 2022-11-21 | AMD | 0.0275 | -1 | +| 19 | 2022-11-21 | AAPL | 0.0322 | -3 | +| 20 | 2022-11-18 | TSLA | 0.0621 | -1 | +| 21 | 2022-11-18 | TQQQ | 0.0246 | -1 | +| 22 | 2022-11-18 | SQQQ | 0.0274 | -1 | +| 23 | 2022-11-18 | SPY | 0.0456 | 1 | +| 24 | 2022-11-18 | QQQ | 0.0235 | -3 | +| 25 | 2022-11-18 | NVDA | 0.0376 | -5 | +| 26 | 2022-11-18 | META | 0.0141 | 3 | +| 27 | 2022-11-18 | AMZN | 0.0338 | 0 | +| 28 | 2022-11-18 | AMD | 0.0379 | 0 | +| 29 | 2022-11-18 | AAPL | 0.0403 | -3 | +| 30 | 2022-11-17 | TSLA | 0.0563 | -1 | \ No newline at end of file diff --git a/website/content/sdk/guides/intros/stocks/fundamental-analysis.md b/website/content/sdk/guides/intros/stocks/fundamental-analysis.md index e463f5226a4..f7c6559ed3b 100644 --- a/website/content/sdk/guides/intros/stocks/fundamental-analysis.md +++ b/website/content/sdk/guides/intros/stocks/fundamental-analysis.md @@ -2,7 +2,130 @@ title: Fundamental Analysis keywords: [ - "stocks, stock, options, option, call, put, earnings, calendar, how-to, guide, scripts, fundamental, analysis, technical, behavioural, analyst, equity, research, api, sdk, application, python, notebook, jupyter", + "stocks", "stock", "options", "option", "call", "put", "earnings", "calendar", "how-to", "guide", "scripts", "fundamental", "analysis", "technical", "behavioural", "analyst", "equity", "research", "api", "sdk", "application", "python", "notebook", "jupyter", ] -excerpt: "This guide introduces the Stocks menu in the context of the OpenBB SDK." +excerpt: "This guide introduces the FA menu in the context of the OpenBB SDK." --- + +The FA module provides programmatic access to the commands from within the OpenBB Terminal. Import the OpenBB SDK module, and then access the functions similarly to how the Terminal menus are navigated. The code completion will be activated upon entering `.`, after, `openbb.fa` +​ + +## How to Use + +​ +The examples provided below will assume that the following import block is included at the beginning of the Python script or Notebook file: +​ + +```python +from openbb_terminal.sdk import openbb +``` + +​ +A brief description below highlights the main Functions and Modules available in the FA SDK + +| Path | Type | Description | +| :------------------------- | :--------: | -------------------------------------------: | +| openbb.fa.mgmt | Function | Company Management Team | +| openbb.fa.score | Function | Investing Score from Warren Buffet | +| openbb.fa.sust | Function | Sustianability Values | +| openbb.fa.divs | Function | Historical Dividends for a Company | +| openbb.fa.shrs | Function | Shareholders (Insiders, Institutions, Funds) | +| openbb.fa.earnings | Function | Earnings Data and EPS | +| openbb.fa.info | Function | Information About a Company | +| openbb.fa.dcf | Function | Shows DCF Values for a Ticker | +| openbb.fa.overview | Function | Overview of the Company | +| openbb.fa.data | Function | Fundamental and Technical Data | +| openbb.fa.ratios | Function | In-Depth Ratios over Time | +| openbb.fa.growth | Function | Growth of Financial Statement Items | +| openbb.fa.enterprise | Function | Company Enterprise Value | +| openbb.fa.analysis | Function | Analysis SEC Fillings with Machine Learning | +| openbb.fa.balance | Function | Company Balance Sheet | +| openbb.fa.dupont | Function | Detailed ROE Breakdown | +| openbb.fa.profile | Function | Company Profile | +| openbb.fa.fraud | Function | Key Fraud Ratios | +| openbb.fa.income | Function | Company Income Statement | +| openbb.fa.key | Function | Company Key Metrics | +| openbb.fa.splits | Function | Stock Splits and Reverse Splits Since IPO | +| openbb.fa.hq | Function | HQ Location for Company | +| openbb.fa.cal | Function | Calendar Earnings and Estimates | +| openbb.fa.mktcap | Function | Estimated Market Cap | +| openbb.fa.metrics | Function | Key Metrics Over Time | +| openbb.fa.cash | Function | Company Cash Flows | + +Alteratively you can print the contents of the FA SDK with: + +```python +help(openbb.fa) +``` + +## Examples + +### shrs + +The shrs command give a breakdown of the top shareholders for a given ticker + +```python +openbb.stocks.fa.shrs("TSLA") +``` + +| | Holder | Shares | Date Reported | Stake | Value | +|---:|:------------------------------|:----------|:--------------------|:--------|:---------| +| 0 | Vanguard Group, Inc. (The) | 213.025 M | 2022-09-29 00:00:00 | 6.75 % | 36.195 B | +| 1 | Blackrock Inc. | 171.861 M | 2022-09-29 00:00:00 | 5.44 % | 29.201 B | +| 2 | State Street Corporation | 99.647 M | 2022-09-29 00:00:00 | 3.16 % | 16.931 B | +| 3 | Capital World Investors | 90.162 M | 2022-09-29 00:00:00 | 2.86 % | 15.319 B | +| 4 | Geode Capital Management, LLC | 47.496 M | 2022-09-29 00:00:00 | 1.50 % | 8.070 B | +| 5 | Price (T.Rowe) Associates Inc | 46.957 M | 2022-09-29 00:00:00 | 1.49 % | 7.978 B | +| 6 | FMR, LLC | 38.429 M | 2022-09-29 00:00:00 | 1.22 % | 6.529 B | +| 7 | Jennison Associates LLC | 29.558 M | 2022-09-29 00:00:00 | 0.94 % | 5.022 B | +| 8 | Baillie Gifford and Company | 27.877 M | 2022-09-29 00:00:00 | 0.88 % | 4.737 B | +| 9 | Northern Trust Corporation | 26.602 M | 2022-09-29 00:00:00 | 0.84 % | 4.520 B | + +### enterprise + +Creates a table showing enterprise value over time for a ticker + +```python +openbb.stocks.fa.enterprise("TSLA") +``` + +| | 2021 | 2020 | 2019 | 2018 | 2017 | +|:--------------------------------|:----------|:----------|:----------|:---------|:---------| +| Symbol | TSLA | TSLA | TSLA | TSLA | TSLA | +| Stock price | 282.117 | 264.510 | 42.721 | 20.585 | 23.055 | +| Number of shares | 2.958 B | 2.880 B | 2.655 B | 2.558 B | 2.486 B | +| Market capitalization | 834.501 B | 761.789 B | 113.423 B | 52.653 B | 57.322 B | +| Minus cash and cash equivalents | 17.576 B | 19.384 B | 6.268 B | 3.686 B | 3.368 B | +| Add total debt | 6.834 B | 11.688 B | 13.704 B | 12.115 B | 10.382 B | +| Enterprise value | 823.759 B | 754.093 B | 120.859 B | 61.082 B | 64.337 B | + +​ + +### income + +Income statements over time for a ticker + +```python +openbb.stocks.ca.income(["tsla", "aapl"]) +``` + +| Breakdown | ttm | 2021-12-31 | 2020-12-31 | 2019-12-31 | +|:--------------------------------------------|--------------:|----------------:|-----------------:|-------------:| +| Total revenue | 2.5508e+09 | 1.60937e+09 | 8.31576e+08 | 3.61384e+08 | +| Cost of revenue | 2.16586e+09 | 1.4104e+09 | 6.66345e+08 | 2.84672e+08 | +| Gross profit | 3.84936e+08 | 1.98969e+08 | 1.65231e+08 | 7.6712e+07 | +| Selling general and administrative | 4.07964e+08 | 2.52133e+08 | 1.03962e+08 | 5.9148e+07 | +| Total operating expenses | 4.90874e+08 | 3.01574e+08 | 1.22461e+08 | 6.597e+07 | +| Operating income or loss | -1.05938e+08 | -1.02605e+08 | 4.277e+07 | 1.0742e+07 | +| Interest expense | 5.7796e+07 | 5.1291e+07 | 3.4002e+07 | 1.0163e+07 | +| Total other income/expenses net | 3.8987e+07 | 3.7169e+07 | -7.9501e+07 | -2.0678e+07 | +| Income before tax | -1.2474e+08 | -1.16723e+08 | -7.0413e+07 | -1.978e+07 | +| Income tax expense | 1.417e+06 | 14000 | 651000 | 0 | +| Income from continuing operations | -1.26157e+08 | -1.16737e+08 | -7.1064e+07 | -1.978e+07 | +| Net income | -5.824e+07 | -1.802e+07 | -7.1064e+07 | -1.978e+07 | +| Net income available to common shareholders | -5.824e+07 | -1.802e+07 | -7.1064e+07 | -1.978e+07 | +| Basic EPS | nan | -0.11 | -0.43 | -0.12 | +| Diluted EPS | nan | -0.28 | -0.43 | -0.12 | +| Basic average shares | nan | 1.70507e+08 | 1.66243e+08 | 1.66243e+08 | +| Diluted average shares | nan | 4.75697e+08 | 1.66243e+08 | 1.66243e+08 | +| EBITDA | nan | -1.5991e+07 | -1.7912e+07 | -2.795e+06 | diff --git a/website/content/sdk/quickstart/faq.md b/website/content/sdk/quickstart/faq.md index baeb8761db0..940ecb0a1b4 100644 --- a/website/content/sdk/quickstart/faq.md +++ b/website/content/sdk/quickstart/faq.md @@ -5,19 +5,19 @@ sidebar_position: 3 # FAQ -
Where does the data comes from? +
Where does the data from the OpenBB SDK come from?

-OpenBB does not provide any data. OpenBB is a data aggregator that connects to -multiple data providers and provides a unified API to access the data. +The OpenBB platform does not provide any data itself as the platform is a data aggregator that connects to +almost a hundred different data providers and APIs to access the data.

-
What programs do I need to allow for Windows Firewall? +
What programs need to be allowed for Windows Firewall?

-Windows Firewall will need to allow the following applications through (if not already): +When issues arise regarding Windows Firewall, please allow the following applications through (if not already): - BranchCache - Hyper-V @@ -29,11 +29,11 @@ From the Windows Security menu, click on the Firewall & Network Protection tab,

-
In what Operating Systems can I run OpenBB SDK? +
On what Operating Systems can I run OpenBB SDK?

-OpenBB SDK is compatible with Windows, Linux, and Mac OS. Check the -[installation guide](/sdk/getstarted/installation) and [requirements outline](/sdk/getstarted/requirements) for more details. +The OpenBB Terminal is compatible with Windows, Mac Os and Linux. Check the +[installation guide](/sdk/quickstart/installation) and [requirements outline](/sdk/quickstart/requirements) for more details.

diff --git a/website/content/sdk/quickstart/requirements.md b/website/content/sdk/quickstart/requirements.md index e2e5c474652..b2beadb8fa1 100644 --- a/website/content/sdk/quickstart/requirements.md +++ b/website/content/sdk/quickstart/requirements.md @@ -6,7 +6,7 @@ sidebar_position: 1 import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -The OpenBB SDK is a set of tools for financial and data analysis. We recommend starting out with some experience in finance, markets, and Python. The purpose of this page is to outline some of the background information required to get the most out of the software. +The OpenBB SDK is a set of tools for financial and data analysis. The purpose of this page is to outline some of the background information required to get the most out of the software. ### Minimum System Requirements @@ -69,23 +69,6 @@ Virtual Python environments are containers for Python applications, and allows t It is not recommended to install in a Global environment, such as Homebrew or `usr/local/bin/python3`. Create and activate a new environment before installing the OpenBB SDK. -### Common Python Libraries - -A good way to learn about the Python language is to recognize the libraries which are frequently used and attempt to understand how they form the foundations for workflows. A working knowledge of the libraries listed below is recommended: - -- Datetime -- JSON -- Math -- Matplotlib -- Numpy -- Pandas -- Plotly -- Quandl -- Requests -- URLlib3 - -The best way to learn is to get involved. Tinkering with the code from an open-source project is an excellent way to dabble. - ### GitHub [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal) is where the OpenBB source code is maintained. Cloning the [repository](https://github.com/OpenBB-finance/OpenBBTerminal.git) is recommended for those wishing to develop functions, inspect the source code, or be on the bleeding-edge of development. Syncing the local folder - `git pull` - to the main branch of the repository will keep the installation up-to-date, and `git checkout` provides a way to test development or contributor branches. GitHub can also be used as a personal storage vault and take advantage of automated actions. It's a great place to build, store, and maintain small databases for free. @@ -118,4 +101,4 @@ Then the view: openbb.economy.ycrv_chart() ``` -![Screenshot 2022-11-21 at 4 29 17 PM](https://user-images.githubusercontent.com/85772166/203185342-f019414d-24e2-4d8a-a718-10eeedb59e8c.png) +![Screenshot 2022-11-21 at 4 29 17 PM](https://user-images.githubusercontent.com/85772166/203185342-f019414d-24e2-4d8a-a718-10eeedb59e8c.png) \ No newline at end of file diff --git a/website/content/terminal/guides/advanced/_category_.json b/website/content/terminal/guides/advanced/_category_.json new file mode 100644 index 00000000000..dd1c52f2ee6 --- /dev/null +++ b/website/content/terminal/guides/advanced/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Advanced", + "position": 3 +} diff --git a/website/content/terminal/guides/advanced/changing_sources.md b/website/content/terminal/guides/advanced/changing_sources.md new file mode 100644 index 00000000000..fc45e81cac5 --- /dev/null +++ b/website/content/terminal/guides/advanced/changing_sources.md @@ -0,0 +1,46 @@ +--- +sidebar_position: 1 +title: Changing Sources +--- + +Many commands have multiple datasources attached to it. A great example is `/stocka/fa/income` that allows you to select `YahooFinance, Polygon, AlphaVantage, FinancialModelingPrep or EODHD`. Many have a default source, for example for `income` this is Yahoo Finance but you can change the default sources of each command via the `sources` menu. + +``` +2022 Nov 23, 06:09 (🦋) / $ sources + +╭─────────────────────────────────────────────────────────────────────────────────────────────────── Data Sources ───────────────────────────────────────────────────────────────────────────────────────────────────╮ +│ │ +│ Get and set default data sources: │ +│ get get available data sources associated with command │ +│ set set default data source for a command │ +│ │ +╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── OpenBB Terminal v2.0.0rc1 (https://openbb.co) ─╯ +``` + +For example, if you would like to change the default data provider from the `income` command from the `stocks/fa` menu you can first run the command `get` following by `stocks_load`. This returns the following: + +``` +2022 Nov 23, 06:12 (🦋) /sources/ $ get stocks_fa_income + + +Default : YahooFinance +Available : YahooFinance, Polygon, AlphaVantage, FinancialModelingPrep, EODHD + +``` + +Then, with `set` you can change the default data provider. For example, we can change the data provider to `Polygon` with +the following: + +``` +2022 Nov 23, 06:12 (🦋) /sources/ $ set stocks_fa_income Polygon + +The data source was specified successfully. + + +2022 Nov 23, 06:12 (🦋) /sources/ $ get stocks_fa_income + + +Default : Polygon +Available : Polygon, YahooFinance, AlphaVantage, FinancialModelingPrep, EODHD + +``` \ No newline at end of file diff --git a/website/content/terminal/guides/advanced/customizing_the_terminal.md b/website/content/terminal/guides/advanced/customizing_the_terminal.md new file mode 100644 index 00000000000..3d44b39ef01 --- /dev/null +++ b/website/content/terminal/guides/advanced/customizing_the_terminal.md @@ -0,0 +1,41 @@ +--- +sidebar_position: 2 +title: Customizing the Terminal +--- + +To adjust the lay-out and settings of the OpenBB Terminal you can access the `settings` menu. This menu allows you to +tweak how the terminal behaves. This includes the following: + +- `dt` adds or removes the datetime from the flair (which is next to the emoji). +- `flair` allows you to change the emoji that is used. +- `lang` gives the ability to change the terminal language. At this moment, the terminal is only available in English. +- `export` defines the folder you wish to export data you acquire from the terminal. Use quotes for custom locations. +- `tz` allows you to change the timezone if this is incorrectly displayed for you. +- `autoscaling` automatically scales plots for you if enabled (when green). +- `pheight` sets the percentage height of the plot (graphs) displayed (if autoscaling is enabled). +- `pwidth` sets the percentage width of the plot (graphs) displayed (if autoscaling is enabled). +- `height` sets the height of the plot (graphs) displayed (if autoscaling is disabled). +- `width` sets the width of the plot (graphs) displayed (if autoscaling is disabled). +- `dpi` refers to the resolution that is used for the plot (graphs) +- `backend` allows you to change the backend that is used for the graphs +- `monitor` choose which monitors to scale the plot (graphs) to if applicable +- `source` allows you to select a different source file in which the default data sources are written down + +Next to that, to enable or disable certain functionalities of the terminal you can use the `featflags` menu which +includes the following: + +- `retryload` whenever you misspell commands, try to use the `load` command with it first (default is no). +- `tab` whether to use tabulate to print DataFrames, to prettify these DataFrames (default is yes). +- `cls` whether to clear the command window after each command (default is no). +- `color` whether to use colors within the terminal (default is yes). +- `promptkit` whether you wish to enable autocomplete and history (default is yes). +- `thoughts` whether to receive a thought of the day (default is no). +- `reporthtml` whether to open reports as HTML instead of Jupyter Notebooks (default is yes). +- `exithelp` whether to automatically print the help message when you use `q` (default is yes). +- `rcontext` whether to remember loaded tickers and similar while switching menus (default is yes). +- `rich` whether to apply a colorful rich terminal (default is yes). +- `richpanel` whether to apply a colorful rich terminal panel (default is yes). +- `ion` whether to enable interactive mode of MATPLOTLIB (default is yes). +- `watermark` whether to include the watermark of OpenBB Terminal in figures (default is yes). +- `cmdloc` whether the location of the command is displayed in figures (default is yes). +- `tbhint` whether usage hints are displayed in the bottom toolbar (default is yes). \ No newline at end of file diff --git a/website/content/terminal/guides/advanced/scripts_and_routines.md b/website/content/terminal/guides/advanced/scripts_and_routines.md new file mode 100644 index 00000000000..2cf19f1d280 --- /dev/null +++ b/website/content/terminal/guides/advanced/scripts_and_routines.md @@ -0,0 +1,110 @@ +--- +sidebar_position: 3 +title: Scripts & Routines +--- +The `.openbb` scripts offer the ability to automatically run a set of commands in the form of a **routine**. Furthermore, the scripts can be adapted, and documented, at any moment giving the user full control over the type of analysis you wish to do (and repeat). This can fundamental research, understanding market movements, finding hidden gems and even doing advanced statistical/econometric research. + +### Explanation of scripts + +The script file below is titled _stocks_demo.openbb_. This file follows the following logic: + +- Comments: any text after a `#` is referred to as a comment. This is used to explain what is happening within the script and is not taking into account when running terminal commands. +- Commands: any text **without** a `#` is being ran inside the OpenBB Terminal. E.g. on the second line it says `stocks` thus within the OpenBB Terminal the script will enter `stocks` and run this for you. + +These scripts have a 1-to-1 relationship with how you would normally use the terminal. To get a better understanding of how the terminal is used, please see Structure of the OpenBB Terminal. + +``` +# Go into the stocks context +stocks + +# Load a company ticker, e.g. Amazon +load AMZN + +# Show a simple plot of the ticker, adding in any basic indicator. +candle --ma 20,30 + +# Switch over to the due diligence menu +dd + +# Show the price targets of Amazon, both the graph as well as the table +pt +pt --raw + +# Show estimates of annual and quarterly earnings as well as quarterly revenue estimates +est + +# Open the comparison analysis menu (ca) +../ca + +# Pick stocks that are competitors to Amazon +add ebay,wmt,tgt,baba,jd + +# Determine the valuations of each company +valuation + +# Show correlations between each company +hcorr + +# Figure out the sentiment perceived of each company +sentiment + +# Return to home +/ +``` + +### Executing a script + +By going to the main menu as depicted below (accessible with `home`), the exe command can be used. With this command you can run any `.openbb` script. These scripts are located where the application is located inside the routines folder. + +![Routine Exe OpenBB Terminal](https://user-images.githubusercontent.com/46355364/174588513-5c52ea20-548a-4c2b-a4c1-6054e2d71786.png) + +Thus, using the earlier mentioned script, we can enter `exe stocks_demo.openbb` which automatically runs all commands within the script file. Thus, it will return a candle chart with moving averages, price targets from Analysts, valuations of related companies, the correlation between these companies and the market sentiment. This results in the following: + +![Script Stocks Demo OpenBB](https://user-images.githubusercontent.com/46355364/176903147-720eb2af-7e5d-40df-8ec6-7363cbc08430.png) + +#### Custom arguments + +Next to that, it is also possible to add in custom arguments to your script making the script more interactive and allow you to do the same analysis for multiple companies. This is done in the following script: + +``` +# This script requires you to use arguments. This can be done with the following: +# exe example_with_inputs.openbb -i TSLA,AAPL,MSFT + +# Go to the stocks menu +stocks + +# Load a ticker, given the argument used. E.g. -i TSLA +load $ARGV[0] + +# Enter the Technical Analysis (ta) menu +ta + +# Show the fibonacci retracement levels +fib + +# Enter the comparison analysis (ca) menu +../ca + +# Set two extra tickers based on the arguments used. E.g. -i TSLA,AAPL,MSFT +set $ARGV[1],$ARGV[2] + +# Plot the historical prices +historical + +# Return to home +/ +``` + +This script includes `$ARGV[0]`, `$ARGV[1]` and `$ARGV[2]`. This means that the script requires you to submit three arguments. In this case, they refer to stock tickers. Therefore, like the script also says, you can include these arguments with `-i` followed by three tickers (e.g. `exe example_with_inputs.openbb -i TSLA,AAPL,MSFT`). This results in the following: + +![Example with Inputs OpenBB Script](https://user-images.githubusercontent.com/46355364/176903205-3cb55bf5-8710-4ad6-8eef-f9a99294ea3b.png) + +It is an incredibly simple script, but it gives an understanding what the possibilities are. + +### Create your own script + +Scripts and routines reside in the `routines` folder and are automatically shown when you type `exe` from the home screen (`home`). To create your own, you can use the `template.openbb` file as a basis but you are free to duplicate one of the demo files. + +Within this template, some examples are given, and you are free to change the contents and rename the file. As long as the file remains in the same folder, you will be able to find your file automatically as shown below: + +![Show Copy of Template OpenBB Script](https://user-images.githubusercontent.com/46355364/176903253-00a5b0f9-a6e7-49c7-a1d8-49ae819e28e3.png) \ No newline at end of file diff --git a/website/content/terminal/guides/basics/_category_.json b/website/content/terminal/guides/basics/_category_.json new file mode 100644 index 00000000000..daec3e16a89 --- /dev/null +++ b/website/content/terminal/guides/basics/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Basics", + "position": 1 +} diff --git a/website/content/terminal/guides/basics/data.md b/website/content/terminal/guides/basics/data.md new file mode 100644 index 00000000000..9021f340da8 --- /dev/null +++ b/website/content/terminal/guides/basics/data.md @@ -0,0 +1,24 @@ +--- +title: Importing and Exporting Data +sidebar_position: 4 +--- +The `OpenBBUserData` folder's default location is the home of the system user account. By default this will be the following paths: +- macOS: `Macintosh HD/Users//OpenBBUserData` +- Windows: `C:/Users//OpenBBUserData` + +Within the folder you can find files that you have exported as well as files that you wish to import directly into the OpenBB Terminal. For example, this could be an orderbook which you can store in `OpenBBUserData/portfolio/holdings`. + +Screen Shot 2022-10-13 at 6 45 01 PM + +This folder contains all things user-created: + + - Screener presets + - Portfolio files + - Exported files + - Files to be imported by various functions + - Styles and themes + - Preferred data sources + + The location of this folder can be set by the user from the `/settings` menu. + +The settings menu \ No newline at end of file diff --git a/website/content/terminal/quickstart/keys.md b/website/content/terminal/guides/basics/keys.md similarity index 94% rename from website/content/terminal/quickstart/keys.md rename to website/content/terminal/guides/basics/keys.md index 0bec8df89ec..9cfc365f073 100644 --- a/website/content/terminal/quickstart/keys.md +++ b/website/content/terminal/guides/basics/keys.md @@ -1,10 +1,8 @@ --- -sidebar_position: 7 -title: API keys +title: Set API keys +sidebar_position: 3 --- -## Accessing other sources of data via API keys - -Within this menu you can define your, often free, API key from various platforms like Alpha Vantage, FRED, IEX, Twitter, DeGiro, Binance and Coinglass. API keys are in essence a set of random characters that is unique to you. +Within the `keys` menu you can define your, often free, API key from various platforms like Alpha Vantage, FRED, IEX, Twitter, DeGiro, Binance and Coinglass. API keys are in essence a set of random characters that is unique to you. You can access this menu from the homepage with `keys` which will open the menu as shown below: @@ -46,4 +44,4 @@ To figure out where you can obtain the API key, you can enter the command (e.g. | eodhd | EODHD | https://eodhistoricaldata.com/ | | santiment | Santiment | https://app.santiment.net/ | | shroom | ShroomDK | https://sdk.flipsidecrypto.xyz/shroomdk | -| tokenterminal | Token Terminal | https://tokenterminal.com/ | +| tokenterminal | Token Terminal | https://tokenterminal.com/ | \ No newline at end of file diff --git a/website/content/terminal/quickstart/commands.md b/website/content/terminal/guides/basics/menus_commands.md similarity index 67% rename from website/content/terminal/quickstart/commands.md rename to website/content/terminal/guides/basics/menus_commands.md index 6f1a1863842..8186ae3e79b 100644 --- a/website/content/terminal/quickstart/commands.md +++ b/website/content/terminal/guides/basics/menus_commands.md @@ -1,6 +1,22 @@ --- -sidebar_position: 4 -title: Commands +sidebar_position: 2 +title: Menus & Commands +--- + +### Explanation of Menus + +Menus, depicted in Dark Blue, take you to a new section of the terminal referred to as a menu. For example, if you wish to view information about stocks, you can do so by typing `stocks` and pressing `ENTER` (⏎). This opens a new menu as depicted below. + +Stocks Menu + +Depending on the menu you are in, you are presented with a new set of commands and menus you can select. There are interactions in place between each menu. For example, when selecting a company within the `stocks` menu, the terminal will remember your selection when you visit the `fa` or `options` menu. See Introduction to Stocks for more information. + +--- + +**Pro tip:** you can quickly jump between menus by using a forward slash (`/`). For example, if I want to access the options +menu, I can type `/stocks/options` to instantly arrive at this menu. You can do this from any location within the +OpenBB Terminal! + --- ### Explanation of Commands @@ -14,11 +30,10 @@ the commands that you are able to use from any menu in the terminal (see this guide. + command. For example, within the `stocks` menu, `about candle` opens this guide. - `wiki`: search for a given expression on the Wikipedia without leaving the terminal. -Continuing with the example mentioned at `quit`, revisit the `stocks` menu and look at the commands. At the top you -will see a command named load. To understand what this command can do, you can use `load -h` followed by `ENTER` (⏎). The `-h` stands for `help` and every command will have this feature. This will return the following: +Continuing with the example mentioned at `quit`, revisit the `stocks` menu and look at the commands. At the top you will see a command named load. To understand what this command can do, you can use `load -h` followed by `ENTER` (⏎). The `-h` stands for `help` and every command will have this feature. This will return the following: ``` 2022 May 19, 05:27 (🦋) /stocks/ $ load -h @@ -49,10 +64,7 @@ optional arguments: For more information and examples, use 'about load' to access the related guide. ``` -This shows you all **arguments** the command has. These are additional options you can provide to the command. Each -default value is also displayed which is used when you do not select this option. For example, if I would use the -stock ticker -of Amazon (AMZN, which can also be found with `search amazon`), I can use `load AMZN` which will return the following: +This shows you all **arguments** the command has. These are additional options you can provide to the command. Each default value is also displayed which is used when you do not select this option. For example, if I would use the stock ticker of Amazon (AMZN, which can also be found with `search amazon`), I can use `load AMZN` which will return the following: ``` 2022 May 19, 05:27 (🦋) /stocks/ $ load AMZN @@ -73,14 +85,9 @@ Company: Amazon.com, Inc. └─────────┴────────┴──────────┴──────────┴──────────┴─────────────────┴──────────────────┴────────────┘ ``` -The default values you see within `load -h` have been inputted here. E.g. the starting period is 2019-05-15. I can -decide to change these default values by calling the argument and inputting a different value. +The default values you see within `load -h` have been inputted here. E.g. the starting period is 2019-05-15. I can decide to change these default values by calling the argument and inputting a different value. -Whenever you wish to apply an optional argument, you use the related shortcode, e.g. `-s` or `--start`. Then, if there -is an additional word behind the argument (in this case there is, which is `START`) it implies the argument expects you -to define a value. Within the documentation you can read that the format must be `YYYY-MM-DD` implying that `2010-01-01` -will be valid. If there is not an additional word behind it, it is enough to write down `load AMZN -p` (which refers to -the prepost optional argument) +Whenever you wish to apply an optional argument, you use the related shortcode, e.g. `-s` or `--start`. Then, if there is an additional word behind the argument (in this case there is, which is `START`) it implies the argument expects you to define a value. Within the documentation you can read that the format must be `YYYY-MM-DD` implying that `2010-01-01` will be valid. If there is not an additional word behind it, it is enough to write down `load AMZN -p` (which refers to the prepost optional argument) Let's change the starting and ending period of the data that is being loaded in by doing the following: @@ -103,8 +110,7 @@ Company: Amazon.com, Inc. └─────────┴─────────┴─────────┴──────────┴──────────┴─────────────────┴──────────────────┴────────────┘ ``` -We can check that this period has changed by looking into the candle chart with `candle`. This, again shares the same `-h` argument. This results in the following which indeed depicts our -selected period. +We can check that this period has changed by looking into the candle chart with `candle`. This, again shares the same `-h` argument. This results in the following which indeed depicts our selected period. ``` 2022 May 19, 05:44 (🦋) /stocks/ $ candle @@ -112,6 +118,4 @@ selected period. Amazon Candle Chart -As mentioned in the Explanation of Menus, some information also transfers over to other menus and this includes the -loaded market data from load. -So, if you would visit the `ta` menu (which stands for Technical Analysis) you will see that, by running any command, the selected period above is depicted again. Return to the Stocks menu again by using `q` and use it again to return to the home screen which can be shown with `?`. +As mentioned in the Explanation of Menus, some information also transfers over to other menus and this includes the loaded market data from load. So, if you would visit the `ta` menu (which stands for Technical Analysis) you will see that, by running any command, the selected period above is depicted again. Return to the Stocks menu again by using `q` and use it again to return to the home screen which can be shown with `?`. \ No newline at end of file diff --git a/website/content/terminal/quickstart/structure.md b/website/content/terminal/guides/basics/structure.md similarity index 96% rename from website/content/terminal/quickstart/structure.md rename to website/content/terminal/guides/basics/structure.md index b567947a784..35a146f8a84 100644 --- a/website/content/terminal/quickstart/structure.md +++ b/website/content/terminal/guides/basics/structure.md @@ -1,9 +1,7 @@ --- -sidebar_position: 2 +sidebar_position: 1 +title: Structure of the Terminal --- - -# Structure of Terminal - The OpenBB Terminal is based off the Command Line Interface (CLI) which is installed by default on every computer. By opening the application you have installed via "Accessing the OpenBB Terminal", you are greeted with the following interface: diff --git a/website/content/terminal/guides/intros/_category_.json b/website/content/terminal/guides/intros/_category_.json index d8fcbac4250..f93c4207f74 100644 --- a/website/content/terminal/guides/intros/_category_.json +++ b/website/content/terminal/guides/intros/_category_.json @@ -1,4 +1,4 @@ { "label": "Intros", - "position": 1 + "position": 2 } diff --git a/website/content/terminal/guides/intros/crypto/dd.md b/website/content/terminal/guides/intros/crypto/dd.md index 9a77082628f..f2adbce5163 100644 --- a/website/content/terminal/guides/intros/crypto/dd.md +++ b/website/content/terminal/guides/intros/crypto/dd.md @@ -6,8 +6,8 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Cryptocurrency Due Diligence menu gives the user the ability to delve deeper into the coin of choice. To be able to do this, the menu offers basic information about the loaded coin (info and basic), information about the project, technology details, audits etcetera (pi), insights into the coin balance and order book (balance and cbbook), the tokenomics and fundraising details -(tk and fr) and lastly, social media activity (mt, twitter and dev). +The Cryptocurrency Due Diligence menu gives the user the ability to delve deeper into the coin of choice. To be able to do this, the menu offers basic information about the loaded coin (info and basic), information about the project, technology details, audits etcetera (pi), insights into the coin balance and order book (balance and cbbook), the tokenomics and fundraising details +(tk and fr) and lastly, social media activity (mt, twitter and dev). ### How to use diff --git a/website/content/terminal/guides/intros/crypto/defi.md b/website/content/terminal/guides/intros/crypto/defi.md index cf576af9ed5..bf413c4e96f 100644 --- a/website/content/terminal/guides/intros/crypto/defi.md +++ b/website/content/terminal/guides/intros/crypto/defi.md @@ -6,7 +6,7 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Cryptocurrency Decentralized Finance (DeFi) menu gives the user the ability to delve deeper into Uniswap statistics (tokens, stats and pairs), information about dApps (gdapps and stvl) as well as information about terra blockchain (sreturn) and luna (lcsc). +The Cryptocurrency Decentralized Finance (DeFi) menu gives the user the ability to delve deeper into Uniswap statistics (tokens, stats and pairs), information about dApps (gdapps and stvl) as well as information about terra blockchain (sreturn) and luna (lcsc). ### How to use diff --git a/website/content/terminal/guides/intros/crypto/disc.md b/website/content/terminal/guides/intros/crypto/disc.md index f57a10ba2b0..d9c408680d1 100644 --- a/website/content/terminal/guides/intros/crypto/disc.md +++ b/website/content/terminal/guides/intros/crypto/disc.md @@ -6,7 +6,7 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Cryptocurrency Discovery menu provides the option to look into a wide selection of trending coins (trending), gainers and losers (gainers and losers) as well as NFTs (nft) and decentralized apps (dex). +The Cryptocurrency Discovery menu provides the option to look into a wide selection of trending coins (trending), gainers and losers (gainers and losers) as well as NFTs (nft) and decentralized apps (dex). ### How to use diff --git a/website/content/terminal/guides/intros/crypto/index.md b/website/content/terminal/guides/intros/crypto/index.md index d722e5a82e3..79c47851e02 100644 --- a/website/content/terminal/guides/intros/crypto/index.md +++ b/website/content/terminal/guides/intros/crypto/index.md @@ -4,9 +4,9 @@ keywords: ["crypto", "web3", "nft", "blockchain", "cryptocurrency"] excerpt: "The Introduction to Cryptocurrency explains how to use the Cryptocurrency and provides a brief description of its sub-menus" --- -The Cryptocurrency menu allows you to discover trending cryptocurrency (disc). Additionally, you can also perform research of a specific coin (dd), -NFT (nft) or most lucrative Defi project -(defi). It does so by handing you tools to (among other things) perform technical analysis (ta), quantitative analysis (qa), and give you access to onchain information (onchain). +The Cryptocurrency menu allows you to discover trending cryptocurrency (disc). Additionally, you can also perform research of a specific coin (dd), +NFT (nft) or most lucrative Defi project +(defi). It does so by handing you tools to (among other things) perform technical analysis (ta), quantitative analysis (qa), and give you access to onchain information (onchain). ### How to use diff --git a/website/content/terminal/guides/intros/crypto/nft.md b/website/content/terminal/guides/intros/crypto/nft.md index 6b6a2811c61..01eaa9c5440 100644 --- a/website/content/terminal/guides/intros/crypto/nft.md +++ b/website/content/terminal/guides/intros/crypto/nft.md @@ -6,7 +6,7 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Non Fungible Token (NFT) menu gives the user the ability to dive into certain NFTs (stats). +The Non Fungible Token (NFT) menu gives the user the ability to dive into certain NFTs (stats). ### How to use diff --git a/website/content/terminal/guides/intros/crypto/onchain.md b/website/content/terminal/guides/intros/crypto/onchain.md index 6b9b0e9c9c3..5f69d7ecc2c 100644 --- a/website/content/terminal/guides/intros/crypto/onchain.md +++ b/website/content/terminal/guides/intros/crypto/onchain.md @@ -6,7 +6,7 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Cryptocurrency Onchain menu provides access to features that determine market sentiment by leveraging data from a blockchain ledger. This can be, among other things, blockchain hashrates (hr), confirmed transactions (btcct) and top traded crypto pairs (ttcp). +The Cryptocurrency Onchain menu provides access to features that determine market sentiment by leveraging data from a blockchain ledger. This can be, among other things, blockchain hashrates (hr), confirmed transactions (btcct) and top traded crypto pairs (ttcp). ### How to use @@ -47,13 +47,13 @@ Alternatively, you can also type `/crypto/onchain`. Within the Cryptocurrency On ### Examples -To access much of the Ethereum features, an ethereum address needs to be entered. As an example this can be `0x71C7656EC7ab88b098defB751B7401B5f6d8976F` which can be entered within the address command as follows: +To access much of the Ethereum features, an ethereum address needs to be entered. As an example this can be `0x71C7656EC7ab88b098defB751B7401B5f6d8976F` which can be entered within the address command as follows: ``` 2022 Jul 12, 08:48 (🦋) /crypto/onchain/ $ address 0x71C7656EC7ab88b098defB751B7401B5f6d8976F ``` -Which allows you to see the balance and hist which returns the following for `balance`: +Which allows you to see the balance and hist which returns the following for `balance`: ``` 2022 Jul 12, 08:53 (🦋) /crypto/onchain/ $ balance diff --git a/website/content/terminal/guides/intros/crypto/ov.md b/website/content/terminal/guides/intros/crypto/ov.md index 3807f3df44c..34a01a57411 100644 --- a/website/content/terminal/guides/intros/crypto/ov.md +++ b/website/content/terminal/guides/intros/crypto/ov.md @@ -6,7 +6,7 @@ with a brief overview of the features." geekdocCollapseSection: true --- -The Cryptocurrency Overview menu provides, among other things, the option to obtain a complete overview of the Cryptocurrency market (global), find cryptocurrency indices and derivatives (indices and derivatives), news feeds (news) and fees (wf and ewf) +The Cryptocurrency Overview menu provides, among other things, the option to obtain a complete overview of the Cryptocurrency market (global), find cryptocurrency indices and derivatives (indices and derivatives), news feeds (news) and fees (wf and ewf) ### How to use diff --git a/website/content/terminal/guides/intros/forecast.md b/website/content/terminal/guides/intros/forecast.md deleted file mode 100644 index 904189be9f4..00000000000 --- a/website/content/terminal/guides/intros/forecast.md +++ /dev/null @@ -1,1300 +0,0 @@ ---- -title: Introduction to Forecast -keywords: - [ - "machine learning", - "statistics", - "deep learning", - "neural networks", - "time series", - "research", - "feature engineering", - "hyperparameters", - ] -excerpt: "The introduction to Forecast explains how to use the menu" - ---- - -The forecasting menu is a machine learning toolkit that provides practitioners -with high-level components that can quickly provide state-of-the-art results, be -it with with classical or deep learning models, while also providing researchers -with low-level components that can be mixed, matched and fine tuned to build new -approaches and custom tuned models. Bring in multiple datasets and train machine -learning models with unlimited external factors to see how underlying data may -change future forecasting predictions and accuracy. - -## Accessing the Forecast menu - -The Forecast menu is called upon by typing `forecast` which opens the following -menu: - -``` -(🦋) / $ forecast -``` - -image - -## How to Use - -To begin any machine learning, you must first load in data. The menu supports -importing both terminal datasets found in `stocks` and `cryptocurrency`, along -with external datasets in the form of `.csv` that can be placed in the following -location: `~/OpenBBUserData/custom_imports/forecast/` - -It is important to note, if you ever have troubles on running a command, please -read the help commands to guide you on what is available. You can do this by -trailing any command with a `-h` - -``` -(🦋) /forecast/ $ -h -``` - -### Loading Data - -Importing data from within the terminal can be simply done as follows: - -``` -(🦋) / $ stocks -(🦋) /stocks/ $ load AAPL -(🦋) /stocks/ $ forecast -``` - -The menu can support loading in unlimited datasets. Once the first one is -loaded, the menu options should turn blue to allow you to begin working through -the menu options. - -To load external data in the form as `.csv`, please place them into the folder -shown within your terminal menu and then load them in as follows: - -``` -(🦋) / $ forecast -(🦋) /forecast/ $ load msft.csv -(🦋) /forecast/ $ load btc.csv -``` - -image - -### Understanding the structure of the menu - -The menu is broken up into three distinct sections. These sections represent the -typical order of operations a machine learning professional would have to take -in order to complete a project. - -Once data is loaded in, begin with `Exploration`. - -`Exploration`: Explore the datasets loaded into the menu to further understand -your data and create unique new datasets by combining and analyzing features. -Functions to note: `plot`, `combine`, `desc`, `corr` -image - -`Feature Engineering`: Manuipulate datasets (addition, deletion, combination, -mutation) of your data set to potentially improve machine learning model -training, by providing new features and covariates that may leading to better -performance and greater accuracy. -image - -`TimeSeries Forecasting`: Train state of the art models on custom datasets and -experiment tuning hyperparameters. For more information on specific model -implementations, please see -[Darts Models](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.html) -for in depth documentation. -image - -## Sample workflow #1 (beginner) - -Let's begin by using one of the datasets we loaded in previously : `AAPL` - -We will be forecasting `5 Business days` ahead for the remaider of these -workflows unless specified. - -Note: All models automatically perform Historical backtesting on the test split -before providing a prediction. - -Note: `MAPE` = -[mean average percentage error](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error). -We use MAPE as it is quite convenient and scale independent since it calculates -error as a percentage instead of an absolute value. THere are many more metrics -to compare time series. The metrics will compare only common slices of series -when the two series are not aligned, and parallelize computation over a large -number of pairs of series. More metrics will be released in future versions of -the menu. - -``` -(🦋) /forecast/ $ plot AAPL.close -``` - -image - -``` -(🦋) /forecast/ $ desc AAPL - - Showing Descriptive Statistics for Dataset AAPL -┏━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ -┃ ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ -┡━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ -│ count │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ mean │ 121.57 │ 123.07 │ 120.15 │ 121.68 │ 120.72 │ 113864153.69 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ std │ 36.37 │ 36.74 │ 35.95 │ 36.34 │ 36.61 │ 56098731.72 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ min │ 53.47 │ 54.11 │ 52.77 │ 53.54 │ 52.38 │ 41000000.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 25% │ 83.10 │ 86.41 │ 83.04 │ 84.16 │ 83.00 │ 77032650.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 50% │ 128.18 │ 129.64 │ 126.63 │ 127.86 │ 126.70 │ 98135650.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 75% │ 149.05 │ 150.38 │ 147.69 │ 149.14 │ 148.50 │ 131152875.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ max │ 182.63 │ 182.94 │ 179.12 │ 182.01 │ 181.26 │ 426510000.00 │ -└───────┴────────┴────────┴────────┴────────┴───────────┴──────────────┘ -``` - -Let's use a simple **Probabilistic Exponential Smoothing Model** to predict the -close price. Keep in mind all models are perform automatic historical -backtesting before providing future forecasts. - -Note: All models forecaste `close` by default. - -``` -(🦋) /forecast/ $ expo AAPL -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:17<00:00, 6.44it/s] -Exponential smoothing obtains MAPE: 3.86% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 157.99 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 157.89 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 158.32 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 158.25 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 158.52 │ -└────────────┴────────────┘ -``` - -image - -That looks great, but we might want to see it a little more up close. Lets set -the flag for `--forecast-only`. - -image - -We can also play with some models that are bit more advanced. As we go down the -list, models begin to become larger in parameter size and complexity. This will -play a key role later on when we want to train models with `past_covariates` -(aka. external factors). - -This time lets test with a **Recurrent Neural Network** which by default uses an -`LSTM` backbone. We can also choose to test out a `GRU` backbone to experiment. -Let's do both and see if we can improve our accuracy and reduce the overall -MAPE. - -``` -(🦋) /forecast/ $ rnn AAPL --forecast-only -Epoch 193: 100%|███████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 129.49it/s, loss=-2.74, train_loss=-2.75, val_loss=-2.22] -Predicting RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 86.89it/s] -RNN model obtains MAPE: 3.69% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 155.71 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 155.42 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 155.11 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 154.81 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 154.51 │ -└────────────┴────────────┘ -``` - -image - -Seems like we improved the performance and reduced MAPE! - -Now for the second task, we would like to change the model type from `LSTM` --> -`GRU`. Use the `-h` flag to understand the particular parameters one can change -for RNN. (Please note that the parameters are different for each model). - -``` -(🦋) /forecast/ $ rnn -h -usage: rnn [--hidden-dim HIDDEN_DIM] [--training_length TRAINING_LENGTH] [--naive] [-d {AAPL,msft}] [-c TARGET_COLUMN] [-n N_DAYS] [-t TRAIN_SPLIT] - [-i INPUT_CHUNK_LENGTH] [--force-reset FORCE_RESET] [--save-checkpoints SAVE_CHECKPOINTS] [--model-save-name MODEL_SAVE_NAME] - [--n-epochs N_EPOCHS] [--model-type MODEL_TYPE] [--dropout DROPOUT] [--batch-size BATCH_SIZE] [--end S_END_DATE] [--start S_START_DATE] - [--learning-rate LEARNING_RATE] [--residuals] [--forecast-only] [-h] [--export EXPORT] - -Perform RNN forecast (Vanilla RNN, LSTM, GRU) - -optional arguments: - --hidden-dim HIDDEN_DIM - Size for feature maps for each hidden RNN layer (h_n) (default: 20) - --training_length TRAINING_LENGTH - The length of both input (target and covariates) and output (target) time series used during training. Generally speaking, - training_length should have a higher value than input_chunk_length because otherwise during training the RNN is never run for as - many iterations as it will during training. (default: 20) - --naive Show the naive baseline for a model. (default: False) - -d {AAPL,msft}, --target-dataset {AAPL,msft} - The name of the dataset you want to select (default: None) - -c TARGET_COLUMN, --target-column TARGET_COLUMN - The name of the specific column you want to use (default: close) - -n N_DAYS, --n-days N_DAYS - prediction days. (default: 5) - -t TRAIN_SPLIT, --train-split TRAIN_SPLIT - Start point for rolling training and forecast window. 0.0-1.0 (default: 0.85) - -i INPUT_CHUNK_LENGTH, --input-chunk-length INPUT_CHUNK_LENGTH - Number of past time steps for forecasting module at prediction time. (default: 14) - --force-reset FORCE_RESET - If set to True, any previously-existing model with the same name will be reset (all checkpoints will be discarded). (default: - True) - --save-checkpoints SAVE_CHECKPOINTS - Whether to automatically save the untrained model and checkpoints. (default: True) - --model-save-name MODEL_SAVE_NAME - Name of the model to save. (default: rnn_model) - --n-epochs N_EPOCHS Number of epochs over which to train the model. (default: 300) - --model-type MODEL_TYPE - Either a string specifying the RNN module type ("RNN", "LSTM" or "GRU") (default: LSTM) - --dropout DROPOUT Fraction of neurons afected by Dropout. (default: 0) - --batch-size BATCH_SIZE - Number of time series (input and output) used in each training pass (default: 32) - --end S_END_DATE The end date (format YYYY-MM-DD) to select for testing (default: None) - --start S_START_DATE The start date (format YYYY-MM-DD) to select for testing (default: None) - --learning-rate LEARNING_RATE - Learning rate during training. (default: 0.001) - --residuals Show the residuals for the model. (default: False) - --forecast-only Do not plot the hisotorical data without forecasts. (default: False) - -h, --help show this help message (default: False) - --export EXPORT Export figure into png, jpg, pdf, svg (default: ) -``` - -Lets change the `--model-type` parameter to `GRU` and rerun. - -``` -(🦋) /forecast/ $ rnn AAPL --model-type GRU --forecast-only -Epoch 35: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 125.85it/s, loss=-2.72, train_loss=-2.74, val_loss=-2.13] -Predicting RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 85.14it/s] -RNN model obtains MAPE: 3.64% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 155.28 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 155.02 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 154.65 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 154.30 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 153.96 │ -└────────────┴────────────┘ -``` - -image - -Looks like we squeezed out a little bit more accuracy! Good work. - -The take away for this is that all models should work out of the box when -forecasting for a particular Timeseries. One can switch the target by specifying -a `-c` for `TARGET_COLUMN` and test out performance with multiple different -models with a single command. - -## Sample workflow #2 (advanced) - -To build successful models and improve accuracy over time, it is important to -capture external data related to the timeseries you are training on. This can be -seen in everyday applications: - -- Observed rainfalls and known weather forecasts can help to predict hydro and - solar electricity production -- Recently-observed activity on an e-commerce website can help predict future - sales. -- Making the model aware of up-coming holidays can help sales forecasting. - -_In fact, more often than not, strictly relying on the history of a time series -to predict its future is missing a lot of valuable information._ - -**Past covariates** are time series whose past values are known at prediction -time. Those series often contain values that have to be observed to be known. - -![image](https://user-images.githubusercontent.com/105685594/190244764-ce8cf01f-c959-4827-a326-62b0e172332d.png) - -If you would like to explore this topic more, please read the -[blog post](https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993) -written by the authors of Darts. - -Note: Only the following models can handle `past_covariates`: `BlockRNNModel`, -`NBEATSModel`, `TCNModel`, `TransformerModel`, `RegressionModel` (incl. -`LinearRegressionModel`), `Temporal Fusion Transformer` - -In this work flow lets do the following: - -- add in some correlation analysis -- combine datasets -- perform feature engineering -- train models with `past_covariates` - -Let's begin by loading in our datasets of AAPL and MSFT once again. In this work -flow we are going to test if `close` price of MSFT is at all affected by the -`close` of AAPL. - -For a refresher, we will grab data from the `stocks` menu found on the terminal. - -``` -(🦋) / $ stocks -(🦋) /stocks/ $ load AAPL -(🦋) /stocks/ $ forecast -(🦋) /forecast/ $ .. -(🦋) /stocks/ $ load MSFT -(🦋) /stocks/ $ forecast -``` - -image - -Before we go combining them, let's train a simple `Block RNN` model on MSFT -`close` price to see how to use `past_covariates` - -Make sure to always check your current data set to know the column names: - -``` -(🦋) /forecast/ $ show MSFT -MSFT dataset has shape (row, column): (759, 7) - - Dataset MSFT | Showing 10 of 759 rows -┏━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓ -┃ ┃ date ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ -┡━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩ -│ 0 │ 2019-09-10 00:00:00 │ 136.80 │ 136.89 │ 134.51 │ 136.08 │ 132.22 │ 28903400 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 1 │ 2019-09-11 00:00:00 │ 135.91 │ 136.27 │ 135.09 │ 136.12 │ 132.26 │ 24726100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 2 │ 2019-09-12 00:00:00 │ 137.85 │ 138.42 │ 136.87 │ 137.52 │ 133.62 │ 27010000 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 3 │ 2019-09-13 00:00:00 │ 137.78 │ 138.06 │ 136.57 │ 137.32 │ 133.42 │ 23363100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 4 │ 2019-09-16 00:00:00 │ 135.83 │ 136.70 │ 135.66 │ 136.33 │ 132.46 │ 16731400 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 5 │ 2019-09-17 00:00:00 │ 136.96 │ 137.52 │ 136.43 │ 137.39 │ 133.49 │ 17814200 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 6 │ 2019-09-18 00:00:00 │ 137.36 │ 138.67 │ 136.53 │ 138.52 │ 134.59 │ 23982100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 7 │ 2019-09-19 00:00:00 │ 140.30 │ 142.37 │ 140.07 │ 141.07 │ 137.07 │ 35772100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 8 │ 2019-09-20 00:00:00 │ 141.01 │ 141.65 │ 138.25 │ 139.44 │ 135.48 │ 39167300 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 9 │ 2019-09-23 00:00:00 │ 139.23 │ 139.63 │ 138.44 │ 139.14 │ 135.19 │ 17139300 │ -└───┴─────────────────────┴────────┴────────┴────────┴────────┴───────────┴──────────┘ -``` - -Without any covariates: - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only -Epoch 87: 100%|█████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 33.84it/s, loss=-2.06, train_loss=-2.27, val_loss=-1.82] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:06<00:00, 18.76it/s] -Block RNN model obtains MAPE: 4.62% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 258.52 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 258.01 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 258.95 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.20 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 257.66 │ -└────────────┴────────────┘ -``` - -image - -With covariates: - -To use any covariates, you have 2 options: - -- specify specific columns with `--past-covariates` -- specify all columns as past covariates except the one you are forecasting - `--all-past-covariates` - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --past-covariates volume -Covariate #0: volume -Epoch 37: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.03it/s, loss=-2.16, train_loss=-2.08, val_loss=-1.44] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 88.94it/s] -Block RNN model obtains MAPE: 5.10% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 257.02 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 255.82 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 260.13 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 258.32 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 258.52 │ -└────────────┴────────────┘ -``` - -image - -You can see here that adding in the external variable of `volume` negatively -affected the accuracy. - -Let's add in all remaining columns from our dataset as covariates and see what -happens to the accuracy. - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Epoch 50: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.29it/s, loss=-2.41, train_loss=-2.53, val_loss=-1.71] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 89.43it/s] -Block RNN model obtains MAPE: 4.26% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 259.28 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 259.13 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 259.42 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.73 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 258.64 │ -└────────────┴────────────┘ -``` - -image - -**Final Result:** Using `open`, `high`, `low`, `adj_close`, `volume` as -`past_covariates` improved MAPE from 4.62 --> 4.26. - -Now that we know how to use covariates and understand their effect, why don't we -also use `AAPL`'s ticker data as `past_covariates` to check whether this -correlates and improves our forecasting accuracy. - -**Remember: You can use unlimited number of `past_covariates` but they must all -be combined into a single dataframe with the target forecast timeseries before -training.** - -We will combine `MSFT` and `AAPL`. - -``` -(🦋) /forecast/ $ combine MSFT -c AAPL -(🦋) /forecast/ $ show MSFT -MSFT dataset has shape (row, column): (759, 13) -Dataframe has more than 10 columns. Please export to see all of the data. - - - Dataset MSFT | Showing 10 of 759 rows -┏━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓ -┃ ┃ date ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ AAPL_open ┃ AAPL_high ┃ AAPL_low ┃ -┡━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩ -│ 0 │ 2019-09-10 00:00:00 │ 136.80 │ 136.89 │ 134.51 │ 136.08 │ 132.22 │ 28903400 │ 53.47 │ 54.19 │ 52.93 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 1 │ 2019-09-11 00:00:00 │ 135.91 │ 136.27 │ 135.09 │ 136.12 │ 132.26 │ 24726100 │ 54.52 │ 55.93 │ 54.43 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 2 │ 2019-09-12 00:00:00 │ 137.85 │ 138.42 │ 136.87 │ 137.52 │ 133.62 │ 27010000 │ 56.20 │ 56.60 │ 55.72 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 3 │ 2019-09-13 00:00:00 │ 137.78 │ 138.06 │ 136.57 │ 137.32 │ 133.42 │ 23363100 │ 55.00 │ 55.20 │ 54.26 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 4 │ 2019-09-16 00:00:00 │ 135.83 │ 136.70 │ 135.66 │ 136.33 │ 132.46 │ 16731400 │ 54.43 │ 55.03 │ 54.39 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 5 │ 2019-09-17 00:00:00 │ 136.96 │ 137.52 │ 136.43 │ 137.39 │ 133.49 │ 17814200 │ 54.99 │ 55.21 │ 54.78 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 6 │ 2019-09-18 00:00:00 │ 137.36 │ 138.67 │ 136.53 │ 138.52 │ 134.59 │ 23982100 │ 55.26 │ 55.71 │ 54.86 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 7 │ 2019-09-19 00:00:00 │ 140.30 │ 142.37 │ 140.07 │ 141.07 │ 137.07 │ 35772100 │ 55.50 │ 55.94 │ 55.09 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 8 │ 2019-09-20 00:00:00 │ 141.01 │ 141.65 │ 138.25 │ 139.44 │ 135.48 │ 39167300 │ 55.35 │ 55.64 │ 54.37 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 9 │ 2019-09-23 00:00:00 │ 139.23 │ 139.63 │ 138.44 │ 139.14 │ 135.19 │ 17139300 │ 54.74 │ 54.96 │ 54.41 │ -└───┴─────────────────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴───────────┴───────────┴──────────┘ -``` - -Now we can run the same `BRNN` model with all `past_covariates` of both tickers. -The output will show the model grabbing all covariates to bring into training to -predict `close` of `MSFT`. - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Covariate #5: AAPL_open -Covariate #6: AAPL_high -Covariate #7: AAPL_low -Covariate #8: AAPL_close -Covariate #9: AAPL_adj_close -Covariate #10: AAPL_volume -Epoch 116: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.26it/s, loss=-2.5, train_loss=-2.61, val_loss=-1.93] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 90.26it/s] -Block RNN model obtains MAPE: 3.93% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 259.78 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 258.15 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 261.44 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.57 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 257.99 │ -└────────────┴────────────┘ -``` - -image - -For one last experiment, we can perform some other feature engineering on `MSFT` -and add it to our `past_covariates` to train on. - -In this case, let's add in `Momentum` over past 10 days of `MSFT` and append it -to our `past_covariates` - -``` -(🦋) /forecast/ $ mom MSFT -Successfully added 'Momentum_10' to 'MSFT' dataset - -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -The data contains inf or nan values. They will be removed. - -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Covariate #5: AAPL_open -Covariate #6: AAPL_high -Covariate #7: AAPL_low -Covariate #8: AAPL_close -Covariate #9: AAPL_adj_close -Covariate #10: AAPL_volume -Covariate #11: Momentum_10 -Epoch 71: 100%|████████████████████████████████████████████████████████████| 24/24 [00:00<00:00, 147.87it/s, loss=-2.52, train_loss=-2.57, val_loss=-1.82] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 114/114 [00:01<00:00, 90.05it/s] -Block RNN model obtains MAPE: 3.72% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 255.14 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 256.20 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 256.67 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 258.63 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 256.62 │ -└────────────┴────────────┘ -``` - -image - -There we have it. Bringing in another ticker has allowed us to further improve -the model accuracy. Furthermore, adding in a new feature to the dataset allowed -us to improve the accuracy further. - -**MAPE = 4.62%** (no past covariates) - -**MAPE = 4.26%** (`open`,`high`,`low`,`adj_close`,`volume`) - -**MAPE = 3.93%** -(`open`,`high`,`low`,`adj_close`,`volume`,`AAPL_open`,`AAPL_high`,`APPL_low`,`APPL_adj_close`,`APPL_volume`,`APPL_close`) - -**MAPE = 3.72%** -(`open`,`high`,`low`,`adj_close`,`volume`,`AAPL_open`,`AAPL_high`,`APPL_low`,`APPL_adj_close`,`APPL_volume`,`APPL_close`,`Momentum_10`) - -More workflows coming soon! - -## If you have any questions or would like to request for new feature engineering or model additions, please join us on discord. Happy hacking! - -title: Introduction to Forecast keywords: ['machine learning', 'statistics', -'deep learning', 'neural networks', 'time series', 'research', 'feature -engineering', 'hyperparameters'] excerpt: "The introduction to Forecast explains -how to use the menu" - ---- - -The forecasting menu is a machine learning toolkit that provides practitioners -with high-level components that can quickly provide state-of-the-art results, be -it with with classical or deep learning models, while also providing researchers -with low-level components that can be mixed, matched and fine tuned to build new -approaches and custom tuned models. Bring in multiple datasets and train machine -learning models with unlimited external factors to see how underlying data may -change future forecasting predictions and accuracy. - -## Accessing the Forecast menu - -The Forecast menu is called upon by typing `forecast` which opens the following -menu: - -``` -(🦋) / $ forecast -``` - -image - -## How to Use - -To begin any machine learning, you must first load in data. The menu supports -importing both terminal datasets found in `stocks` and `cryptocurrency`, along -with external datasets in the form of `.csv` that can be placed in the following -location: `~/OpenBBUserData/custom_imports/forecast/` - -It is important to note, if you ever have troubles on running a command, please -read the help commands to guide you on what is available. You can do this by -trailing any command with a `-h` - -``` -(🦋) /forecast/ $ -h -``` - -### Loading Data - -Importing data from within the terminal can be simply done as follows: - -``` -(🦋) / $ stocks -(🦋) /stocks/ $ load AAPL -(🦋) /stocks/ $ forecast -``` - -The menu can support loading in unlimited datasets. Once the first one is -loaded, the menu options should turn blue to allow you to begin working through -the menu options. - -To load external data in the form as `.csv`, please place them into the folder -shown within your terminal menu and then load them in as follows: - -``` -(🦋) / $ forecast -(🦋) /forecast/ $ load msft.csv -(🦋) /forecast/ $ load btc.csv -``` - -image - -### Understanding the structure of the menu - -The menu is broken up into three distinct sections. These sections represent the -typical order of operations a machine learning professional would have to take -in order to complete a project. - -Once data is loaded in, begin with `Exploration`. - -`Exploration`: Explore the datasets loaded into the menu to further understand -your data and create unique new datasets by combining and analyzing features. -Functions to note: `plot`, `combine`, `desc`, `corr` -image - -`Feature Engineering`: Manuipulate datasets (addition, deletion, combination, -mutation) of your data set to potentially improve machine learning model -training, by providing new features and covariates that may leading to better -performance and greater accuracy. -image - -`TimeSeries Forecasting`: Train state of the art models on custom datasets and -experiment tuning hyperparameters. For more information on specific model -implementations, please see -[Darts Models](https://unit8co.github.io/darts/generated_api/darts.models.forecasting.html) -for in depth documentation. -image - -## Sample workflow #1 (beginner) - -Let's begin by using one of the datasets we loaded in previously : `AAPL` - -We will be forecasting `5 Business days` ahead for the remaider of these -workflows unless specified. - -Note: All models automatically perform Historical backtesting on the test split -before providing a prediction. - -Note: `MAPE` = -[mean average percentage error](https://en.wikipedia.org/wiki/Mean_absolute_percentage_error). -We use MAPE as it is quite convenient and scale independent since it calculates -error as a percentage instead of an absolute value. THere are many more metrics -to compare time series. The metrics will compare only common slices of series -when the two series are not aligned, and parallelize computation over a large -number of pairs of series. More metrics will be released in future versions of -the menu. - -``` -(🦋) /forecast/ $ plot AAPL.close -``` - -image - -``` -(🦋) /forecast/ $ desc AAPL - - Showing Descriptive Statistics for Dataset AAPL -┏━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━┓ -┃ ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ -┡━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━┩ -│ count │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ 758.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ mean │ 121.57 │ 123.07 │ 120.15 │ 121.68 │ 120.72 │ 113864153.69 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ std │ 36.37 │ 36.74 │ 35.95 │ 36.34 │ 36.61 │ 56098731.72 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ min │ 53.47 │ 54.11 │ 52.77 │ 53.54 │ 52.38 │ 41000000.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 25% │ 83.10 │ 86.41 │ 83.04 │ 84.16 │ 83.00 │ 77032650.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 50% │ 128.18 │ 129.64 │ 126.63 │ 127.86 │ 126.70 │ 98135650.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ 75% │ 149.05 │ 150.38 │ 147.69 │ 149.14 │ 148.50 │ 131152875.00 │ -├───────┼────────┼────────┼────────┼────────┼───────────┼──────────────┤ -│ max │ 182.63 │ 182.94 │ 179.12 │ 182.01 │ 181.26 │ 426510000.00 │ -└───────┴────────┴────────┴────────┴────────┴───────────┴──────────────┘ -``` - -Let's use a simple **Probabilistic Exponential Smoothing Model** to predict the -close price. Keep in mind all models are perform automatic historical -backtesting before providing future forecasts. - -Note: All models forecaste `close` by default. - -``` -(🦋) /forecast/ $ expo AAPL -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:17<00:00, 6.44it/s] -Exponential smoothing obtains MAPE: 3.86% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 157.99 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 157.89 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 158.32 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 158.25 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 158.52 │ -└────────────┴────────────┘ -``` - -image - -That looks great, but we might want to see it a little more up close. Lets set -the flag for `--forecast-only`. - -image - -We can also play with some models that are bit more advanced. As we go down the -list, models begin to become larger in parameter size and complexity. This will -play a key role later on when we want to train models with `past_covariates` -(aka. external factors). - -This time lets test with a **Recurrent Neural Network** which by default uses an -`LSTM` backbone. We can also choose to test out a `GRU` backbone to experiment. -Let's do both and see if we can improve our accuracy and reduce the overall -MAPE. - -``` -(🦋) /forecast/ $ rnn AAPL --forecast-only -Epoch 193: 100%|███████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 129.49it/s, loss=-2.74, train_loss=-2.75, val_loss=-2.22] -Predicting RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 86.89it/s] -RNN model obtains MAPE: 3.69% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 155.71 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 155.42 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 155.11 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 154.81 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 154.51 │ -└────────────┴────────────┘ -``` - -image - -Seems like we improved the performance and reduced MAPE! - -Now for the second task, we would like to change the model type from `LSTM` --> -`GRU`. Use the `-h` flag to understand the particular parameters one can change -for RNN. (Please note that the parameters are different for each model). - -``` -(🦋) /forecast/ $ rnn -h -usage: rnn [--hidden-dim HIDDEN_DIM] [--training_length TRAINING_LENGTH] [--naive] [-d {AAPL,msft}] [-c TARGET_COLUMN] [-n N_DAYS] [-t TRAIN_SPLIT] - [-i INPUT_CHUNK_LENGTH] [--force-reset FORCE_RESET] [--save-checkpoints SAVE_CHECKPOINTS] [--model-save-name MODEL_SAVE_NAME] - [--n-epochs N_EPOCHS] [--model-type MODEL_TYPE] [--dropout DROPOUT] [--batch-size BATCH_SIZE] [--end S_END_DATE] [--start S_START_DATE] - [--learning-rate LEARNING_RATE] [--residuals] [--forecast-only] [-h] [--export EXPORT] - -Perform RNN forecast (Vanilla RNN, LSTM, GRU) - -optional arguments: - --hidden-dim HIDDEN_DIM - Size for feature maps for each hidden RNN layer (h_n) (default: 20) - --training_length TRAINING_LENGTH - The length of both input (target and covariates) and output (target) time series used during training. Generally speaking, - training_length should have a higher value than input_chunk_length because otherwise during training the RNN is never run for as - many iterations as it will during training. (default: 20) - --naive Show the naive baseline for a model. (default: False) - -d {AAPL,msft}, --target-dataset {AAPL,msft} - The name of the dataset you want to select (default: None) - -c TARGET_COLUMN, --target-column TARGET_COLUMN - The name of the specific column you want to use (default: close) - -n N_DAYS, --n-days N_DAYS - prediction days. (default: 5) - -t TRAIN_SPLIT, --train-split TRAIN_SPLIT - Start point for rolling training and forecast window. 0.0-1.0 (default: 0.85) - -i INPUT_CHUNK_LENGTH, --input-chunk-length INPUT_CHUNK_LENGTH - Number of past time steps for forecasting module at prediction time. (default: 14) - --force-reset FORCE_RESET - If set to True, any previously-existing model with the same name will be reset (all checkpoints will be discarded). (default: - True) - --save-checkpoints SAVE_CHECKPOINTS - Whether to automatically save the untrained model and checkpoints. (default: True) - --model-save-name MODEL_SAVE_NAME - Name of the model to save. (default: rnn_model) - --n-epochs N_EPOCHS Number of epochs over which to train the model. (default: 300) - --model-type MODEL_TYPE - Either a string specifying the RNN module type ("RNN", "LSTM" or "GRU") (default: LSTM) - --dropout DROPOUT Fraction of neurons afected by Dropout. (default: 0) - --batch-size BATCH_SIZE - Number of time series (input and output) used in each training pass (default: 32) - --end S_END_DATE The end date (format YYYY-MM-DD) to select for testing (default: None) - --start S_START_DATE The start date (format YYYY-MM-DD) to select for testing (default: None) - --learning-rate LEARNING_RATE - Learning rate during training. (default: 0.001) - --residuals Show the residuals for the model. (default: False) - --forecast-only Do not plot the hisotorical data without forecasts. (default: False) - -h, --help show this help message (default: False) - --export EXPORT Export figure into png, jpg, pdf, svg (default: ) -``` - -Lets change the `--model-type` parameter to `GRU` and rerun. - -``` -(🦋) /forecast/ $ rnn AAPL --model-type GRU --forecast-only -Epoch 35: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 125.85it/s, loss=-2.72, train_loss=-2.74, val_loss=-2.13] -Predicting RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 85.14it/s] -RNN model obtains MAPE: 3.64% - - - Actual price: 157.37 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-12 │ 155.28 │ -├────────────┼────────────┤ -│ 2022-09-13 │ 155.02 │ -├────────────┼────────────┤ -│ 2022-09-14 │ 154.65 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 154.30 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 153.96 │ -└────────────┴────────────┘ -``` - -image - -Looks like we squeezed out a little bit more accuracy! Good work. - -The take away for this is that all models should work out of the box when -forecasting for a particular Timeseries. One can switch the target by specifying -a `-c` for `TARGET_COLUMN` and test out performance with multiple different -models with a single command. - -## Sample workflow #2 (advanced) - -To build successful models and improve accuracy over time, it is important to -capture external data related to the timeseries you are training on. This can be -seen in everyday applications: - -- Observed rainfalls and known weather forecasts can help to predict hydro and - solar electricity production -- Recently-observed activity on an e-commerce website can help predict future - sales. -- Making the model aware of up-coming holidays can help sales forecasting. - -_In fact, more often than not, strictly relying on the history of a time series -to predict its future is missing a lot of valuable information._ - -**Past covariates** are time series whose past values are known at prediction -time. Those series often contain values that have to be observed to be known. - -![image](https://user-images.githubusercontent.com/105685594/190244764-ce8cf01f-c959-4827-a326-62b0e172332d.png) - -If you would like to explore this topic more, please read the -[blog post](https://medium.com/unit8-machine-learning-publication/time-series-forecasting-using-past-and-future-external-data-with-darts-1f0539585993) -written by the authors of Darts. - -Note: Only the following models can handle `past_covariates`: `BlockRNNModel`, -`NBEATSModel`, `TCNModel`, `TransformerModel`, `RegressionModel` (incl. -`LinearRegressionModel`), `Temporal Fusion Transformer` - -In this work flow lets do the following: - -- add in some correlation analysis -- combine datasets -- perform feature engineering -- train models with `past_covariates` - -Let's begin by loading in our datasets of AAPL and MSFT once again. In this work -flow we are going to test if `close` price of MSFT is at all affected by the -`close` of AAPL. - -For a refresher, we will grab data from the `stocks` menu found on the terminal. - -``` -(🦋) / $ stocks -(🦋) /stocks/ $ load AAPL -(🦋) /stocks/ $ forecast -(🦋) /forecast/ $ .. -(🦋) /stocks/ $ load MSFT -(🦋) /stocks/ $ forecast -``` - -image - -Before we go combining them, let's train a simple `Block RNN` model on MSFT -`close` price to see how to use `past_covariates` - -Make sure to always check your current data set to know the column names: - -``` -(🦋) /forecast/ $ show MSFT -MSFT dataset has shape (row, column): (759, 7) - - Dataset MSFT | Showing 10 of 759 rows -┏━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓ -┃ ┃ date ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ -┡━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩ -│ 0 │ 2019-09-10 00:00:00 │ 136.80 │ 136.89 │ 134.51 │ 136.08 │ 132.22 │ 28903400 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 1 │ 2019-09-11 00:00:00 │ 135.91 │ 136.27 │ 135.09 │ 136.12 │ 132.26 │ 24726100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 2 │ 2019-09-12 00:00:00 │ 137.85 │ 138.42 │ 136.87 │ 137.52 │ 133.62 │ 27010000 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 3 │ 2019-09-13 00:00:00 │ 137.78 │ 138.06 │ 136.57 │ 137.32 │ 133.42 │ 23363100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 4 │ 2019-09-16 00:00:00 │ 135.83 │ 136.70 │ 135.66 │ 136.33 │ 132.46 │ 16731400 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 5 │ 2019-09-17 00:00:00 │ 136.96 │ 137.52 │ 136.43 │ 137.39 │ 133.49 │ 17814200 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 6 │ 2019-09-18 00:00:00 │ 137.36 │ 138.67 │ 136.53 │ 138.52 │ 134.59 │ 23982100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 7 │ 2019-09-19 00:00:00 │ 140.30 │ 142.37 │ 140.07 │ 141.07 │ 137.07 │ 35772100 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 8 │ 2019-09-20 00:00:00 │ 141.01 │ 141.65 │ 138.25 │ 139.44 │ 135.48 │ 39167300 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┤ -│ 9 │ 2019-09-23 00:00:00 │ 139.23 │ 139.63 │ 138.44 │ 139.14 │ 135.19 │ 17139300 │ -└───┴─────────────────────┴────────┴────────┴────────┴────────┴───────────┴──────────┘ -``` - -Without any covariates: - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only -Epoch 87: 100%|█████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 33.84it/s, loss=-2.06, train_loss=-2.27, val_loss=-1.82] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:06<00:00, 18.76it/s] -Block RNN model obtains MAPE: 4.62% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 258.52 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 258.01 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 258.95 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.20 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 257.66 │ -└────────────┴────────────┘ -``` - -image - -With covariates: - -To use any covariates, you have 2 options: - -- specify specific columns with `--past-covariates` -- specify all columns as past covariates except the one you are forecasting - `--all-past-covariates` - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --past-covariates volume -Covariate #0: volume -Epoch 37: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.03it/s, loss=-2.16, train_loss=-2.08, val_loss=-1.44] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 88.94it/s] -Block RNN model obtains MAPE: 5.10% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 257.02 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 255.82 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 260.13 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 258.32 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 258.52 │ -└────────────┴────────────┘ -``` - -image - -You can see here that adding in the external variable of `volume` negatively -affected the accuracy. - -Let's add in all remaining columns from our dataset as covariates and see what -happens to the accuracy. - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Epoch 50: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.29it/s, loss=-2.41, train_loss=-2.53, val_loss=-1.71] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 89.43it/s] -Block RNN model obtains MAPE: 4.26% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 259.28 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 259.13 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 259.42 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.73 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 258.64 │ -└────────────┴────────────┘ -``` - -image - -**Final Result:** Using `open`, `high`, `low`, `adj_close`, `volume` as -`past_covariates` improved MAPE from 4.62 --> 4.26. - -Now that we know how to use covariates and understand their effect, why don't we -also use `AAPL`'s ticker data as `past_covariates` to check whether this -correlates and improves our forecasting accuracy. - -**Remember: You can use unlimited number of `past_covariates` but they must all -be combined into a single dataframe with the target forecast timeseries before -training.** - -We will combine `MSFT` and `AAPL`. - -``` -(🦋) /forecast/ $ combine MSFT -c AAPL -(🦋) /forecast/ $ show MSFT -MSFT dataset has shape (row, column): (759, 13) -Dataframe has more than 10 columns. Please export to see all of the data. - - - Dataset MSFT | Showing 10 of 759 rows -┏━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓ -┃ ┃ date ┃ open ┃ high ┃ low ┃ close ┃ adj_close ┃ volume ┃ AAPL_open ┃ AAPL_high ┃ AAPL_low ┃ -┡━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩ -│ 0 │ 2019-09-10 00:00:00 │ 136.80 │ 136.89 │ 134.51 │ 136.08 │ 132.22 │ 28903400 │ 53.47 │ 54.19 │ 52.93 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 1 │ 2019-09-11 00:00:00 │ 135.91 │ 136.27 │ 135.09 │ 136.12 │ 132.26 │ 24726100 │ 54.52 │ 55.93 │ 54.43 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 2 │ 2019-09-12 00:00:00 │ 137.85 │ 138.42 │ 136.87 │ 137.52 │ 133.62 │ 27010000 │ 56.20 │ 56.60 │ 55.72 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 3 │ 2019-09-13 00:00:00 │ 137.78 │ 138.06 │ 136.57 │ 137.32 │ 133.42 │ 23363100 │ 55.00 │ 55.20 │ 54.26 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 4 │ 2019-09-16 00:00:00 │ 135.83 │ 136.70 │ 135.66 │ 136.33 │ 132.46 │ 16731400 │ 54.43 │ 55.03 │ 54.39 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 5 │ 2019-09-17 00:00:00 │ 136.96 │ 137.52 │ 136.43 │ 137.39 │ 133.49 │ 17814200 │ 54.99 │ 55.21 │ 54.78 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 6 │ 2019-09-18 00:00:00 │ 137.36 │ 138.67 │ 136.53 │ 138.52 │ 134.59 │ 23982100 │ 55.26 │ 55.71 │ 54.86 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 7 │ 2019-09-19 00:00:00 │ 140.30 │ 142.37 │ 140.07 │ 141.07 │ 137.07 │ 35772100 │ 55.50 │ 55.94 │ 55.09 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 8 │ 2019-09-20 00:00:00 │ 141.01 │ 141.65 │ 138.25 │ 139.44 │ 135.48 │ 39167300 │ 55.35 │ 55.64 │ 54.37 │ -├───┼─────────────────────┼────────┼────────┼────────┼────────┼───────────┼──────────┼───────────┼───────────┼──────────┤ -│ 9 │ 2019-09-23 00:00:00 │ 139.23 │ 139.63 │ 138.44 │ 139.14 │ 135.19 │ 17139300 │ 54.74 │ 54.96 │ 54.41 │ -└───┴─────────────────────┴────────┴────────┴────────┴────────┴───────────┴──────────┴───────────┴───────────┴──────────┘ -``` - -Now we can run the same `BRNN` model with all `past_covariates` of both tickers. -The output will show the model grabbing all covariates to bring into training to -predict `close` of `MSFT`. - -``` -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Covariate #5: AAPL_open -Covariate #6: AAPL_high -Covariate #7: AAPL_low -Covariate #8: AAPL_close -Covariate #9: AAPL_adj_close -Covariate #10: AAPL_volume -Epoch 116: 100%|████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 149.26it/s, loss=-2.5, train_loss=-2.61, val_loss=-1.93] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 115/115 [00:01<00:00, 90.26it/s] -Block RNN model obtains MAPE: 3.93% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 259.78 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 258.15 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 261.44 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 257.57 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 257.99 │ -└────────────┴────────────┘ -``` - -image - -For one last experiment, we can perform some other feature engineering on `MSFT` -and add it to our `past_covariates` to train on. - -In this case, let's add in `Momentum` over past 10 days of `MSFT` and append it -to our `past_covariates` - -``` -(🦋) /forecast/ $ mom MSFT -Successfully added 'Momentum_10' to 'MSFT' dataset - -(🦋) /forecast/ $ brnn MSFT --forecast-only --all-past-covariates -The data contains inf or nan values. They will be removed. - -Covariate #0: open -Covariate #1: high -Covariate #2: low -Covariate #3: adj_close -Covariate #4: volume -Covariate #5: AAPL_open -Covariate #6: AAPL_high -Covariate #7: AAPL_low -Covariate #8: AAPL_close -Covariate #9: AAPL_adj_close -Covariate #10: AAPL_volume -Covariate #11: Momentum_10 -Epoch 71: 100%|████████████████████████████████████████████████████████████| 24/24 [00:00<00:00, 147.87it/s, loss=-2.52, train_loss=-2.57, val_loss=-1.82] -Predicting Block RNN for 5 days -100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 114/114 [00:01<00:00, 90.05it/s] -Block RNN model obtains MAPE: 3.72% - - - Actual price: 251.99 -┏━━━━━━━━━━━━┳━━━━━━━━━━━━┓ -┃ Datetime ┃ Prediction ┃ -┡━━━━━━━━━━━━╇━━━━━━━━━━━━┩ -│ 2022-09-14 │ 255.14 │ -├────────────┼────────────┤ -│ 2022-09-15 │ 256.20 │ -├────────────┼────────────┤ -│ 2022-09-16 │ 256.67 │ -├────────────┼────────────┤ -│ 2022-09-19 │ 258.63 │ -├────────────┼────────────┤ -│ 2022-09-20 │ 256.62 │ -└────────────┴────────────┘ -``` - -image - -There we have it. Bringing in another ticker has allowed us to further improve -the model accuracy. Furthermore, adding in a new feature to the dataset allowed -us to improve the accuracy further. - -**MAPE = 4.62%** (no past covariates) - -**MAPE = 4.26%** (`open`,`high`,`low`,`adj_close`,`volume`) - -**MAPE = 3.93%** -(`open`,`high`,`low`,`adj_close`,`volume`,`AAPL_open`,`AAPL_high`,`APPL_low`,`APPL_adj_close`,`APPL_volume`,`APPL_close`) - -**MAPE = 3.72%** -(`open`,`high`,`low`,`adj_close`,`volume`,`AAPL_open`,`AAPL_high`,`APPL_low`,`APPL_adj_close`,`APPL_volume`,`APPL_close`,`Momentum_10`) - -More workflows coming soon! - -If you have any questions or would like to request for new feature engineering -or model additions, please join us on discord. Happy hacking! diff --git a/website/content/terminal/guides/intros/reports.md b/website/content/terminal/guides/intros/reports.md deleted file mode 100644 index c8c59903dc9..00000000000 --- a/website/content/terminal/guides/intros/reports.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: Introduction to Reports -keywords: ['report'] -excerpt: "The Introduction to Reports" - ---- - -TODO = Proper Reports diff --git a/website/content/terminal/guides/intros/scripts.md b/website/content/terminal/guides/intros/scripts.md deleted file mode 100644 index 270419bb1ad..00000000000 --- a/website/content/terminal/guides/intros/scripts.md +++ /dev/null @@ -1,150 +0,0 @@ ---- -title: Scripts & Routines -keywords: ["scripts", "openbb", "routines", "script", "routine", ".openbb", "automation"] -excerpt: "The Scripts & Routines explain how to use .openbb scripts in detail -for every part of the terminal" - ---- - -The `.openbb` scripts offer the ability to automatically run a set of commands -in the form of a **routine**. Furthermore, the scripts can be adapted, and -documented, at any moment giving the user full control over the type of analysis -you wish to do (and repeat). This can fundamental research, understanding market -movements, finding hidden gems and even doing advanced statistical/econometric -research. - -{{< toc >}} - -## Explanation of scripts - -The script file below is titled _stocks_demo.openbb_. This file follows the -following logic: - -- Comments: any text after a `#` is referred to as a comment. This is - used to explain what is happening within the script and is not taking into - account when running terminal commands. -- Commands: any text **without** a `#` is being ran inside the OpenBB - Terminal. E.g. on the second line it says `stocks` thus within the OpenBB - Terminal the script will enter `stocks` and run this for you. - -These scripts have a 1-to-1 relationship with how you would normally use the -terminal. To get a better understanding of how the terminal is used, please see -Structure -of the OpenBB Terminal. - -``` -# Go into the stocks context -stocks - -# Load a company ticker, e.g. Amazon -load AMZN - -# Show a simple plot of the ticker, adding in any basic indicator. -candle --ma 20,30 - -# Switch over to the due diligence menu -dd - -# Show the price targets of Amazon, both the graph as well as the table -pt -pt --raw - -# Show estimates of annual and quarterly earnings as well as quarterly revenue estimates -est - -# Open the comparison analysis menu (ca) -../ca - -# Pick stocks that are competitors to Amazon -add ebay,wmt,tgt,baba,jd - -# Determine the valuations of each company -valuation - -# Show correlations between each company -hcorr - -# Figure out the sentiment perceived of each company -sentiment - -# Return to home -/ -``` - -## Executing a script - -By going to the main menu as depicted below (accessible with `home`), the -exe command can be used. With this command you -can run any `.openbb` script. These scripts are located where the application is -located inside the routines folder. - -![Routine Exe OpenBB Terminal](https://user-images.githubusercontent.com/46355364/174588513-5c52ea20-548a-4c2b-a4c1-6054e2d71786.png) - -Thus, using the earlier mentioned script, we can enter `exe stocks_demo.openbb` -which automatically runs all commands within the script file. Thus, it will -return a candle chart with moving averages, price targets from Analysts, -valuations of related companies, the correlation between these companies and the -market sentiment. This results in the following: - -![Script Stocks Demo OpenBB](https://user-images.githubusercontent.com/46355364/176903147-720eb2af-7e5d-40df-8ec6-7363cbc08430.png) - -### Custom arguments - -Next to that, it is also possible to add in custom arguments to your script -making the script more interactive and allow you to do the same analysis for -multiple companies. This is done in the following script: - -``` -# This script requires you to use arguments. This can be done with the following: -# exe example_with_inputs.openbb -i TSLA,AAPL,MSFT - -# Go to the stocks menu -stocks - -# Load a ticker, given the argument used. E.g. -i TSLA -load $ARGV[0] - -# Enter the Technical Analysis (ta) menu -ta - -# Show the fibonacci retracement levels -fib - -# Enter the comparison analysis (ca) menu -../ca - -# Set two extra tickers based on the arguments used. E.g. -i TSLA,AAPL,MSFT -set $ARGV[1],$ARGV[2] - -# Plot the historical prices -historical - -# Return to home -/ -``` - -This script includes `$ARGV[0]`, `$ARGV[1]` and `$ARGV[2]`. This means that the -script requires you to submit three arguments. In this case, they refer to -stock -tickers. Therefore, like the script also says, you can include these -arguments with `-i` followed by three tickers (e.g. -`exe example_with_inputs.openbb -i TSLA,AAPL,MSFT`). This results in the -following: - -![Example with Inputs OpenBB Script](https://user-images.githubusercontent.com/46355364/176903205-3cb55bf5-8710-4ad6-8eef-f9a99294ea3b.png) - -It is an incredibly simple script, but it gives an understanding what the -possibilities are. - -## Create your own script - -Scripts and routines reside in the `routines` folder and are automatically shown -when you type `exe` from the home screen (`home`). To create your own, you can -use the `template.openbb` file as a basis but you are free to duplicate one of -the demo files. - -Within this template, some examples are given, and you are free to change the -contents and rename the file. As long as the file remains in the same folder, -you will be able to find your file automatically as shown below: - -![Show Copy of Template OpenBB Script](https://user-images.githubusercontent.com/46355364/176903253-00a5b0f9-a6e7-49c7-a1d8-49ae819e28e3.png) diff --git a/website/content/terminal/index.md b/website/content/terminal/index.md index 40fbead7686..749a9f29b1a 100644 --- a/website/content/terminal/index.md +++ b/website/content/terminal/index.md @@ -9,11 +9,7 @@ financial terminal that will help you make more informed decisions, faster." import Card from "@site/src/components/General/Card"; -OpenBB is a leading open source investment research software platform for -accessing and analyzing financial market data. We represent millions of -investors who want to leverage state-of-the-art data science and machine -learning technologies to make sense of raw unrefined data. Our mission is to -make investment research effective, powerful and accessible to everyone. +OpenBB is a leading open source investment research software platform for accessing and analyzing financial market data. We represent millions of investors who want to leverage state-of-the-art data science and machine learning technologies to make sense of raw unrefined data. Our mission is to make investment research effective, powerful and accessible to everyone. --- @@ -30,25 +26,6 @@ make investment research effective, powerful and accessible to everyone. --- - - - - + + + + --- -Want to learn how to use OpenBB Terminal? Check out our -[Getting Started](/docs/getting-started) guide. Installation ? Check out our -[Installation](/docs/installation) guide. +### Obtaining support and giving feedback -How to import and export data? Check out our -[Data Import/Export](/docs/data-import-export) +Being an open source platform that wishes to tailor to the needs of any type of investor, we highly encourage anyone to share with us their experience and/or how we can further improve the OpenBB Terminal. This can be anything from a very small bug to a new feature to the implementation of a highly advanced Machine Learning model. -User routines? Check out our [User Routines](/docs/user-routines) +You are able to directly send us information about a bug or question/suggestion from inside the terminal by using the `support` command which is available everywhere in the terminal. Here you can select which command you want to report a bug, ask a question or make a suggestion on. When you press `ENTER` (⏎), you are taken to the Support form which is automatically filled with your input. You are only required to include the type (e.g. bug, suggestion or question) and message in the form although this can also be set directly from inside the terminal (see `support -h`). + +Support Command + +Alternatively, you can contact us via the following routes: + +- If you notice that a feature is missing inside the terminal, please fill in the Request a Feature form. +- If you wish to report a bug, have a question/suggestion or anything else, please fill in the Support form. +- If you wish to speak to us directly, please contact us via Discord. + +Want to learn how to install the OpenBB Terminal? Check out our [Installation](/quickstart/installation) guide. + +Want to know how to import and export data? Check out our [Data Import/Export](/guides/basics/data) + +Looking for personalized routines? Check out our [User Routines](/guides/advanced/scripts) [contributors-shield]: https://img.shields.io/github/contributors/OpenBB-finance/OpenBBTerminal.svg?style=for-the-badge [contributors-url]: https://github.com/OpenBB-finance/OpenBBTerminal/graphs/contributors @@ -86,4 +93,4 @@ User routines? Check out our [User Routines](/docs/user-routines) [license-shield]: https://img.shields.io/github/license/OpenBB-finance/OpenBBTerminal.svg?style=for-the-badge [license-url]: https://github.com/OpenBB-finance/OpenBBTerminal/blob/main/LICENSE [twitter-shield]: https://img.shields.io/twitter/follow/openbb_finance?style=for-the-badge&color=blue -[twitter-url]: https://twitter.com/openbb_finance +[twitter-url]: https://twitter.com/openbb_finance \ No newline at end of file diff --git a/website/content/terminal/quickstart/data.md b/website/content/terminal/quickstart/data.md deleted file mode 100644 index 9f5302d8f5b..00000000000 --- a/website/content/terminal/quickstart/data.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -sidebar_position: 6 -title: Importing and Exporting Data ---- - -## Importing and Exporting Data in the OpenBB Terminal - -Screen Shot 2022-10-13 at 6 45 01 PM - -The `OpenBBUserData` folder's default location is the home of the system user account. The location of this folder can be set by the user from the `/settings` menu. This folder is where exported files are saved, it is also where files being imported to the Terminal functions are kept. These include all things user-created: - - - Screener presets - - Portfolio files - - Exported files - - Files to be imported by various functions - - Styles and themes - - Preferred data sources - - Anonymized usage logs - -The settings menu diff --git a/website/content/terminal/quickstart/faq.md b/website/content/terminal/quickstart/faq.md index 66fa5dd37dd..c94f50284f6 100644 --- a/website/content/terminal/quickstart/faq.md +++ b/website/content/terminal/quickstart/faq.md @@ -2,14 +2,67 @@ title: FAQ sidebar_position: 3 --- - -# FAQ - -
Where does the data comes from? +
Where does the data from the OpenBB Terminal come from?

-OpenBB does not provide any data. OpenBB is a data aggregator that connects to -multiple data providers and provides a unified API to access the data. +The OpenBB platform does not provide any data itself as the platform is a data aggregator that connects to +almost a hundred different data providers and APIs to access the data. + +

+
+ +
What programs need to be allowed for Windows Firewall? +

+ +When issues arise regarding Windows Firewall, please allow the following applications through (if not already): + +- BranchCache +- Hyper-V +- VcXsrv +- Windows Terminal + +From the Windows Security menu, click on the Firewall & Network Protection tab, then click on "Allow an app through firewall". If the applications below are not allowed to communicate through Windows Defender Firewall, change the settings to allow. + +

+
+ +
On what Operating Systems can I run the OpenBB Terminal? +

+ +The OpenBB Terminal is compatible with Windows, Mac Os and Linux. Check the +[installation guide](/terminal/quickstart/installation) and [requirements outline](/terminal/quickstart/requirements) for more details. + +

+
+ +
How much hard drive space is required? +

+ +An installation will use approximately 2GB of space. + +

+
+ +
Why is data from today missing when I use the load function? +

+ +By default, the load function requests end-of-day daily data and is not included until the EOD summary has been published. The current day's data is considered intraday and is loaded when the `interval` argument is present. + +

+
+ +
How do I report a bug? +

+ +First, search the open issues for another report. If one already exists, attach any relevant information and screenshots as a comment. If one does not exist, start one with this [link](https://github.com/OpenBB-finance/OpenBBTerminal/issues/new?assignees=&labels=type%3Abug&template=bug_report.md&title=%5BBug%5D) + +

+
+ +
How can I get help with OpenBB Terminal? +

+ +You can get help with OpenBB SDK by joining our [Discord server](https://openbb.co/discord) or contact us in our support form [here](https://openbb.co/support).

diff --git a/website/content/terminal/quickstart/installation.md b/website/content/terminal/quickstart/installation.md index 0398dba1b4a..355675c5233 100644 --- a/website/content/terminal/quickstart/installation.md +++ b/website/content/terminal/quickstart/installation.md @@ -81,68 +81,80 @@ Right-Click the app and select Open. You will see a message saying Install with Docker Here are the steps to get OpenBBTerminal using the Docker containers that we provide: -Installing Docker and Docker Compose Pulling and running OpenBBTerminal Docker -Container Configuring your X-server to show plots Each of this steps need to be -followed to have a working version of OpenBBTerminal. +1. Installing `Docker` and `Docker Compose` +2. Pulling and running `OpenBBTerminal Docker Container` +3. Configuring your X-server to show plots + +Each of this steps need to be followed to have a working version of OpenBBTerminal. We will detail these steps in the rest of this document. -### 1. Installing Docker and Docker Compose +## 1. Installing `Docker` and `Docker Compose` -INSTALL DOCKER +**INSTALL DOCKER** -Installing Docker Desktop is the easiest way for most people: +Installing Docker Desktop is an alternative way and has a `Graphical User Interface` built in with `Docker Compose`. -It has a Graphical User Interface and comes with Docke Compose in it. +You can find `Docker Desktop` installation file for your `OS` here: [Download Docker Desktop](https://www.docker.com/products/docker-desktop) -You can find Docker Desktop installation file for your OS here: +**START DOCKER** -Download Docker Desktop +Once you have `Docker` installed and running, you can use the following command to check the state: -START DOCKER +```bash +docker info +``` -Once you have Docker installed and running, you can use the following command to -check the state: +It should output a text like this one: -docker info It should output a text like this one: +```text +Client: + Context: default + Debug Mode: false -Client: Context: default Debug Mode: false +Server: + Containers: 14 + Running: 2 + Paused: 1 + Stopped: 10 +``` -Server: Containers: 14 Running: 2 Paused: 1 Stopped: 10 ... If you see a message -like the following, it most likely means you need to start Docker. +If you see a message like the following, it most likely means you need to start `Docker`. Open the docker desktop app in this case. -Server: ERROR: Cannot connect to the Docker daemon at -unix:///var/run/docker.sock. Is the docker daemon running? +```text +Server: +ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. +Is the docker daemon running? +``` -### 2. Pulling and running OpenBBTerminal Docker Container +## 2. Pulling and running `OpenBBTerminal Docker Container` -DOCKER COMPOSE +**DOCKER COMPOSE** -Here are the commands to use Docker Compose to pull and run the OpenBBTerminal -Docker Container: +Here are the commands to use `Docker Compose` to pull and run the `OpenBBTerminal Docker Container`: ```bash curl -o docker-compose.yaml https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml + docker compose run poetry ``` -The command line with curl is downloading this file : docker-compose.yaml. +The command line with `curl` is downloading this file : [`docker-compose.yaml`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml). -The docker-compose.yaml file is a configuration file telling Docker Compose: +The `docker-compose.yaml` file is a configuration file telling `Docker Compose`: -where to find OpenBBTerminal Docker Container how to run this container. The -second command runs Docker Compose on the service poetry defined in this -docker-compose.yaml file. +- where to find `OpenBBTerminal Docker Container` +- how to run this container. -This second command must be run in the same folder in which docker-compose.yaml -file is. +The second command runs `Docker Compose` on the service `poetry` defined in this `docker-compose.yaml` file. -DOCKER +This second command must be run in the same folder in which `docker-compose.yaml` file is. -If you don't have Docker Compose you can also use Docker directly to run the -OpenBBTerminal Docker Container. +**DOCKER** + +If you don't have `Docker Compose` you can also use `Docker` directly to run the `OpenBBTerminal Docker Container`. Here is the commands to run: @@ -152,51 +164,57 @@ docker pull ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z docker run -v ~/.openbb_terminal/:/home/python/.openbb_terminal -v ~/OpenBBUserData:/home/python/OpenBBUserData -it --rm ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z ``` -Be sure to replace X.Y.Z with the version you want to pull and run. +Be sure to replace `X.Y.Z` with the version you want to pull and run. Note for windows: - Ignore this message if you are using Powershell or a more evolved interpreters. +```text + Ignore this message if you are using Powershell or a more evolved interpreters. If you are using the builtin Windows interpreter. Replace `~` by `%USERPROFILE%` in the command above. +``` -### 3. Configuring your X-server to show plots +## 3. Configuring your X-server to show plots -In order to display plots in the docker container, we need to configure the -XServer on the host machine. Without this configuration the interactive charts -will not be displayed. +In order to display plots in the docker container, we need to configure the XServer on the host machine. Without this configuration the interactive charts will not be displayed. -On Windows Download and install : VcXsrv +### On Windows + +Download and install : [VcXsrv](https://sourceforge.net/projects/vcxsrv/) When running the program is important to check "Disable access control" -Run Docker Compose like this: +Run `Docker Compose` like this: ```bash curl -o docker-compose.yaml https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/docker/docker-compose.yaml + docker compose run poetry ``` -Or run Docker directly: +Or run `Docker` directly: ```bash docker run -v ~/.openbb_terminal:/home/python/.openbb_terminal -v ~/OpenBBUserData:/home/python/OpenBBUserData -it --rm --env DISPLAY=host.docker.internal:0.0 ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z ``` -X-Server on macOS Users familiar with Docker and X-Server can set the DISPLAY -variable in the file setenv described above. If you use this approach remember -to add :0 at the end of your inet address. E.g. DISPLAY=192.168.1.155:0. +### X-Server on macOS + +Users familiar with Docker and X-Server can set the `DISPLAY` variable in the file [setenv](/docker/setenv) described above. If you use this approach remember to add `:0` at the end of your inet address. E.g. `DISPLAY=192.168.1.155:0`. For help setting up the X-Server continue reading: -Setting up X Quartz/X11 On macOS the X11 client of choice is XQuartz. On Windows -it's Xming. XQuartz will be used as an example further on. +#### Setting up X Quartz/X11 -Install X Quartz from https://www.xquartz.org/ With X Quartz open: go to -Preferences -> Security and make sure both options are enabled. -image -Adding the display for Docker From the command prompt or terminal, run the -following to add your local configuration to the list of allowed access control: +On macOS the X11 client of choice is [XQuartz](https://www.xquartz.org/). On Windows it's [Xming](http://www.straightrunning.com/XmingNotes/). XQuartz will be used as an example further on. + +0. Install X Quartz from +1. With X Quartz open: go to Preferences -> Security and make sure both options are enabled. + ![Screen Shot 2021-09-08 at 12 21 48 PM](https://user-images.githubusercontent.com/18151143/132548605-235d774b-9aa6-4a45-afcf-58fb775d376a.png) + +#### Adding the display for Docker + +From the command prompt or terminal, run the following to add your local configuration to the list of allowed access control: ```bash IP=$(ifconfig | grep inet | grep -v -e "127.0.0.1" | awk '$1=="inet" {print $2}') @@ -209,15 +227,17 @@ Now we can run the docker container, adding the display to the environment: docker run -v ~/.openbb_terminal/:/home/python/.openbb_terminal -v ~/OpenBBUserData:/home/python/OpenBBUserData -it --rm --env-file=path/to/setenv --env DISPLAY=$IP:0 ghcr.io/openbb-finance/openbbterminal-poetry:X.Y.Z ``` -This container will be able to display all the same plots as the terminal -interface. +This container will be able to display all the same plots as the terminal interface. -X-Server on Linux Desktop X-Server is default in Linux distribution. There is no -need to install any clients. +### X-Server on Linux Desktop -Local docker container We can use IPC socket to connect Desktop. +X-Server is default in Linux distribution. There is no need to install any clients. -Add this setting to your .env file. +#### Local docker container + +We can use IPC socket to connect Desktop. + +Add this setting to your `.env` file. ```bash OPENBB_BACKEND=Qt5Agg @@ -231,7 +251,7 @@ docker run -it --rm --name openbb --env-file=./.env -e DISPLAY=$DISPLAY -v /tmp/ xhost -local: ``` -If you're using remote docker host, you can connect with `ssh -X `. +If you're using remote docker host, you can connect with "ssh -X ". Then run the previous docker command. @@ -239,11 +259,7 @@ Then run the previous docker command. This installation type supports both Windows and Unix systems (Linux + MacOS). -**NOTE for Windows users:** Some _not all_ Windows users would prefer to use an environment -similar to what Linux and macOS users use. In this case it is easier to use Windows Subsystem -for Linux (WSL). WSL emulates a Linux machine inside your Windows system. If this is the case - -jump to the Installing WSL (Only for Windows users) -section before proceeding. +**NOTE for Windows users:** Some _not all_ Windows users would prefer to use an environment similar to what Linux and macOS users use. In this case it is easier to use Windows Subsystem for Linux (WSL). WSL emulates a Linux machine inside your Windows system. If this is the case - jump to the Installing WSL (Only for Windows users) section before proceeding. ### Installing the terminal @@ -251,11 +267,7 @@ These steps are common in all operating systems (Windows with or without WSL, Ma This project supports Python 3.8 and 3.9. By default, the newly created virtual environment will use Python 3.9.13 -Our current recommendation is to use this project with Anaconda's Python distribution - either full -[**Anaconda3 Latest**](https://www.anaconda.com/products/distribution) or -[**Miniconda3 Latest**](https://docs.conda.io/en/latest/miniconda.html) (recommended). -Several features in this project utilize Machine Learning. Machine Learning Python dependencies are optional. For MacOS systems, the "Miniconda3 MacOSX 64-bit" version that works on both Intel and M1 -macs is recommended. +Our current recommendation is to use this project with Anaconda's Python distribution - either full [**Anaconda3 Latest**](https://www.anaconda.com/products/distribution) or [**Miniconda3 Latest**](https://docs.conda.io/en/latest/miniconda.html) (recommended). Several features in this project utilize Machine Learning. Machine Learning Python dependencies are optional. For MacOS systems, the "Miniconda3 MacOSX 64-bit" version that works on both Intel and M1 macs is recommended. **NOTE:** We recommend using `conda` and `poetry` because it just works. You can use other python distributions and use raw `pip` instead of `poetry` but you will very likely bump into installation @@ -274,10 +286,7 @@ have the dependencies working straight away. **ONLY REQUIRED ON WINDOWS IF NOT USING WSL**, Install/update Microsoft C++ Build Tools from here: - **NOTE for macOS users:** The link above gets you the Intel version of miniconda meaning if you're on an - Apple Silicon powered machine you will be using the terminal through Apple's rosetta2 layer. We recommend - sticking to this distribution for better compatibility until the dependency developers fully catch up with - Apple's transition to Apple Silicon. + **NOTE for macOS users:** The link above gets you the Intel version of miniconda meaning if you're on an Apple Silicon powered machine you will be using the terminal through Apple's rosetta2 layer. We recommend sticking to this distribution for better compatibility until the dependency developers fully catch up with Apple's transition to Apple Silicon. - After following the steps, confirm that you have it by opening a terminal and running: `conda -V`. The output should be something along the lines of: `conda 22.9.0` @@ -311,8 +320,7 @@ brew install cmake **On Windows:** -If you have followed the instructions in step 1 of this guide CMake was installed as a -part of you Microsoft C++ Build Tools +If you have followed the instructions in step 1 of this guide CMake was installed as a part of you Microsoft C++ Build Tools #### 3. Install git @@ -333,13 +341,9 @@ cd OpenBBTerminal/ #### 6. Create Environment -You can name the environment whatever you want. Although you could use names such as: -`welikethestock`, `thisistheway` or `diamondhands`, we recommend something simple and -intuitive like `obb`. This is because this name will be used from now onwards. +You can name the environment whatever you want. Although you could use names such as: `welikethestock`, `thisistheway` or `diamondhands`, we recommend something simple and intuitive like `obb`. This is because this name will be used from now onwards. -Please note, the following setup has been confirmed to work for all OS (including M1) -with the standard miniconda distribution. If you are using a different distribution, -you will need to install it manually before proceeding. +Please note, the following setup has been confirmed to work for all OS (including M1) with the standard miniconda distribution. If you are using a different distribution, you will need to install it manually before proceeding. ```bash conda env create -n obb --file build/conda/conda-3-9-env.yaml @@ -369,8 +373,7 @@ Install the main dependencies with poetry install ``` -You are good to go with the core of the OpenBB Terminal. To install additional toolkits -proceed with the following commands: +You are good to go with the core of the OpenBB Terminal. To install additional toolkits proceed with the following commands: To install the Portfolio Optimization Toolkit run: @@ -402,26 +405,17 @@ Or if you are old-fashioned run: python terminal.py ``` -**NOTE:** When you close the terminal and re-open it, the only command you need to re-call -is `conda activate obb` before you call `openbb` again. +**NOTE:** When you close the terminal and re-open it, the only command you need to re-call is `conda activate obb` before you call `openbb` again. -**TROUBLESHOOT:** If you are having troubles to install, check out the -[troubleshoot page](https://github.com/OpenBB-finance/OpenBBTerminal/blob/master/TROUBLESHOOT.md). - -You can also reach for help on our [discord](https://discord.gg/Up2QGbMKHY). +**TROUBLESHOOT:** If you are having troubles to install, check out the [troubleshoot page](/terminal/quickstart/troubleshooting). You can also reach for help on our [discord](https://discord.gg/Up2QGbMKHY). ## Advanced User Install - Custom installation procedures -By default we advice using `conda` and `poetry` for environment setup and dependency management. -Conda ships binaries for packages like `numpy` so these dependencies are not built from source locally by `pip`. -Poetry solves the dependency tree in a way that the dependencies of dependencies of dependencies -use versions that are compatible with each other. +By default we advice using `conda` and `poetry` for environment setup and dependency management. Conda ships binaries for packages like `numpy` so these dependencies are not built from source locally by `pip`. Poetry solves the dependency tree in a way that the dependencies of dependencies of dependencies use versions that are compatible with each other. -If you are using a conda environment the `build/conda` folder contains multiple `.yaml` configuration -files that you can choose from. +If you are using a conda environment the `build/conda` folder contains multiple `.yaml` configuration files that you can choose from. -If you are using other python distributions we highly recommend that you use some virtual -environment like `virtualenv` or `pyenv` for installing the terminal dependency libraries. +If you are using other python distributions we highly recommend that you use some virtual environment like `virtualenv` or `pyenv` for installing the terminal dependency libraries. Requirements files that you can find in the project root: @@ -436,19 +430,13 @@ You can install them with with pip The dependency tree is solved by poetry. -Note: The libraries specified in the requirements files have been tested and work for -the purpose of this project, however, these may be older versions. Hence, it is recommended -for the user to set up a virtual python environment prior to installing these. This allows -to keep dependencies required by different projects in separate places. +Note: The libraries specified in the requirements files have been tested and work for the purpose of this project, however, these may be older versions. Hence, it is recommended for the user to set up a virtual python environment prior to installing these. This allows to keep dependencies required by different projects in separate places. ### Installing WSL (Only for Windows users) -If you are using Windows you first you need to install WSL. The process is simple and a tutorial can be found [here](https://www.sitepoint.com/wsl2/). -Once you reach the section **Update Linux** on that tutorial, you should have a linux machine installed and can proceed -to the next steps. +If you are using Windows you first you need to install WSL. The process is simple and a tutorial can be found [here](https://www.sitepoint.com/wsl2/). Once you reach the section **Update Linux** on that tutorial, you should have a linux machine installed and can proceed to the next steps. -Since WSL installation is headless by default (i.e., you have only access to a terminal running a linux distribution) -you need some extra steps to be able to visualize the charts produced by the terminal (more detailed tutorial [here](https://medium.com/@shaoyenyu/make-matplotlib-works-correctly-with-x-server-in-wsl2-9d9928b4e36a)): +Since WSL installation is headless by default (i.e., you have only access to a terminal running a linux distribution) you need some extra steps to be able to visualize the charts produced by the terminal (more detailed tutorial [here](https://medium.com/@shaoyenyu/make-matplotlib-works-correctly-with-x-server-in-wsl2-9d9928b4e36a)): 1. Dynamically export the DISPLAY environment variable in WSL2: @@ -466,34 +454,27 @@ After this, `VcXsrv` should be running successfully and we can proceed to termin ## Update Terminal -The terminal is constantly being updated with new features and bug fixes, hence, for your terminal to be update, -you can run: +The terminal is constantly being updated with new features and bug fixes, hence, for your terminal to be update, you can use `git pull` to get the latest changes: ```bash git pull ``` -to get the latest changes. - If this fails due to the fact that you had modified some python files, and there's a conflict with the updates, you can use: ```bash git stash ``` -Then, re-run `poetry install` to get any new dependencies. - -Once installation is finished, you're ready to openbb. - -If you `stashed` your changes previously, you can un-stash them with: +Then, re-run `poetry install` to get any new dependencies. Once installation is finished, you're ready to use the OpenBB Terminal again. If you `stashed` your changes previously, you can un-stash them with: ```bash git stash pop ``` -**NOTE:** When you close the terminal and re-open it, the only command you need to re-call is `conda activate gst` +**NOTE:** When you close the terminal and re-open it, the only command you need to re-call is `conda activate obb` before you call `openbb` again. -::: +::: \ No newline at end of file diff --git a/website/content/terminal/quickstart/menus.md b/website/content/terminal/quickstart/menus.md deleted file mode 100644 index d240c6a3c65..00000000000 --- a/website/content/terminal/quickstart/menus.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -sidebar_position: 3 -title: Menus ---- - -### Explanation of Menus - -Menus, depicted in Dark Blue, take you to a new section of the terminal referred -to as a menu. For example, if you wish to view information about stocks, you can do so by typing `stocks` and pressing `ENTER` (⏎). -This opens a new menu as depicted below. - -Stocks Menu - -Depending on the menu you are in, you are presented with a new set of commands (see Explanation of Commands) -and menus you can select. There are interactions in place between each menu. For example, when selecting a company within -the `stocks` menu, the terminal will remember your selection when you visit the `fa` or `options` menu. -See Introduction to Stocks for more information. - ---- - -**Pro tip:** you can quickly jump between menus by using a forward slash (`/`). For example, if I want to access the options -menu, I can type `/stocks/options` to instantly arrive at this menu. You can do this from any location within the -OpenBB Terminal! diff --git a/website/content/terminal/quickstart/requirements.md b/website/content/terminal/quickstart/requirements.md new file mode 100644 index 00000000000..47a3d03f7e8 --- /dev/null +++ b/website/content/terminal/quickstart/requirements.md @@ -0,0 +1,15 @@ +--- +title: Requirements +sidebar_position: 1 +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; + +Installation of the OpenbB Terminal requires downloading around 500mb of data. Querying data (by running commands) does not require a lot of bandwidth but you will certainly have a more pleasant experience if you will be on a fast internet line. 4G networks provide a good enough experience so if you're traveling your personal hot-spot will do. While it's technically possible to use a subset of the functionality in off-line mode, you will not be able to use any data that is queried from the APIs of data providers and services. + +### Minimum System Requirements + +- 4GB RAM or more +- 5GB free storage +- Internet connection \ No newline at end of file diff --git a/website/content/terminal/quickstart/scripts.md b/website/content/terminal/quickstart/scripts.md deleted file mode 100644 index 79c7f77e411..00000000000 --- a/website/content/terminal/quickstart/scripts.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -sidebar_position: 5 -title: Scripts / Routines ---- - -### Explanation of Scripts - -The `.openbb` scripts offer the ability to automatically run a set of commands in the form of a **routine**. Furthermore, -the scripts can be adapted, and documented, at any moment giving the user full control over the type of analysis you wish -to do (and repeat). This can fundamental research, understanding market movements, finding hidden gems and even -doing advanced statistical/econometric research. - -For a thorough guide on how to setup these files, please see the Scripts & Routines guide. - ---- - -## Importing and Exporting Data in the OpenBB Terminal - -Screen Shot 2022-10-13 at 6 45 01 PM - -The `OpenBBUserData` folder's default location is the home of the system user account. The location of this folder can be set by the user from the `/settings` menu. This folder is where exported files are saved, it is also where files being imported to the Terminal functions are kept. These include all things user-created: - - - Screener presets - - Portfolio files - - Exported files - - Files to be imported by various functions - - Styles and themes - - Preferred data sources - - Anonymized usage logs - -The settings menu diff --git a/website/content/terminal/quickstart/troubleshooting.md b/website/content/terminal/quickstart/troubleshooting.md new file mode 100644 index 00000000000..4f420795aa7 --- /dev/null +++ b/website/content/terminal/quickstart/troubleshooting.md @@ -0,0 +1,119 @@ +--- +sidebar_position: 4 +title: Troubleshooting +--- +If you are visiting this page it means that you're having issues installing. We apologize for that. Since the start of the project we've come across different types of issues experienced by the users. This page tries to combine issues, and their solutions. This will allow to give the best install experience to everyone regardless of any programming skills. + +**Note:** these instructions are meant for users that use the source version (from [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal)) + +## Standard Install Troubles + +### Microsoft Visual C++ + +If you are on Windows and your terminal has too many red error lines, it is likely that this is the issue. Go to this page and install the 2019 Build Tools (not the latest) found [here](https://answers.microsoft.com/en-us/windows/forum/windows_other-windows_install/microsoft-visual-c-140/6f0726e2-6c32-4719-9fe5-aa68b5ad8e6d) + +### Wheel + +If you receive any notifications regarding `wheel` missing, this could be due to this dependency missing. + +`conda install -c conda-forge wheel` or `pip install wheel` + +### Poetry + +If you get errors about .whl files not existing (usually on Windows) you have to reinitialize the following folder. +Just removing the 'artifacts' folder could also be enough: + +| Platform | Location | +| -------- | ------------------------------- | +| Linux | "~/.cache/pypoetry" | +| Mac | "~/Library/Caches/pypoetry" | +| Windows | "%localappdata%/pypoetry/cache" | + +When you try to add a package to Poetry it is possible that it causes a similar issue. Here you can remove the +'artifacts' folder again to reinitialize Poetry. + +If you run into trouble with Poetry, and the advice above did not help, your best bet is to try + +1. `poetry update --lock` + +2. `conda deactivate` -> `conda activate obb`, then try again + +3. Track down the offensive package and purge it from your anaconda `` folder, then try again + (removing through conda can sometimes leave locks behind) + + | Platform | Location | + | --------- | -------------------------------------------- | + | Linux/Mac | "~/anaconda3/envs" or "~/opt/anaconda3/envs" | + | Windows | "%userprofile%/anaconda3/envs" | + +4. Completely nuke your conda environment folder and make a new environment from scratch + +5. Reboot your computer and try again + +6. Submit a ticket on GitHub + +#### JSONDecodeError during `poetry install` + +Sometimes poetry can throw a `JSONDecodeError` on random packages while running `poetry install`. +This can be observed on macOS 10.14+ running python 3.8+. +This is because of the use of an experimental installer that can be switched off to avoid the mentioned error. +Run + +```bash +poetry config experimental.new-installer false +``` + +as advised [here](https://github.com/python-poetry/poetry/issues/4210) and it should fix the installation process. + +## ModuleNotFoundError Trouble + +In the case when you run into an error of the form `ModuleNotFoundError: No module named '_______'` before you start installing these modules that have not been found please check that you have most followed the recommended installation instructions. +These errors often can occur when you have not activated the virtual environment where you have installed the terminal, or you have not used the `poetry install` command to install the dependencies. + +In case you wish to proceed with an alternative way to install the terminal feel free to install the missing packages via pip. For example if you get the error that `yfinance` is not found, you would run `pip install yfinance` + +## Machine Learning Troubles + +If you run into issues installing or `Cannot convert a symbolic Tensor...` at runtime, try this: + +```bash +poetry install +poetry install -E prediction +``` + +_Commands that may help you in case of an error:_ + +- `python -m pip install --upgrade pip` +- `poetry update --lock` +- `poetry install` + +## CRLF versus LF + +When trying to commit code changes, pylint will prevent you from doing so if your line break settings are set to +CRLF (default for Windows). +This is because the entire package uses LF (default for Linux/Mac), and it is therefore +important that you change this setting to LF _before_ you make any changes to the code. + +It is possible that CRLF automatically turns back on, you can correct this with: + +```bash +git config --global core.autocrlf false +``` + +In case you already made coding adjustments, you have to reset your cache, and the changes you made to the code with +the following: + +```bash +git rm --cached -r . +git reset --hard +``` + +## Unable to run openbb from VS Code integrated terminal + +Occurs when vscode terminal python version/path is different from the terminal version. + +To fix it add this to vscode JSON settings ([ref](https://stackoverflow.com/questions/54582361/vscode-terminal-shows-incorrect-python-version-and-path-launching-terminal-from)): + +```bash + "terminal.integrated.inheritEnv": false, +``` \ No newline at end of file diff --git a/website/src/theme/Footer/index.js b/website/src/theme/Footer/index.js index 56c63cc8557..68e894f525e 100644 --- a/website/src/theme/Footer/index.js +++ b/website/src/theme/Footer/index.js @@ -295,7 +295,7 @@ const FOOTER_CONTENT = { label: "OpenBB Terminal", }, { - href: "https://openbb-finance.github.io/OpenBBTerminal/", + href: "https://docs.openbb.co/terminal", label: "Terminal Docs", }, {