Files
OpenBB/openbb_terminal
montezdesousa 732a614937 Allow users to change api keys through OpenBB SDK (#2642)
* set fred keys through api

* linting

* doc change

* expose api keys dict

* allow optional persistency of api key change

* black

* place decorators

* add keys paths

* av and fmp keys

* some fixes

* rewrite av, fmp, fred cassettes

* quandl key

* polygon key

* rewrite polygon cassette

* news and tradier

* cmc key

* change some docstring

* change some docstring

* filtered keys dict on mykeys

* simplify mykeys collection

* revert arg to persist

* finnhub

* iex key

* reddit key

* reddit docstring

* check for prefix before removing

* glassnode key

* glassnode virtual path

* bitquery keys

* twitter key

* robinhood key

* degiro keys

* oanda key

* binance keys

* si key

* coinbase key

* refactor btcrb dates and logic

* walert key

* coinglass key

* regenerate cassettes

* cpanic keys

* ethplorer keys

* replace ouput status failed msg

* smartstake keys

* gitub keys

* messari keys

* eodhd keys

* santiment keys

* remove unused imports

* typo formatting test

* remove forbidden arg

* fix btcrb test

* fix test import

* revert cassette changes

* fix more tests

* fix last failing

* black

* try to change status to int code

* change status to int

* first try at keys controller refactoring

* fix some tests accordingly

* display msg on set

* change some vars

* remove elifs

* add progress bar and remove duplicate key check

* add mykeys and refresh to menu

* remove unused import

* change mykeys output to df

* remove check keys from keys ctrl tests

* update test split to new structure

* reduce duplicate code

* upper case

* change var name

* rename mykeys df cols

* add keys integrations tests

* add mykeys test to keys controller

* fix mykeys

* fix wrong eodhd var name

* pylint

* test_set_key

* black

* patch dotenv kinda

* fix mypy

* remove refresh

* filter mykeys df

* filter mykeys df

* update integration tests

* first unit test for set api key

* add 2 more test cases to fred

* add persist test to fred unittest

* add av unittest

* add polygon unittest

* generate av cassettes

* add unittests av, fmp, quandl, polygon, fred

* change si token to key

* forgot controller func

* unittest: news, tradier, cmc, finnhub, iex, bitquery

* add si unittest

* update eodhd key name

* add unittest: walert, glassnode, coinglass, cpanic, ethplorer, github, messari, eodhd, santiment

* black

* add reddit unittest

* refactor unittests to handle multi args

* regenerate cassettes

* add more unittests

* add degiro, oanda and rh unittest

* generate oanda records

* fix rh and degiro unittest

* fix oanda unittest

* black

* flake8

* fix si controller test

* move tmp

* pylint

* fix av test

* fix fmp key check

* rewrite fmp

* rewrite oanda output

* fix quandl check

* rewrite expected

* supress logger exception

* fix iex cloud key

* remove iex warning

* fix coinglass check key

* cpanic fix

* regenerate cpanic cassettes

* flake8

* fix smartstake keys check

* regenerate expected output

* fix eodhd

* try at binance

* black

* fix controller tests

* pylint

* disable warning

* flake

* add option to show keys

* add Enum to model

* regenerate test output

* update contributing

* flake

* fix type hint

* remove status msg dict

* pylint

* regenerate and remove unecessary cassettes

* remove duplicate return

* supress traceback

* remove unused imports

* update some docstrings

* add todo

* add export option to mykeys

* minor change

* refactor color scheme

* update contributing

* update contributing

* regenerate cassettes

* fix check output

* small msg change

* small details

* rewrite some tests

Co-authored-by: hjoaquim <h.joaquim@campus.fct.unl.pt>
Co-authored-by: Colin Delahunty <72827203+colin99d@users.noreply.github.com>
2022-10-08 12:30:23 +02:00
..
2022-10-04 20:51:57 -04:00
2022-10-06 17:56:29 -04:00
2022-10-04 01:06:53 +02:00
2022-10-05 10:24:24 +01:00
2022-09-28 15:38:34 +02:00
2022-10-06 17:56:29 -04:00
2022-09-30 11:58:20 +02:00
2022-10-06 17:56:29 -04:00
2022-09-07 12:14:17 +01:00
…
2022-10-06 17:56:29 -04:00
2022-10-06 17:56:29 -04:00
2022-10-06 17:56:29 -04:00

OpenBB Terminal

Table of Contents

  1. Anaconda & Python Installation
  2. Docker Installation
  3. Docker Web UI Installation
  4. Raspberry-Pi-Installation - Portable GST
  5. API Keys

There are currently four options to install the terminal:

First step in all options is to star the project

OpenBB Terminal GitHub Stars

Anaconda & Python

This installation type supports both Windows and Unix systems (Linux + MacOS). However, on Windows it can become messy so it is easier to use Windows Subsystem Linux (WSL) on this operating system. WSL emulates a Linux machine inside your Windows system.

If you are using macOS or other Linux operating systems you can jump the next section Installing the terminal.

Installing WSL (Only for Windows users)

If you are using Windows you first you need to install WSL. The process is simple and a tutorial can be found here. Once you reach the section Update Linux on that tutorial, you should have a linux machine installed and can proceed to the next steps.

Since WSL installation is headless by default (i.e., you have only access to a terminal running a linux distribution) you need some extra steps to be able to visualize the charts produced by the terminal (more detailed tutorial here):

  1. Dynamically export the DISPLAY environment variable in WSL2:

    # add to the end of ~/.bashrc file
    export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):0
    # source the file
    source ~/.bashrc
    
  2. Download and install VcXsrv

  3. When running the program is important to check "Disable access control"

After this, VcXsrv should be running successfully and we can proceed to terminal installation.

Although we extremely recommend using WSL to run the terminal on windows, if you don't want or can't for some reason, you can try install the terminal directly on Windows without WSL. If you'd like to see a video recording of the installation on Windows without WSL, @JohnnyDankseed has made one available here.

Installing the terminal

These steps are common in all operating systems (Windows with or without WSL, MacOS or Linux).

This project supports Python 3.8 and 3.9. By default, the newly created virtual environment will use Python 3.9.13

Our current recommendation is to use this project with Anaconda's Python distribution - either full Anaconda3 Latest or Miniconda3 Latest (recommended). Several features in this project utilize Machine Learning. Machine Learning Python dependencies are optional. For MacOS systems, the "Miniconda3 MacOSX 64-bit" version that works on both Intel and M1 macs is recommended.

NOTE: We recommend using conda and poetry because it just works. You can use other python distributions and use raw pip instead of poetry but you will very likely bump into installation issues.

  1. Install Anaconda (It's on the AUR as anaconda or miniconda3!)

  2. Install git

    conda install -c anaconda git
    

    For Linux users only, run these additional commands:

    sudo apt update
    sudo apt install -y cmake gcc
    pip install cmake
    
  3. Clone the Project

    • Via HTTPS: git clone https://github.com/OpenBB-finance/OpenBBTerminal.git
    • via SSH: git clone git@github.com:OpenBB-finance/OpenBBTerminal.git
  4. Navigate into the project's folder

    cd OpenBBTerminal/
    
  5. Create Environment

    You can name the environment whatever you want. Although you could use names such as: welikethestock, thisistheway or diamondhands, we recommend something simple and intuitive like obb. This is because this name will be used from now onwards.

    conda env create -n obb --file build/conda/conda-3-9-env.yaml
    

    Note: Using python 3.10 can lead to undesirable functionality for certain commands.

  6. Activate the virtual environment

    conda activate obb
    

    Note: At the end, you can deactivate it with: conda deactivate.

  7. Install dependencies with poetry

    Install the main dependencies with

    poetry install
    

    To enable the prediction menu install additional dependencies after installing main dependencies:

    conda install -c conda-forge tensorflow
    

    If you are having trouble with Poetry (e.g. on a non-conda python), simply install requirements.txt with pip

    pip install -r requirements.txt
    
  8. You're ready to use the terminal!

    python terminal.py
    

    If you are using macOS and if you get SyntaxError, you might need to run this instead

    python3 terminal.py
    
  9. (Windows - Optional and only if you are not using WSL) Speeding up opening process in the future

    After you've installed OpenBB Terminal, you'll find a file named "OpenBB Terminal.bat". You can use this file to open OpenBB Terminal quicker. This file can be moved to your desktop if you'd like. If you run into issues while trying to run the batch file. If you run into issues with the batch files, edit the file and check to see if the directories match up. This file assumes you used the default directories when installing.

  10. Jupyter Lab (Optional. Early alpha). User the Terminal from Jupyter Lab

    You can install Jupyter Lab extensions that help you manage settings and launch the terminal in a JL bash console using the commands in the jupyterlab/README.md

NOTE: When you close the terminal and re-open it, the only command you need to re-call is conda activate gst before you call python terminal.py again.

TROUBLESHOOT: If you are having troubles to install, check our newest troubleshoot page. You can also reach for help on our discord.

Advanced User Install - Custom installation procedures

By default we advice using conda and poetry for environment setup and dependency management. Conda ships binaries for packages like numpy and tensorflow so these dependencies are not built from source locally by pip. Poetry solves the dependency tree in a way that the dependencies of dependencies of dependencies use versions that are compatible with each other.

If you are using a conda environment the build/conda folder contains multiple .yaml configuration files that you can choose from.

If you are using other python distributions we highly recommend that you use some virtual environment like virtualenv or pyenv for installing the terminal dependency libraries.

Requirements files that you can find in the project root:

  • requirements.txt list all the dependencies without Machine Learning libraries
  • requirements-full.txt list all the dependencies without Machine Learning libraries

The dependency tree is solved by poetry.

Note: The libraries specified in the requirements files have been tested and work for the purpose of this project, however, these may be older versions. Hence, it is recommended for the user to set up a virtual python environment prior to installing these. This allows to keep dependencies required by different projects in separate places.

Update Terminal

The terminal is constantly being updated with new features and bug fixes, hence, for your terminal to be update, you can run:

git pull

to get the latest changes.

If this fails due to the fact that you had modified some python files, and there's a conflict with the updates, you can use:

git stash

Then, re-run poetry install or pip install -r requirements.txt to get any new dependencies.

Once installation is finished, you're ready to openbb.

If you stashed your changes previously, you can un-stash them with:

git stash pop

Docker Installation

  1. Make sure docker desktop is installed. Install links can be found here. To confirm that your docker desktop is downloaded and running, open a command prompt or terminal and enter docker info. If you get the following you are not running the docker desktop:

    Server:
    ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock.
    Is the docker daemon running?
    

    Open the docker desktop app in this case.

  2. Download the latest docker image.

    docker pull ghcr.io/openbb-finance/openbbterminal-poetry:latest
    

    Upon running this the first time, you should see the various layers downloading (note the random series of letters numbers will vary). The first time this is run, it will take a few minutes. Subsequent updates will be much faster, as the changes will be in the MB instead of GB.

    Screen Shot 2021-09-08 at 10 41 08 AM

    Once the download is complete, confirm that the image has been created by doing docker images. You should see something similar to

    REPOSITORY                             TAG       IMAGE ID       CREATED        SIZE
    ghcr.io/openbb-finance/openbbterminal-poetry   latest    b1409a304750   42 hours ago   2.29GB
    
  3. Run a container

    To launch the container you need to run the following command:

    docker run -v ~/.openbb_terminal/:/home/python/.openbb_terminal -v ~/OpenBBUserData:/home/python/OpenBBUserData -it --rm ghcr.io/openbb-finance/openbbterminal-poetry:1.6.0

    NOTE FOR WINDOWS

    Replace ~ by %USERPROFILE% in the command above. This applies if you are using the builtin Windows interpreter. You won't have to do this inside Powershell or more evolved interpreters.

    At this point, you should be able to use the majority of the features using Docker. To read more on how to configure your X-server to show plots, hop over to the Advanced Docker Setup.

Web UI - Docker

  1. Ensure Docker is installed.
  2. Navigate to the location of the Dockerfile in the repo (cd openbb_terminal_web)
  3. Ensure the launcher is executable with chmod +x ./launch
  4. Launch it with ./launch. If you get a permission denied error, do sudo ./launch instead
  5. Once it's launched, you will be able to access it by going to http://host-ip:8080 in a browser, or http://localhost:8080 if you are running it on your local machine.

For API keys, create the setenv file if it doesn't already exist. It will automatically get created on the first launch, and won't get committed to Git because it is on the .gitignore. Set the API keys as explained here. Once you've put the API keys in that file, re-run the launch script, and it will use your API keys. There are a few things that still don't work, and you can see what works and what doesn't here.

Local Installation of GST on Raspberry Pi

  1. Running raspbian lite(headless) or desktop (both 64bit) we should first start off with an update/upgrade.

    sudo apt update && sudo apt upgrade
    

    Once completed reboot and lets reopen.

  2. Install Miniforge The community version of anaconda/miniconda that has Arm support.

    • Follow the on screen prompts to install miniforge(conda).
    wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
    
    • After following the steps, confirm that you have it by opening a terminal and running: conda -V. The output should be something along the lines of: conda 4.9.2
  3. Install git

    conda install -c anaconda git
    
  4. Install Cmake

    sudo apt update
    sudo apt install -y cmake
    pip install cmake
    
  5. Clone the Project

    • Via HTTPS: git clone https://github.com/OpenBB-finance/OpenBBTerminal.git
    • via SSH: git clone git@github.com:OpenBB-finance/OpenBBTerminal.git
  6. Navigate into the project's folder

    cd OpenBB/
    
  7. Create Environment

    You can name the environment whatever you want. Although you could use names such as: welikethestock, thisistheway or diamondhands, we recommend something simple and intuitive like gst. This is because this name will be used from now onwards.

    conda env create -n openbb python=3.9
    
  8. Activate the virtual environment

    conda activate openbb
    

    Note: At the end, you can deactivate it with: conda deactivate.

  9. Update all poetry dependencies

    poetry update --lock
    

    Note: This is done to solve any issues with poetry dependencies for the arm architecture of the rpi

  10. Install using our updated poetry installation

    poetry install
    
  11. Ready to rock on a Raspberry Pi!

    python terminal.py
    

    Note: For a headless installation using raspbian lite also follow the Jupyter Lab installation

  12. Jupyter Lab (Optional. Early alpha). User the Terminal from Jupyter Lab

You can install Jupyter Lab extensions that help you manage settings and launch the terminal in a JL bash console using the commands in the jupyterlab/README.md

NOTE: When you close the terminal and re-open it, the only command you need to re-call is conda activate gst before you call python terminal.py again.

API Keys

The project is build around several different API calls, whether it is to access historical data or financials. The table below shows the ones where a key is necessary. The easiest way is of updating the keys is by using the terminal, see this guide.

You can also use the environment variable to set your API Keys directly instead of using the Terminal, for the variable name in the code one just needs to remove the "GT_", this can be found in config_terminal.py.

Website Environment Variables
Alpha Vantage OPENBB_API_KEY_ALPHAVANTAGE
Binance OPENBB_API_BINANCE_KEY
OPENBB_API_BINANCE_SECRET
BitQuery OPENBB_API_BITQUERY_KEY
Coinbase OPENBB_API_COINBASE_KEY
OPENBB_API_COINBASE_SECRET
OPENBB_API_COINBASE_PASS_PHRASE
Coinglass OPENBB_API_COINGLASS_KEY
CoinMarketCap OPENBB_API_CMC_KEY
Cryptopanic OPENBB_API_CRYPTO_PANIC_KEY
DEGIRO OPENBB_DG_USERNAME
OPENBB_DG_PASSWORD
OPENBB_DG_TOTP_SECRET
Ethplorer OPENBB_API_ETHPLORER_KEY
Financial Modeling Prep OPENBB_API_KEY_FINANCIALMODELINGPREP
Finnhub OPENBB_API_FINNHUB_KEY
FRED OPENBB_API_FRED_KEY
GitHub OPENBB_API_GITHUB_KEY
Glassnode OPENBB_API_GLASSNODE_KEY
News OPENBB_API_NEWS_TOKEN
Oanda OPENBB_OANDA_TOKEN
OPENBB_OANDA_ACCOUNT
Polygon OPENBB_API_POLYGON_KEY
Quandl OPENBB_API_KEY_QUANDL
Reddit OPENBB_API_REDDIT_CLIENT_ID
OPENBB_API_REDDIT_CLIENT_SECRET
OPENBB_API_REDDIT_USERNAME
OPENBB_API_REDDIT_USER_AGENT
OPENBB_API_REDDIT_PASSWORD
SentimentInvestor OPENBB_API_SENTIMENTINVESTOR_TOKEN
OPENBB_API_SENTIMENTINVESTOR_KEY
Tradier OPENBB_TRADIER_TOKEN
Twitter OPENBB_API_TWITTER_KEY
OPENBB_API_TWITTER_SECRET_KEY
OPENBB_API_TWITTER_BEARER_TOKEN
Whale Alert OPENBB_API_WHALE_ALERT_KEY

Example:

export OPENBB_API_REDDIT_USERNAME=SexyYear

Environment variables can also be set in a .env file at the top of the repo. This file is ignored by git so your API keys will stay secret. The above example stored in .env would be:

OPENBB_API_REDDIT_USERNAME=SexyYear

Note that the OPENBB_API_REDDIT_USER_AGENT is the name of the script that you set when obtained the Reddit API key. Note that it is not necessary to have a valid Alpha Vantage key to get daily OHLC values.