bdist-workspace

This repository provides containerized applications and microservices for the Information Systems and Databases Course @ Instituto Superior Técnico.

This guide helps setting up a workspace environment for completing lab classes, which includes a database management system and other auxiliary services.

Services provided include:

• PostgreSQL Open Source database management system
• pgAdmin4 Open Source administration and development platform for PostgreSQL
• bdist/notebook Jupyter Notebook Databases Stack

Prerequisites

Install Docker Desktop

Docker Desktop is an application for MacOS, Linux, and Windows machines for the building and sharing of containerized applications and microservices.

Install Docker Desktop on Mac, Windows or Linux

Install Git

Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

Install Git on Mac Windows or Linux

Your first time launching the Workspace

1. Open Terminal.

2. Change the current working directory to the location where you want the cloned directory.

3. Run the following command to create your local clone

   git clone https://github.com/bdist/bdist-workspace.git

4. Change the current working directory to the location of the cloned directory.

   cd bdist-workspace/

5. From the cloned directory, start up bdist-workspace by running

   docker compose up

   Note: The services are attached to this Terminal, so the logs for all the services provided will be printed on its window. To shutdown safely you will need to issue CTRL+C in the Terminal window and wait until all the services stop gracefuly.

Using the Jupyter Notebook (Docs)

The Jupyter Notebook service runs on the port 9999. Token authentication is enabled.

1. You need to find your Authentication Token to login every time the bdist-workspace is launched (e.g., after a reboot)

2. Find the section of the logs towards the bottom of the Terminal window that look like this excerpt:

   bdist-workspace-notebook-1  |     Or copy and paste one of these URLs:
   bdist-workspace-notebook-1  |         http://7fd8c38e99bd:9999/lab?token=f83ee982668ebe66bee2dbeb5875d14131a1d118d1e0fa12
   bdist-workspace-notebook-1  |         http://127.0.0.1:9999/lab?token=f83ee982668ebe66bee2dbeb5875d14131a1d118d1e0fa12
   

   Note: You can also view the logs for the bdist-workspace-notebook-1 in the Containers tab in the Docker Desktop application.

3. Follow the link printed last with host 127.0.0.1. The authentication token is embedded in the URL.

Included services

pgAdmin

pgAdmin is the most popular and feature rich Open Source administration and development platform for PostgreSQL, the most advanced Open Source database in the world.

1. Login here.

   Username: pgadmin@tecnico.pt Password: pgadmin

2. Click the button Add New Server.

3. Set the Name in the General tab to postgres.

4. Set the Hostname in the Connection tab to postgres.

5. Use the same username and password you would provide psql.

   Username: postgres Password: postgres

Flask Web App

1. Make sure the app/ directory is right next to the bdist-workspace/ directory. If needed, move the app/ directory out of bdist-workspace/.

   $ ls
   app/  bdist-workspace/

2. The docker-compose..app.yml file contains the instructions to launch the app. Relaunch the workspace with the following command to use it:

   docker compose -f docker-compose..app.yml up

   Note: Please run the instructions in the How do I update my workspace? section if this file is does not exist.

3. Check if the app is running and open ping.

4. Do you get an API-like HTTP JSON-formatted response like this?

   {
     "message": "pong!"
   }

5. Try modifying the message in ../app/app.py while it is running.

6. In the logs, check if an automatic reload of the Flask Web App is triggered when you save your changes.

FAQ and Troubleshooting

I get an initdb: error: directory "/var/lib/postgresql/data" exists but is not empty

You need to delete the data volume used by postgres and recreate it.

1. Open a new Terminal.

2. Run this to shutdown the containers and remove all associated volumes.

   docker compose down -v

3. Run the steps in the following section to update bdist-workspace.

How do I update my workspace?

1. Firstly, from the project directory run

   git pull

   This updates bdist-workspace to the latest version.

2. Secondly, from the project directory run

   docker compose pull

   This updates container images to the latest version.

3. Lastly, from the project directory run

   docker compose up --build --force-recreate --remove-orphans

   This starts up the bdist-workspace and cleans up unused images.

I get Permission Errors and/or the notebook Save Button is disabled

You need to set the owner of every directory under /home/jovyan/ to the user jovyan and groups users. This is the default username and group for the user in the notebook container.

You only need to run this once.

1. Open Terminal.

2. Run this to reset the permissions for files and folders under home/jovyan/.

   chown -R jovyan:users /home/jovyan/

If this still does not solve the issue with permissions then follow this recipe:

1. Open Terminal.

2. Run this do access the notebook container as administrator (root):

   docker compose exec -e GRANT_SUDO=yes --user root notebook /bin/bash

3. Run this to reset the permissions for files and folders under home/jovyan/.

   chown -R jovyan:users /home/jovyan/

A psycopg.errors.UndefinedTable exception is thrown

(psycopg.errors.UndefinedTable) relation "depositor" does not exist

If the exception thrown looks like the one in the example, then follow this checklist:

1. Is the database you are connected to the correct one?

   %sql postgresql://bank:bank@postgres/**bank**

2. Is the database empty?

   Connect to the target database on the Terminal via psql and run the command \d.

   $ psql -h postgres -U bank
   Password for user bank:
   psql (16.1 (Debian 16.1-1.pgdg120+1))
   Type "help" for help.
   
   postgres bank@bank=>\d
           List of relations
   Schema |   Name    | Type  | Owner
   --------+-----------+-------+-------
   public | account   | table | bank
   public | borrower  | table | bank
   public | branch    | table | bank
   public | customer  | table | bank
   public | depositor | table | bank
   public | loan      | table | bank
   (6 rows)

   Note: Alternatively, get the list of relations inside the notebook. Run this on a new notebook cell:

   %sqlcmd tables

Issues

Please use GitHub Issues to report any issues you might have with bdist-workspace.

Credits

Flavio Martins