JupyterHub in minutes with Rancher OS on an OpenStack cloud.

11 min readFeb 8, 2021

I recently set up an OpenStack cloud at home and started playing around with a few things. I have been running JupyterHub at work on our HPC with good results. In that case people log in though the web interface and the notebooks are spawned into compute nodes on the cluster through a job scheduler.

I had read about running JupyterHub in Docker and Kubernetes and wanted to give it a try. I don’t have a Kubernetes deployment set up to play with just yet. However I found that Rancher OS was pretty quick to get up and running to mess around with.

Rancher OS is a small Linux distribution built for running Docker containers. With a few commands I was able to get an instance of Rancher OS up and running and with a few more I had a working JupyterHub.

The following will get JupyterHub up and running. I do not go through the steps of securing it with SSL, setting up authentication, or persistent storage.

First things first log into Openstack and source the keystone_admin file.

[root@firefly ~]# . keystonerc_admin[root@firefly ~(keystone_admin)]#

Download the Rancher OS OpenStack image.

[root@firefly images(keystone_admin)]# wget https://releases.rancher.com/os/v1.5.7/rancheros-openstack.img

Load Rancher into glance

[root@firefly images(keystone_admin)]# openstack image create — container-format bare — disk-format qcow2 — file rancheros-openstack.img rancheros-1.5.7

You will need to create a configuration file to set up the instance. In this case I give it a hostname “jupyterhub” and add an SSH key to access the instance.

[root@firefly images(keystone_admin)]# cat /root/images/rancheros.txt#cloud-confighostname: jupyterhubmanage_etc_hosts: yesssh_authorized_keys:- ssh-rsa AAAAB3NNNNNNNGFGGGGGGOOOOOO*****BnwNoldstewtweqtqwetGm+MVFHsdgsdgdgk6s7IdydFfgjfgjfgjGenerated-by-Nova

In my environment I have two networks. The 10.10 network is internal to OpenStack. The 192.168 network is directly accessible on my home network. I will be launching my instance in the internal network. I will create a floating IP on the 192.168 network to access the instance.

[root@firefly images(keystone_admin)]# openstack subnet list+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — +| ID | Name | Network | Subnet |+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — +| 8cb98547-e305–43a3-ba22-ae83ff9b4eb0 | internal_subnet | bc526ae3–21ed-4f05–9043–17eed412a80b | 10.10.1.0/24 || cd1e095a-ad69–4b35–9592–85ca0e884280 | public_subnet | 68bd8d18–215e-4247–8f29-e965b889b96c | 192.168.1.0/24 |+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — +

In OpenStack security groups are essentially your firewall. In this case I have a security group called SSH which allows access on port 22. I will be enabling this on my instance so that I can log in and configure it.

[root@firefly images(keystone_admin)]# openstack security group list+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — + — — — +| ID | Name | Description | Project | Tags |+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — + — — — +| 2d785644-da90–447b-9e8d-531b876f7688 | default | Default security group | 01d4cc12ae4641a7b91bdd8f0bd4c11e | [] || 3157df56–1993–4248–9d3a-af0778e08fd1 | ssh | | 01d4cc12ae4641a7b91bdd8f0bd4c11e | [] | |+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — + — — — +

Launch a Rancher OS instance.

[root@firefly images(keystone_admin)]# openstack server create — config-drive true — image “rancheros-1.5.7” — flavor m1.small — nic net-id=bc526ae3–21ed-4f05–9043–17eed412a80b — security-group 3157df56–1993–4248–9d3a-af0778e08fd1 — max 1 — user-data /root/images/rancheros.txt jupyterhub

Create a floating IP 192.168.1.97.

[root@firefly images(keystone_admin)]# openstack floating ip create public

#List floating IP

[root@firefly images(keystone_admin)]# openstack floating ip list+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — -+ — — — — — — — — — + — — — + — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — +| ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project |+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — -+ — — — — — — — — — + — — — + — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — +| 8962ec91–503c-4949–905a-e4adf080d7a2 | 192.168.1.97 | None | None | 68bd8d18–215e-4247–8f29-e965b889b96c | 01d4cc12ae4641a7b91bdd8f0bd4c11e |+ — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — -+ — — — — — — — — — + — — — + — — — — — — — — — — — — — — — — — — — + — — — — — — — — — — — — — — — — — +

Assign floating ip to our instance.

[root@firefly images(keystone_admin)]# openstack server add floating ip jupyterhub 192.168.1.97

Now when we list he instances we see that the floating IP has been assigned.

[root@firefly images(keystone_admin)]# openstack server list+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — + — — — — — — — — — — — — — — — — — -+ — — — — — — — — -+ — — — — — +| ID | Name | Status | Networks | Image | Flavor |+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — + — — — — — — — — — — — — — — — — — -+ — — — — — — — — -+ — — — — — +| bb1d8625-df90–4361–9544–0415039e5cc7 | jupyterhub | ACTIVE | internal=10.10.1.66, 192.168.1.97 | rancheros-1.5.7 | m1.small |+ — — — — — — — — — — — — — — — — — — — + — — — — — — + — — — — + — — — — — — — — — — — — — — — — — -+ — — — — — — — — -+ — — — — — +

We are up and running with an accessible IP of 192.168.1.97, port 22 is open, and we have configured the instance with an SSH key. We can go ahead and log in.

Macintosh:~ stevex0r$ ssh -i Downloads/smohr.pem rancher@192.168.1.97[rancher@jupyterhub ~]$

With a single command we can check out the jupyterhub docker image and get it running on port 8000.

[rancher@jupyterhub ~]$ docker run -d -p 8000:8000 — name jupyterhub jupyterhub/jupyterhub jupyterhubUnable to find image ‘jupyterhub/jupyterhub:latest’ locallylatest: Pulling from jupyterhub/jupyterhub54ee1f796a1e: Pull completef7bfea53ad12: Pull complete46d371e02073: Pull completeb66c17bbf772: Pull completefcf01b6b7722: Pull completeb9f344a86a36: Pull complete420e0b505636: Pull complete15612a7c1218: Pull completea6aabc8a2567: Pull complete76f04b09bbf9: Pull complete0cd320fe4add: Pull completeDigest: sha256:38fa6c5349afa5f59e372f447f7836e8c7ca239275f6d273ca79a8b249a68b86Status: Downloaded newer image for jupyterhub/jupyterhub:latesta62aebb171d0e3ca97035b44aab6e843b2bd81408c1ecec0da68e729e90b7c08

We will need to configure users and install a few things before we can log in and run a notebook. Execute bash on the jupyterhub container to access it.

[rancher@ranch ~]$ docker exec -it jupyterhub bash

Inside of the container create a user.

root@a62aebb171d0:/srv/jupyterhub# adduser smohrAdding user `smohr’ …Adding new group `smohr’ (1000) …Adding new user `smohr’ (1000) with group `smohr’ …Creating home directory `/home/smohr’ …Copying files from `/etc/skel’ …New password:Retype new password:passwd: password updated successfullyChanging the user information for smohrEnter the new value, or press ENTER for the defaultFull Name []:Room Number []:Work Phone []:Home Phone []:Other []:Is the information correct? [Y/n] y

Switch to the newly created user account.

root@a62aebb171d0:/srv/jupyterhub# su — smohr

Use pip to install notebook and any other Python modules.

smohr@a62aebb171d0:~$ pip install notebook pandas

Back in OpenStack land we will need to create another security group.

[root@firefly images(keystone_admin)]# openstack security group create jupyterhub

Our JupyterHub is configured on port 8000. We will create a rule to allow port 8000.

[root@firefly images(keystone_admin)]# openstack security group rule create jupyterhub — protocol tcp — dst-port 8000:8000 — remote-ip 0.0.0.0/0+ — — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — +| Field | Value |+ — — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — +| created_at | 2021–02–07T21:49:22Z || description | || direction | ingress || ether_type | IPv4 || id | b9c06dbf-0bdb-4e18-a9a8–6a1162fa4c75 || name | None || port_range_max | 8000 || port_range_min | 8000 || project_id | 01d4cc12ae4641a7b91bdd8f0bd4c11e || protocol | tcp || remote_group_id | None || remote_ip_prefix | 0.0.0.0/0 || revision_number | 0 || security_group_id | bf8c9d8d-2f53–4f49–9526–72fdcd68ceab || tags | [] || updated_at | 2021–02–07T21:49:22Z |

We see that port 8000 is enabled in the security group.

[root@firefly images(keystone_admin)]# openstack security group rule list jupyterhub+ — — — — — — — — — — — — — — — — — — — + — — — — — — -+ — — — — — -+ — — — — — -+ — — — — — — + — — — — — — — — — — — -+| ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group |+ — — — — — — — — — — — — — — — — — — — + — — — — — — -+ — — — — — -+ — — — — — -+ — — — — — — + — — — — — — — — — — — -+| 15ab00bd-7cf0–43ee-bc9e-5d30baa0cd40 | None | IPv6 | ::/0 | | None || b9c06dbf-0bdb-4e18-a9a8–6a1162fa4c75 | tcp | IPv4 | 0.0.0.0/0 | 8000:8000 | None || cf155e1e-872e-4725-a24b-9c919e9e5bb8 | None | IPv4 | 0.0.0.0/0 | | None |+ — — — — — — — — — — — — — — — — — — — + — — — — — — -+ — — — — — -+ — — — — — -+ — — — — — — + — — — — — — — — — — — -+

Assign the security group jupyterhub to the totally not confusing instance which is also called jpuyerthub.

[root@firefly images(keystone_admin)]# openstack server add security group jupyterhub jupyterhub

When we take a look at the instance we see that the IP is configured, our ports are set up, and we are ready to log in.

[root@firefly images(keystone_admin)]# openstack server show jupyterhub+ — — — — — — — — — — — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — — — — — — — — — — — +| Field | Value |+ — — — — — — — — — — — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — — — — — — — — — — — +| OS-DCF:diskConfig | MANUAL || OS-EXT-AZ:availability_zone | nova || OS-EXT-SRV-ATTR:host | reaver || OS-EXT-SRV-ATTR:hypervisor_hostname | reaver || OS-EXT-SRV-ATTR:instance_name | instance-00000050 || OS-EXT-STS:power_state | Running || OS-EXT-STS:task_state | None || OS-EXT-STS:vm_state | active || OS-SRV-USG:launched_at | 2021–02–07T21:30:56.000000 || OS-SRV-USG:terminated_at | None || accessIPv4 | || accessIPv6 | || addresses | internal=10.10.1.66, 192.168.1.97 || config_drive | True || created | 2021–02–07T21:30:50Z || flavor | m1.small (2) || hostId | 9d4c561db67b9ed6e972424e4d5a6cb52d22dc0d10c67dd36920a421 || id | bb1d8625-df90–4361–9544–0415039e5cc7 || image | rancheros-1.5.7 (f7d438fb-0eae-4db6-b8de-73364350a39c) || key_name | None || name | jupyterhub || progress | 0 || project_id | 01d4cc12ae4641a7b91bdd8f0bd4c11e || properties | || security_groups | name=’ssh’ || | name=’jupyterhub’ || status | ACTIVE || updated | 2021–02–07T21:31:01Z || user_id | 1b29256ea2d04eb29b701985908d7998 || volumes_attached | |+ — — — — — — — — — — — — — — — — — — -+ — — — — — — — — — — — — — — — — — — — — — — — — — — — — — +