Hosting 'The Littlest JupyterHub'

Back in 2024 I ran a pyrolite workshop as part of the SGTSG conference (The Geological Society of Australia's Specialist Group in Tectonics and Structural Geology Biennual Meeting), where I used a hosted Jupyterlab instance to provide easy access to an environment with pyrolite installed for the attendees, in an attempt to maximise the use of time and minimise technical issues.

This proved a to be both technically fairly simple (in terms of getting a JupyterHub running) and logistically hard (in terms of getting infrastructure to run it on). Since I've wondered whether I can host such thing from a server (desktop running Proxmox) I own, and save both the hassle and the bill. I think I've figured out one way which will work, and this post covers it.

Getting a Large VPS is Hard

I had a bunch of headaches trying to set up a server in multiple of the major players (AWS, Azure, OVH, Digital Ocean). Azure even blocked my account for trying to do this, with no explanation as to why - not helpful a few days out from the workshop. The other major issue was that to access nodes with enough resources to handle 40-odd simultaneous users was nigh on impossible without having a previous record (and series of bills) with each of these providers. I think in the end I went back to AWS and settled for something subpar which would at least have a few folks running on the night. But the experience wasn't particularly fun.

Setting up The Littlest JupyterHub (TLJH)

As you'll find in other posts here, I use Proxmox as a host for many of my self-hosted services (small and not-to-small). Within this, I principally use Linux Containers (LXC) as my container solution. Typically, installing into an LXC will be pretty similar to installing on a Virtual Private Server (if you're e.g. renting a cloud server, as I did for the workshop originally) and especially similar to installing on any Linux host.

Here I'm going to start with a Debian 13 image (I think TLJH uses Debian 12 by default, which might explain one or two of the changes I needed), which we can quickly update after starting:

apt update && apt upgrade -y

Note: I don't use sudo here as I'm running as root for the majority of important tasks within the container, but on your device or in your container you might need it. Particularly for this one, you might need sudo -E.

We'll loosely follow the official guide for getting set up on your own server, which starts with installing some basic pre-requisites. I've had issues running the bootstrap.py install script on Debian 13 and Python 3.13, so have simplified this down to a few commands below. Here this is updated to include python3-venv, which for Python 3.13 as of writing this requires it:

apt install python3 python3-dev python3-venv git curl -y

The bootstrap.py script as per the installation instructions, with defaults, essentially reduces to creating a venv, cloning a repo and running the installer:

python3 -m venv /opt/tljh/hub
/opt/tljh/hub/bin/pip install --upgrade git+https://github.com/jupyterhub/the-littlest-jupyterhub.git
/opt/tljh/hub/bin/python -m tljh.installer --admin admin

When this is finished, you can visit your local IP address (found with, for example, ip a) and you should see the Jupyter login page. It'll probably warn you that this is being served over HTTP rather than HTTPS - and that's something we can deal with later. Note that by default, the Jupyterhub Service is running on ports 80 and 8443 (HTTP/HTTPS), but if you visit the IP address locally you should be mapped to the right place. You can login with the admin username we set above (here, admin), and choose a password to associate with this account on first login.

Setting up Default Authentication

Here we're going to modify the default authentication method (which allows anyone to sign up, using a username and password of their choice at first login). We'll instead stop this method from being able to create new users, so essentially whitelisting usernames.

This allows whitelisted users to create a password at first login, but it won't automatically create a user for any random person trying to access your server; this means you'll need to add a username for anyone wanting to access it (or a set of usernames which people can take one of, which is what I did for the workshop). This strikes a bit of a balance in terms of security - while the presence of e.g. an admin user is fairly guessable, the absence of others makes it harder to gain access. As more users are added, there's a bit of a security hole before they set a password, and if they set a bad password but it's manageable for e.g. workshops (the intended use case of exposing this, where you do).

We can modify the YAML configuration file at /opt/tljh/config/config.yaml to have an authenticator set as follows, then reload:

authenticator:
  type: firstuseauthenticator.FirstUseAuthenticator
  create_users: false

tljh-config reload

Default Environment Management

To add packages to the default environment (apt, pip or conda), you can log into the JupyterHub interface with your admin user, open a terminal and use e.g. conda install -c conda-forge <package_name> or pip install <package_name>.

For workshop-style work, consider using an environment.yml or requirements.txt you manage in git for this purpose, and use e.g. conda env update -n base -f environment.yml to update it.

Using nbgitpuller

If you're running a workshop with some standardised content you want everyone to have a copy of - their own copy - then nbgitpuller can come in handy. This assumes you have git installed (we do), and in this instance works through providing a specific URL to attendees, which itself includes instructions to pull the repository on login For example, in my pyrolite workshop I used this to pull a repository for each user on login, and also some extra parameters to i) open a specific file within that repository (here ./notebooks/00_overview.ipynb relative to the root of that repository), and ii) pull down the develop branch: https://{HUB_URL_OR_IP}/hub/user-redirect/git-pull?repo={REPOSITORY}&urlpath=lab%2Ftree%2F{REPOSITORY}%2Fnotebooks%2F00_overview.ipynb&branch=develop. nbgitpuller provides a link generator for you, which is pretty simple to use. Note this is a good way to distribute content, but in terms of environment and packages, see Default Enviroment Management to save doing this per-user for packages/software.

Overriding Default Jupyter Settings

For example, we can create a file called overrides.json, and add ruff as the default formatter:

{
  "jupyterlab_code_formatter:settings": {
    "preferences": {
      "default_formatter": {
        "python": "ruff"
      }
    }
  }
}

Caches for matplotlib and numba

Some packages use cache directories, and wil complain about these not existing for each user. You can set the environment variables for the spawner with an extra configuration file:

from pathlib import Path

def tmpdir(instance, key):
    return str(Path("~") / ".{}".format(key))

def mpl_usr_tmp(instance):
    return tmpdir(instance, "mpl")

def nb_usr_tmp(instance):
    return tmpdir(instance, "numba")

 
c.Spawner.environment.update( # noqa: F821 # type: ignore
    {"MPLCONFIGDIR": mpl_usr_tmp, "NUMBA_CACHE_DIR": nb_usr_tmp}
)

If you write this to a file at e.g. /opt/tljh/config/jupyterhub_config.d/usr_env_caches.py the environment variables should be loaded when the notebook server starts.

Adding Users

We've set up our server to only allow users which are explicitly added to a whitelist (i.e. already created). You can add these in bulk, separated by lines. To add users in the JupyterHub interface, you can you can go to https://<HUB_DOMAIN>/hub/admin#/add-users, and add them there.

Quickly Purging Users

If you need to quickly purge the users of your hub (e.g. I used this after the workshop), you can use this kind of setup where you obtain an API token configuration (at http://<HUB_DOMAIN_OR_IP>/hub/token):

import requests

token = '<API_TOKEN>'
api_url = 'https://<HUB_DOMAIN_OR_IP>/hub/api'

r = requests.get(api_url + '/users',
    headers={
             'Authorization': 'token %s' % token,
            "Content-Type": "application/json",
            }
    )

r.raise_for_status()
users = r.json()

matching_users = [u['name'] for u in users if not u['name'].startswith('admin')]
for u in matching_users:
    requests.delete(api_url + "/users/" + u,
         headers={
             'Authorization': 'token %s' % token,
            "Content-Type": "application/json",
            })

Adding a Tailscale Funnel for JupyterHub

To make this locally hosted app available to external users, we'll use Tailscale (and more specifically, Tailscale funnel). This only requires a Tailscale account, which is free. You'll end up with a publicly-accessible URL for your JupyterHub, which if you want you could map to a subdomain in DNS configuration for a domain you own.

If you're using an LXC, you'll need to mount /dev/net/tun to use Tailscale.

On Linux, you can install Tailscale with a basic install script:

curl -fsSL https://tailscale.com/install.sh | sh

To start a funnel providing access to JupyterHub on port 80, you can run:

tailscale funnel -bg  http://localhost:80

This will make the JupyterHub instance available on your tailnet; if you wanted, you could point the NDS record on your custom domain to this also.

Note: I did try doing this with Cloudflare tunnels, but was only half successful - I could get a tunnel to show parts of the JupyterHub interface (the home, admin and token pages) but it wouldn't load a notebook. I suspect there's either some part of JupyterHub running on different ports/protocols which I didn't see, or some setting I needed to enable to get it working. After an evening of headaches trying to make this work I switched to Tailscale and had it up after only a little fiddling.