DEV Community

Mike Lockhart
Mike Lockhart

Posted on • Originally published at milosophical.me

Use Jupyter Notebooks for bash scripting

#literate #bash #jupyter #vscode

I'm experimenting with using Jupyter to capture Linux server commands in a Notebook while I figure out doing GitLab Geo setup. It's complicated, so I want to keep good notes, but copy/paste from the terminal to a text editor is a chore I'd rather not deal with. Instead I would like to keep a literate-programming style notebook. I learnt recently that there is a Jupyter kernel for running bash instead of python, and that would be ideal.

This post describes how to set up using Jupyter Notebooks on a remote host, and then running and editing notebooks using Visual Studio Code's support for Jupyter.

You can also do this without VSCode, by SSH tunnelling a local web browser to the host. That would be better in cases where users' /home volume size is restricted. I briefly discuss this too.

Install Python, Jupyter, and the Bash kernel

First we need an environment for running Jupyter. I did try this globally, since I'm using a dedicated host, but it works better as a virtual environment ("venv").

This server runs Ubuntu, so the software package installation is specific to that OS.

Install the python-venv package

In Ubuntu, Python is separated into multiple .deb packages, and the python virtual environment functions are in pythonX.Y-venv. Don't worry about the specific version in the package name: if you install python3-venv then APT will get the latest version available:

sudo apt install python3 python3-venv
Enter fullscreen mode Exit fullscreen mode 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3 is already the newest version (3.8.2-0ubuntu2).
The following NEW packages will be installed:
  python3-venv
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 1228 B of archives.
After this operation, 11.3 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://us-west1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 python3-venv amd64 3.8.2-0ubuntu2 [1228 B]
Fetched 1228 B in 0s (17.2 kB/s)       
Selecting previously unselected package python3-venv.
(Reading database ... 197373 files and directories currently installed.)
Preparing to unpack .../python3-venv_3.8.2-0ubuntu2_amd64.deb ...
Unpacking python3-venv (3.8.2-0ubuntu2) ...
Setting up python3-venv (3.8.2-0ubuntu2) ...
Processing triggers for man-db (2.9.1-1) ...
Enter fullscreen mode Exit fullscreen mode

Create a python virtual environment

Make a directory for keeping your notebooks, and navigate to it:

NB=~/lab/notebooks
mkdir -p $NB; cd $NB
Enter fullscreen mode Exit fullscreen mode

Create a virtual environment using Python's venv module, installed above. I'm following the usual practice of putting the venv within the project directory, rather than my preferred practice of putting venvs in ~/lib/venv:

python3 -m venv venv
Enter fullscreen mode Exit fullscreen mode

You can now activate the venv for the current shell session:

source venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Install Jupyter and bash_kernel

Now that the venv is activated, install Jupyter and the bash_kernel into it:

pip install -U pip jupyter bash_kernel
python -m bash_kernel.install
Enter fullscreen mode Exit fullscreen mode

At this point we can start Jupyter with the bash kernel.

Running Jupyter notebook

Launch the notebook, without automatically spawning a web browser, since that makes no sense on the remote host:

jupyter notebook --no-browser
Enter fullscreen mode Exit fullscreen mode

Jupyter will print status information to the terminal. Among it are details for connecting securely to the server to view and run notebooks:


[I 2025-03-08 21:30:42.929 ServerApp] Serving notebooks from local directory: /home/mjl/lab/notebooks
[I 2025-03-08 21:30:42.929 ServerApp] Jupyter Server 2.14.2 is running at:
[I 2025-03-08 21:30:42.929 ServerApp] http://localhost:8888/tree?token=REDACTEDTOKEN
[I 2025-03-08 21:30:42.929 ServerApp]     http://127.0.0.1:8888/tree?token=REDACTEDTOKEN
[I 2025-03-08 21:30:42.929 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 2025-03-08 21:30:42.933 ServerApp]

    To access the server, open this file in a browser:
        file:///home/mjl/.local/share/jupyter/runtime/jpserver-7453-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/tree?token=REDACTEDTOKEN
        http://127.0.0.1:8888/tree?token=REDACTEDTOKEN
Enter fullscreen mode Exit fullscreen mode

Note that a fresh token is generated each time that Jupyter starts.

Tunnel to Jupyter and open in a local web browser

Typically, the next step when running Jupyter on a remote host, is to "tunnel to the host" and then connect a local web browser through the tunnel to view the notebook server. The SSH command for that is:

ssh -N -L localhost:8888:localhost:8888 username@remote_host
Enter fullscreen mode Exit fullscreen mode

After that is connected, use your local web browser to view the URL printed by Jupyter above. If the tunnel held, you are in and have a web interface to Jupyter Notebook.

Running bash in a notebook

You can create notebooks on the remote host using the Bash kernel, and run bash commands in cells of the notebook, executing on the remote host!

What's great about this is that the bash kernal maintains state, just like any other Jupyter kernel. So the environment persists between cells, as it is the same process executing each cell, unlike with Python-kernel cell-magics which fork a sub-process to execute the shell commands.

So this makes the bash kernel for Jupyter ideal for exploring and documenting complicated Linux server setup.

Connect to the host from VSCode

A nicer way to do this is with VSCode Remote SSH. It has these advantages:

  1. Automatically forward ports, so port 8888 will be browsable locally, without the SSH command above
  2. Nicer / more powerful editing than Jupyer in a browser
  3. Open the notebook directly and connect to the server running on the host
  4. Absolute filenames and directories that appear in the shell output can be clicked on and opened with VSCode
  5. View the Jupyter console output, and the notebook, together in the same window
  6. Navigate an outline of the notebook (like Jupyter Lab), browse files in the remote filesystem, run a remote terminal
  7. Other IDE features, like IntelliSense for shell aliases and functions from within the Notebook!

If you followed along the above with SSH in a normal Terminal application, but now want to try using VSCode, then exit the Jupyter server with ⌃-C (Ctrl-C), and open a new VSCode Remote SSH connection to the host, from VSCode:

  • F1 Remote-SSH: Connect Current Window to Host...

(If the host is in your SSH configuration, it'll be selectable from the quick-pick)

Install Python and Jupyter VSCode extensions

You must install these VSCode extensions in the Remote-SSH session. They are necessary for the integration to work:

Launch Jupyter inside VSCode

Having installed the extensions, we can now open a new notebook in VSCode.

Open the VSCode integrated terminal and change to the directory with the notebooks:

cd ~/lab/notebooks
Enter fullscreen mode Exit fullscreen mode

Make sure to activate the environment:

source venv/bin/activate
Enter fullscreen mode Exit fullscreen mode

Then start Jupyter again, like before:

jupyter notebook --no-browser
Enter fullscreen mode Exit fullscreen mode

The Jupyter server output will contain the new authentication token information, for this instance of Jupyter.

Making new notebooks and running cells

With Jupyter running inside VSCode, there's an automatic port-forwarding for Jupyter.

Open a local web browser through the VSCode tunnel

VSCode automatically detects listening TCP ports and tunnels them for us. We can connect our local web browser through the tunnel that VSCode provides, without need for special SSH parameters. Review the PORTS panel to confirm the forwarded ports, but Jupyter usually opens port 8888 by default:

http://localhost:8888/?token=PUT_TOKEN_HERE

Use the connection details shown on the TERMINAL panel for the token.

Open a Notebook directly in VSCode

You can create and interact with notebooks directly in VSCode:

  1. Create a new buffer.
  2. Save the buffer with a .ipynb filename extension. The file re-loads as a Notebook, in VSCode's Notebook editor.
  3. Select the Bash kernel from the Select a kernel button in the top-right.
    1. Choose Jupyter kernel... from the quick-pick list.
    2. Then choose Bash (venv/bin/python).
  4. Add a code cell by pressing A and enter some bash code in it. Try uname -a to convince yourself it is the remote host.
  5. Run the cell to test that it works, using the ⇧-return keyboard shortcut.

Enjoy editing and executing bash cells with the full features of the VSCode editor and any other extensions that you would like to add.

Side-note: Storage volume consumption by VSCode and Jupyter

Do note that after installing all of this, there will be quite a lot of data in your $HOME directory. The VSCode server components added 510MB:

du -sh ~/.vscode-server/* | sort -h
Enter fullscreen mode Exit fullscreen mode 
21M /home/mjl/.vscode-server/code-e54c774e0add60467559eb0d1e229c6452cf8447
60M /home/mjl/.vscode-server/data
195M    /home/mjl/.vscode-server/cli
234M    /home/mjl/.vscode-server/extensions
Enter fullscreen mode Exit fullscreen mode

Breaking down the 234MB of extensions:

du -sh ~/.vscode-server/extensions/* | sort -h
Enter fullscreen mode Exit fullscreen mode 
8.0K    /home/mjl/.vscode-server/extensions/extensions.json
116K    /home/mjl/.vscode-server/extensions/ms-toolsai.jupyter-keymap-1.1.2
512K    /home/mjl/.vscode-server/extensions/ms-toolsai.vscode-jupyter-slideshow-0.1.6
556K    /home/mjl/.vscode-server/extensions/ms-toolsai.vscode-jupyter-cell-tags-0.1.9
21M /home/mjl/.vscode-server/extensions/ms-toolsai.jupyter-2025.1.0-linux-x64
25M /home/mjl/.vscode-server/extensions/ms-toolsai.jupyter-renderers-1.1.0
38M /home/mjl/.vscode-server/extensions/ms-python.debugpy-2025.4.0-linux-x64
48M /home/mjl/.vscode-server/extensions/ms-python.python-2025.2.0-linux-x64
103M    /home/mjl/.vscode-server/extensions/ms-python.vscode-pylance-2025.3.1
Enter fullscreen mode Exit fullscreen mode

Also the python virtual environment, and Jupyter itself, are pretty significant:

du -sh ~/lab/notebooks/venv
Enter fullscreen mode Exit fullscreen mode 
247M    /home/mjl/lab/notebooks/venv
Enter fullscreen mode Exit fullscreen mode 
$ du -sh ~/lab/notebooks/venv/share/* | sort -h
Enter fullscreen mode Exit fullscreen mode 
12K /home/mjl/lab/notebooks/venv/share/applications
12K /home/mjl/lab/notebooks/venv/share/man
40K /home/mjl/lab/notebooks/venv/share/icons
2.2M    /home/mjl/lab/notebooks/venv/share/python-wheels
23M /home/mjl/lab/notebooks/venv/share/jupyter
Enter fullscreen mode Exit fullscreen mode

Be mindful of this storage consumption! Running Jupyter within VSCode on remote hosts is probably something only to be done in environments where you can afford this extra ¾ GB storage.

It's also installed per user, so a shared host — such as a university mainframe — will quickly balloon the /home volume.

Top comments (0)

Read next

arjun98k profile image

Zed IDE vs. NeoVim and Other IDEs: The Future of Modern Coding Environments

arjun -

frorning profile image

Building a Basic Testing Framework in Bash 🐚

Piyush Chauhan -

arjun98k profile image

Shell Scripting for Beginners: Automating Common Coding Tasks

arjun -

midhunkrishnan profile image

To access terminal profile of VS Code and add commands to be run initially

Midhun Krishnan R -