Podman

Create a Linux dev environment

I use Podman to do HPC development in MacOS because some tools (like Darshan) rely on Linux-specific hooks. Here’s the process:

Create the container image

Create a base container image:

podman pull ubuntu:22.04

Then launch an interactive session within the container to set up the dev environment:

# Launch the container
podman run -it --name hpc-dev ubuntu:22.04 bash
 
# Once inside the container
apt update && apt install -y gcc g++ git build-essential

After setting up the environment, commit the container’s state so it can be reused without reinitializing everything:

podman commit hpc-dev my-hpc-dev-env

The above can be done whenever changes worth saving are made.

Create a data volume

Create a persistent volume to store data outside the container:

# Create a persistent volume
podman volume create hpc-dev-vol
 
# Launch the container with the volume mounted
podman run -it --name hpc-dev --volume hpc-dev-vol:/workspace my-hpc-dev-env bash

Or just bind mount a local directory:

podman run -it --name hpc-dev -v /path/on/macos:/workspace my-hpc-dev-env bash

Re-use the container

Once the container is exited, it can be stopped and started again without losing the environment:

podman start -ai hpc-dev

The container can also be used in other systems:

# Export the container
podman save -o hpc-dev.tar my-hpc-dev-env
 
# Import the container
podman load -i hpc-dev.tar

Machine configuration

The default Podman install on MacOS only gave me 4 cores and 2 GB of RAM. You can change this:

podman machine stop
podman machine set --cpus 8
podman machine set --memory 8192
podman machine start
 
podman run --rm -it --name hpc-dev --volume hpc-dev-vol:/workspace my-hpc-dev-env bash

To see the number of cores on MacOS:

sysctl -n hw.ncpu

Inspecting containers

I find myself having to dissect containers that are poorly documented. Here’s what I do.

Start the VM

podman machine init
podman machine start

Pull the image

export img="docker.io/vastdataorg/insight-engine:latest"
podman pull "$img"

Inspect metadata

podman image inspect "$img" \
     --format 'Entrypoint={{json .Config.Entrypoint}} Cmd={{json .Config.Cmd}}'
podman history --no-trunc "$img"

Export and unpack the filesystem (no app execution)

export cid=$(podman create "$img")
podman export "$cid" -o rootfs.tar
podman rm "$cid"
mkdir -p rootfs
tar -xf rootfs.tar -C rootfs

Look at

• Entrypoint/command: image inspect
• Likely app paths: rootfs/app, rootfs/opt, rootfs/usr/local/bin, rootfs/etc
• Search for startup scripts: rg -n "entrypoint|start\.sh|uvicorn|gunicorn|supervisord|s6|tini" rootfs