Troubleshooting

Every frame is an entry in the boot menu, and it boots a writable copy of the system as it was. If the system no longer boots, you therefore do not need the live medium. Commands for busybox tools such as reboot are written with their path; see system.

The system does not start after a change

In the boot menu, choose the newest Frame entry. The initramfs tells you that it is booting the frame and that changes are dropped at the next frame boot. To make the old state permanent, boot the normal entry and schedule a flash, which cannot run while the root is mounted:

# scram roll
# scram flash --boot FRAME
flash to FRAME scheduled for the next boot
# /rod/init/bin/reboot

Early in that boot, the initramfs takes a frame named pre-flash, replaces the subvolumes, and tells you which frame it flashed and how to undo it. --root after --boot replaces only @ and leaves the home directories as they are. scram cancel removes a scheduled action, and scram status and scram doctor show it while it is waiting.

scram flash refuses

# scram flash FRAME
scram: @ is mounted, run offline

On the running system, use --boot. The same goes for scram commit. If nothing is waiting, scram commit tells you so.

An update was reverted

An update that you applied with scram commit --boot runs on trial. The trial service waits five minutes, runs geiger up to three times, and reboots into the old state if it still fails. The initramfs then tells you that the update was reverted and which frame holds the state before the revert. Find out what geiger complained about on the new state before you try again. If you want to keep an update that geiger rejects, run scram confirm before the five minutes are over.

The boot menu does not show

The initramfs drops to a shell

The initramfs reports what failed and then drops to a shell. These are the causes:

No root device is found
No block device with that label appeared within about twenty seconds. The btrfs label must be uranium. Check it with lsblk -f from a rescue system, and set it with btrfs filesystem label DEVICE uranium. A storage driver that is not built into the kernel cannot show the disk.
The device cannot be mounted
The filesystem is damaged or is not btrfs.
The root subvolume cannot be mounted
The subvolume @ is missing. scram doctor from a rescue system reports that the subvolume is missing.
A frame cannot be opened
The id in uranium.frame= has no frame.
The encrypted device is not found, or it cannot be unlocked
The device from uranium.luks= was not found, or the passphrase was wrong three times. The prompt uses the default US layout, see keyboard.
The live medium is not found, or its image cannot be mounted
The ISO was not found within thirty seconds, or it's image is bad. Check the download, see download.

Add uranium.shell to the boot arguments, or choose Rescue shell, to get a shell before the root is mounted. Exit the shell to continue the boot.

Services do not start

# geiger
FAIL  not running: bad1 bad2 bad3
# sv status /etc/service/*
down: /etc/service/bad1: 1s, normally up, want up

If sv status shows a service that stays at one or two seconds and is "normally up, want up", the run script exits at once and runsv starts it again every second. The reasons are written to the console, not to a log. These are the common ones:

runsv cannot start the run script because access is denied
run is not executable. Run chmod +x /etc/sv/NAME/run.
The run script names a program that does not exist
The program does not exist. Scripts have to name programs by full path or set PATH; busybox tools are in /rod/init/bin.
sv cannot find the service directory
The name is not in SVDIR. In a script, or without a login shell, use /etc/service/NAME.

Add exec 2>&1 to the script so the errors go to the log, and read /var/log/NAME/current. An sshd that exits at once usually has a bad configuration, check with sshd -t -f /etc/ssh/sshd_config, see ssh. A log directory added to a service that already runs is not used. See services.

No network

# ip addr
# ip route
# sv status /etc/service/dhcp
# tail /var/log/dhcp/current
# cat /etc/resolv.conf
# geiger

The package index is missing

# rod +i hello
rod: no package index, run: rod +s
# rod +p
rod: no pin, run: rod +s

The index is not on the ISO or in the stage, and frames leave it out. Run rod +s with a network. rod +f needs it too, and geiger warns until the index exists and warns again when it is older than 60 days. hazmat and silo need the index as well. rod +s needs a working clock for the https certificates; see time. Installing package sets with @NAME reads /etc/rod/sets/NAME.

doas

See users.

A user cannot log in

The clock or the zone is wrong

A date in 1970 or the wrong hour breaks certificate checks and makes the frame ids misleading. Check ls -l /etc/localtime (a broken link shows UTC without a message), sv status ntpd and /var/log/ntpd/current. Set the time with date -s; see time.

The disk is full

Frames hold old versions of files, and deleting files does not free space until the frames that reference them are gone. geiger warns at 85% and fails at 95% use of /. See storage for scram prune and rod +c old. scram doctor warns below 10% free space.

A key does not produce the right letter

If a name in /etc/keymap is wrong, boot ignores the error. Run loadkeys NAME by hand and read the message.

Find out what changed

scram log lists the frames with the number of files that changed since the frame before. scram diff FRAME lists the files, scram blame PATH lists the frames in which a file changed, scram cat PATH FRAME shows an old version, scram restore PATH FRAME brings one file back, and scram bisect GOOD BAD COMMAND finds the first frame in which a command fails. geiger and scram doctor are the first two commands to run on a system that behaves oddly; see geiger.

Report a problem

See contributing for what a report should contain.

← System and filesFAQ →