Linux Cockpit Terminal screen illegible

If the Terminal screen on a server’s Cockpit page is illegible, you’ll need to set the site permissions from your client web browser.

Go to the site at http://x.x.x.x:9090, login, click the Terminal option.

Click the lock icon in the address bar.

Click on Connection Not Secure.

Click on More Information.

Click on the Permissions icon.

Under “Extract Canvas Data”, unselect “Use Default” and select “Allow”.

Refresh the page. You may have to change the font size.

upgrade to Debian 13 Trixie

Pre-Upgrade System Preparation

elevate to root

Create System Backup
mkdir mkdir /mnt/athena_backups/"$HOSTNAME"_backups
tar -czf /mnt/athena_backups/"$HOSTNAME"_backups/etc-backup-$(date +%Y%m%d).tar.gz /etc
dpkg --get-selections > /mnt/athena_backups/"$HOSTNAME"_backups/package-selections-$(date +%Y%m%d).txt
Ensure Debian 12 is fully updated before proceeding
apt update
apt upgrade -y
apt dist-upgrade -y
apt clean
apt autoremove -y
Verify system integrity before upgrade
apt --fix-broken install
dpkg --configure -a
Create safety copy of repository settings
cp /etc/apt/sources.list /etc/apt/sources.list.bookworm-backup
cp -r /etc/apt/sources.list.d /etc/apt/sources.list.d.bookworm-backup
Update Main Sources List and Additional Repositories
sed -i 's/bookworm/trixie/g' /etc/apt/sources.list
find /etc/apt/sources.list.d -name "*.list" -exec sed -i 's/bookworm/trixie/g' {} ;

This systematically updates all repository references from Bookworm to Trixie across all configuration files.

apt update

Third-party repositories may not immediately support Trixie. Consider temporarily disabling them by adding ‘#’ at the beginning of their lines in sources configuration files.

Minimal Upgrade First
apt upgrade --without-new-pkgs
Full Distribution Upgrade
apt full-upgrade
Confirm successful upgrade completion
cat /etc/debian_version
lsb_release -a
cat /etc/os-release
Clean Obsolete Packages
apt autoremove -y
apt autoclean
Update Package Cache
apt update
apt list --upgradable
System Reboot: Complete the upgrade process
systemctl reboot

Adjust History file to erase duplicates and set larger capacity

vim .bashrc

look for the following lines and adjust HISTCONTROL, HISTSIZE, and HISTFILESIZE accordingly:

#don't put duplicate lines or lines starting with space in the history.
#See bash(1) for more options
HISTCONTROL=ignoreboth:erasedups
#append to the history file, don't overwrite it
shopt -s histappend
#for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=9999
HISTFILESIZE=9999

lvm cheat sheet

Logical Volume Manager (LVM)

This is a quick and dirty cheat sheet on LVM using Linux, I have highlighted many of the common attributes for each command however this is not an extensive list, make sure you look up the command.

With the pvs, vgs and lvs commands, the number of verboses added the more verbose information for example pvs -vvvvv

Directory and Files
Directories and Files## Directories
/etc/lvm                    – default lvm directory location
/etc/lvm/backup         – where the automatic backups go
/etc/lvm/cache          – persistent filter cache
/etc/lvm/archive        – where automatic archives go after a volume group change
/var/lock/lvm             – lock files to prevent metadata corruption

# Files
/etc/lvm/lvm.conf       – main lvm configuration file
$HOME/.lvm               – lvm history
Tools
diagnosticlvmdump
lvmdump -d
dmsetup [info|ls|status]

Note: by default the lvmdump command creates a tar ball
Physical Volumes
displaypvdisplay -v
pvs -v
pvs -a
pvs –segments (see the disk segments used)

pvs attributes are:
1. (a)llocatable
2. e(x)ported
scanningpvscan -v

Note: scans for disks for non-LVM and LVM disks
addingpvcreate /dev/sdb1

## Create physical volume with specific UUID, used to recover volume groups (see miscellaneous section)
pvcreate –uuid /dev/sdb1

Common Attributes that you may want to use:

-M2 create a LVM2 physical volume
removingpvremove /dev/sdb1
checkingpvck -v /dev/sdb1

Note: check the consistency of the LVM metadata
change physical attributes## do not allow allocation of extents on this drive, however the partition must be in a vg otherwise you get an error
pvchange -x n /dev/sdb1

Common Attributes that you may want to use:–addtag add a tag
-x allowed to allocate extents
-u change the uuid
movingpvmove -v /dev/sdb2 /dev/sdb3

Note: moves any used extents from this volume to another volume, in readiness to remove that volume. However you cannot use this on mirrored volumes, you must convert back to non-mirror using “lvconvert -m 0”
Volume Groups
displayvgdisplay -v
vgs -v
vgs -a -o +devices

vgs flags:
#PV – number of physical devices
#LV – number of configured volumes        

vgs attributes are:
1. permissions (r)|(w)
2. resi(z)eable
3. e(x)ported
4. (p)artial
5. allocation policy – (c)ontiguous, c(l)ing, (n)ormal, (a)nywhere, (i)nherited
6. (c)luster
scanningvgscan -v
creatingvgcreate VolData00 /dev/sdb1 /dev/sdb2 /dev/sdb3
vgcreate VolData00 /dev/sdb[123]

## Use 32MB extent size
vgcreate VolData00 -s 32 /dev/sdb1

Common Attributes that you may want to use:

-l  maximum logical volumes
-p maximum physical volumes
-s physical extent size (default is 4MB)
-A autobackup
extendingvgextend VolData00 /dev/sdb3
reducingvgreduce VolData00 /dev/sdb3

vgreduce –removemissing –force VolData00
removingvgremove VolData00

Common Attributes that you may want to use:

-f force the removal of any logical volumes
checkingvgck VolData00

Note: check the consistency of the LVM metadata
change volume attributesvgchange -a n VolData00

Common Attributes that you may want to use:

-a control availability of volumes within the group
-l  maximum logical volumes
-p maximum physical volumes
-s physical extent size (default is 4MB)
-x resizable yes or no (see VG status in vxdisplay)
renamingvgrename VolData00 Data_Vol_01

note: the volume group must not have any active logical volumes
converting metadata typevgconvert -M2 VolData00

Note: vgconvert allows you to convert from one type of metadata format to another for example from LVM1 to LVM2, LVM2 offers bigger capacity, clustering and mirroring
merging# the old volumes group will be merged into the new volume group
vgmerge New_Vol_Group Old_Vol_Group

Note: you must unmount any fielsystems and deactivate the vg that is being merged “vgchange -a n “, then you can activiate it again afterwards “vgchange -a y “, then perform a vgscan, dont forget to backup the configuration
splitingvgsplit Old_Vol_Group New_Vol_Group [physical volumes] [-n logical volume name]
importingvgimport VolData00

Common Attributes that you may want to use:

-a import all exported volume groups
exporting## to see if a volume has already been export use “vgs” and look at the third attribute should be a x
vgexport VolData00

Common Attributes that you may want to use:

-a export all inactive volume groups
backing up## Backup to default location (/etc/lvm/backup)
vgcfgbackup VolData00

# Backup to specific location
vgcfgbackup -f /var/backup/VolData00_bkup VolData00# Backup to specific location all volume groups (notice the %s)
vgcfgbackup -f /var/backup/vg_backups_%s

Note: the backup is written in plain text and are by default located in /etc/lvm/backup
restoringvgcfgrestore -f /var/backup/VolData00_bkup VolData00

Common Attributes that you may want to use:

-l list backups of file
-f backup file
-M metadataype 1 or 2
cloningvgimportclone /dev/sdb1

Note: used to import and rename duplicated volume group
special filesvgmknodes VolData00

Note: recreates volume group directory and logical volume special files in /dev
Logical Volumes
displaylvdisplay -v
lvdisplay –maps      display mirror volumes

lvs -v
lvs -a -o +devices

## lvs commands for mirror volumes
lvs -a -o +devices
lvs -a -o +seg_pe_ranges –segments## Stripe size
lvs -v –segments
lvs -a -o +stripes,stripesize## use complex command
lvs -a -o +devices,stripes,stripesize,seg_pe_ranges –segments

lvs attributes are:
1. volume type: (m)irrored, (M)irrored without initail sync, (o)rigin, (p)vmove, (s)napshot, invalid (S)napshot, (v)irtual, mirror (i)mage
                      mirror (I)mage out-of-sync, under (c)onversion
2. permissions: (w)rite, (r)ead-only
3. allocation policy – (c)ontiguous, c(l)ing, (n)ormal, (a)nywhere, (i)nherited
4. fixed (m)inor
5. state: (a)ctive, (s)uspended, (I)nvalid snapshot, invalid (S)uspended snapshot, mapped (d)evice present with-out tables,
             mapped device present with (i)nactive table
6. device (o)pen (mounted in other words)
scanninglvscan -v
lvmdiskscan
creating## plain old volume
lvcreate -L 10M VolData00## plain old volume but use extents, use 10 4MB extents (if extent size is 4MB)
lvcreate -l 10 VolData00

## plain old volume but with a specific name web01
lvcreate -L 10M -n web01 VolData00

## plain old volume but on a specific disk
lvcreate -L 10M VolData00 /dev/sdb1

## a striped volume called lvol1 (note the captial i for the stripe size), can use -l (extents) instead of -L
lvcreate -i 3 -L 24M -n lvol1 vg01## Mirrored volume
lvcreate -L 10M -m1 -n data01 vg01

## Mirrored volume without a mirror log file
lvcreate -L 10M -m1 –mirrorlog core -n data01 vg01

Common Attributes that you may want to use:

-L size of the volume [kKmMgGtT]
-l number of extents
-C contiguous [y|n]
-i stripes
-I stripe size
-m mirrors
–mirrorlog
-n volume name
extendinglvextend -L 20M /dev/VolData00/vol01Common Attributes that you may want to use:-L size of the volume [kKmMgGtT]
-l number of extents
-C contiguous [y|n]
-i stripes
-I stripe size

Note: you can extend a ext2/ext3 filesystem using the “resize2fs” or “fsadm” command

fsadm resize /dev/VolData01/data01
resize2fs -p /dev/mapper/VolData01-data01 [size]

The -p option displays bars of progress while extendingthe filesystem
reducing/resizinglvreduce -L 5M /dev/VolData00/vol01
lvresize -L 5M /dev/VolData00/vol01

Note: rounding will occur when extending and reducing volumes to the next extent (4MB by default), you can use resize2fs or fsadm to shrink the filesystemfsadm resize /dev/VolData01/data01 [size]
resize2fs -p /dev/mapper/VolData01-data01 [size]
removinglvremove /dev/VolData00/vol01
adding a mirror to a non-mirrored volumelvconvert -m1 –mirrorlog core /dev/VolData00/vol01 /dev/sdb2Note: you can also use the above command to remove a unwanted log
removing a mirror from a mirrored volumelvconvert -m0 /dev/VolData00/vol01 /dev/sdb2Note: the disk in the command is the one you want to remove
Mirror a volume that has stripeslvconvert –stripes 3 -m1 –mirrorlog core /dev/VolData00/data01 /dev/sdd1 /dev/sde1 /devsdf1
change volume attributeslvchange -a n /dev/VolData00/vol01

Common Attributes that you may want to use:-a availability
-C contiguous [y|n]
renaminglvrename /dev/VolData00/vol_old /dev/VolData00/vol_new
snapshottinglvcreate –size 100M –snapshot -name snap /dev/vg01/data01
Miscellaneous
Simulating a disk failuredd if=/dev/zero of=/dev/sdb2 count=10
reparing a failed mirror no LVM corruption## check volume, persume /dev/sdb2 has failed
lvs -a -o +devices

# remove the failed disk from the volume (if not already done so) , this will convert volume into a non-mirrored volume
vgreduce –removemissing –force VolData00

## replace the disk physically, remember to partion it with type 8e
fdisk /dev/sdb
……..

## add new disk to LVM
pvcreate /dev/sdb2

## add the disk back into volume group
vgextend VolData00 /dev/sdb2

## mirror up the volume
lvconvert -m1 –mirrorlog core /dev/VolData00/vol02 /dev/sdb2
corrupt LVM metadata without replacing drive# attempt to bring the volume group online
vgchange -a y VolData00

# Restore the LVM configation
vgcfgrestore VolData00

# attempt to bring the volume grou online
vgchange -a y VolData00

# file system check
e2fsck /dev/VolData00/data01
corrupt LVM metadata but replacing the faulty disk# attempt to bring the volume group online but you get UUID conflict errors make note of the UUID number
vgchange -a y VolData00
vgchange -a n VolData00

## sometimes it my only be a logical volume problem
lvchange -a y /dev/VolData00/web02
lvchange -a n /dev/Voldata00/web02

## replace the disk physically, remember to partion it with type 8e
fdisk /dev/sdb
……..

# after replacing the faulty drive the disk must have the previuos UUID number or you can get it from /etc/lvm directory
pvcreate –uuid <previous UUID number taken from above command> /dev/sdb2

# Restore the LVM configation
vgcfgrestore VolData00

# attempt to bring the volume group online or logical volume
vgchange -a y VolData00
lvchange -a y /dev/VolData00/web02

# file system check
e2fsck /dev/VolData00/data01Note: if you have backed the volume group configuration you can obtain the UUID number in the backup file by default located in /etc/lvm/backup or running “pvs -v”

ip command examples

The linux ip command is similar to ifconfig, but more powerful and is intended to be a replacement for it. With ip you have the advantage of performing several network administration tasks with only one command.
In this tutorial, we are going to discuss 15 most common uses for ‘ip’ command, so let’s get going.

1. Identify available network interface with ip command

If you run ip link show command it will list all available network interfaces on your server.

$ ip link show

Example output:

$ ip link show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s3:  mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000
    link/ether 08:00:27:64:ab:7a brd ff:ff:ff:ff:ff:ff

2. How to Check an IP Address

To get the depth information of your network interfaces like IP Address, MAC Address information, use the following command as shown below.

$ ip addr show

The output will display the currently assign IP configuration for all network interfaces.

$ ip addr show
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp0s3:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:64:ab:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.8/24 brd 192.168.100.255 scope global enp0s3
       valid_lft forever preferred_lft forever
    inet 192.168.100.20/24 scope global secondary enp0s3:1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe64:ab7a/64 scope link 
       valid_lft forever preferred_lft forever

3. How to Enable Network Interface

The “up” flag with interface name (enp0s3) enables a network interface. For example, the following command will activates the enp0s3 network interface.

$ ip link enp0s3 up
Or
$ sudo ip link set enp0s3 up

4. How to Disable Network Interface

The “down” flag with interface name  disables a network interface. For example, the following command will De-activates the enp0s3 network interface.

$ ip link set enp0s3 down
or
$ sudo ip link set enp0s3 down

5.How to Assign a IP Address to Specific Interface

To assign IP address to interface, we will use:

$ sudo ip addr add 192.168.100.4/255.255.255.0 dev enp0s3

We can also set broadcast address to interface with ‘ip’ command. By default no broadcast address is set, so to set a broadcast address command is:

$ $ sudo  ip addr add broadcast 192.168.100.255 dev enp0s3

We can also set standard broadcast address along with IP address by using the following command,

$ sudo ip addr add 192.168.100.4/24 brd + dev enp0s3

As shown in the above example, we can also use ‘brd’ in place on ‘broadcast’ to set broadcast ip address.

6. How to Remove an IP Address

The following command will remove an assigned IP address from the given interface:

$ sudo ip addr del 192.168.100.4/24 dev enp0s3

7. Adding an Alias for an interface

To add an alias i.e. assign more than one IP to an interface, execute below command:

$ sudo ip addr add 192.168.100.20/24 dev enp0s3 label enp0s3:1

Example outputs:

$ ip addr show enp0s3
2: enp0s3:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:64:ab:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.100.8/24 brd 192.168.100.255 scope global enp0s3
       valid_lft forever preferred_lft forever
    inet 192.168.100.20/24 scope global secondary enp0s3:1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fe64:ab7a/64 scope link 
       valid_lft forever preferred_lft forever

8. Check routing table

Checking routing information shows us the route a packet will take to reach the destination. To check the network routing information, execute the following command:

$ ip route show

Example output:

$ ip route show
default via 192.168.100.1 dev enp0s3 proto static 
192.168.100.0/24 dev enp0s3 proto kernel scope link src 192.168.100.8

Suppose now that you have an IP address which you need to know the route packets will take. You can use route option as follows:

$ ip route get 192.168.100.4

9. Adding a static route

To change the default route, the ip command can be used as follows:

$ sudo ip route add default via 192.168.100.150/24

So now all network packets will travel via 192.168.100.150 as opposed to old default route. For changing the default route for a single interface & to make change route further, execute

$ sudo ip route add 10.0.0.2 via 192.168.100.150/24 dev enp0s3

10. Removing a static route

To remove assigned static route, simply type the following command.

$ sudo ip route del 10.0.0.2

11. How do i add default gateway

Default gateway can be specified globally or for in interface-specific config file. Advantage of default gateway is If we have more than one NIC is present in the system. You can add default gateway on the fly as shown below command.

sudo ip route add default via 192.168.100.1

12. Show network statistics

The ip command can also be used to show the statistics of the various network interfaces. To do this you can use the ip command with the option -s and then specify the network device.

$ ip -s link
ip command - show network statistics
ip command – show network statistics

When you need to get information about a particular network interface, add the option ls followed by the name of the network interface. The option -s when used more than once gives you more information about that particular interface. This can be very useful especially when trouble shooting errors in network connectivity.

ip -s -s link ls enp0s3

13.Checking ARP entries

Address Resolution Protocol (ARP) is used to translate an IP address to its corresponding physical address, commonly known as MAC address. With ip command you can view the MAC address of the devices connected in your LAN by using the option neigh or neighbour.

$ ip neigh

14. Monitor netlink messages

It is also possible to view netlink messages with ip command. The monitor option allows you to see the state of your network devices. For instance a computer on your LAN could be categorized as REACHABLE or STALE depending on its status. The command can be used as follows:

$ ip monitor all

Example output:

$ ip monitor all
[NEIGH]192.168.100.4 dev enp0s3 lladdr c8:f7:33:de:30:d7 REACHABLE
[NEIGH]192.168.100.4 dev enp0s3 lladdr c8:f7:33:de:30:d7 STALE
[NEIGH]192.168.100.4 dev enp0s3 lladdr c8:f7:33:de:30:d7 PROBE
[NEIGH]192.168.100.4 dev enp0s3 lladdr c8:f7:33:de:30:d7 REACHABLE

15. View help

If you want to find a option which is not listed in above examples, then you can look for manual.

$ man ip

Conclusion

The command ip is a must have tool for network administrators and all Linux users alike. It is time to move from ifconfig, especially when you are writing scripts.

Ubuntu 20.04 Setting Background Colour

Newer versions of Ubuntu Desktop have removed the capability to set the background to a solid colour. You may run the following command in a terminal window to accomplish this task:

To clear the current image setting:

Newer versions of Ubuntu Desktop have removed the capability to set the background to a solid colour. You may run the following command in a terminal window to accomplish this task:

To clear the current image setting:

> gsettings set org.gnome.desktop.background picture-uri ""

To set the primary colour (the first colour in a gradient or the solid colour):

> gsettings set org.gnome.desktop.background primary-color '#000000'

To set the secondary colour (the second colour in a gradient, ignored for solid):

> gsettings set org.gnome.desktop.background secondary-color '#000000'

To set the background type (‘solid’ for one color, ‘horizontal’ or ‘vertical’ for gradient):

> gsettings set org.gnome.desktop.background color-shading-type 'solid'

systemctl / systemd commands

Starting and Stopping Services

To start a systemd service, executing instructions in the service’s unit file, use the start command:

  • sudo systemctl start application.service

Sincesystemd knows to look for *.service files for service management commands, so the command could just as easily be typed like this:

  • sudo systemctl start application

To stop a currently running service, you can use the stop command instead:

  • sudo systemctl stop application.service

Restarting and Reloading

To restart a running service, you can use the restart command:

  • sudo systemctl restart application.service

If the application in question is able to reload its configuration files (without restarting), you can issue the reload command to initiate that process:

  • sudo systemctl reload application.service

If you are unsure whether the service has the functionality to reload its configuration, you can issue the reload-or-restart command. This will reload the configuration in-place if available. Otherwise, it will restart the service so the new configuration is picked up:

  • sudo systemctl reload-or-restart application.service

Enabling and Disabling Services

To start a service at boot, use the enable command:

  • sudo systemctl enable application.service

This will create a symbolic link from the system’s copy of the service file, usually in /lib/systemd/system or /etc/systemd/system) into the location on disk where systemd looks for autostart files (usually /etc/systemd/system/some_target.target.wants.

To disable the service from starting automatically, you can type:

  • sudo systemctl disable application.service

This will remove the symbolic link.

Enabling and Starting Services together

To enable and start a service in one command, use

  • sudo systemctl enable application.service –now

Checking the Status of Services

To check the status of a service on your system, you can use the status command:

  • systemctl status application.service

There are also methods for checking for specific states. For instance, to check to see if a unit is currently active (running), you can use the is-active command:

  • systemctl is-active application.service

This will return the current unit state, which is usually active or inactive. The exit code will be “0” if it is active, making the result simpler to parse in shell scripts.

To see if the unit is enabled, you can use the is-enabled command:

  • systemctl is-enabled application.service

This will output whether the service is enabled or disabled and will again set the exit code to “0” or “1” depending on the answer to the command question.

A third check is whether the unit is in a failed state. This indicates that there was a problem starting the unit in question:

  • systemctl is-failed application.service

This will return active if it is running properly or failed if an error occurred. If the unit was intentionally stopped, it may return unknown or inactive. An exit status of “0” indicates that a failure occurred and an exit status of “1” indicates any other status.

System State Overview

The commands so far have been useful for managing single services, but they are not very helpful for exploring the current state of the system. There are a number of systemctl commands that provide this information.

Listing Current Units

To see a list of all of the active units that systemd knows about, we can use the list-units command:

  • systemctl list-units

This will show you a list of all of the units that systemd currently has active on the system. The output will look something like this:

Output

UNIT                                      LOAD   ACTIVE SUB     DESCRIPTION
atd.service                               loaded active running ATD daemon
avahi-daemon.service                      loaded active running Avahi mDNS/DNS-SD Stack
dbus.service                              loaded active running D-Bus System Message Bus
dcron.service                             loaded active running Periodic Command Scheduler
dkms.service                              loaded active exited  Dynamic Kernel Modules System
getty@tty1.service                        loaded active running Getty on tty1
. . .

The output has the following columns:

  • UNIT: The systemd unit name
  • LOAD: Whether the unit’s configuration has been parsed by systemd. The configuration of loaded units is kept in memory.
  • ACTIVE: A summary state about whether the unit is active. This is usually a fairly basic way to tell if the unit has started successfully or not.
  • SUB: This is a lower-level state that indicates more detailed information about the unit. This often varies by unit type, state, and the actual method in which the unit runs.
  • DESCRIPTION: A short textual description of what the unit is/does.

Since the list-units command shows only active units by default, all of the entries above will show loaded in the LOAD column and active in the ACTIVE column. This display is actually the default behavior of systemctl when called without additional commands, so you will see the same thing if you call systemctl with no arguments:

  • systemctl

We can tell systemctl to output different information by adding additional flags. For instance, to see all of the units that systemd has loaded (or attempted to load), regardless of whether they are currently active, you can use the --all flag, like this:

  • systemctl list-units –all

This will show any unit that systemd loaded or attempted to load, regardless of its current state on the system. Some units become inactive after running, and some units that systemd attempted to load may have not been found on disk.

You can use other flags to filter these results. For example, we can use the --state= flag to indicate the LOAD, ACTIVE, or SUB states that we wish to see. You will have to keep the --all flag so that systemctl allows non-active units to be displayed:

  • systemctl list-units –all –state=inactive

Another common filter is the --type= filter. We can tell systemctl to only display units of the type we are interested in. For example, to see only active service units, we can use:

  • systemctl list-units –type=service

Listing All Unit Files

The list-units command only displays units that systemd has attempted to parse and load into memory. Since systemd will only read units that it thinks it needs, this will not necessarily include all of the available units on the system. To see every available unit file within the systemd paths, including those that systemd has not attempted to load, you can use the list-unit-files command instead:

  • systemctl list-unit-files

Units are representations of resources that systemd knows about. Since systemd has not necessarily read all of the unit definitions in this view, it only presents information about the files themselves. The output has two columns: the unit file and the state.

Output

UNIT FILE                                  STATE   
proc-sys-fs-binfmt_misc.automount          static  
dev-hugepages.mount                        static  
dev-mqueue.mount                           static  
proc-fs-nfsd.mount                         static  
proc-sys-fs-binfmt_misc.mount              static  
sys-fs-fuse-connections.mount              static  
sys-kernel-config.mount                    static  
sys-kernel-debug.mount                     static  
tmp.mount                                  static  
var-lib-nfs-rpc_pipefs.mount               static  
org.cups.cupsd.path                        enabled
. . .

The state will usually be enabled, disabled, static, or masked. In this context, static means that the unit file does not contain an install section, which is used to enable a unit. As such, these units cannot be enabled. Usually, this means that the unit performs a one-off action or is used only as a dependency of another unit and should not be run by itself.

We will cover what masked means momentarily.

Unit Management

So far, we have been working with services and displaying information about the unit and unit files that systemd knows about. However, we can find out more specific information about units using some additional commands.

Displaying a Unit File

To display the unit file that systemd has loaded into its system, you can use the cat command (this was added in systemd version 209). For instance, to see the unit file of the atd scheduling daemon, we could type:

  • systemctl cat atd.service

Output

[Unit]
Description=ATD daemon
[Service]
Type=forking
ExecStart=/usr/bin/atd
[Install]
WantedBy=multi-user.target

The output is the unit file as known to the currently running systemd process. This can be important if you have modified unit files recently or if you are overriding certain options in a unit file fragment (we will cover this later).

Displaying Dependencies

To see a unit’s dependency tree, you can use the list-dependencies command:

  • systemctl list-dependencies sshd.service

This will display a hierarchy mapping the dependencies that must be dealt with in order to start the unit in question. Dependencies, in this context, include those units that are either required by or wanted by the units above it.

Output

sshd.service
├─system.slice
└─basic.target
  ├─microcode.service
  ├─rhel-autorelabel-mark.service
  ├─rhel-autorelabel.service
  ├─rhel-configure.service
  ├─rhel-dmesg.service
  ├─rhel-loadmodules.service
  ├─paths.target
  ├─slices.target
. . .

The recursive dependencies are only displayed for .target units, which indicate system states. To recursively list all dependencies, include the --all flag.

To show reverse dependencies (units that depend on the specified unit), you can add the --reverse flag to the command. Other flags that are useful are the --before and --after flags, which can be used to show units that depend on the specified unit starting before and after themselves, respectively.

Checking Unit Properties

To see the low-level properties of a unit, you can use the show command. This will display a list of properties that are set for the specified unit using a key=value format:

  • systemctl show sshd.service

Output

Id=sshd.service
Names=sshd.service
Requires=basic.target
Wants=system.slice
WantedBy=multi-user.target
Conflicts=shutdown.target
Before=shutdown.target multi-user.target
After=syslog.target network.target auditd.service systemd-journald.socket basic.target system.slice
Description=OpenSSH server daemon
. . .

If you want to display a single property, you can pass the -p flag with the property name. For instance, to see the conflicts that the sshd.service unit has, you can type:

  • systemctl show sshd.service -p Conflicts

Output

Conflicts=shutdown.target

Masking and Unmasking Units

We saw in the service management section how to stop or disable a service, but systemd also has the ability to mark a unit as completely unstartable, automatically or manually, by linking it to /dev/null. This is called masking the unit, and is possible with the mask command:

  • sudo systemctl mask nginx.service

This will prevent the Nginx service from being started, automatically or manually, for as long as it is masked.

If you check the list-unit-files, you will see the service is now listed as masked:

  • systemctl list-unit-files

Output

. . .
kmod-static-nodes.service              static  
ldconfig.service                       static  
mandb.service                          static  
messagebus.service                     static  
nginx.service                          masked
quotaon.service                        static  
rc-local.service                       static  
rdisc.service                          disabled
rescue.service                         static
. . .

If you attempt to start the service, you will see a message like this:

  • sudo systemctl start nginx.service

Output

Failed to start nginx.service: Unit nginx.service is masked.

To unmask a unit, making it available for use again, use the unmask command:

  • sudo systemctl unmask nginx.service

This will return the unit to its previous state, allowing it to be started or enabled.

Editing Unit Files

While the specific format for unit files is outside of the scope of this tutorial, systemctl provides built-in mechanisms for editing and modifying unit files if you need to make adjustments. This functionality was added in systemd version 218.

The edit command, by default, will open a unit file snippet for the unit in question:

  • sudo systemctl edit nginx.service

This will be a blank file that can be used to override or add directives to the unit definition. A directory will be created within the /etc/systemd/system directory which contains the name of the unit with .d appended. For instance, for the nginx.service, a directory called nginx.service.d will be created.

Within this directory, a snippet will be created called override.conf. When the unit is loaded, systemd will, in memory, merge the override snippet with the full unit file. The snippet’s directives will take precedence over those found in the original unit file.

If you wish to edit the full unit file instead of creating a snippet, you can pass the --full flag:

  • sudo systemctl edit –full nginx.service

This will load the current unit file into the editor, where it can be modified. When the editor exits, the changed file will be written to /etc/systemd/system, which will take precedence over the system’s unit definition (usually found somewhere in /lib/systemd/system).

To remove any additions you have made, either delete the unit’s .d configuration directory or the modified service file from /etc/systemd/system. For instance, to remove a snippet, we could type:

  • sudo rm -r /etc/systemd/system/nginx.service.d

To remove a full modified unit file, we would type:

  • sudo rm /etc/systemd/system/nginx.service

After deleting the file or directory, you should reload the systemd process so that it no longer attempts to reference these files and reverts back to using the system copies. You can do this by typing:

  • sudo systemctl daemon-reload

Adjusting the System State (Runlevel) with Targets

Targets are special unit files that describe a system state or synchronization point. Like other units, the files that define targets can be identified by their suffix, which in this case is .target. Targets do not do much themselves, but are instead used to group other units together.

This can be used in order to bring the system to certain states, much like other init systems use runlevels. They are used as a reference for when certain functions are available, allowing you to specify the desired state instead of the individual units needed to produce that state.

For instance, there is a swap.target that is used to indicate that swap is ready for use. Units that are part of this process can sync with this target by indicating in their configuration that they are WantedBy= or RequiredBy= the swap.target. Units that require swap to be available can specify this condition using the Wants=, Requires=, and After= specifications to indicate the nature of their relationship.

Getting and Setting the Default Target

The systemd process has a default target that it uses when booting the system. Satisfying the cascade of dependencies from that single target will bring the system into the desired state. To find the default target for your system, type:

  • systemctl get-default

Output

multi-user.target

If you wish to set a different default target, you can use the set-default. For instance, if you have a graphical desktop installed and you wish for the system to boot into that by default, you can change your default target accordingly:

  • sudo systemctl set-default graphical.target

Listing Available Targets

You can get a list of the available targets on your system by typing:

  • systemctl list-unit-files –type=target

Unlike runlevels, multiple targets can be active at one time. An active target indicates that systemd has attempted to start all of the units tied to the target and has not tried to tear them down again. To see all of the active targets, type:

  • systemctl list-units –type=target

Isolating Targets

It is possible to start all of the units associated with a target and stop all units that are not part of the dependency tree. The command that we need to do this is called, appropriately, isolate. This is similar to changing the runlevel in other init systems.

For instance, if you are operating in a graphical environment with graphical.target active, you can shut down the graphical system and put the system into a multi-user command line state by isolating the multi-user.target. Since graphical.target depends on multi-user.target but not the other way around, all of the graphical units will be stopped.

You may wish to take a look at the dependencies of the target you are isolating before performing this procedure to ensure that you are not stopping vital services:

  • systemctl list-dependencies multi-user.target

When you are satisfied with the units that will be kept alive, you can isolate the target by typing:

  • sudo systemctl isolate multi-user.target

Using Shortcuts for Important Events

There are targets defined for important events like powering off or rebooting. However, systemctl also has some shortcuts that add a bit of additional functionality.

For instance, to put the system into rescue (single-user) mode, you can just use the rescue command instead of isolate rescue.target:

  • sudo systemctl rescue

This will provide the additional functionality of alerting all logged in users about the event.

To halt the system, you can use the halt command:

  • sudo systemctl halt

To initiate a full shutdown, you can use the poweroff command:

  • sudo systemctl poweroff

A restart can be started with the reboot command:

  • sudo systemctl reboot

These all alert logged in users that the event is occurring, something that only running or isolating the target will not do. Note that most machines will link the shorter, more conventional commands for these operations so that they work properly with systemd.

For example, to reboot the system, you can usually type:

  • sudo reboot

Tmux Tips

Summary of commands

  • tmux create a new tmux session
  • tmux ls list any existing tmux sessions
  • tmux a reattach to the last open tmux session
  • ctrl + b the default tmux command prefix
  • prefix + d detach from current tmux session
  • prefix + c create a new window
  • prefix + 09 to switch to the numbered window
  • prefix + , rename the existing window
  • prefix + % split the current pane into two vertical panes, left and right
  • prefix + " split the current pane into two horizontal panes, top and bottom
  • prefix + q view numbered panes for current window
  • prefix + q, 09 switch to pane immediately after displaying pane numbers
  • prefix + z to zoom and unzoom

Creating Tmux sessions

To create a new Tmux session and attach to it, run the following command from the Terminal:

$ tmux

Or,

$ tmux new

Detaching from Tmux sessions

To detach from a current Tmux session, just press Ctrl+b and d. You don’t need to press this both Keyboard shortcut at a time. First press "Ctrl+b" and then press "d".

Once you’re detached from a session, you will see an output something like below.

[detached (from session 0)]

Creating named sessions

If you use multiple sessions, you might get confused which programs are running on which sessions. In such cases, you can just create named sessions.

For example if you wanted to perform some activities related to web server in a session, just create the Tmux session with a custom name, for example “webserver” (or any name of your choice).

$ tmux new -s webserver

List Tmux sessions

To view the list of open Tmux sessions, run:

$ tmux ls

Attaching to Tmux sessions

You can attach to the last created session by running this command:

$ tmux attach

Or,

$ tmux a

If you want to attach to any specific named session, for example “myoffice”, run:

$ tmux attach -t myoffice

Or, shortly:

$ tmux a -t myoffice

Kill Tmux sessions

When you’re done and no longer required a Tmux session, you can kill it at any time with command:

$ tmux kill-session -t myoffice

To kill when attached, press Ctrl+b and x. Hit "y" to kill the session.

You can verify if the session is closed with tmux ls command.

To Kill Tmux server along with all Tmux sessions, run:

$ tmux kill-server

Be careful! This will terminate all Tmux sessions even if there are any running jobs inside the sessions without any warning.

When there were no running Tmux sessions, you will see the following output:

$ tmux ls
no server running on /tmp/tmux-1000/default

Split Tmux Session Windows

Tmux has an option to split a single Tmux session window into multiple smaller windows called Tmux panes.

This way we can run different programs on each pane and interact with all of them simultaneously.

Each pane can be resized, moved and closed without affecting the other panes. We can split a Tmux window either horizontally or vertically or both at once.

Split panes horizontally

To split a pane horizontally, press Ctrl+b and " (single quotation mark).

Split panes vertically

To split a pane vertically, press Ctrl+b and %.

Split panes horizontally and vertically

We can also split a pane horizontally and vertically at the same time. Take a look at the following screenshot.

First, I did a horizontal split by pressing Ctrl+b " and then split the lower pane vertically by pressing Ctrl+b %.

As you see in the above screenshot, I am running three different programs on each pane.

Switch between panes

To switch between panes, press Ctrl+b and Arrow keys (Left, Right, Up, Down).

Send commands to all panes

In the previous example, we run three different commands on each pane. However, it is also possible to run send the same commands to all panes at once.

To do so, press Ctrl+b and type the following command and hit ENTER:

:setw synchronize-panes

Now type any command on any pane. You will see that the same command is reflected on all panes.

Swap panes

To swap panes, press Ctrl+b and o.

Show pane numbers

Press Ctrl+b and q to show pane numbers.

Kill panes

To kill a pane, simply type exit and ENTER key. Alternatively, press Ctrl+b and x. You will see a confirmation message. Just press "y" to close the pane.

Zoom in and Zoom out Tmux panes

Now, I am going to zoom the left pane which has top command running. To do so, switch to left pane and press and z. The tmux prefix command is Ctrl+b, so I type Ctrl+b followed by z to zoom the pane. It is that simple!

Note: To switch between panes, press Ctrl+b and Arrow keys (Left, Right, Up, Down).

Autostart Tmux Session On Remote System When Logging In Via SSH

To autostart Tmux session when connecting via SSH, edit your remote system’s ~/.bash_profile file:

$ nano ~/.bash_profile

If the file is not available, just create it.

And add the following lines in it:

if [ -z "$TMUX" ]; then
    tmux attach -t default || tmux new -s default
fi

Save and close the file. Log out and log back into the remote systems. You will be landed into a new Tmux session named ‘default’.

Now, start any process inside the Tmux session and detach it from by pressing Ctrl+b followed by d. The process will keep running inside the Tmux session even after the network connection is dropped. Once the network connect is back, SSH into the remote system and you will see that the job is still running!!

Create session in tmux

Toggling Status Line

Turn it off:

$ tmux set-option status off

And, turn it on:

$ tmux set-option status on

You can also bind a key (t in this example) to toggle the status line on and off, as follows:

$ tmux bind-key t set-option status

Open a terminal and use the following command:

# tmux

There’s a difference between Session vs Window vs Pane (split) in tmux

Windows:

Create windows using:

[ctrl + b] + c

Switching windows in a tmux session

PreviousNextn (0, 1, 2, 3 etc.)
[ctrl + b] + p [ctrl + b] + nSwitch to n Window: [ctrl + b] + 0

Switch using window list

Another option uses [ctrl+b] + w

Name your current window using:

[ctrl+b] + ,

Name your tmux session using:

tmux new -s sessionname

If you already created a session, rename it using:

[ctrl+b] + $

Detach a session (come out of tmux session but keep it running in background)

use:

[ctrl + b] + d

List tmux sessions:

tmux ls

Reenter a session in Tmux (also known as attach)

tmux attach -t session_name_or_number

Creating multiple panes in tmux

Horizontal (Left/Right)Vertical (Up/Down)
[ctrl + b] + %[ctrl + b] + “

Switching between panes

[ctrl+b] + arrow keys

Zoom into or out of a pane

[ctrl+b] + z

Killing or closing panes

[ctrl + b] + x

Resize panes

This assumes that you’ve hit ctrl + b and : to get to the command prompt

  • :resize-pane -L

Here are some additional tmux pane resizing examples:

  • :resize-pane -D (Resizes the current pane down)
  • :resize-pane -U (Resizes the current pane upward)
  • :resize-pane -L (Resizes the current pane left)
  • :resize-pane -R (Resizes the current pane right)
  • :resize-pane -D 10 (Resizes the current pane down by 10 cells)
  • :resize-pane -U 10 (Resizes the current pane upward by 10 cells)
  • :resize-pane -L 10 (Resizes the current pane left by 10 cells)
  • :resize-pane -R 10 (Resizes the current pane right by 10 cells)

Killing sessions in tmux

tmux kill-session -t session_name_or_number

Kill all tmux sessions

tmux kill-server

CommandExplanation
tmux kill-session -t nameKill the mentioned session
tmux kill-session -aKill all sessions except the last used session
tmux kill-serverKill all tmux sessions
tmux attach -t nameAttach/enter the mentioned session
[Ctrl +b] + dDetach/exit the current session
[Ctrl +b] + $Name or rename current session
[Ctrl +b] + cCreate new window
[Ctrl +b] + ,Name or rename current window
[Ctrl +b] + wList all windows
[Ctrl +b] + &Kill the current window
[Ctrl +b] + n/p/NMove to next/previous/Nth window
[Ctrl +b] + %Create horizontal pane
[Ctrl +b] + “Create vertical pane
[Ctrl +b] + space keyToggle the layout of current pane
[Ctrl +b] + zZoom into the current pane
[Ctrl +b] + xKill the current pane