DevMemo – Blog

Blog: How To Debug Emacs Cpp Indentation

Sat, 26 Aug 2023 21:16:11 +0000

Do Syntactic Analysis

Do c-show-syntactic-information or C-c C-s to get an syntactic analysis first, you should get something like this

Syntactic analysis: ((statement-block-intro 930))

At the same time, emacs will also highlight the anchor point.

Set `c-set-offset`

Do the following to adjust the indentation.

(defun c-indentation-hook ()
 (c-set-offset 'statement-block-intro 2)
)

(add-hook 'c-mode-common-hook 'c-indentation-hook)

You’ll need to reopen the file to allow the above change to take effect.

Blog: How To Resize Disk On Linux

Thu, 13 Jul 2023 05:40:50 +0000

Assuming you have a Linux VM, and you enlarged the disk. Take the following steps on your Linux to apply the changes.

Step 1: Resize the volume

The easiest way to resize your volume is to use cfdisk.

sudo cfdisk

This is what it looks like on my VM.

 Disk: /dev/sda
 Size: 64 GiB, 68719476736 bytes, 134217728 sectors
 Label: gpt, identifier: 89F6152F-43FB-4262-8B2E-06EF3344C5A3

 Device Start End Sectors Size Type
>> /dev/sda1 2048 4095 2048 1M BIOS boot
 /dev/sda2 4096 134217694 134213599 64G Linux filesystem



 ┌────────────────────────────────────────────────────────────────────────────────────┐
 │Partition UUID: 1BDD55B2-1581-4671-878E-5891F4802220 │
 │Partition type: BIOS boot (21686148-6449-6E6F-744E-656564454649) │
 └────────────────────────────────────────────────────────────────────────────────────┘
 [ Delete ] [ Resize ] [ Quit ] [ Type ] [ Help ] [ Write ] [ Dump ]

Go to your target device, and then resize it as you need. For my case, it is /dev/sda2, because that’s where the root directory is mounted.

Step 2: Apply the change to your filesystem.

To apply the change to your file system, run the following command

sudo resize2fs /dev/sda2

Now you have added more spaces to your disk.

Blog: Set Up VPN Server And VPN Router

Sun, 02 Jul 2023 04:57:29 -0700

Set Up VPN Server

Deploy an ubuntu 22.04 server on your target network. Run the following commands to install vpn server on it.

wget https://raw.githubusercontent.com/Angristan/openvpn-install/master/openvpn-install.sh -O ubuntu-22.04-lts-vpn-server.sh
chmod -v +x ubuntu-22.04-lts-vpn-server.sh
sudo ./ubuntu-22.04-lts-vpn-server.sh

Once it is done, an ovpn file will be generated, for example,

The configuration file has been written to /home/ubuntu/linux.ovpn.

Set Up VPN Router

Step 1: Install An Physical Ubuntu Server

Install ubuntu on a device which has:

A physical ethernet port connected to your home router. This port will be used to exchange data from and to the Internet.
A wifi adapter. This port will be used as a hotspot for your devices to connect. Any device connected to this hotspot will be able use send traffic through the vpn server.

This ubuntu server will be used as your VPN router.

Step 2: Set Up DNS

Run the following command to set up your DNS on your VPN router.

echo "[Resolve]
DNS=8.8.8.8 8.8.4.4
" | sudo tee /etc/systemd/resolved.conf.d/99-dns.conf

Restart your VPN router,

Step 3: Create A Wifi Hotspot

Run the following command to create a wifi hotspot.

sudo nmcli d wifi hotspot ifname {interface-name} ssid {wifi-ssid} password {wifi-passwd}

Step 4: Connect To Your VPN Server From Your VPN Router

Copy the opvn config file from your VPN server to your VPN router, and run the following command.

sudo openvpn --config {ovpn-config-file}

Step 5: Enable NAT On Ubuntu Server

iptables -t nat -A POSTROUTING -j MASQUERADE

This config will go away if you reboot your linux server. To make it persistent, do the following

sudo apt update -y && sudo apt install iptables-persistent

Now your VPN router is ready to use.

Blog: How To Use mDNS Within Docker Container

Sat, 17 Jun 2023 09:56:48 -0700

mDNS is a service that allows you to access hosts within your LAN by using their hostnames. However, it is not working by default in a docker container.

In order to use mDNS in a docker container, you’ll have to:

Install avahi-utils within your container.
Map these two files into your container, /var/run/dbus and /var/run/avahi-daemon/socket.

For example, you can start your container like:

docker run -v /var/run/dbus:/var/run/dbus -v /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket -it debian:10

and then run the following commands within your contain:

apt update && apt install avahi-utils -y

Now, you can ping your hosts within your LAN, e.g.

root@cccbde9152f2:/# ping nfs-server.local -c 4
PING nfs-server.local (192.168.1.101) 56(84) bytes of data.
64 bytes from 192.168.1.101 (192.168.1.101): icmp_seq=1 ttl=63 time=0.193 ms
64 bytes from 192.168.1.101 (192.168.1.101): icmp_seq=2 ttl=63 time=0.208 ms
64 bytes from 192.168.1.101 (192.168.1.101): icmp_seq=3 ttl=63 time=0.241 ms
64 bytes from 192.168.1.101 (192.168.1.101): icmp_seq=4 ttl=63 time=0.252 ms

--- nfs-server.local ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 36ms
rtt min/avg/max/mdev = 0.193/0.223/0.252/0.028 ms
root@cccbde9152f2:/#

Blog: Automate Proxmox VM Creation With Cloud-Init

Mon, 06 Feb 2023 15:10:26 -0800

This post gives you step-by-step instructions of how to automate proxmox VM creation. I’ll use ubuntu as an example.

Overview

The overall strategy is to use the ubuntu cloud image to create a Proxmox VM template, and attach a cloud-init disk to it. After that, we can clone the template to create new VM, and new VM will automatically bootstrap itself, e.g., changing the hostname.

Prepare tools

Install virt-customize

apt-get install libguestfs-tools

Prepare The Ubuntu Cloud Image

Use the following commands to prepare the ubuntu cloud image:

# Create a new directory for our image building.
mkdir ubuntu-cloud-image
cd ubuntu-cloud-image

# Download the cloud image.
wget https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img

# Resize the image to 16GB. Feel free to change it to your desired size.
qemu-img resize noble-server-cloudimg-amd64.img 16G

# Install avahi-deamon for mdns.
virt-customize -a noble-server-cloudimg-amd64.img --run-command 'apt update && apt install avahi-daemon -y && cloud-init clean --logs --machine-id --configs all --reboot && sudo truncate -s 0 /etc/machine-id /var/lib/dbus/machine-id'

Prepare The Proxmox VM Template

Then, we prepare the Proxmox VM template with the following steps.

Create A Temporary VM

# Create a VM with the following configuration:
# * vmid: 1000 (feel free to change this)
# * memory: 4GB
# * network: attached to bridge vmbr0
qm create 1000 --name ubuntu-noble-template --memory 4096 --net0 virtio,bridge=vmbr0

Attach The Ubuntu Cloud Image

# Attach the ubuntu cloud image onto the VM.
# Note that here ssd is my storage name. You'll need to replace it with yours.
qm importdisk 1000 noble-server-cloudimg-amd64.img ssd

# Make the attached disk scsi0.
qm set 1000 --scsihw virtio-scsi-pci --scsi0 ssd:vm-1000-disk-0

# Set scsi0 as the first boot device.
qm set 1000 --boot c --bootdisk scsi0

Create A Cloud-init Drive

# Create a cloud-init drive.
qm set 1000 --ide2 ssd:cloudinit

# Configure your cloud-init drive. Replace the username, password and sshkeys for your case.
qm set 1000 --ciuser swe --cipassword swe-passwd --sshkeys ~/.ssh/id_rsa.pub --ipconfig0 ip=dhcp

Convert The VM Into A Template

# Convert the VM into a template.
qm template 1000

Create VM

# Create a VM using the template, and set the name to ubuntu-vm1.
qm clone 1000 1101 --full --name ubuntu-vm1

# Create another VM using the template, and set the name to ubuntu-vm2.
qm clone 1000 1102 --full --name ubuntu-vm2

To access these VMs, you can do

# Use the following command to access ubuntu-vm1 if you have mdns included in your template, and you are within the same LAN as ubuntu-vm1.
ssh swe@ubuntu-vm1.local

# Use the following command to access ubuntu-vm2 if you have mdns included in your template, and you are within the same LAN as ubuntu-vm2.
ssh swe@ubuntu-vm2.local

Blog: Automate Ubuntu Installation

Wed, 01 Feb 2023 16:22:53 -0800

As a software engineer, I frequently needed to create new Linux VMs for testing and other purposes. The traditional way of installing Linux on a VM is very inefficient, which consumes a lot of my time. Automating the process is very appealing to me.

In this post, I’m going to give you step-by-step instructions of how to do that.

Build Ubuntu Auto-Install Iso Image

Prerequisites

To build an auto-install ubuntu image, we’ll need to install the following packages first

sudo apt update && sudo apt install p7zip wget xorriso whois

Set Up Build Environment

Next, let’s download the latest ubuntu image from the official website. The version we are going to be using is ubuntu 20.04.

# Go to your home directory.
cd

# Create a directory for our image building process.
mkdir ubuntu-auto-install-image-building
cd ubuntu-auto-install-image-building/

# Download the latest package.
wget https://releases.ubuntu.com/22.04.1/ubuntu-22.04.1-live-server-amd64.iso

# Create a source directory.
mkdir ubuntu-iso-sources

# Unpack the iso to the ubuntu-iso-sources dir.
7z -y x ubuntu-22.04.1-live-server-amd64.iso -oubuntu-iso-sources

# Go into the source directory.
cd ubuntu-iso-sources

# Move BOOT directory outside, since this is not needed in the final iso.
mv \[BOOT\] ../BOOT

Update Image Files

After the source files are unpacked, let’s update some files.

Create An Grub Entry

The first file to update is the grub file, i.e., ~/ubuntu-auto-install-image-building/ubuntu-iso-sources/boot/grub/grub.cfg. We’ll need to do two things:

Add an autoinstall entry.
Make wait time shorter (this is optional).

The updated file looks like this. The highlighted lines are modified.

set timeout=5

loadfont unicode

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray

menuentry "Autoinstall Ubuntu Server" {
 set gfxpayload=keep
 linux /casper/vmlinuz quiet autoinstall ds=nocloud\;s=/cdrom/server/ ---
 initrd /casper/initrd
}

menuentry "Try or Install Ubuntu Server" {
 set gfxpayload=keep
 linux /casper/vmlinuz ---
 initrd /casper/initrd
}
grub_platform
if [ "$grub_platform" = "efi" ]; then
menuentry 'Boot from next volume' {
 exit 1
}
menuentry 'UEFI Firmware Settings' {
 fwsetup
}
else
menuentry 'Test memory' {
 linux16 /boot/memtest86+.bin
}
fi

Create File user-data

Next, we’ll need to create a user-data file, ~/ubuntu-auto-install-image-building/ubuntu-iso-sources/server/user-data, with the following content.

#cloud-config
autoinstall:
 version: 1
 keyboard:
 layout: us
 identity:
 hostname: ubuntu-server
 password: "$y$j9T$qiKEYsg5oLCDV.XxExdEB/$41TEJJWV7he/aTyHQEDAWJyfKceDtCYVk.Agh9aAvk3"
 username: swe
 ssh:
 allow-pw: true
 install-server: true

This is a minimum setup, which only creates a user with a password. The password is devmemo-passwd. The password string above is generated using the following command

mkpasswd "devmemo-passwd"

Note that the above config is really a minimum one. Check out this page if you need more configurations.

Create File meta-data

For our use case, we don’t need any data in meta-data file. So we’ll just need to create an empty file.

touch ~/ubuntu-auto-install-image-building/ubuntu-iso-sources/server/meta-data

Create An Auto-Installation Iso Image

# Create the auto-installation iso image.
xorriso -as mkisofs -r \
 -V 'Ubuntu 22.04 LTS (Auto Install)' \
 -o ../ubuntu-22.04-autoinstall.iso \
 --grub2-mbr ../BOOT/1-Boot-NoEmul.img \
 -partition_offset 16 \
 --mbr-force-bootable \
 -append_partition 2 28732ac11ff8d211ba4b00a0c93ec93b ../BOOT/2-Boot-NoEmul.img \
 -appended_part_as_gpt \
 -iso_mbr_part_type a2a0d0ebe5b9334487c068b6b72699c7 \
 -c '/boot.catalog' \
 -b '/boot/grub/i386-pc/eltorito.img' \
 -no-emul-boot -boot-load-size 4 -boot-info-table --grub2-boot-info \
 -eltorito-alt-boot \
 -e '--interval:appended_partition_2:::' \
 -no-emul-boot .

# Check out the image.
cd ..
ls -alh ubuntu-22.04-autoinstall.iso

Now the ubuntu auto-install image is created. You can use it to auto install ubuntu on physical machines or VMs.

Create Your VMs

After you finish the last section, you already have an ubuntu auto-installation image. This section is totally optional.

In this section, I’ll demonstrate how to use this image on proxmox.

Upload The Image To Proxmox Host

First, let’s upload the image to the VM host. You can either use the web UI, or directly copy the iso into /var/lib/vz/template/iso.

Create A VM

Next, let’s create a VM, put the iso into its cdrom, and make the cdrom the boot device.

qm create 1000 --name vm1000 --cores 4 --memory 8192 \
 --scsi0 file=ssd:32 --net0 virtio,bridge=vmbr0 \
 --cdrom local:iso/ubuntu-22.04-autoinstall.iso

For example, the above command will create a 4 core, 8GB ram, and 32GB ssd VM. Start the VM, and then all you have to do is to wait until the OS is fully installed.

Blog: Debug Gitlab Pages Build Locally

Tue, 24 Jan 2023 16:27:49 -0800

Gitlab pages is an awesome tool, which allows you to publish any static website to the internet.

One of the biggest frustration I had is that I don’t know whether a commit is breaking the build before submitting the changes for CI. I was wishing for a local mode for running the CI job.

Luckily there is. It takes a few steps, like summarized in the following sections.

Step 1: Download Gitlab Runner

Before anything, let’s set your arch environment variable. If you are using amd64, do it like this

export arch=amd64

Then, go to a download folder, and then download the gitlab runner image

For Ubuntu or Debian

# Replace ${arch} with any of the supported architectures, e.g. amd64, arm, arm64
# A full list of architectures can be found here https://gitlab-runner-downloads.s3.amazonaws.com/latest/index.html
curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_${arch}.deb"

For CentOS or Red Hat Enterprise Linux

# Replace ${arch} with any of the supported architectures, e.g. amd64, arm, arm64
# A full list of architectures can be found here https://gitlab-runner-downloads.s3.amazonaws.com/latest/index.html
curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/rpm/gitlab-runner_${arch}.rpm"

Step 2: Install Gitlab Runner

To install gitlab runner on Ubuntu or Debian,

sudo dpkg -i "gitlab-runner_$arch.deb"

To install gitlab runner on CentOS or Red Hat Enterprise Linux

sudo rpm -i "gitlab-runner_$arch.rpm"

Step 3: Register Your Runner

Next, let’s register your runner. To register your runner, you’ll first need to acquire a token from gitlab. Go to your project, and then Settings -> CD/CD -> Runner -> Project Runners. You’ll see something like this

Grab the registration token, and run the following command

sudo gitlab-runner register --url https://gitlab.com/ --registration-token $REGISTRATION_TOKEN

The executor I selected is docker+machine, because I found it is used in the gitlab CI.
The docker image I used is the hugo image registry.gitlab.com/pages/hugo/hugo_extended:latest.

Step 4: Run Build Job Locally

Once you have finish the previous step, use the following command to run your job locally

gitlab-runner exec docker <job-name>

The job-name is defined in your .gitlab-ci.yml file. For me, the real command to run is:

gitlab-runner exec docker pages

Blog: Using LVM For Proxmox

Thu, 19 Jan 2023 15:02:22 -0800

If you are trying to use Proxmox to set up a homelab for the first time, it must be very confusing to see all the storage related terms, e.g., LVM, LVM-Thin. This post talks about everything you need to know to understand how Proxmox uses LVM as its storage infrastructure.

LVM Basics

LVM (Logical Volume Management) is actually a Linux technology. It is not Proxmox proprietary. Proxmox can leverage LVM as its storage infrastructure, but LVM itself is not part of Proxmox.

Let’s try to understand the basics of LVM, and then we try to understand how Proxmox uses it.

What Is LVM?

The overall goal of LVM is to create a virtual storage layer on top of the physical storage layer, so that the storage usage can be more flexible.

For example, if you have two 1TB disks, but you want to use them together as your root directory. LVM provides a solution to this.

How Does LVM Work?

The following image presents the overall architecture of LVM. (The image comes from wikipedia)

There are a couple of layers involved (from bottom to top):

On the bottom layer, we have the physical disk itself. These are your hard drives.
Above the hard drives, we have partitions. Each hard drive is splitted into a few partitions.
PV (physical volume) is a LVM concept. It is a representation of a physical volume. It is mapped to a single disk partition.
VG (volume group) is also a LVM concept. A VG contains a couple of PVs.
LV (logical volume) is a logical volume, which is basically a virtual volume.
On top of each LV, a file system can be created, and the LV device can be mounted to an accessible directory.

In this design, logical volume does not have to tie to a single physical disk. Logical volume can be created over multiple physical disks.

Installation LVM

On debian and ubuntu, you can do

sudo apt install lvm2

How To Use LVM?

Here are some commands to manage PV, VG and LV.

# Create a PV
pvcreate <disk-device-name>

# Remove a PV
pvremove <disk-device-name>

# List all PVs
pvs

# Create a VG
vgcreate <vg-name> <disk-device-name>

# Remove a VG
vgremove <vg-name>

# List all VGs
vgs

# Create a LV
lvcreate -L <lv-size> -n <lv-name> <vg-name>

# Remove a LV
lvremove <vg-name>/<lv-name>

# List all LVs
lvs

An Example

For example, I have a empty /dev/sdb device,

swe@ubuntu-server:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 63.3M 1 loop /snap/core20/1778
loop1 7:1 0 141.4M 1 loop /snap/docker/2285
loop2 7:2 0 55.6M 1 loop /snap/core18/2667
loop3 7:3 0 55.4M 1 loop /snap/core18/2066
loop5 7:5 0 67.6M 1 loop /snap/lxd/20326
loop6 7:6 0 49.6M 1 loop /snap/snapd/17883
loop7 7:7 0 49.8M 1 loop /snap/snapd/17950
loop8 7:8 0 91.9M 1 loop /snap/lxd/24061
loop9 7:9 0 139.4M 1 loop /snap/docker/2343
sda 8:0 0 32G 0 disk
├─sda1 8:1 0 1M 0 part
└─sda2 8:2 0 32G 0 part /
sdb 8:16 0 32G 0 disk
sr0 11:0 1 4M 0 rom
sr1 11:1 1 1024M 0 rom
swe@ubuntu-server:~$

let’s create a PV on it

swe@ubuntu-server:~$ sudo pvcreate /dev/sdb
 Physical volume "/dev/sdb" successfully created.
swe@ubuntu-server:~$ sudo pvs
 PV VG Fmt Attr PSize PFree
 /dev/sdb lvm2 --- 32.00g 32.00g
swe@ubuntu-server:~$

and then create a VG on the PV

swe@ubuntu-server:~$ sudo vgcreate test-vg /dev/sdb
 Volume group "test-vg" successfully created
swe@ubuntu-server:~$ sudo vgs
 VG #PV #LV #SN Attr VSize VFree
 test-vg 1 0 0 wz--n- <32.00g <32.00g
swe@ubuntu-server:~$

and then create a LV on the VG

swe@ubuntu-server:~$ sudo lvcreate -L 10G -n test-lv test-vg
 Logical volume "test-lv" created.
swe@ubuntu-server:~$ sudo lvs
 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 test-lv test-vg -wi-a----- 10.00g
swe@ubuntu-server:~$

Now we can create a file system on the LG, and mount it for real usage:

swe@ubuntu-server:~$ sudo mkfs.ext4 /dev/test-vg/test-lv
mke2fs 1.45.5 (07-Jan-2020)
Discarding device blocks: done
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: 5c329fcf-76ec-450a-8d96-dfb816538e3e
Superblock backups stored on blocks:
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

swe@ubuntu-server:~$ cd /
swe@ubuntu-server:/$ sudo mkdir mount
swe@ubuntu-server:/$ sudo mount /dev/test-vg/test-lv /mount
swe@ubuntu-server:/$ cd mount/
swe@ubuntu-server:/mount$ ls
lost+found
swe@ubuntu-server:/mount$

Now, we have an ext4 filesystem on the LV:

swe@ubuntu-server:/mount$ lsblk -f
NAME FSTYPE LABEL UUID FSAVAIL FSUSE% MOUNTPOINT
loop0 squashfs 0 100% /snap/core20/1778
loop1 squashfs 0 100% /snap/docker/2285
loop2 squashfs 0 100% /snap/core18/2667
loop3 squashfs 0 100% /snap/core18/2066
loop5 squashfs 0 100% /snap/lxd/20326
loop6 squashfs 0 100% /snap/snapd/17883
loop7 squashfs 0 100% /snap/snapd/17950
loop8 squashfs 0 100% /snap/lxd/24061
loop9 squashfs 0 100% /snap/docker/2343
sda
├─sda1
└─sda2 ext4 761b4178-9020-4642-ac79-d172a2a0a06d 21G 28% /
sdb LVM2_member biMIK2-WI2D-ypLl-tVGw-p8hZ-CynW-vo8swy
└─test--vg-test--lv ext4 5c329fcf-76ec-450a-8d96-dfb816538e3e 9.2G 0% /mount
sr0 iso9660 cidata 2023-01-12-13-13-29-00
sr1

How Does Proxmox Use LVM?

Now you have a basic understanding of LVM. So how does Proxmox use LVM as its storage infrastructure?

In a few words, Proxmox uses LVs for two purposes:

Use LVs as the client OS disks.
Use a LV as the host OS root directory.

For example,

root@host3:~$ pvs
 PV VG Fmt Attr PSize PFree
 /dev/nvme0n1p3 ssd lvm2 a-- <1.82t 738.50g
root@host3:~$
root@host3:~$ vgs
 VG #PV #LV #SN Attr VSize VFree
 ssd 1 13 0 wz--n- <1.82t 738.50g
root@host3:~$
root@host3:~$ lvs
 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 root ssd -wi-ao---- 96.00g
 vm-1001-disk-0 ssd -wi-ao---- 16.00g
 vm-2000-cloudinit ssd -wi-ao---- 4.00m
 vm-2000-disk-0 ssd -wi-ao---- 16.00g
 vm-2002-cloudinit ssd -wi-ao---- 4.00m
 vm-2002-disk-0 ssd -wi-ao---- 16.00g
 vm-5000-disk-0 ssd -wi-ao---- 200.00g
 vm-7000-disk-0 ssd -wi-a----- 16.00g
 vm-7000-disk-1 ssd -wi-a----- 500.00g
 vm-8000-cloudinit ssd -wi-a----- 4.00m
 vm-8000-disk-0 ssd -wi-a----- 32.00g
 vm-8000-disk-1 ssd -wi-a----- 200.00g
 vm-8001-disk-0 ssd -wi-a----- 32.00g
root@host3:~$

The above commands shows that on one of my Proxmox servers, I have a PV, /dev/nvme0n1p3, a VG, ssd, and a few LVs. Each of the client OS has its own LVs.

Using LV as the host OS storage device makes a lot of sense, which is the whole point of LVM.

LVM-Thin

By default, Proxmox does not use LVM. Instead, it uses LVM-thin. The difference between LVM-thin and LVM is that:

LVM allocates your storage blocks when you create LVs. With LVM, your LV’s storage is guaranteed for your use.
LVM-thin allocates your storage blocks when you write data to LVs. LVM-thin provides a higher disk utilization, but it can’t guarantee you can use all your LV storage.

Blog: Create Multi-Boot Usb Flash Drive With Ventoy

Wed, 18 Jan 2023 13:38:32 -0800

A multi-boot usb drive allows you to install multiple operating systems from the it. It is pretty handy. With a multi-boot usb drive, you don’t need to create a installation usb drive for each OS.

This post gives you step by step instructions to create a multi-boot usb drive.

Step 1: Install Ventoy

First of all, download the ventoy installer, from this page.

Find a usb drive. Better to have a decent size, e.g., 64G or 128G, so that you can place multiple OS images in it.

Open the ventoy installer,

and select the target usb drive, and make the ventoy usb flash drive.

NOTE: The above snapshot was taken from a Windows machine. On Linux, ventoy has a similar installer.

Step 2: Copy The OS Images

The next thing is to download your target OS installation images, and copy them into the flash drive.

In the above example, I copied five images.

Step 3: Boot Your Pc With The Multi-Boot Usb Drive

After your multi-boot usb drive is prepared, insert it into the target machine, and boot it up.

You’ll see the above menu, which allows you to select the target OS to install.

Blog: Build Static Website With Hugo And Gitlab

Mon, 16 Jan 2023 16:21:53 -0800

This article talks about everything you need to know to build an internet accessible static website, like the one you are seeing right now.

Introduction

This section introduces the key concepts of a static websites.

What Is Static Website?

A static website is basically a website which only serves static files to your browser. These files are served as they are stored on the server side. Once your browser gets back the files, it will not circle back to the server to query more information. Any action needed to render your page needs to happen on your browser side. A great example is wikipedia.

A dynamic website behaves differently. The server side not only serves static files to the browser, but also respond to API calls from the broswer. For example, amazon would need to call server side APIs to place your orders.

Dynamic websites are definitely more powerful, because it allows you to interact with the server. Static websites are less powerful, but it’s a lot easier to maintain. You can even get free internet static website servers today.

Why Hugo?

There are many ways to build static websites. There seems to be two straightforward solutions.

Option 1: Writing your website manually

The most straightforward one is to write html, css, js files manually. Although it is perfectly doable, it is not quite desirable, because there are many duplicated work which can be automated. Here’s what we need to do for a static website typically (for example, this website):

Build a navigation menu on the top, and include articles in the websites.
Build a left side structure tree.
Build a table of content for each article on the right.
Build a tagging system for the website, and users can search articles based on tags.

Ideally, website developers should only need to focus on providing the content. If we were to do it without a static website generator, it doesn’t seem to be that easy. Everytime you add a new article, you’ll have to carefully update the above items, which is definitely a nightmare. That being said, building a static website manually is not preferred, because there are two many duplicated manual work.

Option 2: Using pure python to generate your website

So certain kind of automation needs to be done here. Can we use pure python to do that? Of course we can. However, pure python doesn’t provide a html template system. Writing python code to generate html files look ugly. Part of the python code needs to generate the texts, and part of it needs to generate automated items. As a developer, you are not focusing on the content itself, but the python script to generate the content. On the other hand, python definitely does not offer you the best performance, because it is a scripting language. So using pure python to automate the website generation is also not the best way.

Both options are not the most desirable one, but it makes us realize that we need:

A html template system, which allows us to define html files with auto-generated component in it.
A tool to generate the final output fast enough.

Hugo works exactly like this. Hugo built a html template system on top of go. Go is definitely fast enough, and the template system allows developers to truly focus on writing the content itself.

Why Gitlab?

Gitlab is one of the best places in the internet to host free static websites. You can also manage your repo there, and gitlab provides CI/CD for you to auto build and deploy your change.

Another choice here is github, but I found there are two major differences between them.

In github, each user can only host one single static websites. While in gitlab, you can create many groups, and each group can have one website hosted.
In github, you can only host the website if you open source your project (unless you pay for it). In github, your source code doesn’t have to be open-sourced.

So Gitlab is clearly a better choice here in terms of the above factors.

Development

This section talks about how to develop a website with hugo, and deploy it on gitlab pages.

Step 1: Create A Gitlab Group And A Project

Log on gitlab, and go to https://gitlab.com/groups/new. Fill up the forms

Group name: let’s use examplesite.
Group URL: it’s filled up automatically based on your group name.
Visibility level: this is to say whether you repo is publicly visible. Either private or public is ok.
Role: Software Developer (or others based on your case).
Who will be using this group: Just me (or My company of team if you have a team).
What will you use this group for?: I want to store my code.

Then click on create.

Once a group is created, create an blank project in it:

Project name: This has to be examplesite.gitlab.io
Project deployment target: Gitlab Pages

Then create project.

Now let’s go back to our mac or Linux server, and find a place (say home folder) to clone your project from gitlab:

git clone https://gitlab.com/examplesite/examplesite.gitlab.io.git

you can also clone by ssh if you want,

git clone git@gitlab.com:examplesite/examplesite.gitlab.io.git

Now you have an empty git project on gitlab, which is cloned into your ~/examplesite directory.

Step 2: Create An Empty Hugo Website

To use hugo, we’ll first need to install it. To install hugo on ubuntu,

sudo apt install hugo

To install hugo on mac,

brew install hugo

After hugo is installed, let’s create a hugo website by typing in the following command from your home folder.

cd
hugo new site examplesite.gitlab.io --force

Note we need the parameter --force because the examplesite directory already exists.

Next, we need to copy a theme. There are many themes available in here. In this example, we’ll use the ananke theme.

cd examplesite.gitlab.io
git submodule add https://github.com/theNewDynamic/gohugo-theme-ananke themes/ananke
echo "theme = 'ananke'" >> config.toml

After that, we can start our hugo website by typing in

hugo server --bind 0.0.0.0

We bind the ip 0.0.0.0 so that you can access it from outside the server. Then you can access your server from {your-server-ip}:1313. If your hugo server is running on the same machine with your browser, you can use localhost:1313. In my case, I got

Step 3: Create A Post

Next, let’s create a post on the website. To create a post, type in:

hugo new posts/my-first-post.md

Edit the file content/posts/my-first-post.md and update its content like the following:

---
title: "My First Post"
date: 2023-01-17T07:19:39Z
tags: ["post"]
---

This is my first hugo post!

Then go to the page again, and you’ll find the post.

Step 4: Submit Our Git Changes

Now, submit our changes onto gitlab.

git add -A
git commit -m "Example hugo website"
git push origin main:main

Step 5: Host It On Gitlab Pages

Now, it is time to deploy it to gitlab pages and make it public to internet.

Step 5.1: Create A .gitlab-ci.yml

First, let’s create a .gitlab-ci.yml file in your repo root directory to trigger the gitlab CI/CD with the following content.

image: registry.gitlab.com/pages/hugo/hugo_extended:latest

before_script:
 - apk add --update nodejs npm
 - npm install --prefix themes/ananke

variables:
 GIT_SUBMODULE_STRATEGY: recursive

pages:
 script:
 - hugo -e production --minify
 artifacts:
 paths:
 - public
 rules:
 - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Then, submit the file with the following command.

git add -A
git commit -m "Add .gitlab-ci.yml file"
git push origin main:main

Now, if you go to your gitlab project CI/CD -> Pipelines, you should see your commit is triggering a new build and deployment. After your pipeline job is done, you should be able to see your website from https://examplesite.gitlab.io/.

Step 5.2: Make Your Pages Public To Internet

Although you are able to see the website from a public url, others cannot. You’ll have to make it public. To do that, Go to your project -> Settings -> Visibility, project features, permissions -> Pages, select Everyone. Then goto the bottom and save changes.

Now a hugo static website on the Internet is done! For any of the future changes, just edit your hugo website locally, and then push the changes to the gitlab main branch. Then your website will be updated automatically.

Blog: Efficient Mac Window Management

Sat, 14 Jan 2023 14:55:09 -0800

As a software engineer, you might need to open up multiple windows at the same time when doing development work, e.g., a terminal window, a browser, a IDE window, a chat application, etc. You might also have multiple monitors as well. Window management becomes a neccessary part of your work. Unfortunately, mac doesn’t come with a decent solution to this.

In this article, I’ll talk about how you can effectively manage your windows on your mac. I’ll cover two topics specifically:

How to switch between windows?
How to place the target window on the desired position?

Window Switching

Switching between windows is probably one of the most common operations you’ll need when using any kind of operating system.

If you’ve used a Windows Operating System before, you’ll know that it provides a pretty good solution to switch among windows, which is ctrl + tab. Whenever you press ctrl + tab, you’ll see a window poping up in the middle of your screen, presenting you all the current visible windows. By selecting the target window, you can switch to it pretty easily.

This feature works really well on Windows, and Linux also provides the same feature. However, for some reason, mac doesn’t do that by default. Mac has a cmd + tab hotkey, and it pops up a similar window. However, that window presents active applications in your system, and those active applications might include ones without a window. In most cases, you won’t care about them, but it is annoying. I typically only have a few windows opened when I’m working on mac. I’d like to see a clean list, instead of a whole bunch of background running applications, which makes windows switching a lot more inefficient.

Over years, I’ve always tried to find softwares that can provide similar feature as Windows and Linux do. I found two of them work really well.

HyperSwitch

A few years ago, I found this software called HyperSwitch, and it does exactly what I want. It works pretty well, just like Windows and Linux.

HyperSwitch has been pretty reliable before I switched to use the M1 macbook air from my x86 based macbook pro. On M1, I’ve encounter cases where HyperSwitch just failed to find certain windows. I’m not sure why that happened. I also noticed that HyperSwitch is not under active maintenance anymore. The last time it was updated was before 2021. Then I started to search for other solutions.

AltTab

AltTab is another software that provides similar feature. It pretty much does the same thing, but it has richer features. AltTab is also opened sourced at github.

I started to use AltTab ever since 2021. It works pretty good, and I’m still using it now. AltTab is definitely my recommended solution to mac window switching.

Window Placement

Another big topic of window management is window placement. If you’ve used Windows before, you’ll noticed it provides a few pretty handy window management hotkeys:

win + left: Place window on the left part of the screen.
win + right: Place window on the right part of the screen.
win + up: Maximize the window.
win + down: Restore the previous window position if it is currently maximized, otherwise, minimize the window.

I wish mac had come with these features by default, but it had never came. Then I started to search solutions for that.

There are tools to do that, e.g., Magnet, BetterSnapTool, HazeOver, etc. But as a software engineer, I definitely can’t accept these solutions, because they not customizable enough.

And then I found hammerspoon, which is truely a game changer on mac. Hammerspoon is not a window management tool. Instead, it is a powerful automation framework on mac. Hammerspoon connects a lot of system APIs with lua script. In other words, you can use lua script to call system APIs to implement your feature.

This is a great tool for us to define some window management tricks. Based on that, I wrote a script to help me place windows. It is similar to the one on Windows and Linux, but there is a little bit difference.

win + left: Place window on the left part of the screen. If your window is already on the left side of the screen, it will be moved to the monitor to the left (if there is any).
win + right: Place window on the right part of the screen. If your window is already on the right side of the screen, it will be moved to the monitor to the right (if there is any).
win + up: Maximize the window.
win + down: Place the window in the center.

Window placement with hammerspoon is definitely the most preferred solution to me. I attached the script on the bottom. To use it, install hammerspoon (follow this page) first, then copy the following content to ~/.hammerspoon/init.lua, and restart hammerspoon.

------------------------------------------------------------------------------------------
-- The following script comes from https://devmemo.gitlab.io/blog/mac_window_management/
------------------------------------------------------------------------------------------

local devmemo_config = { }


------------------------------------------------------------------------------------------
-- Window Management
------------------------------------------------------------------------------------------

function devmemo_config.focused_window()
 return hs.window.focusedWindow()
end

function devmemo_config.set_frame(unit)
 return devmemo_config.focused_window():setFrame(unit, 0)
end

function devmemo_config.maximize_window()
 local sf = devmemo_config.focused_window():screen():frame()
 devmemo_config.set_frame({ x = sf.x, y = sf.y, w = sf.w, h = sf.h })
end

function devmemo_config.center_window()
 local sf = devmemo_config.focused_window():screen():frame()
 devmemo_config.set_frame({ x = sf.x + sf.w / 4, y = sf.y + sf.h / 4, w = sf.w / 2, h = sf.h / 2 })
end

function devmemo_config.send_window_left()
 local sf = devmemo_config.focused_window():screen():frame()
 local old_wf = devmemo_config.focused_window():frame()
 devmemo_config.set_frame({ x = sf.x, y = sf.y, w = sf.w / 2, h = sf.h })

 if old_wf == devmemo_config.focused_window():frame() then
 local west_screen = devmemo_config.focused_window():screen():toWest()
 if west_screen ~= nil then
 local wsf = west_screen:frame()
 devmemo_config.set_frame({ x = wsf.x + wsf.w / 2, y = wsf.y, w = wsf.w / 2, h = wsf.h })
 end
 end
end

function devmemo_config.send_window_right()
 local sf = devmemo_config.focused_window():screen():frame()
 local old_wf = devmemo_config.focused_window():frame()
 devmemo_config.set_frame({ x = sf.x + sf.w / 2, y = sf.y, w = sf.w / 2, h = sf.h })

 if old_wf == devmemo_config.focused_window():frame() then
 local east_screen = devmemo_config.focused_window():screen():toEast()
 if east_screen ~= nil then
 local esf = east_screen:frame()
 devmemo_config.set_frame({ x = esf.x, y = esf.y, w = esf.w / 2, h = esf.h })
 end
 end
end


------------------------------------------------------------------------------------------
-- Key Bindings and Taps
------------------------------------------------------------------------------------------

hs.hotkey.bind({"cmd"}, "Left", function() devmemo_config.send_window_left() end)
hs.hotkey.bind({"cmd"}, "Right", function() devmemo_config.send_window_right() end)
hs.hotkey.bind({"cmd"}, "Up", function() devmemo_config.maximize_window() end)
hs.hotkey.bind({"cmd"}, "Down", function() devmemo_config.center_window() end)