About James Cherti

James Cherti is an experienced Infrastructure and Software engineer. He has written high-quality Python, Bash, C/C++, PHP, JavaScript source codes, administered Linux servers, and delivered several presentations in universities. He is also an active Open Source contributor.

Preventing Emacs from entering the debugger when a specific error occurs

Emacs provides a built-in feature called debug-on-error, which can be activated with (setq debug-on-error t). The feature determines whether Emacs should enter the debugger when an error occurs. By default, Emacs enters the debugger for all errors, which can be helpful for diagnosing issues. However, there are cases where one may want to ignore specific errors that are not critical to their workflow.

Ignoring specific errors when (setq debug-on-error t) is activated

Open the Emacs configuration file, typically located at ~/.emacs or ~/.emacs.d/init.el. Then use the debug-ignored-errors variable to make Emacs ignore a specific error:

(add-to-list 'debug-ignored-errors 'specific-error-symbol)Code language: plaintext (plaintext)

Replace 'specific-error-symbol with the actual error that the specific function might raise.

Example

Suppose you encounter the search-failed error in Emacs (Evil package) when searching for a non-existent pattern:

Debugger entered--Lisp error: (search-failed "my-pattern")
  signal(search-failed ("my-pattern"))
  evil-ex-start-search(forward nil)
  evil-ex-search-forward(nil)
  funcall-interactively(evil-ex-search-forward nil)
  command-execute(evil-ex-search-forward)Code language: plaintext (plaintext)

To make Emacs ignore the search-failed error above, the following can be used:

(add-to-list 'debug-ignored-errors 'search-failed)Code language: plaintext (plaintext)

Installing Debian from another Linux Distribution such as Gentoo or Arch Linux

There are various scenarios in which one might need to install a Debian-based system (e.g., Debian, Ubuntu, etc.) from another distribution (e.g., Arch Linux, Gentoo, etc.). One common reason is when a user wants to set up a Debian-based system alongside an existing distribution. This could be for the purpose of testing software compatibility, development, or simply to have a dual-boot.

A Debian-based distribution can be installed from any other distribution (e.g. Arch Linux, Gentoo…) using debootstrap. The debootstrap command-line tool allows installing a Debian or Ubuntu base system within a subdirectory of an existing, installed system. Unlike traditional installation methods using a CD or a USB Key, debootstrap only requires access to a Debian repository.

Step 1: Create a new LVM partition, format it, and mount it

# Create the root LVM partition
lvcreate  -L 20G -n debian_root VOL_NAME

# Format the partition
mkfs.ext4 /dev/VOL_NAME/debian_root

# Mount the partition
mkdir /mnt/debian_root
mount /dev/VOL_NAME/debian_root /mnt/debian_rootCode language: plaintext (plaintext)

Step 2: Install the debootstrap command-line tool

On Arch Linux, debootstrap can be installed using:

sudo pacman -Sy debian-archive-keyring debootstrapCode language: plaintext (plaintext)

On Gentoo, it can be installed using:

sudo emerge -a dev-util/debootstrapCode language: plaintext (plaintext)

Step 3: Install the Debian base system

Use the debootstrap command to install Debian into the target directory:

debootstrap  --arch=amd64 stable /mnt/debian_root http://deb.debian.org/debianCode language: plaintext (plaintext)

You can replace stable with another Debian release like testing or unstable if desired. You can also add the flag --force-check-gpg to force checking Release file signatures.

In the above example, it will install the Debian-based system from the repository http://deb.debian.org/debian into the local directory /mnt/debian_root.

Step 4: Chroot into the Debian system

Since you are installing a Debian-based system inside another distribution (Arch Linux, Gentoo, etc.), you’ll need to ensure that the directory where the Debian system is mounted is ready. You can achieve this by mounting certain directories and chrooting into the Debian system:

sudo mount --bind /dev /mnt/debian_root/dev
sudo mount --bind /proc /mnt/debian_root/proc
sudo mount --bind /sys /mnt/debian_root/sys
sudo mount --bind /boot /mnt/debian_root/boot
sudo cp /etc/resolv.conf /mnt/debian_root/etc/resolv.conf
sudo cp /etc/fstab /mnt/debian_root/etc/fstab
sudo chroot /mnt/debian_root /bin/bash -lCode language: plaintext (plaintext)

The chroot command will open a new shell in the Debian environment.

Step 5: Configure the Debian-based system

Now that you’re inside the Debian-based system, you can configure it as desired. You can install packages, modify configurations, set up users, etc.

Here is an example:

apt-get update

# Install the Linux Kernel
apt-get install linux-image-amd64 firmware-linux-free firmware-misc-nonfree 

# Install cryptsetup if you are using a LUKS encrypted partition
apt-get install cryptsetup cryptsetup-initramfs

# Install misc packages
apt-get install console-setup vim lvm2 sudo

# Reconfigure locales
dpkg-reconfigure locales

# Configure the host name and the time zone
echo yourhostname > /etc/hostname
ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
Code language: plaintext (plaintext)

Do not forget to:

  • Modify /mnt/debian_root/etc/fstab (The mount point “/” has to point to the Debian system)
  • Modify /mnt/debian_root/etc/crypttab (If you are using a LUKS encrypted partition)
  • Configure the bootloader (Grub, Syslinux, etc.).

Congratulations! You have successfully installed a Debian-based system using debootstrap from another distribution such as Arch Linux, Gentoo, etc.

Emacs .dir-locals.el – Add project path to $PYTHONPATH (Python Development in Emacs)

In order to ensure that the processes executed by Emacs and its packages, such as Flycheck or Flymake, can access the Python modules of a project, it is essential to correctly configure the $PYTHONPATH environment variable.

This article provides a solution by introducing a .dir-locals.el file that adds the directory path of .dir-locals.el to the $PYTHONPATH environment variable.

The .dir-locals.el file should be placed in the root directory of a Python project.

File name: .dir-locals.el

;; -*- mode: emacs-lisp; -*-
;; File: .dir-locals.el
;; Description:
;; This file adds the path where `.dir-locals.el` is located to the
;; `$PYTHONPATH` environment variable to ensure that processes executed by
;; Emacs and its packages, such as Flycheck or Flymake, can access the Python
;; modules of a project.
;;
;; Author: James Cherti
;; License: MIT
;; URL: https://www.jamescherti.com/emacs-dir-locals-add-path-to-pythonpath/

((python-mode . ((eval . (progn
                           (let ((project_path
                                  (car (dir-locals-find-file
                                        (buffer-file-name))))
                                 (python_path_env (getenv "PYTHONPATH")))
                             (setq-local process-environment
                                         (cons
                                          (concat "PYTHONPATH="
                                                  project_path
                                                  (if python_path_env
                                                      (concat ":" python_path_env)
                                                    ""))
                                          process-environment))))))))
Code language: Lisp (lisp)

A Git Tool that can decide whether to use ‘git mv’ or ‘mv’ to move files and/or directories

The git-smartmv command-line tool, written by James Cherti, allows moving files and/or directories without having to worry about manually choosing whether to use mv or git mv.

  • If the file or directory is being moved within the same Git repository, git-smartmv uses git mv.
  • If the file or directory is being moved between a Git repository and a non-Git directory or a different Git repository, git-smartmv uses mv.

Installation

sudo pip install git-smartmvCode language: plaintext (plaintext)

Shell alias

To simplify the usage of this tool, you can add the following line to your ~/.bashrc:

alias mv="git-smartmv"Code language: plaintext (plaintext)

Usage

The git-smartmv command-line tool accepts similar arguments as the mv command, including the source file or directory to be moved, and the destination file or directory.

Example:

git smartmv file1 file2 directory/

Second example (rename):

git smartmv file1 file2

Links related to git-smartmv

Gentoo: How to Speed Up emerge ‐‐sync

Synchronizing with the Gentoo Portage ebuild repository using emerge --sync can be slow when utilizing the rsync protocol. However, an effective solution exists that can greatly improve the synchronization speed: Configuring emerge --sync to synchronize using Git instead.

In this article, we will explore how to set up emerge to synchronize from the official Gentoo ebuild Git repository and save valuable time during the synchronizing process.

Step 1: Install Git using the following command:

sudo emerge -a dev-vcs/gitCode language: plaintext (plaintext)

Step 2: Remove any file from the directory /etc/portage/repos.conf/ that configures the emerge command to use rsync.

Step 3: Create the file /etc/portage/repos.conf/gentoo.conf containing:

[DEFAULT]
main-repo = gentoo

[gentoo]

# The sync-depth=1 option speeds up initial pull by fetching 
# only the latest Git commit and its immediate ancestors, 
# reducing the amount of downloaded Git history.
sync-depth = 1

sync-type = git
auto-sync = yes
location = /var/db/repos/gentoo
sync-git-verify-commit-signature = yes
sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
sync-uri = https://github.com/gentoo-mirror/gentoo.gitCode language: plaintext (plaintext)

Step 4: Finally, run the following command to synchronize with the Gentoo ebuild repository using Git:

sudo emerge --sync

The initial download of the entire Git repository will cause the first emerge --sync command to take some time. However, subsequent synchronizations will be significantly quicker, taking only a few seconds.

Using Git can be a great way to speed up synchronization with the Gentoo ebuild repository. By following the steps outlined in this article, you can clone the Portage repository to your local machine and keep it up-to-date with the latest changes using Git. This can save you a lot of time when syncing your local repository.

Set up Vim to use Fasd for quickly jumping to directories

Fasd is a command-line tool that enhances productivity by providing quick access to files and directories. It is inspired by autojump, z and v.

By using the code provided below, you can configure Vim to leverage Fasd for swiftly jumping to directories:

" Language: Vim script
" Description: Vim Fasd integration
" Usage: Execute the command: 
"        :Fasd <QUERY>
" License: MIT
" Author: James Cherti
" URL: https://www.jamescherti.com/vim-fasd-integration

function! Fasd(query)
  " -d: Match directories only.
  " -l: List paths without scores.
  let l:cmd = 'fasd -d -l ' . shellescape(a:query)
  let l:result = systemlist(l:cmd)
  if v:shell_error !=# 0
    echoerr 'Fasd error: ' . join(l:result, "\n")
    return
  endif

  if len(l:result) ==# 0 || !isdirectory(l:result[0])
    echo 'Fasd has not found anything for: ' . a:query
    return
  endif

  let l:path = l:result[0]
  exec 'edit ' . fnameescape(l:path)
  exec 'lcd ' . fnameescape(l:path)
  echo l:path
endfunction

command! -bar -nargs=1 Fasd call Fasd(<f-args>)
Code language: Vim Script (vim)

Vim to Emacs: Converting code snippets from Ultisnips to YASnippet format

Ultyas is a command-line tool, written by James Cherti, designed to simplify the process of converting code snippets from Ultisnips (Vim) to YASnippet (Emacs) format. With Ultyas, you can effortlessly migrate your code snippets to the YASnippet format, saving you valuable time and effort.

Installation

Ultyas can be installed locally to ~/.local/bin/ultyas using pip:

pip install --user ultyasCode language: plaintext (plaintext)

Usage

You can convert Ultisnips snippets to the Yasnippet format by using the following command:

~/.local/bin/ultyas ~/.vim/UltiSnips/python.snippets -o ~/.emacs.d/snippets/python-mode/Code language: plaintext (plaintext)

Available command line options

Here are the available command line options for Ultyas:

usage: ultyas <file.snippets> -o <yasnippet-major-mode-dir>

A command-line tool for converting code snippets from Ultisnips to YASnippet format.

positional arguments:
  ultisnips_file        The Ultisnips .snippets file (e.g.
                        '~/.vim/UltiSnips/python.snippets')

options:
  -h, --help            show this help message and exit
  -o YASNIPPET_DIR, --yasnippet-dir YASNIPPET_DIR
                        The YASnippet snippets major mode directory (e.g.
                        '~/.emacs.d/snippets/python-mode/')
  -i {auto,fixed,nothing}, --yas-indent-line {auto,fixed,nothing}
                        Add one of the following comments to the YASnippet snippets that
                        will be generated: "# expand-env: ((yas-indent-line 'fixed))" or "#
                        expand-env: ((yas-indent-line 'auto))". For more information on
                        'yas-indent-line', visit:
                        https://joaotavora.github.io/yasnippet/snippet-reference.html#yas-
                        indent-line
  -t CONVERT_TABS_TO, --convert-tabs-to CONVERT_TABS_TO
                        Convert the tabs that are in the generated snippets to the string
                        passed to this option (Default: The indentation marker '$>')
  -v, --verbose         Verbose mode
  -q, --quiet           Quiet modeCode language: plaintext (plaintext)

Links related to Ultyas

A Docker container for Oddmuse, a Wiki engine that does not require a database

Oddmuse is a wiki engine. Unlike other wiki engines that rely on local or remote databases to store and modify content, Oddmuse utilizes the local file system. This means that users can create and manage Wiki pages on their local machine and easily transfer them to other locations or servers. By leveraging the local file system, Oddmuse eliminates the need for complex and costly database setups. Oddmuse is used by many websites around the world, including the website emacswiki.org.

To make it even easier to use Oddmuse, I have created a Docker container that includes the wiki engine and all of its dependencies. This container can be downloaded and run on any machine that has Docker installed.

Pull and run the Oddmuse Docker container

The Oddmuse Docker container can be pulled from the Docker hub repository jamescherti/oddmuse using the following command:

docker pull jamescherti/oddmuseCode language: Bash (bash)

And here is an example of how to run the Docker container:

docker run --rm \
  -v /local/path/oddmuse_data:/data \
  -p 8080:80 \
  --env ODDMUSE_URL_PATH=/wiki \
  jamescherti/oddmuseCode language: Bash (bash)

Once the container up and running, you can start using Oddmuse to create and manage your own wiki pages.

Alternative method: Compile the Oddmuse Docker container

Alternatively, you can build the Oddmuse Docker container using the Dockerfile that is hosted in the GitHub repository jamescherti/docker-oddmuse:

git clone https://github.com/jamescherti/docker-oddmuse
docker build -t jamescherti/oddmuse docker-oddmuseCode language: Bash (bash)

The Oddmuse Docker container is a convenient and efficient way to use the Oddmuse Wiki engine without the need for complex setups.

Arch Linux: Preserving the kernel modules of the currently running kernel during and after an upgrade

One potential issue when upgrading the Arch Linux kernel is that the modules of the currently running kernel may be deleted. This can lead to a number of problems, including unexpected behavior, system crashes, or the inability to mount certain file systems (e.g. the kernel fails to mount a vfat file system due to the unavailability of the vfat kernel module).

The Arch Linux package linux-keep-modules (also available on AUR: linux-keep-modules @AUR), written by James Cherti, provides a solution to ensure that the modules of the currently running Linux kernel remain available until the operating system is restarted. Additionally, after a system restart, the script automatically removes any unnecessary kernel modules that might have been left behind by previous upgrades (e.g. the kernel modules that are not owned by any Arch Linux package and are not required by the currently running kernel).

The linux-keep-modules package keeps your system running smoothly and maintains stability even during major Linux kernel upgrades.

Make and install the linux-keep-modules package

Clone the repository and change the current directory to ‘archlinux-linux-keep-modules/’:

$ git clone https://github.com/jamescherti/archlinux-linux-keep-modules.git
$ cd archlinux-linux-keep-modules/Code language: plaintext (plaintext)

Use makepkg to make linux-keep-modules package:

$ makepkg -fCode language: plaintext (plaintext)

Install the linux-keep-modules package:

$ sudo pacman -U linux-keep-modules-*-any.pkg.tar.*Code language: plaintext (plaintext)

Finally, enable the cleanup-linux-modules service:

$ sudo systemctl enable cleanup-linux-modulesCode language: plaintext (plaintext)

(The cleanup-linux-modules service will delete the Linux kernel modules that are not owned by any a package at boot time)

The linux-keep-modules Arch Linux package offers a solution to preserve kernel modules during and after upgrades, ensuring that the necessary modules for the currently running kernel remain present in the system even after the kernel is upgraded. This solution keeps your system running smoothly and maintains stability even during major upgrades.

Links related to the pacman package linux-keep-modules

Helper script to upgrade Arch Linux

In this article, we will be sharing a Python script, written by James Cherti, that can be used to upgrade Arch Linux. It is designed to make the process of upgrading the Arch Linux system as easy and efficient as possible.

The helper script to upgrade Arch Linux can:

  • Delete the ‘/var/lib/pacman/db.lck’ when pacman is not running,
  • upgrade archlinux-keyring,
  • upgrade specific packages,
  • download packages,
  • upgrade all packages,
  • remove from the cache the pacman packages that are no longer installed.

The script provides a variety of options and is perfect for those who want to automate the process of upgrading their Arch Linux system (e.g. execute it from cron) and ensure that their system is always up to date.

Requirements: psutil
Python script name: archlinux-update.py

#!/usr/bin/env python
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/script-update-arch-linux/
"""Helper script to upgrade Arch Linux."""

import argparse
import logging
import os
import re
import subprocess
import sys
import time

import psutil


class ArchUpgrade:
    """Upgrade Arch Linux."""

    def __init__(self, no_refresh: bool):
        self._download_package_db = no_refresh
        self._keyring_and_pacman_upgraded = False
        self._delete_pacman_db_lck()

    @staticmethod
    def _delete_pacman_db_lck():
        """Delete '/var/lib/pacman/db.lck' when pacman is not running."""
        pacman_running = False
        for pid in psutil.pids():
            try:
                process = psutil.Process(pid)
                if process.name() == "pacman":
                    pacman_running = True
                    break
            except psutil.Error:
                pass

        if pacman_running:
            print("Error: pacman is already running.", file=sys.stderr)
            sys.exit(1)

        lockfile = "/var/lib/pacman/db.lck"
        if os.path.isfile(lockfile):
            os.unlink(lockfile)

    def upgrade_specific_packages(self, package_list: list) -> list:
        """Upgrade the packages that are in 'package_list'."""
        outdated_packages = self._outdated_packages(package_list)
        if outdated_packages:
            cmd = ["pacman", "--noconfirm", "-S"] + outdated_packages
            self.run(cmd)

        return outdated_packages

    def _outdated_packages(self, package_list: list) -> list:
        """Return the 'package_list' packages that are outdated."""
        outdated_packages = []
        try:
            output = subprocess.check_output(["pacman", "-Qu"])
        except subprocess.CalledProcessError:
            output = b""

        for line in output.splitlines():
            line = line.strip()
            pkg_match = re.match(r"^([^\s]*)\s", line.decode())
            if not pkg_match:
                continue

            pkg_name = pkg_match.group(1)
            if pkg_name in package_list:
                outdated_packages += [pkg_name]

        return outdated_packages

    @staticmethod
    def upgrade_all_packages():
        """Upgrade all packages."""
        ArchUpgrade.run(["pacman", "--noconfirm", "-Su"])

    def download_all_packages(self):
        """Download all packages."""
        self.download_package_db()
        self.run(["pacman", "--noconfirm", "-Suw"])

    def download_package_db(self):
        """Download the package database."""
        if self._download_package_db:
            return

        print("[INFO] Download the package database...")
        ArchUpgrade.run(["pacman", "--noconfirm", "-Sy"])
        self._download_package_db = True

    def upgrade_keyring_and_pacman(self):
        self.download_package_db()

        if not self._keyring_and_pacman_upgraded:
            self.upgrade_specific_packages(["archlinux-keyring"])
            self._keyring_and_pacman_upgraded = True

    def clean_package_cache(self):
        """Remove packages that are no longer installed from the cache."""
        self.run(["pacman", "--noconfirm", "-Scc"])

    @staticmethod
    def run(cmd, *args, print_command=True, **kwargs):
        """Execute the command 'cmd'."""
        if print_command:
            print()
            print("[RUN] " + subprocess.list2cmdline(cmd))

        subprocess.check_call(
            cmd,
            *args,
            **kwargs,
        )

    def wait_download_package_db(self):
        """Wait until the package database is downloaded."""
        successful = False
        minutes = 60
        hours = 60 * 60
        seconds_between_tests = 15 * minutes
        for _ in range(int((10 * hours) / seconds_between_tests)):
            try:
                self.download_package_db()
            except subprocess.CalledProcessError:
                minutes = int(seconds_between_tests / 60)
                print(
                    f"[INFO] Waiting {minutes} minutes before downloading "
                    "the package database...",
                    file=sys.stderr,
                )
                time.sleep(seconds_between_tests)
                continue
            else:
                successful = True
                break

        if not successful:
            print("Error: failed to download the package database...",
                  file=sys.stderr)
            sys.exit(1)


def parse_args():
    """Parse the command-line arguments."""
    usage = "%(prog)s [--option] [args]"
    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0],
                                     usage=usage)
    parser.add_argument("packages",
                        metavar="N",
                        nargs="*",
                        help="Upgrade specific packages.")

    parser.add_argument(
        "-u",
        "--upgrade-packages",
        default=False,
        action="store_true",
        required=False,
        help="Upgrade all packages.",
    )

    parser.add_argument(
        "-d",
        "--download-packages",
        default=False,
        action="store_true",
        required=False,
        help="Download the packages that need to be upgraded.",
    )

    parser.add_argument(
        "-c",
        "--clean",
        default=False,
        action="store_true",
        required=False,
        help=("Remove packages that are no longer installed from "
              "the cache."),
    )

    parser.add_argument(
        "-n",
        "--no-refresh",
        default=False,
        action="store_true",
        required=False,
        help=("Do not download the package database (pacman -Sy)."),
    )

    parser.add_argument(
        "-w",
        "--wait-refresh",
        default=False,
        action="store_true",
        required=False,
        help=("Wait for a successful download of the package database "
              "(pacman -Sy)."),
    )

    return parser.parse_args()


def command_line_interface():
    """The command-line interface."""
    logging.basicConfig(level=logging.INFO, stream=sys.stdout,
                        format="%(asctime)s %(name)s: %(message)s")

    if os.getuid() != 0:
        print("Error: you cannot perform this operation unless you are root.",
              file=sys.stderr)
        sys.exit(1)

    nothing_to_do = True
    args = parse_args()
    upgrade = ArchUpgrade(no_refresh=args.no_refresh)

    if args.wait_refresh:
        upgrade.wait_download_package_db()
        nothing_to_do = False

    if args.packages:
        print("[INFO] Upgrade the packages:", ", ".join(args.packages))
        upgrade.upgrade_keyring_and_pacman()
        if not upgrade.upgrade_specific_packages(args.packages):
            print()
            print("[INFO] The following packages are already up-to-date:",
                  ", ".join(args.packages))
        nothing_to_do = False

    if args.download_packages:
        print("[INFO] Download all packages...")
        upgrade.download_all_packages()
        nothing_to_do = False

    if args.upgrade_packages:
        print("[INFO] Upgrade all packages...")
        upgrade.upgrade_keyring_and_pacman()
        upgrade.upgrade_all_packages()

        nothing_to_do = False

    if args.clean:
        print("[INFO] Remove packages that are no longer installed "
              "from the cache...")
        upgrade.clean_package_cache()
        nothing_to_do = False

    if nothing_to_do:
        print("Nothing to do.")
        print()

    sys.exit(0)


def main():
    try:
        command_line_interface()
    except subprocess.CalledProcessError as err:
        print(f"[ERROR] Error {err.returncode} returned by the command: "
              f"{subprocess.list2cmdline(err.cmd)}",
              file=sys.stderr)
        sys.exit(1)


if __name__ == '__main__':
    main()Code language: Python (python)

Python: Extract variables/values from source code comments

The source code in this article can be used to extract variables and values from source code comments. The code is written in Python and uses a combination of regular expressions and Python’s built-in string functions to extract specific information from source code comments.

#!/usr/bin/env python
# Author: James Cherti
# License: MIT
# URL: https://www.jamescherti.com/python-extract-variables-values-from-source-code-comments/
"""Extract variables/values from source code comments."""

import re
from typing import Dict


def get_variables_from_comments(
        source_code_content: str,
        comment_pattern: str = r'[^\w\s]+') -> Dict[str, list]:
    """Extract variables/values from source code comments.

    Source code example:
        #!/usr/bin/env python
        # This is a simple comment.
        print("Hello world")
        #
        # myvar: value 1
        # myvar: value 2
        # myvar: value 3
        # AnotherVar: value 1

    Here is how to extract the variables and their values from the
    source code above:
    >>> get_variables_from_comments(source_code_content)
    {'AnotherVar': ['value 1'], 'myvar': ['value 1', 'value 2', 'value 3']}

    """
    source_code_lines = source_code_content.splitlines()

    result: Dict[str, list] = {}
    for line in source_code_lines:
        re_str = (r'^\s*' +
                  comment_pattern +
                  r'\s*([\w\d]+)\s*:\s*(.*)\s*$')
        match_result = re.search(re_str, line)
        if match_result:
            var_name = match_result.group(1)
            var_value = match_result.group(2)

            if var_name not in result:
                result[var_name] = []

            result[var_name].append(var_value)

    return result


def main():
    """Try the method 'get_variables_from_comments()'."""

    source_code = (
        "#!/usr/bin/env python\n"
        "# This is a simple comment.\n"
        "print(\"Hello world\")\n"
        "#\n"
        "# myvar: value 1\n"
        "# myvar: value 2\n"
        "#\n"
        "# myvar: value 3\n"
        "# AnotherVar: value 1\n"
    )

    __import__('pprint').pprint(get_variables_from_comments(
        source_code_content=source_code,
        comment_pattern=re.escape('#'))
     )


if __name__ == '__main__':
    main()Code language: Python (python)

Vim: Quickly replace the word that is under the cursor

" Language: Vim script
" Description: Quickly replace the word that is under the cursor.
" Usage: Press the keys '<Leader>rr', type the new word, and 
"        then leave insert mode (<Esc>).
" License: MIT
" Author: James Cherti
" URL: https://www.jamescherti.com/vim-quickly-replace-word-under-cursor/

function! s:replace_word_under_cursor(...) abort
  let l:action = 'main'
  if len(a:000) > 0
    let l:action = a:000[0]
  endif
  
  "
  " Main action
  "
  if l:action ==# 'main'
    let b:replace_word_data = {}
    let b:replace_word_data['changenr'] = changenr()

    let l:regex_prefix = ''
    let l:regex_suffix = ''
    if mode() ==# 'n'
      normal! diw
      let l:regex_prefix = '\<'
      let l:regex_suffix = '\>'
    else
      echoerr 'Unsupported mode: ' . mode()
      return
    endif

    let b:replace_word_data['string'] = substitute(@@, '\v\n+$', '', 'g')
    let b:replace_word_data['escaped_string'] =
        \ '\V' . l:regex_prefix . 
        \ escape(b:replace_word_data['string'], '/\') . 
        \ l:regex_suffix
    startinsert

    augroup ReplaceString
      autocmd!
      autocmd InsertLeave * call s:replace_word_under_cursor('insert_leave')
    augroup END

    return
  endif

  
  "
  " Insert Leave
  "
  if l:action ==# 'insert_leave'
    autocmd! ReplaceString InsertLeave
    
    let l:new_string = expand('<cword>')
    if empty(l:new_string) 
      execute 'undo ' . b:replace_word_data['changenr']
      return
    endif
    
    let l:cursor_pos = getpos('.')
    execute 'keeppatterns silent! %substitute/' . 
        \ b:replace_word_data['escaped_string'] . '/' . 
        \ escape(l:new_string, '/\') . '/g'
    call setpos('.', l:cursor_pos)
    
    echo 'Replace: ' . b:replace_word_data['string'] . ' -> ' . l:new_string
    
    unlet b:replace_word_data
    return
  endif
endfunction

nnoremap <Leader>rr :call <SID>replace_word_under_cursor()<CR>Code language: Vim Script (vim)

Python: Tab completion against a list of strings (readline)

#!/usr/bin/env python
# License: MIT
# Author: James Cherti
# URL: https://www.jamescherti.com/python-tab-completion-readline-against-list/
"""Tab completion against a list of strings (readline)"""

import readline
from typing import Any, List, Union


class ReadlineCompleter:
    """A readline completer."""

    def __init__(self, options: List[str]):
        """Store the options = ['word1', 'word2']."""
        readline.set_completer_delims('')
        self.options = options
        self.matches: List[str] = []

    def complete(self, _, state):
        """Complete a readline sentence."""
        if state == 0:
            origline = readline.get_line_buffer()
            begin = readline.get_begidx()
            end = readline.get_endidx()
            being_completed = origline[begin:end]
            words = origline.split()

            if not words:
                self.matches = self.options[:]
            else:
                try:
                    if begin == 0:
                        matches = self.options[:]  # First word
                    else:
                        first = words[0]  # Later word
                        matches = self.options[first]

                    if being_completed:
                        # Match options with portion of input
                        # being completed
                        self.matches = [w for w in matches
                                        if w.startswith(being_completed)]
                    else:
                        # Matching empty string so use all candidates
                        self.matches = matches
                except (KeyError, IndexError):
                    self.matches = []

        try:
            return self.matches[state]
        except IndexError:
            return None


def input_completion(prompt: Any,
                     list_options: Union[None, List[str]] = None):
    """Read a string from standard input and complete against 'list_options'.

    The trailing newline is stripped. The prompt string is printed to
    standard output without a trailing newline before reading input.

    If the user hits EOF (*nix: Ctrl-D, Windows: Ctrl-Z+Return), raise
    EOFError. On *nix systems, readline is used if available.

    """
    readline.parse_and_bind('tab: complete')
    if list_options is None:
        list_options = []

    save_completer = readline.get_completer()
    try:
        readline.set_completer(
            ReadlineCompleter(list_options).complete
        )
        return input(prompt)
    finally:
        readline.set_completer(save_completer)


def main():
    """Try input_completion()."""
    list_options = ["yes", "no", "cancel"]
    value = input_completion("Proceed (press the Tab key)? ",
                             list_options=list_options)
    print("Value:", value)


if __name__ == "__main__":
    main()Code language: Python (python)

A tool to Execute a Command in a new Tmux Window

The Python script tmux-run.py allows executing a command in a new tmux window. A tmux window is similar to a tab in other software.

If the script is executed from within a tmux session, it creates a tmux window in the same tmux session. However, if the script is executed from outside of a tmux session, it creates a new tmux window in the first available tmux session.

(Requirement: libtmux)

The Python script: tmux-run.py

#!/usr/bin/env python
# License: MIT
# Author: James Cherti
# URL: https://www.jamescherti.com/python-script-run-command-new-tmux-window/
"""Execute a command in a new tmux window.

This script allows executing a command in a new tmux window (a tmux window is
similar to a tab in other software).

- If it is executed from within a tmux session, it creates a tmux window
in the same tmux session.
- However, if the script is executed from outside of a tmux
session, it creates a new tmux window in the first available tmux session.

"""

import os
import shlex
import shutil
import sys

import libtmux


SCRIPT_NAME = os.path.basename(sys.argv[0])


def parse_args():
    if len(sys.argv) < 2:
        print(f"Usage: {SCRIPT_NAME} <command> [args...]",
              file=sys.stderr)
        sys.exit(1)

    args = sys.argv[1:]
    args[0] = shutil.which(args[0])
    if args[0] is None:
        print(f"{SCRIPT_NAME}: no {args[0]} in "
              f"({os.environ.get('PATH', '')})", file=sys.stderr)
        sys.exit(1)

    return args


def get_tmux_session():
    tmux_server = libtmux.Server()
    if not tmux_server.sessions:
        print(f"{SCRIPT_NAME}: the tmux session was not found",
              file=sys.stderr)
        sys.exit(1)

    tmux_session_id = os.environ["TMUX"].split(",")[-1]
    if tmux_session_id:
        try:
            return tmux_server.sessions.get(id=f"${tmux_session_id}")
        except Exception:  # pylint: disable=broad-except
            pass

    return tmux_server.sessions[0]


def run_in_tmux_window():
    try:
        command_args = parse_args()
        tmux_session = get_tmux_session()
        command_str = shlex.join(command_args)
        tmux_session.new_window(attach=True, window_shell=command_str)
    except libtmux.exc.LibTmuxException as err:
        print(f"Error: {err}.", file=sys.stderr)
        sys.exit(1)


if __name__ == '__main__':
    run_in_tmux_window()


Code language: Python (python)

Gentoo Linux: Unlocking a LUKS Encrypted LVM Root Partition at Boot Time using a Key File stored on an External USB Drive

Gentoo can be configured to use a key file stored on an external USB drive to unlock a LUKS encrypted LVM root partition.

We will explore in this article the general steps involved in configuring Gentoo to use an external USB drive as a key file to unlock a LUKS encrypted LVM root partition.

1. Create a key file on the USB stick and add it to the LUKS encrypted partition

Generate a key file on a mounted ext4 or vfat partition of a USB stick, which will be used by initramfs to unlock the LUKS partition:

dd if=/dev/urandom of=/PATH/TO/USBSTICK/keyfile bs=1024 count=4Code language: plaintext (plaintext)

Ensure that the partition on the USB drive has a label, as the initramfs will use this label to find where the key file is located.

Afterward, add the key file to the LUKS partition to enable decryption of the partition using that key file:

cryptsetup luksAddKey /dev/PART1 /PATH/TO/USBSTICK/keyfile

In this example, “/dev/PART1” is the partition where the LUKS encryption is enabled, and “/PATH/TO/USBSTICK/keyfile” is the location of the keyfile.

2 – Find the UUID of the encrypted partition and the label of the USB drive

Use the lsblk command to find the UUID of the encrypted partition and the label of the USB drive:

lsblk -o +UUID,LABEL

3. Configure the boot loader (such as Systemd-boot, GRUB, Syslinux…)

Add to the boot loader configuration the following initramfs kernel parameters:

  • crypt_root=UUID=A1111111-A1AA-11A1-AAAA-111AA11A1111
  • root=/dev/LVMVOLUME/root
  • root_keydev=/dev/disk/by-label/LABELNAME
  • root_key=keyfile

Here is an example for Systemd-boot:

options dolvm crypt_root=UUID=A1111111-A1AA-11A1-AAAA-111AA11A1111 root=/dev/LVMVOLUME/root root_keydev=/dev/disk/by-label/LABELNAME root_key=keyfileCode language: plaintext (plaintext)

To ensure proper setup:

  • Customize the initramfs options for LVMVOLUME, LABELNAME, and UUID=A1111111-A1AA-11A1-AAAA-111AA11A1111 to match your specific case.
  • Verify that the ext4 or vfat partition of the USB drive that is labeled “LABELNAME” contains a file named “keyfile”.
  • Make sure that the modules “dm_mod” and “usb_storage” are included in the initramfs.

This method offers a convenient way to unlock a LUKS encrypted root LVM partition. The implementation process is well-documented, making it a suitable choice for those looking to secure their Gentoo Linux systems.