Welcome to MoonSet One Docs

Objectives: Docs with tips or tricks within Python, Ruby, Linux, Data Ops, Data Engineering, some Javascript and Frontends.

This is only scripts with minimal explanation. The full explanation is going on my blog.

TILs: "Today I learned" are simple tricks that you annotated to remind yourself on a later date, or share with fellow devs.

ArchLinux | Manjaro Based Setup

Tips and tricks for a ideal initial setup

Ranking mirrors for faster downloads

Manjaro

 sudo pacman-mirrors -c Brazil,United_States

ArchLinux

curl -s "https://www.archlinux.org/mirrorlist/?country=BR&use_mirror_status=on" | sed -e 's/^#Server/Server/' -e '/^#/d' | rankmirrors -n 5 -

Update system

sudo pacman -Suy sudo pacman -S yay # only works on Manjaro

Interesting terminal companions and programming languages

yay -S micro-manjaro
yay -S --needed thefuck nodenv nodenv-node-build-git rbenv ruby-build elixir fish tmux autoconf automake bison bind-tools fasd htop make patch ed fzf gcc mosh ruby tk yarn php php-fpm python-pip python-pillow python-numpy nfs-utils lsof strace tldr mosh
tldr --update

Interesting graphical interface apps and tools

yay -S --needed adobe-source-sans-pro-fonts chromium deluge firefox-developer-edition otf-fira-code otf-fira-sans p7zip ttf-roboto ttf-ubuntu-font-family
yay -S --needed albert-lite atom-editor-bin visual-studio-code-bin ttf-iosevka pinta simple-scan xsel python-pyusb

Interesting python packages

Interesting utilities

yay -S --needed telegram-desktop foliate flameshot neofetch

No password for sudo

sudo visudo -f /etc/sudoers.d/99-mine

Be careful with %wheel, I prefer to put just myself :D

%wheel ALL=(ALL) NOPASSWD: ALL
rodrigo ALL=(ALL) NOPASSWD: ALL

Using my .dotfiles with TMUX and Fish Shell

```sh
cd
git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm
git clone https://gitlab.com/rdlu/dotfiles.git .dotfiles
ln -s ~/.dotfiles/tmux.conf .tmux.conf
tmux source ~/.tmux.conf
ln -s ~/.dotfiles/fish/config.fish ~/.config/fish/config.fish
ln -s ~/.dotfiles/fish/omf ~/.config/omf
curl -L https://get.oh-my.fish | fish
sudo chsh (whoami) -s /usr/bin/fish
```

After that restart your fish prompt and:

omf update

CTRL+A; SHIFT+U for tmux plugins

Fixes and Workarounds for ArchLinux

I can use in emergencies or having bugs.

Using dnsmasq like in Ubuntu for DNS resolutions

It's useful when using Docker containers + Reverse Proxies. You can just add your local address for wildcard resolution later.

sudo pacman -S dhclient dnsmasq

Then configure NetworkManager:

micro /etc/NetworkManager/NetworkManager.conf

To this:

[main]
plugins=keyfile
dhcp=dhclient
dns=dnsmasq

Rereading /etc/hosts and restarting:

echo 'addn-hosts=/etc/hosts' | sudo tee /etc/NetworkManager/dnsmasq.d/hosts.conf > /dev/null
sudo systemctl restart NetworkManager

Adding wildcard subdomains to dnsmasq

Create/edit a NetworkManager configuration for dnsmasq hosts:

sudo micro /etc/NetworkManager/dnsmasq.d/hosts.conf

Add, changing the domain to your preference:

address=/domain.tld/127.0.0.3
address=/sub.domain.tld/127.0.0.4

Restart NetworkManager

sudo systemctl restart NetworkManager

Comparing new configuration files after updates

SUDO_EDITOR=meld sudo -e /etc/file{,.pacnew}

Bluetooth AD2P not working with Gnome

yay -S pulseaudio-bluetooth-a2dp-gdm-fix

Firefox Text fields: Using dark themes with Gnome

Create a new property using about:config

Name: widget.content.gtk-theme-override Value: Adwaita:light

Adwaita must be present, KDE users can rely on breeze.

Firefox versus HiDPI and Wayland

Firefox Developer Edition

cp /usr/share/applications/firefox-developer-edition.desktop ~/.local/share/applications/
sed -i 's/Exec=/Exec=env MOZ_ENABLE_WAYLAND=1 /g' ~/.local/share/applications/firefox-developer-edition.desktop

Regular Firefox

cp /usr/share/applications/firefox.desktop ~/.local/share/applications/
sed -i 's/Exec=/Exec=env MOZ_ENABLE_WAYLAND=1 /g' ~/.local/share/applications/firefox.desktop

Undoing

rm ~/.local/share/applications/firefox*

Slow Playback / Hardware Acceleration

If your video playback is slow, change in about:config this setting: layers.acceleration.force-enabled to true

It works just fine on my three main computers (AMD A8 with open source radeon/mesa; Intel i5 with iGPU and mesa; Intel i7 with nVidia GTX 1050Ti and proprietary nvidia)

GPG, SSH Auth with Keybase

Installing requirements:

sudo pacman -S kbfs keybase keybase-gui micro

If you are having problems with gpg-agent as ssh agent mode, restart gnupg from scratch (export your important keys first)

rm -rf ~/.gnupg

Login to Keybase using the GUI or keybase login. Now import private key and public keys from keybase:

keybase pgp export --secret | gpg --allow-secret-key-import --import
keybase pgp export | gpg --import

Setup GPG Agent:

micro ~/.gnupg/gpg-agent.conf

With the below contents:

enable-ssh-support

default-cache-ttl-ssh 10800
max-cache-ttl-ssh 10800

pinentry-program /usr/bin/pinentry-gnome3

Setup PAM environment as extra:

micro ~/.pam-environment

With the below contents:

SSH_AGENT_PID	DEFAULT=
SSH_AUTH_SOCK	DEFAULT="${XDG_RUNTIME_DIR}/gnupg/S.gpg-agent.ssh"

Add your keys to SSH control file, that keeps the "authorized" keys to use by the emulated SSH agent:

ls ~/.gnupg/private-keys-v1.d/ | sed s/.key// >> ~/.gnupg/sshcontrol

Now logout and login again, or event better, reboot your machine. Check your SSH keys with:

ssh-add -l

If you are getting errors like error fetching identities: agent refused operation then you need to remove offending keys that are not Authorization keys. You can check them with:

gpg --list-keys --with-keygrip

Then comment out the keys with # or remove auth with ! at ~/.gnupg/sshcontrol:

micro ~/.gnupg/sshcontrol

Virtualization in Archlinux - Docker, Kubernetes, KVM

Tips and tricks for a ideal initial setup

Docker Setup

yay -S docker nvidia-container-toolkit
sudo usermod -a -G docker $(whoami)
sudo systemctl start docker
docker info

Kubernetes using K3s (K8s on K3s)

First you need to setup docker like above.

yay -S helm kubectl k3s-bin
set -gx KUBECONFIG /etc/rancher/k3s/k3s.yaml  #for fish shell
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml   #for bash shell
sudo systemctl start k3s
sudo chown -R root.docker /etc/rancher/k3s
kubectl get pods --all-namespaces
helm ls --all-namespaces

Kubernetes Dashboard using fish shell

set GITHUB_URL https://github.com/kubernetes/dashboard/releases
set VERSION_KUBE_DASHBOARD (curl -w '%{url_effective}' -I -L -s -S ${GITHUB_URL}/latest -o /dev/null | sed -e 's|.*/||')
k3s kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/${VERSION_KUBE_DASHBOARD}/aio/deploy/recommended.yaml

Now create a RBAC authentication for a single user:

Using mkdir -pv ~/Projects/k8s; micro ~/Projects/k8s/dashboard.admin-user.yml, paste this:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

Using micro ~/Projects/k8s/dashboard.admin-user-role.yml, paste this:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

Now apply these new settings:

cd ~/Projects/k8s
k3s kubectl create -f dashboard.admin-user.yml -f dashboard.admin-user-role.yml

Accessing the dashboard

Obtain the auth token (Bearer token), copying to clipboard:

k3s kubectl -n kubernetes-dashboard describe secret admin-user-token | grep '^token' | sed -r 's/token:[[:blank:]]+//g' | xclip -sel clip

Create the proxy

k3s kubectl proxy

Now access: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/

More info: https://rancher.com/docs/k3s/latest/en/installation/kube-dashboard/

Deleting the dashboard

k3s kubectl delete ns kubernetes-dashboard

KVM Setup on Arch

Basic installation

sudo pacman -S --needed qemu virt-manager virt-viewer dnsmasq vde2 bridge-utils openbsd-netcat ebtables iptables
sudo yay -S --needed libguestfs
sudo systemctl enable --now libvirtd.service

Enable normal user account to use KVM

sudo micro /etc/libvirt/libvirtd.conf

Uncomment these lines:

unix_sock_group = "libvirt"
unix_sock_rw_perms = "0770"

Back to shell:

sudo usermod -a -G libvirt $(whoami)
newgrp libvirt
sudo systemctl restart libvirtd.service

In my test we need more configuration to not get a laggy 2D experience, even on Manjaro KDE.

Julia

nteract notebooks plus IJulia Packages managed by pacman

It's a wonderful idea to delegate the package updates to the operating system.

ArchLinux AUR has many Julia packages already, including julia-ijulia for notebooks

yay -S julia-distrohelper julia-loadpath

Reboot or logout/login from your user, because these 2 packages manages the julia's loadpath using system packs as a fallback. Without that Julia envs will not recognize the system installed packages.

yay -S --needed julia-ijulia nteract-bin

Now you can install any packages starting with julia-* and notebook apps like nteract-bin will recognize Julia as a option to notebook environments.

Vivaldi Browser

Installation

yay -S --needed vivaldi

Proprietary codecs

You can install from AUR also, but they are slow to download and build (downloading an entire Google Chrome deb, etc)

yay -S --needed vivaldi-ffmpeg-codecs vivaldi-widevine vivaldi-codecs-ffmpeg-extra-bin

Or, below you can use the prebuilt binaries from Vivaldi to install in the user space.

Vivaldi Browser Proprietary Media on user space (no root)

The update-ffmpeg and update-widevine scripts included in the Vivaldi install directory are provided to fix situations where proprietary media (AVC/H.264 and AAC) and Widevine (DRM/EME) respectively, are not setup correctly.

These scripts are primarily intended to be run as root (or under sudo) as they create and update files and directories that are root owned. However both support a command line option (--user) that adjusts their installation directories and thus allows them to be run without escalation.

The --user options were made for internal usage, with locally ‘unpacked’ copies of Vivaldi (i.e. not installed). However, it is possible to use them with standard installs (albeit with a little tweaking in the case of update-widevine).

update-ffmpeg

To install proprietary media for just your current user issue the following:

/opt/vivaldi/update-ffmpeg --user

update-widevine

To install Widevine for just your current user, a couple more steps are required. This is because the script still attempts to adjust symlinks within the (typically root owned) Vivaldi install directory.

You can adjust the script as it runs, to disable the symlink creation, which would otherwise cause failure:

sed -r 's/^ *(rm|ln) -f.*/:/' /opt/vivaldi/update-widevine | sh -eus -- --user

Now create meta data in your Vivaldi user data directory that points to alternative Widevine install location–triple click to select the entire line:

echo "{\"Path\":\"$HOME/.local/lib/vivaldi/WidevineCdm\"}" > "${XDG_CONFIG_HOME:-$HOME/.config}/vivaldi/WidevineCdm/latest-component-updated-widevine-cdm"

Note: Change all “vivaldi” references in the above commands to “vivaldi-snapshot”, if you use the snapshot version.

(Original)[https://gist.github.com/ruario/995728dd8123540d331052e0ce648439]

Elixir language

List comprehensions

Filtering

animals = [ {"Liz", :dog}, {"Margot", :cat} ]

# pattern match
for {name, :cat} <- prefs, do: name
# ["Margot"]

# guard clause
for {name, pet_choice} <- animals, pet_choice == :dog, do: name
# ["Liz"]

# anonymous function in a guard clause
cat_lover? = fn(choice) -> choice == :cat end
for {name, pet_choice} <- prefs, cat_lover?.(pet_choice), do: name
# ["Margot"]

# destructuring plus filtering
pets = [
  %{name: "Fluffy", species: :cat, sleeping: true},
  %{name: "Joe", species: :dog, sleeping: false}
]
for pet <- pets,
    name = pet.name,
    sleeping = pet.sleeping do
  "Is #{name} sleeping? #{sleeping}"
end
# => ["Is Fluffy sleeping? true"]

# applying for each list
for i <- 1..2, IO.inspect(i), j <- 5..6 do
  {i, j}
end

for i <- 1..10, Integer.is_even(i), j <- 5..6 do
  {i, j}
end
# [{2, 5}, {2, 6}, {4, 5}, ...

Atomize keys - into

You can use into and do option from comprehensions to achieve this:

style = %{"color" => "#FFF", "display" => "flex", "border" => "2px"}
for {key, val} <- style, into: %{}, do: {String.to_atom(key), val}
# %{border: "2px", color: "#FFF", display: "flex"}

Cartesian product

names = ~w[John Jane]
surnames = ~w[Doe Silva]

for name <- names,
    surname <- surnames do
  "#{name} #{surname}"
end

Unique

for n <- [1, 1, 2, 2, 2, 3], uniq: true do
  n
end

Reduce

pets = [
  %{name: "Fluffy", species: :cat, sleeping: true},
  %{name: "Joe", species: :dog, sleeping: false},
  %{name: "Joe", species: :cat, sleeping: false}
]

for pet <- pets, reduce: %{} do
  acc ->
    Map.update(acc, pet.name, [pet], &[pet|&1])
end

# Groups by name, returns a new map
%{
  "Fluffy" => [%{name: "Fluffy", sleeping: true, species: :cat}],
  "Joe" => [
    %{name: "Joe", sleeping: false, species: :cat},
    %{name: "Joe", sleeping: false, species: :dog}
  ]
}

Filter, map at same time

for %{sleeping: false} = pet <- pets, into: %{} do
  {pet.name, pet}
end

# returns
%{"Joe" => %{name: "Joe", sleeping: false, species: :cat}}

Elixir async and state management

• Use a Task if you want to perform a one-off computation or query asynchronously.
• Use an Agent if you just need a simple process to keep the state.
• Use a GenServer if you need a long-running server process that store states and performs work concurrently.
• Use a dedicated GenServer process if you need to serialize access to a shared resource or service used by many concurrent processes.
• Use a GenServer process if you need to schedule background work to run on a periodic interval.
• You can start with one and migrate to another later, like from Agent to GenServer

GenServer callback functions

handle_call(message, from, state)

You want synchronous requests sent by the client. So the client is waiting for an immediate answer - like an ack answer - with the new state. from is usually ignored. It's format is {pid, tag}.

The return is typically {:reply, reply, new_state}.

handle_cast(message, state)

Handles asynchronous requests sent by the client. The client isn't waiting an acknowledgment.

The return is typically {:noreply, new_state}.

handle_info(message, state)

Handles all other requests. Useful to handle unknown messages as informational actions.

The return is typically {:noreply, state}.

init(args)

When GenServer.start(__MODULE__, [], name: @process_name) is called, [] is the initial state that will the init function will receive.

You can use to customize the initial state passed by the caller.

terminate(reason, state)

A handle_call or handle_cast function might return {:stop, reason, new_state}. terminate handles the process exit routines, like saving the state and closing files.

Debugging and Tracing

Check the current state

iex> :sys.get_state(pid)
%HttpServer.PledgeServer.State{cache_size: 5, pledges: [{"rodrigo", 15}, {"margot", 25}]}

Trace changes

iex> :sys.trace(pid, true)
:ok
iex> HttpServer.PledgeServer.create_pledge("moe", 20)

*DBG* pledge_server got call {create_pledge,<<"moe">>,20} from <0.152.0>

Full status

:sys.get_status(pid)

Phoenix Framework

LiveView

Powerful components

Render Slot is a powerful tool to render many similar cells inside a component

Extremely useful for lists, table rows, table cells, repeatable items that needs to be inside another, like with ul>li, tr>td, etc.

https://hexdocs.pm/phoenix_live_view/0.17.10/Phoenix.LiveView.Helpers.html#render_slot/2

<.unordered_list let={item} items={~w(Apples Oranges Bananas)}>

def unordered_list(assigns) do
  ~H"""
  <ul>
  <%= for item <- @items do %>
    <li><%= render_slot(@inner_block, item)%></li>
  <% end %>
  </ul>
  """
end

More on https://hexdocs.pm/phoenix_live_view/Phoenix.Component.html

Ruby

Summary

• Language Patterns
• Structure Patterns
  • Struct and OpenStruct
• Architectural Patterns
  • Double Dispatch
  • Visitor
  • Iterator

Frameworks

• Rails

Ruby Struct

Structs: Simple Objects

Objective: Fancy hashes that responds to attributes names as a function, like in JS Objects

Car = Struct.new(:model, :maker, :year, keyword_init: true)

vw_up = Car.new model: 'Up!', year: 2015, maker: 'VW'
polo = Car.new('Polo', 'VW', 2018)
golf = Car.new 'Golf', 'VW'


polo.maker
# "VW"
golf.year
# nil

Equality: It uses their attributes instead object ids like default classes

# vw_up2 = Car.new model: 'Up!', maker: 'VW', year: 2015
vw_up == vw_up2
# True

OpenStruct: Simpler alternative

require 'ostruct'

cat = OpenStruct.new(color: 'black')
puts cat.class
puts cat.color

Differences from Struct

• Struct creates a new class with predefined attributes, equality method (==) & enumerable
• OpenStruct creates a new object with the given attributes

Double Dispatch

Objective: Turn a type checking imperative procedure in a more maintenable OO version.

Related to Visitor Pattern.

Some differences between the two on Refactoring Guru

Applications

• Avoiding case for type checking in Ruby to comply with polymorphic collections
• Use the Visitor when you need to perform an operation on all elements of a complex object structure (trasversing a tree). Like Visitor

One Example

Using a simple game, Rock, Paper, Scissors, like in RubyGuides we can start understand why:

class Scissors
    def wins_against?(other_move)
        case other_move
        when Rock then false
        when Paper then true
        end
    end
end

We want to avoid that example of case with this:

class Scissors
    def wins_against?(other_move)
        other_move.do_you_beat_scissors?
    end
    def do_you_beat_rock?
        false
    end
    def do_you_beat_paper?
        true
    end
end

Another example

# Hosts
class Car
    def dispatch_maintenance(caretaker)
      caretaker.maintenance_on_car(self)
    end
end

class Truck
    def dispatch_maintenance(caretaker)
      caretaker.maintenance_on_truck(self)
    end
end

# Visitors
class Mechanic
    def work_on(vehicle)
        vehicle.dispatch_maintenance(self)
    end
    def maintenance_on_car(vehicle)
        change_motor_oil(vehicle, tools: LightTools.grab())
    end
    def maintenance_on_truck(vehicle)
        change_motor_oil(vehicle, tools: HeavyTools.grab())
        check_tires_condition(vehicle) && change_tires(vehicle)
    end
end

class CarWasher
    def work_on(vehicle)
       vehicle.dispatch_work(self)
    end
    def maintenance_on_car(vehicle)
        grab_some_cleaning_cloth()
        do_wash(vehicle, using: LightMachine)
    end
    def maintenance_on_truck(vehicle)
        check_water_reservoir()
        do_wash(vehicle, using: HeavyMachine)
    end
end

mechanic = Mechanic.new
mechanic.work_on(vw_golf)

Here we see caretakers [Mechanic, CarWasher] visiting the patients instances [Car, Truck]. The hosts are responsible to dispatch the correct action for caretakers. It's odd on reality of this example.

Using a in View

Now from Sandi Metz's blog.

class MyView
  attr_reader :target
  
  def initialize(target)
    @target = target
  end

  def double
    case target
      when Numeric  then target * 2 
      when String   then target.next    # lazy example fail
      when Array    then target.collect {|o| MyView.new(o).double}
      else
        raise “don’t know how to double #{target.class} #{target.inspect}”
    end
  end
end

class Numeric
  def double
    self * 2
  end
end

class String
  def double
    self.next 
  end
end

class Array
  def double
    collect {|e| e.double}
  end
end

class Object
  def double
    raise "don't know how to double #{self.class} #{self.inspect}"
  end
end

class MyView
  attr_reader :target
  
  def initialize(target)
    @target = target
  end

  def double
    target.double
  end
end

Visitor

Objective: Separate Auxiliary procedures and algorithms from the object they operate.

We can consider the Visitor pattern a more complex version of double dispatch. It uses the double dispatch principle to make it work.

On top of double dispatch, Visitor lets you add “external” operations to a whole class hierarchy without changing the existing code of these classes.

Visitor isn’t a very common pattern because of its complexity and narrow applicability.

Related to Double Dispatch. Refactoring Guru

Applications

• Use the Visitor when you need to perform an operation on all elements of a complex object structure (trasversing a tree).
• Use the Visitor to clean up the business logic of auxiliary behaviors.

Example

class Node
  def accept visitor
    raise NotImpelementedError.new
  end
end

module Visitable
  def accept visitor
    visitor.visit(self)
  end
end

class IntegerNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class StringNode < Node
  include Visitable

  attr_reader :value
  def initialize value
    @value = value
  end
end

class Ast < Node
  def initialize
    @nodes = []
    @nodes << IntegerNode.new(2)
    @nodes << StringNode.new("3")
  end

  def accept visitor
    @nodes.each do |node|
      node.accept visitor
    end
  end
end

class BaseVisitor
  def visit subject
    method_name = "visit_#{subject.class}".intern
    send(method_name, subject )
  end
end

class DoublerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 2
  end

  def visit_StringNode subject
    puts subject.value.to_i * 2
  end
end

class TriplerVisitor < BaseVisitor
  def visit_IntegerNode subject
    puts subject.value * 3
  end

  def visit_StringNode subject
    puts subject.value.to_i * 3
  end
end

ast = Ast.new
puts "Doubler:"
ast.accept DoublerVisitor.new
puts "Tripler:"
ast.accept TriplerVisitor.new

# =>
Doubler:
4
6
Tripler:
6
9

Real World Usage

Arel uses this pattern to build specific database queries. See Arel PostgreSQL

Iterator

# It's just a class to demonstrate a collection capability that answers to enumerable idiom
class ACollection
  include Enumerable

  # @return [Array]
  attr_accessor :collection
  private :collection


  # @param [Array] collection
  def initialize(collection = [])
    @collection = collection
  end

  def each(&block)
    @collection.each(&block)
  end

  def <<(item)
    @collection << item
  end

  def append(item)
    self << item
  end
end


collection = ACollection.new
collection.append('First')
collection << 'Second'
collection.append('Third')

puts 'Straight traversal:'
collection.each { |item| puts item }
puts "\n"

puts 'Reverse traversal:'
collection.reverse_each { |item| puts item }

Python Tips

Install

ArchLinux or Manjaro system packages

For Data Science / Engineering

yay -S --needed python-ipykernel python-pandas python-numexpr python-beautifulsoup4 python-sqlalchemy python-xlrd python-xlwt python-tabulate python-psycopg2 python-pymysql python-seaborn python-statsmodels gnuplot python-numpy python-pandas-datareader python-scipy python-openpyxl python-matplotlib python-bottleneck mysql-python python-mysql-connector cython python-thrift python-fastparquet

For development

yay -S --needed python-flask python-django python-bcrypt python-coverage python-pytest-pylint python-pylint python-sqlalchemy

Python VirtualEnvs

VirtualEnvs on Fish Shell

First you need to use oh my fish, preferably with my .dotfiles

omf install virtualfish
pip install virtualfish
vf install compat_aliases projects environment auto_activation global_requirements update_python 

Then for existing projects:

vf new myprojectenv
vf connect
cd
vf disconnect
cd ~/Projects/myproject

Check if the environment auto changes.

Pipenv

pacman -S python-pipenv pyenv
omf install pyenv
pyenv install 3.7.7
pipenv

Then on any project folder:

pipenv install --python 3.7

Django

Using Django Shell Plus inside a VS Code Notebook

Preparing Django and the Python Virtual Env

Django Shell Plus is a nice way to explore your django project.

First on your project venv install the requisites:

pip install jupyterlab jupyter ipython django-extensions

Now add django-extensions to your DJANGO_PROJECT/settings.py:

INSTALLED_APPS = [
    'django_extensions',

Setting a fixed authentication

It's boring to copy and paste the token everytime you start the notebook to the VSCode, so we have two ways:

ps: disabling jupyter security does not work currently in vscode

With a Password

A nice trick is to set a password instead copying and pasting the jupyter token everytime you starts it, because jupyter changes it everytime you starts it.

Run this:

jupyter notebook password

To define a local password. This way VSCode will ask your password at least 4 times, but with a known value.

With a fixed token

Open ~/.jupyter/jupyter_notebook_config.json with your favorite editor then paste:

{
  "NotebookApp": {
    "token": "MY_SECURE_TOKEN"
  }
}

The real trick is here: you need to use a "remote" jupyter notebook, because the notebook started by VS Code is in a state that I cannot load Django successfully, even settings paths, messing with os.change_dir and similar workarounds.

Press CTRL+SHIFT+P in VSCode and type remote jupyter. When asked for the server address put this:

http://localhost:8888/?token=MY_SECURE_TOKEN

Now reload VSCode. Run in a separate terminal the external Django Shell Plus notebook, inside your project dir:

cd DJANGO_PROJECT

Then

python manage.py shell_plus --notebook --no-browser

Create a new notebook file anywhere inside your project, like in DJANGO_PROJECT/notebooks/test.ipynb.

Before running your first django code, run any notebook cell, it will connect to your "remote" jupyter server and ask for the password, unless you are using the fixed token.

Now FINALLY change the kernel to Django Shell Plus on the VSCode GUI, upper right. Only then you can import your models and start exploring your django data.

On your first cell put this:

import os
os.environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = "true"
from myapp.models import Model1, Model2
Model1.objects.first()

Fluent Python

Chapter 9 on pytest

Chapter 9 of the Fluent Python book, by Luciano Ramalho, presents a Vector2d class.

It's a good reminder and example for experienced developers that are learning python. For instance, I can remember lots of concepts at once and compare with my other known programming languages, like Ruby.

Directory Structure suggestion

I also converted to the follow structure, recommended for a isolated component/module in python. I was used to Rails framework conventions with a different autoloader, so this reminds me that I need to create this structure by myself in python way.

And yes, as a seasoned Rubist, it's weird to put __init__ everywhere. Don't forget that.

+ fluentpy
|- __init__.py
|+ chapter9
 |- __init__.py
 |- vector2d.py
|+ tests
 |- __init__.py
 |+ chapter9
  |- __init__.py
  |- test_vector2d.py

Tests, converted to pytest

The original tests Ramalho presents on the book are the also widely used doctest. Buy the book and see there the original. The book is worthy every penny. I just converted (and added some bits) to pytest to make most of my learning and to remember in the future.

The coverage is 100% of the original code.

In the file fluentpy/tests/chapter9/test_vector2d.py:

import math
import pytest

import fluentpy.chapter9.vector2d as v2d

class TestVector2d:
    v1 = v2d.Vector2d(3, 4)

    def test_init_int(self):
        x, y = self.v1
        assert (x, y) == (3.0, 4.0), 'Must return a tuple'

    def test_clone_eval(self):
        v1_clone = eval('v2d.' + repr(self.v1))
        assert self.v1 == v1_clone, \
            'repr must be evaluated'

    def test_str(self):
        assert str(self.v1) == '(3.0, 4.0)', \
            'str conversion need to print a tuple'

    def test_bytes(self):
        assert bytes(self.v1) == b'd\x00\x00\x00\x00\x00\x00\x08@\x00\x00\x00\x00\x00\x00\x10@', \
            'byte representation must match'

    def test_abs(self):
        assert abs(self.v1) == 5.0, \
            'abs must be working'

    def test_bool(self):
        assert (bool(self.v1), bool(v2d.Vector2d(0, 0))) == (True, False), \
            'bool conversion must evaluate False on null vectors'

    def test_from_bytes(self):
        v1_clone = v2d.Vector2d.frombytes(bytes(self.v1))
        assert self.v1 == v1_clone

    def test_format(self):
        assert format(self.v1) == '(3.0, 4.0)', \
            'format without params'
        assert format(self.v1, '.2f') == '(3.00, 4.00)', \
            'format 2 decimal places'
        assert format(self.v1, '.3e') == '(3.000e+00, 4.000e+00)', \
            'format cientific exp mode'

    def test_angle(self):
        assert v2d.Vector2d(0, 0).angle() == 0.0, \
            'angle on null vector is zero'
        assert v2d.Vector2d(1, 0).angle() == 0.0, \
            'angle with y component zero is zero'
        epsilon = 10 ** -8
        assert abs(v2d.Vector2d(0, 1).angle() - math.pi / 2) < epsilon, \
            'angle must be valid on circle'
        # interesting reading: https://www.euclideanspace.com/maths/geometry/trig/inverse/index.htm
        assert abs(v2d.Vector2d(1, 1).angle() - math.pi / 4) < epsilon

    def test_format_polar(self):
        v_1_1 = v2d.Vector2d(1, 1)
        assert format(v_1_1, 'p') == '<1.4142135623730951, 0.7853981633974483>', \
            'vector in polar format must match'
        assert format(v_1_1, '.3ep') == '<1.414e+00, 7.854e-01>', \
            'vector in polar format, with 3 decimals and exponential rest must match'
        assert format(v_1_1, '0.5fp') == '<1.41421, 0.78540>', \
            'vector in polar format, with 5 decimals, truncated rest must match'

    def test_x_y_handling(self):
        # x and y must be read only
        with pytest.raises(AttributeError):  # as e_info:
            self.v1.x = 1
        with pytest.raises(AttributeError):
            self.v1.y = 2

    def test_hashing(self):
        v2 = v2d.Vector2d(3.1, 4.2)
        assert hash(self.v1) == 7, \
            'hash of vector (3, 4) is 7'
        assert hash(v2) == 384307168202284039, \
            'hash of a non integer components must work'
        assert hash(self.v1) != hash(v2), \
            'hash of (3, 4) must not match to (3.n, 4.n)'
        assert hash(self.v1) == hash(v2d.Vector2d(3, 4)), \
            'two different instance of vectors with same components must match'

    def test_set(self):
        assert len({self.v1, v2d.Vector2d(0, 0)}) == 2, \
            'converting to a set must work, with two different vectors'
        assert len({self.v1, self.v1}) == 1, \
            'converting to set must work, with two equivalent vectors'

The original code to be tested

File: fluentpy/chapter9/vector2d.py

from array import array
import math


class Vector2d:
    type_code = 'd'

    def __init__(self, x, y):
        self.__x = float(x)
        self.__y = float(y)

    @property  # getter
    def x(self):
        return self.__x

    @property  # getter
    def y(self):
        return self.__y

    def __iter__(self):
        return (i for i in (self.x, self.y))

    def __repr__(self):
        class_name = type(self).__name__
        return "{}({!r}, {!r})".format(class_name, *self)

    def __str__(self):
        return str(tuple(self))

    def __bytes__(self):
        return (bytes([ord(self.type_code)])) + \
               bytes(array(self.type_code, self))

    def __eq__(self, other):
        return tuple(self) == tuple(other)

    def __hash__(self):
        return hash(self.x) ^ hash(self.y)

    def __abs__(self):
        return math.hypot(self.x, self.y)

    def __bool__(self):
        return bool(abs(self))

    def angle(self):
        return math.atan2(self.y, self.x)

    def __format__(self, format_spec=''):
        if format_spec.endswith('p'):
            format_spec = format_spec[:-1]
            coords = (abs(self), self.angle())
            outer_format = '<{}, {}>'
        else:
            coords = self
            outer_format = '({}, {})'
        components = (format(c, format_spec) for c in coords)
        return outer_format.format(*components)

    @classmethod
    def frombytes(cls, octets):
        type_code = chr(octets[0])
        memv = memoryview(octets[1:]).cast(type_code)
        return cls(*memv)

In this code we have lots of methods that are used by common python functions. For instance str(something) converts the something to string. The something needs to respond to __str__, that will be called by str(). It's the equivalent in ruby to implement to_s in a class/module and use the something.to_s, that is a convetion to string conversions in Ruby.

Another example is __iter__ that must be implemented to make the object compatible with splat/unpack syntax *something or "matching" with x, y = something.

Every detail here is important for a reason. You will not use everything always.

Jupyter Notebook Tips

Notebook folder using remote VSCode

When you use a docker container with Jupyter Notebook and VSCode Remote Connect feature to develop your notebooks, you can have difficulties to load local python libraries inside you folder.

You need to set the correct working directory with this:

# Set the correct folder for Remote VSCode
import os

try:
    # '.' if the path is to current folder
    os.chdir(os.path.join(os.getcwd(), './notebooks'))
    # print(os.getcwd())
except:
    pass
os.getcwd()

Then you will have easier time to import your libs.

Helix Editor

Selection

:tutor Chapter 3 Recap

• Type w to select forward until the next word.

  • Type e to select to the end of the current word.
  • Type b to select backward to the start of the current word.
  • Use uppercase counterparts, W,E,B, to traverse WORDS.

• Type d to delete the entire selection.

  • Type c to delete the selection and enter Insert mode.

• Type a number before a motion to repeat it that many times.

• Type v to enter Select mode, where all motions extend the selection.

• Type x to select the entire current line. Type x again to select the next line.

• Type semicolon ( ; ) to collapse selection.

Undo / Copy / Search

:tutor Chapter 4 Recap

• Type u to undo. Type U to redo.

• Type y to yank (copy) text and p to paste.

  • Use Space + y and Space + p to yank / paste on the system clipboard.

• Type / to search forward in file, and ? to search backwards.

  • Use n and N to cycle through search matches.

Multi Cursor

:tutor Chapter 5 Recap

• Type C to duplicate the cursor to the next suitable line and Alt-C for previous suitable line.

• Type s to select all instances of a regex pattern inside the current selection.

• Type & to align selections.

• Type Alt-s to split the selection into lines.

Selection with Search / Replace

:tutor Chapter 6 Recap

• Type f / F to extend selection up to & including a character.

  • Type t / T to extend selection until a character.

• Type r to replace selected characters.

• Type . to repeat the last insertion.

  • Type Alt-. to repeat the last f / t selection.

Advanced Manipulation

:tutor Chapter 7 Recap

• Type R to replace the selection with yanked text.

• Type J to join lines in selection.

• Type < and > to indent / outdent lines.

• Type Ctrl-a to increment the selected number.

  • Type Ctrl-x to decrement the selected number.

Search

:tutor Chapter 9 Recap

• Type * to set the search register to the primary selection.

• Type n / N in Visual mode to add selections on each search match.

• Type Ctrl-s to save position to the jumplist.

  • Type Ctrl-i and Ctrl-o to go forward and backward in the jumplist.

Manipulating Selections

:tutor Chapter 10 Recap

• Use ) and ( to cycle the primary selection back and forward through selections respectively.

  • Type Alt-, to remove the primary selection.

• Type ~ to alternate case of selected letters.

  • Use and Alt- to set the case of selected letters to upper and lower respectively.

• Type S to split selections on regex.

Troubleshooting Gnome Alt+` backtick

You can set only Super+` for App switch.

 gsettings set org.gnome.desktop.wm.keybindings switch-group "['<Super>Above_Tab']"

Other Data Tools

More languages to play on Manjaro and a notebook

yay -S nteract-bin ihaskell-git r sagemath-jupyter julia

NVIDIA Accelerated Docker containers on ArchLinux

yay -S nvidia-container-runtime
sudo tee /etc/docker/daemon.json <<EOF
{
    "default-runtime": "nvidia",
    "runtimes": {
        "nvidia": {
            "path": "/usr/bin/nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
docker run nvidia/cuda:10.1-base nvidia-smi

Compiling Psycopg2 for Lambda on ArchLinux

Please pay attention to the version numbers, change to newer versions as appropriate.

cd ~/Downloads
pacman -S postgresql postgresql-libs
wget http://initd.org/psycopg/tarballs/PSYCOPG-2-7/psycopg2-2.7.5.tar.gz
tar xf psycopg2-2.7.5.tar.gz
cd psycopg2-2.7.5/
sed -i 's/libpq.a/libpq.so/g' setup.py
gedit setup.cfg

Change setup.cfg build_ext section to that:

[build_ext]
define = 
pg_config = /usr/bin/pg_config
use_pydatetime = 1
mx_include_dir = 
have_ssl = 0
static_libpq = 1
libraries = ssl crypto

After that, we will finally build:

python setup.py build
cd build/lib.linux-x86_64-3.7
cp -R psycopg2/ ~/Projects/{MY_LAMBDA_PROJECT}/

Exporting PDF using jupyter nbconvert

yay -S texlive-most --needed

Redshift

Grant read only access to a group on a schema

-- Create Read-Only Group     
CREATE GROUP ro_group;

-- Create User
CREATE USER ro_user WITH password PASSWORD;

-- Add User to Read-Only Group
ALTER GROUP ro_group ADD USER ro_user;

-- Grant Usage permission to Read-Only Group to specific Schema
GRANT USAGE ON SCHEMA "ro_schema" TO GROUP ro_group;

-- Grant Select permission to Read-Only Group to specific Schema
GRANT SELECT ON ALL TABLES IN SCHEMA "ro_schema" TO GROUP ro_group;

-- Alter Default Privileges to maintain the permissions on new tables
ALTER DEFAULT PRIVILEGES IN SCHEMA "ro_schema" GRANT SELECT ON TABLES TO GROUP ro_group;

-- Revoke CREATE privileges from group
REVOKE CREATE ON SCHEMA "ro_schema" FROM GROUP ro_group;

Testing sorting keys

create table datasci.test_table_compound
compound sortkey (created_at, id)
as select * from public.test_table;

create table datasci.test_table_interleaved
interleaved sortkey (created_at, id)
as select * from public.test_table;

This copy for me over a million rows tables works pretty fast (< 1 minute), so it's worthy to copy some optimized tables.

MySQL

Number generator

The purpose is to join or count in special ocasions you can rely on the index, or use a custom order, on old versions without powerful window functions.

You can use maths + join on numbers to create discrete time windows based on created_at, for instance.

CREATE OR REPLACE VIEW generator_16
AS SELECT 0 n UNION ALL SELECT 1  UNION ALL SELECT 2  UNION ALL 
   SELECT 3   UNION ALL SELECT 4  UNION ALL SELECT 5  UNION ALL
   SELECT 6   UNION ALL SELECT 7  UNION ALL SELECT 8  UNION ALL
   SELECT 9   UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL
   SELECT 12  UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL 
   SELECT 15;

CREATE OR REPLACE VIEW generator_256
AS SELECT ( ( hi.n << 4 ) | lo.n ) AS n
     FROM generator_16 lo, generator_16 hi;

CREATE OR REPLACE VIEW generator_4k
AS SELECT ( ( hi.n << 8 ) | lo.n ) AS n
     FROM generator_256 lo, generator_16 hi;

CREATE OR REPLACE VIEW generator_64k
AS SELECT ( ( hi.n << 8 ) | lo.n ) AS n
     FROM generator_256 lo, generator_256 hi;

CREATE OR REPLACE VIEW generator_1m
AS SELECT ( ( hi.n << 16 ) | lo.n ) AS n
     FROM generator_64k lo, generator_16 hi;

-- create table for results

drop table if exists numbers ;

create table `numbers` (
  `i` int(11) signed 
  , primary key(`i`)
) ENGINE=myisam DEFAULT CHARSET=latin1;

INSERT INTO numbers(i)
SELECT n FROM generator_64k WHERE n < 64000

Creating database and privileges

mysql -h 127.0.0.1 -uroot -p

CREATE USER 'ghost'@'%';
GRANT ALL PRIVILEGES ON ghost.* To 'ghost'@'%' IDENTIFIED BY 'ghost123';
FLUSH PRIVILEGES;

CTRL-D to disconnect

OpenWRT tricks

TIM Live Brazil Fiber (FTTH)

It uses PPPoE like the VDSL, but you need to set the VLAN.

Unlike the VDSL connections, for FTTH connections you need the VLAN ID 100.

Unfortunately you can't use the Luci GUI interface. Tested with OpenWrt 21.02.

Edit /etc/config/network, specifically the find wan and wan6 interfaces:

config interface 'wan'
        option proto 'pppoe'
        option password 'guest'
        option ipv6 'auto'
        option username 'guest'
        option device 'eth0.2'
        option ifname 'dsl0.100'  ## This is essential

config interface 'wan6'
        option proto 'dhcpv6'
        option device 'eth0.2'

IPv6 needs more work, since my connection does not have it enabled, trying to enable inside the wan6 prevents me the connection with PADO timeouts.

Be aware that device 'eth0.2' can be different for your router model.

Slow PPPoE

If you're having slow throughput using PPPoE, specially under the FTTH connection above, enable Software flow offloading inside Firewall settings.

It will make QoS SQM to not work, but since I have plenty bandwith now, I just disabled it.

Blocking URLs at dnsmasq

uci add_list dhcp.@dnsmasq[0].address='/address.com/127.0.0.1'
uci commit dhcp
/etc/init.d/dnsmasq restart

Some useful packages

ca-bundle curl ddns-scripts ddns-scripts_cloudflare.com-v4 etherwake flent-tools ip-tiny iptables-mod-conntrack-extra iptables-mod-ipopt kmod-ifb kmod-ipt-conntrack-extra kmod-ipt-ipopt kmod-ipt-raw kmod-sched-cake kmod-sched-core kmod-udptunnel4 kmod-udptunnel6 kmod-wireguard libcap libcurl4 libelf1 libmbedtls12 libmnl0 libncurses6 libqrencode librt luci-app-ddns luci-app-sqm luci-app-wireguard luci-app-wol luci-compat luci-lib-ipkg luci-proto-wireguard nano netperf qrencode speedtest-netperf sqm-scripts tc terminfo wireguard wireguard-tools zlib

Ubuntu/Debian Based

Keybase

cd ~/Downloads
curl -O https://prerelease.keybase.io/keybase_amd64.deb
sudo dpkg -i keybase_amd64.deb
sudo apt-get install -f
run_keybase

Node / Yarn

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt update
sudo apt install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev software-properties-common libffi-dev nodejs yarn

Linux Brew

sudo apt install build-essential curl file git
sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"
nano ~/.profile

test -d ~/.linuxbrew && eval $(~/.linuxbrew/bin/brew shellenv)
test -d /home/linuxbrew/.linuxbrew && eval $(/home/linuxbrew/.linuxbrew/bin/brew shellenv)
test -r ~/.bash_profile && echo "eval \$($(brew --prefix)/bin/brew shellenv)" >>~/.bash_profile
echo "eval \$($(brew --prefix)/bin/brew shellenv)" >>~/.profile

Configure a Ghost Blog at GCE

Node + Yarn + Requisites

DBUS and SETCAP are not installed by default on GCE Debian

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -

echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update && sudo apt-get install -y yarn nodejs dbus libcap2-bin wget

Update Practices

sudo apt update && sudo apt upgrade

weekly

Ghost Engine

echo "PATH=$PATH:$HOME/.yarn/bin" >> ~/.bashrc
source ~/.bashrc
yarn global add ghost-cli@latest
sudo rm -rf /var/www/ghost
sudo mkdir -p /var/www/ghost
sudo chown $(whoami): /var/www/ghost

ghost install --url=https://MYSITE.COM --admin-url=https://ADMIN.MYSITE.COM --db=sqlite3 --mail=SMTP --mailservice=Mailgun [email protected] --mailpass=MAILGUN_SMTP_PW --no-stack --no-setup-ssl --no-prompt -d /var/www/ghost

Update Practices

ghost update

whenever a new version comes

Caddy Webserver

Auto HTTPS using Let's Encrypt

Caddyfile

First we need to create a Caddyfile, that is a configuration file for Caddy HTTPS Server. nano Caddyfile or vim Caddyfile and paste this:

MYSITE.COM {
  status 404 /ghost
  proxy / http://127.0.0.1:2368 {
    transparent
    fail_timeout 300s
    header_upstream X-Forwarded-Ssl on
  }

  tls [email protected]
  gzip
}

ADMIN.MYSITE.COM {
  status 404 //
  proxy / http://127.0.0.1:2368/ {
    transparent
    fail_timeout 300s
    header_upstream X-Forwarded-Ssl on
  }
  tls [email protected]
  gzip
}

www.MYSITE.COM {
  redir https://MYSITE.COM
  tls [email protected]
}

Obviously modify to your site settings.

I separated the admin site for hardening access. It has stricter rules at Cloudflare Edge. The ADMIN is some long string (kind of phrase password). It can make life harder for bots. Plus with stricter rules at my Cloudflare Edge for these bots (for instance, mandatory Captcha thoughter rate limits), it will not eat Network Fees from Google Cloud (or whatever you are using).

Installation

Now we will install and start the service:

curl https://getcaddy.com | bash -s personal
sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddy
sudo mkdir /etc/caddy
sudo chown -R root:www-data /etc/caddy
sudo mkdir /etc/ssl/caddy
sudo chown -R root:www-data /etc/ssl/caddy
sudo chmod 0770 /etc/ssl/caddy
sudo cp ~/Caddyfile /etc/caddy/
sudo chown www-data:www-data /etc/caddy/Caddyfile
sudo chmod 444 /etc/caddy/Caddyfile
sudo chown www-data:www-data /var/www
sudo chmod 555 /var/www
wget https://raw.githubusercontent.com/mholt/caddy/master/dist/init/linux-systemd/caddy.service
sudo cp caddy.service /etc/systemd/system/
sudo chown root:root /etc/systemd/system/caddy.service
sudo chmod 644 /etc/systemd/system/caddy.service
sudo systemctl daemon-reload
sudo systemctl start caddy.service
sudo systemctl enable caddy.service

Update practices

Whenever a new caddy version comes, or weekly:

curl https://getcaddy.com | bash -s personal
sudo pkill -USR2 caddy

Automating this is out of this doc scope (tip: CRON)

Logs

To follow connection logs on Caddy:

journalctl -f -u caddy.service

Or requests at node server:

journalctl -f -u ghost_MYSITE.service

Git

Redundancy

Add Multiple repositories

Refer to my dotfiles/fish/functions.fish file

Push to all remotes at same time

Creating an alias is a good idea

git config --global alias.pushall '!git remote | xargs -L1 git push --all'

Keychron with VIA inside Linux

It is supported only through Chromium

Add to /etc/udev/rules.d/50-keychron-chrome.rules

KERNEL=="hidraw*", ATTRS{idVendor}=="3434", MODE="0664", GROUP="plugdev"

Reboot your system to be sure, then go to https://usevia.app

Fedora does not have a plugdev group like ArchLinux, so change the group to something your user belongs.

WSL

Backup using WSL2 + Compression

Inside a Power Shell terminal:

wsl --export Distrod  - | zstd -10 -o "D:\archlinux-$(Get-Date -Format FileDateUniversal).tar"

You'll need zstd installed. Use scoop.sh inside PowerShell.

scoop install zstd

Podman

Running a container with Wireguard && Current User

set -lx UID (id -u)
set -lx GID (id -g)

podman run -d --user 0:0 --userns keep-id:uid=$UID,gid=$GID \
    --sysctl="net.ipv4.conf.all.src_valid_mark=1" \
    --privileged=true \

Wayland

Overriding electron apps with Wayland

This was tested using Fedora 37. The example below is using Logseq.

flatpak override --user --socket=wayland --env=GDK_BACKEND=wayland com.logseq.Logseq

Then try to open with:

GDK_BACKEND=wayland flatpak run com.logseq.Logseq --enable-features=UseOzonePlatform --ozone-platform=wayland

Now you can edit with Alacarte/Menu Editor:

Instead of

/usr/bin/flatpak run --branch=stable --arch=x86_64 --command=run.sh --file-forwarding com.logseq.Logseq @@u %U @@

Use this

/usr/bin/flatpak run --branch=stable --arch=x86_64 --command=run.sh --file-forwarding com.logseq.Logseq --enable-features=UseOzonePlatform --ozone-platform=wayland @@u %U @@

Obsidian

Obsidian has a dedicated environment variable:

flatpak override --user --env=OBSIDIAN_USE_WAYLAND=1 md.obsidian.Obsidian

Fedora Setup

Yubikey with GPG

This can help with Yubikey card not available after sleep.

Remove opensc

sudo dnf remove opensc

File ~/.gnupg/scdaemon.conf:

reader-port Yubico Yubi
pcsc-shared

File /etc/polkit-1/rules.d/10-pcsc-custom.rules:

polkit.addRule(function(action, subject) {
    if (action.id == "org.debian.pcsc-lite.access_pcsc" &&
        subject.user == "rdlu") {
            return polkit.Result.YES;
    }
});

polkit.addRule(function(action, subject) {
    if (action.id == "org.debian.pcsc-lite.access_card" &&
        action.lookup("reader") == 'Yubico YubiKey OTP+FIDO+CCID 00 00' &&
        subject.user == "rdlu") {
            return polkit.Result.YES;
    }
});