Tutorials on cache cascade

Using PAM for SSH Login Telephony

tai@taigrr.com (Tai Groot) — Sun, 18 May 2025 22:31:25 +0600

Pluggable Authentication Modules ( PAM) provide a flexible mechanism for authenticating users on Linux systems. By integrating PAM with Gotify, you can receive real-time notifications for SSH login attempts, enhancing your system’s security and awareness.

I’ve been using this approach for over 5 years across many systems — it’s lightweight, reliable, and highly effective. Let’s walk through how to set it up.

Prerequisites

Before you begin, make sure you have the following:

A Linux server with SSH access.
Root or sudo privileges.
curl installed.

Step 1: Install and Configure Gotify

Gotify is a simple self-hosted server for sending notifications. You can also use a hosted instance if desired.

Install Gotify or use an existing instance.
Create an application in Gotify and note down the application token.

Test that your setup works by sending a test message:

curl -X POST "http://<gotify-server>/message" \
 -H "X-Gotify-Key: <application-token>" \
 -F "title=Test Notification" \
 -F "message=This is a test message"

Step 2: Create the PAM Notification Script

We’ll configure PAM to trigger this script every time someone logs in via SSH.

Create the script:

sudo vim /usr/local/bin/pam-ssh-notify.sh

Paste the following:

#!/usr/bin/env bash

GOTIFY_URL="http://<gotify-server>/message"
GOTIFY_TOKEN="<gotify-token>"

/usr/bin/curl -s -X POST "$GOTIFY_URL" \
 -H "X-Gotify-Key: $GOTIFY_TOKEN" \
 -F "title=SSH Login Alert" \
 -F "message=User \"$PAM_USER\" logged in from \"$PAM_RHOST\""

Make it executable:

sudo chmod +x /usr/local/bin/pam-ssh-notify.sh

Step 3: Update PAM Configuration

Edit the PAM configuration for SSH:
```
sudo vim /etc/pam.d/sshd
```

Add this line at the end of the file:

session optional pam_exec.so /usr/local/bin/pam-ssh-notify.sh

Why optional? If Gotify is unreachable and this script fails, PAM will ignore the failure. If you use required, login failures could result from simple network errors — not ideal for a monitoring hook.

Step 4: Enable and Test It

Ensure the UsePAM setting is set to ‘yes’ inside of /etc/ssh/sshd_config (or wherever your configuration is).
Restart SSH:
```
sudo systemctl restart sshd
```
Log in from another machine and confirm that your Gotify app receives a notification.

Appendix: PAM Environment Variables

When PAM executes your custom script, it passes in useful environment variables that can be used to enhance logic, filtering, and alerting.

Variable	Description	Example Use Case
`PAM_USER`	User being authenticated	Notify only on root logins
`PAM_RHOST`	Remote IP or hostname	GeoIP filter or blacklist
`PAM_SERVICE`	Service name (e.g. `sshd`, `sudo`)	Alert only on `sshd` logins
`PAM_TTY`	Terminal device (e.g. `pts/0`)	Log which TTY was used
`PAM_TYPE`	Event type (`auth`, `session`, etc.)	Differentiate login vs session start
`PAM_RUSER`	Originating user (e.g. for `sudo`)	Log who used `sudo`

Considerations

While this PAM-to-Gotify setup is simple and effective, it’s best suited for single-node systems like personal VPS setups or small self-hosted boxes, systems you’re willing to give your full attention to. It is not scalable for large fleets or production environments with dozens or hundreds of machines. For those cases, you’d want something more robust and asynchronous — like a logging agent with webhook triggers or a centralized security event bus. For a midrange use case with an audit trail, I’ve also swapped out Gotify for a custom Slack App, using the channel webhooks in the past.

Also, a critical caution: setting your PAM module to required instead of optional can be dangerous. If Gotify (or DNS) is down, a required script failure will prevent user logins entirely. This is not hypothetical — in 2021 Facebook famously locked themselves out of their own data center because DNS broke, and they couldn’t log in to fix the DNS server. By marking this hook as optional, you avoid cascading failures. Learn from their mistakes.

PAM is capable of much more–adding 2-factor authentication, YubiKey support, LDAP auth, encrypted filesystem mounting, etc. If you want to learn more, the ArchLinux Wiki is a great source, even if you’re not running Arch yourself.

Conclusion

With just a few lines of shell and PAM configuration, you can build powerful real-time login alerts and security automation using Gotify. The $PAM_* variables give you rich context for decision-making and logging, and the system is lightweight and reliable.

Feel free to extend this setup to logins via sudo, su, or even failed login attempts — and adapt it to your environment as needed.

Stay secure!

Temporal Server: Self-hosting a Production-Ready Instance

tai@taigrr.com (Tai Groot) — Tue, 29 Apr 2025 18:19:25 +0600

Temporal is an open-source workflow orchestration engine designed to manage, execute, and monitor complex workflows and activities. In plain terms, Temporal allows you to write crash-proof code: it ensures that your long-running processes (workflows) continue execution reliably even if your services (or external services!) crash or restart.

Temporal’s walkthroughs and CLI provide extremely easy adoption and low-friction for getting started: the temporal CLI contains an embedded development server (temporal server start-dev), which binds to localhost and allows for easy, unauthenticated workflows. Connecting to Temporal’s managed service is similarly easy, but at writing, has a minimum cost of $100/month, which is not always affordable for smaller orgs, especially when running a production-grade self-managed server can cost less than a quarter of that.

In this article, we’ll set up a single-node Temporal server backed by PostgreSQL, and we’ll secure it with TLS using NGINX as a reverse proxy. The end result will be Temporal’s services running in Docker containers, accessible at https://temporal.example.com (our example domain) with encrypted connections.

Note: This setup is suitable for small-scale, development and production deployments. Temporal is designed to scale horizontally and run multiple nodes for high availability and heavy workloads. For production, you should consider a multi-node cluster and follow Temporal’s production best practices (e.g. running each Temporal service separately, managing database schemas manually, etc.) community.temporal.io. There is significant effort required to deploy and manage multiple horizontal Temporal servers, between certificate rotation and configuration management, especially as Temporal often changes configuration requirements. Once you get to the stage where you need the reliability of multiple Temporal servers, my recommendation is that you pay for their managed service.

Overview of the Deployment

Our deployment will consist of the following components:

Temporal Server – runs Temporal’s core services (History, Matching) in one container. We will use the official temporalio/auto-setup image for convenience, which launches all services in one process and auto-initializes the database schema docs.temporal.io.
Temporal Web UI – runs in a separate container (temporalio/ui image) to provide a web interface for viewing workflows, tasks, and Temporal namespaces
PostgreSQL – a single Postgres instance as Temporal’s persistence store (keeping workflow state, history, etc.). I recommend using a managed database for this, like DigitalOcean (Disclaimer: this is my personal RefLink to DigitalOcean) to ensure uptime and reliability.
NGINX – runs on the host as a reverse proxy. It will terminate TLS (HTTPS), forwarding requests to the Temporal WebUI container, and provide Basic Auth support.

All these services can run on a single machine via Docker Compose (again, I do recommend using a managed Postgres instance for stability). The domain temporal.example.com will point to this machine (use an A or CNAME record!), and we’ll obtain an SSL certificate for it using Let’s Encrypt.

Prerequisites

Before we begin, make sure you have the following on your VPS:

Docker and Docker Compose: This is how we’ll be standing up our temporal services and ensuring they automatically restart
A domain name: for accessing Temporal (e.g., temporal.example.com), with DNS records pointing to your server’s IP. Don’t forget to reserve a public IP! Some cloud platforms will auto-release and reassign a new IP for you if you reboot, and this will break your DNS!
Firewall Rules: Block all incoming traffic to your server, especially port 8080, and only allow through TCP/7233, TCP/80, and TCP/443. Later, you can whitelist your worker nodes for 7233, or just allow all (0.0.0.0) to access this port.
Certbot: Let’s Encrypt’s CertBot utility can be helpful for provisioning and rotating SSL certs in nginx
GRPCurl: A grpc CLI for doing quick spot-checks on the installation
temporal (cli): A handy CLI utility for interacting with the Temporal server

If you’re running an Ubuntu/Debian/Debian Derivative, this shell snippet may help:

apt install docker-compose-v2 nginx python3-certbot-nginx apache2-utils golang
go install github.com/fullstorydev/grpcurl/cmd/grpcurl@latest
curl -L 'https://temporal.download/cli/archive/latest?platform=linux&arch=amd64' > x.tar.gz
mv temporal /usr/bin
rm LICENSE x.tar.gz

Now, with that out of the way, let’s get started!

Setting Up Our Docker Compose

First, create a directory for your Temporal deployment and navigate into it. Then create a file named docker-compose.yml with the following content:

version: "3.5"
services:
 temporal:
 container_name: temporal
 image: temporalio/auto-setup:${TEMPORAL_VERSION:-1.24.2.1} # You can substitute for the latest Temporal version
 env_file: .env
 restart: always
 networks:
 - temporal-network
 ports:
 - 7233:7233 # Expose gRPC frontend port
 volumes:
 - ./certs:/etc/temporal/certs
 - ./base.yaml:/etc/temporal/config/base.yaml

 temporal-ui:
 container_name: temporal-ui
 image: temporalio/ui:${TEMPORAL_VERSION:-1.24.2.1} # Use the latest Temporal Web UI version
 env_file: .env.ui
 restart: always
 depends_on:
 - temporal
 networks:
 - temporal-network
 ports:
 - 8080:8080 # Expose Web UI port
 volumes:
 - ./ui.yaml:/etc/temporal/config/development.yaml
 - ./certs:/etc/temporal/certs
networks:
 temporal-network:
 driver: bridge
 name: temporal-network

Next, let’s set up Postgres.

Configuring Postgres

There are many ways to configure your Postgres instance, and plenty of ways to restrict permissions. Here are the minimum requirements to make it work:

Create a database named temporal
Create a database named temporal_visibility
Create a user for the Temporal server to use (here we’ll assume tuser)
Using psql on your personal/work machine, you’ll need to connect to the Postgres instance as an admin and grant full access to the user on these two databases:

Note: Don’t forget to whitelist the IP address of your temporal server to access the database! Similarly, you may need to whitelist whichever machine you’re using to connect to the database with psql.

GRANT CONNECT ON DATABASE temporal TO tuser;
GRANT USAGE ON SCHEMA public TO tuser;
GRANT CREATE ON SCHEMA public TO tuser;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO tuser;

Do the same for temporal_visibility:

GRANT CONNECT ON DATABASE temporal_visibility TO tuser;
GRANT USAGE ON SCHEMA public TO tuser;
GRANT CREATE ON SCHEMA public TO tuser;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO tuser;

Now, let’s create some TLS certificates.

TLS certificates

Make a directory called certs, and cd into it. Inside this directory, we’ll generate a RootCA file:

openssl genrsa -out rootCA.key 2048
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 3650 -out rootCA.pem

You can adjust the expiration time or keysize as you wish.

Create a client and server key:

openssl genrsa -out client.key 2048
openssl genrsa -out server.key 2048

Now, using our RootCA, we’ll mint some certificates. Let’s start by creating some Certificicate Signing Request (.csr) files:

openssl req -new -key server.key -out server.csr -config server.cnf

You’ll be prompted to answer some questions about your organization.

Be sure the commonName and field matches your DNS record, i.e. temporal.example.com
For DNS Alt-names, provide the following:
- DNS.1: temporal
- DNS.2: temporal.example.com
- IP.1: 127.0.0.1
- IP.2: <YOUR RESERVED IP HERE>

And similarly, for the client:

openssl req -new -key client.key -out client.csr -config client.cnf

Here, the values don’t really matter. Configure as you see fit.

Now that we have our .csrs, let’s generate the certificates:

openssl x509 -req -in client.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out client.crt -days 3650 -sha256 -extfile client.cnf -extensions v3_req
openssl x509 -req -in server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out server.crt -days 3650 -sha256 -extfile server.cnf -extensions v3_req

Now, you should have a directory that looks like this:

temporal@temporal-server:~$ tree
.
├── certs
│   ├── client.cnf
│   ├── client.crt
│   ├── client.csr
│   ├── client.key
│   ├── rootCA.key
│   ├── rootCA.pem
│   ├── rootCA.srl
│   ├── server.cnf
│   ├── server.crt
│   ├── server.csr
│   └── server.key
└── docker-compose.yml

Next, we need to create our .env and .env.ui files.

Configuring Env Vars

Create a .env file next to your docker-compose.yml, replacing the <SUBSTITUTIONS>:

POSTGRES_USER=<YOUR USER HERE>
POSTGRES_PWD=<PASSWORD HERE>
DB=postgres12_pgx
DBNAME=temporal
VISIBILITY_DBNAME=temporal_visibility
DB_PORT=<YOUR DB PORT>
POSTGRES_TLS_ENABLED=true
POSTGRES_SEEDS=<YOU DB HOSTNAME/IP HERE>
POSTGRES_TLS_DISABLE_HOST_VERIFICATION=true
SQL_TLS=true
SQL_TLS_DISABLE_HOST_VERIFICATION=true
SQL_TLS_ENABLED=true
SQL_HOST_VERIFICATION=false
SKIP_DB_CREATE=true
TEMPORAL_AUTH_ENABLED=true
TEMPORAL_ADDRESS=temporal:7233
TEMPORAL_TLS_REQUIRE_CLIENT_AUTH=true
TEMPORAL_TLS_CERTS_DIR=/etc/temporal/certs
SKIP_DEFAULT_NAMESPACE_CREATION=false
DEFAULT_NAMESPACE=default
TEMPORAL_CLI_TLS=true
TEMPORAL_CLI_TLS_CERT=/etc/temporal/certs/client.crt
TEMPORAL_CLI_TLS_KEY=/etc/temporal/certs/client.key
TEMPORAL_CLI_TLS_CA_CERT=/etc/temporal/certs/rootCA.pem
TEMPORAL_ADDRESS=temporal:7233

Then, create a .env.ui file to configure the webui:

TEMPORAL_TLS_CA_DATA=<REPLACE WITH BASE64 CERT rootCA.pem>
TEMPORAL_TLS_CERT_DATA=<REPLACE WITH BASE64 CERT client.crt>
TEMPORAL_TLS_KEY_DATA=<REPLACE WITH BASE64 CERT client.key>

You can create these base64-encoded values as follows:

echo TEMPORAL_TLS_CA_DATA=$(cat certs/rootCA.pem | base64 -w0) > .env.ui
echo TEMPORAL_TLS_CERT_DATA=$(cat certs/client.crt | base64 -w0) >> .env.ui
echo TEMPORAL_TLS_KEY_DATA=$(cat certs/client.key | base64 -w0) >> .env.ui

Temporal Configuration

Next to our docker-compose.yml, you need to create two configuration files, one for temporal-server and one for the webui.

Create base.yml:

global:
 tls:
 internode:
 server:
 certFile: /etc/temporal/certs/server.crt
 keyFile: /etc/temporal/certs/server.key
 requireClientAuth: true
 clientCaFiles:
 - /etc/temporal/certs/rootCA.pem
 client:
 serverName: temporal
 rootCaFiles:
 - /etc/temporal/certs/rootCA.pem
 frontend:
 server:
 certFile: /etc/temporal/certs/server.crt
 keyFile: /etc/temporal/certs/server.key
 requireClientAuth: true
 clientCaFiles:
 - /etc/temporal/certs/rootCA.pem
 rootCaFiles:
 - /etc/temporal/certs/rootCA.pem
 client:
 serverName: temporal
 rootCaFiles:
 - /etc/temporal/certs/rootCA.pem

services:
 history:
 matching:
 worker:

log:
 level: debug

And ui.yaml:

tls:
 caFile: /etc/temporal/certs/rootCA.pem
 certFile: /etc/temporal/certs/client.crt
 keyFile: /etc/temporal/certs/client.key
 enableHostVerification: false
 serverName: temporal

Spinning Up

Next, run docker compose up and see your services come to life!

You should see a whole mess of tables populate in your Postgres instance, as the auto-setup image does its thing.

Once the logs start to slow down and your server starts into a retry loop, go ahead and kill the process with Ctrl+c. Wait for the containers to spin down, and we have one change to make to our docker-compose.yml: swap out the word autosetup for server to change our specified container image. autosetup is meant to help you easily provision database tables and schema, but isn’t suitable for production deployments, so once our tables are set up, we switch it to the production-ready temporalio/server image.

Configuring NGINX as a TLS Reverse Proxy

We’re so close to complete! We only have one step left: setting up our nginx reverse-proxy with password auth.

Here’s a config you can drop in on top of /etc/nginx/nginx.conf (optionally: instead, add a sites file and link it to sites-enabled):

user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;

events {
 worker_connections 768;
}
http {
 sendfile on;
 tcp_nopush on;
 types_hash_max_size 2048;
 include /etc/nginx/mime.types;
 default_type application/octet-stream;
 ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
 ssl_prefer_server_ciphers on;
 access_log /var/log/nginx/access.log;
 gzip on;
 upstream temporal {
 server 127.0.0.1:8080;
 }

 server {
 auth_basic "Temporal Admins Only!";
 auth_basic_user_file /etc/nginx/.htpasswd;
 server_name temporal.example.com;
 location / {
 proxy_pass http://temporal;
 proxy_http_version 1.1;
 # Ensuring it can use websockets
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "upgrade";
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header X-Forwarded-Proto http;
 proxy_redirect http:// $scheme://;
 proxy_set_header Host $http_host;
 }
 }
 include /etc/nginx/conf.d/*.conf;
 include /etc/nginx/sites-enabled/*;
}

Add the following to your /etc/hosts:

cat << EOF >> /etc/hosts
127.0.0.1 temporal
EOF

Generate a .htpasswd file to control user authentication (you will be prompted to enter a password):

htpasswd -c /etc/nginx/.htpasswd <YOUR USERNAME>

Enable and reload nginx:

systemctl enable --now nginx && systemctl reload nginx

Setup Certbot (this is interactive and may prompt you for your email):

certbot -d temporal.example.com

Note: don’t forget to add certbot renew to a cron job! LetsEncrypt certs typically expire every 3 months, so check for renewals every 1-2 months.

Testing the Setup

Moment of truth: open your browser and navigate to https://temporal.example.com. You should be greeted with a Basic Auth prompt. Enter your username and password you generated earlier. Then, you should see Temporal’s Web UI homepage.

If you see something similar to the screenshot above, congratulations! 🎉 You now have a single-node Temporal server running with TLS. The connection is secure (check that browser lock icon), and you can start building or testing workflows. The UI will show zero workflows initially (since none have been run); it’s expected.

If you see errors regarding “no default namespace”, you can manually create one using the following command:

temporal operator namespace create --namespace default --tls-ca-path certs/rootCA.pem --tls-cert-path certs/client.crt --tls-key-path certs/client.key

Troubleshooting

Health Check

To further verify that gRPC traffic is working through TLS, you can use grpcurl to check the health probe:

~/go/bin/grpcurl --cacert certs/rootCA.pem --cert certs/client.crt --key certs/client.key -d '{"service": "temporal.api.workflowservice.v1.WorkflowService"}' temporal:7233 grpc.health.v1.Health/Check

If you see the following, temporal-server is properly configured:

{
 "status": "SERVING"
}

Creating a Test Workflow

To verify your task queues are set up, you can create a HELLO WORLD workflow (note the nested quotes, which are necessary):

temporal workflow start --address temporal:7233 --type HelloWorldWorkflow --task-queue=HELLO_WORLD_TASK_QUEUE --tls-ca-path certs/rootCA.pem --tls-cert-path certs/client.crt --tls-key-path certs/client.key --input '"Hello, World!"'

Cleaning up Tests

If you’ve created too many test workflows, you can Terminate them from within the Web UI or run the following to delete all of them:

for x in $(temporal workflow list --address temporal:7233 --tls-ca-path certs/rootCA.pem --tls-cert-path certs/client.crt --tls-key-path certs/client.key | awk '{ print $2 }'); do
 temporal workflow delete --address temporal:7233 --tls-ca-path certs/rootCA.pem --tls-cert-path certs/client.crt --tls-key-path certs/client.key -w $x
done

Connecting a Client

Because we have self-signed certificates, we need some way to get the certificates into our client.

Here’s how I do it in go (it will be similar in other languages):

package tclient

import (
 "context"
 "crypto/tls"
 "crypto/x509"
 "encoding/base64"
 "fmt"
 "math/rand"
 "os"
 "sync"

 "github.com/taigrr/temporal-worker/vars"
 "go.temporal.io/sdk/client"
)

var (
 tClient client.Client
 clientTex sync.Mutex
)

func New() (client.Client, string, error) {
 clientTex.Lock()
 defer clientTex.Unlock()
 if tClient != nil {
 return tClient, "", nil
 }
 // base64-encoded TLS certs created the same way we did for .env.ui
 clientCertData := os.Getenv(vars.TemporalTLSCertData)
 clientCertKeyData := os.Getenv(vars.TemporalTLSKeyData)
 serverRootCAData := os.Getenv(vars.TemporalTLSCAData)

 // the FQDN:PORT i.e. temporal.example.com:7233
 hostPort := os.Getenv(vars.TemporalHostPort)

 // default
 namespace := os.Getenv(vars.TemporalNamespace)

 // Decode the base64 encoded keys and certs
 clientCert, err := base64.StdEncoding.DecodeString(clientCertData)
 if err != nil {
 return nil, "", fmt.Errorf("unable to decode client cert data: %w", err)
 }
 clientCertKey, err := base64.StdEncoding.DecodeString(clientCertKeyData)
 if err != nil {
 return nil, "", fmt.Errorf("unable to decode client cert key data: %w", err)
 }
 serverRootCA, err := base64.StdEncoding.DecodeString(serverRootCAData)
 if err != nil {
 return nil, "", fmt.Errorf("unable to decode server root CA data: %w", err)
 }
 cert, err := tls.X509KeyPair(clientCert, clientCertKey)
 if err != nil {
 return nil, "", fmt.Errorf("unable to load cert and key pair: %w", err)
 }

 // Create Certpool to add the server root CA to the client pool
 certPool := x509.NewCertPool()
 certPool.AppendCertsFromPEM(serverRootCA)

 // Ensure no two workers have the same ID
 randString := fmt.Sprintf("%04x", rand.Intn(0x10000))
 // Embed the git commit of the currently running code into the worker ID
 commitHash := os.Getenv(vars.WorkerCommit)
 if len(commitHash) > 7 {
 commitHash = commitHash[:7]
 }
 workerID := fmt.Sprintf("%s-%s-%s", os.Getenv(vars.WorkerID), commitHash, randString)

 // Add the cert to the tls certificates in the ConnectionOptions of the Client
 clientOptions := client.Options{
 HostPort: hostPort,
 Namespace: namespace,
 ConnectionOptions: client.ConnectionOptions{
 TLS: &tls.Config{
 Certificates: []tls.Certificate{cert},
 RootCAs: certPool,
 },
 },
 Identity: workerID,
 }
 temporalClient, err := client.Dial(clientOptions)
 if err != nil {
 return nil, "", fmt.Errorf("unable to connect to Temporal: %w", err)
 }
 tClient = temporalClient
 return tClient, workerID, nil
}

Wrap Up

We’ve successfully deployed Temporal on a single node with Docker Compose and added TLS termination with NGINX.

In summary, we:

Introduced Temporal and set up a PostgreSQL-backed Temporal server using Docker Compose
Used Temporal’s auto-setup image to provision our databases
Switched to the Temporal production server
Deployed the Temporal Web UI and passed it through NGINX via a friendly HTTPS URL
Configured NGINX to handle TLS, forwarding web requests to the UI, thereby securing the traffic

This setup provides a starting point for working with Temporal’s powerful workflow capabilities. You can now write workflows in your language of choice (Go, Java, Python, etc.), point the SDK client to temporal.example.com:7233, and begin executing workflows on your server–just make sure your self-signed certificates are imported (see my go example above)!

Before we conclude, a few final notes and best practices:

Security: Your NGINX config can also restrict access by IP. If you intend to expose the Temporal UI or gRPC port more broadly, consider additional security layers, i.e. cloud infra-level firewalls
Persistence: Your temporal workflows are only as stable as your database. Again, I highly recommend using a stable, managed solution
Certificates: A time will come when your certificates expire. Be sure to have a rotation plan in place for scheduled maintenance!
Postgres: We initially granted sweeping access to the tuser in Postgres for our temporal and temporal_visibility databases. Now that the schema is set up, consider locking that down.

Temporal brings reliability superpowers to your applications – even in this modest single-node setup, you can develop and test workflows that will never give up in the face of failures. Now that you have Temporal running at temporal.example.com, it’s time to start orchestrating some workflows. Have fun exploring Temporal, and may your workflows always complete (eventually)! 🚀

Setting up a WireGuard VPN for Remote Access

tai@taigrr.com (Tai Groot) — Thu, 24 Mar 2022 18:19:25 +0600

What is WireGuard?

WireGuard is a VPN, which tunnels traffic over encrypted, secure UDP packet streams. Once a tunnel is established, the endpoints are allowed to roam (change IP addresses, such as might happen switching between WiFi networks or when connected to a cell tower). It is special when compared to other VPN technologies for many technical reasons, which led to its rapid adoption into the Linux Kernel, but three features in particular make WireGuard stand out: the CryptoKey Routing Table, the way public/private keypairs are handled, and the “silent by default” behavior.

According to the wireguard website,

WireGuard® is an extremely simple yet fast and modern VPN that utilizes state-of-the-art cryptography. It aims to be faster, simpler, leaner, and more useful than IPsec, while avoiding the massive headache. It intends to be considerably more performant than OpenVPN. WireGuard is designed as a general purpose VPN for running on embedded interfaces and super computers alike, fit for many different circumstances. Initially released for the Linux kernel, it is now cross-platform (Windows, macOS, BSD, iOS, Android) and widely deployable. It is currently under heavy development, but already it might be regarded as the most secure, easiest to use, and simplest VPN solution in the industry.

WireGuard Basics

Unlike traditional VPN technologies, WireGuard has no concept of Client or Server, but instead all nodes on the network are simply Peers, connecting Point-to-Point. This allows for some really interesting scenarios where packets sent through the VPN can be sent through very specific routes by setting IP masks appropriately, and if all Peers on the VPN have public IP addresses (or else they can ‘see’ each other in some other way, perhaps by sharing an insecure local network), the single point of failure built into the Hub-and-Spoke model becomes optional.

However, even if WireGuard doesn’t specifically recognize the Hub and Spoke model, it’s still a valid configuration,and we’ll be using it for this tutorial. Let’s see how easy it is to configure a Hub and Spoke-style, simple WireGuard VPN ready for personal or commercial use.

Server Instructions

Choose a private IP block to reserve. In order to use WireGuard inside of a Spoke and Hub paradigm, it’s necessary to reserve a block of IP addresses for all the nodes. (When running Peer to Peer, it’s not necessary for these addresses to share a subnet, as a CIDR of /32 can be used for each peer.) Private IP ranges include:
- 10.0.0.0 to 10.255.255.255
- 172.16.0.0 to 172.31.255.255
- 192.168.0.0 to 192.168.255.255
An example range for WireGuard might look like: 192.168.2.1-192.168.2.255 (be sure this range doesn’t conflict with that of your router). In CIDR notation, this range would be: 192.168.2.0/24. You can use a CIDR Calculator to determine the proper format for a given range.
Choose a port for your tunneled traffic to use. WireGuard traffic flows over UDP, and by default uses port 51820. For this tutorial, we will assume you are using the default port, but if you are running multiple WireGuard VPNs on the same server, you will need to choose a different port for each one.
Choose a cloud provider that provides you with a public IP address. My personal preference is Linode/Akamai (by using my signup link you’ll get free credits, as will I.)
Once you are ready to spin up a server, be sure to choose a distribution/flavor which ships Linux 5.6 or above. (Some versions of Ubuntu with older kernels also received backports, but if you have the option, go with the latest stable release available.) It is possible to use WireGuard in Userspace (if you don’t know what this means, don’t worry about it, just use a newer installation image) but it is much slower than the version running in the kernel. Going forward, we will assume you have the public IP of 93.184.216.34 (used for example.com).
Install WireGuard and its helper utilities. The name of the helper utilities package is usually wireguard-tools, and installing that package should bring in wireguard as well. If you don’t already have iptables installed, you will need to install that as well. (On Debian-based systems, run apt install wireguard-tools iptables.)
Enable IP forwarding. This is a necessary step for the server to act as a router. On Linux, this is done by running sysctl -w net.ipv4.ip_forward=1. To make this change permanent, edit or create /etc/sysctl.conf and add the line net.ipv4.ip_forward=1.
Create a keypair for your server. Once WireGuard is installed, you can generate a private key by running wg genkey. To get the public key, run wg pubkey and paste (or pipe) in the private key. This keypair is just like an SSH public/private keypair, so keep it safe!
Use the keypairs to create a configuration file. The next step is to create a configuration file for your server. There are three main pieces of information required:
- the interface and port to listen on
- the WireGuard private IP and netmask (CIDR Notation) you are committing to use (such as 192.168.2.1/24)
- the Private Key of your server
Create a file at /etc/wireguard/my-vpn.conf and add in the following:
```
[Interface]
ListenPort = 51820
PrivateKey = <SERVER PRIVATE KEY HERE> # <SERVER PUBLIC KEY HERE>
Address = 192.168.2.1/24
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
```
Many systems use eth0 as the name of the primary network interface, but if yours is different, you will need to change it in the PostUp and PostDown commands. If you are unsure of your default gateway’s interfacve name, run ip a and look for the interface with the public IP address you are using to connect to the server.

Additionally, you will add a block for each Peer, but we’ll need to generate those keypairs first. More on that later.
Be sure the connection port is exposed through your hosting provider (for UDP traffic). Instructions for Linode can be found here. You may need to consult your provider’s documentation for more information.
Before we start up our server, we need some more information from our clients. We’ll come back to the server later.

Client Instructions

Linux instructions:

Ensure your kernel has support for WireGuard, or install the userspace version. On Ubuntu/Debian-based systems, run apt install wireguard-tools
Create the folder /etc/wireguard/
Generate a new keypair the same way you did on the server: wg genkey
Place the keypair into a config file named my-vpn.conf and save it, along with the details you got from the server. The format is as follows:

[Interface]
PrivateKey = <CLIENT PRIVATE KEY HERE> # <CLIENT PUBLIC KEY HERE>
Address = 192.168.2.4/24 # Or whatever IP you want your client to use on the VPN
[Peer]
PublicKey = <SERVER PUBLIC KEY HERE>
AllowedIPs = 192.168.2.0/24 # This is the subnet you chose for your VPN, and should match the server's
Endpoint = 93.184.216.34:51820 # This is the public IP of your server, and the port you chose
# Note you can also use a domain name, if you want:
# Endpoint = example.com:51820
PersistentKeepalive = 25 # allow your client to roam without dropping the connection

chmod the config file and the /etc/wireguard directory to 400 and chown it to root:root.
On systemd-based systems, run systemctl enable wg-quick@my-vpn.
Move on to the server section, when complete move on to the next step.

Mac instructions:

Install Wireguard for Mac
Follow prompts to fill in the info from your server.
Choose an IP from the subnet you chose for your VPN.
Save your Public Key for later.
Enable start on demand.
Complete the server steps next, before hitting Save.

Final Server Instructions

Inside of your my-vpn.conf file, create a Peer block following the pattern of the previous entries:

[Peer] # name of client
PublicKey = <CLIENT PUBLIC KEY HERE>
AllowedIPs = 192.168.2.<n>/32 # replace `<n>` with the last octet of the Client's WireGuard private IP

Save and close the config file.
Ensure the config file is owned by root and has permissions of 400.
Run systemctl enable --now wg-quick@my-vpn to start the WireGuard tunnel and set it to start automatically on boot.
Continue setting up from the client side.

Linux

Same as for the server, run systemctl enable --now wg-quick@my-vpn

Validate the VPN is running with wg. You should see something like this:

interface: my-vpn
public key: <CLIENT PUBLIC KEY HERE>
private key: (hidden)
listening port: 58341
peer: <SERVER PUBLIC KEY HERE>
endpoint: 93.184.216.34:51820
allowed ips: 192.168.2.0/24
latest handshake: 13 seconds ago
transfer: 5.37 MiB received, 6.40 MiB sent
persistent keepalive: every 25 seconds

If you see an error, check the logs with journalctl -u wg-quick@my-vpn.

MacOS

Hit Save.
Optionally, in your Mac’s System Preferences, go to Network and check the box for Enable VPN on Demand.

Next Steps

You should now have a working WireGuard VPN, ready for roaming and personal use. There’s only one client for now, but you can add as many as you like by repeating the client steps on each new device. (Be sure to pick a unique private IP address inside of the subnet you chose for your VPN for each client.) Then, you can jump directly from one client to another using the WireGuard private IP addresses. This can be particularly useful when pairing the WireGuard app for Android with Termux or JuiceSSH to access your laptop or desktop on the go.

Once the client side setup is done, simply add a Peer block for each client in the server’s config file, using a unique private IP address, then run systemctl restart wg-quick@my-vpn on the server to restart the tunnel with the new configuration. It is normal for the tunnel to drop when you run the restart command, but it should come back up within a few seconds.

If you have any issues, check the logs with journalctl -u wg-quick@my-vpn. If you’re stumped, feel free to reach out to the community in my Discord, and someone may be able to help you out.

Going Further

If you want to take your WireGuard VPN to the next level, you can add a DNS server to your server’s config file, and then add a DNS entry to each client’s Interface block. You can read more about that here.

It’s possible to use WireGuard to route all of your traffic through the VPN, pair WireGuard with netns or Docker containers to force applications to use the VPN, or even use WireGuard to create a mesh network. I’ll likely be writing more about these topics in the future. For now, there are great resources available on the WireGuard website and on YouTube.

“WireGuard” and the “WireGuard” logo are registered trademarks of Jason A. Donenfeld.

Tutorials on cache cascade

Using PAM for SSH Login Telephony

Real-Time SSH Login Alerts Using PAM and Gotify

Prerequisites

Step 1: Install and Configure Gotify

Step 2: Create the PAM Notification Script

Step 3: Update PAM Configuration

Step 4: Enable and Test It

Appendix: PAM Environment Variables

Considerations

Conclusion

Temporal Server: Self-hosting a Production-Ready Instance

Overview of the Deployment

Prerequisites

Setting Up Our Docker Compose

Configuring Postgres

TLS certificates

Configuring Env Vars

Temporal Configuration

Spinning Up

Configuring NGINX as a TLS Reverse Proxy

Testing the Setup

Troubleshooting

Health Check

Creating a Test Workflow

Cleaning up Tests

Connecting a Client

Wrap Up

Setting up a WireGuard VPN for Remote Access

What is WireGuard?

WireGuard Basics

Server Instructions

Client Instructions

Linux instructions:

Mac instructions:

Final Server Instructions

Linux

MacOS

Next Steps

Going Further