This document describes the internals of PostgreSQL for database administrators and system developers.

It was 5:17pm today, just as I was wrapping up work for the day, and my manager pinged me with the following chat: <manager>: Hi Jeremy - we have a <other team> ticket - escalated to <leader>, <leader>, etc. <principal> is on trying to advise as well. Are you available this evening if needed for…

If you run Linux in production for any significant amount of time, you have likely run into the "Linux Assassin" that is, the OOM (out-of-memory) killer. When Linux detects that the system is using too much memory, it will identify processes for termination and, well, assassinate them. The OOM killer has a noble role in ensuring a system does not run out of memory, but this can lead to unintended consequences.

For years the PostgreSQL community has made recommendations on how to set up Linux systems to keep the Linux Assassin away from PostgreSQL processes, which I will describe below. These recommendations carried forward from bare metal machines to virtual machines, but what about containers and Kubernetes?

Below is an explanation of experiments and observations I've made on how the Linux Assassin works in conjunction with containers and Kubernetes, and methods to keep it away from PostgreSQL clusters in your environment.

Tutorial and documentation

Crunchy Hardened PostgreSQL is a state-of-the-art Postgres solution for security and compliance-focused enterprises. Crunchy Hardened PostgreSQL extends Crunchy PostgreSQL in order to enforce additional security controls to protect data from unauthorized access and comply with regulatory and privacy requirements. Crunchy Hardened Postgres combines advanced security features including enhanced RBAC, TDE, and superuser lockdown with commercial support requirements like High Availability, Disaster Recovery, and Certification.

All available containers for crunchydata postgres operator

Patroni can use Kubernetes objects in order to store the state of the cluster and manage the leader key. That makes it capable of operating Postgres in Kubernetes environment without any consistency store, namely, one doesn't need to run an extra Etcd deployment. There are two different type of Kubernetes objects Patroni can use to store the leader and the configuration keys, they are configured with the kubernetes.use_endpoints or PATRONI_KUBERNETES_USE_ENDPOINTS environment variable.