Enable huge pages in Linux
Enable huge pages
Using huge pages reduces overhead when ADPG/PostgreSQL utilizes large contiguous chunks of memory. In particular, this occurs when the shared_buffers parameter is set to a large value.
In ADPG, the default value of the huge_pages
configuration parameter is try
, but typically huge pages are not enabled in the operating system. To effectively use huge pages, enable them on operating system side. To do this, perform the steps below:
-
Check if a Linux kernel has the settings required for huge pages:
CONFIG_HUGETLBFS=y
andCONFIG_HUGETLB_PAGE=y
. Utilize the following command:$ grep HUGETLB /boot/config-$(uname -r)
The output should be similar to:
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y CONFIG_CGROUP_HUGETLB=y CONFIG_HUGETLBFS=y CONFIG_HUGETLB_PAGE=y
-
Check that currently huge pages are not in use. Obtain the information from the virtual memory subsystem:
$ cat /proc/sys/vm/nr_hugepages
Output:
0
Also, you can get information related to huge pages from /proc/meminfo — the file that reports statistics about memory usage on the system:
$ grep ^HugePages /proc/meminfo
Output:
HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0
-
Make sure that the
huge_pages
ADPG parameter value is set totry
, for example, using thepsql
tool:SHOW huge_pages;
Result:
huge_pages ------------ try
-
Find the number of huge pages to reserve.
Stop the cluster:
$ sudo systemctl stop adpg16-patroni.service
Get a huge page size used in the system:
$ grep ^Hugepagesize /proc/meminfo
Hugepagesize: 2048 kB
The runtime-computed PostgreSQL parameter
shared_memory_size_in_huge_pages
reports the number of huge pages required. This parameter value can be displayed with thepostgres
command./pg_data1/adpg16
is the path to postgresql.conf, it can be replaced by the$PGDATA
environment variable if it is set:$ postgres -D /pg_data1/adpg16 -C shared_memory_size_in_huge_pages
Output:
73
In this example, at least 73 huge pages are required, a larger setting would be appropriate if other programs also need huge pages.
The command below displays a number of subdirectories of the form
hugepages-<size>kB
, where<size>
is the size of the huge pages supported by the kernel/CPU combination.$ ls /sys/kernel/mm/hugepages
Output:
hugepages-1048576kB hugepages-2048kB
The default huge page size is 2 MB, but you can also explicitly request either 2 MB or 1 GB with the huge_page_size configuration parameter. To set this parameter, you can use the ADPG configurations section on the Primary configuration page of the ADPG service (see Configuration parameters). The number of pages calculated by
shared_memory_size_in_huge_pages
will be adapted to the page size. To determine the optimal size of huge pages, test different page sizes with your database. -
Set the
vm.nr_hugepages
kernel parameter that specifies the number of huge pages to reserve. You need to execute the following command on behalf of theroot
user:$ sudo su - $ echo 'vm.nr_hugepages = 73' >> /etc/sysctl.d/30-postgresql.conf
Load kernel parameters from all files:
$ sysctl -p --system
Check the result using the commands below:
$ grep ^HugePages /proc/meminfo
Output:
HugePages_Total: 73 HugePages_Free: 73 HugePages_Rsvd: 0 HugePages_Surp: 0
$ sudo sysctl -a | grep 'huge*page'
The output should contain the specified
vm.nr_hugepages
value:... vm.nr_hugepages = 73 ...
-
Restart ADPG. Press
y
andEnter
when needed to confirm the operation:$ sudo systemctl start adpg16-patroni.service $ /usr/lib/adpg16/usr/bin/patronictl -c /etc/adpg16-patroni/adpg16-patroni.yml restart a80bfefc-755f-4ccb-8614-989f0ddda0e7 test_adpg_cluster_eek1-adpg-adpg3
Where:
-
a80bfefc-755f-4ccb-8614-989f0ddda0e7
is a cluster UUID. You can find it in ADCM UI on the cluster Configuration tab. -
test_adpg_cluster_eek1-adpg-adpg3
is a patroni name of the current node. You can determine its name from the output of thelist
patroni command:/usr/lib/adpg16/usr/bin/patronictl -c /etc/adpg16-patroni/adpg16-patroni.yml list
.
-
See how the statistics changed:
$ grep ^HugePages /proc/meminfo
Output:
HugePages_Total: 73 HugePages_Free: 63 HugePages_Rsvd: 63 HugePages_Surp: 0
Calculate the required number of huge pages on a running cluster
The optimal size and number of huge pages are determined as a result of testing, and if the first calculation method described above does not suit your needs, you can try to calculate the number of huge pages on a running cluster. The results obtained also need to be tested.
-
Determine pid of the postmaster process:
$ sudo head -1 /pg_data1/adpg16/postmaster.pid
Output:
895
-
Get the peak memory usage of the postmaster process using pid:
$ grep ^VmPeak /proc/895/status
Output:
VmPeak: 403440 kB
-
Calculate the number of huge pages as
VmPeak
divided by the huge page size plus1
—403440/2048+1
. In this example, the required number of huge pages is 197.