Enable huge pages in Linux

Enable huge pages

Using huge pages reduces overhead when ADPG/PostgreSQL utilizes large contiguous chunks of memory. In particular, this occurs when the shared_buffers parameter is set to a large value. In ADPG, the default value of the huge_pages configuration parameter is try, but typically huge pages are not enabled in the operating system. To effectively use huge pages, enable them on operating system side. To do this, perform the steps below:

  1. Check if a Linux kernel has the settings required for huge pages: CONFIG_HUGETLBFS=y and CONFIG_HUGETLB_PAGE=y. Utilize the following command:

    $ grep HUGETLB /boot/config-$(uname -r)

    The output should be similar to:

    CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
    CONFIG_CGROUP_HUGETLB=y
    CONFIG_HUGETLBFS=y
    CONFIG_HUGETLB_PAGE=y
  2. Check that currently huge pages are not in use. Obtain the information from the virtual memory subsystem:

    $ cat /proc/sys/vm/nr_hugepages

    Output:

    0

    Also, you can get information related to huge pages from /proc/meminfo — the file that reports statistics about memory usage on the system:

    $ grep ^HugePages /proc/meminfo

    Output:

    HugePages_Total:       0
    HugePages_Free:        0
    HugePages_Rsvd:        0
    HugePages_Surp:        0
  3. Make sure that the huge_pages ADPG parameter value is set to try, for example, using the psql tool:

    SHOW huge_pages;

    Result:

     huge_pages
    ------------
     try
  4. Find the number of huge pages to reserve.

    Stop the cluster:

    $ sudo systemctl stop adpg16-patroni.service

    Get a huge page size used in the system:

    $ grep ^Hugepagesize /proc/meminfo
    Hugepagesize:       2048 kB

    The runtime-computed PostgreSQL parameter shared_memory_size_in_huge_pages reports the number of huge pages required. This parameter value can be displayed with the postgres command. /pg_data1/adpg16 is the path to postgresql.conf, it can be replaced by the $PGDATA environment variable if it is set:

    $ postgres -D /pg_data1/adpg16 -C shared_memory_size_in_huge_pages

    Output:

    73

    In this example, at least 73 huge pages are required, a larger setting would be appropriate if other programs also need huge pages.

    The command below displays a number of subdirectories of the form hugepages-<size>kB, where <size> is the size of the huge pages supported by the kernel/CPU combination.

    $ ls /sys/kernel/mm/hugepages

    Output:

    hugepages-1048576kB  hugepages-2048kB

    The default huge page size is 2 MB, but you can also explicitly request either 2 MB or 1 GB with the huge_page_size configuration parameter. To set this parameter, you can use the ADPG configurations section on the Primary configuration page of the ADPG service (see Configuration parameters). The number of pages calculated by shared_memory_size_in_huge_pages will be adapted to the page size. To determine the optimal size of huge pages, test different page sizes with your database.

  5. Set the vm.nr_hugepages kernel parameter that specifies the number of huge pages to reserve. You need to execute the following command on behalf of the root user:

    $ sudo su -
    
    $ echo 'vm.nr_hugepages = 73' >> /etc/sysctl.d/30-postgresql.conf

    Load kernel parameters from all files:

    $ sysctl -p --system

    Check the result using the commands below:

    $ grep ^HugePages /proc/meminfo

    Output:

    HugePages_Total:      73
    HugePages_Free:       73
    HugePages_Rsvd:        0
    HugePages_Surp:        0
    $ sudo sysctl -a | grep 'huge*page'

    The output should contain the specified vm.nr_hugepages value:

    ...
    vm.nr_hugepages = 73
    ...
  6. Restart ADPG. Press y and Enter when needed to confirm the operation:

    $ sudo systemctl start adpg16-patroni.service
    
    $ /usr/lib/adpg16/usr/bin/patronictl -c /etc/adpg16-patroni/adpg16-patroni.yml restart a80bfefc-755f-4ccb-8614-989f0ddda0e7 test_adpg_cluster_eek1-adpg-adpg3

    Where:

    • a80bfefc-755f-4ccb-8614-989f0ddda0e7 is a cluster UUID. You can find it in ADCM UI on the cluster Configuration tab.

    • test_adpg_cluster_eek1-adpg-adpg3 is a patroni name of the current node. You can determine its name from the output of the list patroni command: /usr/lib/adpg16/usr/bin/patronictl -c /etc/adpg16-patroni/adpg16-patroni.yml list.

       

See how the statistics changed:

$ grep ^HugePages /proc/meminfo

Output:

HugePages_Total:      73
HugePages_Free:       63
HugePages_Rsvd:       63
HugePages_Surp:        0

Calculate the required number of huge pages on a running cluster

The optimal size and number of huge pages are determined as a result of testing, and if the first calculation method described above does not suit your needs, you can try to calculate the number of huge pages on a running cluster. The results obtained also need to be tested.

  1. Determine pid of the postmaster process:

    $ sudo head -1 /pg_data1/adpg16/postmaster.pid

    Output:

    895
  2. Get the peak memory usage of the postmaster process using pid:

    $ grep ^VmPeak /proc/895/status

    Output:

    VmPeak:   403440 kB
  3. Calculate the number of huge pages as VmPeak divided by the huge page size plus 1 — 403440/2048+1. In this example, the required number of huge pages is 197.

Found a mistake? Seleсt text and press Ctrl+Enter to report it