LMDB Database File Sizes and Memory Utilization
This article provides insights into storage requirements for OpenLDAP mdb databases and how to monitor active database storage requirements
Table of Contents
Date: 11-04-2022
Introduction
LMDB (the Lightning Memory-mapped Data Base) is an “in-memory database.” In OpenLDAP, LMDB is the database engine under the “mdb” database type. This is the only currently supported database type used in OpenLDAP.
All of the contents of the database are loaded into Virtual Memory. As much of the contents are kept in the server's memory as possible. This is why, for performance, server memory size must be large enough to ensure the LMDB data stays in memory.
Setting the size of the mdb database
The size is set in the configuration for the server. It is specified in bytes. For servers using slapd.conf for configuration information the default looks like:
maxsize 1073741824
# 1 gigabyte
For servers using dynamic configuration the default looks like:
dn: cn=module{0},cn=config
changetype: modify
add: maxsize
maxsize: 1073741824
-
When the server (slapd) starts up it looks for its database(s). If they are not where they are supposed to be, slapd allocates the space and initializes them. slapd writes a file as long as the number of bytes specified in the configuration! Even if it is empty, it takes up that much space on the storage device.
Operating systems utilities do not understand memory-mapped files. The mdb database is a memory mapped file. So space usage reporting is often misleading. Remember, the file size is what was specified.
Finding the size of the data in the mdb database
OpenLDAP provides the mdb_stat utility to tell you about your mdb data. The simple version of the mdb_stat command looks like:
mdb_stat -f /var/symas/openldap-data/symas/
mdb_stat produces a report that looks like:
root@symas-ldap0:~# mdb_stat -f /var/symas/openldap-data/symas/
Freelist Status
Tree depth: 1
Branch pages: 0
Leaf pages: 1
Overflow pages: 0
Entries: 26
Free pages: 179
Status of Main DB
Tree depth: 1
Branch pages: 0
Leaf pages: 1
Overflow pages: 0
Entries: 24
To calculate the amount of space containing active data, we multiply the number of active pages times 4K (mdb uses 4k pages to hold data.). In the example above there is {TODO}kb of data.
How much room is left in the mdb database
The repost shows you how many pages are in the “free list”. They are not in use and are available for temporary use (updates create copies while slapd works and frees them when the operation is done) and growth. Because slapd needs free pages for updates, the size of the free list is not a simple indication of how much more space “you have”.
First, calculate the percentage of the space that is free. To do that, divide the number of free pages ({TODO}) by the sum of the pages in use and in the free list ({TODO} + {TODO}). From the example we come up with {TODO}% free.
You should generally have at least 30%{TODO _ CHECK???} free for ongoing operations. When the database uses more than that, performance may be impacted during busy times or, worse yet, the database could “run out of space” which generally causes an outage.
Making an mdb database bigger
If using slapd.conf, increase the “maxsize”. Once the changes have been restart the ‘slapd’ service.
maxsize 10737418240
# 10 gigabyte
If using a dynamic configuration, increasing the database size would look like this:
dn: cn=module{0},cn=config
changetype: modify
replace: maxsize
maxsize: 10737418240
-