Diagnosing Performance Issues
Where to Start
OpenLDAP Logging
We recommend that the logLevel setting in slapd.conf
or
the olcLogLevel setting in cn=config
be set to “STATS
SYNC”. This logs information about general server operations and
sync-repl
activity. This is the level that gives you the
detail to analyze many kinds of problems related to the server. It is
very important to have this level of data to diagnose performance
problems.
We do not recommend higher levels of logging for normal operations. They bloat log file sizes and make diagnosing simple problems more time-consuming.
Log Sample Sizes
The logs from busy OpenLDAP directories are often very large. For diagnosing general performance issues, much smaller samples are just as useful. Slices of the log are easy to take. For example, you might take just the first 100,000 lines from the beginning of the current log with:
head -n 100000 <path>/slapd.log > <tmpfile>
With the smaller sample you can inspect the parameters to individual requests looking for things likely to be more costly to process.
Symas Log Analysis tools.
Symas maintains a project on https://gitlab.symas.net/mheyman/sylog that contains
python (2 or 3) programs that do simple log analysis. These programs are
a work in progress and there is no formal installer or Makefile yet.
However, two are worth using: logreduce
which produces a
simple summry of system performance and a couple of indicators of system
problems and logload
which prints out the number of
requests for common LDAP operations arrived every 11 seconds.
logreduce
Overview
TODO usage
TODO sample output
logload
Overview
TODO usage
TODO sample output
INTERNAL FACTORS
INDEXING
- Why indexing
- What to index
- What not to index
- Link to Symas Log Analyzer
- How to use Log Analyzer
- Interpreting Log Analyzer results
- Applying index updates
BDB/HDB CACHE SIZE
- How to determine if your BDB/HDB parameters need tuning
- You should just migrate to MDB
- If you can’t migrate to MDB, this will tell you how to tune BDB/HDB
- You really should migrate, though
REPLICATION TYPE
- Are you using the right type of replication (standard vs. delta-sync) for your data?
FILESYSTEM TYPE & LOCATION
- For LMDB, ext2 is the standard, but jfs with an external journal is best for situations that require synchronous writes
- For other backends, see Backend/Filesystem Benchmarks
- Storage on a SAN: good or bad?
LOGGING
- Set appropriate log level
- Assess log rotation strategy
- Write log to separate physical disk
COMPETING SOFTWARE
- Check for software (RDBMS, services) that competes for resources with OpenLDAP
- Are there cron jobs or scheduled tasks that are monopolizing system resources?
INADEQUATE HARDWARE
- Profile processor, memory, disk and network interface usage
- Look for excessive swapping
VIRTUAL MACHINE RESOURCES
- Are the VM and VM host properly configured?
- Are other guest VMs stealing CPU/RAM from OpenLDAP VM?
EXTERNAL FACTORS
INEFFICIENT QUERIES
Do you really need every single entry and their objects in one query?
Do your query criteria search against non-indexed values?
NETWORK ISSUES
- Are there other systems on the network that monopolize bandwidth?
- Are the replicas properly distributed across the network?