Diagnosing Performance Issues
Where to Start
We recommend that the logLevel setting in
the olcLogLevel setting in
cn=config be set to “STATS
SYNC”. This logs information about general server operations and
sync-repl activity. This is the level that gives you the
detail to analyze many kinds of problems related to the server. It is
very important to have this level of data to diagnose performance
We do not recommend higher levels of logging for normal operations. They bloat log file sizes and make diagnosing simple problems more time-consuming.
Log Sample Sizes
The logs from busy OpenLDAP directories are often very large. For diagnosing general performance issues, much smaller samples are just as useful. Slices of the log are easy to take. For example, you might take just the first 100,000 lines from the beginning of the current log with:
head -n 100000 <path>/slapd.log > <tmpfile>
With the smaller sample you can inspect the parameters to individual requests looking for things likely to be more costly to process.
Symas Log Analysis tools.
Symas maintains a project on https://gitlab.symas.net/mheyman/sylog that contains
python (2 or 3) programs that do simple log analysis. These programs are
a work in progress and there is no formal installer or Makefile yet.
However, two are worth using:
logreduce which produces a
simple summry of system performance and a couple of indicators of system
logload which prints out the number of
requests for common LDAP operations arrived every 11 seconds.
TODO usage TODO sample output
TODO usage TODO sample output
- Why indexing
- What to index
- What not to index
- Link to Symas Log Analyzer
- How to use Log Analyzer
- Interpreting Log Analyzer results
- Applying index updates
BDB/HDB CACHE SIZE
- How to determine if your BDB/HDB parameters need tuning
- You should just migrate to MDB
- If you can’t migrate to MDB, this will tell you how to tune BDB/HDB
- You really should migrate, though
- Are you using the right type of replication (standard vs. delta-sync) for your data?
FILESYSTEM TYPE & LOCATION
- For LMDB, ext2 is the standard, but jfs with an external journal is best for situations that require synchronous writes
- For other backends, see Backend/Filesystem Benchmarks
- Storage on a SAN: good or bad?
- Set appropriate log level
- Assess log rotation strategy
- Write log to separate physical disk
- Check for software (RDBMS, services) that competes for resources with OpenLDAP
- Are there cron jobs or scheduled tasks that are monopolizing system resources?
- Profile processor, memory, disk and network interface usage
- Look for excessive swapping
VIRTUAL MACHINE RESOURCES
- Are the VM and VM host properly configured?
- Are other guest VMs stealing CPU/RAM from OpenLDAP VM?
Do you really need every single entry and their objects in one query?
Do your query criteria search against non-indexed values?
- Are there other systems on the network that monopolize bandwidth?
- Are the replicas properly distributed across the network?