Symas OpenLDAP Knowledge Base

Diagnosing Performance Issues

Where to Start

OpenLDAP Logging

We recommend that the logLevel setting in slapd.conf or the olcLogLevel setting in cn=config be set to “STATS SYNC”. This logs information about general server operations and sync-repl activity. This is the level that gives you the detail to analyze many kinds of problems related to the server. It is very important to have this level of data to diagnose performance problems.

We do not recommend higher levels of logging for normal operations. They bloat log file sizes and make diagnosing simple problems more time-consuming.

Log Sample Sizes

The logs from busy OpenLDAP directories are often very large. For diagnosing general performance issues, much smaller samples are just as useful. Slices of the log are easy to take. For example, you might take just the first 100,000 lines from the beginning of the current log with:

        head -n 100000 <path>/slapd.log > <tmpfile>

With the smaller sample you can inspect the parameters to individual requests looking for things likely to be more costly to process.

Symas Log Analysis tools.

Symas maintains a project on https://gitlab.symas.net/mheyman/sylog that contains python (2 or 3) programs that do simple log analysis. These programs are a work in progress and there is no formal installer or Makefile yet. However, two are worth using: logreduce which produces a simple summry of system performance and a couple of indicators of system problems and logload which prints out the number of requests for common LDAP operations arrived every 11 seconds.

logreduce Overview

TODO usage

TODO sample output

logload Overview

TODO usage

TODO sample output

INTERNAL FACTORS

INDEXING

  • Why indexing
  • What to index  
  • What not to index
  • Link to Symas Log Analyzer
  • How to use Log Analyzer
  • Interpreting Log Analyzer results
  • Applying index updates 

BDB/HDB CACHE SIZE

  • How to determine if your BDB/HDB parameters need tuning
  • You should just migrate to MDB
  • If you can’t migrate to MDB,  this will tell you how to tune BDB/HDB
  • You really should migrate, though 

REPLICATION TYPE

  • Are you using the right type of  replication (standard vs. delta-sync) for your data?

FILESYSTEM TYPE & LOCATION

  • For LMDB, ext2 is the standard, but jfs with an external journal is best for situations that require synchronous writes
  • For other backends, see Backend/Filesystem Benchmarks 
  • Storage on a SAN: good or bad? 

LOGGING

  • Set appropriate log level
  • Assess log rotation strategy
  • Write log to separate physical disk 

COMPETING SOFTWARE

  • Check for software (RDBMS, services) that competes for resources with OpenLDAP
  • Are there cron jobs or scheduled tasks that are monopolizing system resources? 

INADEQUATE HARDWARE

  • Profile processor, memory, disk and network interface usage
  • Look for excessive swapping 

VIRTUAL MACHINE RESOURCES

  • Are the VM and VM host properly configured?
  • Are other guest VMs stealing CPU/RAM from OpenLDAP VM? 

EXTERNAL FACTORS

INEFFICIENT QUERIES

  • Do you really need every single entry and their objects in one query?

  • Do your query criteria search against non-indexed values? 

NETWORK ISSUES

  • Are there other systems on the network that monopolize bandwidth?
  • Are the replicas properly distributed across the network? 

 

Solution