Postprocess Local File Logs

The Local File Logging capability introduced in OpenLDAP 2.6 introduces a new header format for the log records. The entire header is a date-time-stamp in “epoch” format (hexadecimal numbers). The rest of the log information is unmodified.

The header is in two parts. They are separated by a period (“.”). The first is the date in “epoch” format and in hexadecimal.

The following awk script will process the entire log, “piping” it to the output pager with all of the headers converted:

#### /^[0-9a-f]+\.[0-9a-f]/{ split($1, stamp, ".") ts = "0x" stamp[1] "" us = "0x" stamp[2] "" tsn = strtonum( ts ) usn = strtonum( us ) sus = sprintf( "%0" (length( stamp[2] ) + 1) "d", usn ) $1 = strftime( "%Y-%m-%d %T", tsn ) "." sus } { print } #### 
...
  awk -f stamp.awk < slapd.log | less 

The command could have been:

  awk -f stamp.awk < slapd.log > newlog.log

to produce a copy with the date and time converted. The following python (2 or 3) snippet converts the date portion:

>>> from datetime import date
>>> # variable logline contains a line read in prior code
>>> splitline = logline.split(' ', 1)
>>> datepart = splitline[0].split('.', 1)
>>> intDatePart = int(datePart, 16)  # convert from hex string to int
>>> ts = date.fromtimestamp(int(intDatePart, 16))
>>> ts
datetime.date(2021, 11, 9)
>>> 

The time part can wait … it’ll be more of same