- ldapdiff 
- (c) 2000-2005 Thomas.Reith@rhoen.de 
Fri Nov 18 22:39:58 CET 2005

Sections
0. intro
1. compile ldapdiff
2. commandline options
3. config file options
4. return values
5. misc
6. thanks

0. intro

ldapdiff combines classical "diff" and "patch" functionality in one 
application. the difference is, that ldapdiff is not designed for use 
on flat ascii files, it is designed for "patching" ldap directories 
using ldif files.

many people using ldap based directories have problems updating their 
ldap directory automatically, because the only offline interface between 
the company database and the ldap directory are ldif files. 

normally the primary instance which stores most of a company's data, 
is a relational database. since there are normally no ldap servers available, 
which request their information directly from a relational database, it
is difficult, to keep a ldap directory up to date. one could create 
"ldif formatted deltas" for every database change, but it is time 
consuming and difficult. with no access to the database application and
schema, it is nearly impossible. much easier is it, to implement a procedure, 
which dumps the whole data pool into a ldif formatted file and run ldapdiff.

with ldapdiff it is possible to check every entry/attribute of an ldif file 
against a running ldap directory. ldapdiff is able to produce ldif delta 
files, which can be fed, into standard tools like "ldapmodify", "ldapdelete" etc. 
ldapdiff is also able to modify, delete and add entries/attributes online.

the main idea of ldapdiff, is not to synchronize two different ldap
directories. this can be done much better with internal replication 
tools (e.g. slurpd). ldapdiff's main operational domain are updates 
of user, groups etc.organized in ldap "ou" branches.

1. compile ldapdiff

run ./configure --with-ldap-dir=[your ldap SDK]

[required] ldap sdk (recommended openldap) from ftp://ftp.openldap.org
[optional] libiconv is located at ftp://ftp.gnu.org.

libiconv has been integrated into glibc. on
modern linux distributions the --with-iconv-dir option
should not be necessary. if you run into compiler or linker
problems like:

...
ldapiconv.c:29: iconv.h: No such file or directory
make: *** [ldapiconv.o] Error 1
...
/tmp/ldapdiff-0.9.1/ldapiconv.c:39: undefined reference to `libiconv_open'
ldapiconv.o: In function `ldificonv':
/tmp/ldapdiff-0.9.1/ldapiconv.c:56: undefined reference to `libiconv'
ldapiconv.o: In function `ldificonvclose':
/tmp/ldapdiff-0.9.1/ldapiconv.c:68: undefined reference to `libiconv_close'
collect2: ld returned 1 exit status
...

your glibc/libc doesn't include the iconv functions and you should use
"configure --with-iconv-dir=[your iconv SDK] --with-ldap-dir=[your ldap SDK]"
instead

if you run into compile problems with the new ldap API (e.g. undefined reference to `ldap_initialize`) try setting the environment variable "CPPFLAGS=-DUSE_DEPRECATED_API"

2. commandline options

most of the options are stored in the file "ldapdiff.conf", but there
are still some commandline options:

-c            : name of the config file
                default: ./ldapdiff.conf

-f ldiffile   : ldiffile is a ldif formatted file, which will be compared
                with the running ldap server. 
                "changetypes" lines will not work
                if this option is not used, stdin will be used

-l logfile    : name of the logfile
                default: stdout

-p profile    : profile is specified in ldapdiff.conf. profiles are all other 
                sections not named [global]

-s local[0-7] : use syslog, example for /etc/syslog.conf
                "local4.*\t/var/log/ldapdiff.log"

-v            : print version and exit

3. config file options

the ldapdiff config file (normally ./ldapdiff.conf) is
structured into sections and variables. every mentioned variable
has to be set. there are no optional variables, yet. 

3.1 section [global]

ldaphost:         hostname, where the ldapserver is running
ldapport:         port, where the ldapserver is running
rootdn:           rootdn of the ldap directory
rootpw:           cleartext password of rootdn (will be improved by upcoming 
                  releases)
modfile:          delta file for found differences in existing 
                  entries/attributes
addfile:          delta file for new entries, which doesn't exist in the ldap 
                  server
onlineupdate:     if "yes", ldapdiff tries to update the ldapserver directly
                  (ATTENTION ATTENTION ATTENTION) 
                  especially take care in combination with the option 
                  "deleteentry: yes"  and "deleteattribute: yes"
offlineupdate:    if "yes", ldapdiff creates "modfile" and "addfile" with the
                  differences in ldif format.
onlineerrfatal:   if "yes", every error updating the ldap server online
                  causes ldapdiff to exit. it can be useful to set this 
                  parameter to "no", if there are a lot of entries to
                  check and ldapdiff should ignore these errors
iconv:            if "yes" libiconv is used for conversion between different
                  charset
ldifcharset:      name of the source charset (eg. "ISO-8859-3" for western 
                  europe). see (iconv_open 3) for the available charsets
ldapcharset:      name of the ldap charset (recommendet "UTF-8")
                  see (iconv_open 3) for the available charsets
schemacheck:      if "yes" all attributetypes will be requested from
                  the ldapserver (works with openldap). alias checking avoids 
                  that ldapdiff gets confused with the ambiguity
                  of identical attribute names like "commonName" and "cn" etc.
schemabase:       the basedn of the ldapserver to query the schema.
                  openldap: "cn=subschema"
schemafilter:     the filter for querying schema objectclasses
                  openldap: "objectClass=*"
schemaattribute:  schema attribute to query 
                  openldap: "attributetypes"

3.2 section [profile]

since there are a lot of different objectclasses in a running ldap server, 
ldapdiff needs to know what kind of objectclasses have to be compared. this 
can be configured with the "group" parameter. ldapdiff needs a unique attribute
for every group, which can be configured with "filter". ("dn" can't be used)

ldapdiff real life example:
for me, it is only of interest to update user and groups exported from SAP. 
all other entries are stored with java, php or other graphical frontends directly in
the ldap server. the SAP database dumps two ldif formatted files every 
hour (4000 persons and 60 groups). 
the first file contains every person with attributes like name, uid, gid,
password etc. the second file contains the groups with the uniqueMember 
attributes. the groups are used from apache, squid, cyrus imap and various
other applications. I have configured two profiles in "ldapdiff.conf": [user]
and [group].


basedn:           ldapdiff uses the library function ldap_search with the
                  scope LDAP_SCOPE_SUBTREE. this mean that ldapdiff
                  searches recursively from the beginning of "basedn"
                  for objects. in large ldap directories, it's a good
                  idea to store ldapdiff's searchbase in a separate "ou"
                  for example "ou=pople,dc=webtomware,dc=com"
                  moves from one "ou" to another ldapdiff has to be run twice!!!
filter:           the filter is the unique attribute, which ldapdiff uses to 
                  search for. for example "uidNumber" or "cn"
                  hint: use an "eq" index (openldap) for this attribute to 
                  boost performance
                  DON'T USE "DN", WHICH DOESN'T WORK!!!
group:            the grouping filter will be associated by logical "AND" to 
                  the filter above, to match only the objectclass you want to 
                  update. use an attribute, which is unique for the
                  group of entries you want to match, for example 
                  "objectClass=posixAccount" or "objectClass=groupOfUniqueNames"
ignore:           a comma seperated list of attributes, which will be ignored
                  by ldapdiff. "none" if no attributes have to be ignored
ignoreval:        a comma seperated list of attribute values, which will be
                  ignored by ldapdiff. "none" if no attributes have to be
                  ignored.  This is useful if you are merging more than one
                  profile into a single ldap server.
noequality:       if "schemacheck" is disabled, this attribute tells 
                  ldapdiff in a comma separated list, which ldap attributes
                  are not defined with an "EQUALITY" function (e.g. jpegPhoto,
                  facsimilieTelephoneNumber etc.)
                  "none", if there are no such attributes in the ldif source.
                  this option is necessary, because ldap attributes with no 
                  "EUQALITY" method cannot be replaced. they have to 
                  be deleted and then added again.
mapalias:         "alias=realname, ..." 
                  there are some attributetypes, which are aliasnames, like
                  fax and facsimileTelephoneNumber, since openldap stores
                  attributes with it's real name, ldapdiff becomes
                  confused. therefore a parameter "fax=facsimileTelephoneNumber"
                  can be configured, which tells ldapdiff to use the real
                  name for further operations.
                  if "schemacheck" is enabled this attribute will be ignored,
                  because this information will be retrieved from the schema
                  automatically.
addentry:         if "yes" ldapdiff tries to add every entry (dn), which
                  matches the "group" filter and cannot be found in the ldap
                  server. (This matches the behavior of ldapdiff prior to
                  this version.) If "no", ldapdiff will not add an entry
                  (dn) that is in the ldif file but not present in the ldap
                  server. This is useful if you are merging more than one
                  ldif file into a single ldap server.
deleteentry:      if "yes" ldapdiff tries to delete every entry (dn), which 
                  matches the "group" filter and cannot be found in the ldif
                  file.
                  (ATTENTION ATTENTION ATTENTION) 
                  especially take care in combination with the option 
                  "onlineupdate: yes"!!!
deleteattribute:  if "yes" ldapdiff tries to delete an attribute from an
                  entry, which is listed in the ldap server, and cannot be
                  found in the ldif file.  This option will also accept a
                  comma seperated list of attributes that ldapdiff can
                  delete.  This is useful if you are merging more than one
                  LDIF file into a single ldap server.
                  (ATTENTION ATTENTION ATTENTION) 
                  especially take care in combination with the option 
                  "onlineupdate: yes"!!!
modifybydeladd:   if "yes" ldapdiff doesn't try to modify the attributes of the
                  ldap entry. ldapdiff simply deletes the different entry 
                  completely and adds the one from the ldif file. this feature
                  can be very useful, if there are differences in attributes 
                  like "objectClass". 
                  if you don't use "onlineupdate: yes" run ldapmodify 
                  (ldapdiff.mod) before ldapadd (ldapdiff.add)
                  (btw. this method is nearly 15% slower, then attribute 
                  replacement)

4. return values

0: no errors
1: an error occured
2: an online modification error (insert,update,delete) occured 

5. misc

5.1. handling of empty attributes

empty attributes cannot be stored in the ldap server. therefore, they
will be ignored from ldapdiff with a warning message shown in loglevel 2

...
Thu Feb 24 20:43:34 2005: ldifread: warning, empty attribute in line 13
...

6. thanks

Tzafrir Cohen 
- for contributing the rpm spec file ldapdiff.spec
Jeff Waugh
- for creating the man page and maintaing ldapdiff for gnu debian linux
David Goncalves
- for the idea, that "delete and add" of the whole ldap entry can be useful 
  for modifying different attributes
Tod Thomas
- for sending a patch, if there are "empty" attributes in the ldap server
Steven Premeau
- for adding the features "addentry" and "ignoreval" and extending 
  "deleteattribute"
Rodney Hee
- for the hint to make dn comparison case insensitive
