Comparison of Directory / LDAP Servers

A couple of weeks ago I began the exercise of investigation Directory Servers again.  This post provides some background on my previous experience, plus what I have found in my current investigation thus far.

The Past

I have some experience with Directory Servers from around 2001-2004, when I went through a very similar effort compared to what I am doing now, including a code evaluation of those that were open source.  At that time it really came down to whether or not it was appropriate for the company to use either LDAPd (now Apache DS) or OpenLDAP.  It came down to these two based primarily on the fact that they were more open or easier to install than some of the other contenders.  We looked at Sun Directory server, MS Active Directory, and Novell eDirectory, among others.  At the time the company company I worked for delivered a Windows based solution, but Active Directory just did not meet our needs, and the Sun Directory Server proved troublesome to install properly in our environment.  The Novell offering, while strong, relatively easy to install, and affordable (free for the first 1MM nodes at the time – we weren’t going to have that many) wasn’t the politically acceptable option and didn’t quite match up to our particular needs.

That left us with OpenLDAP and LDAPd, in part because they both supported database backends (we had a particular reason for requiring that).  Personally, I really liked LDAPd and offered a few code snippets to the project where I found some issues around the database backend modules.  I very much liked the idea of a true directory server that supported many different types of backends based on need, which I believe is what led Apache DS to be the encompassing project it is now.  The issue at that time was really more about maturity – was this project going to be able to support us moving forward?  On the other hand, OpenLDAP had history, a large install base, and unfortunately, seemed to have developers that honestly couldn’t have cared less about supporting the Windows environment.  On checking out the code, I found that it was impossible to build within Windows as it was.  That offered an experience to work in sussing through C conditional compile statements that I did not much relish, but never-the-less took on.  After some time fooling around with the code I did get it to compile, and was able to get the backend components to work with Oracle and MS SQL server as well.

While not thrilled about my Windows based experience with OpenLDAP, it was the winner.  It had the features we needed and it was solid and trusted.

The Situation Now

Now several years later I am back undertaking a similar effort at a different company and with a different set of requirements.  I am no longer constrained by the Windows environment as we run our entire environment in Linux.  I also don’t have the need (as far as I can tell right now) to support a database backend.  So, here we are again – OpenLDAP is still the same stalwart it has always been, Active Directory still doesn’t meet our needs (for different reasons), and I didn’t even bring up Novell this time (although I am still a fan).  The playing field has changed though – LDAPd is now ApacheDS, with a larger supporting cast, an increased scope, and a changed architecture.  The new kid on the block is OpenDS, which has come out of the Sun world, has had a 1.0 release this year, as well as some more recent 1.1 builds.  Given our platform and our requirements we decided to take a look at ApacheDS, OpenLDAP and OpenDS.  After some quick conversations with other folks out there, I decided to defer my test of ApacheDS and wait for some additional work to be done on the 1.5 branch to get some additional performance and “ready for prime-timeness” (I am happy to hear from the Apache DS folks if they believe I should reconsider).  So at the end of the day, I am off to install OpenDS and OpenLDAP ….

The Install

My test environment is CentOS, and I created two virtual servers on the same physical box with the exact same CPU and memory configurations.  The intent was to work with a more or less default install of each of the servers without performing any tuning, tweaking, or custom building.  CentOS always tends to be a few package versions back in their repositories, but I decided to use what was there for the sake of sticking with the rules above.  On the OpenDS side, I started with OpenDS 1.0, instead of the new builds in OpenDS 1.1.

From an install standpoint, OpenDS seemed quite a bit easier to me, even though I had never seen it before.  OpenLDAP was just exactly how I remembered it.  I did take the installs a bit further and went to set them up to use TLS for security.  Again, OpenDS was much easier to set this up with, I needed to do a lot more configuring with OpenLDAP and ended up deferring until later (we didn’t really need it for the test, or for our environmental set up in production either … for now.)

In terms of features, well again, LDAP is mostly LDAP.  However, one feature that I very much liked (and we need) from OpenDS was virtual attributes, specifically isMemberOf.  This is EXTREMELY useful to us, and the default install of OpenLDAP from the CentOS repos doesn’t have it.  To be fair, if I were to build the latest versions of OpenLDAP that are available, they do have the memberOf feature.  Based on the fact that we do not want to get into maintaining and building 3rd party code whenever possible, this ended up being a ‘+’ in the OpenDS column and a ‘-‘ in the OpenLDAP column.

The verdict, I installed OpenDS using both the GUI installer as well as the command line interface, and used command line interface for OpenLDAP.  All things considered, setting up one LDAP server feels much like setting up any other, it isn’t that hard.  In the end though, I just felt OpenDS was easier, and would certainly be more straightforward for someone without much backround in the technology .. and it had the nifty isMemberOf attribute we wanted.

Testing by Hand

First I just wanted to make sure that the servers did what they should do.  I should be able to use any LDAP client to add, remove, search, and modify LDAP entries.  Everything worked fine using both the command line tools, and mostly OK using available GUI LDAP clients.  My client tool of choice is the Apache Directory Manager client.  For the purposes of this initial set of tests we decided to go the simple route – we won’t have thousands of objects in the LDAP store, maybe one day, but now we are talking about hundreds.  I only created a handful of users, groups, and orgs for this test.    The only caveat I had in my testing with this client was that I had some very specific problems connecting to the OpenDS server using a startTLS connection from a Linux client.  I did not have the same issue with the Windows client, and have had some Linux clients that I have been able to “clear up”, but I can’t explain why yet (see incident DIRSTUDIO-414 at the ApacheDS project).  I did not have this same problem connecting to an ApacheDS server (I did set one up when I ran into this problem – very easy!), and I still haven’t set up a secure OpenLDAP server yet, so I can’t provide any feedback on that.

My next step was to write some sample code to exercise OpenLDAP and OpenDS a little.  Again, no problems with either, including a secure connections using TLS to the OpenDS server (which makes the problem above even more perplexing).  I put in a little data, updated it etc.

The last thing I did was to do some data exports / imports from OpenDS to OpenDS, OpenLDAP to OpenLDAP, OpenLDAP to OpenDS, and OpenDS to OpenLDAP.  The only thing that was unexpected here was that OpenDS was placing UUID information into the LDIF, which OpenLDAP was NOT HAPPY with (as it shouldn’t be).  Removing the UUID from the LDIF took care of the issue.  After that, everything was great.

Automated Testing

My initial instinct was to go for my good old trusted friend JMeter.  I did run a couple of tests through JMeter before I decided that it was a little more work than I wanted to take on, so I went looking for another tool.  That is when I ran across SLAMD.  I have been very happy with SLAMD thus far, even though I know I have only scratched the surface. Keeping in mind that I don’t have a ton of data in the stores, I headed off to run on of the LDAP Load Generator projects.  I created two – one for OpenDS and one for OpenLDAP.  These tests ran continuously over an 8 hour period with 2 client machines running 40 threads apiece, with the first 10 minutes and last 10 minutes of the 8 hours was used as a warm up and cool down time. Some graphs from those tests are included below.

The Verdict

While I have not spent a ton of time doing any analysis on these results, and certainly have not doen any tweaking to affect them in any way, it would appear that OpenDS has better performance than OpenLDAP in our simple tests.  With a few exceptions, the overall variability of performance with OpenLDAP seemed to vary a little more widely than the OpenDS performance, which suprised me a little.  There were some very higly variable points with the OpenDS server, but they are very high peaks spread out over what looks like very consistant intervals.  I would tend to interpret these as garbage collection since OpenDS is java based.  As I said, I have only begun to really investigate these results, so I welcome any other interpretations or experiences that other have and will continue to do some further investigation, run additional tests, and test out synchronization.