Comparison of Directory / LDAP Servers

A couple of weeks ago I began the exercise of investigation Directory Servers again.  This post provides some background on my previous experience, plus what I have found in my current investigation thus far.

The Past

I have some experience with Directory Servers from around 2001-2004, when I went through a very similar effort compared to what I am doing now, including a code evaluation of those that were open source.  At that time it really came down to whether or not it was appropriate for the company to use either LDAPd (now Apache DS) or OpenLDAP.  It came down to these two based primarily on the fact that they were more open or easier to install than some of the other contenders.  We looked at Sun Directory server, MS Active Directory, and Novell eDirectory, among others.  At the time the company company I worked for delivered a Windows based solution, but Active Directory just did not meet our needs, and the Sun Directory Server proved troublesome to install properly in our environment.  The Novell offering, while strong, relatively easy to install, and affordable (free for the first 1MM nodes at the time – we weren’t going to have that many) wasn’t the politically acceptable option and didn’t quite match up to our particular needs.

That left us with OpenLDAP and LDAPd, in part because they both supported database backends (we had a particular reason for requiring that).  Personally, I really liked LDAPd and offered a few code snippets to the project where I found some issues around the database backend modules.  I very much liked the idea of a true directory server that supported many different types of backends based on need, which I believe is what led Apache DS to be the encompassing project it is now.  The issue at that time was really more about maturity – was this project going to be able to support us moving forward?  On the other hand, OpenLDAP had history, a large install base, and unfortunately, seemed to have developers that honestly couldn’t have cared less about supporting the Windows environment.  On checking out the code, I found that it was impossible to build within Windows as it was.  That offered an experience to work in sussing through C conditional compile statements that I did not much relish, but never-the-less took on.  After some time fooling around with the code I did get it to compile, and was able to get the backend components to work with Oracle and MS SQL server as well.

While not thrilled about my Windows based experience with OpenLDAP, it was the winner.  It had the features we needed and it was solid and trusted.

The Situation Now

Now several years later I am back undertaking a similar effort at a different company and with a different set of requirements.  I am no longer constrained by the Windows environment as we run our entire environment in Linux.  I also don’t have the need (as far as I can tell right now) to support a database backend.  So, here we are again – OpenLDAP is still the same stalwart it has always been, Active Directory still doesn’t meet our needs (for different reasons), and I didn’t even bring up Novell this time (although I am still a fan).  The playing field has changed though – LDAPd is now ApacheDS, with a larger supporting cast, an increased scope, and a changed architecture.  The new kid on the block is OpenDS, which has come out of the Sun world, has had a 1.0 release this year, as well as some more recent 1.1 builds.  Given our platform and our requirements we decided to take a look at ApacheDS, OpenLDAP and OpenDS.  After some quick conversations with other folks out there, I decided to defer my test of ApacheDS and wait for some additional work to be done on the 1.5 branch to get some additional performance and “ready for prime-timeness” (I am happy to hear from the Apache DS folks if they believe I should reconsider).  So at the end of the day, I am off to install OpenDS and OpenLDAP ….

The Install

My test environment is CentOS, and I created two virtual servers on the same physical box with the exact same CPU and memory configurations.  The intent was to work with a more or less default install of each of the servers without performing any tuning, tweaking, or custom building.  CentOS always tends to be a few package versions back in their repositories, but I decided to use what was there for the sake of sticking with the rules above.  On the OpenDS side, I started with OpenDS 1.0, instead of the new builds in OpenDS 1.1.

From an install standpoint, OpenDS seemed quite a bit easier to me, even though I had never seen it before.  OpenLDAP was just exactly how I remembered it.  I did take the installs a bit further and went to set them up to use TLS for security.  Again, OpenDS was much easier to set this up with, I needed to do a lot more configuring with OpenLDAP and ended up deferring until later (we didn’t really need it for the test, or for our environmental set up in production either … for now.)

In terms of features, well again, LDAP is mostly LDAP.  However, one feature that I very much liked (and we need) from OpenDS was virtual attributes, specifically isMemberOf.  This is EXTREMELY useful to us, and the default install of OpenLDAP from the CentOS repos doesn’t have it.  To be fair, if I were to build the latest versions of OpenLDAP that are available, they do have the memberOf feature.  Based on the fact that we do not want to get into maintaining and building 3rd party code whenever possible, this ended up being a ‘+’ in the OpenDS column and a ‘-‘ in the OpenLDAP column.

The verdict, I installed OpenDS using both the GUI installer as well as the command line interface, and used command line interface for OpenLDAP.  All things considered, setting up one LDAP server feels much like setting up any other, it isn’t that hard.  In the end though, I just felt OpenDS was easier, and would certainly be more straightforward for someone without much backround in the technology .. and it had the nifty isMemberOf attribute we wanted.

Testing by Hand

First I just wanted to make sure that the servers did what they should do.  I should be able to use any LDAP client to add, remove, search, and modify LDAP entries.  Everything worked fine using both the command line tools, and mostly OK using available GUI LDAP clients.  My client tool of choice is the Apache Directory Manager client.  For the purposes of this initial set of tests we decided to go the simple route – we won’t have thousands of objects in the LDAP store, maybe one day, but now we are talking about hundreds.  I only created a handful of users, groups, and orgs for this test.    The only caveat I had in my testing with this client was that I had some very specific problems connecting to the OpenDS server using a startTLS connection from a Linux client.  I did not have the same issue with the Windows client, and have had some Linux clients that I have been able to “clear up”, but I can’t explain why yet (see incident DIRSTUDIO-414 at the ApacheDS project).  I did not have this same problem connecting to an ApacheDS server (I did set one up when I ran into this problem – very easy!), and I still haven’t set up a secure OpenLDAP server yet, so I can’t provide any feedback on that.

My next step was to write some sample code to exercise OpenLDAP and OpenDS a little.  Again, no problems with either, including a secure connections using TLS to the OpenDS server (which makes the problem above even more perplexing).  I put in a little data, updated it etc.

The last thing I did was to do some data exports / imports from OpenDS to OpenDS, OpenLDAP to OpenLDAP, OpenLDAP to OpenDS, and OpenDS to OpenLDAP.  The only thing that was unexpected here was that OpenDS was placing UUID information into the LDIF, which OpenLDAP was NOT HAPPY with (as it shouldn’t be).  Removing the UUID from the LDIF took care of the issue.  After that, everything was great.

Automated Testing

My initial instinct was to go for my good old trusted friend JMeter.  I did run a couple of tests through JMeter before I decided that it was a little more work than I wanted to take on, so I went looking for another tool.  That is when I ran across SLAMD.  I have been very happy with SLAMD thus far, even though I know I have only scratched the surface. Keeping in mind that I don’t have a ton of data in the stores, I headed off to run on of the LDAP Load Generator projects.  I created two – one for OpenDS and one for OpenLDAP.  These tests ran continuously over an 8 hour period with 2 client machines running 40 threads apiece, with the first 10 minutes and last 10 minutes of the 8 hours was used as a warm up and cool down time. Some graphs from those tests are included below.

SLAMd Comparis on Overall Operations Attempted
SLAMd Comparison of Overall Operations Attempted
SLAMd Comparison of Overall Operation Time
SLAMd Comparison of Overall Operation Time

The Verdict

While I have not spent a ton of time doing any analysis on these results, and certainly have not doen any tweaking to affect them in any way, it would appear that OpenDS has better performance than OpenLDAP in our simple tests.  With a few exceptions, the overall variability of performance with OpenLDAP seemed to vary a little more widely than the OpenDS performance, which suprised me a little.  There were some very higly variable points with the OpenDS server, but they are very high peaks spread out over what looks like very consistant intervals.  I would tend to interpret these as garbage collection since OpenDS is java based.  As I said, I have only begun to really investigate these results, so I welcome any other interpretations or experiences that other have and will continue to do some further investigation, run additional tests, and test out synchronization.


9 thoughts on “Comparison of Directory / LDAP Servers

  1. Hm… Lots of comments, I guess I’ll start with The Past. Frankly your characterization of the situation Windows is, to be blunt, totally wrong. Symas fleshed out the Windows support in 1999-2000 and it has been a fully supported platform ever since then, just “configure; make” like all the rest of our platforms. We don’t spend a lot of time talking about its quirks because for the most part It Just Works. So no, it’s *not* impossible to build on Windows, and no, the Project doesn’t just ignore the Windows platform. We just don’t pay any more attention to it than any other platform.

    In fact Symas has spent quite a bit of time and energy into checking how well OpenLDAP performs on Windows.

    (Quite well, thank you very much.) Of course, that round of testing did reveal some other opportunities for optimization, which were committed to the CVS repository 2 weeks later.

    As for the situation Now, I’m inclined to point out that it’s unfair of you to take a crufty old CentOS release and compare it against an extremely new OpenDS release. If you’re going to use an OpenDS that was written within the past 6 months then you ought to be comparing against an OpenLDAP release from within the past 6 months; Red Hat is well known to be an extremely poor distro in terms of OpenLDAP packaging, perennially out of date, and CentOS doesn’t improve on that. But that’s rather beside the point…

    A properly configured OpenLDAP will blow the doors off a properly configured OpenDS and every other directory server on the planet.

    Your graphs show average operation response times on the order of 10 seconds, which I’ve never seen even with every possible debug option turned on. There’s something seriously wrong with your setup, and I’d be curious to see what the problem is.

  2. Fair comments, let me try to address them …

    On running on Windows – I know that quite a bit of work was done on Windows, and when I got it running it ran quite well. I did choose my words carefully though – I said “SEEMED to have developers that honestly couldn’t have cared less about supporting the Windows environment”. Configure:make did not work, and it took quite a bit of time to make it work. What it looked like to me was that someone had done a lot of work to make it run on Windows, but then much of that work was stepped on and I could not compile the code without making a significant number of changes. The part that led me to believe that no one cared was that when I asked some question I was told that Windows was not supported and I would need to figure it out myself. I was basically told to go it my own, which I did. You have to admit, that was not a very positive experience.

    Not too mention this particular statement in the FAQs on the site:

    “Information useful in building OpenLDAP 2.X-devel (HEAD) on MS Windows NT/2000. The NT/2000 port is experimental. Other Win32 platforms have yet to experimented with. Your mileage may vary! ”

    This was certainly true at the time for Win2000, which is what were were trying to compile on.

    All things considered, I did get it running, and we did choose it because other than that glitch and the surliness of the team supporting we were happy with the product.

    On the older version of OpenLDAP compared to the new OpenDS – you have a good point, but I don’t control what is in the CentOS repository. The rules we set up for this test were that we did not want to build the software, but wanted to get a prepackaged software. I do think that a new version would have run better, but probably not that significantly, it is only a year old … it’s not like the software is 5 years old. The comparison is 1 year old software vs. 6 month old software.

    I specifically did not do any tuning at this point – I was quite clear that I was running it out of the box – packaged software configured for my org, but not tweaked in any way.

    I can’t explain the slowness of the response times, that was very surprising to me as well. I am more than happy to work with others on doing some additional work in this area. I have already spoken with people from OpenDS and Apache about various aspects of these tests, and I am more than happy to talk to folks from OpenLDAP. I want to choose the best tool for our company, but also provided valuable, accurate information based on MY experience, not on how others think it should work.

  3. re: old vs new – OpenDS doesn’t come bundled with CentOS. If you’re going to have to install a 3rd party package to get it onto the machine, then it would be fair to install a 3rd party OpenLDAP build on the machine too.

    In the meantime, it’s well documented that you must tune various settings first. The default BerkeleyDB cache is only 256KB. The default data layout stores everything in one place but you really need to keep transaction logs on a separate spindle from the database files, etc. All of this is spelled out in the Admin Guide.

    I’m not telling you how I “think it should work” – I’m telling you how it *does* work, when you follow the documentation. That’s not theoretical, that’s actual.

    There’s other factors that may come into play, not all of which are directly under our control. E.g., the default malloc implementation in glibc was extremely bad up until very recently.

    (To be honest, I haven’t benchmarked a current glibc yet to see if it’s really fixed. I’ve pretty much switched to tcmalloc for all my critical installations and haven’t looked back.)

    You can email me if you want to investigate this further.

  4. Correct, OpenDS doesn’t come bundled, but there isn’t any different between installing a package in Linux (such as OpenLDAP) and setting up the tree and then running setup with OpenDS and setting up the tree . Making sure you have all the packages, configuring everything, and then making it is a different experience (it is easy enough to do, but the comparison was with pre-packaged software).

    I have been very clear that past setting up the tree, I have not done much to configure either OpenLDAP or OpenDS and that the test are based on an “out-of-the-box” install. I have no doubt that they would both perform better if I spent time tuning them, which I will. This is just the first set of testing.

    One thing won’t change with testing though, and that is the ease of use of OpenDS vs OpenLDAP – you guys (OpenLDAP) I believe are more focused on high performance and use by folks who tend to use CLI tools and who have deep experience with LDAP, while I think OpenDS (and ApacheDS too) apply some of their focus on a different kind of user experience that can extend LDAP into the hands of new folks. Honestly, my opinion is that those tools do have a better user experience and are less intimidating to folks than OpenLDAP.

    I do appreciate your comments and your offer to work with me on this and will contact you via e-mail.

  5. I came across the site since i am trying to collect information about alternatives to OpenLDAP. It is now the second time that i had a bad experience with the latter one. Basically your test sounds all right to me since it shows: OpenLDAP is poorely configured by default. It actually takes a lot of time to get what you want. Reading documentation, following up on problems in case of an error are common tasks then. The points are: getting a secured connection, populating a scheme, or configure the authentication. Btw, i made my experience at Debian/Lenny using the default packages.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s