AD LDS ADAMSync stops synchronizing with a DNManip error

I’ve been with a customer recently to work with them in our Windows Server 2012 RDP. RDP stands for “Rapid Deployment Program” in which we’re helping customers adopt Windows Server 2012. That’s right – we’re supporting customers on their way to upgrading to Server 2012.

In a Active Directory deployment with Server 2012, we wanted to spawn two new load-balanced AD Leightweight Directory Service (LDS) servers as a side activity. We would synchronize AD data into AD LDS instance and point applications to them, to abstract applications from our Domain Controllers.

While starting the initial sync, AD LDS would think for a moment, start synchronizing and would stop with a cryptic and not-so-helpful error message:

An internal error occurred: DnManip::DnManip

1

Hm – interesting. DnManip::DnManip looks like a name of a function that’s called and that produces an error. LDS stopped synchronizing pretty early in the process but has synchronized a number of users – so it does some work after all. We were thinking what to do next and started a quick web search. Funny enough, not much turned up.

We finally decided to enable logging – we appended the “/log” parameter with a file name (Logging is described in http://support.microsoft.com/kb/923835). We let it log into a dnmanip_error.txt text file in the ADAM directory. We started synchronizing again and looked at the log file. Sure enough, ADAMSync would tell us the exact same error at the bottom of the log file – along with the GUID of the object that it has trouble with:

3

With that GUID at hand, we can examine what object we’re dealing with in AD DS and see what we can do to help AD LDS sync it. How do we find an object based on its objectGuid? We could have searched with ADFind or Powershell, but we felt kind of lazy, so we opened LDP.exe, connected, performed a bind and searched for the GUID:

4

The GUID just goes into the BaseDN box – with the brackets in the beginning and the end. Hit “Run” and sure enough LDP.exe would find our object in question:

5

Look at the bold part in LDP’s results pane – the DN of the object we’re looking at is a conflict object! That’s why LDP has trouble synchronizing it. Conflict objects get a new name that’s comprised of the RDN (user-0), a special character that cannot be entered manually (/OA) and the conflict prefix followed by the object’s GUID: (CNF:cb23a754-….). Read more about conflict resolution in AD here: http://technet.microsoft.com/en-us/library/cc772726(v=WS.10).aspx, Section “Multimaster Conflict Resolution Policy”.

Cleraly, the special character, /OA can’t be sychronized (‘cause it’s special) which is why LDS refuses to continue. Now the “DnManip” error message makes sense: we’re looking at a special character in the “distinguishedName”, DN, attribute – and the Dn-Manipulation function which is where the error occurs. D’oh!

It’s worth noting that Active Directory has built-in means of handling replication conflicts – renaming the conflict victim is one of the means. And during the rename, the /OA special character is added. If you’ve looked closer at how AD stores and treats deleted objects, you’ll notice AD renames them, too. Deleted objects carry the special /OA character in their DN too, followed by the “DEL": prefix and their GUID:

2 

Fortunately, deleted users are not in ADAMSync’s scope and deleted users are not sync’ed. If we moved the user out of the “Deleted Objects” container and reset their “isDeleted” flag, the same thing would have happened to the deleted user – just like it did with our conflict user.

So – now that we know, we searched for “real” object (the conflict winner), made sure it was available and used and deleted the conflict object.

For a “deleted object”, outside of the “Deleted Object” container, that is an object that has the “/OADEL” in its DN, we’d have to look closer at the object. Chances are the object is still in use and it got restored throughout its life with an obscure(?) restore method. Deleting a “deleted object” again,

Kick off ADAMSync again, it would run happily for a few seconds until it stopped with the same error again – this time a different GUID, a different object. Sigh. Looks like there was more than one conflict object. Hm. Now – rather than fixing each single object and running ADAMSync again – is there a better method to detect conflict and deleted objects in our NC (other than the ‘Deleted Objects’ container)?

I did some thinking, as the customer wanted a good way of knowing how many object there are, and this is what I think can be done:

  • Dump all users in Excel and look through the list manually (look for CNF and DEL in the object DNs)
  • Script a logic with Powershell

I won’t go into detail on how to dump all users into Excel and manually dig through the list, looking for DEL and CNF DNs on object. Instead, I’ll post a Powershell one-liner that makes our search a little easier. Since the conflict or deleted objects have their GUID in the DN (see how the DN is constructed above) – we can ask Powershell to do the following:

Powershell, return all objects from objectClass ‘user’ that have their objectGUID as part of their DN.

And here’s how we build it in Powershell:

Get-ADobject -LDAPFilter:"(objectClass=user)" | ?{$_.DistinguishedName.Contains($_.objectGUID)} | %{write-host $_}

I am sure there are more elegant ways of doing it – but this is what I came up with after a couple minutes of looking at the problem, thinking and scripting. I set objectClass=user so that both user and computer objects are returned. Just to be sure. In my environment, the one-liner returns the following:

6

And – oh wonder – user-0 is one of the conflict objects. Looks like I only have three problematic objects in my environment. After cleaning them up, ADAMSync ran happily ever after!

No Comment