- Remove From My Forums
-
Question
-
Dear all,
Im exeprence replication errors bettwen site to site replication.
Error:
Repadmin can’t connect to a «home server», because of the following error. Try
specifying a different
home server with /homeserver:[dns name]
Error: An LDAP lookup operation failed with the following error:LDAP Error 81(0x51): Server Down
Server Win32 Error 0(0x0):
Extended Information:This error occurs if I’m connected by the main WAN line (satilite), but if I change to backup line wokrs fine.
I was in the way to troubleshooting this issue with Netowrk TEAM, and they ask me wich ports are use when replication occurs so they can troubleshoot it.
I will kindly ask your help and add on this issue and also if you could provide details about all ports and flow of the replication.
Thanks in advance
Answers
-
Hi,
You need to check couple of the options to fix this issue.
1. Check DNS settings on NIC (preferred should be itself if it holds DNS role)
2. Repadmin /replsum at elivated command prompt. If you notice any errors work on that.
3. Add Antivirus exceptions for SYSVOL, NTDS folders
4. Restart Netlogon, DNS and ipconfig /flushdns & ipconfig /registerdns
5. If none of the above options doesn’t work, provide us ipconfig /all and DCDiag /v logs for better understanding about the issue.
-
Edited by
Saturday, December 1, 2012 3:06 PM
-
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:23 AM
-
Edited by
-
-
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:23 AM
-
Marked as answer by
-
-
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:23 AM
-
Marked as answer by
-
You have to start that the traffic on your main WAN connection is not filtered. Active Directory ports used for AD replication should be opened in both directions:
http://technet.microsoft.com/en-us/library/bb727063.aspxYou can use PortQryUI to check the filtering.
Of course, you should also check that the routing is correctly set for this connection. This can be checked using ping requests / responses (I suppose that ICMP traffic is not blocked).
Note also that AD replication behind a NAT device is not supported. That is why you will need to check if one of the routers used or the main WAN connection is using NAT. If it is the case, you will need to disable it (In your case, with a dedicated site
to site connection, NAT should not be required).Of course, dcdiag and repadmin commands should provide you with more details about the issue.
This posting is provided «AS IS» with no warranties or guarantees , and confers no rights.
-
Edited by
Mr XMVP
Sunday, December 2, 2012 3:19 PM -
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:23 AM
-
Edited by
-
-
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:23 AM
-
Marked as answer by
-
-
Marked as answer by
Yan Li_
Thursday, December 6, 2012 2:24 AM
-
Marked as answer by
Содержание
- Ldap error 81 0x51 server down
- Answered by:
- Question
- Answers
- All replies
Ldap error 81 0x51 server down
This forum has migrated to Microsoft Q&A. Visit Microsoft Q&A to post new questions.
Answered by:
Question
Im exeprence replication errors bettwen site to site replication.
Repadmin can’t connect to a «home server», because of the following error. Try
specifying a different
home server with /homeserver:[dns name]
Error: An LDAP lookup operation failed with the following error:
LDAP Error 81(0x51): Server Down
Server Win32 Error 0(0x0):
Extended Information:
This error occurs if I’m connected by the main WAN line (satilite), but if I change to backup line wokrs fine.
I was in the way to troubleshooting this issue with Netowrk TEAM, and they ask me wich ports are use when replication occurs so they can troubleshoot it.
I will kindly ask your help and add on this issue and also if you could provide details about all ports and flow of the replication.
Thanks in advance
Answers
You need to check couple of the options to fix this issue.
1. Check DNS settings on NIC (preferred should be itself if it holds DNS role)
2. Repadmin /replsum at elivated command prompt. If you notice any errors work on that.
3. Add Antivirus exceptions for SYSVOL, NTDS folders
4. Restart Netlogon, DNS and ipconfig /flushdns & ipconfig /registerdns
5. If none of the above options doesn’t work, provide us ipconfig /all and DCDiag /v logs for better understanding about the issue.
This article should help:
It mentions the ports used for various services.
Richard Mueller — MVP Directory Services
If the replication is working fine with backup line then it seems the issue somewhere with routing or ports of main line. You may refer the following article for AD replication ports.
If still issue persist, post dcdiag /q and repadmin /replsum or replication error events to assist you further.
Abhijit Waikar.
MCSA | MCSA:Messaging | MCITP:SA | MCC:2012
Blog: http://abhijitw.wordpress.com
Disclaimer: This posting is provided «AS IS» with no warranties or guarantees and confers no rights.
You have to start that the traffic on your main WAN connection is not filtered. Active Directory ports used for AD replication should be opened in both directions: http://technet.microsoft.com/en-us/library/bb727063.aspx
You can use PortQryUI to check the filtering.
Of course, you should also check that the routing is correctly set for this connection. This can be checked using ping requests / responses (I suppose that ICMP traffic is not blocked).
Note also that AD replication behind a NAT device is not supported. That is why you will need to check if one of the routers used or the main WAN connection is using NAT. If it is the case, you will need to disable it (In your case, with a dedicated site to site connection, NAT should not be required).
Of course, dcdiag and repadmin commands should provide you with more details about the issue.
This posting is provided «AS IS» with no warranties or guarantees , and confers no rights.
- Edited by Mr X MVP Sunday, December 2, 2012 3:19 PM
- Marked as answer by Yan Li_ Thursday, December 6, 2012 2:23 AM
Portquery is free tool from the MS which can be downloaded and installed to verify the necessary ports are opened or not.
PortQryUI — User Interface for the PortQry Command Line Port Scanner (GUI version)
http://www.microsoft.com/en-us/download/details.aspx?id=24009
Agreed with MX regarding the DCs used with NAT are not supported configuration.
How to design AD and DNS System with NAT Networks
http://social.technet.microsoft.com/Forums/en-US/winserverDS/thread/9fc98b8e-df73-4133-a87c-44c550225fce
Hope this helps
Disclaimer: This posting is provided «AS IS» with no warranties or guarantees , and confers no rights.
Did you see below article from the DS team.
DCDIAG Advertising test with error 81
Awinish Vishwakarma — MVP
Disclaimer This posting is provided AS-IS with no warranties/guarantees and confers no rights.
Im exeprence replication errors bettwen site to site replication.
Repadmin can’t connect to a «home server», because of the following error. Try
specifying a different
home server with /homeserver:[dns name]
Error: An LDAP lookup operation failed with the following error:
LDAP Error 81(0x51): Server Down
Server Win32 Error 0(0x0):
Extended Information:
This error occurs if I’m connected by the main WAN line (satilite), but if I change to backup line wokrs fine.
I was in the way to troubleshooting this issue with Netowrk TEAM, and they ask me wich ports are use when replication occurs so they can troubleshoot it.
I will kindly ask your help and add on this issue and also if you could provide details about all ports and flow of the replication.
Thanks in advance
Just as an add I will like to mention that I have try the following :
If I test smb connection from both side and this happens:
Server b to a : \servernamesysvol connection fail
If I change to line backup all works.
You need to check couple of the options to fix this issue.
1. Check DNS settings on NIC (preferred should be itself if it holds DNS role)
2. Repadmin /replsum at elivated command prompt. If you notice any errors work on that.
3. Add Antivirus exceptions for SYSVOL, NTDS folders
4. Restart Netlogon, DNS and ipconfig /flushdns & ipconfig /registerdns
5. If none of the above options doesn’t work, provide us ipconfig /all and DCDiag /v logs for better understanding about the issue.
This article should help:
It mentions the ports used for various services.
Richard Mueller — MVP Directory Services
If the replication is working fine with backup line then it seems the issue somewhere with routing or ports of main line. You may refer the following article for AD replication ports.
If still issue persist, post dcdiag /q and repadmin /replsum or replication error events to assist you further.
Abhijit Waikar.
MCSA | MCSA:Messaging | MCITP:SA | MCC:2012
Blog: http://abhijitw.wordpress.com
Disclaimer: This posting is provided «AS IS» with no warranties or guarantees and confers no rights.
You have to start that the traffic on your main WAN connection is not filtered. Active Directory ports used for AD replication should be opened in both directions: http://technet.microsoft.com/en-us/library/bb727063.aspx
You can use PortQryUI to check the filtering.
Of course, you should also check that the routing is correctly set for this connection. This can be checked using ping requests / responses (I suppose that ICMP traffic is not blocked).
Note also that AD replication behind a NAT device is not supported. That is why you will need to check if one of the routers used or the main WAN connection is using NAT. If it is the case, you will need to disable it (In your case, with a dedicated site to site connection, NAT should not be required).
Of course, dcdiag and repadmin commands should provide you with more details about the issue.
This posting is provided «AS IS» with no warranties or guarantees , and confers no rights.
- Edited by Mr X MVP Sunday, December 2, 2012 3:19 PM
- Marked as answer by Yan Li_ Thursday, December 6, 2012 2:23 AM
Portquery is free tool from the MS which can be downloaded and installed to verify the necessary ports are opened or not.
PortQryUI — User Interface for the PortQry Command Line Port Scanner (GUI version)
http://www.microsoft.com/en-us/download/details.aspx?id=24009
Agreed with MX regarding the DCs used with NAT are not supported configuration.
How to design AD and DNS System with NAT Networks
http://social.technet.microsoft.com/Forums/en-US/winserverDS/thread/9fc98b8e-df73-4133-a87c-44c550225fce
Hope this helps
Disclaimer: This posting is provided «AS IS» with no warranties or guarantees , and confers no rights.
Just checking in to see if the suggestions were helpful. Please let us know if you would like further assistance.
If you have any feedback on our support, please click here .
Cataleya Li
TechNet Community Support
Did you see below article from the DS team.
DCDIAG Advertising test with error 81
Awinish Vishwakarma — MVP
Disclaimer This posting is provided AS-IS with no warranties/guarantees and confers no rights.
I have done with above troubleshooting and everything is fine, but issue is still persist. whenever I run «repadmin /replicate dest-ADC1 source-DC01 » it’s giving me error «LDAP Error 81(0x51): Server Down Server Win32 Error 0(0x0):» as per you I proving ipconfig /all and dcdiag result. please have a look and help me.
ipconfig /all
Windows IP Configuration
Host Name . . . . . . . . . . . . : DC1
Primary Dns Suffix . . . . . . . : rk19.com
Node Type . . . . . . . . . . . . : Hybrid
IP Routing Enabled. . . . . . . . : No
WINS Proxy Enabled. . . . . . . . : No
DNS Suffix Search List. . . . . . : rk19.com
Ethernet adapter Ethernet0:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) 82574L Gigabit Network Connection
Physical Address. . . . . . . . . : 00-0C-29-59-41-83
DHCP Enabled. . . . . . . . . . . : No
Autoconfiguration Enabled . . . . : Yes
Link-local IPv6 Address . . . . . : fe80::98ac:492b:fccb:b411%12(Preferred)
IPv4 Address. . . . . . . . . . . : 172.16.0.3(Preferred)
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 172.16.0.1
DHCPv6 IAID . . . . . . . . . . . : 301993001
DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1F-93-61-78-00-0C-29-59-41-83
DNS Servers . . . . . . . . . . . : ::1
172.16.0.3
172.16.0.9
NetBIOS over Tcpip. . . . . . . . : Enabled
DCDiag
Directory Server Diagnosis
Performing initial setup:
Trying to find home server.
Home Server = DC1
* Identified AD Forest.
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site-NameDC1
Starting test: Connectivity
. DC1 passed test Connectivity
Doing primary tests
Testing server: Default-First-Site-NameDC1
Starting test: Advertising
. DC1 passed test Advertising
Starting test: FrsEvent
. DC1 passed test FrsEvent
Starting test: DFSREvent
There are warning or error events within the last 24 hours after
. DC1 failed test DFSREvent
Starting test: SysVolCheck
. DC1 passed test SysVolCheck
Starting test: KccEvent
. DC1 passed test KccEvent
Starting test: KnowsOfRoleHolders
. DC1 passed test KnowsOfRoleHolders
Starting test: MachineAccount
. DC1 passed test MachineAccount
Starting test: NCSecDesc
. DC1 passed test NCSecDesc
Starting test: NetLogons
. DC1 passed test NetLogons
Starting test: ObjectsReplicated
. DC1 passed test ObjectsReplicated
Starting test: Replications
. DC1 passed test Replications
Starting test: RidManager
. DC1 passed test RidManager
Starting test: Services
. DC1 passed test Services
Starting test: SystemLog
. DC1 passed test SystemLog
Starting test: VerifyReferences
. DC1 passed test VerifyReferences
Running partition tests on : ForestDnsZones
Starting test: CheckSDRefDom
. ForestDnsZones passed test CheckSDRefDo
Starting test: CrossRefValidation
. ForestDnsZones passed test CrossRefVali
Running partition tests on : DomainDnsZones
Starting test: CheckSDRefDom
. DomainDnsZones passed test CheckSDRefDo
Starting test: CrossRefValidation
. DomainDnsZones passed test CrossRefVali
Running partition tests on : Schema
Starting test: CheckSDRefDom
. Schema passed test CheckSDRefDom
Starting test: CrossRefValidation
. Schema passed test CrossRefValidation
Running partition tests on : Configuration
Starting test: CheckSDRefDom
. Configuration passed test CheckSDRefDom
Starting test: CrossRefValidation
. Configuration passed test CrossRefValid
Running partition tests on : rk19
Starting test: CheckSDRefDom
. rk19 passed test CheckSDRefDom
Starting test: CrossRefValidation
. rk19 passed test CrossRefValidation
Running enterprise tests on : rk19.com
Starting test: LocatorCheck
. rk19.com passed test LocatorCheck
Starting test: Intersite
. rk19.com passed test Intersite
Источник
ldp.exe LDAPS Cannot open connection Error 81
April 21, 2020 04:30PM
Part 1: Install and configure certificate authority (CA) on Microsoft Windows server with Group Policy
Part 2: Configuring Secure LDAPs on Domain Controller
ldp.exe LDAPS Cannot open connection Error 81
Part 3: Install and Configure Active Directory Federation Service (ADFS)
While setting up a lab for Configuring Secure LDAPs on Domain Controller I faced an error. After deploying SSL on LDAP and testing AD connection using Ldp.exe utility, I was using IP address to connect.
Below is the error log on the screen.
ld = ldap_sslinit(«192.168.34.11», 636, 1);
Error 0 = ldap_set_option(hLdap, LDAP_OPT_PROTOCOL_VERSION, 3);
Error 81 = ldap_connect(hLdap, NULL);
Server error: <empty>
Error <0x51>: Fail to connect to 192.168.34.11.
There was also a popup box with warning message Cannot open connection.
To dig deeper I checked Event Viewer >> System Log and found Error Event ID 36869.
The TLS server credential’s certificate does not have a private key information property attached to it. This most often occurs when a certificate is backed up incorrectly and then later restored. This message can also indicate a certificate enrollment failure.
To troubleshoot further I checked SSL certificate deployed for LDAP on Domain Controller. On the personal (my) computer account go and check properties of LDAPS certificate. I checked Issued to, Subject CN and Certification Path. There was no IP mentioned. I used FQDN to connect on ldp.exe.
After using FQDN (fully qualified domain name), LDAP connection over SSL to domain controller established successfully.
If you want to make sure LDAPs connection is using only your assigned SSL certificate, You can remove/delete unused and unwanted certificates from LocalMachine Personal (my) store on Domain Controller. Use below command to verify certificate
certutil -VerifyStore MY
If you have renewed certificate on the server make sure you update the same information on the Domain Controller with below procedure. Save below information to text file.
dn:
changetype: modify
add: renewServerCertificate
renewServerCertificate: 1
-
And execute command.
ldifde -i -f c:certenable_ldap.txt
Useful Articles
Generate new self-signed certificates for ESXi using OpenSSL
Push SSL certificates to client computers using Group Policy
Replacing a default ESXi certificate with a CA-Signed certificate
Troubleshooting replacing a corrupted certificate on Esxi server
How to import default vCenter server appliance VMCA root certificate and refresh CA certificate on ESXi
How to replace default vCenter VMCA certificate with Microsoft CA signed certificate
Go Back
I would have expected more debug output. This looks like it’s from the very end of the TLS negotiation process, right when it fails. It would be helpful to see what led up to it.
However, as far as I can tell, it looks like the failure is in some way related to TLS renegotiation. The «waiting for close_notify or alert» indicates that the connection already considers itself closed, and it’s just waiting for either the socket to be closed or an SSL alert message with additional information, and the «state 3» indicates that it’s in the process of attempting to renegotiate the session.
Immediately after that, you can see that it reads data, but it’s not able to interpret it, and that leads to the «fatal, unexpected_message». It follows that by invalidating two existing TLS sessions, closing the connection, and throwing the exception that the LDAP SDK catches and wraps with its own LDAPException.
All of this is happening inside Java’s TLS processing code, and the LDAP SDK doesn’t really have much more control over or insight into what’s going on. Without debug information from earlier in the process, I can’t really say what caused the failure. I also don’t know why inserting a delay would change things. However, it does look like there had already been a couple of TLS sessions that had been successfully established. Given that there is a renegotiation in progress, maybe it’s related to an attempt to reuse a session that’s being destroyed or invalidated.
You mention that you’re using a custom socket factory. That also raises some questions:
- Does that socket factory do anything unusual? Might it be invalidating sessions?
- Might there be a thread safety problem in that socket factory that you had been protected from by defaulting to not allowing concurrent socket factory use in unrecognized JVM implementations in earlier versions of the LDAP SDK? Does the problem still occur if you call LDAPConnectionOptions.setAllowConcurrentSocketFactoryUse(false)? Perhaps injecting the delay has a similar effect to reducing contention and avoids the thread safety problem.
- Do you see the problem if you take your socket factory out of the picture?
It also sounds like it only happens in one specific environment. It would obviously be good to look at what might be different in that environment, and especially related to the server’s TLS configuration, whether you have any networking equipment in the picture that might do weird things (especially something that might involve TLS, like a load balancer that acts as an SSL endpoint).
All of this is just speculation based on a few pieces of information, but it’s really about all there is to go on right now.
so, I it’s been about week since my last update — sorry. Been stumped by this a while now obviously, but it only just now came to be pressing, as our CEO got a new Blackberry and we weren’t able to access the BES controls… likely due to this LDAP issue. (It was configured to used windows authentication)
Following up on your last comment britv8 — the [isGlobalCatalogReady is false] was apparently false because those DCs did not have the global catalog enabled. I’ve since enabled it on those servers for a better control group with fewer variables as I try to determine what is going on here. I am still getting the LDAP errors.
Here’s what I’ve found thus far:
——————————————————————————————————
every DC gives the same error message response to: repadmin /showreps /all /verbose («LDAP error 81 (Server Down) Win32 Err 58»)
——————————————————————————————————
nltest /dsgetdc:wcnb /force /gc
every DC passes this test, and advertises the same flags: with the exception of the PDC which includes that flag: (GC DS LDAP KDC TIMESERV WRITABLE DNS_FOREST CLOSE_SITE)
——————————————————————————————————
AD Replication Status Tool on my first scan revealed no errors in the replication status collection details, but did show a connection error of some sort — I’ve been unable to get that error message since I refreshed the replication status at any time.
——————————————————————————————————
Both my primary and secondary (at primary site) failed a [Netdiag /test:DNS]. error message as follows:
————-
DNS test . . . . . . . . . . . . . : Failed
[WARNING] Cannot find a primary authoritative DNS server for the name
‘<mydc2>.’. [WSAEADDRNOTAVAIL ]
The name ‘<mydc2>.’ may not be registered in DNS.
[WARNING] Cannot find a primary authoritative DNS server for the name
‘<mydc2>.’. [ERROR_TIMEOUT]
The name ‘<mydc2>.’ may not be registered in DNS.
[WARNING] Cannot find a primary authoritative DNS server for the name
‘<mydc2>.’. [WSAEADDRNOTAVAIL ]
The name ‘<mydc02>.’ may not be registered in DNS.
[WARNING] Cannot find a primary authoritative DNS server for the name
‘<mydc02>.’. [WSAEADDRNOTAVAIL ]
The name ‘<mydc02>.’ may not be registered in DNS.
[WARNING] Cannot find a primary authoritative DNS server for the name
‘<mydc2>.’. [ERROR_TIMEOUT]
The name ‘<mydc2>.’ may not be registered in DNS.
[WARNING] The DNS entries for this DC are not registered correctly on DNS server ‘0.0.0.0’. Please wait for 30 minutes for DNS server replication.
[FATAL] No DNS servers have the DNS records for this DC registered.
————-
an alternate site DC passed with warning as follows:
————-
DNS test . . . . . . . . . . . . . : Passed
PASS — All the DNS entries for DC are registered on DNS server ‘<alternate site DC IP>‘ and other DCs also have some of the names registered.
[WARNING] The DNS entries for this DC are not registered correctly on DNS se
rver ‘<PDC IP Address>’. Please wait for 30 minutes for DNS server replication.
————-
The other 3 DCs passed.
——————————————————————————————————
I’ve checked that each DC (IPV4) is looking to itself for primary DNS, and the PDC as it’s alternate.
In the process I notice that the PDC and the primary site alternate DC both have IPV6 enabled, and accompanying tunnel adapters. (Some post that I have read have me leaning to this as my next direction in this investigation.)
——————————————————————————————————
While investigating DNS (which I am still learning) I noticed that one <mylocaldomain>_msdcs>GC>_sites was missing along with the tcp>_Ldap entry.
Restarting the Net Logon service seemed to resolve this missing site, without affecting the other errors already
——————————————————————————————————
Any suggestions are greatly appreciated. Thanks!
Was this post helpful?
thumb_up
thumb_down
Возможные причины:
1. Строка слишком большой длины.
2. Неверный тип — строка записывается в числовой атрибут.
3. Неправильное значение, например, атрибут может принимать только определённое значение, либо одно из набора значений.
Возможные причины:
1. При добавлении записи — один или несколько атрибутов в LDIF (или операции добавления/замены) для записи в точности совпадают (дублируются).
Дополнительный текст: unable to get TLS Client DN (невозможно получить DN клиента TLS).
Возможные причины:
1. Не предоставлен сертификат клиента в случае, если директива TLSVerifyClient установлена в ‘demand’.
2. Не предоставлен сертификат клиента в случае, если директива TLSVerifyClient установлена в ‘never’. В этом случае данное сообщение об ошибке не является фатальным и обслуживание клиента продолжается.
Дополнительный текст: no global superior knowledge (нет сведений о глобальном вышестоящем каталоге) — имя записи, которую собираются добавить или модифицировать, не находится ни в одном из контекстов именования и у сервера нет правильной отсылки на вышестоящий каталог.
Возможная причина: не задан атрибут olcSuffix (директива suffix в slapd.conf) для DIT, на которое идёт ссылка.
Дополнительный текст: Shadow context; no update referral (теневой контекст (реплика); отсылки для выполнения обновлений не указано) — DIT, в которое собираются вносить изменения, является репликой в режиме «только для чтения», и, из-за отсутствия директивы updateref, невозможно возвратить отсылку.
Возможные причины:
1. Была попытка произвести запись в реплику «только для чтения» (в конфигурации syncrepl потребитель всегда в режиме «только для чтения»).
2. В конфигурации syncrepl multi-master в файле slapd.conf возможно пропущена директива mirrormode true.
3. Если slapd при запуске использовал файл slapd.conf, а директория slapd.d (cn=config) также существует, то при последующих модификациях DIT могут возникать ошибки с выдачей этого сообщения. В частности, в FreeBSD требуется наличие явного указания в rc.conf (slapd_cn_config=»YES») для принудительного использования slapd.d.
Возможная причина:
Попытка удаления атрибута (особенно в cn=config), удаление которого запрещено.
Дополнительный текст: olcDbDirectory: value #0: invalid path: No such file or directory
Возможная причина: перед инициализацией новой базы данных директория для её размещения должна существовать.
Recently I run into the problem where Exchange return with the error:
“An Active Directory error 0x51 occured when trying to check the suitability of Server…”
Weird thing this happened not for all commands. It was somehow randomly, but this caused several issues:
- prompt for credential (which was the most ugly side effect!)
- errors in scripts
- CmdLets didn’t return all values
- …..
When I first saw this error I had a déjà-vu. Last year I had a very long running case with Microsoft, where I had the very similar errors. But back in time Exchange 2010 on Windows Server 2008 R2 was affected. After several Gigabyte of network and LDAP traces it turned out to be an ICMP issue on the OS level:
The LDAP check is using ICMP to evaluate whether the server is up or down. And there was a bug in the ICMP stack, which result in an 0x51 LDAP error even the server was up and healthy. Read this KB for more information.
But now I started seeing this for Exchange 2013 CU10 on Windows Server 2012 R2.
How bad is it?
First I needed to know if this happens on a few server or on all. Therefore I needed to crawl the event logs across all Exchange servers for the EventID 2070. To speed things up I wrote the following function:
function Collect-Events (){
param(
[Parameter(ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$true,Position=0)]
[Alias('fqdn')]
[string] $computername = $env:computername,
[parameter( Mandatory=$false, ValueFromPipelineByPropertyName=$false,Position=1)]
[string]$EventID = '2070',
[parameter( Mandatory=$false, ValueFromPipelineByPropertyName=$false,Position=2)]
[string]$Eventlog = 'Application',
[parameter( Mandatory = $false, ValueFromPipelineByPropertyName=$false,Position=3)]
[DateTime]$StartTime = $((Get-Date).AddHours(-12)),
[parameter( Mandatory=$false, ValueFromPipelineByPropertyName=$false,Position=4)]
[ValidateSet("Critical","Error","Warning","Information","Verbose")]
[string]$Severity
)
process
{
Write-Host "Processing $Computername....."
If ($Severity) {
switch ($Severity) {
"Critical" {$level = 1}
"Error" {$level = 2}
"Warning" {$level = 3}
"Information" {$level = 4}
"Verbose" {$level = 5}
}
Get-WinEvent -ComputerName $Computername -FilterHashtable @{logname=$Eventlog;id=$EventID;StartTime=$StartTime;Level=$level} -ErrorAction SilentlyContinue
}
Else {
Get-WinEvent -ComputerName $Computername -FilterHashtable @{logname=$Eventlog;id=$EventID;StartTime=$StartTime} -ErrorAction SilentlyContinue
}
}
}
This function has already the correct Event Log(Application) and EventID(2070) predefined. Now you can easily search across your Exchange servers. The following example search for EventID 2070 within the last 2 hours:
$2070 = Get-ExchangeServer | Collect-Events -StartTime (Get-Date).addhours(-2)
It turned out to be a general issue and not only on a few servers. Feel free to use this function to search for different events.
Root cause
After turning on logging for MSExchange ADAccess I could see the servers were heavily using Out-of-Site DC’s and GC’s. Shortly I’ve found the following KB, which explained a lot:
https://support.microsoft.com/kb/3088777
Before you start panic: This is only an issue in larger environments with multiple AD sites! Smaller ones shouldn’t be affected. Just to get an idea: In my case we have over 280 AD sites across the globe and not always the needed network bandwidth and latency, which is in general okay as in our scenario Exchange shouldn’t contact the most of them.
How to fix?
To fix this issue and change the behavior you just have to follow the KB article and edit the file Microsoft.Exchange.Directory.TopologyService.exe.config, and restart the service MSExchangeADTopology.
I have to admin just to restart this service sounds easy, but in the end you have to reboot the server. Not always all depending services could be gracefully restarted with MSExchangeADTopology service.
Conclusion
This change was made in CU6. From Microsoft perspective I understand why the change was made. Cloudwise it makes sense, but for larger on-premise installations this could really cause issue.
I hope this helps someone!









