Troubleshooting RPC_S_SERVER_UNAVAILABLE (1722) for Cross-Domain COM+ Calls with Strict Firewalls

The RPC_S_SERVER_UNAVAILABLE error, numerically represented as 1722 (0x6BA), is a common yet frustrating issue for developers and system administrators. It signifies that a client application was unable to establish a connection with a Remote Procedure Call (RPC) server. This problem becomes particularly acute when a Windows client attempts to invoke methods on a COM+ server residing in a different Active Directory domain, especially when stringent firewall rules are in place. Such scenarios are frequent in enterprise environments with segmented networks for security or organizational reasons.

This article provides a deep dive into the underlying causes of this error in cross-domain COM+ communication and offers a systematic approach to troubleshooting and resolving it, with a strong focus on firewall configurations and DCOM intricacies.

Understanding the Core Problem: RPC, DCOM, and Firewalls

Before diving into troubleshooting, it’s crucial to understand the technologies involved and how they interact:

Remote Procedure Call (RPC): RPC is a protocol that allows a program on one computer to execute code on another computer remotely. COM+ and DCOM rely heavily on RPC for their remote communication capabilities.
Component Object Model (COM) & COM+: COM is a Microsoft technology that allows software components to interact. COM+ builds upon COM, providing an application server environment for deploying and managing distributed, component-based applications. It handles transactions, security, object pooling, and more.
Distributed COM (DCOM): DCOM extends COM to enable communication between objects on different machines across a network. When a client calls a COM+ component on a remote server, DCOM is the underlying mechanism facilitating this call.
RPC Endpoint Mapper (EPM): When a client wants to connect to an RPC server (like a COM+ application), it first contacts the RPC Endpoint Mapper service on the server, typically listening on TCP port 135. The EPM then informs the client which dynamically assigned port the specific RPC application is listening on.
Dynamic RPC Ports: By default, RPC services, including DCOM servers like COM+ applications, register themselves with the EPM to listen on dynamically allocated TCP ports. This range is quite large by default (e.g., 49152-65535 on modern Windows versions, or a legacy range like 1024-5000 on older systems).
Cross-Domain Communication: When the client and COM+ server are in different Active Directory domains, additional complexities arise related to authentication (domain trusts) and name resolution.
Firewalls: Firewalls, whether host-based (like Windows Defender Firewall) or network-based appliances, inspect and filter traffic. Strict firewall rules often block the wide range of dynamic RPC ports by default, leading to the RPC_S_SERVER_UNAVAILABLE error.

The crux of the problem usually lies in the client successfully contacting the EPM on port 135, but then failing to connect to the COM+ application on the subsequent dynamic port because a firewall blocks it.

Systematic Troubleshooting Steps

Resolving RPC_S_SERVER_UNAVAILABLE requires a methodical approach. Start with basic checks and progressively move to more complex configurations.

1. Verify Essential Services

Ensure critical services are running on both the client and the COM+ server:

Remote Procedure Call (RPC) (RpcSs): Fundamental for RPC communication.
DCOM Server Process Launcher (DcomLaunch): Responsible for launching COM servers.
COM+ Event System (EventSystem): Necessary for COM+ applications.
RPC Endpoint Mapper (RpcEptMapper): Manages dynamic port registrations.

Use PowerShell to check their status:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Run on both client and COM+ server
$services = @("RpcSs", "DcomLaunch", "EventSystem", "RpcEptMapper")
foreach ($serviceName in $services) {
    $service = Get-Service -Name $serviceName -ErrorAction SilentlyContinue
    if ($service) {
        Write-Host ("{0,-25} Status: {1,-10} StartType: {2}" -f `
            $service.DisplayName, $service.Status, $service.StartType)
    } else {
        Write-Host ("Service '{0}' not found." -f $serviceName) `
            -ForegroundColor Red
    }
}

All these services should be Running and typically set to Automatic start type.

2. Basic Network Connectivity and Name Resolution

Ping Test: From the client, ping the COM+ server by its Fully Qualified Domain Name (FQDN) and IP address to verify basic network reachability.
1 2
ping server.domain.com ping <server_IP_address>
DNS Resolution: Ensure the client can correctly resolve the COM+ server’s FQDN and that the server can resolve the client’s FQDN (important for some authentication scenarios).
1
nslookup server.domain.com
On the server, perform a similar nslookup client.otherdomain.com. In cross-domain scenarios, ensure DNS forwarders or conditional forwarders are correctly configured between the domain DNS servers.

3. Firewall Configuration: The Primary Suspect

This is the most common cause, especially with strict firewall policies.

TCP Port 135 (RPC Endpoint Mapper): The firewall on the COM+ server (and any network firewalls in between) must allow inbound connections to TCP port 135 from the client.
Dynamic RPC Ports: The firewalls must also allow inbound connections from the client to the COM+ server on the entire range of dynamic RPC ports that DCOM might use.

Action: Configure firewall rules accordingly. For Windows Defender Firewall, you can use netsh advfirewall or PowerShell cmdlets.

PowerShell example to open TCP port 135 and a common dynamic range (e.g., 49152-65535):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Run on the COM+ Server
# Allow RPC Endpoint Mapper
New-NetFirewallRule -DisplayName "DCOM RPC EPM (TCP-In)" `
    -Direction Inbound -Protocol TCP -LocalPort 135 `
    -Action Allow -Profile Domain,Private `
    -Description "Allow DCOM RPC Endpoint Mapper."

# Allow Dynamic RPC Ports (example range, adjust if restricted)
# For default Windows range (Windows Vista/Server 2008 and later):
New-NetFirewallRule -DisplayName "DCOM Dynamic RPC (TCP-In)" `
    -Direction Inbound -Protocol TCP -LocalPort 49152-65535 `
    -Action Allow -Profile Domain,Private `
    -Description "Allow DCOM Dynamic RPC ports."

It’s highly recommended to restrict this dynamic port range for better security.

4. Restricting RPC Dynamic Port Range (Highly Recommended)

Allowing a vast range of over 16,000 ports is a significant security concern. Restrict the dynamic port range used by DCOM/RPC on the COM+ server and then open only that specific, smaller range in the firewall. Microsoft provides detailed guidance on this in “How to configure RPC dynamic port allocation to work with firewalls”. They suggest a minimum of 100 ports, but 200-500 is often safer for busy servers.

Registry settings on the COM+ server: Navigate to: HKEY_LOCAL_MACHINE\Software\Microsoft\Rpc\Internet

Create the following values (if they don’t exist):

Ports (REG_MULTI_SZ): e.g., 5000-5200 (specify your chosen range).
PortsInternetAvailable (REG_SZ): Y
UseInternetPorts (REG_SZ): Y

A server reboot is required for these settings to take effect. After restricting the range, update your firewall rules to allow only this custom range (e.g., TCP 5000-5200 in the example above) instead of the entire default dynamic range.

5. DCOM Configuration (`dcomcnfg`)

Incorrect DCOM security settings can also lead to connection failures. Use dcomcnfg on the COM+ server.

Enable DCOM:
1. Run dcomcnfg.
2. Expand Component Services -> Computers.
3. Right-click My Computer -> Properties.
4. Go to the Default Properties tab.
5. Ensure Enable Distributed COM on this computer is checked.
6. Default Authentication Level: Connect.
7. Default Impersonation Level: Identify.
Machine-Wide Launch and Access Permissions:
1. In My Computer Properties, go to the COM Security tab.
2. Under Access Permissions, click Edit Limits... and Edit Default.... Ensure that the client’s user account, or relevant groups (like Authenticated Users, Network, or Everyone for testing – use with caution in production), have Local Access and Remote Access permissions.
3. Under Launch and Activation Permissions, click Edit Limits... and Edit Default.... Ensure the same accounts/groups have Local Launch, Remote Launch, Local Activation, and Remote Activation permissions.
Application-Specific Permissions:
1. If the COM+ application uses custom permissions, navigate to it under DCOM Config (e.g., Component Services -> Computers -> My Computer -> DCOM Config -> Your Application).
2. Right-click your application -> Properties.
3. Go to the Security tab and adjust Launch and Activation Permissions and Access Permissions as needed. Often, customizing these to grant specific client user accounts or groups access is more secure than relying solely on machine-wide defaults.
4. Also check the Identity tab to ensure the account running the COM+ application has sufficient privileges.

6. Authentication and Domain Trusts

For cross-domain communication, proper domain trust relationships are essential.

Verify Trust: Ensure a valid trust exists between the client’s domain and the server’s domain. The type of trust (e.g., two-way transitive, external) must permit the required authentication flow.
Test Trust with nltest: Use the nltest utility from a command prompt on a domain-joined machine (preferably the client or server) to verify the trust and secure channel.
1 2 3 4 5
REM Run on client, testing trust with server's domain nltest /sc_verify:<server_domain_FQDN> REM Run on server, testing trust with client's domain nltest /sc_verify:<client_domain_FQDN>
Look for “Trust relationship status” or “Secure channel” messages.
Time Synchronization: Kerberos authentication, commonly used in domain environments, is sensitive to time differences. Ensure client, server, and domain controllers are synchronized with a reliable time source. A skew of more than 5 minutes (default) can cause authentication failures.
Service Principal Names (SPNs): If Kerberos is used, the COM+ server might require correctly registered SPNs. This is less common for generic COM+ but can be a factor.

7. Event Log Analysis

Check the Event Viewer on both the client and COM+ server for related errors around the time of the failure:

Server Logs:
- System log: Look for DCOM errors (e.g., Event IDs 10006, 10010, 10016), RPC errors, or network-related errors.
- Application log: Errors from the specific COM+ application.
- Security log: Audit Failures for logon attempts (Logon Type 3 is network logon).
Client Logs:
- System and Application logs: Client-side RPC errors or application errors detailing the connection failure.

Event ID 10016 often indicates DCOM permission issues. Event ID 1722 directly points to the RPC server being unavailable.

8. Diagnostic Tools

PortQry / PortQryUI: This Microsoft tool is invaluable for testing port connectivity. Download from the Microsoft Download Center for PortQry Command Line Port Scanner Version 2.0. From the client machine, run:

1
2
3
4
5
6
REM Test RPC Endpoint Mapper on the server
PortQry.exe -n <server_FQDN_or_IP> -e 135

REM Test a specific dynamic port or a restricted range if configured
REM Example for a restricted range of 5000-5200
PortQry.exe -n <server_FQDN_or_IP> -p TCP -r 5000:5200

LISTENING: The port is open and a service is responding.
NOT LISTENING: The server responded, but no service is on that port.
FILTERED: No response; likely a firewall is blocking the port.

netstat -ano: On the COM+ server, this command shows listening ports and the Process ID (PID) associated with them. You can use it to verify if your COM+ application’s process (often dllhost.exe) is listening on an expected dynamic port.
1 2 3 4
netstat -ano | findstr "LISTENING" REM Find the PID of your dllhost.exe instance in Task Manager REM and then use it in the command below: netstat -ano | findstr "<PID_of_COM_App_Process>"
Network Monitor / Wireshark: For deep-dive analysis, capture network traffic on both the client and server during a connection attempt. Wireshark is a popular, free network protocol analyzer. Filter for traffic between the client and server IPs. Look for:
1. Client SYN to server TCP port 135.
2. Server SYN/ACK back to client.
3. Client RPC bind request to port 135.
4. Server response, potentially indicating the dynamic port for the COM+ application.
5. Client SYN to the server on that dynamic port. If this last SYN receives no SYN/ACK or gets a TCP RST, a firewall is likely blocking the dynamic port.

Key Configuration Best Practices

Always restrict RPC dynamic ports on servers and configure firewalls accordingly. This is a crucial security measure. Refer to the Microsoft documentation on configuring RPC dynamic port allocation.
Apply the Principle of Least Privilege for DCOM permissions. Grant access only to necessary user accounts or groups. Avoid using “Everyone” in production if possible.
Maintain up-to-date DNS records and ensure robust name resolution across domains.
Ensure domain trusts are healthy and correctly configured for the required authentication flow.
Keep systems patched, including Windows Server and client operating systems.

Advanced Considerations

DCOM Hardening (CVE-2021-26414): Microsoft has released patches to harden DCOM security (e.g., KB5004442—Manage changes for Windows DCOM Server Security Feature Bypass (CVE-2021-26414)). These changes enforce higher authentication levels (like Packet Integrity). If older clients or misconfigured DCOM security settings don’t meet these requirements, connections can fail. You might see specific DCOM error events in the System log on the server.
Network Address Translation (NAT): DCOM generally does not work reliably across NAT that changes IP addresses, as it embeds IP addresses in its marshaling packets. If NAT is involved, consider VPNs or application redesign.
Selective Authentication over Trusts: If the domain trust is configured with “Selective Authentication,” users from the trusted domain do not automatically have permission to authenticate to all resources in the trusting domain. You must explicitly grant “Allowed to Authenticate” permission on the COM+ server’s computer object (in Active Directory Users and Computers) to the client users/groups.

Conclusion

The RPC_S_SERVER_UNAVAILABLE error in cross-domain COM+ scenarios, particularly with firewalls, usually points to network connectivity issues related to dynamic RPC ports, misconfigured DCOM security, or authentication problems. By systematically checking services, name resolution, firewall rules (especially for TCP 135 and the DCOM dynamic port range), DCOM permissions, and domain trusts, administrators can effectively diagnose and resolve this challenging error. Restricting RPC dynamic ports and ensuring proper DCOM security configurations are key not only to fixing the issue but also to maintaining a secure and robust distributed application environment.

Remember to document your firewall rules and DCOM configurations thoroughly, especially in complex, multi-domain environments. This will save significant time in future troubleshooting efforts.