A rejected change to the LDAPS connection handler stops the handler listening until a later change is accepted or the server restarts

[vharseko]

Problem

A dsconfig change to the LDAPS connection handler that the server rejects can still shut LDAPS down. The change is refused, the configuration still says enabled: true, and the handler stops listening on its port. It stays down after the cause is gone, until a later change to the handler is accepted or the server restarts.

It happens whenever the key store of the handler cannot be loaded at the time of the change: the file was replaced by one with another password, it is truncated or corrupt, or it is missing. On a running server, the key store file can change under the server without any configuration change. The Docker image, for instance, copies a renewed key store from SECRET_VOLUME while the server runs (#1087, PR #1100). That PR holds back a new password until the next start for this reason.

The HTTPS connection handler and the administration connector do the same. For the administration connector it is worse: once port 4444 stops listening, dsconfig cannot reach the server any more, so the change that would bring the port back has to go through another channel, such as ldapmodify on cn=Administration Connector,cn=config over plain LDAP, or wait for a restart.

Steps to reproduce

With openidentityplatform/opendj:latest (5.1.2, build 20260717142447), no volumes, LDAPS published on 1638:

docker run -d --name t -p 127.0.0.1:1638:1636 openidentityplatform/opendj:latest
# once the log says "OpenDJ is started":
docker exec t cp /opt/opendj/data/config/keystore /opt/opendj/data/config/keystore.orig
docker exec t keytool -genkeypair -alias other -keyalg RSA -keysize 2048 -validity 30 -dname CN=other -storetype JKS \
  -keystore /opt/opendj/data/config/keystore.new -storepass another -keypass another
docker exec t mv -f /opt/opendj/data/config/keystore.new /opt/opendj/data/config/keystore
docker exec t /opt/opendj/bin/dsconfig set-connection-handler-prop --hostname localhost --port 4444 \
  --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt \
  --handler-name "LDAPS Connection Handler" --set max-request-size:6mb
echo | openssl s_client -connect 127.0.0.1:1638

step	LDAPS on 1636
as set up	serves CN=localhost
key store file replaced by one with another password	still serves CN=localhost (the SSL context was built at start)
dsconfig set-connection-handler-prop … --set max-request-size:6mb	rejected with "Keystore was tampered with, or password was incorrect", and no TLS server on the port
get-connection-handler-prop --property enabled	enabled : true
original key store put back	still no TLS server on the port
the same dsconfig change again	accepted, LDAPS serves CN=localhost again

Error log of the rejected change:

msgID=1527 msg=No usable key was found for 'LDAPS Connection Handler'. Verify the keystore content
msgID=1528 msg=Disabling LDAPS Connection Handler
msgID=277 msg=Stopped listening for new connections on LDAPS Connection Handler 0.0.0.0 port 1636

HTTPS connection handler and administration connector

Same image, with 4444 published on 4445 and 8443 on 8444. The HTTP connection handler is first enabled with use-ssl:true, listen-port:8443 and the JKS key manager provider. Then config/keystore (for HTTPS) or config/admin-keystore (for the administration connector) is replaced by a store with another password, as above.

step	HTTPS on 8443	administration connector on 4444
as set up	serves CN=localhost	serves the administration connector certificate
key store file replaced	still serves it	still serves it
a change to the handler (--set keep-stats:false / set-administration-connector-prop --set ssl-protocol:TLSv1.2 --set ssl-protocol:TLSv1.3)	rejected, no TLS server on the port, enabled : true	rejected, no TLS server on the port
original key store put back	still no TLS server on the port	still no TLS server on the port; dsconfig fails with Unable to connect to the server at "localhost" on port 4444
the same change again	accepted through dsconfig, HTTPS back	accepted only through ldapmodify on cn=Administration Connector,cn=config over LDAP 1389, port 4444 back

Error log of the rejected change to the administration connector. Its ssl-cert-nickname is admin-cert, so the alias branch fires as well:

msgID=1527 msg=No usable key was found for 'Administration Connector'. Verify the keystore content
msgID=1528 msg=Disabling Administration Connector
msgID=1526 msg=The key with alias '[admin-cert]' was not found for 'Administration Connector'. Verify that the keystore is properly configured
msgID=1528 msg=Disabling Administration Connector
msgID=277 msg=Stopped listening for new connections on Administration Connector 0.0.0.0 port 4444

Cause

The acceptance check of a change mutates the live handler.

• LDAPConnectionHandler2.isConfigurationChangeAcceptable (:626-630) calls isConfigurationAcceptable on the running instance, which builds an SSL context for the proposed configuration: createSSLEngine(config, createSSLContext(config)) (:580-591).
• createSSLContext (:947-1000) calls disableAndWarnIfUseSSL(config) when the key manager provider is missing, holds no usable key, or lacks the configured alias (:952-977). disableAndWarnIfUseSSL (:940-945) sets the field enabled = false on the handler.
• FileBasedKeyManagerProvider.containsAtLeastOneKey (:176-186) returns false for any failure to load the store, a wrong password included, so an unreadable key store takes that branch.
• Then getKeyManagers() throws, the change is rejected, and applyConfigurationChange never runs. That method is the only place that sets enabled back from the configuration (:293).
• The handler thread run() (:731-751) sees !enabled, stops the listener, and sleeps, checking again every second.

The legacy org.opends.server.protocols.ldap.LDAPConnectionHandler has the same code (:720, :1294-1344). The default configuration does not use it.

The same check reaches the running handler by other paths too:

• The administration connector is an LDAPConnectionHandler2. AdministrationConnector.isConfigurationChangeAcceptable (:168-174) calls isConfigurationAcceptable on it, so a rejected change to the connector disables it the same way. FileBasedKeyManagerProvider.containsKeyWithAlias (:106-127) also returns false when the store cannot be loaded, so with ssl-cert-nickname set the alias branch disables the handler as well.
• ConnectionHandlerConfigManager.isConfigurationChangeAcceptable (:283-288) calls isJavaClassAcceptable (:325-341), which runs isConfigurationAcceptable on the registered, running handler, not on a new instance. So each change to a handler entry runs the check on the running handler twice.
• HTTPConnectionHandler.isConfigurationAcceptable (:478-511) builds an SSL context through createSSLEngineConfigurator (:500). Its createSSLContext (:877-930) sets enabled = false directly in the same three branches. run() (:632-664) then stops the embedded HTTP server.

The branches in createSSLContext are meant for the start of the handler, where a handler without a usable key is disabled on purpose. During the check of a change they have side effects on the running handler.

Expected

A rejected change leaves the handler as it was: listening, with the SSL context it had. The reason the change was rejected goes back to the client, as it does now.

Possible fix

Keep isConfigurationAcceptable free of side effects. For example, pass createSSLContext a flag saying whether it runs for a check, and have its branches touch enabled only when it does not. A key store that cannot be loaded still makes getKeyManagers() throw, so the change is still rejected with the reason it has today. The start of the handler and applyConfigurationChange keep disabling the handler as they do now. Apply the same change to the legacy LDAPConnectionHandler and to HTTPConnectionHandler. The administration connector is covered by the change to LDAPConnectionHandler2.

A test: start an LDAPS handler (and an HTTPS one), replace its key store file with one that cannot be loaded, submit a change to the handler, and check that it is rejected and that the handler still accepts TLS connections. Run it with and without ssl-cert-nickname, so that both the no-key branch and the alias branch are covered.

Related: #1095 / PR #1101 (the server loads a changed key store file without a restart) and #1105. With #1101, a key store renewed together with its PIN file loads, so the password case no longer reaches this path. A key store that cannot be loaded at all still does: containsAtLeastOneKey() still returns false and getKeyManagers() still throws. There the handler keeps what it last loaded, as #1101 intends, until any change to the handler is submitted.