Personal Blog of Thomas Hampel - Creative Mythbusting in Development and Collaboration

Who am I?

Feeds

Query results for : HTTPServer

How to fix HTTP Server error "Cannot load resource file"- 14 December 2017 - (0) Comments

Thomas Hampel
 14 December 2017

A colleague approached me with a problem where a Domino server did not start up the HTTP task after upgrading to 9.0.1 FP9

After upgrading a Domino server on Windows from 9.0.1 FP8 to FP9 the http task no longer starts up.
HTTP Server: Cannot load resource file. Exiting
Image:How to fix HTTP Server error "Cannot load resource file"
For a different server the same FP9 installation worked fine.
Although the problem is not new, it seems nobody investigated the root cause yet...

Oliver Busse blogged about this problem in combination with FP3 install on Windows and IBM Technote 1298729 indicates that this problem is caused by a missing file: "Similar behavior can be seen in the Windows environment if the nstrings.dll file located in the Domino folder is damaged or missing."

In our case this file did exist and was not the root cause of the problem.

We wanted to avoid reinstalling the server from scratch as done by Oliver.
So we compared the program directory of the server with another server where FP9 has been installed succesfully and found the file "nrshttp.dll" is missing.

The solution is rather simple: just copy the file from a working server...
Image:How to fix HTTP Server error "Cannot load resource file"
then start up the HTTP task and get yourself a coffee.

References

Domino Security - Disable HTTPEnableConnectorHeaders NOW- 9 November 2015 - (1) Comments

Thomas Hampel
 9 November 2015

There is a seucrity issue with Domino which allows anybody to gain access without authentication.
Jesper Kiaer wrote about this problem before in his blog post ( Part1 and Part2 ) and also created a video showing the problem.

If the Notes.ini variable HTTPEnableConnectorHeaders is set to 1, an attacker just needs to pass the user name he wants to be within a request header to get unauthorized access to Domino servers.
This notes.ini variable is referenced in the product documentation as well as in this technote for configuring Domino servers behind an IIS reverse proxy.

So there is a good chance that some people have enable this variable in production.
None of the Domino servers I have checked was affected, however I was able to reproduce the findings and can confirm it is working as described even with Domino 9.0.1 with latest fixes installed.

Steps to reproduce
  • Add the Notes.ini variable "HTTPEnableConnectorHeaders=1" to the Notes.ini of the Domino server
    Remark: This will make the server insecure.
  • Restart the HTTP task
  • Use Firefox and install this plugin => https://addons.mozilla.org/en-US/firefox/addon/modify-headers/
  • Restart Firefox for the plugin to be initialized
  • In Firefox, open the configuration of the new plugin
    Image:Domino Security - Disable HTTPEnableConnectorHeaders NOW
  • Add a new header called $WSRU with the desired username / shortname as available in the target environment
    Image:Domino Security - Disable HTTPEnableConnectorHeaders NOW
    Save + Enable the configuration
  • Start the Plugin
    Image:Domino Security - Disable HTTPEnableConnectorHeaders NOW
  • Navigate to an existing Domino server resource, e.g. https://your-domino-server.your-domain.com/mail/username.nsf
Surprise, surprise... you now have access rights of the user name you have specified in the request header, in my case thats PaulSmith.
Just imagine what can be done when using the name of an administrator...

How to fix it?

Well, as simple as removing the Notes.ini variable in question, using the following two commands at the Domino server console:
set config HTTPEnableConnectorHeaders=0
tell http restart

Of course you would use a configuration document in production to keep your Notes.ini under control.

References:

IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client- 22 September 2014 - (1) Comments

Thomas Hampel
 22 September 2014

Following up on David's post, here is a WIMP's Guide to get a GUI version of an iKeyman which is supproting the CMS format that is used by the IBM HTTP Server.
iKeyman is actually part of your Notes Client, it is available in "/jvm/bin/ikeyman.exe" but this version does not provide support for CMS keystores unfortunately.
Image:IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client
So what does it take to add support for the CMS format
There is a (much) longer method to get the same done by downloading a specific version of ikeyman which includes CMS support... but this I'll explain at the end of this post.

Here is the short version:

What you need:
  • Notes Client (which you should already have installed)
  • Text editor of your choice

Actions:
  • Edit the file /jvm/lib/security/java.security
  • Find the list of security providers, e.g. by searching for "security.provider", which should look like this:
    security.provider.1=com.ibm.jsse2.IBMJSSEProvider2
    security.provider.2=com.ibm.crypto.provider.IBMJCE
    security.provider.3=com.ibm.security.jgss.IBMJGSSProvider
    security.provider.4=com.ibm.security.cert.IBMCertPath
    security.provider.5=com.ibm.security.sasl.IBMSASL
    security.provider.6=com.ibm.xml.crypto.IBMXMLCryptoProvider
    security.provider.7=com.ibm.xml.enc.IBMXMLEncProvider
    security.provider.8=org.apache.harmony.security.provider.PolicyProvider
    security.provider.9=com.ibm.security.jgss.mech.spnego.IBMSPNEGO
  • Append one new line at the end of this list, where [X] is the next integer value available
    security.provider.[X]=com.ibm.security.cmskeystore.CMSProvider
  • Save

So it should look like this:
{...}
#
# List of providers and their preference orders (see above):
#
security.provider.1=com.ibm.jsse2.IBMJSSEProvider2
security.provider.2=com.ibm.crypto.provider.IBMJCE
security.provider.3=com.ibm.security.jgss.IBMJGSSProvider
security.provider.4=com.ibm.security.cert.IBMCertPath
security.provider.5=com.ibm.security.sasl.IBMSASL
security.provider.6=com.ibm.xml.crypto.IBMXMLCryptoProvider
security.provider.7=com.ibm.xml.enc.IBMXMLEncProvider
security.provider.8=org.apache.harmony.security.provider.PolicyProvider
security.provider.9=com.ibm.security.jgss.mech.spnego.IBMSPNEGO
security.provider.10=com.ibm.security.cmskeystore.CMSProvider
{...}

Result:
Voila: CMS support is ready
Image:IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client
Image:IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client

This blog post could end here but I'd like to share what someone would have to do without using the method above:

The (very) long route:
  1. Try downloading IBM HTTP Server from www.ibm.com/software/webservers/httpservers/download or here
  2. Recognize this website only offers version:  7.0.0.0 dated from 15 May 2009
  3. Try anyway and download and install v7.0.0
  4. Notice iKeyman in this version does not Subject Alternate Support
    Image:IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client
  5. Read Technote 1444027
    Notice it clearly describes "later versions of IBM HTTP Server (IHS), after v7.0, do not require these special steps to enable SAN functionality."
  6. Conclude the Technote 1444027 is wrong or needs update
  7. Try downloading t he IBM HTTP Server trial 8.0 or 8.5
  8. Notice the web site does not offer a download link and only shows a blank page (why?)
  9. Get IBM HTTP Server in the latest version, (which is part of Websphere Application Server 8.5.5 Supplements disks, so its just 3 Gbyte to download)
  10. Install the IBM Installation Manager v1.8  and add the WAS Supplements folder location as a new repository
  11. Install IBM HTTP Server
  12. Launch iKeyman with CMS support and Subject Alternative Support
    Image:IBM HTTP Server - iKeyman with support for CMS is already part of your Notes Client
  13. Notice that iKeyman is actually part of the Notes Client anyway and the same could have been done without all those actions before: priceless

For reference:
Creating a certificate request incl. Subject Alternate Names can be done by using the GSKTool command line version
/opt/IBM/HTTPServer/bin/gskcapicmd -certreq -create -db /opt/IBM/HTTPServer/ssl/keystore.kdb -pw passw0rd -label foobar -dn "cn=www.foobar.ibm.com" -size 2048 -file /tmp/foobar.csr -san_dnsname "www.foobar.ibm.com" -san_emailaddr "admin@us.ibm.com" -san_ipaddr "192.168.1.221"

Further reading:

IBM HTTP Server - SSL Handshake Failed and why it matters to keep a backup of the key ring file- 20 September 2014 - (0) Comments

Thomas Hampel
 20 September 2014

All of a sudden (as usual) clients started to receive SSL expiration warnings when accessing the customer's Connections environment.

A quick look into the log file  /opt/IBM/HTTPServer/logs/error.log confirmed the SSL certificate has expired.:
[Sat Sep 20 22:21:05 2014] [error] [client 10.175.198.62] [8222a80] [30015] SSL0221E: SSL Handshake Failed, Either the certificate has expired or the system clock is incorrect. [10.175.198.62:40028 -> 10.175.220.11:443] [22:31:05.000019743]


Opening the *.kdb file with the gsktool showed the default certificate had expired.
Image:IBM HTTP Server -  SSL Handshake Failed and why it matters to keep a backup of the key ring file
Ok, nothing easier than that... so lets create a new signing request and get this signed by the certificate authority.
Once that is completed we can import the new certificate incl. any trusted roots quickly.


However when you try to import/receive your signed certificate keep the following in mind:
  • You can only import a signed certificate into >exactly< the same *.kdb file which was used to create the certificate request.

Within the iKeyman utility, switch from "Personal certificates" to "Personal Certificate Requests" (sorry, only got screenshots in German available and hope the translation is correct)
Image:IBM HTTP Server -  SSL Handshake Failed and why it matters to keep a backup of the key ring file

normally it would look like this...
Image:IBM HTTP Server -  SSL Handshake Failed and why it matters to keep a backup of the key ring file

but if it looks like the following screenshot, then bad luck... you can not import your signed certificate anymore.
Image:IBM HTTP Server -  SSL Handshake Failed and why it matters to keep a backup of the key ring file

Instead you'll see "The certificate request created for the certificate is not in the key database"  / "Die für das Zertifikat erstellte Zertifikatsanforderung ist nicht in der Schlüsseldatenbank vorhanden."
Image:IBM HTTP Server -  SSL Handshake Failed and why it matters to keep a backup of the key ring file

Now your options to solve this are:
a) find the original key ring file (*.kdb) which was used to create the certificate request
b) create a new certificate request, but this time make sure to keep the *.kdb file
c) set up a self signed certificate - although this is an option, it should not be considered
d) update the SSL directives on your IBM HTTP Server and set SSLClientAuth to "noverify". This will not fix the problem but will at least allow the server to be up and running with an expired certificate until you have obtained a new one.
(...if there are other options, please let me know)

Lessons learned:
  • Keep the key ring file backed up
  • Track certificate expiration time
  • When expired, take action well in advance
  • Even when delegating simple work, supply detailled instructions on least 250 pages

Off topic:
Why do I need to troubleshoot such a very very basic problem myself on a Saturday night?


Further reading:
Thomas Hampel, All rights reserved.