Integration tests – Using haproxy for communicating with SSL enabled containers

While writing a few Integration tests(ITs), I ran into a scenario where my test had to make a few calls to the SSL enabled custom keycloak container. I had to communicate over HTTPS with the keycloak container since HTTP was disabled. I also wanted my tests to be compatible with any future customizations to the keycloak image. The keycloak image was using a self-signed certificate.

For the test code to be able to communicate with SSL enabled container, it was imperative that we either add code in our test to manage SSL issues(usually creating a Keystore programmatically) or worse install the self-signed certificate in the test code’s JVM.

Handling SSL in the test code would have been an added nuisance. With the intent of keeping the test code to only deal with the functionality under test(and not the infrastructure), I tried out using haproxy to handle ssl connection. The below picture outlines the basic implementation –

Keystore shared volume

A self-signed certificate was generated and placed onto a shared docker volume. Keycloak and Haproxy containers were made to refer to this shared Keystore volume.

Haproxy configuration – The configuration routes the HTTP traffic to HTTPS endpoint for keycloak i.e. keycloak-backend configuration will encrypt the outgoing traffic to keycloak with the same certificate(present on the shared volume) which is recognized by the keycloak container.

#---------------------------------------------------------------------
# Keycloak
#---------------------------------------------------------------------

frontend integration-tests-keycloak
    bind *:8444
    mode http
    default_backend keycloak-backend

backend keycloak-backend
    mode http
    # https backend
    server keycloak keycloak:8444 check ca-file /test-keystore/integration-tests.pem ssl

In conclusion, SSL management stays out of the test code and is only limited to the test infrastructure. Infrastructure here refers to docker maven plugin https://github.com/fabric8io/docker-maven-plugin

Haproxy image configuration

<image>
  <alias>haproxy</alias>
  <name>${project.artifactId}-haproxy:${project.version}</name>
  <build>
      <from>${dockerHubPublic}/library/haproxy:2.2.3</from>
      <assembly>
          <basedir>/</basedir>
          <inline>
              <fileSets>
                  <fileSet>
                      <directory>${project.basedir}/src/test/config/haproxy</directory>
                      <outputDirectory>usr/local/etc/haproxy</outputDirectory>
                      <lineEnding>unix</lineEnding>
                      <includes>
                          <include>haproxy.cfg</include>
                      </includes>
                  </fileSet>
              </fileSets>
          </inline>
      </assembly>
  </build>
  <run>
      <ports>
          <port>${keycloak.proxy.port}:8444</port>
      </ports>
      <links>
          <link>keycloak</link>
      </links>
      <volumes>
          <from>
              <image>keystore</image>
          </from>
      </volumes>
  </run>
</image>

Keycloak image configuration

<image>
  <alias>keycloak</alias>
  <name>${docker-image-registry}custom-keycloak-base:1.8.0-SNAPSHOT</name>
  <run>
      <ports>
          <port>${keycloak.https.port}:8444</port>
      </ports>
      <env>
          <DB_VENDOR>POSTGRES</DB_VENDOR>
          <KEYCLOAK_DATABASE_HOST>postgres</KEYCLOAK_DATABASE_HOST>
          <KEYCLOAK_DATABASE_PORT>5432</KEYCLOAK_DATABASE_PORT>
          <KEYCLOAK_DATABASE_NAME>${postgres.db}</KEYCLOAK_DATABASE_NAME>
          <KEYCLOAK_DATABASE_APPNAME>Keycloak</KEYCLOAK_DATABASE_APPNAME>
          <KEYCLOAK_DATABASE_USERNAME>${postgres.user}</KEYCLOAK_DATABASE_USERNAME>
          <KEYCLOAK_DATABASE_PASSWORD>${postgres.pass}</KEYCLOAK_DATABASE_PASSWORD>
          <KEYCLOAK_USER>${keycloak.admin.username}</KEYCLOAK_USER>
          <KEYCLOAK_PASSWORD>${keycloak.admin.password}</KEYCLOAK_PASSWORD>
          <SSL_KEYSTORE_JKS>integration-tests.p12</SSL_KEYSTORE_JKS>
          <SSL_CA_CRT>integration-tests.pem</SSL_CA_CRT>
          <SSL_CERT_ALIAS>integration-tests-alias</SSL_CERT_ALIAS>
          <SSL_KEYSTORE_PASSWORD>changeit</SSL_KEYSTORE_PASSWORD>
          <KEYCLOAK_LOGLEVEL>DEBUG</KEYCLOAK_LOGLEVEL>
          <ROOT_LOGLEVEL>DEBUG</ROOT_LOGLEVEL>
      </env>
      <volumes>
          <from>
              <image>keystore</image>
          </from>
      </volumes>
      <dependsOn>
          <container>postgres</container>
      </dependsOn>
      <links>
          <link>postgres</link>
      </links>
      <wait>
          <http>
              <url>https://${docker.host.address}:${keycloak.https.port}/auth/realms/master</url>
              <allowAllHosts>true</allowAllHosts>
          </http>
          <time>180000</time>
          <shutdown>500</shutdown>
      </wait>
  </run>
</image>

Keystore image configuration

<image>
    <alias>keystore</alias>
    <name>${project.artifactId}-keystore:${project.version}</name>
    <build>
        <from>${dockerHubPublic}/library/java:8</from>
        <runCmds>
            <runCmd>mkdir /test-keystore</runCmd>
            <runCmd>$JAVA_HOME/bin/keytool -genkey -noprompt -alias integration-tests-alias -dname "CN=myname, OU=myorganisational.unit, O=myorganisation, L=mycity, S=myprovince, C=GB" -keystore /test-keystore/integration-tests.jks -storepass changeit -keypass changeit -keyalg RSA</runCmd>
            <!--converting java keystore jks to pem to be used in SSL_CA_CRT-->
            <runCmd>$JAVA_HOME/bin/keytool -importkeystore -srckeystore /test-keystore/integration-tests.jks -srcstorepass changeit -deststorepass changeit -srckeypass changeit -destkeypass changeit -destkeystore /test-keystore/integration-tests.p12 -srcalias integration-tests-alias -srcstoretype jks -deststoretype pkcs12</runCmd>
            <runCmd>openssl pkcs12 -in /test-keystore/integration-tests.p12 -out /test-keystore/integration-tests.pem -password pass:changeit</runCmd>
        </runCmds>
        <volumes>
            <volume>/test-keystore</volume>
        </volumes>
    </build>
</image>