• Introduction
  • A Word About Databases 
  • Prerequisites
• Docker - Command line to Compose
  • Docker Networking
  • Properties Files 
  • The Enterprise Dashboard Container
    • Adding the License File
    • Running from Docker Compose
    • Summary
• Moving Towards a "real" Environment 
  • Start the Registry...
  • the Broker...
  • the Portal...
  • the Coordinator...
  • the Simulator...
  • And finally the Virtual Service Environment, with a port range of 8000->8100 exposed for services
  • Moving to MySQL 
    • Docker Compose to the Rescue
  • Building On Our Work So Far
    • Think of the Environment!
    • The Portal
    • Some Notes
      • Port Translation
      • Running Workstation
        • Windows 
        • LINUX/UNIX
      • Data in the Database
    • Sacrificing Flexibility for Convenience
  • In Conclusion
• Where to Obtain Further Assistance.
  • Docker
  • Running DevTest In Docker
  • If you find a defect in DevTest
• Attachments

Introduction

As acceptance of Docker grows, so does running DevTest inside containers. DevTest ships with build scripts to create containers, and the DevTest documentation contains simple examples of running the containers for illustrative purposes.

Since Docker is a separate product. and there are many ways to configure a containerised environment, the level of documentation is appropriate for the DevTest product - further information on running DevTest in this way should be obtained via the DevTest or Docker communities, as appropriate, and, in the event of a defect being found in DevTest then CA Product Support should be contacted. 

This document, therefore, is being made available to the DevTest community in the hope that is proves useful - it will cover running each component from the command line and will progress to an automated deployment using docker-compose

A Word About Databases 

Through the course of this document, we will use both the supplied Derby database that ships with DevTest, and also MySQL. The use of Derby is purely for purposes of demonstrating the principles involved, and is not supported. In order to be running a supported environment you must be using one of supported databases, as described in the DevTest documentation.

In the cases where we use MySQL in a container, the data therein will be available only until the container is deleted. Whilst this may be acceptable for the test and service execution data, it is not acceptable for the Enterprise Dashboard. In order to maintain compliance with your license the data from the Enterprise Dashboard must be maintained and remain available. 

It is therefore recommended that a containerised database not be used for the Enterprise Dashboard, and that a standalone instance be used. This instance may be shared by the other components, whether in containers or not, as long as separate schema are used.

The final configuration files referenced will be attached to this document as a zip file.

The time has come, the author said, to speak of many things. Of images, containers and startup scripts, of files and config strings

Prerequisites

The following need to be available before progressing 

• A working knowledge of Docker and its commands 
• The base images have been built according to the DevTest documentation
• Docker and Docker Compose have been installed (see Install Docker Compose | Docker Documentation)
• A valid devtest license file (devtestlic.xml) is available if an Enterprise Dashboard is required
• A copy of the mysql connector jar is available (these examples will assume that this is named mysql-connector.jar)
• A MySQL database is available and configured to hold the Enterprise Dashboard data (see above)

You should create a directory that can be used as a workspace.

Docker - Command line to Compose

Docker Networking

In order to simplify the initial examples, host-mode networking will be used at first. This has the advantage of all docker containers running in the docker host's network stack, and communications between components are therefore possible without the added complexity of an additional network being used. 

Later, when the Docker Compose examples are provided, Docker networking and name resolution will be used. 

Properties Files 

Those familiar with DevTest will be aware that it is normally configured by the use of properties and vmoptions files. When running in docker this is avoided as far as possible so that docker images remain general purpose. 

Whilst DevTest properties files will not be used in these examples, their use is possible should you wish to include them in an image by modifying the dockerfiles and rebuilding the image. Please use whichever approach bests suits your environment.

The Enterprise Dashboard Container

The first item that needs to be run in any DevTest installation is the Enterprise Dashboard - it may well be that you already have this running in your environment, in which case this should be used, and this section need not be followed 

Using the basic images already created, we will run an Enterprise Dashboard from the command line, initially using the Derby database and then connecting it to an external MySQL instance.

Adding the License File

In order for the Enterprise Dashboard to run, it must be configured with a valid license file (devtestic.xml) in the image. In the workspace directory, create a sub-directory called "dashboard", and change directory to there. 

Create a file called "Dockerfile" that contains the following:-

FROM devtest/dradis-base:latest
ADD devtestlic.xml /opt/devtest/devtestlic.xml

This file contains directives for Dockers image build process:- to actually build the image, type

docker build -t dradis-licensed:latest .

This command builds a image called dradis-licensed and 'tags' it as the latest version, with the build taking place in the current directory. The dockerfile created before is the default input to this build command

You have now created a DevTest Enterprise Dashboard with a valid license. This can be run from the command line with 

docker run -d --net host -p 1506:1506 dradis-licensed:latest

This will run an Enterprise Dashboard, exposing port 1506 on the localhost (see the Docker documents to clarify this if you are not sure). 

Remember, however, that we have not configured a database yet, so Derby will be used, and this is not supported - let's now move to a supported configuration!

Firstly, delete the container you have just created with 

docker rm <your container id>

and the image you created with 

docker rmi dradis-licensed:latest

since neither are now required.

Obtain a copy of the MySQL connector jar - for the purposes of this document we will use the name mysql-connnector.jar for this file. Place this file alongside Dockerfile and edit Dockerfile to read 

FROM devtest/dradis-base:latest
ADD devtestlic.xml /opt/devtest/devtestlic.xml
ADD mysql-connector.jar /opt/devtest/lib/dradis/mysql-connector.jar

And run the command 

docker build -t dradis-licensed-mysql:latest .

This will create a MySQL-ready image ready to be run - running it, however, requires a little more complexity than the previous example since we will configure the database from the command line.

Assuming your database runs on host DBServer is called dashboard, that the database user is DevTestUser and the password is DevTestPass. the dashboard should be started as 

docker run -d --net host -p 1506:1506 dradis-licensed-mysql:latest \
  /opt/devtest/bin/EnterpriseDashboard \
 -J-Ddradis.db.driverClass=com.mysql.jdbc.Driver \
 -J-Ddradis.db.url=jdbc:mysql://DBServer:3306/dashboard \
 -J-Ddradis.db.user=DevTestUser -J-Ddradis.db.password=DevTestPass \
 -J-Ddradis.db.internal.enabled=false

This command supplies all of the necessary database connection information and exposes port 1506, as before.

Running from Docker Compose

It may well be desirable to deploy your Enterprise Dashboard via Docker Compose, which can be used to build and run your dashboard. Should you wish to run the dashbaord in this way, please create a docker-compose.yml (or docker-compose.yaml) file in the same directory as the other files, and add the following:-

version: '3'
services:
  dashboard:
    build:
      context: .
    image: dradis-licensed-mysql:latest
    ports:
    - 1506:1506
    command: /opt/devtest/bin/EnterpriseDashboard -J-Ddradis.db.driverClass=com.mysql.jdbc.Driver -J-Ddradis.db.url=jdbc:mysql://DBServer:3306/database -J-Ddradis.db.user=DevTestUser -J-Ddradis.db.password=DevTestPass -J-Ddradis.db.internal.enabled=false

This contains all the commands necessary to build and run the Enterprise Dashboard when the command

docker-compose up

is executed.

As a final enhancement, create a further sub-directory named patches in the same directory, and create a text file (perhaps readme.txt, content is not important) in this sub-directory so that it is not empty. Now modify the Dockerfile to be 

FROM devtest/dradis-base:latest
ADD devtestlic.xml /opt/devtest/devtestlic.xml
ADD mysql-connector.jar /opt/devtest/lib/dradis
ADD patches/* /opt/devtest/lib/patches

Now, executing 

docker-compose build 

will apply any patches that you place in the patches directory automatically.

At this time, your directory structure should resemble

workspace/
   dashboard/
      devtestlic.xml 
      docker-compose.yml
      Dockerfile
      mysql-connector.jar 
      patches/
          readme.txt

Summary

We have created an Enterprise Dashboard image that contains both the license file and the MySQL JDBC driver - this has then been run as a standalone container with host networking, and also from docker-compose. Docker compose will build the container as required and will patch the image if patches have been provided. 

Moving Towards a "real" Environment 

Now that we have explored what Docker and Docker Compose can do when creating an enterprise dashboard, we can consider how to run DevTest itself.

To do this we need to run the following components inside containers

• Registry
• Broker
• Portal
• Coordinator
• Simulator
• VirtualServiceEnvironment

We will configure and run each one on the command line, and then create a Docker Compose example that performs the same function as the startdefservers script that is shipped with DevTest

Under the workspace, create a folder called startservers

workspace/

dashboard/
startservers/

We can simply start the up the environment using host networking and using the unsupported Derby database without further ado

Start the Registry...

docker run -d --net host -p 2010:2010 -p 1505:1505 -p 1528:1528 \
-e LISA_MORE_VM_PROPS='-Ddevtest.enterprisedashboard.host=localhost' \
devtest/registry-broker-base:latest \
/opt/devtest/bin/Registry -n tcp://localhost:2010/Registry

the Broker...

docker run -d --net host -p 2010:2010 -p 1505:1505 -p 1528:1528 \
-e LISA_MORE_VM_PROPS='-Ddevtest.enterprisedashboard.host=localhost' \
devtest/registry-broker-base:latest \
/opt/devtest/bin/Broker

the Portal...

docker run -d --net host -p 1507:1507 devtest/portal-base:latest

the Coordinator...

docker run -d --net host -p 2011:2011 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=org.apache.derby.jdbc.ClientDriver -Dlisadb.pool.common.url=jdbc:derby://localhost:1528/database/lisa.db -Dlisadb.pool.common.user=rpt -Dlisadb.pool.common.user=rpt' \
devtest/servers-base:latest \
/opt/devtest/bin/CoordinatorServer

the Simulator...

docker run -d --net host -p 2014:2014 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=org.apache.derby.jdbc.ClientDriver -Dlisadb.pool.common.url=jdbc:derby://localhost:1528/database/lisa.db -Dlisadb.pool.common.user=rpt -Dlisadb.pool.common.user=rpt' \
devtest/servers-base:latest \
/opt/devtest/bin/Simulator

And finally the Virtual Service Environment, with a port range of 8000->8100 exposed for services

docker run -d --net host -p 2013:2013 -p 8000-8100:8000-8100 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=org.apache.derby.jdbc.ClientDriver -Dlisadb.pool.common.url=jdbc:derby://localhost:1528/database/lisa.db -Dlisadb.pool.common.user=rpt -Dlisadb.pool.common.user=rpt' \
devtest/servers-base:latest \
/opt/devtest/bin/VirtualServiceEnvironment

This will start an environment using Derby and illustrate the relevant configuration options. As ever, however, running Derby is not supported.

There are some things of note here - firstly, the Coordinator, Simulator and Virtual Service Environment share a common image and secondly that it is necessary to pass the database connections to each component. This is required since the registry does not obtain these values from the site.properties file, as would be normal and they are therefore not shared by the registry. 

Moving to MySQL 

Since is is desirable to use a supported database, we will now add MySQL support

Under the workspace, create a folder called startservers

workspace/
  dashboard/
  startservers/

In this folder, please place a copy of your mysql-connector.jar, and create a text file called Registry.dockerfile

In the same way as was done with the Enterprise Dashboard, the MySQL connector needs to be added to the image, so the file will have the content of

FROM devtest/registry-broker-base:latest
ADD mysql-connector.jar /opt/devtest/lib/shared/mysql-connector.jar

To build this image use

docker build -f Registry.dockerfile -t devtest/registry-broker-mysql:latest .

As with the Enterprise Dashboard, this will create a new Docker image that contains the MySQL connector.

In a similar manner, create a Portal.dockerfile that contains

FROM devtest/portal-base:latest
ADD mysql-connector.jar /opt/devtest/lib/shared/mysql-connector.jar

and build this with

docker build -f Portal.dockerfile -t devtest/portal-mysql:latest .

and a Servers.dockerfile with

FROM devtest/servers-base:latest
ADD mysql-connector.jar /opt/devtest/lib/shared/mysql-connector.jar

that is built with

docker build -f Servers.dockerfile -t devtest/servers-mysql:latest .

Assuming, as before, that we have an external database at DBServer, with DevTestUser/DevTestPass as credentials, but this time named registry, then we could start all of these services with host networking as

docker run -d --net host -p 2010:2010 -p 1505:1505 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass -Dlisadb.internal.enabled=false -Ddevtest.enterprisedashboard.host=localhost' \
devtest/registry-broker-mysql:latest /opt/devtest/bin/Registry -n tcp:localhost:2010/Registry

docker run -d --net host -p 2009:2009 -p 1508:1508 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass' \
devtest/registry-broker-mysql:latest /opt/devtest/bin/Broker

docker run -d --net host -p 1507:1507 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass' \
devtest/portal-mysql:latest /opt/devtest/bin/Portal

docker run -d --net host -p 2011:2011 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass' \
devtest/servers-mysql:latest /opt/devtest/bin/Coordinator -m tcp://localhost:2010/Registry

docker run -d --net host -p 2014:2014 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass' \
devtest/servers-mysql:latest /opt/devtest/bin/Simulator -m tcp://localhost:2010/Registry

docker run -d --net host -p 2011:2011 \
-e LISA_MORE_VM_PROPS='-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://DBServer:3306/registry -Dlisadb.pool.common.user=DevTestUser -Dlisadb.pool.common.password=DevTestPass' \
devtest/servers-mysql:latest /opt/devtest/bin/VirtualServiceEnvironment -m tcp://localhost:2010/Registry

but, as can be seen, this is a task that is crying out to be automated.

Docker Compose to the Rescue

Let us rewind to the unsupported Derby configuration when we first ran the DevTest components as services. Whilst considerably simpler than the last example, there was still a lot of work to do to start and stop services. So, let's take the Docker Compose example that we had for the Enterprise Dashboard, and extend it.

So, still in the startservers directory, create a docker-compose.yml file and place the below into it to create the functional equivalent of startdefservers.sh. and opens ports 8000-8100 for virtual services to run. It is assumed that the Enterprise Dashboard is running on the same machine

version: '3'
services:

 registry:
   image: devtest/registry-broker-base:latest
   network_mode: host
   ports:
    - 2010:2010
    - 1505:1505
    - 1528:1528
   command: /opt/devtest/bin/Registry
   environment:
    LISA_MORE_VM_PROPS: -Ddevtest.enterprisedashboard.host=localhost -Ddevtest.enterprisedashboard.port=1506

 broker:
   depends_on:
    - "registry"
   image: devtest/registry-broker-base:latest
   network_mode: host
   ports:
    - 1508:1508
    - 2009:2009
    - 2099:2099
   command: /opt/devtest/bin/Broker

 portal:
   depends_on:
    - "registry"
   image: devtest/portal-base:latest
   network_mode: host
   ports:
    - 1507:1507
   command: /opt/devtest/bin/Portal

 coordinator:
   depends_on:
    - "registry"
   image: devtest/servers-base:latest
   network_mode: host
   ports:
    - 2011:2011
   command: /opt/devtest/bin/CoordinatorServer

 simulator:
   depends_on:
    - "registry"
    - "coordinator"
   image: devtest/servers-base:latest
   network_mode: host
   ports:
    - 2014:2014
   command: /opt/devtest/bin/Simulator
   environment:
    LISA_MORE_VM_PROPS: -DANDROID_HOME=""

 VSE:
   depends_on:
    - "registry"
   image: devtest/servers-base:latest
   network_mode: host
   ports:
    - 2013:2013
    - 8000-8100:8000-8100
   command: /opt/devtest/bin/VirtualServiceEnvironment

This may seem like a lot of work, but now the entire environment can be started by simply running

docker-compose up

whenever it is needed. 

Building On Our Work So Far

We have been able to run an entire environment from docker, but it is still not a supported configuration, because it is using Derby. Neither is the system flexible, since it maps ports to the host that would prevent more than once instance running. We need to take the work we have done, and improve it - adding a database per registry and the ability to reliably change ports. We will now construct a fully parameterised DevTest environment in Docker - in effect taking things to the extreme. Whilst this environment will be fully functional and will illustrate all of necessary aspects of DevTest and Docker, it may not be what you wish to deploy - please see the Some Notes section following, especially with regard to running Workstation

Think of the Environment!

Docker Compose allows for variables to be defined in environment files, and the defined values to be referenced in the docker compose file.

The default environment file is named .env - we should create one now, and we can include the configuration information that the registry needs to reach the Enterprise Dashboard.

# Environment file for DevTest under Docker Compose
#
# Enterprise Dashboard information
ED_HOST=localhost
ED_PORT=1506
ED_USE_SSL=false

Once this file has been saved, we will modify the docker compose file to use it. Change the registry service section in docker-compose.yml to be

registry:
 image: devtest/registry-broker-base:latest
 network_mode: host
 ports:
  - 2010:2010
  - 1505:1505
  - 1528:1528
 command: /opt/devtest/bin/Registry
 environment:
   LISA_MORE_VM_PROPS: -Ddevtest.enterprisedashboard.host=${ED_HOST} -Ddevtest.enterprisedashboard.port=${ED_PORT} -Ddevtest.enterprisedashboard.https.enabled=${ED_USE_SSL}

We have now moved the configuration data for the Enterprise Dashboard to to the environment file - by moving more of the configuration information there we will be able to control the DevTest configuration from a single point without needing to edit the docker compose file.

At this point, we should mention version control - and the fact that it can save a great deal of trouble as such files are developed.

Having demonstrated the use of variables in the configuration, we now need to move away from Derby to a supported database, and move away from host networking.

Docker Compose is about to make this easy for us!

Edit the docker-compose.yml file and remove all the lines that say "network_mode: host" - then add the following service before the registry, so that the start of the file reads as

version: '3'

services:

database:
  build:
  context: .
  dockerfile: mysql.dockerfile
  image: mysql:utf8
  ports:
   - ${DB_PORT}:${DB_PORT}
  environment:
    MYSQL_DATABASE: ${DATABASE}
    MYSQL_USER: ${DB_USER}
    MYSQL_PASSWORD: ${DB_PASS}

registry:
 ...
 

This section defines a new service called "database" that will be a MySQL server. The image for this file is called mysql.utf8 - if this image does not exist it will be created. As can be seen from the above, we will need to create a mysql.dockerfile and we have some configuration valued to add to the .env file

Create the mysql.dockerfile and put the following content in it

FROM mysql/mysql-server:5.7
ADD my.cnf /etc/my.cnf

We therefore require a my.cnf file - this is needed in order to ensure that the MySQL database is configured to support UTF8, as required by DevTest

This is again a text file, and should be created with the content of

# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
skip-host-cache
skip-name-resolve
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
secure-file-priv=/var/lib/mysql-files
user=mysql

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

collation_server=utf8_unicode_ci
character_set_server=utf8

port = 4306

Now let's look at the environment variables that are spedified in the environment section of the service

MYSQL_DATABASE: ${DATABASE}
MYSQL_USER: ${DB_USER}
MYSQL_PASSWORD: ${DB_PASS}
 
When MYSQL_DATABASE, MYSQL_USER and MYSQL_PASSWORD are all specified, the container will create the named database and user and grant full access on first start. This means that when the container is first run the user and database will be automatically created. On subsequent runs the database is already present

In the .env file, therefore, add the following

#
# MySQL Database configuration
# Note that changing the MySQL port requires a change in my.cnf and a container rebuild

#
DATABASE=DT101
DB_USER=DTUSER
DB_PASS=DTPASS
DB_PORT=4306
USE_SSL=false

This contains not only the above information but also the port to be used and the flagto specifiy whether a SSL encryped connection should be used.

Now that we have a database available, we can configure the registry to use it

version: '3'

services:

database:
 build:
 context: .
 dockerfile: mysql.dockerfile
 image: mysql:utf8
 ports:
 - ${DB_PORT}:${DB_PORT}
 environment:
 MYSQL_DATABASE: ${DATABASE}
 MYSQL_USER: ${DB_USER}
 MYSQL_PASSWORD: ${DB_PASS}

registry:
 build:
 context: .
 dockerfile: Registry.dockerfile
 image: devtest/registry-broker-mysql:latest
 ports:
 - ${REG_PORT}:${REG_PORT}
 - ${REG_API_EXT}:${REG_API}
 - ${BROKER_PORT}:${BROKER_PORT}
 - ${BROKER_CONSOLE_EXT}:${BROKER_CONSOLE}
 command: /opt/devtest/runRegistryBroker.sh ${REG_HOST} ${REG_PORT}
 environment:
 HOSTNAME: ${REG_HOST}
 LISA_MORE_VM_PROPS: -Dlisa.pathfinder.broker.port=${BROKER_PORT} -Dlisa.webserver.port=${REG_API} -Ddevtest.port=${PORTAL_PORT_EXT} -Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:${DB_PORT}/${DATABASE}?useSSL=${USE_SSL} -Dlisadb.pool.common.user=${DB_USER} -Dlisadb.pool.common.password=${DB_PASS} -Ddevtest.enterprisedashboard.host=${ED_HOST} -Ddevtest.enterprisedashboard.port=${ED_PORT} -Ddevtest.enterprisedashboard.https.enabled=${ED_USE_SSL} -Dlisadb.internal.enabled=false
 links:
 - "database"

We appear to have got ahead of ourselves again! In the .environment file, add definitions for the registry and broker ports

#
# ActiveMQ - default 2010
REG_PORT=3010
#
# Hostname - provided only to allow re-purposing
REG_HOST=registry
#
# API Port - default 1505. Translation OK
REG_API=2505
REG_API_EXT=2505
#
# Broker Configuration
#
# ActiveMQ - default 2009
BROKER_PORT=3009
#
# Console - default 2099. Translation OK.
# if you wish to change this value then the rules.xml file must
# be updated and added to the images
BROKER_CONSOLE=2099
BROKER_CONSOLE_EXT=2099

You may also have noticed that we do not appear to be running the correct registry command - when moving away from host based networking, each container will be allocated its own ip address - but the broker and the registry must be on the same machine (or, strictly speaking, the broker needs to contact the registry on localhost).

To allow this, we will create a simple script, and add this to the image. Create runRegistryBroker.sh and insert the following content

#!/bin/bash
# Invoked with three paramaters
# $1 is hostname
# $2 is registry port
/opt/devtest/bin/Registry -n tcp://${1}:${2}/Registry &
sleep 10
/opt/devtest/bin/Broker

This script will start the registry, sleep 10 seconds and then start the broker. Problem solved!

Make the script executable

chmod +x runRegistryBroker.sh

and modify the Registry.dockefile to be

FROM devtest/registry-broker-base:latest
ADD mysql-connector.jar /opt/devtest/lib/shared/mysql-connector.jar
ADD runRegistryBroker.sh /opt/devtest/runRegistryBroker.sh

The Portal

We have a database define, a regisry and a broker - now we must configure the portal

The service definition for the portal becomes

portal:
 depends_on:
   - "registry"
 build:
   context: .
   dockerfile: Portal.dockerfile
 image: devtest/portal-mysql:latest
 ports:
   - ${PORTAL_PORT_EXT}:${PORTAL_PORT}
 command: /opt/devtest/bin/Portal -J-Dregistry.host=${REG_HOST} -J-Dregistry.portal.port=${REG_API} -J-Dregistry.port=${BROKER_PORT} -J-DlisaAutoConnect=tcp://${REG_HOST}:${REG_PORT}/Registry -J-Dlisa.pathfinder.broker.port=${BROKER_PORT} -J-Dphoenix.port=${PORTAL_PORT} -J-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -J-Dlisadb.pool.common.url=jdbc:mysql://database:${DB_PORT}/${DATABASE}?useSSL=${USE_SSL} -J-Dlisadb.pool.common.user=${DB_USER} -J-Dlisadb.pool.common.password=${DB_PASS} 
environment:
   HOSTNAME: portal
 links:
   - "registry"
   - "database"

Here we link it to the registry and the database, and configure it as before. Once more we have a need to add some more configuration to .env to support this - the following definitions should be added

#
# Portal Configuration
#
# Portal - default 1507. Translation OK
PORTAL_PORT=2507
PORTAL_PORT_EXT=2507

Again, the portal image will be created if not present.

The final image to be created by docker-compose is the servers image, which will be common for the Coordinator, the Simulator and the Virtual Service environment

The coordinator service definition in the docker-compose file becomes

coordinator:
 depends_on:
   - "registry"
 build:
 context: .
   dockerfile: Servers.dockerfile
 image: devtest/servers-mysql:latest
 ports:
   - ${COORD_PORT}:${COORD_PORT}
 command: /opt/devtest/bin/CoordinatorServer -n tcp://coordinator:${COORD_PORT}/Coordinator -m tcp://${REG_HOST}:${REG_PORT}/Registry
 environment:
   HOSTNAME: coordinator
   LISA_MORE_VM_PROPS: -Dlisa.pathfinder.broker.port=${BROKER_PORT} -Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:${DB_PORT}/${DATABASE}?useSSL=${USE_SSL} -Dlisadb.pool.common.user=${DB_USER} -Dlisadb.pool.common.password=${DB_PASS} -Ddevtest.enterprisedashboard.host=${ED_HOST} -Ddevtest.enterprisedashboard.port=${ED_PORT}
 links:
 - "registry"
 - "database"

The only new variable here is the port definition for the Coordinator

#
# Coordinator Configuration
#
# ActiveMQ - default 2011
COORD_PORT=3011

The Simulator and the VSE are presented together - the only added complexity is that we need to specify port ranges for the Virtual Service Environment to be able to publish services

simulator:
 depends_on:
   - "coordinator"
 image: devtest/servers-mysql:latest
 ports:
   - ${SIM_PORT}:${SIM_PORT}
 command: /opt/devtest/bin/Simulator -n tcp://simulator:${SIM_PORT}/Simulator -m tcp://${REG_HOST}:${REG_PORT}/Registry
 environment:
   HOSTNAME: simulator
   LISA_MORE_VM_PROPS: -DANDROID_HOME="" -Dlisa.pathfinder.broker.port=${BROKER_PORT} -Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:${DB_PORT}/${DATABASE}?useSSL=${USE_SSL} -Dlisadb.pool.common.user=${DB_USER} -Dlisadb.pool.common.password=${DB_PASS} -Ddevtest.enterprisedashboard.host=${ED_HOST} -Ddevtest.enterprisedashboard.port=${ED_PORT}
 links:
 - "registry"
 - "coordinator"
 - "database"

VSE:
 depends_on:
   - "coordinator"
 image: devtest/servers-mysql:latest
 ports:
   - ${VSE_PORT}:${VSE_PORT}
   - ${SERVICE_EXT}:${SERVICE_INT}
 command: /opt/devtest/bin/VirtualServiceEnvironment -n tcp://VSE:${VSE_PORT}/VSE -m tcp://${REG_HOST}:${REG_PORT}/Registry
 environment: 
   HOSTNAME: VSE
   LISA_MORE_VM_PROPS: -Dlisa.vse.performance.enabled=${PERF_VSE} -Dlisa.pathfinder.broker.port=${BROKER_PORT} -Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:${DB_PORT}/${DATABASE}?useSSL=${USE_SSL} -Dlisadb.pool.common.user=${DB_USER} -Dlisadb.pool.common.password=${DB_PASS} -Ddevtest.enterprisedashboard.host=${ED_HOST} -Ddevtest.enterprisedashboard.port=${ED_PORT}
 links:
   - "registry"
   - "database"

The following, therefore, should be added to the .env file

# Simulator Configuration
# ActiveMQ - default 2014
SIM_PORT=3014
#
#
# Virtual Service Environment Configuration
# ActiveMQ - default 2013
VSE_PORT=3013
#
# Service ports - Translation may be acceptable, depending on service
# Repeat here and in docker YAML file as needed
# internal range
SERVICE_INT=8000-8100
# external range
SERVICE_EXT=8000-8100
#
PERF_VSE=false

Note that VSE performance mode is off - should your license include this option then you can set this to true. If your license does not include this then please be aware that it is a chargeable option

Some Notes

Port Translation

The ActiveMQ ports must not be subjected to port translation - they must be exposed to the greater environment on the same port that they run on within the container. Failure to observe this will prevent the components connecting

Running Workstation

Since there is no site.properties file in this configuration, it is necessary to supply the database connection details to the Workstation at startup. This can be achieved by setting the LISA_MORE_VM_PROPS environmental variable before starting Workstation. For example:

Windows 

set LISA_MORE_VM_PROPS=-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:4306/DT101?useSSL=false -Dlisadb.pool.common.user=DTUSER -Dlisadb.pool.common.password=DTPASS

LINUX/UNIX

export LISA_MORE_VM_PROPS=-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:4306/DT101?useSSL=false -Dlisadb.pool.common.user=DTUSER -Dlisadb.pool.common.password=DTPASS

It may, however, not be prudent to supply the database password in plain text - in this case the encrypted value (perhaps taken from a configuration file) can be used, replacing lisadb.pool.common.password with lisadb.pool.common.password_enc, as below

set LISA_MORE_VM_PROPS=-Dlisadb.pool.common.driverClass=com.mysql.jdbc.Driver -Dlisadb.pool.common.url=jdbc:mysql://database:4306/DT101?useSSL=false -Dlisadb.pool.common.user=DTUSER -Dlisadb.pool.common.password_enc=76f271db3661fd50082e68d4b953fbee

Data in the Database

You may wish to configure an external database, or ensure that your data is otherwise persistent in order to avoid loss when the containers are rebuilt.

Sacrificing Flexibility for Convenience

Should a slightly less flexible arrangement be acceptable, then the database details can be supplied in the site.properties file, eliminating many of the required parameters to services and to workstation 

In this case, add the following line to the Registry.dockerfile 

ADD site.properties /opt/devtest/site.properties

And configure site.properties in the normal way

The following entries in bold can be removed from the .env file

# MySQL Database configuration
# Note that changing the MySQL port requires a change in my.cnf and a container rebuild
#
DATABASE=DT101
DB_USER=DTUSER
DB_PASS=DTPASS
DB_PORT=4306
USE_SSL=false
#
# Registry Configuration
ED_HOST=192.168.56.101
ED_PORT=1506
ED_USE_SSL=false

and the docker-compose.yml file becomes 

version: '3'

services:

  database:
    build:
      context: .
      dockerfile: mysql.dockerfile
    image: mysql:utf8
    ports:
     - ${DB_PORT}:${DB_PORT}
    environment:
      MYSQL_DATABASE: ${DATABASE}
      MYSQL_USER: ${DB_USER}
      MYSQL_PASSWORD: ${DB_PASS}

 registry:
   build:
     context: .
     dockerfile: Registry.dockerfile
   image: devtest/registry-broker-mysql:latest
   ports:
     - ${REG_PORT}:${REG_PORT}
     - ${REG_API_EXT}:${REG_API}
     - ${BROKER_PORT}:${BROKER_PORT}
     - ${BROKER_CONSOLE_EXT}:${BROKER_CONSOLE}
   command: /opt/devtest/runRegistryBroker.sh ${REG_HOST} ${REG_PORT}
   environment:
     HOSTNAME: ${REG_HOST}
     LISA_MORE_VM_PROPS: -Dlisa.pathfinder.broker.port=${BROKER_PORT} -Dlisa.webserver.port=${REG_API} -Ddevtest.port=${PORTAL_PORT_EXT}
   links:
     - "database"

 portal:
   depends_on:
     - "registry"
   build:
     context: .
     dockerfile: Portal.dockerfile
   image: devtest/portal-mysql:latest
   ports:
     - ${PORTAL_PORT_EXT}:${PORTAL_PORT}
   command: /opt/devtest/bin/Portal -J-Dregistry.host=${REG_HOST} -J-Dregistry.portal.port=${REG_API} -J-Dregistry.port=${BROKER_PORT} -J-DlisaAutoConnect=tcp://${REG_HOST}:${REG_PORT}/Registry -J-Dlisa.pathfinder.broker.port=${BROKER_PORT} -J-Dphoenix.port=${PORTAL_PORT}
   environment:
     HOSTNAME: portal
   links:
    - "registry"
    - "database"

 coordinator:
   depends_on:
   - "registry"
   build:
     context: .
     dockerfile: Servers.dockerfile
   image: devtest/servers-mysql:latest
   ports:
     - ${COORD_PORT}:${COORD_PORT}
   command: /opt/devtest/bin/CoordinatorServer -n tcp://coordinator:${COORD_PORT}/Coordinator -m tcp://${REG_HOST}:${REG_PORT}/Registry
   environment:
   HOSTNAME: coordinator
    LISA_MORE_VM_PROPS: -Dlisa.pathfinder.broker.port=${BROKER_PORT}
    links:
     - "registry"
     - "database"

 simulator:
   depends_on:
     - "coordinator"
   image: devtest/servers-mysql:latest
   ports:
    - ${SIM_PORT}:${SIM_PORT}
   command: /opt/devtest/bin/Simulator -n tcp://simulator:${SIM_PORT}/Simulator -m tcp://${REG_HOST}:${REG_PORT}/Registry
   environment:
     HOSTNAME: simulator

 VSE:
   depends_on:
     - "coordinator"
   image: devtest/servers-mysql:latest
   ports:
     - ${VSE_PORT}:${VSE_PORT}
     - ${SERVICE_EXT}:${SERVICE_INT}
   command: /opt/devtest/bin/VirtualServiceEnvironment -n tcp://VSE:${VSE_PORT}/VSE -m tcp://${REG_HOST}:${REG_PORT}/Registry
   environment:
     HOSTNAME: VSE
     LISA_MORE_VM_PROPS: -Dlisa.vse.performance.enabled=${PERF_VSE} -Dlisa.pathfinder.broker.port=${BROKER_PORT}
   links:
     - "registry"
     - "database"

As can bee seen, things have been greatly simplified, but at the expense of needing to configure site.properties and rebuild the containers when the database or Enterprise Dashboard properties change. 

In Conclusion

DevTest can be used within containers in many ways, only some of which have been explored here. The examples offered provide illustrations of some of these methods, and may be used as a starting point for deployment of DevTest with Docker. 

None of the examples here should be considered a solution - and solution will need to match your environment and business needs. It is, however, hoped that they prove useful.

Attached are two zip files that contain the fully parameterised and the site.properties based examples. 

Where to Obtain Further Assistance.

Running DevTest in Docker containers is community supported -  please see below to contact the relevant community.

Docker

If you are experiencing trouble with Docker, you can get further assistance from the following place 

• Docker documentation
• Docker Communities

Running DevTest In Docker

If you are having trouble running DevTest under Docker, please use the DevTest Communities

If you find a defect in DevTest

If you have found a defect in DevTest, then please raise a support case at CA Support  - CA Technologies. Please note that the DevTest Product Support Team are unable to answer questions about Docker.

Attachments

Attached are the examples shown, with the Dockerfiles amended to copy in patches from the patch sub-directory on container rebuild. After unzipping the files, please add your mysql-connector jar and, if it is not called mysql-connector.jar alter the Dockerfiles to match the name you are using.