Concourse CI is a very nice continuous integration server.
However for installs there are a few gotcha’s you need to keep in mind. Mostly these relate to how TLS/SSL works.
The first is that while it is possible to run concourse inside Docker I found this to cause a lot of issues with workers dying and not recovering. I would suggest installing the binarys on bare machines. When I moved from a docker cluser using Amazon’s ECS to a single t2.large instance not only were builds faster but it was a far more reliable solution.
I am also not going to automate this install, and will leave it as an excercise for you the reader to do this yourself. I would suggest using Python Fabric, or something like Puppet, Ansible or Saltstack to achive this.
Also keep in mind that with this install everything is running on a single instance. If you have need to scale out this is not going to work, but as a way to get started quickly it works pretty well.
Prerequisites are that you have a Ubuntu instance running somewhere. If you want to run the fly execute command you will also need a valid domain name to point at your machine. This is an annoying thing caused by GoLang when using SSL certs. Turns out you cannot set a hostfile entry and use it as such. You can in a insecure non SSL mode but otherwise cannot.
If you are using a virual machine from DigitalOcean/AWS/Vultr or other you will need to add some swap space. I noticed a lot of issues where this was missing. You can do so by running the following commands which will configure your server to have 4G of swap space,
We will need to get the concourse binary, and to make it executable. For convenience and to match the concourse documentation lets also rename it to concourse. To do so run the following command.
We now need to generate the keys that concourse requires.
The above commands will create a directory called keys and setup all of the keys that concourse 2.6.0 requires.
We can now create some helper scripts which we can use to run concourse easily.
This script will start running concourse. Keep in mind that the username and password used here are for the main group and as such you should protect them as they have the ability to create additional groups on your concourse instance.
This script will spin up a worker which will communicate with the main concourse instance and do all the building. It can be useful to lower the priority of this command using nice and ionice if you are running on a single core machine.
Now we need to install all of the postgresql packages required,
Once this is done we can create the database to be used
Then login to postgresql and create a user to connect to the database
We also need to need to edit the pg_hba file allowing us to make the connection,
Scroll down and look for the following line,
and change the md5 on the end to trust
Then save the file and restart postgresql
At this point everything we need to run concourse should be there. You will need to setup the concourse scripts we created earlier to run as a service, or just run them in a screen session if you are in a hurry.
What we want to do now is expose it to the big bad internet.
Create a directory using either the domain name you want to use, a desired name or anything if you are going to connect to things using IP addresses.
Nowe we want to swtich to the directory and setup the self signed TLS/SSL keys.
Enter whatever you want for the passphrase but remember it!
Enter the passhrase entered. The most important thing here is that when asked for the Common Name or FQDN you need to enter in the desired domain name.
With that done we need to sign the key.
Remember to enter the same pass phrase as before. Finally sign they key with an expiry of 9999 days.
Make a copy of the file server.crt which will be needed for the concourse fly tool to talk to the server if you are using self signed certs.
With that done lets enable the site,
And enter in the following details,
The above nginx config defines an upstream concourse server running on port 8080. It then defines a server listening on port 80 that redirects all traffic to the same server on HTTPS. The last server config defines out site, sets up the keys and forwards everything to the upstream concourse server.
We can now enable it,
We need to remove the default file above because nginx will not allow you to have two default servers.
At this point everything should be working. You should now be able to connect to your concourse server like so,
Where the file ca.crt exists whereever you are running fly. Everything at this point should work and you can browse to your concourse server.
If you are using the IP address to communicate to your concourse server you have the limitation that you will not be able to run fly execute to upload a task. You can get around this by using a real domain, or running your own DNS to resolve it correctly.
The only bit of homework at this point would be to configure the firewall to disable access to port 8080 so that everything must go though nginx. Enjoy!