Automation is everywhere and it is better to adopt it as soon as possible. Today, in this blog post, we are going to discuss creating the infrastructure. For this, we will be using AWS for hosting our deployment pipeline. Packer will be used to create AMI’s and Terraform will be used for creating the master/slaves. We will be discussing different ways of connecting the slaves and will also run a sample application with the pipeline.

Please remember the intent of the blog is to accumulate all the different components together, this means some of the code which should be available in development code repo is also included here. Now that we have highlighted the required tools, 10000 ft view and intent of the blog. Let’s begin.

Using Packer to Create AMI’s for Jenkins Master and Linux Slave

Hashicorp has bestowed with some of the most amazing tools for simplifying our life. Packer is one of them. Packer can be used to create custom AMI from already available AMI’s. We just need to create a JSON file and pass installation script as part of creation and it will take care of developing the AMI for us. Install packer depending upon your requirement from Packer downloads page. For simplicity purpose, we will be using Linux machine for creating Jenkins Master and Linux Slave. JSON file for both of them will be same but can be separated if needed.

Note: user-data passed from terraform will be different which will eventually differentiate their usage.

We are using Amazon Linux 2 - JSON file for the same.

CODE: https://gist.github.com/velotiotech/d9e05253d8537b49e6a0c25750584c69.js

As you can see the file is pretty simple. The only thing of interest here is the install_amazon.bash script. In this blog post, we will deploy a Node-based application which is running inside a docker container. Content of the bash file is as follows:

CODE: https://gist.github.com/velotiotech/ba0882f2586b0af37118a6519dcdf0a5.js

Now there are a lot of things mentioned let’s check them out. As mentioned earlier we will be discussing different ways of connecting to a slave and for one of them, we need xmlstarlet. Rest of the things are packages that we might need in one way or the other.

Update ami_users with actual user value. This can be found on AWS console Under Support and inside of it Support Center.

Validate what we have written is right or not by running packer validate amazon.json.

Once confirmed, build the packer image by running packer build amazon.json.

After completion check your AWS console and you will find a new AMI created in “My AMI’s”.

It's now time to start using terraform for creating the machines. 

Prerequisite:

1. Please make sure you create a provider.tf file.

CODE: https://gist.github.com/velotiotech/b7a5322999457afe1f9a4066e3243888.js

The ‘credentials file’ will contain aws_access_key_id and aws_secret_access_key.

2.  Keep SSH keys handy for server/slave machines. Here is a nice article highlighting how to create it or else create them before hand on aws console and reference it in the code.

3. VPC:

CODE: https://gist.github.com/velotiotech/9acba1e2864d2a34cd488782f0068487.js

Creating Terraform Script for Spinning up Jenkins Master

Creating Terraform Script for Spinning up Jenkins Master. Get terraform from terraform download page.

We will need to set up the Security Group before setting up the instance.

CODE: https://gist.github.com/velotiotech/f8a72ee7b022c06664fbcfad14dc5def.js

Now that we have a custom AMI and security groups for ourselves let’s use them to create a terraform instance.

CODE: https://gist.github.com/velotiotech/619a3b4d5bcedd2841132817f96390e9.js

As mentioned before, we will be discussing multiple ways in which we can connect the slaves to Jenkins master. But it is already known that every time a new Jenkins comes up, it generates a unique password. Now there are two ways to deal with this, one is to wait for Jenkins to spin up and retrieve that password or just directly edit the admin password while creating Jenkins master. Here we will be discussing how to change the password when configuring Jenkins. (If you need the script to retrieve Jenkins password as soon as it gets created than comment and I will share that with you as well).

Below is the user data to install Jenkins master, configure its password and install required packages.

CODE: https://gist.github.com/velotiotech/74b5712f8f9b4675258cd377308506e0.js

There is a lot of stuff that has been covered here. But the most tricky bit is changing Jenkins password. Here we are using a python script which uses brcypt to hash the plain text in Jenkins encryption format and xmlstarlet for replacing that password in the actual location. Also, we are using xmstarlet to edit the JNLP port for windows slave. Do remember initial username for Jenkins is admin.

Command to run: Initialize terraform - terraform init , Check and apply - terraform plan -> terraform apply

After successfully running apply command go to AWS console and check for a new instance coming up. Hit the <public ip="">:8080 and enter credentials as you had passed and you will have the Jenkins master for yourself ready to be used. </public>

Note: I will be providing the terraform script and permission list of IAM roles for the user at the end of the blog.

Creating Terraform Script for Spinning up Linux Slave and connect it to master

We won't be creating a new image here rather use the same one that we used for Jenkins master.

VPC will be same and updated Security groups for slave are below:

CODE: https://gist.github.com/velotiotech/54c358c39bcdcf0bf22b92975c8c100b.js

Now that we have the required security groups in place it is time to bring into light terraform script for linux slave.

CODE: https://gist.github.com/velotiotech/e989a082af0cf563dd35713e583ee647.js

And now the final piece of code, which is user-data of slave machine.

CODE: https://gist.github.com/velotiotech/24c99af97e78a14462b14d4fdb1be9fa.js

This will not only create a node on Jenkins master but also attach it.

Command to run: Initialize terraform - terraform init, Check and apply - terraform plan -> terraform apply

One drawback of this is, if by any chance slave gets disconnected or goes down, it will remain on Jenkins master as offline, also it will not manually attach itself to Jenkins master.

Some solutions for them are:

1. Create a cron job on the slave which will run user-data after a certain interval.

2. Use swarm plugin.

3. As we are on AWS, we can even use Amazon EC2 Plugin.

Maybe in a future blog, we will cover using both of these plugins as well.

Using Packer to create AMI’s for Windows Slave

Windows AMI will also be created using packer. All the pointers for Windows will remain as it were for Linux.

CODE: https://gist.github.com/velotiotech/4fb2bceebff48e661a0f5cd233a2bba5.js

Now when it comes to windows one should know that it does not behave the same way Linux does. For us to be able to communicate with this image an essential component required is WinRM. We set it up at the very beginning as part of user_data_file. Also, windows require user input for a lot of things and while automating it is not possible to provide it as it will break the flow of execution so we disable UAC and enable RDP so that we can connect to that machine from our local desktop for debugging if needed. And at last, we will execute install_windows.ps1 file which will set up our slave. Please note at the last we are calling two PowerShell scripts to generate random password every time a new machine is created. It is mandatory to have them or you will never be able to login into your machines.

There are multiple user-data in the above code, let’s understand them in their order of appearance.

SetUpWinRM.ps1:

CODE: https://gist.github.com/velotiotech/5a4cc995a5c81f24910e575abf8c09d2.js

The content is pretty straightforward as it is just setting up WInRM. The only thing that matters here is the <powershell> and </powershell>. They are mandatory as packer will not be able to understand what is the type of script. Next, we come across disable-uac.ps1 & enable-rdp.ps1, and we have discussed their purpose before. The last user-data is the actual user-data that we need to install all the required packages in the AMI.

Chocolatey: a blessing in disguise - Installing required applications in windows by scripting is a real headache as you have to write a lot of stuff just to install a single application but luckily for us we have chocolatey. It works as a package manager for windows and helps us install applications as we are installing packages on Linux. install_windows.ps1 has installation step for chocolatey and how it can be used to install other applications on windows.

CODE: https://gist.github.com/velotiotech/923b09afe6ee303de28e333c9be256a3.js

See, such a small script and you can get all the components to run your Windows application in no time (Kidding… This script actually takes around 20 minutes to run :P)

Remaining user-data can be found here.

Now that we have the image for ourselves let’s start with terraform script to make this machine a slave of your Jenkins master.

Creating Terraform Script for Spinning up Windows Slave and Connect it to Master

This time also we will first create the security groups and then create the slave machine from the same AMI that we developed above.

CODE: https://gist.github.com/velotiotech/e2e755b8e451886cc055c58296234079.js

Once security groups are in place we move towards creating the terraform file for windows machine itself. Windows can't connect to Jenkins master using SSH the method we used while connecting the Linux slave instead we have to use JNLP. A quick recap, when creating Jenkins master we used xmlstarlet to modify the JNLP port and also added rules in sg group to allow connection for JNLP. Also, we have opened the port for RDP so that if any issue occurs you can get in the machine and debug it.

Terraform file:

CODE: https://gist.github.com/velotiotech/c7dbab4a7f7e2dd774b4acd991b581fb.js

Finally, we reach the user-data for the terraform plan. It will download the required jar file, create a node on Jenkins and register itself as a slave.

CODE: https://gist.github.com/velotiotech/4e1ce859f05e5bd2f223e1cebc763e78.js

Command to run: Initialize terraform - terraform init, Check and apply - terraform plan -> terraform apply

Same drawbacks are applicable here and the same solutions will work here as well.

Congratulations! You have a Jenkins master with Windows and Linux slave attached to it.

IAM roles for reference

‍Jenkins Master

Linux Slave

Windows Slave

Bonus:

If you want to associate IAM permissions to the user but cannot assign FULL ACCESS here is a curated list below for reference:

Packer Policy

Terraform Policy

Conclusion:

This blog tries to highlight one of the ways in which we can use packer and Terraform to create AMI's which will serve as Jenkins master and slave. We not only covered their creation but also focused on how to associate security groups and checked some of the basic IAM roles that can be applied. Although we have covered almost all the possible scenarios but still depending on use case, the required changes would be very less and this can serve as a boiler plate code when beginning to plan your infrastructure on cloud.