In order to do efficient, modern software development, especially for web applications, it is absolutely necessary to have a system for continuous integration (CI) and continuous delivery (CD). The idea with CI is to continuously integrate working code to the repository so that it can continuously be checked for errors. So, a prerequisite is to be using some sort of test driven development (TDD) method or at least some unit test framework; otherwise there is chance to automatically test for errors when the code is committed to the repository. The best CI systems automate everything after the commit. A developer pushes her code, then the CI systems notices the change and automatically runs the tests. When the application passes all of its test, a continuous delivery system should then automatically put the code into the QA, staging, or testing environment. There can be a lot of steps required to get a decent CI/CD system working, and it can be especially daunting when there is no dedicated DevOps team. So, I created a cheat-sheet for quickly setting up a simple Jenkins CI/CD system on an AWS EC2 instance.

Before proceeding you must have an Amazon Web Services account and be somewhat familiar with AWS.

Create an EC2 Instance

  1. Log in to the AWS console and got to EC2 (Services > Compute > EC2)
  2. Launch an Instance (by clicking the big “Launch Instance” button)
    • Choose an Amazon Machine Image
      • Select Amazon Linux AMI (64-bit)
    • Choose an Instance Type
      • Select t2.micro (in the free tier) for testing
      • Select t2.small (not free tier) when using Docker
    • Configure Instance Details
      • If CodePipeline is needed, then Create a new IAM role which includes the policy AWSCodePipelineCustomActionAccess, if one does not already exist
      • Select the newly created IAM role
      • Leave the remaining options as defaulted, or set according to need
    • Add Storage
    • Tag Instance
    • Configure Security Group
      • Create a security group, if one does not already exist, and make sure it has the following inbound rules
        • HTTP, port 80, (select an appropriate source, not 0.0.0.0 unless you have no other option)
        • Custom TCP Rule, port 8080
        • SSH, port 22
    • Review Instance Launch
    • Create a key to access the instance via SSH (referenced below as SSH_KEY_NAME)
  3. Under Instances > Instances, select the new instance an note its Public DNS name (referenced below as PUBLIC_DNS_NAME)
  4. If test results should be reported in local time rather than UTC time, use the following steps
    • Log in to the instance (open a terminal and type the command listed below)
    • Set the timezone
      • Find the correct timezone in /usr/share/zoneinfo (e.g. /usr/share/zoneinfo/America/New_York)
      • Edit /etc/sysconfig/clock so that it only says ZONE="America/New\_York" (or whatever timezone is required)
        • sudo vi /etc/sysconfig/clock
      • sudo ln -sf /usr/share/zoneinfo/America/New_York /etc/localtime
    • This will take effect after the next reboot

AWS EC2 Login Commands

ssh -i \
	~/.ssh/SSH\_KEY\_NAME \
	ec2-user@PUBLIC\_DNS\_NAME

Install Jenkins

If you are not already logged in to the instance from changing the timezone above, then log in now (see above).

sudo yum update
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
sudo rpm --import http://pkg.jenkins-ci.org/redhat-stable/jenkins-ci.org.key
sudo yum install jenkins
sudo service jenkins start
sudo chkconfig jenkins on

Install Docker (Amazon EC2 Container Service)

sudo yum install -y ecs-init
sudo gpasswd -a jenkins docker
sudo service docker start
sudo chkconfig docker on

Install Other Tools

sudo yum install git

If you are using the GitHub plug-in then you should create a key to log in to GitHub

sudo su -s /bin/bash jenkins
cd /var/lib/jenkins/
ssh-keygen
cat .ssh/id_rsa.pub

Copy the public key, which was just concatenated onto the screen, and add it to your GitHub keys.

sudo reboot

The ssh connection will close while the system reboots, which is fine, we are done with it for now.

Configure Jenkins

After the instance reboots, type PUBLIC_DNS_NAME:8080 (use the Public DNS name from above) into a browser. The Jenkins interface should be seen.

Configure Security Credentials

  • Manage Jenkins > Setup Security
    • Enable Security
    • Security Realm = Jenkins’ own user database
      • Allow users to sign up
    • Authorization = Matrix based
    • CHECK ALL PERMISION BOXES FOR ANONYMOUS
    • Save
  • Sign up (as administrator) and log in
  • Manage Jenkins > Configure Global security
    • Add all permissions for your account
    • Remove all but read from anonymous
    • Save

Update and Install Plugins

Install plugins which are appropriate for your needs, below are the ones that I use.

  1. Update existing installed plugins using the Jenkins interface
  2. Install the AWS CodePipeline plugin
  3. Install the GitHub plugin
  4. Install the NodeJS plugin
  5. Install the TAP plugin

Jenkins needs to restart after all of the plugins are installed. It can take a while to update some of the existing plugins, so I normally take a break at this point and finish the steps below at a later time.

Finish Jenkins Configuration

GitHub Credentials

If you will be getting code from GitHub, then you will need to tell Jenkins to use the proper ssh key.

  • Jenkins > Manage Jenkins
  • Manage Credentials
  • Add Credentials > SSH Username with Private Key
    • Username = YOUR_GITHUB_USERNAME
    • From the Jenkins master ~/.ssh

Jenkins System Configuration

  • Manage Jenkins > Configure System
    • NodeJS
      • Add NodeJS
        • Name the Node installation
        • Install automatically from nodejs.org
        • Select the node version to install (If you do not see a drop-down list of Node version choices, then you may need to restart Jenkins)
        • Global NPM packages to install
          • ember-cli bower phantomjs
    • Jenkins Location
      • System Admin e-mail address = SOME_NAME <SOME_NAME@SOME_DOMAIN.XYZ>
    • E-mail Notification (assuming Gmail)
      • Click advanced
      • SMTP server = smtp.gmail.com
      • CHeck Use SSL
      • SMTP Port = 465
      • Check Use SMTP Authentication
      • User Name = YOUR_EMAIL_ADDRESS@gmail.com
      • Password = ???

Configure Items

With the Jenkins system set up, it is now possible to create build and test processes. The steps required varies based upon which tools are being used, and they will be covered in future posts. The last thing which you may want to do is to go to the EC2 console and save this instance as an image, so that you do not have to go through all of these steps again.

Tags: DevOps AWS EC2 Jenkins Docker CI CD

Published: 2016-01-08