Integration CI/CD Automate Tools

1. CI/CD Tools

What is CI/CD tools?

CI/CD stands for Continuous Integration and Continuous Delivery/Deployment, which are methodologies that enable teams to consistently integrate new code and reliably deliver updates of their software.

Continuous Integration (CI):

This is a development practice where developers integrate code into a shared repository frequently, typically multiple times a day. Each integration can then be verified by an automatic build and automated tests. The key goals of CI are to find and address bugs quicker, improve software quality, and reduce the time it takes to validate and release new software updates.

Continuous Delivery (CD):

Following CI, Continuous Delivery automates the delivery of applications to selected infrastructure environments. Most teams work with multiple environments other than the production, such as development and testing environments, and CD ensures there is an automated way to push code changes to them.

Continuous Deployment (CD):

A step beyond Continuous Delivery, Continuous Deployment goes further by releasing every change that passes through the production pipeline to customers with no human intervention. It requires a highly developed culture of monitoring, automated testing, and quality assurance to work effectively.

CICD Tools As image above, there are most popular CI/CD tools, pricing and the complexity, you can refer:

  • Jenkins: An open-source automation server providing an impressive plugin ecosystem to support building, deploying, and automating any project.

  • GitLab CI/CD: Integrated into the GitLab platform for source code management, it provides a rich set of features for automating different phases of the application lifecycle.

  • CircleCI: Cloud-native CI/CD service that provides infrastructure for automating the software development process.

  • Travis CI: A hosted, distributed continuous integration service to build and test software projects hosted at GitHub and Bitbucket.

  • GitHub Actions: Enables workflows for continuous integration and continuous deployment directly in the GitHub platform.

  • Bamboo: A CI/CD server from Atlassian, it offers automated building, testing, and deploying of applications.

  • Spinnaker: An open-source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence.

  • Azure Pipelines: Part of Microsoft’s Azure DevOps Services, this cloud service is used to automatically build and test code projects and make it available to other users.

2. Install Jenkins on Bastion Host

We will install Jenkins on Bastion Host.

2.1 SSH to Bastion Host

ssh -i public-bastion-host.pem ec2-user@YourEC2PublicIPV4

SSH Successfully to Bastion Host

2.2 Configure Security for Jenkins

At terraform source code, we have configured inbound security group for Jenkins allow access from Internet to Jenkins server via port 8080

ingress {
from_port   = 8080
to_port     = 8080
protocol    = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Warning: This allows traffic from any IP which might not be secure.
}

detail: https://github.com/daotq2000/aws-iaac-terraform/blob/main/aws-resources/05_sg.tf

2.3 Downloading and installing Jenkins

Completing the previous steps enables you to download and install Jenkins on AWS. To download and install Jenkins:

Ensure that your software packages are up to date on your instance by using the following command to perform a quick software update:

[ec2-user ~]$ sudo yum update –y

Add the Jenkins repo using the following command:

[ec2-user ~]$ sudo wget -O /etc/yum.repos.d/jenkins.repo \
https://pkg.jenkins.io/redhat-stable/jenkins.repo

Import a key file from Jenkins-CI to enable installation from the package:

[ec2-user ~]$ sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
[ec2-user ~]$ sudo yum upgrade
Install Java (Amazon Linux 2023):

[ec2-user ~]$ sudo dnf install java-17-amazon-corretto -y

Install Jenkins:

[ec2-user ~]$ sudo yum install jenkins -y

Enable the Jenkins service to start at boot:

[ec2-user ~]$ sudo systemctl enable jenkins

Start Jenkins as a service:

[ec2-user ~]$ sudo systemctl start jenkins

You can check the status of the Jenkins service using the command:

[ec2-user ~]$ sudo systemctl status jenkins

3. Install Docker on Bastion Host

You can refer post install docker on Amazon Linux at here or following constructions step below:

  1. Apply pending updates using the yum command

      sudo yum update

  2. Search for Docker package:

      sudo yum search docker

  3. Get version information:

     sudo yum info docker

  4. Install docker, run:

     sudo yum install docker

  5. Add group membership for the default jenkins user so you can run all docker commands without using the sudo command:

     sudo usermod -a -G docker jenkins

  6. Enable docker service at AMI boot time:

     sudo systemctl enable docker.service

  7. Start the Docker service:

     sudo systemctl start docker.service

  8. Get the docker service status on your AMI instance, run:

     sudo systemctl status docker.service

Outputs:

  docker.service - Docker Application Container Engine
  Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
  Active: active (running) since Wed 2021-09-08 05:03:52 EDT; 18s ago
  Docs: https://docs.docker.com
  Process: 3295 ExecStartPre=/usr/libexec/docker/docker-setup-runtimes.sh (code=exited, status=0/SUCCESS)
  Process: 3289 ExecStartPre=/bin/mkdir -p /run/docker (code=exited, status=0/SUCCESS)
  Main PID: 3312 (dockerd)
  Tasks: 9
  Memory: 39.9M
  CGroup: /system.slice/docker.service
  └─3312 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/c...
  
  Sep 08 05:03:51 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:51 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:51 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:51 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:52 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:52 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:52 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:52 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Sep 08 05:03:52 amazon.example.local systemd[1]: Started Docker Applicatio...
  Sep 08 05:03:52 amazon.example.local dockerd[3312]: time="2021-09-08T05:03...
  Hint: Some lines were ellipsized, use -l to show in full.

4. Configuring Jenkins

Jenkins is now installed and running on your EC2 instance. To configure Jenkins:

  1. Connect to http://<your_server_public_DNS>:8080 from your browser. You will be able to access Jenkins through its management interface: unlock_jenkins.pngJenkins Password
  2. As prompted, enter the password found in /var/lib/jenkins/secrets/initialAdminPassword.

Use the following command to display this password:

[ec2-user ~]$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

  1. The Jenkins installation script directs you to the Customize Jenkins page. Click Install suggested plugins.

  2. Once the installation is complete, the Create First Admin User will open. Enter your information, and then select Save and Continue. create_admin_user.png

  3. On the left-hand side, select Manage Jenkins, and then select Manage Plugins.

  4. Select the Available tab, and then enter Amazon EC2 plugin at the top right.

  5. Select the checkbox next to Amazon EC2 plugin, and then select Install without restart. unlock_jenkins.png

  6. Once the installation is done, select Back to Dashboard.

  7. Select Configure a cloud if there are no existing nodes or clouds. configure_cloud.png We are almost done to configure Jenkins on Bastion Host

3. Install require plugins for Jenkins

Click to install suggested plugins selected-plugsin.png selected-plugsin.png

4. Configure aws credentials on Bastion Host

  1. Using this command to config credentials. By default, aws-cli always available on EC2, we don’t need install aws-cli again.

      aws configure

Output:

  aws configure
  AWS Access Key ID [****************]: ****************
  AWS Secret Access Key [****************r]: ****************
  Default region name [us-east-1]: us-east-1
  Default output format [None]:

5. Install helm on Bastion Host

In this project, we will using Helm Chart to manage kubernetes object on EKS Cluster and Jenkins using helm command to apply this.

  1. install the binaries with the following commands.

      curl https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 > get_helm.sh
  chmod 700 get_helm.sh
  ./get_helm.sh

  2. See the version of Helm that you installed.

      helm version | cut -d + -f 1