Prerequisites

Before we begin, ensure you have the following installed:

• Go (version 1.15 or later)

• Docker

• kubectl

• AWS CLI

• eksctl

• Helm

• Git

You'll also need accounts on:

• GitHub

• Docker Hub

• AWS

Step 1: Containerizing the Application with Docker

Commands Used:

• Cloning the Git repository:

  git clone <repo_url>

  cd <repo_name>

• Building and running the application locally:

  go build -o main.exe

  ./main.exe

• Creating a Dockerfile using a multi-stage build:

    FROM golang:1.22.5 as base
    WORKDIR /app
    COPY go.mod .
    RUN go mod download
    COPY . .
    RUN go build -o main.exe .
  
    FROM gcr.io/distroless/base
    COPY --from=base /app/main.exe .
    COPY --from=base /app/static ./static
    EXPOSE 8080
    CMD ["./main.exe"]
  

• Building the Docker image:

  docker build -t priyatjs/go-web-app:v1 .

• Running the container locally:

  doc``ker run -p 8080:8080 priyatjs/go-web-app:v1

Screenshots**:**

• Docker build output showing successful image creation.

  

  

  Terminal screenshot showing the Docker image being pushed to Docker Hub

  

• Browser screenshot accessing the application on localhost:8080/courses.

Explanation of the Multi-Stage Build Process:

• Stage 1: Base Image

  • Base Image: The first stage of the Dockerfile uses a full-featured base image, such as golang:1.22.5. This image includes all the necessary tools and libraries needed to build the application.

  • Dependencies: The go mod download command ensures that all Go module dependencies are downloaded.

  • Application Code: The application code is copied into the container, and the application is built using the go build command.

• Stage 2: Distroless Image

  • Distroless Image: The second stage uses a minimal, "distroless" base image, such as gcr.io/distroless/base. This image is designed to be lightweight and secure.

  • Copy Application: Only the built application and necessary runtime dependencies are copied from the first stage into this minimal image.

  • Run Configuration: The final image is configured to run the application using the CMD command.

Benefits of the Multi-Stage Build:

• Smaller Image: The final image is much smaller because it only includes the application and its runtime dependencies. All unnecessary build tools and libraries from the first stage are excluded.

• Increased Security: By using a distroless base image, the attack surface is reduced. The final image lacks a shell and other unnecessary tools, making it more secure.

Output:

• The application was successfully built and ran locally, accessible via localhost:8080/courses.

• The Docker image was built successfully despite minor warnings regarding the Dockerfile casing.

Explanation: The Dockerfile was well-constructed using a multi-stage build, which reduced the final image size and increased security by using a distroless base image. One minor issue encountered was a warning about the casing of keywords in the Dockerfile, but this did not affect the build process.

Step 2: Deploying the Application to Kubernetes

Commands Used:

• Creating Kubernetes manifest files (deployment.yaml, service.yaml, ingress.yaml).

• Applying Kubernetes manifests:

  kubectl apply -f k8s/manifest/deployment.yaml kubectl apply -f k8s/manifest/service.yaml kubectl apply -f k8s/manifest/ingress.yaml

• Creating an EKS cluster using eksctl:

  eksctl create cluster --name demo-cluster --region us-east-1

Output:

• Kubernetes resources were successfully created, though initially, the ingress resource did not have an assigned address.

Screenshots:

• Terminal screenshot showing the eks cluster created.

  

  

• Kubernetes dashboard showing deployment, service, and ingress resources.

  

Initial Issue:

• After applying the ingress manifest, I noticed that my ingress resource was not assigned an address. This was because an ingress controller was not yet in place.

Steps Taken to Resolve the Issue:

• To address this, I needed to deploy an ingress controller. However, before doing so, I decided to verify that the service was working by exposing it in NodePort mode.

• I edited the go-web-app service to NodePort using the following command:

  kubectl edit service go-web-app

  

Finding the Node IP:

• After exposing the service, the go-web-app could be accessed on port 30485 on my node's IP address. To determine the node IP addresses, I used:

  kubectl get nodes -o wide

  

• I was able to access the go-web-app on both external IPs by entering the node IP and port in my browser.

  

  

Step 3: Configuring Ingress Controller

Commands Used:

• Installing the Nginx ingress controller:

  kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/cloud/deploy.yaml

  

• Checking the ingress status:

  kubectl get ingress -n ingress-nginx

  

  Then lets check the details of nginx controller pod using below command.

  

  

Output:

• The ingress resource was successfully assigned an address after setting up the ingress controller.

  

Mapping the IP Address to a Domain Name:

• Now, I needed to map the ingress IP address to a DNS host name. I did an nslookup on the assigned address to check the address and then edited the /etc/hosts file on my system for DNS mapping.

  

• As I had forgotten my system password, I manually updated the /etc/hosts file by adding:

  3.93.102.122 go-web-app.local

  • Note: If you're adding it properly in Notepad on Windows, ensure you run Notepad as an administrator to save the file.

• Once I saved my /etc/hosts file correctly, I was able to access my application on a web browser using the host name go-web-app.local followed by /courses.

Output:

• Kubernetes resources were successfully created, and after resolving the ingress issue, the application was accessible via the domain name mapped in the /etc/hosts file.

• The DNS mapping allowed smooth access to the application using a friendly hostname.

Screenshots:

• Browser screenshot accessing the application using the DNS name mapped to the ingress IP.

  

Explanation: Initially, there was a delay in assigning the ingress address, which was fixed by properly configuring the ingress controller. The DNS mapping was manually handled by editing the /etc/hosts file due to the lack of automated DNS management.

Step 4: Using Helm to Manage Kubernetes Workflows

After successfully deploying the application and setting up the ingress, I moved on to using Helm to create and manage Kubernetes workflows. Helm is a powerful tool that simplifies the deployment and management of applications within Kubernetes by using packaged applications called "charts."

Commands Used:

1. Checking if Helm is Installed:

   • Before proceeding, I needed to verify if Helm was installed on my system. I ran the following command:

     helm version

     • This command returns the version of Helm installed. If Helm isn't installed, you can follow the Helm installation guide to set it up.

Screenshot:

• Terminal output showing the installed version of Helm.

  

2. Installing an Application Using a Helm Chart:

   I then used Helm to install a chart.

   • 

Explanation:

Helm greatly simplified the deployment of Kubernetes resources by allowing me to use predefined charts. First, I confirmed that Helm was installed on my system. After adding the necessary Helm repository, I was able to deploy applications within Kubernetes using Helm charts.

One of the key benefits of Helm is its ability to manage complex Kubernetes applications with ease. By using Helm charts, I could deploy and manage applications with a single command, avoiding the need to manually create and apply numerous Kubernetes manifest files.

Additionally, Helm allows for easy upgrades and rollbacks of applications, making it an essential tool in a Kubernetes-based CI/CD pipeline.

Output:

• Successfully installed and managed Kubernetes applications using Helm, demonstrating Helm’s capabilities in streamlining Kubernetes workflows.

Step 5: Setting Up CI/CD with GitHub Actions and ArgoCD

To automate and streamline the deployment process, I implemented CI/CD pipelines using GitHub Actions for CI and ArgoCD for CD. Here’s how I set it up:

Continuous Integration (CI):

I implemented CI using GitHub Actions, with the following stages:

1. Build and Unit Test:

   • The CI pipeline begins by building the Go application and running unit tests.

2. Static Code Analysis:

   • Next, static code analysis is performed to ensure code quality.

3. Docker Image Creation and Pushing:

   • The Docker image is created and pushed to Docker Hub. I used the following command to push the image:

     docker push <your_dockerhub_username>/go-web-app: <tag>

4. Update Helm Chart:

   • The new Docker image tag is updated in the values.yaml file of the Helm chart.

I created a .github/workflows/ci.yaml file in my GitHub repository to define the CI pipeline. Within this YAML file, I configured the pipeline to trigger on every push to the main branch. I added Docker Hub and GitHub secrets to securely store credentials.

After committing and pushing these changes to the remote repository, the CI pipeline was triggered. It successfully completed all stages after resolving several errors.

Screenshots:

• Screenshot showing the successful CI run on GitHub Actions.

  

  

Continuous Deployment (CD):

For CD, I used ArgoCD, which watches the values.yaml file in the Helm chart. Whenever this file is updated, ArgoCD automatically pulls the new version and deploys it to the Kubernetes cluster.

1. Installing ArgoCD:

   • I installed ArgoCD in the Kubernetes cluster and exposed the ArgoCD server to access the UI:

     kubectl create namespace argocd

     kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

     kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

2. Accessing ArgoCD UI:

   • I accessed the ArgoCD UI using the external IP obtained from the above command.

3. Logging into ArgoCD:

   • The default username is admin. To retrieve the password, I decoded it from the argocd-initial-admin-secret:

     kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath="{.data.password}" | base64 --decode

4. Creating an ArgoCD Application:

   • I created a new ArgoCD application by filling out the form with details such as app name, sync policy, GitHub repo link, cluster URL, and Helm files. Once created, ArgoCD automatically deployed the application to the Kubernetes cluster.

5. Verifying the Deployment:

   • I verified that the application was deployed successfully by accessing it via the domain name mapped earlier.

6. Testing the CI/CD Pipeline:

   • To ensure the CI/CD pipeline was working correctly, I made a small change to a static file (home.html) in the application, adding "by Abhishek Veeramalla" to the website home page. After committing this change to the main branch, the CI pipeline was triggered, which updated the Docker image tag and Helm chart.

   • ArgoCD detected the change and deployed the updated application to the Kubernetes cluster. I verified the deployment by checking the web application, which reflected the changes I made.

Screenshots:

• Screenshot of CI pipeline triggered after committing changes in GIT.

• Screenshot of updated tag in values.yaml file and recently created tag within Docker hub.

• Screenshot of web app accessed using host link with updated change in home page.

Conclusion

This project provided hands-on experience with containerization, Kubernetes deployment, and CI/CD integration. There were challenges, particularly with ingress configuration and synchronization between CI/CD tools, but these were resolved through careful troubleshooting and configuration. The resulting deployment is secure, scalable, and automated, making it a robust solution for managing cloud-native applications.

Once upon a time, there was a talented artist named Harry who crafted exquisite ceramic mugs. As word of his artistry spread far and wide, orders came pouring in. However, Harry soon struggled to keep up with the overwhelming demand and ensure safe delivery of the delicate mugs.

A Magical Solution Appears ✨

Desperate for help, Harry sought advice from a wise logistics expert, who introduced him to the wonders of Kubernetes, a powerful system that could transform his shipping process.

Packaging the Mugs 📦(Docker)

Before using Kubernetes, the consultant explained that it works best when applications are packaged into containers. Just as mugs need sturdy packaging to protect them during transit, applications need a consistent and isolated environment to run reliably in different shipping facilities (clusters). Among the various container technologies available, the consultant recommended Docker, which provides a lightweight, portable, and consistent way to package applications, ensuring they run seamlessly across different environments.

Wrapping the Packages🎁(Pods)

After packing the mugs (applications) in Docker containers, Kubernetes wrapped these containers in gift-wrapped packages (Pods), adding an extra layer of protection, like wrapping a fragile mug in bubble wrap. Each Pod could contain one or more containers. However, it is recommended to have one container in a pod until it is a supporting container that is required to run the other container.

The Shipping Fleet 🚚 (Nodes)

Kubernetes then placed these Pods into vehicles or trucks (Nodes). Each Node could carry multiple Pods, and each Pod could contain one or more containers. The Nodes were responsible for transporting the cargo.

The Vehicle Crew 👷(Node Components)

Each Node had its own crew to ensure smooth operations:

• The driver (Kubelet), responsible for managing the Pods on their assigned Node and communicating with the Master Node.

• The loading bay (Container Runtime), preparing and loading the containers onto the Node.

• The navigation system (Kube Proxy), facilitating efficient communication between Pods and guiding the vehicles along optimal routes.

The Logistics Headquarters🏢(Master Node)

However, before the Nodes could begin their journey, they needed instructions and coordination from a central logistics headquarters (Master Node).

The Logistics Team 🧑‍✈️(Master Components)

At the heart of the Master Node, there was a dedicated logistics team:

• The communication hub (API Server), receiving and processing all incoming orders and requests.

• The central database (etcd), storing and tracking every detail about the cargo, vehicles, routes, and more.

• The expert route planner (Scheduler), strategically assigning Pods to the most suitable Nodes based on available resources and requirements.

• The operations manager (Controller Manager), continuously monitoring the shipping process and taking corrective actions when needed.

The Shipping Network🌲(Cluster)

Together, the Master Node and Nodes formed a vast shipping network (Cluster) capable of transporting and managing Pods across multiple locations.

Customer Access Points 🏪(Services)

As orders poured in, Harry utilized access points called Services, which acted as retail stores or outlets, providing a single point for customers to purchase and receive their mugs. Services not only served as entry points or gateways, allowing customers to access Harry's shipping operation from outside, but also offered load balancing capabilities by distributing incoming traffic across multiple Pods running the same application. This ensured no single Pod was overwhelmed, providing better reliability and scalability. Moreover, Services facilitated service discovery, making it easier for different components within the shipping operation to communicate with each other, even if their locations or IP addresses changed.

Service Types:

• ClusterIP Service: An internal access point, accessible only within the shipping network (cluster).

• NodePort Service: A dedicated access point on each Node, allowing direct access from outside the shipping network.

• LoadBalancer Service: An advanced access point with an external load balancer, distributing traffic across multiple Nodes for high availability and scalability.

Service discovery enabled Pods and Nodes to locate and communicate with each other seamlessly, even as their underlying infrastructure changed, ensuring a resilient and efficient shipping operation.

The Management Tools 💻📄

To manage this intricate shipping operation, Harry relied on two essential tools:

• Kubectl 💻: The trusty shipping management software, allowing Harry to interact with the Kubernetes cluster and issue commands to deploy, scale, or manage their mugs (applications) and associated resources (Pods, Nodes, Services, etc.).

• Manifest Files (YAML Files) 📄: Acting as detailed shipping instructions, these YAML files specified the desired state of Harry's Kubernetes resources, from the number of replicas required to resource allocations and configurations.

The Resilient and Efficient Operation ⚡⚡

Harry's business could effortlessly scale by adding or removing Nodes (trucks) to match demand. In case of vehicle breakdowns, Kubernetes automatically rescheduled affected Pods to healthy Nodes, ensuring minimal disruption. Communication between Nodes was seamless, with the navigation system (Kube Proxy) enabling efficient coordination and load balancing.

A Thriving Shipping Empire 🌍☕

Thanks to the well-orchestrated shipping operation of Kubernetes, Harry's ceramic mug business flourished, delighting customers with prompt and secure deliveries. As the tale spread, businesses embraced Kubernetes, recognizing its benefits and challenges.

Key Takeaways and Drawbacks:

• Kubernetes simplified the management of complex, distributed applications by abstracting away underlying infrastructure details.

• Services provided an isolation layer, allowing Harry to scale, update, or replace shipping components without disrupting customer experience.

• However, Kubernetes introduced a steep learning curve due to its complexity and required diligent configuration and monitoring for security.

• Running and maintaining the control plane and system components added overhead, potentially leading to vendor lock-in (vendors like Amazon Web Services (AWS), Google Cloud Platform (GCP), or Microsoft Azure).

Despite these drawbacks, for businesses needing to manage and scale containerized applications efficiently, Kubernetes emerged as the go-to solution, particularly in scenarios involving multiple interconnected applications, mission-critical workloads demanding high availability, rapidly fluctuating demand, and the need for consistent packaging and deployment across different environments.

As the magical tale spread, more businesses recognized the transformative power of Kubernetes, embracing it as the de facto standard for container orchestration and ushering in a new era of seamless application shipping, just like Harry's precious mugs reaching delighted customers worldwide! 🌍☕