DevOps

1. What is DevOps?

DevOps = Dev (Builders) + Ops (Operators)

Simplest Explanation:

Imagine building a car (Dev) and then driving it around (Ops).

• Old way: The builders finish the car, throw the keys over the fence to the drivers, and then the drivers struggle to figure out how it works or fix problems. Lots of arguments and delays.
• DevOps way: The builders and drivers work together from the start. They talk constantly, use tools to automate building and testing, and make sure the car is always ready to drive smoothly. If something breaks, they fix it together, fast.

DevOps is about breaking down walls between teams so they can work together seamlessly to deliver software faster and better.

Key Concepts:

1. Collaboration: Dev and Ops talk and work as one team. (Think: no more throwing keys over the fence!)
2. Automation: Use tools to do repetitive tasks (like building, testing, deploying) automatically. (Think: robots doing the work, not humans making mistakes).
3. Continuous Delivery (CD): Always ready to release new features to customers quickly and safely. (Think: car is always showroom ready).
4. Feedback: Learn from how the software performs in real life to constantly improve. (Think: drivers tell builders what’s working/not working).
5. Faster & Better: The ultimate goal is to deliver software faster with higher quality.

Why it’s important (Benefits):

• Speed: Get new features out super fast.
• Quality: Fewer bugs, more reliable software.
• Efficiency: Less wasted time and effort.

---

2. Key Principles of DevOps

3. Important DevOps Tools

DevOps relies heavily on a diverse set of tools that automate various stages of the software development and delivery pipeline. For your exams, it’s important to know the categories of these tools and some prominent examples within each.

Here are the most important DevOps tools categorized by their primary function:

1. Version Control Systems (VCS) / Source Code Management (SCM):
   • Purpose: To track changes in code, manage different versions, and enable collaboration among multiple developers. This is where all the code lives and is the foundation for CI/CD.
   • Key Tools:
     • Git: The most widely used distributed version control system. It’s open-source and provides powerful branching and merging capabilities.
     • GitHub / GitLab / Bitbucket: Web-based platforms that provide hosting for Git repositories, along with features for collaboration (pull requests, code review), issue tracking, and often built-in CI/CD pipelines.
2. Continuous Integration (CI) / Continuous Delivery (CD) Tools:
   • Purpose: To automate the process of building, testing, and preparing code for deployment whenever changes are committed. They are the heart of the DevOps pipeline.
   • Key Tools:
     • Jenkins: An open-source automation server with a vast plugin ecosystem, making it highly flexible for automating builds, tests, and deployments for almost any project.
     • GitLab CI/CD: Built-in CI/CD functionality within the GitLab platform, offering a complete DevOps solution from code management to deployment.
     • GitHub Actions: A feature of GitHub that allows you to automate workflows directly within your GitHub repositories, including CI/CD.
     • CircleCI / Travis CI / Bamboo: Other popular CI/CD platforms, often cloud-based, that help automate the build and test process.
3. Containerization Tools:
   • Purpose: To package applications and their dependencies into isolated, portable “containers.” This ensures that software runs consistently across different environments (development, testing, production).
   • Key Tools:
     • Docker: The most popular containerization platform. It allows developers to package an application with all its libraries and dependencies into a “Docker image,” which can then be run as a “Docker container.”
     • Kubernetes (K8s): An open-source container orchestration platform. While Docker creates containers, Kubernetes manages and automates the deployment, scaling, and management of these containerized applications across clusters of machines. It ensures high availability and self-healing.
4. Configuration Management / Infrastructure as Code (IaC) Tools:
   • Purpose: To automate the provisioning, configuration, and management of infrastructure (servers, networks, databases) using code, rather than manual processes. This ensures consistency, repeatability, and reduces human error.
   • Key Tools:
     • Ansible: Agentless (doesn’t require software installed on the target machines) and uses simple YAML files (playbooks) to define configurations and automate tasks.
     • Terraform: An open-source IaC tool by HashiCorp that allows you to define and provision infrastructure across various cloud providers (AWS, Azure, GCP) and on-premises environments using a declarative language.
     • Chef / Puppet: Other popular configuration management tools that use a “master-agent” model to manage infrastructure.
5. Monitoring and Logging Tools:
   • Purpose: To collect, analyze, and visualize data (logs, metrics) from applications and infrastructure in real-time. This helps in identifying performance bottlenecks, errors, security issues, and understanding system health.
   • Key Tools:
     • Prometheus: An open-source monitoring system with a powerful query language and time-series database, often used for collecting metrics.
     • Grafana: A popular open-source data visualization and dashboarding tool that integrates well with Prometheus and many other data sources to create insightful dashboards.
     • ELK Stack (Elasticsearch, Logstash, Kibana): A suite for centralized logging and analysis. Logstash collects logs, Elasticsearch indexes and stores them, and Kibana provides powerful visualization.
     • Nagios / Zabbix: Traditional monitoring tools for network devices, servers, and applications.
6. Testing Tools:
   • Purpose: To automate various types of tests (unit, integration, functional, performance) throughout the development pipeline.
   • Key Tools:
     • Selenium: An open-source framework for automating web browser interactions, commonly used for UI (User Interface) testing.
     • JUnit / NUnit / TestNG: Frameworks for unit testing in various programming languages (Java, .NET, etc.).

This list covers the most frequently encountered and important DevOps tools. Remember that the “best” tools often depend on the specific needs of a project or organization.

Purpose	Tool Examples
Version Control	Git, SVN
Continuous Integration	Jenkins, Travis CI
Configuration Mgmt	Ansible, Puppet, Chef
Containerization	Docker, Podman
Orchestration – Coordination of multiple IT processes	Kubernetes, Docker Swarm
Monitoring	Nagios, Prometheus
Cloud Services	AWS, Azure, GCP

---

4. Core DevOps Concepts

Here are the most important core DevOps concepts:

1. Continuous Integration (CI):
   • Concept: Developers frequently merge their code changes into a central shared repository (e.g., Git). Every merge triggers automated builds and tests to quickly detect and address integration errors.
   • Why it’s Core: It ensures that the codebase is always in a working state, preventing “integration hell” where merging large, infrequent changes leads to massive, hard-to-resolve conflicts and bugs.
   • Memorize: “Frequent merges, automated builds, automated tests.”
2. Continuous Delivery (CD):
   • Concept: Building on CI, this concept ensures that software is always in a deployable state. After passing automated tests, the code is ready to be released to production at any time, though the final deployment might still require a manual push.
   • Why it’s Core: It means the software can be released on demand, reducing lead time from development to release and increasing flexibility.
   • Memorize: “Software always ready for release.”
3. Continuous Deployment (CD):
   • Concept: An extension of Continuous Delivery where every code change that passes all automated tests is automatically deployed to production without any human intervention.
   • Why it’s Core: Represents the highest level of automation in the delivery pipeline, leading to extremely fast release cycles.
   • Memorize: “Automated release to production after all tests pass.”
4. Infrastructure as Code (IaC):
   • Concept: Managing and provisioning computing infrastructure (like servers, networks, databases, storage) using machine-readable definition files (code), rather than manual hardware configuration or interactive configuration tools.
   • Why it’s Core: Ensures consistency, repeatability, and prevents “configuration drift.” Infrastructure can be version-controlled, tested, and deployed just like application code.
   • Memorize: “Infrastructure managed by code; automation, consistency.”
5. Monitoring and Logging (Observability):
   • Concept: Continuously collecting and analyzing data (logs, metrics, traces) from all parts of the application and infrastructure in real-time. This provides deep insights into performance, health, and user behavior. “Observability” implies being able to ask arbitrary questions about your system and get answers from its data.
   • Why it’s Core: Essential for quickly detecting issues, understanding their root causes, optimizing performance, and making informed decisions for continuous improvement. Without good monitoring, you don’t know if your “fast” deployments are actually working well.
   • Memorize: “Real-time data collection; insights into system health and performance.”
6. Shift-Left Security:
   • Concept: Integrating security practices and considerations earlier in the software development lifecycle (i.e., “shifting left” from the traditional end-of-lifecycle testing). This means security is baked into design, development, and testing phases.
   • Why it’s Core: Finding and fixing security vulnerabilities early is much cheaper and more efficient than finding them in production. It makes security a shared responsibility.
   • Memorize: “Integrate security early; security is everyone’s job.”
7. Automation Everywhere:
   • Concept: While CI/CD and IaC highlight specific areas, the underlying principle is to automate any repetitive or error-prone task across the entire value stream.
   • Why it’s Core: Reduces human error, speeds up processes, and allows teams to focus on innovation.
   • Memorize: “Automate repetitive tasks to reduce errors and increase speed.”
8. Feedback Loops:
   • Concept: Creating mechanisms to quickly gather information (from monitoring, user feedback, tests) and feed it back into the development process.
   • Why it’s Core: Enables rapid learning and continuous improvement. It allows teams to respond quickly to problems and adapt to changing requirements.
   • Memorize: “Collect info, learn, adapt.”

To simplify further for memorization, think of these core concepts as the “C-A-I-S-M-F” framework of DevOps:

• Continuous (CI/CD)
• Automation (everywhere)
• Infrastructure as Code (IaC)
• Shift-Left Security
• Monitoring (Observability)
• Feedback Loops

---

5. Popular DevOps Pipelines

Here are the most popular and commonly recognized stages of a DevOps pipeline:

1. Plan:
   • Purpose: Defining the project scope, requirements, user stories, and overall strategy. This is where ideas are formed and priorities are set.
   • Activities: Requirement gathering, sprint planning (Agile/Scrum), backlog management, risk assessment.
   • Common Tools (examples): Jira, Azure Boards, Trello, Asana (Project Management & Issue Tracking tools).
   • Hint: The starting point; focuses on “what to build.”
2. Code:
   • Purpose: Developers write, review, and commit source code.
   • Activities: Writing code, code reviews, static code analysis (checking for errors/security flaws in code itself), version control.
   • Common Tools: Git (version control), GitHub, GitLab, Bitbucket (repository hosting), IDEs (Integrated Development Environments like VS Code, IntelliJ IDEA).
   • Hint: Focus on “source code management” and “collaboration.”
3. Build:
   • Purpose: Compiling the source code into executable binaries or artifacts, and packaging them. This also involves resolving dependencies.
   • Activities: Compiling code, running unit tests, creating deployable packages (e.g., JAR files, Docker images), dependency management.
   • Common Tools: Jenkins, GitLab CI/CD, GitHub Actions, Azure Pipelines, CircleCI (CI/CD tools); Maven, Gradle (build automation for Java); Docker (for containerization).
   • Hint: The outcome is a “deployable artifact.” This stage often initiates the CI process.
4. Test:
   • Purpose: Ensuring the quality, functionality, performance, and security of the software. This phase involves various types of automated and sometimes manual tests.
   • Activities: Running automated unit tests, integration tests, functional tests, performance tests, security scans (SAST – Static Application Security Testing, DAST – Dynamic Application Security Testing), user acceptance testing (UAT – often manual).
   • Common Tools: Selenium, JUnit, TestNG (testing frameworks); SonarQube (code quality/security); various CI/CD tools (for orchestrating tests).
   • Hint: Emphasizes “automated testing” and “quality assurance” early in the cycle (Shift-Left).
5. Release (or Staging/Pre-production):
   • Purpose: Preparing the validated software for deployment, often to a production-like environment (staging) for final checks and approvals.
   • Activities: Creating release notes, tagging versions, obtaining approvals, deploying to a staging/pre-production environment for final validation.
   • Common Tools: CI/CD tools (Jenkins, GitLab CI/CD), release orchestration tools, sometimes artifact repositories (like Nexus, Artifactory).
   • Hint: Marks the point where the software is “ready to go live.”
6. Deploy:
   • Purpose: Automating the actual deployment of the software to the production environment, making it available to end-users.
   • Activities: Pushing code to production servers, updating configurations, managing infrastructure (IaC), using deployment strategies (e.g., blue/green, canary).
   • Common Tools: Kubernetes, Docker (for container orchestration/runtime); Ansible, Terraform, Chef, Puppet (IaC/configuration management); various CI/CD tools (for orchestrating deployments); Cloud services’ deployment tools (AWS CodeDeploy, Azure DevOps Pipelines).
   • Hint: The “go live” stage; focuses on “automation” and “consistency.”
7. Operate:
   • Purpose: Managing and maintaining the application and its underlying infrastructure in the production environment.
   • Activities: Infrastructure management, system administration, ensuring uptime, incident management.
   • Common Tools: Infrastructure as Code tools (Terraform, Ansible), cloud provider management consoles.
   • Hint: Focuses on “running the system” after deployment.
8. Monitor:
   • Purpose: Continuously observing the performance, health, security, and user experience of the application and infrastructure in production.
   • Activities: Collecting logs, metrics, and traces; setting up alerts; creating dashboards; analyzing performance data; gathering user feedback.
   • Common Tools: Prometheus, Grafana, ELK Stack (Elasticsearch, Logstash, Kibana), Datadog, Splunk (Monitoring & Logging tools).
   • Hint: Critical for “feedback loops” and “continuous improvement.” This feedback often loops back to the “Plan” stage for future iterations.

The “Infinite Loop” or “Cycle”:

---

6. Common DevOps Practices

• Version Control: Track code changes (e.g., Git).
• Automated Testing: Test code using tools (e.g., Selenium).
• Infrastructure Automation: Use scripts to set up servers (e.g., Ansible).
• Monitoring and Logging: Keep track of performance (e.g., ELK Stack).

---

7. Benefits of DevOps

• Faster delivery of software.
• Improved collaboration between teams.
• Early detection of bugs.
• Increased automation reduces manual effort.

---

8. Challenges in DevOps

• Resistance to change.
• Lack of skilled professionals.
• Tool integration complexity.
• Security concerns in CI/CD pipelines.

---

9. Cloud and DevOps

• DevOps often uses cloud platforms (AWS, Azure, GCP) for scalable infrastructure.
• Cloud provides flexibility, scalability, and cost-effectiveness for DevOps processes.

---

10. Key Terms to Remember

• Blue-Green Deployment: Deploy new changes without downtime by having two environments.
• Canary Deployment: Gradual release of updates to a small group before full rollout.
• Rolling Deployment: Updates are applied gradually without downtime.
• GitOps: Managing infrastructure using Git as a single source of truth.

---

11. Popular Exam Topics

• DevOps Lifecycle: CI/CD, Automation, Monitoring.
• Tools and Use Cases: Jenkins (CI), Docker (Containers), Kubernetes (Orchestration).
• IaC Concepts: Terraform, Ansible.
• Version Control: Git commands (clone, push, pull, merge).
• Containers vs Virtual Machines: Lightweight containers vs heavy VMs.

---

12. Important Commands

Git:

Git is for managing your code’s history and collaborating with others.

1. Basic Setup & Configuration:

* git config --global user.name "Your Name" * git config --global user.email "your_email@gyanarth.com"

Purpose: Sets your name and email for commits. (Important for tracking who did what).

2. Starting a Repository:

* git init

Purpose: Initializes a new Git repository in the current directory. (Starts tracking files). * git clone <repository_url> * Purpose: Creates a local copy of an existing remote repository. (Downloads code from GitHub/GitLab).

3. Making Changes & Committing:

* git status

Purpose: Shows the current status of your repository (modified, staged, untracked files). (Check what’s going on).

* git add <file_name> * Purpose: Stages a specific file for the next commit. (Prepares file for saving).

* git add .

Purpose: Stages all modified and new files in the current directory for the next commit. (Prepares all changes for saving).

* git commit -m "Your commit message"

Purpose: Records the staged changes to the repository history with a descriptive message. (Saves your work).

* git commit -am "Your commit message"

Purpose: Stages all modified and deleted files and commits them. (Does add and commit for tracked files). Doesn’t add new, untracked files.

4. Viewing History:

* git log

Purpose: Shows the commit history. (See who did what, when).

5. Syncing with Remote Repository:

* git pull

Purpose: Fetches changes from the remote repository and merges them into your current branch. (Downloads and applies updates from others). * git push * Purpose: Uploads your local commits to the remote repository. (Shares your saved work with others).

6. Branching (Crucial for Collaboration):

* git branch

Purpose: Lists all local branches.

* git branch <branch_name>

Purpose: Creates a new branch.

* git checkout <branch_name>

Purpose: Switches to an existing branch. (Moves to a different line of development).

* git checkout -b <new_branch_name>

Purpose: Creates a new branch AND switches to it. (Create and go to new line).

* git merge <branch_name>

Purpose: Integrates changes from a specified branch into the current branch. (Combines work from different lines).

* git branch -d <branch_name>

Purpose: Deletes a local branch (only if it’s merged).

---

Docker:

Docker is for packaging applications into portable “containers.”

1. Docker Images (Blueprints):

* docker build -t <image_name> .

Purpose: Builds a Docker image from a Dockerfile in the current directory. (. indicates current directory). (Creates the application package).

* docker images

Purpose: Lists all local Docker images. (See all your created packages).

* docker pull <image_name>:<tag>

Purpose: Downloads a Docker image from a registry (like Docker Hub). (Gets a pre-built application package).

* docker rmi <image_name>

Purpose: Removes a Docker image. (Deletes a package).

2. Docker Containers (Running Instances of Images):

* docker run -p 80:80 <image_name>

Purpose: Runs a Docker container from an image. -p 80:80 maps port 80 on your host to port 80 in the container. (Starts an instance of your application package).

* docker run -d <image_name>

Purpose: Runs a Docker container in detached mode (in the background).

* docker ps

Purpose: Lists all running Docker containers. (See what applications are currently active).

* docker ps -a

Purpose: Lists all Docker containers (running and stopped).

* docker stop <container_id/name>

Purpose: Stops a running container gracefully.

* docker kill <container_id/name>

Purpose: Forces a stop on a running container.

* docker rm <container_id/name>

Purpose: Removes a stopped container. (Deletes a stopped instance).

* docker exec -it <container_id/name> /bin/bash

Purpose: Executes a command inside a running container. -it allows interactive access to the container’s shell. (Go inside a running application to troubleshoot).

3. Docker Compose (Multi-Container Applications):

* docker-compose up -d

Purpose: Starts and runs all services defined in a docker-compose.yml file in detached mode. (Starts a whole set of interconnected applications).

* docker-compose down

Purpose: Stops and removes containers, networks, and volumes defined in the docker-compose.yml file. (Shuts down the whole set).

---

Kubernetes:

Kubernetes is for managing and scaling containerized applications across clusters of machines. kubectl is the command-line tool for interacting with a Kubernetes cluster.

1. Basic Cluster Interaction:

* kubectl get nodes

Purpose: Lists the nodes (worker machines) in your Kubernetes cluster. (See what machines are available).

* kubectl get pods

Purpose: Lists the pods (smallest deployable units in K8s) in your current namespace. (See your running application components).

* kubectl get deployments

Purpose: Lists the deployments, which manage the desired state of your pods. (See your application deployments).

* kubectl get services

Purpose: Lists the services, which provide network access to your pods. (See how your applications are exposed).

* kubectl get all

Purpose: Lists several common resources (pods, deployments, services, replicasets, etc.) in the current namespace.

2. Deploying and Managing Applications:

* kubectl apply -f <filename.yaml>

Purpose: Creates or updates Kubernetes resources (like Pods, Deployments, Services) defined in a YAML file. (Applies your application configuration). This is the most common way to deploy.

* kubectl delete -f <filename.yaml>

Purpose: Deletes Kubernetes resources defined in a YAML file. (Removes your application configuration).

* kubectl rollout status deployment/<deployment_name>

Purpose: Checks the status of a deployment rollout (e.g., if it’s successfully updated).

* kubectl scale deployment/<deployment_name> --replicas=<number>

Purpose: Scales the number of running instances (replicas) of a deployment. (Increase or decrease how many copies of your app are running).

3. Inspection and Debugging:

* kubectl describe pod <pod_name>

Purpose: Shows detailed information about a specific pod, including events, conditions, and resource usage. (Get detailed info about a specific running component).

* kubectl logs <pod_name>

Purpose: Displays the logs from a container in a pod. (See what your application is printing out).

* kubectl exec -it <pod_name> -- /bin/bash

Purpose: Executes a command inside a running container within a pod. -- separates kubectl options from the command to be executed in the container. (Go inside a running container to troubleshoot).

4. Namespaces:

* kubectl get namespaces

Purpose: Lists all namespaces in the cluster. Namespaces are a way to divide cluster resources.

* kubectl config set-context --current --namespace=<namespace_name>

Purpose: Sets the default namespace for subsequent kubectl commands.

---

13. Last-Minute Tips

• Remember core DevOps tools and their purposes.
• Understand CI/CD pipelines and how they improve software delivery.
• Focus on IaC tools like Ansible and Terraform.
• Know containerization basics with Docker and orchestration with Kubernetes.
• Practice basic commands for Git, Docker, and Kubernetes.
• Stay updated on popular DevOps practices and cloud integrations.

• X
• Instagram
• WhatsApp