Terraform
Before Terraform(IaC)
Everything was done manually before Infrastructure as Code came into the picture.Any component that helps to run application is Infrastructure Engineering.
We have servers,storage and networking and different components,helping us to deploy and run applications.
At that time earlier IT Infrastructure engineering and management was tough Job ,as everything was done manually.
System Administrators were the one to manage and configure all hardware and software manually to run applications.
Traditional process to put servers physically in place and configure them to deploy and start using application.
Applications were deployed on machines only after correct setting and components configuration as per OS.
Major issues faced were:-
1.Time Taken to set-up infra.
2.Scalability and availability
As manual configuration is very slow,applications used to strugle with spike in access and system engineers and administrators get into loop of setting up servers and managing load.
Eg:- Imagine manual process of going,adding and deploying more servers in place to manage load with increase in users.Thus, IaC comes and helps to overcome manual process.
What is IaC
IaC is process of managing & provisioning computer data centers through machine readable definition files,rather than physical hardware configuration or interactive configuration tools.
Writing code could be in YAML/JSON & by executing same we get Infrastructure.Consider setting IT Infra(VMS, storage,networking components) creating and executing code configuration files.
Infrastructure Configuration takes form of a code file.
All requirements :-
1.how many servers
2.storage capacity
3.networking capacity
4.different software components
Everything gets defined in code in text format.Thua,it is easy to copy,edit,distribute and re-use.
These all code can be edit at any point of time,can be copied and create new version from it as well.
Shared to other teams to start to set-up new infrastructure.
Why IaC
Company adopting IaC , have benefits as follows:-
1.Speed :- Simple and fast run well-maintained scripts to setup infrastructure quickly.
Same Scripts for different - different environment:-
1.Development to Production.
2.Passing through staging.
3.QA,UAT,POC,Pre-Prod,etc.
Entire Development lifecycle becomes shorter,quicker and efficient.
2.Consistency :-
Imagine manually Setting up Infrastucture for dev environment ,then Staging, then UAT, then pre-prod, POC, Production,etc.
High Chance of making mistakes.
Any manual process increases high risk:-
1.misconfiguration
2.forget something while working up with different environment.
We humans may fail.Our memory and communication may get bad/difficult usually.
N number of discrepancies will be there, no matter how hard our team is working for it.
Well maintained config code IaC files are just like single point of contact to get same quality infra no matter where script is being run.
Same configurations will be deployed over and over again tat is consistency.
3.Accountability:-
Manually difficult to know who has setup infrastructure and when it was set-up.(Bad Practices given by leaders passed manager and seniors creds to juniors.All Circle- Circle. )
Getting accountability becomes very difficult.
With this IaC, because we are maintaining everything as code.
We can utilize it with version control and can identify who made change,when change was done.
No more guessing and blaming games on each other, who did what and when type of harassment. This is the main benefit of traceability.
Terraform Introduction
Terraform is a popular IaC tool started by HashiCorp and contributed to and by the open source community initially.
HashiCorp is a company specializing in producing tools and applications for DevOps, security and cloud infrastructure management.
Terraform itself is a cloud agnostic infrastructure provisioning tool that helps to ease the burden of infrastructure builds and maintenance.
Apart from open source version of Terraform, installed locally, Terraform is also available in a cloud and enterprise edition.
| Feature | Open Source | Terraform Cloud | Terraform Enterprise |
| Cost | Free | Paid SaaS model | Paid, self-hosted license |
| Deployment | Local machine | Hosted by HashiCorp | On-premises or cloud (self-hosted) |
| Collaboration | Limited (manual) | Workspaces, team management | Workspaces, team management, multi-tenancy |
| State Management | Local storage (manual) | Remote state storage (optional) | Remote state storage with locking and access control |
| Integrations | Limited | Integrations with various cloud providers and tools | Extensive integrations |
| Support | Community forums | Paid support plans | Paid support plans |
Terraform Enterprise is focused on being a self-hosted solution addressing the needs of data localization and operational security policies.
Terraform Cloud is Hashicorp's managed service offering.
| Feature | Public Cloud | Private Cloud | Hybrid Cloud | Multi-Cloud |
| Deployment Model | Shared resources over the internet | Dedicated infrastructure for a single organization | Combination of public and private clouds | Multiple public clouds from different providers |
| Management | By cloud service provider | By the organization or a third-party | By the organization (both private and public) | By the organization |
| Cost | Pay-as-you-go | Upfront investment and ongoing maintenance | Varies depending on usage of both private and public resources | Varies depending on chosen cloud providers |
| Security | Shared responsibility model | Highest level of control | Balance between control and scalability | Varies depending on chosen cloud providers |
| Scalability | Highly scalable | Limited scalability | Scalable by leveraging public cloud resources | High scalability across multiple cloud providers |
| Use Cases | Non-critical workloads, development/testing, startups | Sensitive data, applications requiring high control | Workloads requiring both control and scalability | Leveraging best-of-breed services from different providers, avoiding vendor lock-in |
Terraform Alternatives
AWS CloudFormation: While working on AWS platform,AWS CloudFormation service is used to model and provision cloud infra using JSON or YAML templates.
Azure Resource Manager (ARM): Microsoft Azure service to define your infrastructure for Azure using JSON or Bicep templates.
Pulumi:- Good IaC tool for developer who can utilize Java/ Python/ .NET / Javascript / C# / Go / Typescript etc.
Google Cloud Deployment Manager:-
It is Google's Infrastructure Deployment Service.
YAML or Python scripts.
Ansible: An open-source automation tool that for infrastructure provisioning(to some-extent compared to Terraform), configuration management, and application deployment.It can have procedural approach and different capabilities than Terraform.Agentless architecture & YAML for playbook definitions
Salt-Stack:- Another configuration management tool that can manage infrastructure.Event-driven automation written in YAML.
| Terraform | Chef/Puppet/Ansible | |
| Primary Use Case | Infrastructure Provisioning | Configuration Management and Application Deployment |
| Approach | Declarative (What to achieve) | Procedural/Imperative (How to achieve) |
| State Management | Manages and tracks infrastructure state | Limited state management, focuses on desired system state |
| Focus | Infrastructure layer | Configuration layer and system state |
| Use with Other Tools | Can be used alongside for best of both worlds | Often used in conjunction with |
| Ecosystem Integration | Wide range, suitable for multi-cloud | Strong in system configurations, less focused on multi-cloud |
Reactivity to Infrastructure State Changes:
Event-driven automation reacts to events or changes in your environment. For example, if a new virtual machine is created or a particular threshold is exceeded, it can trigger automated tasks or workflows.
IaC can be part of the response to such events. For example, if monitoring tools detect a high load on a server, an event-driven system can trigger Terraform to provision additional resources according to a predefined template.
Automated Infrastructure Management:
With IaC, infrastructures are defined using code, which makes it easier to create, update, and delete resources systematically and reproducibly.
Event-driven automation can automatically apply these IaC definitions in response to specific triggers or conditions, thereby ensuring that the infrastructure can dynamically adapt to changes in demand or configuration.
Integration in CI/CD Pipelines:
IaC is often integrated into Continuous Integration/Continuous Deployment (CI/CD) pipelines to automate the deployment of infrastructure alongside application code.
Event-driven automation can enhance these pipelines by triggering infrastructure updates or rollbacks based on events such as a successful build, a failed test, or deployment completion.
Enhancing Security and Compliance:
IaC ensures that the infrastructure is provisioned in a consistent, repeatable manner, which is crucial for maintaining security and compliance standards.
Event-driven automation can monitor for compliance drift or security incidents and automatically apply IaC templates to rectify issues, ensuring that the infrastructure remains in the desired state.
Scalability and Efficiency:
IaC allows for efficient scaling of infrastructure by defining it in code, which can be quickly executed to match demand.
Event-driven automation can detect when scaling is needed, either up or down, and execute the relevant IaC procedures to adjust the infrastructure accordingly without human intervention.
Why Terraform
1.For Professional use-case, while having my hands dirty with open source version of Terraform using it to provision AWS infrastructure, will further lead me to provision multi-cloud deployments later-on.
2.The infrastructure that Terraform manages can be deployed on public cloud platforms like Amazon Web Services, Azure, and Google Cloud Platform. It can also be used for on-premises or private cloud environments, including OpenStack, VMware vSphere, or CloudStack.
3.Terraform Infrastructure integrations also allow you to manage software and services including databases like MySQL, source control systems like GitHub, configuration management tools like Chef and Puppet and much more.
Eg:- over 100 publicly available infrastructure integrations
4.codifying infrastructure requirements into machine readable definition files.
In doing so, we are effectively creating executable documentation.
5.New project team-mate can examine the projects infrastructure as code templates and immediately understand the infrastructure configuration.
6.using code to generate infrastructure, the same environment can not only be recreated multiple times, it can be done so consistently without era or unintentional divergence.
7.infrastructure as code can address environmental drift.situations where the initial infrastructure is drifted away from the initial day zero configuration.Comparing the current site of an infrastructure and baselining it back against your existing infrastructure as code templates, you can deduce any drift and receipt it back to the recorded baseline.
8.templates can be stored in a version control system such as Git allowing teams to collaborate on infrastructure.
9.With specific code version from git, team create their own development or test environments.
10.a pain point that often existed for developers before moving to cloud infrastructure, were the delays encountered with operations, having to budget, plan, create and deliver physical infrastructure.
11.Now with the elasticity of the cloud allowing resources to be created on demand, developers can instead provision the infrastructure they need when they need it.
12.Terraform, the open source version is packaged into a single executable file lightweight and easy to install regardless of operating system.
13.After installing, features can be accessed via the terminal.
14.Validate & Plan infrastructure within local environment first.
15.Later on in the project lifecycle, you can always use the Terraform destroy command to destroy your infrastructure if and when required.
Terraform with AWS
TBC