In the ever-evolving landscape of cloud-native technologies, Kubernetes has emerged as a powerhouse for orchestrating containerized applications. Amazon Elastic Kubernetes Service (EKS) is a popular managed Kubernetes service, simplifying the deployment and operation of Kubernetes clusters on AWS. While EKS provides a robust foundation, addressing the challenge of securely exposing services to the public internet often involves navigating the intricacies of Ingress controllers.
In this blog post, I’ll delve into an alternative approach to managing external access to your EKS services using Cloudflare Tunnels. This solution not only simplifies the process but also enhances security by design. I’ll explore the integration of Cloudflare Tunnels with EKS, highlighting the benefits of managing infrastructure as code with Terraform.
Table of contents
Security at the core
I’d like to emphasize the security-centric decisions made for my EKS cluster: I’m running the cluster only in private subnets ensuring resources are not directly exposed to the public internet.
Here’s a snippet of Terraform code showcasing the deployment of EKS in private subnets:
locals {
subnets = 6
private_cidr_block = "10.90.16.0/20"
public_cidr_block = "10.90.32.0/20"
}
(...)
resource "aws_subnet" "private" {
count = local.subnets
vpc_id = aws_vpc.k8s.id
cidr_block = cidrsubnet(local.private_cidr_block, 6, count.index)
availability_zone = data.aws_availability_zones.available.names[count.index]
tags = {
Name = "k8s-private-${count.index}"
}
}
One of the main drivers of opting for private subnets as the primary environment for the EKS deployment is to establish a robust security foundation, in terms of:
- Isolation from external threats: By using private subnets, we create a network barrier that shields the infrastructure from external threarts. This isolation is particularly crucial for applications dealing with sensitive data or compliance requirements, where minimizing exposure is paramount. This is not my case, but I always keep this in mind.
- Controlled ingress and egress: Private subnets allow for fine-grained control over inbound and outbound traffic. This control is advantageous for implementing security groups, network ACLs, and other network-level policies, enabling you to define precisely what traffic is allowed to enter or exit the EKS cluster.
- Secure communication within the VPC: Services within the private subnets can communicate securely with each other without traversing the public internet. This private communication minimizes the risk of eavesdropping and interception, contributing to a more secure intra-cluster communication environment.
This enhanced security posture ensures that no resources, including worker nodes and control plane components, are directly accessible from the public internet significantly reducing the attach surface, mitigating the risk of an unauthorized access and potential securyt breaches.
As always, there are some drawbacks that needs to be considered:
- Increased complexity for external access: Traditional methods of exposing services, such as using an Ingress controller, face challenges when operating within a private network. This is where innovative solutions like Cloudflare Tunnels come into play.
- Aditional configuration overhead: Managing an EKS cluster in private subnets may require additional configuration steps compared to a setup with public subnets. For example, setting up NAT gateways or NAT instances is necessary for worker nodes to communicate with external resources like container registries or AWS services.
Bridging the Gap: Cloudflare Tunnels for External Access
With the foundational understanding of the security benefits of running Amazon EKS in private subnets, the next critical consideration is how to securely expose services to the external world. Traditional approaches face challenges in this private network setup. However, innovating the way we handle external access is where Cloudflare Tunnels steps in as a game-changer.
Running Cloudflare Tunnel as sidecar
Considering that only certain services requries access from the public internet,
the best way to expose them is running the cloudflare tunnel service right next
to them. Using a sidecar container within the same pod allow us to configure
cloudflare tunnel to use http://localhost:5000 to establish the connection.
apiVersion: apps/v1
kind: Deployment
metadata:
name: some-app
spec:
replicas: 3
selector:
matchLabels:
app: some-app
spec:
volumes:
- name: some-app-data
secret:
secretName: some-app-config
containers:
- name: cf-tunnel
image: cloudflare/cloudflared
env:
- name: CF_TOKEN
valueFrom:
secretKeyRef:
name: cf-tunnel-token-some-app
key: token
args:
- tunnel
- run
- --token
- $(CF_TOKEN)
- name: some-app
image: some-image:some-tag
ports:
- containerPort: 5000
Worth noting that the configuration above assumes the existance of some resources:
- Secret for
CF_TOKEN - Image for
some-image:some-tag
Deployment of new services
Now that we have established a secure foundation with EKS in private subnets and addressed external access challenges using Cloudflare Tunnels, the focus turns to the efficient deployment of new services within this robust architecture. Enter ArgoCD, THE GitOps-driver continuous delivery tool that everybody is using (or should be using).
As mentioned, ArgoCD follows the GitOps methodology, allowing us to define and manage our application configurations declaratively in git repositories. This approach brings numerous advantages, including version control, auditability, and the ability to roll back to a known, stable state in case of issues.
ArgoCD in action
Just for educational purposes, let’s ilustrate the deployment of a new service using ArgoCD with a very simplified example.
- Create the application definition, as we did here
- Push the application definition to git.
- Automatic synchronization: ArgoCD will detect the changes and automatically synchronizes the updated configuration to our EKS cluster.
All the above assumes we already have an application defined and configured in ArgoCD. This can also be done using terraform:
resource "argocd_application" "kustomize" {
metadata {
name = "kustomize-app"
namespace = "argocd"
labels = {
test = "true"
}
}
cascade = false # disable cascading deletion
wait = true
spec {
project = "myproject"
destination {
server = "https://kubernetes.default.svc"
namespace = "foo"
}
source {
repo_url = "https://github.com/kubernetes-sigs/kustomize"
path = "examples/helloWorld"
target_revision = "master"
kustomize {
name_prefix = "foo-"
name_suffix = "-bar"
images = ["hashicorp/terraform:light"]
common_labels = {
"this.is.a.common" = "la-bel"
"another.io/one" = "true"
}
}
}
sync_policy {
automated {
prune = true
self_heal = true
allow_empty = true
}
# Only available from ArgoCD 1.5.0 onwards
sync_options = ["Validate=false"]
retry {
limit = "5"
backoff {
duration = "30s"
max_duration = "2m"
factor = "2"
}
}
}
ignore_difference {
group = "apps"
kind = "Deployment"
json_pointers = ["/spec/replicas"]
}
ignore_difference {
group = "apps"
kind = "StatefulSet"
name = "someStatefulSet"
json_pointers = [
"/spec/replicas",
"/spec/template/spec/metadata/labels/bar",
]
# Only available from ArgoCD 2.1.0 onwards
jq_path_expressions = [
".spec.replicas",
".spec.template.spec.metadata.labels.bar",
]
}
}
}
(Source: terraform-provider-argocd/examples)
Closing thoughts
In summary, the decision to run the EKS cluster in private subnets is a strategic move to fortify the security posture of the infrastructure. It aligns with the principle of least privilege, restricting access to only the necessary entities. While it introduces certain complexities, these can be effectively addressed with thoughtful design and the integration of solutions like Cloudflare Tunnels to facilitate secure external access. The enhanced security and control offered by private subnets outweigh the challenges, making it a prudent choice for safeguarding your Kubernetes deployments.