Documentation

Karpenter is an open-source node provisioning project built for Kubernetes. Adding Karpenter to a Kubernetes cluster can dramatically improve the efficiency and cost of running workloads on that cluster. Karpenter works by:

Watching for pods that the Kubernetes scheduler has marked as unschedulable
Evaluating scheduling constraints (resource requests, nodeselectors, affinities, tolerations, and topology spread constraints) requested by the pods
Provisioning nodes that meet the requirements of the pods
Scheduling the pods to run on the new nodes
Removing the nodes when the nodes are no longer needed

As someone using Karpenter, once your Kubernetes cluster and the Karpenter controller are up and running (see Getting Started, you can:

Set up provisioners: By applying a provisioner to Karpenter, you can configure constraints on node provisioning and set timeout values for node expiry or Kubelet configuration values. Provisioner-level constraints related to Kubernetes and your cloud provider (AWS, for example) include:
- Taints (taints): Identify taints to add to provisioned nodes. If a pod doesn’t have a matching toleration for the taint, the effect set by the taint occurs (NoSchedule, PreferNoSchedule, or NoExecute). See Kubernetes Taints and Tolerations for details.
- Labels (labels): Apply arbitrary key-value pairs to nodes that can be matched by pods.
- Requirements (requirements): Set acceptable (In) and unacceptable (Out) Kubernetes and Karpenter values for node provisioning based on Well-Known Labels and cloud-specific settings. These can include instance types, zones, computer architecture, and capacity type (such as AWS spot or on-demand).
- Limits (limits): Lets you set limits on the total CPU and Memory that can be used by the cluster, effectively stopping further node provisioning when those limits have been reached.
Deploy workloads: When deploying workloads, you can request that scheduling constraints be met to direct which nodes Karpenter provisions for those workloads. Use any of the following Pod spec constraints when you deploy pods:
- Resources (resources): Make requests and set limits for memory and CPU for a Pod. See Requests and limits for details.
- Nodes (nodeSelector): Use nodeSelector to ask to match a node that includes one or more selected key-value pairs. These can be arbitrary labels you define, Kubernetes well-known labels, or Karpenter labels.
- Node affinity (NodeAffinity): Set nodeAffinity to have the Pod run on nodes that have matching nodeSelectorTerms set or not set. Matching affinity can be a particular operating system or zone. You can set the node affinity to be required or simply preferred. NotIn and DoesNotExist allow you to define node anti-affinity behavior.
- Pod affinity and anti-affinity (podAffinity/podAntiAffinity): Choose to run a pod on a node based on whether certain pods are running (podAffinity) or not running (podAntiAffinity) on the node. See Inter-pod affinity and anti-affinity for details.
- Tolerations (tolerations): Identify that a pod must match (tolerate) a taint on a node before the pod will run on it. Without the toleration, the effect set by the taint occurs (NoSchedule, PreferNoSchedule, or NoExecute). See Kubernetes Taints and Tolerations for details.
- Topology spread (topologySpreadConstraints): Request that pods be spread across zones (topology.kubernetes.io/zone) or hosts (kubernetes.io/hostname), or cloud provider capacity types (karpenter.sh/capacity-type). See Pod Topology Spread Constraints for details.
- Persistent volume topology: Indicate that the Pod has a storage requirement that requires a node running in a particular zone that can make that storage available to the node.