[Daily morning study] Kubernetes Admission Controller์™€ OPA(Open Policy Agent)

#daily morning study

Image


Admission Controller๋ž€

Kubernetes API ์„œ๋ฒ„๋กœ ๋“ค์–ด์˜ค๋Š” ์š”์ฒญ์€ ์ธ์ฆ(Authentication) โ†’ ์ธ๊ฐ€(Authorization) โ†’ Admission ์ˆœ์„œ๋กœ ์ฒ˜๋ฆฌ๋œ๋‹ค.

Admission Controller๋Š” ์ด ์„ธ ๋ฒˆ์งธ ๋‹จ๊ณ„์—์„œ ๋™์ž‘ํ•˜๋ฉฐ, ์˜ค๋ธŒ์ ํŠธ๊ฐ€ ํด๋Ÿฌ์Šคํ„ฐ์— ์‹ค์ œ๋กœ ๋ฐ˜์˜๋˜๊ธฐ ์ „์— ์š”์ฒญ์„ ๊ฐ€๋กœ์ฑ„์„œ ๊ฒ€์ฆํ•˜๊ฑฐ๋‚˜ ๋ณ€๊ฒฝํ•˜๋Š” ํ”Œ๋Ÿฌ๊ทธ์ธ์ด๋‹ค.

kubectl apply โ†’ API Server โ†’ Authn โ†’ Authz โ†’ [Admission] โ†’ etcd ์ €์žฅ

๋‘ ๊ฐ€์ง€ ํƒ€์ž…์ด ์žˆ๋‹ค.

ํƒ€์ž…์—ญํ• 
Mutating Admission์š”์ฒญ์„ ์ˆ˜์ • (์˜ˆ: ๊ธฐ๋ณธ๊ฐ’ ์ฃผ์ž…, ์‚ฌ์ด๋“œ์นด ์ž๋™ ์ถ”๊ฐ€)
Validating Admission์š”์ฒญ์„ ๊ฒ€์ฆ ํ›„ ํ—ˆ์šฉ ๋˜๋Š” ๊ฑฐ๋ถ€

Mutating์ด ๋จผ์ € ์‹คํ–‰๋œ ํ›„ Validating์ด ์‹คํ–‰๋œ๋‹ค.


๋นŒํŠธ์ธ Admission Controller

Kubernetes์—๋Š” ๊ธฐ๋ณธ์œผ๋กœ ๋‚ด์žฅ๋œ Admission Controller๋“ค์ด ์žˆ๋‹ค.

  • NamespaceLifecycle: ์‚ญ์ œ ์ค‘์ธ ๋„ค์ž„์ŠคํŽ˜์ด์Šค์— ๋ฆฌ์†Œ์Šค ์ƒ์„ฑ ๊ธˆ์ง€
  • LimitRanger: ๋ฆฌ์†Œ์Šค ์š”์ฒญ/์ œํ•œ ๊ธฐ๋ณธ๊ฐ’ ์ ์šฉ
  • ServiceAccount: ํŒŒ๋“œ์— ์„œ๋น„์Šค ๊ณ„์ • ์ž๋™ ๋งˆ์šดํŠธ
  • ResourceQuota: ๋„ค์ž„์ŠคํŽ˜์ด์Šค๋ณ„ ๋ฆฌ์†Œ์Šค ์ด๋Ÿ‰ ์ œํ•œ
  • PodSecurity: Pod Security Standards ์ ์šฉ (PSP ๋Œ€์ฒด)
  • MutatingAdmissionWebhook: ์™ธ๋ถ€ ์›นํ›… ํ˜ธ์ถœ (๋ณ€๊ฒฝ)
  • ValidatingAdmissionWebhook: ์™ธ๋ถ€ ์›นํ›… ํ˜ธ์ถœ (๊ฒ€์ฆ)

Webhook ๊ธฐ๋ฐ˜ Admission Controller

๋นŒํŠธ์ธ์œผ๋กœ๋Š” ๋ณต์žกํ•œ ์ •์ฑ… ํ‘œํ˜„์— ํ•œ๊ณ„๊ฐ€ ์žˆ๋‹ค. ์ด๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด Kubernetes๋Š” ์™ธ๋ถ€ HTTP ์„œ๋ฒ„๋กœ ์š”์ฒญ์„ ์œ„์ž„ํ•˜๋Š” Webhook ๋ฐฉ์‹์„ ์ง€์›ํ•œ๋‹ค.

๋™์ž‘ ํ๋ฆ„

kubectl apply
    โ†“
API Server
    โ†“
MutatingAdmissionWebhook ์„ค์ • ํ™•์ธ
    โ†“
์™ธ๋ถ€ ์›นํ›… ์„œ๋ฒ„๋กœ AdmissionReview ์ „์†ก (HTTP POST)
    โ†“
์›นํ›… ์„œ๋ฒ„: ํ—ˆ์šฉ(allow) / ๊ฑฐ๋ถ€(deny) / ํŒจ์น˜(patch) ์‘๋‹ต
    โ†“
ValidatingAdmissionWebhook ์„ค์ • ํ™•์ธ
    โ†“
์ตœ์ข… ๊ฒฐ์ •

MutatingWebhookConfiguration ์˜ˆ์‹œ

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: my-mutating-webhook
webhooks:
  - name: inject-sidecar.example.com
    clientConfig:
      service:
        name: webhook-service
        namespace: webhook-system
        path: "/mutate"
    rules:
      - operations: ["CREATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]
    admissionReviewVersions: ["v1"]
    sideEffects: None
    failurePolicy: Fail   # ์›นํ›… ์‹คํŒจ ์‹œ ์š”์ฒญ ๊ฑฐ๋ถ€

OPA(Open Policy Agent)๋ž€

OPA๋Š” ์ •์ฑ…(Policy)์„ ์ฝ”๋“œ๋กœ ํ‘œํ˜„ํ•˜๊ณ  ์ง‘ํ–‰ํ•˜๋Š” ๋ฒ”์šฉ ์ •์ฑ… ์—”์ง„์ด๋‹ค. ์ฟ ๋ฒ„๋„คํ‹ฐ์Šค์— ์ข…์†๋˜์ง€ ์•Š์œผ๋ฉฐ ์„œ๋น„์Šค ๋ฉ”์‹œ, API ๊ฒŒ์ดํŠธ์›จ์ด, ๋งˆ์ดํฌ๋กœ์„œ๋น„์Šค ๋“ฑ ๋‹ค์–‘ํ•œ ํ™˜๊ฒฝ์—์„œ ํ™œ์šฉ๋œ๋‹ค.

ํ•ต์‹ฌ ๊ตฌ์„ฑ์š”์†Œ

๊ตฌ์„ฑ์š”์†Œ์„ค๋ช…
RegoOPA ์ „์šฉ ์ •์ฑ… ์–ธ์–ด
PolicyRego๋กœ ์ž‘์„ฑ๋œ ๊ทœ์น™ ์ง‘ํ•ฉ
Inputํ‰๊ฐ€ ๋Œ€์ƒ ๋ฐ์ดํ„ฐ (JSON)
Data์ •์ฑ… ํŒ๋‹จ์— ์ฐธ๊ณ ํ•˜๋Š” ์™ธ๋ถ€ ๋ฐ์ดํ„ฐ

Rego ์–ธ์–ด ๊ธฐ์ดˆ

Rego๋Š” ์„ ์–ธํ˜• ์ฟผ๋ฆฌ ์–ธ์–ด๋‹ค. deny ๊ทœ์น™์ด ํ•˜๋‚˜๋ผ๋„ ์ฐธ์ด๋ฉด ์š”์ฒญ์ด ๊ฑฐ๋ถ€๋œ๋‹ค.

package kubernetes.admission

# ํŠน๊ถŒ ์ปจํ…Œ์ด๋„ˆ ๊ธˆ์ง€
deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    container.securityContext.privileged == true
    msg := sprintf("์ปจํ…Œ์ด๋„ˆ '%v'์— privileged ๋ชจ๋“œ๊ฐ€ ํ—ˆ์šฉ๋˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค", [container.name])
}

# latest ํƒœ๊ทธ ์ด๋ฏธ์ง€ ๊ธˆ์ง€
deny[msg] {
    input.request.kind.kind == "Pod"
    container := input.request.object.spec.containers[_]
    endswith(container.image, ":latest")
    msg := sprintf("์ปจํ…Œ์ด๋„ˆ '%v'์— ':latest' ํƒœ๊ทธ ์ด๋ฏธ์ง€๋Š” ์‚ฌ์šฉํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค", [container.name])
}

Gatekeeper: OPA์˜ Kubernetes ํ†ตํ•ฉ

Gatekeeper๋Š” OPA๋ฅผ Kubernetes Admission Controller๋กœ ํ†ตํ•ฉํ•˜๋Š” ํ”„๋กœ์ ํŠธ๋‹ค. CRD(Custom Resource Definition) ๊ธฐ๋ฐ˜์œผ๋กœ ์ •์ฑ…์„ ์„ ์–ธ์ ์œผ๋กœ ๊ด€๋ฆฌํ•œ๋‹ค.

์ฃผ์š” CRD

CRD์—ญํ• 
ConstraintTemplateRego ์ •์ฑ… ์ •์˜ ๋ฐ CRD ์ƒ์„ฑ
Constraint์‹ค์ œ ์ •์ฑ… ์ธ์Šคํ„ด์Šค (๋Œ€์ƒ ๋ฆฌ์†Œ์Šค ์ง€์ •)

ConstraintTemplate ์˜ˆ์‹œ

apiVersion: templates.gatekeeper.sh/v1
kind: ConstraintTemplate
metadata:
  name: k8srequiredlabels
spec:
  crd:
    spec:
      names:
        kind: K8sRequiredLabels
      validation:
        openAPIV3Schema:
          type: object
          properties:
            labels:
              type: array
              items:
                type: string
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8srequiredlabels

        violation[{"msg": msg}] {
          provided := {label | input.review.object.metadata.labels[label]}
          required := {label | label := input.parameters.labels[_]}
          missing := required - provided
          count(missing) > 0
          msg := sprintf("ํ•„์ˆ˜ ๋ ˆ์ด๋ธ”์ด ์—†์Šต๋‹ˆ๋‹ค: %v", [missing])
        }

Constraint ์˜ˆ์‹œ (์ •์ฑ… ์ ์šฉ)

apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
  name: pod-must-have-team-label
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
    namespaces: ["production"]
  parameters:
    labels: ["team", "app"]

์ด ์„ค์ •์œผ๋กœ production ๋„ค์ž„์ŠคํŽ˜์ด์Šค์˜ ๋ชจ๋“  ํŒŒ๋“œ๋Š” team๊ณผ app ๋ ˆ์ด๋ธ”์ด ์—†์œผ๋ฉด ๋ฐฐํฌ๊ฐ€ ๊ฑฐ๋ถ€๋œ๋‹ค.


Gatekeeper Audit ๊ธฐ๋Šฅ

Gatekeeper๋Š” ์ƒˆ๋กœ ๋“ค์–ด์˜ค๋Š” ์š”์ฒญ๋งŒ ๊ฒ€์ฆํ•˜๋Š” ๊ฒŒ ์•„๋‹ˆ๋ผ, ์ด๋ฏธ ํด๋Ÿฌ์Šคํ„ฐ์— ์กด์žฌํ•˜๋Š” ๋ฆฌ์†Œ์Šค๊ฐ€ ์ •์ฑ…์„ ์œ„๋ฐ˜ํ•˜๋Š”์ง€๋„ ์ฃผ๊ธฐ์ ์œผ๋กœ ๊ฐ์‚ฌ(Audit)ํ•œ๋‹ค.

# ์ •์ฑ… ์œ„๋ฐ˜ ํ˜„ํ™ฉ ํ™•์ธ
kubectl get constraint pod-must-have-team-label -o yaml

# ์œ„๋ฐ˜ ๋ชฉ๋ก์€ .status.violations ํ•„๋“œ์— ํ‘œ์‹œ๋จ
status:
  violations:
    - enforcementAction: deny
      kind: Pod
      name: legacy-pod
      namespace: production
      message: "ํ•„์ˆ˜ ๋ ˆ์ด๋ธ”์ด ์—†์Šต๋‹ˆ๋‹ค: {\"team\"}"

enforcementAction ์„ค์ •

์‹ค์ œ ์šด์˜ ํ™˜๊ฒฝ์—์„œ ์ƒˆ ์ •์ฑ…์„ ๋ฐ”๋กœ deny๋กœ ์ ์šฉํ•˜๋ฉด ๊ธฐ์กด ์›Œํฌ๋กœ๋“œ์— ์˜ํ–ฅ์„ ์ค„ ์ˆ˜ ์žˆ๋‹ค. ์ ์ง„์  ์ ์šฉ์„ ์œ„ํ•œ ์˜ต์…˜์ด ์žˆ๋‹ค.

enforcementAction๋™์ž‘
deny์ •์ฑ… ์œ„๋ฐ˜ ์‹œ ์š”์ฒญ ๊ฑฐ๋ถ€
warn๊ฒฝ๊ณ  ๋ฐ˜ํ™˜, ์š”์ฒญ์€ ํ—ˆ์šฉ (k8s 1.19+)
dryrun๊ฐ์‚ฌ๋งŒ ์ˆ˜ํ–‰, ์‹ค์ œ ๊ฑฐ๋ถ€ ์—†์Œ
spec:
  enforcementAction: warn   # ๋จผ์ € warn์œผ๋กœ ์˜ํ–ฅ๋„ ํŒŒ์•…

Admission Controller ์„ค๊ณ„ ์‹œ ๊ณ ๋ ค์‚ฌํ•ญ

failurePolicy

์›นํ›… ์„œ๋ฒ„๊ฐ€ ์‘๋‹ตํ•˜์ง€ ์•Š์„ ๋•Œ์˜ ๋™์ž‘์„ ๊ฒฐ์ •ํ•œ๋‹ค.

failurePolicy: Fail    # ์›นํ›… ์‹คํŒจ โ†’ ์š”์ฒญ ๊ฑฐ๋ถ€ (๋ณด์•ˆ ์šฐ์„ )
failurePolicy: Ignore  # ์›นํ›… ์‹คํŒจ โ†’ ์š”์ฒญ ํ—ˆ์šฉ (๊ฐ€์šฉ์„ฑ ์šฐ์„ )

ํ”„๋กœ๋•์…˜์—์„œ๋Š” Fail์ด ๊ธฐ๋ณธ์ด์ง€๋งŒ, ์›นํ›… ์„œ๋ฒ„ ์ž์ฒด์˜ ๊ณ ๊ฐ€์šฉ์„ฑ(HA)์„ ๋ฐ˜๋“œ์‹œ ํ™•๋ณดํ•ด์•ผ ํ•œ๋‹ค.

namespaceSelector

์›นํ›… ์ ์šฉ ๋Œ€์ƒ ๋„ค์ž„์ŠคํŽ˜์ด์Šค๋ฅผ ๋ ˆ์ด๋ธ”๋กœ ํ•„ํ„ฐ๋งํ•œ๋‹ค. ์‹œ์Šคํ…œ ๋„ค์ž„์ŠคํŽ˜์ด์Šค(kube-system)๋Š” ์ •์ฑ… ์ ์šฉ์—์„œ ์ œ์™ธํ•˜๋Š” ๊ฒฝ์šฐ๊ฐ€ ๋งŽ๋‹ค.

namespaceSelector:
  matchExpressions:
    - key: admission-control
      operator: In
      values: ["enabled"]

์„ฑ๋Šฅ ์˜ํ–ฅ

๋ชจ๋“  API ์š”์ฒญ์ด ์›นํ›…์„ ๊ฑฐ์น˜๋ฏ€๋กœ ์›นํ›… ์‘๋‹ต ์ง€์—ฐ์ด ํด๋Ÿฌ์Šคํ„ฐ ์ „์ฒด์— ์˜ํ–ฅ์„ ์ค€๋‹ค. timeoutSeconds๋ฅผ ํ•ฉ๋ฆฌ์ ์œผ๋กœ ์„ค์ •ํ•˜๊ณ  ์›นํ›… ์„œ๋ฒ„ ์‘๋‹ต ์‹œ๊ฐ„์„ ๋ชจ๋‹ˆํ„ฐ๋งํ•ด์•ผ ํ•œ๋‹ค.


OPA vs Kyverno ๋น„๊ต

Kyverno๋Š” Kubernetes ์ „์šฉ์œผ๋กœ ์„ค๊ณ„๋œ ๋˜ ๋‹ค๋ฅธ ์ •์ฑ… ์—”์ง„์ด๋‹ค.

ํ•ญ๋ชฉOPA/GatekeeperKyverno
์ •์ฑ… ์–ธ์–ดRego (๋ฒ”์šฉ, ํ•™์Šต ๊ณก์„  ๋†’์Œ)YAML (k8s ๋„ค์ดํ‹ฐ๋ธŒ)
์ ์šฉ ๋ฒ”์œ„k8s ์™ธ ๋‹ค์–‘ํ•œ ํ™˜๊ฒฝ ๊ฐ€๋Šฅk8s ์ „์šฉ
Mutate ์ง€์›์ œํ•œ์ ๊ฐ•๋ ฅํ•œ Mutate ์ง€์›
Generate ์ง€์›๋ฏธ์ง€์›ConfigMap/Secret ์ž๋™ ์ƒ์„ฑ ์ง€์›
๋„์ž… ๋‚œ์ด๋„์ƒ๋Œ€์ ์œผ๋กœ ๋†’์Œ์ƒ๋Œ€์ ์œผ๋กœ ๋‚ฎ์Œ

๋‹จ์ˆœํ•œ k8s ์ •์ฑ… ๊ด€๋ฆฌ๋ผ๋ฉด Kyverno๊ฐ€ ์ง„์ž… ์žฅ๋ฒฝ์ด ๋‚ฎ๊ณ , ๋ฉ€ํ‹ฐ ํ”Œ๋žซํผ ์ •์ฑ… ํ†ตํ•ฉ์ด ํ•„์š”ํ•˜๋‹ค๋ฉด OPA๊ฐ€ ์ ํ•ฉํ•˜๋‹ค.


์ •๋ฆฌ

  • Admission Controller๋Š” k8s ๋ฆฌ์†Œ์Šค๊ฐ€ etcd์— ์ €์žฅ๋˜๊ธฐ ์ „ ์š”์ฒญ์„ ๊ฐ€๋กœ์ฑ„ ๊ฒ€์ฆ/๋ณ€๊ฒฝํ•œ๋‹ค
  • Webhook ๋ฐฉ์‹์œผ๋กœ ์™ธ๋ถ€ ์ •์ฑ… ์„œ๋ฒ„์™€ ์—ฐ๋™ํ•  ์ˆ˜ ์žˆ๋‹ค
  • OPA๋Š” Rego ์–ธ์–ด๋กœ ์ •์ฑ…์„ ํ‘œํ˜„ํ•˜๋Š” ๋ฒ”์šฉ ์—”์ง„์ด๋ฉฐ, Gatekeeper๋ฅผ ํ†ตํ•ด k8s์™€ ํ†ตํ•ฉ๋œ๋‹ค
  • ConstraintTemplate โ†’ Constraint ๊ตฌ์กฐ๋กœ ์ •์ฑ…์„ ์„ ์–ธ์ ์œผ๋กœ ๊ด€๋ฆฌํ•œ๋‹ค
  • Audit ๊ธฐ๋Šฅ์œผ๋กœ ๊ธฐ์กด ๋ฆฌ์†Œ์Šค์˜ ์ •์ฑ… ์œ„๋ฐ˜๋„ ๊ฐ์ง€ํ•  ์ˆ˜ ์žˆ๋‹ค
  • enforcementAction: warn/dryrun์œผ๋กœ ์ ์ง„์  ์ •์ฑ… ์ ์šฉ์ด ๊ฐ€๋Šฅํ•˜๋‹ค