Skip to content

Feature/115/multi tenancy - #154

Open
jack1902 wants to merge 4 commits into
confidential-containers:mainfrom
jack1902:feature/115/multi-tenancy
Open

Feature/115/multi tenancy#154
jack1902 wants to merge 4 commits into
confidential-containers:mainfrom
jack1902:feature/115/multi-tenancy

Conversation

@jack1902

@jack1902 jack1902 commented May 29, 2026

Copy link
Copy Markdown

This empowers the operator to manage trustee' per namespace. If a TrusteeConfig is created inside a namespace tenant-a then the respective resources are deployed into tenant-a.

I have successfully used this to create a trustee in tenant-a and tenant-b, i deployed the built operator into a local kind cluster as make run didn't work (paths in code assume location of /templates which resulted in things being very unhappy)

Dev Loop

Script used to iteratively build / deploy the operator into a local throwaway cluster (kind)

#!/usr/bin/env bash
set -eou pipefail

export IMG=quay.io/jack1902/trustee-operator:test
make docker-build IMG=$IMG
./tests/scripts/kind-with-registry.sh || true
kind load docker-image "${IMG}"
make install
make deploy IMG=$IMG
kubectl logs -n trustee-operator-system -l control-plane=controller-manager -f
image

Only bit i do observe within the logs is that lines such as 2026-05-29T13:36:01Z INFO kbsconfig-controller Created a new deployment {"kbsconfig": "", "Deployment.Namespace": "tenant-a", "Deployment.Name": "trustee-deployment"} don't appear to actually hold the name of the kbsconfig

closes: #115

When reconciling, we set the namespace on r, similar to how other parts of the codebase work. Additionally ammend the Dockerfile to remove a warning and enable the build to work on a hardened linux build machine

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
@jack1902
jack1902 force-pushed the feature/115/multi-tenancy branch 2 times, most recently from 18d254c to cb4f152 Compare May 29, 2026 15:33
@lmilleri

Copy link
Copy Markdown
Member

@jack1902 thanks, running copilot review as well

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds multi-tenancy support so KbsConfig reconciliation (and its dependent ConfigMap/Secret watches) can operate per-namespace rather than being restricted to the operator’s namespace, enabling tenants to deploy trustee instances in their own namespaces.

Changes:

  • Update KbsConfigReconciler to use req.Namespace and watch referenced ConfigMaps/Secrets across all namespaces.
  • Adjust the container image to copy templates with non-root ownership.
  • Update default KBS TOML templates and add tenant-facing RBAC example manifests + README.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
internal/controller/kbsconfig_controller.go Switch reconciliation namespace to request namespace; remove namespace-scoped watch predicates; add events RBAC marker.
Dockerfile Copy templates into runtime image with correct non-root ownership.
config/templates/kbs-config-restricted.toml Update admin auth configuration template schema.
config/templates/kbs-config-permissive.toml Update admin auth configuration template schema.
config/rbac/tenant-rolebinding.yaml Add example RoleBinding for tenant access.
config/rbac/tenant-role.yaml Add example ClusterRole for tenant users to manage TrusteeConfigs and view generated resources.
config/rbac/role.yaml Grant manager-role permissions to create/patch Events.
config/rbac/README.md Document how to apply the tenant RBAC examples.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/controller/kbsconfig_controller.go Outdated
Comment thread internal/controller/kbsconfig_controller.go Outdated
Comment thread config/rbac/README.md Outdated
Comment on lines 8 to +14
[admin]
type = "DenyAll"
insecure_api = false
auth_public_key = "/etc/auth-secret/publicKey"

[[admin.personas]]
id = "admin"
public_key_path = "/etc/auth-secret/publicKey"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left this in play @lmilleri since if happy with this as a change i can look through the rest of the codebase to "align"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jack1902 the auth_public_key has been already removed from the both restrictive/permissive template. The [[admin.personas]] section should have no effect since the authorization_mode is DenyAll. This change can be useful for the future when integrating the trustee admin api.

Comment thread config/templates/kbs-config-permissive.toml
Comment thread internal/controller/kbsconfig_controller.go
jack1902 added 3 commits May 29, 2026 17:03
Addresses 'cannot patch resource' in logs

Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
Signed-off-by: jack1902 <39212456+jack1902@users.noreply.github.com>
@jack1902
jack1902 force-pushed the feature/115/multi-tenancy branch from cb4f152 to 813d9ed Compare May 29, 2026 16:03
@jack1902
jack1902 marked this pull request as ready for review July 17, 2026 11:00
@jack1902
jack1902 requested a review from a team as a code owner July 17, 2026 11:00

@lmilleri lmilleri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jack1902 Please rebase and fix lint issues.

Comment on lines 8 to +14
[admin]
type = "DenyAll"
insecure_api = false
auth_public_key = "/etc/auth-secret/publicKey"

[[admin.personas]]
id = "admin"
public_key_path = "/etc/auth-secret/publicKey"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jack1902 the auth_public_key has been already removed from the both restrictive/permissive template. The [[admin.personas]] section should have no effect since the authorization_mode is DenyAll. This change can be useful for the future when integrating the trustee admin api.

@jack1902

Copy link
Copy Markdown
Author

@lmilleri will do (I'm off for 2 weeks so will do it once back). Just checking that once those changes are complete that the changes are ok?

@lmilleri

Copy link
Copy Markdown
Member

@jack1902 My only concern is that now the operator watches for all configmaps/secrets in all namespaces (cluster-wide). Any idea on how to improve this, e.g. leaving the predicate with different implementation?

@jack1902

Copy link
Copy Markdown
Author

@jack1902 My only concern is that now the operator watches for all configmaps/secrets in all namespaces (cluster-wide). Any idea on how to improve this, e.g. leaving the predicate with different implementation?

I guess the options might be to have this :

Configmaps/secrets in existing namespace (non breaking)

Configmaps/secrets with a label / name prefix across the remainder of the cluster

Effectively avoiding a breaking change, whilst ensuring a defined pattern outside the existing namespaces

@lmilleri

Copy link
Copy Markdown
Member

Yes, maybe an hybrid approach by watching all the resources in the operator namespace and configmaps/secrets having a custom label (all namespaces)? This label can be set by the TrusteeConfigController in most cases. WDYT?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for multi-tenancy

3 participants