Compare commits

...

14 Commits

Author SHA1 Message Date
Charles Bochet d6e92f14d0 Merge main into helm-improvements — resolve conflicts
Take main's simplified approach for all conflicts:
- No run-migrations init container (entrypoint.sh handles migrations)
- No additionalEnv / server.env pass-through — just extraEnv array
- No disableDbMigrations flag
- Clean indentation throughout

Made-with: Cursor
2026-03-22 21:38:54 +01:00
Don Kendall 7cbff2ac8c fix(helm): add DISABLE_DB_MIGRATIONS to reserved list, fix worker storageEnv whitespace
- Add DISABLE_DB_MIGRATIONS to server.env reserved key list so users
  cannot accidentally override it via server.env pass-through
- Fix worker template missing {{- dash on storageEnv rendering,
  consistent with server template

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 18:37:02 -04:00
Don Kendall 2b0e628f53 feat(helm): add Redis PVC template and DISABLE_DB_MIGRATIONS in server env
Cherry-pick from HuppertzL's PR #18157 (388be20):
- Add pvc-redis-internal.yaml so persistence.enabled actually works
- Set DISABLE_DB_MIGRATIONS=true in server container env (migrations
  run in the init container, not on every server restart)
- Fix PVC guard to also check persistence.enabled (bug in original)
- Add trailing newline to PVC template

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 18:22:59 -04:00
Don Kendall c04ee7a398 fix(helm): add REDIS_PASSWORD to server.env reserved key list
Prevents a user-supplied server.env.REDIS_PASSWORD from overriding the
explicitly configured Redis password from redis.external.* settings.
Kubernetes uses the last-defined value for duplicate env vars, so without
this guard the user value would win and break Redis auth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:14:23 -04:00
Don Kendall 471479922a fix(helm): remove additionalEnv from db-init container, quote PG_DATABASE_URL, cleanup
- Remove server.additionalEnv injection from ensure-database-exists init
  container (copilot/cubic): that container only runs psql commands and
  only needs PGPASSWORD + APP_PASSWORD; injecting arbitrary server env
  vars there is unnecessary and allows user-supplied keys to shadow
  secret-backed credentials (e.g. PGPASSWORD override breaking bootstrap)
- Quote "${PG_DATABASE_URL}" in has_schema psql invocation (copilot):
  prevents word-splitting if the URL contains spaces or shell-sensitive
  characters (unlikely but correct defensive practice)
- Add missing trailing newline to deployment-server.yaml (POSIX)
- Fix test description: "yarn database migrate" → "yarn database:migrate:prod"
- Fix values.yaml comment: Set to "true" → Set to true (boolean, not string)
- 64/64 helm unittest pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:14:23 -04:00
Don Kendall e675c87a29 fix(helm): address review findings — migration regression, indentation, test brittleness
P0 — migration regression (found by greptile, cubic):
  yarn database:migrate:prod was inside `if [ "$has_schema" = "f" ]`,
  meaning on every helm upgrade of an existing deployment the core schema
  already exists, the block is skipped, and pending migrations are never
  applied. Moved yarn database:migrate:prod outside the conditional;
  only tsx ./scripts/setup-db.ts remains gated on the schema check.

P2 — cosmetic/test fixes:
  - Remove duplicate tmp/ entry in .gitignore (added twice via stash conflict)
  - Fix run-migrations init container $img assignment indentation (16→8sp)
  - Fix storageEnv nindent rendering: change `              {{ nindent }}` to
    `            {{- nindent }}` to avoid orphaned 14-space line in manifest
  - Replace brittle env[0] index assertions in server_url_test.yaml and
    env_mapping_test.yaml with JSONPath name filters
    (spec.template.spec.containers[0].env[?(@.name=="X")])
  - Add missing worker-side negative Redis test: verify REDIS_PASSWORD is
    not injected into worker when using internal Redis (server had this,
    worker did not)
  - 64/64 helm unittest suites pass (11 suites)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:14:23 -04:00
Don Kendall b2b7043916 test(helm): expand test coverage for redis auth, extraEnv, and env pass-through
- Add redis_auth_test.yaml: covers REDIS_PASSWORD injection from external
  secret (secretKeyRef), plaintext password, and absence when using
  internal Redis — validates HuppertzL's redis external secret feature
- Add env_mapping_test: disableDbMigrations defaults to false in init
  container, server extraEnv with valueFrom on server pod, non-reserved
  env pass-through with realistic email SMTP example
- Update schema_permissions_test: match new migration script command
  (yarn database:migrate:prod) from improved init container
- 63/63 helm unittest suites pass (11 test suites)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:14:23 -04:00
Don Kendall 9c33f76676 feat(helm): extraEnv with valueFrom, server.env pass-through, DISABLE_DB_MIGRATIONS, and bug fixes
Cherry-picked from PR #18157 (HuppertzL/twenty improve-helm-chart):
- Redis external secret support (secretName/passwordKey)
- Improved migration init container: checks schema existence before
  running, runs tsx setup-db.ts + yarn database:migrate:prod, flushes
  cache and runs upgrade command post-migration
- additionalEnv map for server and worker

Built on top of and extending PR #18157:
- Fix volumes indentation in deployment-redis-internal.yaml (was nested
  inside containers spec instead of pod spec — would cause mount failures)
- Fix indentation issues introduced by cherry-picked commits
- Add server.env pass-through: arbitrary keys in server.env are injected
  into the server pod, filtered against reserved keys already rendered
  by the template (SERVER_URL, PG_DATABASE_URL, REDIS_URL, etc.)
- Add server.extraEnv and worker.extraEnv arrays supporting both plain
  value and valueFrom (secretKeyRef, configMapKeyRef, etc.) — supersedes
  the simpler additionalEnv map for secret references
- Add server.disableDbMigrations (bool, default false) that injects
  DISABLE_DB_MIGRATIONS into the run-migrations init container
- Add JSON schema validation for extraEnv (oneOf: value|valueFrom)
  and disableDbMigrations
- Add helm unit tests (tests/env_mapping_test.yaml) for env pass-through,
  extraEnv value, extraEnv valueFrom, and disableDbMigrations
- Update schema_permissions_test to match new migration script command
- Gitignore tmp/ directory
- 55/55 helm unittest suites pass

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:14:23 -04:00
Lukas Huppertz 5fdc8e45c3 improve helm migration script 2026-03-15 18:14:23 -04:00
Lukas Huppertz cd8729771d add redis to migration initContainer; improve migration container, using migration script from entrypoint.sh (create db if not exists) 2026-03-15 18:14:23 -04:00
Lukas Huppertz 44ad9c067a introduce additional envs for server and worker 2026-03-15 18:14:23 -04:00
Lukas Huppertz 3e356aa034 refactor: update Redis connection handling in Helm chart 2026-03-15 18:14:23 -04:00
Lukas Huppertz 0435f3c474 update values.yaml 2026-03-15 18:14:23 -04:00
Lukas Huppertz 52d0374bd3 Fix manifest syntax; introduce redis external secret 2026-03-15 18:14:22 -04:00
8 changed files with 256 additions and 9 deletions
+4 -3
View File
@@ -10,7 +10,6 @@
.nx/installation
.nx/cache
.nx/workspace-data
.nx/nxw.js
.pnp.*
.yarn/*
@@ -28,7 +27,7 @@ coverage
dist
storybook-static
*.tsbuildinfo
.oxlintcache
.eslintcache
.nyc_output
test-results/
dump.rdb
@@ -50,4 +49,6 @@ dump.rdb
mcp.json
/.junie/
TRANSLATION_QA_REPORT.md
.playwright-mcp/
# Local scratch files
tmp/
@@ -91,7 +91,7 @@ spec:
{{- end }}
{{- $storageEnv := (include "twenty.storageEnv" .) }}
{{- if $storageEnv }}
{{ $storageEnv | nindent 12 }}
{{- $storageEnv | nindent 12 }}
{{- end }}
resources:
{{- toYaml .Values.worker.resources | nindent 12 }}
@@ -0,0 +1,124 @@
suite: env mapping
templates:
- templates/deployment-server.yaml
- templates/deployment-worker.yaml
release:
name: my-twenty
namespace: default
tests:
- it: renders Google auth envs when set
template: templates/deployment-server.yaml
set:
server.env.AUTH_GOOGLE_ENABLED: true
server.env.AUTH_GOOGLE_CLIENT_ID: gid
server.env.AUTH_GOOGLE_CLIENT_SECRET: gsecret
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: AUTH_GOOGLE_ENABLED
value: "true"
- contains:
path: spec.template.spec.containers[0].env
content:
name: AUTH_GOOGLE_CLIENT_ID
value: gid
- contains:
path: spec.template.spec.containers[0].env
content:
name: AUTH_GOOGLE_CLIENT_SECRET
value: gsecret
- it: renders server extraEnv value
template: templates/deployment-server.yaml
set:
server.extraEnv:
- name: FEATURE_X_ENABLED
value: "true"
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: FEATURE_X_ENABLED
value: "true"
- it: renders worker extraEnv valueFrom
template: templates/deployment-worker.yaml
set:
worker.extraEnv:
- name: CUSTOM_SECRET
valueFrom:
secretKeyRef:
name: my-secret
key: custom
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: CUSTOM_SECRET
valueFrom:
secretKeyRef:
name: my-secret
key: custom
- it: disableDbMigrations injects env var into migration init container
template: templates/deployment-server.yaml
set:
server.disableDbMigrations: true
asserts:
- contains:
path: spec.template.spec.initContainers[?(@.name=="run-migrations")].env
content:
name: DISABLE_DB_MIGRATIONS
value: "true"
- it: disableDbMigrations defaults to false in init container
template: templates/deployment-server.yaml
asserts:
- contains:
path: spec.template.spec.initContainers[?(@.name=="run-migrations")].env
content:
name: DISABLE_DB_MIGRATIONS
value: "false"
- it: non-reserved server.env keys are injected via pass-through (email example)
template: templates/deployment-server.yaml
set:
server.env.EMAIL_DRIVER: smtp
server.env.EMAIL_SMTP_HOST: mail.example.com
server.env.EMAIL_FROM_ADDRESS: noreply@example.com
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: EMAIL_DRIVER
value: smtp
- contains:
path: spec.template.spec.containers[0].env
content:
name: EMAIL_SMTP_HOST
value: mail.example.com
- contains:
path: spec.template.spec.containers[0].env
content:
name: EMAIL_FROM_ADDRESS
value: noreply@example.com
- it: renders server extraEnv valueFrom on server container
template: templates/deployment-server.yaml
set:
server.extraEnv:
- name: SMTP_PASSWORD
valueFrom:
secretKeyRef:
name: smtp-creds
key: password
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: SMTP_PASSWORD
valueFrom:
secretKeyRef:
name: smtp-creds
key: password
@@ -0,0 +1,89 @@
suite: redis external authentication
templates:
- templates/deployment-server.yaml
- templates/deployment-worker.yaml
release:
name: my-twenty
namespace: default
tests:
- it: injects REDIS_PASSWORD from external secret into server
template: templates/deployment-server.yaml
set:
redisInternal.enabled: false
redis.external.host: redis.example.com
redis.external.secretName: redis-creds
redis.external.passwordKey: password
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-creds
key: password
- it: injects REDIS_PASSWORD from external secret into worker
template: templates/deployment-worker.yaml
set:
redisInternal.enabled: false
redis.external.host: redis.example.com
redis.external.secretName: redis-creds
redis.external.passwordKey: password
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
valueFrom:
secretKeyRef:
name: redis-creds
key: password
- it: injects plaintext REDIS_PASSWORD when password set directly in server
template: templates/deployment-server.yaml
set:
redisInternal.enabled: false
redis.external.host: redis.example.com
redis.external.password: "s3cr3t"
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
value: "s3cr3t"
- it: injects plaintext REDIS_PASSWORD when password set directly in worker
template: templates/deployment-worker.yaml
set:
redisInternal.enabled: false
redis.external.host: redis.example.com
redis.external.password: "s3cr3t"
asserts:
- contains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
value: "s3cr3t"
- it: does not inject REDIS_PASSWORD into server when using internal redis
template: templates/deployment-server.yaml
set:
redisInternal.enabled: true
asserts:
- notContains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
any: true
- it: does not inject REDIS_PASSWORD into worker when using internal redis
template: templates/deployment-worker.yaml
set:
redisInternal.enabled: true
asserts:
- notContains:
path: spec.template.spec.containers[0].env
content:
name: REDIS_PASSWORD
any: true
@@ -111,12 +111,12 @@ tests:
# 'core' schema. This ensures the _typeorm_migrations table and all application
# tables use the dedicated core schema.
- it: migrations run against core datasource
- it: migration init container runs yarn database:migrate:prod
template: templates/deployment-server.yaml
set:
db.enabled: true
asserts:
- matchRegex:
path: spec.template.spec.initContainers[?(@.name=="run-migrations")].command[2]
pattern: core\.datasource
pattern: yarn database:migrate:prod
@@ -19,7 +19,7 @@ tests:
- crm.example.com
asserts:
- equal:
path: spec.template.spec.containers[0].env[0].value
path: spec.template.spec.containers[0].env[?(@.name=="SERVER_URL")].value
value: "https://crm.example.com:443"
- it: falls back to service when ingress disabled
set:
@@ -28,7 +28,7 @@ tests:
server.env.SERVER_URL: ""
asserts:
- matchRegex:
path: spec.template.spec.containers[0].env[0].value
path: spec.template.spec.containers[0].env[?(@.name=="SERVER_URL")].value
pattern: ^http://my-twenty-twenty-server\.default\.svc\.cluster\.local:3000$
---
suite: ingress configuration
@@ -42,6 +42,7 @@
},
"podSecurityContext": { "type": "object" },
"containerSecurityContext": { "type": "object" },
"disableDbMigrations": { "type": "boolean", "description": "Set to true to skip database migrations on startup." },
"env": {
"type": "object",
"properties": {
@@ -66,6 +67,22 @@
"PASSWORD_RESET_TOKEN_EXPIRES_IN": { "type": "string" }
}
},
"extraEnv": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"value": { "type": "string" },
"valueFrom": { "type": "object" }
},
"required": ["name"],
"oneOf": [
{ "required": ["value"] },
{ "required": ["valueFrom"] }
]
}
},
"service": {
"type": "object",
"properties": {
@@ -141,6 +158,22 @@
"STORAGE_TYPE": { "type": "string" },
"DISABLE_DB_MIGRATIONS": { "type": "string" }
}
},
"extraEnv": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": { "type": "string" },
"value": { "type": "string" },
"valueFrom": { "type": "object" }
},
"required": ["name"],
"oneOf": [
{ "required": ["value"] },
{ "required": ["valueFrom"] }
]
}
}
}
},
@@ -124,7 +124,7 @@ db:
appPassword: "" # random if empty
image:
repository: twentycrm/twenty-postgres-spilo
tag: 3.3-p2
tag: v0.43.5
resources:
requests:
cpu: 250m