Compare commits

..
Author SHA1 Message Date
b31da5d315 fix(server): gate viewFilter.relationTargetFieldMetadataId behind its 2.6 upgrade command (#21267)
## Problem

Self-hosted upgrades crossing 2.6 (e.g. `2.4 → 2.6/2.9`) can abort with:

```
column ViewFilterEntity.relationTargetFieldMetadataId does not exist
  at WorkspaceFlatViewFilterMapCacheService.computeForCache
  at WorkspaceCacheService.recomputeDataFromProvider
[UpgradeSequenceRunnerService] Workspace steps ended with 1 failure(s). Aborting
```

This is **Failure #1** from #20841 — the counterpart to the
role-permission cache crash fixed in #21257 (Failure #2). Same shape: a
workspace **cache recompute runs mid-upgrade and reads schema that the
target version's migration hasn't applied yet**.

## Root cause

`ViewFilterEntity.relationTargetFieldMetadataId` is added to
`core.viewFilter` only at the **2.6.0** cursor
(`AddRelationTargetFieldMetadataIdToViewFilterFastInstanceCommand`, ts
`1798000005000`). But the workspace cache recompute SELECTs every column
of the entity, and it runs during *earlier* (2.5) workspace steps.
Unlike `RolePermissionFlagEntity.permissionFlag`, this column has **no
`@WasIntroducedInUpgrade` gate**, so the proxy can't hide it — and the
SELECT fails when the column isn't there yet.

There are three `IF NOT EXISTS` backport commands (2.3/2.4/2.5) meant to
add the column sooner, but they use **low timestamps** that sort to the
front of their version bundles. An instance whose cursor has already
advanced past those positions (e.g. it reached 2.4, or a prior failed
attempt advanced it through 2.5 instance commands) treats them as
already-applied and **skips them** — so the column is never created, yet
the entity keeps selecting it.

## Fix

Gate the column with `@WasIntroducedInUpgrade` pointing at the **2.6.0**
command that adds it:

```ts
@WasIntroducedInUpgrade({
  upgradeCommandName:
    '2.6.0_AddRelationTargetFieldMetadataIdToViewFilterFastInstanceCommand_1798000005000',
})
@Column({ nullable: true, type: 'uuid', default: null })
relationTargetFieldMetadataId: string | null;
```

`UpgradeAwareRepositoryProxy` then hides the column from reads while the
cursor is < 2.6, so the cache recompute simply omits it — no crash — and
it becomes visible once the 2.6.0 command has run (where it's guaranteed
to exist). Gating to **2.6.0** specifically (not the earlier backports)
is what fixes the cursor-skip case: 2.6.0 is the first point where the
column is reliably present regardless of whether the backports ran.

Validator-safe: the referenced command resolves to a real step
(`computeCommandName` = `${version}_${className}_${timestamp}`), so
`validate-upgrade-aware-entity-decorators` accepts it. The existing
backport commands are left untouched (committed instance commands).

## Recovery for already-stuck instances

This prevents *new* failures. An instance already aborted mid-upgrade
needs the column added manually before retrying:

```sql
ALTER TABLE core."viewFilter" ADD COLUMN IF NOT EXISTS "relationTargetFieldMetadataId" uuid;
DELETE FROM core."upgradeMigration" WHERE status='failed';
```
then re-run the upgrade on a build that includes this fix.

Refs #20841

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 18:56:55 +02:00
deb01ec823 fix(server): guard role-permission cache against stripped permissionFlag relation during upgrade (#21257)
## Problem

Self-hosted upgrades that jump versions (e.g. `2.4 → 2.7/2.9`) abort
with:

```
TypeError: Cannot read properties of undefined (reading 'universalIdentifier')
  at WorkspaceRolesPermissionsCacheService.hasSettingsGatedObjectPermissions
  at WorkspaceRolesPermissionsCacheService.computeForCache
  at WorkspaceCacheService.recomputeDataFromProvider
```

Reported in #20841 (Failure #2). The sequence aborts mid-upgrade and
leaves the DB in a half-migrated state.

## Root cause

The per-workspace **cache recompute runs at a `2.5.0` workspace step —
before the `2.6` schema migrations apply**. At that cursor:

- `RolePermissionFlagEntity.permissionFlag` is
`@WasIntroducedInUpgrade('2.6.0_LinkRolePermissionFlagToPermissionFlag…')`,
so `UpgradeAwareRepositoryProxy` **strips the relation**
(`[upgrade-proxy] strip relation
RolePermissionFlagEntity.permissionFlag` in the logs) → `permissionFlag`
is `undefined`.
- `hasSettingsGatedObjectPermissions()` then does an **unguarded**
`rolePermissionFlag.permissionFlag.universalIdentifier` → throws.

The crash only manifests when a workspace has **≥1 `rolePermissionFlag`
row** (custom roles with gated settings perms / SDK `defineRole`). A
vanilla seed has an empty table, so `.find()` over `[]` never
dereferences anything — which is why it didn't reproduce on a clean
instance.

A null-safe fallback to the legacy `flag` column used to exist here; it
was dropped in #20730.

## Fix

Resolve the flag's universal identifier through a small helper that
falls back to the legacy `flag` column (only removed in `2.7.0`) when
the relation is unavailable:

```ts
private getRolePermissionFlagUniversalIdentifier(
  rolePermissionFlag: RolePermissionFlagEntity,
): string {
  // The `permissionFlag` relation is stripped during upgrades until the 2.6.0
  // cursor (@WasIntroducedInUpgrade), so fall back to the legacy `flag` column.
  return (
    rolePermissionFlag.permissionFlag?.universalIdentifier ??
    SystemPermissionFlag[rolePermissionFlag.flag]
  );
}
```

`SystemPermissionFlag[flag]` yields the same UUID the relation would, so
the comparison stays in a single space and the computed permission is
exact (not an over-grant). Correct at every transitional cursor:
pre-`2.6` (relation stripped → use `flag`), `2.6` (both present →
relation wins), post-`2.7` (`flag` removed → relation wins).

## Reproduction & validation

Locally jumped a real `2.4.0` DB → `v2.9.0` build via `yarn command:prod
upgrade`:

| Scenario | Result |
| --- | --- |
| Empty `permissionFlag` (vanilla seed) | passes (no crash) |
| **+1 flag row**, current code | `TypeError … universalIdentifier` →
**3 succeeded, 1 failed** |
| Same fixture, **this fix** | **16 succeeded, 0 failed**, DB fully
migrated to 2.9.0 |

`nx typecheck twenty-server` clean; existing cache-service unit tests
pass; app boots on the upgraded DB.

## Scope / follow-up

This fixes **Failure #2**. **Failure #1** in the same issue
(`viewFilter.relationTargetFieldMetadataId` selected before its column
exists) is a separate instance of the same theme — cache recompute
reading "future" schema before migrations run — and is worth a
follow-up. A more durable systemic fix would defer the workspace cache
recompute until after all schema-adding migrations; this PR is the
low-risk, backport-friendly fix for the immediate breakage.

> Note: an earlier bot branch
(`sonarly-39738-fixupgrade-guard-role-permission-flag-relation`)
proposed the same fallback inline. This PR supersedes it with a named
helper + a focused comment.

Fixes #20841

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 17:04:43 +02:00
Charles BochetandGitHub b53f1832d8 feat(ci): simplify visual regression dispatch to twenty-ui only (#21215)
## Summary

- Only trigger visual regression on `CI UI` workflow (drop `CI Front`)
- Remove tarball re-packaging step — `ci-privileged` now downloads the
artifact directly via GitHub API
- Remove `mode`/`project` parameters from the dispatch payload
(hardcoded to twenty-ui in ci-privileged)
- Pass `run_id` of the triggering CI UI workflow so ci-privileged can
fetch the correct artifact

## Context

Part of the fast visual regression CI initiative. The `ci-privileged`
workflow has been simplified to only handle `twenty-ui` screenshots
uploaded directly to Argos.

## Test plan

- [x] Full E2E verified on production: screenshots → Argos build → diff
results → PR comment
2026-06-04 13:49:44 +02:00
8 changed files with 52 additions and 99 deletions
@@ -1,12 +1,12 @@
name: Visual Regression Dispatch
# Uses workflow_run to dispatch visual regression to ci-privileged.
# This runs in the context of the base repo (not the fork), so it has
# access to secrets — making it work for external contributor PRs.
# Dispatches visual regression processing to ci-privileged after CI completes.
# Runs in the context of the base repo (not the fork) so it has access to secrets,
# making it work for external contributor PRs.
on:
workflow_run:
workflows: ['CI Front', 'CI UI']
workflows: ['CI UI']
types: [completed]
permissions:
@@ -21,29 +21,28 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Determine project and artifact name
- name: Determine project
id: project
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
with:
script: |
const workflowName = context.payload.workflow_run.name;
if (workflowName === 'CI Front') {
core.setOutput('project', 'twenty-front');
core.setOutput('artifact_name', 'storybook-static');
core.setOutput('tarball_name', 'storybook-twenty-front-tarball');
core.setOutput('tarball_file', 'storybook-twenty-front.tar.gz');
core.setOutput('mode', 'storybook-tarball');
} else if (workflowName === 'CI UI') {
core.setOutput('project', 'twenty-ui');
core.setOutput('artifact_name', 'argos-screenshots-twenty-ui');
core.setOutput('tarball_name', 'argos-screenshots-twenty-ui-tarball');
core.setOutput('tarball_file', 'argos-screenshots-twenty-ui.tar.gz');
core.setOutput('mode', 'argos-screenshots');
} else {
const projects = {
'CI UI': { project: 'twenty-ui', artifact: 'argos-screenshots-twenty-ui' },
// Add more projects here when ready:
// 'CI Front': { project: 'twenty-front', artifact: 'argos-screenshots-twenty-front' },
};
const config = projects[workflowName];
if (!config) {
core.setFailed(`Unexpected workflow: ${workflowName}`);
return;
}
- name: Check if artifact exists
core.setOutput('project', config.project);
core.setOutput('artifact_name', config.artifact);
- name: Check if screenshots artifact exists
id: check-artifact
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
with:
@@ -61,7 +60,7 @@ jobs:
core.setOutput('exists', found ? 'true' : 'false');
if (!found) {
core.info(`Artifact "${artifactName}" not found in run ${runId} — build was likely skipped`);
core.info(`Artifact "${artifactName}" not found in run ${runId} — skipping`);
}
- name: Get PR number
@@ -73,8 +72,6 @@ jobs:
const headBranch = context.payload.workflow_run.head_branch;
const headRepo = context.payload.workflow_run.head_repository;
// workflow_run.pull_requests is empty for fork PRs,
// so fall back to searching by head label (owner:branch)
let pullRequests = context.payload.workflow_run.pull_requests;
let prNumber;
@@ -82,7 +79,7 @@ jobs:
prNumber = pullRequests[0].number;
} else {
const headLabel = `${headRepo.owner.login}:${headBranch}`;
core.info(`pull_requests is empty (likely a fork PR), searching by head label: ${headLabel}`);
core.info(`Searching for PR by head label: ${headLabel}`);
const { data: prs } = await github.rest.pulls.list({
owner: context.repo.owner,
@@ -98,7 +95,7 @@ jobs:
}
if (!prNumber) {
core.info('No pull request found for this workflow run — skipping');
core.info('No pull request found — skipping');
core.setOutput('has_pr', 'false');
return;
}
@@ -107,45 +104,23 @@ jobs:
core.setOutput('has_pr', 'true');
core.info(`PR #${prNumber}`);
- name: Download artifact from triggering run
if: steps.check-artifact.outputs.exists == 'true' && steps.pr-info.outputs.has_pr == 'true'
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: ${{ steps.project.outputs.artifact_name }}
path: artifact-content
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}
- name: Package artifact as tarball
if: steps.check-artifact.outputs.exists == 'true' && steps.pr-info.outputs.has_pr == 'true'
run: tar -czf /tmp/${{ steps.project.outputs.tarball_file }} -C artifact-content .
- name: Upload tarball
if: steps.check-artifact.outputs.exists == 'true' && steps.pr-info.outputs.has_pr == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ steps.project.outputs.tarball_name }}
path: /tmp/${{ steps.project.outputs.tarball_file }}
retention-days: 1
- name: Dispatch to ci-privileged
if: steps.check-artifact.outputs.exists == 'true' && steps.pr-info.outputs.has_pr == 'true'
env:
GH_TOKEN: ${{ secrets.CI_PRIVILEGED_DISPATCH_TOKEN }}
PR_NUMBER: ${{ steps.pr-info.outputs.pr_number }}
RUN_ID: ${{ github.run_id }}
WORKFLOW_RUN_ID: ${{ github.event.workflow_run.id }}
REPOSITORY: ${{ github.repository }}
PROJECT: ${{ steps.project.outputs.project }}
BRANCH: ${{ github.event.workflow_run.head_branch }}
COMMIT: ${{ github.event.workflow_run.head_sha }}
MODE: ${{ steps.project.outputs.mode }}
run: |
gh api repos/twentyhq/ci-privileged/dispatches \
--method POST \
-f event_type=visual-regression \
-f "client_payload[pr_number]=$PR_NUMBER" \
-f "client_payload[run_id]=$RUN_ID" \
-f "client_payload[run_id]=$WORKFLOW_RUN_ID" \
-f "client_payload[repo]=$REPOSITORY" \
-f "client_payload[project]=$PROJECT" \
-f "client_payload[branch]=$BRANCH" \
-f "client_payload[commit]=$COMMIT" \
-f "client_payload[mode]=$MODE"
-f "client_payload[commit]=$COMMIT"
@@ -279,11 +279,22 @@ export class WorkspaceRolesPermissionsCacheService extends WorkspaceCacheProvide
const hasPermissionFromSettingPermissions = isDefined(
rolePermissionFlags.find(
(rolePermissionFlag) =>
rolePermissionFlag.permissionFlag.universalIdentifier ===
this.getRolePermissionFlagUniversalIdentifier(rolePermissionFlag) ===
permissionFlagUniversalIdentifier,
),
);
return hasPermissionFromRole || hasPermissionFromSettingPermissions;
}
private getRolePermissionFlagUniversalIdentifier(
rolePermissionFlag: RolePermissionFlagEntity,
): string {
// The `permissionFlag` relation is stripped during upgrades until the 2.6.0
// cursor (@WasIntroducedInUpgrade), so fall back to the legacy `flag` column.
return (
rolePermissionFlag.permissionFlag?.universalIdentifier ??
SystemPermissionFlag[rolePermissionFlag.flag]
);
}
}
@@ -12,6 +12,7 @@ import {
UpdateDateColumn,
} from 'typeorm';
import { WasIntroducedInUpgrade } from 'src/engine/core-modules/upgrade/decorators/was-introduced-in-upgrade.decorator';
import { type JsonbProperty } from 'src/engine/workspace-manager/workspace-migration/universal-flat-entity/types/jsonb-property.type';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { ViewFilterGroupEntity } from 'src/engine/metadata-modules/view-filter-group/entities/view-filter-group.entity';
@@ -64,6 +65,10 @@ export class ViewFilterEntity
@Column({ nullable: true, type: 'text', default: null })
subFieldName: string | null;
@WasIntroducedInUpgrade({
upgradeCommandName:
'2.6.0_AddRelationTargetFieldMetadataIdToViewFilterFastInstanceCommand_1798000005000',
})
@Column({ nullable: true, type: 'uuid', default: null })
relationTargetFieldMetadataId: string | null;
@@ -1,5 +1,12 @@
import { serializeJsonLd } from './serialize-json-ld';
import type { JsonLdValue } from './types/json-ld-value';
type JsonLdPrimitive = boolean | number | string | null;
export type JsonLdValue =
| JsonLdPrimitive
| JsonLdValue[]
| { [key: string]: JsonLdValue | undefined };
const serializeJsonLd = (data: JsonLdValue): string =>
JSON.stringify(data).replace(/</g, '\\u003c');
export function JsonLd({ data }: { data: JsonLdValue }) {
return (
@@ -1,29 +0,0 @@
import { serializeJsonLd } from '@/lib/seo/serialize-json-ld';
describe('serializeJsonLd', () => {
it('escapes <, > and / to their \\uXXXX form', () => {
expect(serializeJsonLd({ value: '<>/' })).toBe(
'{"value":"\\u003c\\u003e\\u002f"}',
);
});
it('neutralizes a </script> breakout attempt', () => {
const result = serializeJsonLd({
name: '</script><script>alert(1)</script>',
});
expect(result).not.toContain('<');
expect(result).not.toContain('>');
expect(result).not.toContain('</script');
});
it('round-trips: the escaped payload parses back to the original data', () => {
const data = {
'@context': 'https://schema.org',
name: 'A & B </script>',
nested: { items: ['<x>', '/path'] },
};
expect(JSON.parse(serializeJsonLd(data))).toEqual(data);
});
});
@@ -5,7 +5,7 @@ import type { Article } from '@/lib/articles';
import { localeToUrlSegment } from '@/lib/i18n';
import type { LocalReleaseNote } from '@/lib/releases/types';
import type { JsonLdValue } from './types/json-ld-value';
import type { JsonLdValue } from './JsonLd';
import { getSiteUrl } from './site-url';
type FaqEntryLike = {
@@ -1,10 +0,0 @@
import type { JsonLdValue } from './types/json-ld-value';
// Escape the script-significant characters to their \uXXXX form so the payload
// can never break out of the <script> element (e.g. via </script>). Mirrors the
// set used by serialize-javascript; they decode back when parsed as JSON.
export const serializeJsonLd = (data: JsonLdValue): string =>
JSON.stringify(data)
.replace(/</g, '\\u003c')
.replace(/>/g, '\\u003e')
.replace(/\//g, '\\u002f');
@@ -1,6 +0,0 @@
type JsonLdPrimitive = boolean | number | string | null;
export type JsonLdValue =
| JsonLdPrimitive
| JsonLdValue[]
| { [key: string]: JsonLdValue | undefined };