15eb3e7edccdf4e9770a00a07bfbd026420f7c3b
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Charles BochetandGitHub
|
15eb3e7edc |
feat(sdk): use config file as single source of truth, remove env var fallbacks (#19409)
## Summary - **Config as source of truth**: `~/.twenty/config.json` is now the single source of truth for SDK authentication — env var fallbacks have been removed from the config resolution chain. - **Test instance support**: `twenty server start --test` spins up a dedicated Docker instance on port 2021 with its own config (`config.test.json`), so integration tests don't interfere with the dev environment. - **API key auth for marketplace**: Removed `UserAuthGuard` from `MarketplaceResolver` so API key tokens (workspace-scoped) can call `installMarketplaceApp`. - **CI for example apps**: Added monorepo CI workflows for `hello-world` and `postcard` example apps to catch regressions. - **Simplified CI**: All `ci-create-app-e2e` and example app workflows now use a shared `spawn-twenty-app-dev-test` action (Docker-based) instead of building the server from source. Consolidated auth env vars to `TWENTY_API_URL` + `TWENTY_API_KEY`. - **Template publishing fix**: `create-twenty-app` template now correctly preserves `.github/` and `.gitignore` through npm publish (stored without leading dot, renamed after copy). ## Test plan - [x] CI SDK (lint, typecheck, unit, integration, e2e) — all green - [x] CI Example App Hello World — green - [x] CI Example App Postcard — green - [x] CI Create App E2E minimal — green - [x] CI Front, CI Server, CI Shared — green |
||
|
|
4ea2e32366 |
Refactor twenty client sdk provisioning for logic function and front-component (#18544)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <[email protected]>
|
||
|
Charles BochetandGitHub
|
ef499b6d47 |
Re-enable disabled lint rules and right-size CI runners (#18461)
## Summary - Re-enable one lint rule that was temporarily disabled during the ESLint-to-Oxlint migration: - **`twenty/sort-css-properties-alphabetically`** in twenty-front — 578 violations auto-fixed across 390 files - Document why **`typescript/consistent-type-imports`** cannot be auto-fixed in twenty-server: NestJS relies on `emitDecoratorMetadata` for DI, so converting constructor parameter imports to `import type` erases them at compile time and breaks dependency injection at runtime - Right-size CI runners, reducing 8-core usage from 18 jobs to 3: | Change | Jobs | Rationale | |--------|------|-----------| | **Keep 8-core** | `ci-merge-queue/e2e-test`, `ci-front/front-sb-build`, `ci-front/front-build` | Heavy builds needing max CPU + memory (10GB NODE_OPTIONS, full Storybook webpack bundling) | | **8-core → 4-core** | `ci-server` (build, lint-typecheck, validation, test, integration-test), `ci-front/front-sb-test`, `ci-zapier/server-setup`, `ci-sdk/sdk-e2e-test` | Already sharded into 10-12 parallel instances, I/O-bound (DB/Redis), or moderate single builds | | **8-core → 2-core** | `ci-emails/emails-test` | Trivially lightweight (build + curl health check) | | **Removed** | `ci-front/front-chromatic-deployment` | Dead code — permanently disabled with `if: false` | - Fix merge queue CI issues: - **Concurrency**: Use `merge_group.base_ref` instead of unique merge group ref so new queue entries cancel previous runs - **Required status checks**: Add `merge_group` trigger to all 6 required CI workflows (front, server, shared, website, docker-compose, sdk) with `changed-files-check` auto-skipped for merge_group events — status check jobs auto-pass without re-running full CI - **Build caching**: Add Nx build cache restore/save to E2E test job with fallback to `main` branch cache for faster frontend and server builds ## Test plan - [ ] CI passes on this PR (verifies lint rule auto-fix works) - [ ] Verify 4-core runner jobs complete within their 30-minute timeouts - [ ] Verify merge queue status checks auto-pass (ci-front-status-check, ci-server-status-check, etc.) - [ ] Verify merge queue E2E concurrency cancels previous runs when a new PR enters the queue |
||
|
Charles BochetandGitHub
|
9d57bc39e5 |
Migrate from ESLint to OxLint (#18443)
## Summary Fully replaces ESLint with OxLint across the entire monorepo: - **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint configs (`.oxlintrc.json`) for every package: `twenty-front`, `twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`, `twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`, `twenty-apps/*`, `create-twenty-app` - **Migrated custom lint rules** from ESLint plugin format to OxLint JS plugin system (`@oxlint/plugins`), including `styled-components-prefixed-with-styled`, `no-hardcoded-colors`, `sort-css-properties-alphabetically`, `graphql-resolvers-should-be-guarded`, `rest-api-methods-should-be-guarded`, `max-consts-per-file`, and Jotai-related rules - **Migrated custom rule tests** from ESLint `RuleTester` + Jest to `oxlint/plugins-dev` `RuleTester` + Vitest - **Removed all ESLint dependencies** from `package.json` files and regenerated lockfiles - **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in `nx.json` and per-project `project.json` to use `oxlint` commands with proper `dependsOn` for plugin builds - **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more ESLint executor - **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with `oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and format-on-save with Prettier - **Replaced all `eslint-disable` comments** with `oxlint-disable` equivalents across the codebase - **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint - **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules` ### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`) | Rule | Package | Violations | Auto-fixable | |------|---------|-----------|-------------| | `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes | | `typescript/consistent-type-imports` | twenty-server | 3814 | Yes | | `twenty/max-consts-per-file` | twenty-server | 94 | No | ### Dropped plugins (no OxLint equivalent) `eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`, `import/order`, `prefer-arrow/prefer-arrow-functions`, `eslint-plugin-mdx`, `@next/eslint-plugin-next`, `eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial coverage for `jsx-a11y` and `unused-imports`. ### Additional fixes (pre-existing issues exposed by merge) - Fixed `EmailThreadPreview.tsx` broken import from main rename (`useOpenEmailThreadInSidePanel`) - Restored truthiness guard in `getActivityTargetObjectRecords.ts` - Fixed `AgentTurnResolver` return types to match entity (virtual `fileMediaType`/`fileUrl` are resolved via `@ResolveField()`) ## Test plan - [x] `npx nx lint twenty-front` passes - [x] `npx nx lint twenty-server` passes - [x] `npx nx lint twenty-docs` passes - [x] Custom oxlint rules validated with Vitest: `npx nx test twenty-oxlint-rules` - [x] `npx nx typecheck twenty-front` passes - [x] `npx nx typecheck twenty-server` passes - [x] CI workflows trigger correctly with `dependsOn: ["twenty-oxlint-rules:build"]` - [x] IDE linting works with `oxc.oxc-vscode` extension |
||
|
Paul RastoinandGitHub
|
0b766464e4 |
Composite action: Spawn twenty instance (#18317)
# Introduction ## Runs: Public personal repo: - [latest](https://github.com/prastoin/twenty-app/actions/runs/22568051680/job/65368592903) |
||
|
Félix MalfaitandGitHub
|
0223975bbd |
Harden GitHub Actions: fix injections, isolate privileged operations to ci-privileged repo (#18318)
## Summary - Fix expression injection vulnerabilities in composite actions (`restore-cache`, `nx-affected`) and workflow files (`claude.yml`) - Reduce overly broad permissions in `ci-utils.yaml` (Danger.js) and `ci-breaking-changes.yaml` - Restructure `preview-env-dispatch.yaml`: auto-trigger for members, opt-in for contributor PRs via `preview-app` label (safe because keepalive has no write tokens) - Isolate all write-access operations (PR comments, cross-repo posting) to a new dedicated [`twentyhq/ci-privileged`](https://github.com/twentyhq/ci-privileged) repo via `repository_dispatch`, so that workflows in twenty that execute contributor code never have write tokens - Create `post-ci-comments.yaml` (`workflow_run` bridge) to dispatch breaking changes results to ci-privileged, solving the [fork PR comment issue](https://github.com/twentyhq/twenty/pull/13713#issuecomment-3168999083) - Delete 5 unused secrets and broken `i18n-qa-report` workflow - Remove `TWENTY_DISPATCH_TOKEN` from twenty (moved to ci-privileged as `CORE_TEAM_ISSUES_COMMENT_TOKEN`) - Use `toJSON()` for all `client-payload` values to prevent JSON injection ## Security model after this PR | Workflow | Executes fork code? | Write tokens available? | |----------|---------------------|------------------------| | preview-env-keepalive | Yes | None (contents: read only) | | preview-env-dispatch | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN only | | ci-breaking-changes | Yes | None (contents: read only) | | post-ci-comments (workflow_run) | No (default branch) | CI_PRIVILEGED_DISPATCH_TOKEN only | | claude.yml | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN, CLAUDE_CODE_OAUTH_TOKEN | | ci-utils (Danger.js) | No (base branch) | GITHUB_TOKEN (scoped) | All actual write tokens (`TWENTY_PR_COMMENT_TOKEN`, `CORE_TEAM_ISSUES_COMMENT_TOKEN`) live in `twentyhq/ci-privileged` with strict CODEOWNERS review and branch protection. ## Test plan - [ ] Verify preview environment comments still appear on member PRs - [ ] Verify adding `preview-app` label triggers preview for contributor PRs - [ ] Verify breaking changes reports still post on PRs (including fork PRs) - [ ] Verify Claude cross-repo responses still post on core-team-issues - [ ] Confirm ci-privileged branch protection is enforced |
||
|
Paul RastoinandGitHub
|
e3ffdb0c2b |
[BREAKING_CHANGE_NESTED_WORKSPACE]Refactor FlatEntity typing in aim of introducing UniversalFlatEntity (#16701)
# Introduction
Added a `WorkspaceRelated` and `AllNonWorkspaceRelatedEntity` to
simplify the `FlatEntityFrom` that now do not expect a string literal to
omit and itself builds the related many to one entities foreign key
aggregators
We now have the type grain over relation to syncable or just workspace
related entities
Added a migrations that sets the fk on missing entities
## Next
In upcoming PR we will be able to introduce such below type
```ts
import { type CastRecordTypeOrmDatePropertiesToString } from 'src/engine/metadata-modules/flat-entity/types/cast-record-typeorm-date-properties-to-string.type';
import { type ExtractEntityManyToOneEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-many-to-one-entity-relation-properties.type';
import { type ExtractEntityOneToManyEntityRelationProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-one-to-many-entity-relation-properties.type';
import { type ExtractEntityRelatedEntityProperties } from 'src/engine/metadata-modules/flat-entity/types/extract-entity-related-entity-properties.type';
import { type RemoveSuffix } from 'src/engine/workspace-manager/workspace-migration-v2/workspace-migration-builder-v2/types/remove-suffix.type';
import { type SyncableEntity } from 'src/engine/workspace-manager/workspace-sync/types/syncable-entity.interface';
export type UniversalFlatEntityFrom<TEntity extends SyncableEntity> = Omit<
TEntity,
| `${ExtractEntityManyToOneEntityRelationProperties<TEntity> & string}Id`
| ExtractEntityRelatedEntityProperties<TEntity>
| 'application'
| 'workspaceId'
| 'applicationId'
| keyof CastRecordTypeOrmDatePropertiesToString<TEntity>
> &
CastRecordTypeOrmDatePropertiesToString<TEntity> & {
[P in ExtractEntityManyToOneEntityRelationProperties<TEntity> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifier`]: string;
} & {
[P in ExtractEntityOneToManyEntityRelationProperties<
TEntity,
SyncableEntity
> &
string as `${RemoveSuffix<P, 's'>}UniversalIdentifiers`]: string[];
};
```
|
||
|
Félix MalfaitandGitHub
|
e6491d6a80 |
feat(i18n): fix translation QA issues and add automation (#16756)
## Summary This PR fixes translation QA issues and adds automation to prevent future issues. ### Translation Fixes - Fixed **escaped Unicode sequences** in translations (e.g., `\u62db\u5f85` → `招待`) - Removed **corrupted control characters** from .po files (null bytes, invalid characters) - Fixed **missing/incorrect placeholders** in various languages - Deleted **35 problematic translations** via Crowdin API that had variable mismatches ### New Scripts (in `packages/twenty-utils/`) - `fix-crowdin-translations.ts` - Auto-fixes encoding issues and syncs to Crowdin - `fix-qa-issues.ts` - Fixes specific QA issues via Crowdin API - `translation-qa-report.ts` - Generates weekly QA report from Crowdin API ### New Workflow - `i18n-qa-report.yaml` - Weekly workflow that creates a PR with translation QA issues for review ### Other Changes - Moved GitHub Actions from `.github/workflows/actions/` to `.github/actions/` - Fixed `date-utils.ts` to avoid nested `t` macros in plural expressions (root cause of confusing placeholders) ### QA Status After Fixes | Category | Count | Status | |----------|-------|--------| | variables | 0 ✅ | Fixed | | tags | 1 | Minor | | empty | 0 ✅ | Fixed | | spaces | 127 | Low priority | | numbers | 246 | Locale-specific | | special_symbols | 268 | Locale-specific | |