4ea2e32366
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <[email protected]>
240 lines
8.0 KiB
JSON
240 lines
8.0 KiB
JSON
{
|
|
"name": "twenty-server",
|
|
"description": "",
|
|
"author": "",
|
|
"private": true,
|
|
"license": "AGPL-3.0",
|
|
"scripts": {
|
|
"nx": "NX_DEFAULT_PROJECT=twenty-server node ../../node_modules/nx/bin/nx.js",
|
|
"start:prod": "node dist/main",
|
|
"command:prod": "node dist/command/command",
|
|
"worker:prod": "node dist/queue-worker/queue-worker",
|
|
"database:init:prod": "npx ts-node ./scripts/setup-db.ts && yarn database:migrate:prod",
|
|
"database:migrate:prod": "npx -y typeorm migration:run -d dist/database/typeorm/core/core.datasource",
|
|
"clickhouse:migrate:prod": "node dist/database/clickHouse/migrations/run-migrations.js",
|
|
"typeorm": "../../node_modules/typeorm/.bin/typeorm"
|
|
},
|
|
"dependencies": {
|
|
"@ai-sdk/amazon-bedrock": "^3.0.83",
|
|
"@ai-sdk/anthropic": "^3.0.46",
|
|
"@ai-sdk/google": "^3.0.30",
|
|
"@ai-sdk/mistral": "^3.0.20",
|
|
"@ai-sdk/openai": "^3.0.30",
|
|
"@ai-sdk/provider-utils": "^4.0.15",
|
|
"@ai-sdk/xai": "^3.0.57",
|
|
"@aws-sdk/client-lambda": "3.1001.0",
|
|
"@aws-sdk/client-s3": "3.1001.0",
|
|
"@aws-sdk/client-sesv2": "3.1001.0",
|
|
"@aws-sdk/client-sts": "3.1001.0",
|
|
"@aws-sdk/credential-providers": "3.1001.0",
|
|
"@aws-sdk/s3-request-presigner": "3.1001.0",
|
|
"@azure/msal-node": "^3.8.4",
|
|
"@babel/preset-env": "7.26.9",
|
|
"@blocknote/server-util": "^0.47.1",
|
|
"@clickhouse/client": "^1.18.1",
|
|
"@dagrejs/dagre": "^1.1.2",
|
|
"@e2b/code-interpreter": "^1.0.4",
|
|
"@envelop/core": "4.0.3",
|
|
"@envelop/on-resolve": "4.1.0",
|
|
"@esbuild-plugins/node-modules-polyfill": "^0.2.2",
|
|
"@faker-js/faker": "9.8.0",
|
|
"@file-type/pdf": "^0.2.0",
|
|
"@graphql-tools/schema": "10.0.4",
|
|
"@graphql-tools/utils": "9.2.1",
|
|
"@graphql-yoga/nestjs": "patch:@graphql-yoga/[email protected]#./patches/@graphql-yoga+nestjs+2.1.0.patch",
|
|
"@jrmdayn/googleapis-batcher": "^0.10.1",
|
|
"@lingui/conf": "5.1.2",
|
|
"@lingui/core": "^5.1.2",
|
|
"@lingui/format-po": "5.1.2",
|
|
"@lingui/react": "5.1.2",
|
|
"@lingui/vite-plugin": "5.1.2",
|
|
"@microsoft/microsoft-graph-client": "3.0.7",
|
|
"@microsoft/microsoft-graph-types": "^2.40.0",
|
|
"@nestjs/axios": "3.1.2",
|
|
"@nestjs/cache-manager": "^2.3.0",
|
|
"@nestjs/common": "11.1.16",
|
|
"@nestjs/config": "3.3.0",
|
|
"@nestjs/core": "11.1.16",
|
|
"@nestjs/event-emitter": "2.1.0",
|
|
"@nestjs/graphql": "patch:@nestjs/[email protected]#./patches/@nestjs+graphql+12.1.1.patch",
|
|
"@nestjs/jwt": "11.0.1",
|
|
"@nestjs/passport": "11.0.5",
|
|
"@nestjs/platform-express": "11.1.16",
|
|
"@nestjs/schedule": "^6.0.1",
|
|
"@nestjs/serve-static": "5.0.4",
|
|
"@nestjs/terminus": "11.0.0",
|
|
"@nestjs/typeorm": "11.0.0",
|
|
"@node-saml/node-saml": "5.1.0",
|
|
"@node-saml/passport-saml": "^5.1.0",
|
|
"@opentelemetry/api": "^1.9.0",
|
|
"@opentelemetry/auto-instrumentations-node": "^0.60.0",
|
|
"@opentelemetry/exporter-metrics-otlp-http": "^0.200.0",
|
|
"@opentelemetry/exporter-prometheus": "^0.211.0",
|
|
"@opentelemetry/sdk-metrics": "^2.0.0",
|
|
"@opentelemetry/sdk-node": "^0.202.0",
|
|
"@ptc-org/nestjs-query-core": "4.4.0",
|
|
"@ptc-org/nestjs-query-graphql": "patch:@ptc-org/[email protected]#./patches/@ptc-org+nestjs-query-graphql+4.2.0.patch",
|
|
"@ptc-org/nestjs-query-typeorm": "4.2.1-alpha.2",
|
|
"@react-email/render": "^1.2.3",
|
|
"@sentry/nestjs": "^10.27.0",
|
|
"@sentry/node": "^10.27.0",
|
|
"@sentry/profiling-node": "^10.27.0",
|
|
"@sniptt/guards": "0.2.0",
|
|
"addressparser": "1.0.1",
|
|
"ai": "6.0.97",
|
|
"apollo-server-core": "3.13.0",
|
|
"archiver": "7.0.1",
|
|
"axios": "^1.13.5",
|
|
"axios-retry": "^4.5.0",
|
|
"babel-plugin-module-resolver": "5.0.2",
|
|
"bcrypt": "5.1.1",
|
|
"bullmq": "5.40.0",
|
|
"bytes": "3.1.2",
|
|
"cache-manager": "^5.4.0",
|
|
"cache-manager-redis-yet": "^4.1.2",
|
|
"chalk": "4.1.2",
|
|
"class-transformer": "0.5.1",
|
|
"class-validator": "0.14.0",
|
|
"class-validator-jsonschema": "^5.0.2",
|
|
"cloudflare": "^4.5.0",
|
|
"connect-redis": "^7.1.1",
|
|
"cron-parser": "5.1.1",
|
|
"dataloader": "2.2.2",
|
|
"date-fns": "2.30.0",
|
|
"deep-equal": "2.2.3",
|
|
"dompurify": "3.3.3",
|
|
"dotenv": "16.4.5",
|
|
"express": "4.22.1",
|
|
"express-session": "^1.18.2",
|
|
"file-type": "^21.3.1",
|
|
"fuse.js": "^7.1.0",
|
|
"gaxios": "5.1.3",
|
|
"google-auth-library": "8.9.0",
|
|
"googleapis": "105.0.0",
|
|
"graphql": "16.8.1",
|
|
"graphql-fields": "2.0.3",
|
|
"graphql-middleware": "^6.1.35",
|
|
"graphql-redis-subscriptions": "2.7.0",
|
|
"graphql-scalars": "1.23.0",
|
|
"graphql-subscriptions": "2.0.0",
|
|
"graphql-tag": "2.12.6",
|
|
"graphql-type-json": "0.3.2",
|
|
"graphql-upload": "16.0.2",
|
|
"graphql-yoga": "4.0.5",
|
|
"html-to-text": "^9.0.5",
|
|
"imapflow": "1.2.1",
|
|
"ioredis": "5.6.0",
|
|
"jsdom": "^26.1.0",
|
|
"json-schema": "0.4.0",
|
|
"jsonwebtoken": "9.0.2",
|
|
"libphonenumber-js": "1.11.5",
|
|
"lodash.camelcase": "4.3.0",
|
|
"lodash.chunk": "4.2.0",
|
|
"lodash.compact": "3.0.1",
|
|
"lodash.differencewith": "^4.5.0",
|
|
"lodash.groupby": "4.6.0",
|
|
"lodash.isempty": "4.4.0",
|
|
"lodash.isequal": "4.5.0",
|
|
"lodash.isobject": "3.0.2",
|
|
"lodash.kebabcase": "4.1.1",
|
|
"lodash.merge": "^4.6.2",
|
|
"lodash.omit": "4.5.0",
|
|
"lodash.omitby": "^4.6.0",
|
|
"lodash.snakecase": "4.1.1",
|
|
"lodash.uniq": "^4.5.0",
|
|
"lodash.uniqby": "^4.7.0",
|
|
"lodash.upperfirst": "4.3.1",
|
|
"mailparser": "3.9.3",
|
|
"microdiff": "1.4.0",
|
|
"mrmime": "^2.0.1",
|
|
"ms": "2.1.3",
|
|
"nest-commander": "^3.19.1",
|
|
"node-ical": "^0.20.1",
|
|
"nodemailer": "^7.0.11",
|
|
"openapi-types": "12.1.3",
|
|
"openid-client": "^5.7.0",
|
|
"otplib": "^12.0.1",
|
|
"passport": "^0.7.0",
|
|
"passport-google-oauth20": "2.0.0",
|
|
"passport-jwt": "4.0.1",
|
|
"passport-microsoft": "2.1.0",
|
|
"path-to-regexp": "^8.2.0",
|
|
"pg": "8.12.0",
|
|
"planer": "1.2.0",
|
|
"pluralize": "8.0.0",
|
|
"postal-mime": "^2.6.1",
|
|
"psl": "^1.9.0",
|
|
"react": "18.3.1",
|
|
"react-dom": "18.3.1",
|
|
"redis": "^4.7.0",
|
|
"reflect-metadata": "0.2.2",
|
|
"rxjs": "7.8.1",
|
|
"semver": "7.6.3",
|
|
"sharp": "0.32.6",
|
|
"stripe": "19.3.1",
|
|
"tar": "^7.5.9",
|
|
"temporal-polyfill": "^0.3.0",
|
|
"transliteration": "2.3.5",
|
|
"tsconfig-paths": "^4.2.0",
|
|
"tsdav": "^2.1.5",
|
|
"tslib": "2.8.1",
|
|
"type-fest": "4.10.1",
|
|
"typeorm": "patch:[email protected]#./patches/typeorm+0.3.20.patch",
|
|
"unzipper": "^0.12.3",
|
|
"uuid": "9.0.1",
|
|
"vite-tsconfig-paths": "4.3.2",
|
|
"zod": "^4.1.11"
|
|
},
|
|
"devDependencies": {
|
|
"@faker-js/faker": "^9.8.0",
|
|
"@lingui/cli": "^5.1.2",
|
|
"@nestjs/cli": "^11.0.16",
|
|
"@nestjs/devtools-integration": "^0.2.1",
|
|
"@nestjs/schematics": "^11.0.9",
|
|
"@nestjs/testing": "11.1.16",
|
|
"@types/archiver": "^6.0.0",
|
|
"@types/babel__preset-env": "7.10.0",
|
|
"@types/bytes": "^3.1.1",
|
|
"@types/dompurify": "^3.0.5",
|
|
"@types/express": "^4.17.13",
|
|
"@types/express-session": "^1.18.0",
|
|
"@types/graphql-upload": "^16.0.7",
|
|
"@types/html-to-text": "^9.0.4",
|
|
"@types/lodash.chunk": "^4.2.9",
|
|
"@types/lodash.differencewith": "^4.5.9",
|
|
"@types/lodash.isempty": "^4.4.7",
|
|
"@types/lodash.isequal": "^4.5.8",
|
|
"@types/lodash.isobject": "^3.0.7",
|
|
"@types/lodash.merge": "^4.6.9",
|
|
"@types/lodash.omit": "^4.5.9",
|
|
"@types/lodash.omitby": "^4.6.9",
|
|
"@types/lodash.snakecase": "^4.1.7",
|
|
"@types/lodash.uniq": "^4.5.9",
|
|
"@types/lodash.uniqby": "^4.7.9",
|
|
"@types/lodash.upperfirst": "^4.3.7",
|
|
"@types/mailparser": "^3.4.6",
|
|
"@types/ms": "^0.7.31",
|
|
"@types/node": "^24.0.0",
|
|
"@types/nodemailer": "^7.0.3",
|
|
"@types/openid-client": "^3.7.0",
|
|
"@types/passport-google-oauth20": "^2.0.11",
|
|
"@types/passport-jwt": "^3.0.8",
|
|
"@types/passport-microsoft": "^2.1.0",
|
|
"@types/pluralize": "^0.0.33",
|
|
"@types/psl": "^1.1.3",
|
|
"@types/react": "^18.2.39",
|
|
"@types/tar": "^7.0.87",
|
|
"@types/unzipper": "^0",
|
|
"@yarnpkg/types": "^4.0.0",
|
|
"rimraf": "^5.0.5",
|
|
"twenty-client-sdk": "workspace:*",
|
|
"twenty-emails": "workspace:*",
|
|
"twenty-shared": "workspace:*"
|
|
},
|
|
"engines": {
|
|
"node": "^24.5.0",
|
|
"npm": "please-use-yarn",
|
|
"yarn": "^4.0.2"
|
|
}
|
|
}
|