As per title, update a documentation on how to upload a file given
increasing amount of questions for this problem
CC: @StephanieJoly4
---------
Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
## Summary
- When `STORAGE_S3_PRESIGNED_URL_BASE` is configured, the file
controller returns a **302 redirect** to a presigned S3 URL instead of
proxying every byte through the server. This eliminates server bandwidth
and CPU overhead for S3-backed deployments.
- For local storage or S3 without a public endpoint, behavior is
unchanged (stream + pipe with security headers).
- Added `getPresignedUrl` to the `StorageDriver` interface (required
method returning `string | null`), with implementations in S3Driver
(uses a separate presign client with the public endpoint), LocalDriver
(returns `null`), and ValidatedStorageDriver (path traversal protection
+ delegation).
- Added a unified `getFileResponseById` method in `FileService` that
performs a single DB lookup and returns either a redirect URL or a
stream, avoiding double lookups.
- Extracted `getContentDisposition` from the header util so both the
proxy path and presigned URL path share the same inline/attachment
allowlist.
- Added MinIO service to `docker-compose.dev.yml` (optional `s3`
profile) for local S3 testing.
- Documented S3 presigned URL setup, CORS, and `nosniff` requirements in
the self-hosting docs.
## Test plan
- [x] All 63 unit tests pass across 5 test suites (util, S3 driver,
validated driver, file storage service, controller)
- [x] `npx nx typecheck twenty-server` passes
- [ ] Manual E2E test with MinIO: `docker compose --profile s3 up -d`,
configure S3 env vars, verify `curl -I` returns 302 with `Location`
header pointing to MinIO
- [ ] Verify local storage (no `STORAGE_S3_PRESIGNED_URL_BASE`) still
streams files with 200 + security headers
- [ ] Verify public assets endpoint still proxies (no redirect)
Made with [Cursor](https://cursor.com)
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
## Summary
This PR implements OAuth 2.0 Dynamic Client Registration (RFC 7591) and
OAuth 2.0 Protected Resource Metadata (RFC 9728) support, enabling
third-party applications to dynamically register as OAuth clients
without manual configuration.
## Key Changes
### OAuth Dynamic Client Registration
- **New Controller**: `OAuthRegistrationController` at `POST
/oauth/register` endpoint
- Validates client metadata according to RFC 7591 specifications
- Enforces PKCE-only public client model (no client secrets)
- Supports only `authorization_code` grant type and `code` response type
- Rate limits registrations to 10 per hour per IP address
- Returns `client_id` and registration metadata in response
- **Input Validation**: `OAuthRegisterInput` DTO with constraints on:
- Client name (max 256 chars)
- Redirect URIs (max 20, validated for security)
- Grant types, response types, scopes, and auth methods
- Logo and client URIs (max 2048 chars)
- **Discovery Endpoint Update**: Added `registration_endpoint` to OAuth
discovery metadata
### Stale Registration Cleanup
- **Cleanup Service**: Automatically removes OAuth-only registrations
older than 30 days that have no active installations
- **Cron Job**: Runs daily at 02:30 AM UTC with batch processing (100
records per batch)
- **CLI Command**: `cron:stale-registration-cleanup` to manually trigger
cleanup
### MCP (Model Context Protocol) Authentication
- **New Guard**: `McpAuthGuard` implements RFC 9728 compliance
- Wraps JWT authentication with proper error responses
- Returns `WWW-Authenticate` header with protected resource metadata URL
on 401
- Enables OAuth-protected MCP endpoints
### Protected Resource Metadata
- **New Endpoint**: `GET /.well-known/oauth-protected-resource` (RFC
9728)
- Advertises MCP resource as OAuth-protected
- Lists supported scopes and bearer token methods
- Enables OAuth clients to discover authorization requirements
### Application Registration Updates
- **New Source Type**: `OAUTH_ONLY` enum value for OAuth-only
registrations
- **Install Service**: Skips artifact installation for OAuth-only apps
(no code artifacts)
### Frontend Updates
- **Authorization Page**: Support both snake_case (standard OAuth) and
camelCase (legacy) query parameters
- `client_id` / `clientId`
- `code_challenge` / `codeChallenge`
- `redirect_uri` / `redirectUrl`
## Implementation Details
- **Rate Limiting**: Uses token bucket algorithm with 10 registrations
per 3,600,000ms window per IP
- **Scope Validation**: Requested scopes are capped to allowed OAuth
scopes; defaults to all scopes if not specified
- **Redirect URI Validation**: Uses existing `validateRedirectUri`
utility for security
- **Cache Headers**: Registration responses include `Cache-Control:
no-store` and `Pragma: no-cache`
- **Batch Processing**: Cleanup operations process 100 records at a time
to avoid memory issues
- **Grace Period**: 30-day grace period before cleanup to allow time for
client activation
https://claude.ai/code/session_01PxcuWFFRuXMASMaMGTLYk2
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
## Summary
This PR fixes several small documentation issues in the contributor and
setup guides:
- fixes broken docs links in the root README
- corrects multiple typos and capitalization issues in contributor docs
- fixes malformed Markdown for the Redis command in local setup
- improves wording in the Docker Compose self-hosting guide
## Changes
- updated README installation links to the current docs routes
- changed `Open-source` to `open-source`
- fixed `specially` -> `especially` in the frontend style guide
- normalized `MacOS` -> `macOS`, `powershell` -> `PowerShell`, and
`Postgresql` -> `PostgreSQL`
- replaced the invalid `localhost:5432` Markdown link with inline code
- fixed the malformed fenced code block for `brew services start redis`
- cleaned up Redis naming/capitalization and a few grammar issues in the
setup docs
- improved the warning and environment-variable wording in the Docker
Compose guide
## Testing
- not run; docs-only changes
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
