# Introduction
Now only using typeorm to generate migrations up and down statement
We handle and maintain our own migration table history
## What's new
Now all the instance commands will live within the same module and
folder than the upgrade commands
Sequentiality comes from the timestamp located in the filename
Same sequentiality also applies to the workspace commands in the future,
for the moment still expected a as code explicit declaration
( below screen is an example see below section )
<img width="1382" height="634" alt="image"
src="https://github.com/user-attachments/assets/5610a246-4eae-485e-99f4-98fb89ad5ac8"
/>
## Existing 1.21 migrations
We won't start following this pattern in 1.21 yet at least not with the
migration that has already been released as typeorm migrations in cloud
production as they would rerun
## Small duplication
Duplicating the legacy typeorm and instance commands run in the
`run-instance-commands` to avoid any merge of interest for the moment
## Concurrency
Not handling any run in parrallel of the upgrade for the moment
# Introduction
Typeorm migration are now associated to a given twenty-version from the
`UPGRADE_COMMAND_SUPPORTED_VERSIONS` that the current twenty core engine
handles
This way when we upgrade we retrieve the migrations that need to be run,
this will be useful for the cross-version incremental upgrade so we
preserve sequentiality
## What's new
To generate
```sh
npx nx database:migrate:generate twenty-server -- --name add-index-to-users
```
To apply all
```sh
npx nx database:migrate twenty-server
```
## Next
Introduce slow and fast typeorm migration in order to get rid of the
save point pattern in our code base
Create a clean and dedicated `InstanceUpgradeService` abstraction
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption
## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )
For our cloud we still need a way to bypass this security explaining the
-f flag
## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor
Related to https://github.com/twentyhq/twenty-infra/pull/529
## 1. The `twenty-client-sdk` Package (Source of Truth)
The monorepo package at `packages/twenty-client-sdk` ships with:
- A **pre-built metadata client** (static, generated from a fixed
schema)
- A **stub core client** that throws at runtime (`CoreApiClient was not
generated...`)
- Both ESM (`.mjs`) and CJS (`.cjs`) bundles in `dist/`
- A `package.json` with proper `exports` map for
`twenty-client-sdk/core`, `twenty-client-sdk/metadata`, and
`twenty-client-sdk/generate`
## 2. Generation & Upload (Server-Side, at Migration Time)
**When**: `WorkspaceMigrationRunnerService.run()` executes after a
metadata schema change.
**What happens in `SdkClientGenerationService.generateAndStore()`**:
1. Copies the stub `twenty-client-sdk` package from the server's assets
(resolved via `SDK_CLIENT_PACKAGE_DIRNAME` — from
`dist/assets/twenty-client-sdk/` in production, or from `node_modules`
in dev)
2. Filters out `node_modules/` and `src/` during copy — only
`package.json` + `dist/` are kept (like an npm publish)
3. Calls `replaceCoreClient()` which uses `@genql/cli` to introspect the
**application-scoped** GraphQL schema and generates a real
`CoreApiClient`, then compiles it to ESM+CJS and overwrites
`dist/core.mjs` and `dist/core.cjs`
4. Archives the **entire package** (with `package.json` + `dist/`) into
`twenty-client-sdk.zip`
5. Uploads the single archive to S3 under
`FileFolder.GeneratedSdkClient`
6. Sets `isSdkLayerStale = true` on the `ApplicationEntity` in the
database
## 3. Invalidation Signal
The `isSdkLayerStale` boolean column on `ApplicationEntity` is the
invalidation mechanism:
- **Set to `true`** by `generateAndStore()` after uploading a new client
archive
- **Checked** by both logic function drivers before execution — if
`true`, they rebuild their local layer
- **Set back to `false`** by `markSdkLayerFresh()` after the driver has
successfully consumed the new archive
Default is `false` so existing applications without a generated client
aren't affected.
## 4a. Logic Functions — Local Driver
**`ensureSdkLayer()`** is called before every execution:
1. Checks if the local SDK layer directory exists AND `isSdkLayerStale`
is `false` → early return
2. Otherwise, cleans the local layer directory
3. Calls `downloadAndExtractToPackage()` which streams the zip from S3
directly to disk and extracts the full package into
`<tmpdir>/sdk/<workspaceId>-<appId>/node_modules/twenty-client-sdk/`
4. Calls `markSdkLayerFresh()` to set `isSdkLayerStale = false`
**At execution time**, `assembleNodeModules()` symlinks everything from
the deps layer's `node_modules/` **except** `twenty-client-sdk`, which
is symlinked from the SDK layer instead. This ensures the logic
function's `import ... from 'twenty-client-sdk/core'` resolves to the
generated client.
## 4b. Logic Functions — Lambda Driver
**`ensureSdkLayer()`** is called during `build()`:
1. Checks if `isSdkLayerStale` is `false` and an existing Lambda layer
ARN exists → early return
2. Otherwise, deletes all existing layer versions for this SDK layer
name
3. Calls `downloadArchiveBuffer()` to get the raw zip from S3 (no disk
extraction)
4. Calls `reprefixZipEntries()` which streams the zip entries into a
**new zip** with the path prefix
`nodejs/node_modules/twenty-client-sdk/` — this is the Lambda layer
convention path. All done in memory, no disk round-trip
5. Publishes the re-prefixed zip as a new Lambda layer via
`publishLayer()`
6. Calls `markSdkLayerFresh()`
**At function creation**, the Lambda is created with **two layers**:
`[depsLayerArn, sdkLayerArn]`. The SDK layer is listed last so it
overwrites the stub `twenty-client-sdk` from the deps layer (later
layers take precedence in Lambda's `/opt` merge).
## 5. Front Components
Front components are built by `app:build` with `twenty-client-sdk/core`
and `twenty-client-sdk/metadata` as **esbuild externals**. The stored
`.mjs` in S3 has unresolved bare import specifiers like `import {
CoreApiClient } from 'twenty-client-sdk/core'`.
SDK import resolution is split between the **frontend host** (fetching &
caching SDK modules) and the **Web Worker** (rewriting imports):
**Server endpoints**:
- `GET /rest/front-components/:id` —
`FrontComponentService.getBuiltComponentStream()` returns the **raw
`.mjs`** directly from file storage. No bundling, no SDK injection.
- `GET /rest/sdk-client/:applicationId/:moduleName` —
`SdkClientController` reads a single file (e.g. `dist/core.mjs`) from
the generated SDK archive via
`SdkClientGenerationService.readFileFromArchive()` and serves it as
JavaScript.
**Frontend host** (`FrontComponentRenderer` in `twenty-front`):
1. Queries `FindOneFrontComponent` which returns `applicationId`,
`builtComponentChecksum`, `usesSdkClient`, and `applicationTokenPair`
2. If `usesSdkClient` is `true`, renders
`FrontComponentRendererWithSdkClient` which calls the
`useApplicationSdkClient` hook
3. `useApplicationSdkClient({ applicationId, accessToken })` checks the
Jotai atom family cache for existing blob URLs. On cache miss, fetches
both SDK modules from `GET /rest/sdk-client/:applicationId/core` and
`/metadata`, creates **blob URLs** for each, and stores them in the atom
family
4. Once the blob URLs are cached, passes them as `sdkClientUrls`
(already blob URLs, not server URLs) to `SharedFrontComponentRenderer` →
`FrontComponentWorkerEffect` → worker's `render()` call via
`HostToWorkerRenderContext`
**Worker** (`remote-worker.ts` in `twenty-sdk`):
1. Fetches the raw component `.mjs` source as text
2. If `sdkClientUrls` are provided and the source contains SDK import
specifiers (`twenty-client-sdk/core`, `twenty-client-sdk/metadata`),
**rewrites** the bare specifiers to the blob URLs received from the host
(e.g. `'twenty-client-sdk/core'` → `'blob:...'`)
3. Creates a blob URL for the rewritten source and `import()`s it
4. Revokes only the component blob URL after the module is loaded — the
SDK blob URLs are owned and managed by the host's Jotai cache
This approach eliminates server-side esbuild bundling on every request,
caches SDK modules per application in the frontend, and keeps the
worker's job to a simple string rewrite.
## Summary Diagram
```
app:build (SDK)
└─ twenty-client-sdk stub (metadata=real, core=stub)
│
▼
WorkspaceMigrationRunnerService.run()
└─ SdkClientGenerationService.generateAndStore()
├─ Copy stub package (package.json + dist/)
├─ replaceCoreClient() → regenerate core.mjs/core.cjs
├─ Zip entire package → upload to S3
└─ Set isSdkLayerStale = true
│
┌────────┴────────────────────┐
▼ ▼
Logic Functions Front Components
│ │
├─ Local Driver ├─ GET /rest/sdk-client/:appId/core
│ └─ downloadAndExtract │ → core.mjs from archive
│ → symlink into │
│ node_modules ├─ Host (useApplicationSdkClient)
│ │ ├─ Fetch SDK modules
└─ Lambda Driver │ ├─ Create blob URLs
└─ downloadArchiveBuffer │ └─ Cache in Jotai atom family
→ reprefixZipEntries │
→ publish as Lambda ├─ GET /rest/front-components/:id
layer │ → raw .mjs (no bundling)
│
└─ Worker (browser)
├─ Fetch component .mjs
├─ Rewrite imports → blob URLs
└─ import() rewritten source
```
## Next PR
- Estimate perf improvement by implementing a redis caching for front
component client storage ( we don't even cache front comp initially )
- Implem frontent blob invalidation sse event from server
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
## Summary
- **Add `lingui:compile` to Dockerfile** before both the server and
frontend build stages, ensuring compiled translation catalogs are always
fresh regardless of git state
- **Add `repository-dispatch` to i18n workflows** (`i18n-push.yaml` and
`i18n-pull.yaml`) to trigger reactive automerge in `twenty-infra` when
the i18n PR is ready, replacing the 15-minute polling approach
## Context
Users sometimes see "Uncompiled message detected" errors because
releases can be cut from `main` before the i18n PR (with freshly
compiled translation catalogs) has been merged. This creates a race
condition between new translatable strings landing on `main` and their
compiled catalogs being available.
These changes fix this in two ways:
1. **Safety net in builds**: Every Docker build now compiles
translations before building, so even if compiled catalogs in git are
stale, the build artifact is always correct
2. **Faster i18n PR merges**: Instead of a 15-minute cron polling for
i18n PRs, the workflows now notify `twenty-infra` immediately when
translations are ready, reducing merge latency from ~15 minutes to ~1
minute
Companion PR in twenty-infra: twentyhq/twenty-infra
(feat/i18n-reactive-automerge)
## Test plan
- [ ] Verify `TWENTY_INFRA_TOKEN` secret is available to i18n workflows
- [ ] Docker build still succeeds with the added `lingui:compile` steps
- [ ] i18n-push triggers automerge in twenty-infra after pushing changes
- [ ] i18n-pull triggers automerge in twenty-infra after pulling
translations
Made with [Cursor](https://cursor.com)
- enrich SSE events with relations
- remove queries from sse metadata events
- on sse event, manage store
- on object/field metadata changes, manage store
TODO left:
- fix other metadata items
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
## Add standard command menu items
### Summary
This PR introduces standard command menu items, migrating hardcoded
command menu actions to the backend command menu item architecture
powered by front components. It adds a new `twenty-standard-application`
package that defines, builds, and registers front components as standard
command menu items, gated behind the `IS_COMMAND_MENU_ITEM_ENABLED`
feature flag.
### Description
- **New `twenty-standard-application` package**: Contains front
component definitions with an esbuild-based build pipeline that
generates minified `.mjs` bundles and a manifest with checksums.
- **Server-side registration**: New constants register all items with
metadata (labels, icons, positions, availability types, conditional
expressions). A `StandardFrontComponentUploadService` uploads built
components to file storage.
- **`FALLBACK` availability type**: New enum value for command menu
items that appear as fallback options (e.g., "Search Records" fallback).
- **`CommandMenuContextApi` refactor**
- **Conditional availability enhancements**: New array-based helper
functions for evaluating multi-record conditions.
- **Frontend wiring** (twenty-front):
`useCommandMenuItemFrontComponentCommands`
## Next steps
Only simple commands have been implemented for now:
- **Navigation (9)** -- `CommandLink`: go-to-companies,
go-to-dashboards, go-to-notes, go-to-opportunities, go-to-people,
go-to-runs, go-to-settings, go-to-tasks, go-to-workflows
- **Side panel (4)** -- `CommandOpenSidePanelPage`: ask-ai,
search-records, search-records-fallback, view-previous-ai-chats
We still have to implement front components for all the following
commands:
All have placeholder `execute` logic (`async () => {}`) with a `// TODO:
implement execute logic` comment:
**Record (22)**
- `add-to-favorites`, `remove-from-favorites`
- `create-new-record`, `create-new-view`
- `delete-single-record`, `delete-multiple-records`
- `destroy-single-record`, `destroy-multiple-records`
- `restore-single-record`, `restore-multiple-records`
- `export-from-record-index`, `export-from-record-show`,
`export-multiple-records`, `export-note-to-pdf`, `export-view`
- `hide-deleted-records`, `see-deleted-records`
- `import-records`, `merge-multiple-records`, `update-multiple-records`
- `navigate-to-next-record`, `navigate-to-previous-record`
**Page layout (3)** -- `cancel-record-page-layout`,
`edit-record-page-layout`, `save-record-page-layout`
**Dashboard (4)** -- `cancel-dashboard-layout`, `duplicate-dashboard`,
`edit-dashboard-layout`, `save-dashboard-layout`
**Workflow (10)** -- `activate-workflow`, `add-node-workflow`,
`deactivate-workflow`, `discard-draft-workflow`, `duplicate-workflow`,
`see-active-version-workflow`, `see-runs-workflow`,
`see-versions-workflow`, `test-workflow`, `tidy-up-workflow`
**Workflow version (4)** -- `see-runs-workflow-version`,
`see-versions-workflow-version`, `see-workflow-workflow-version`,
`use-as-draft-workflow-version`
**Workflow run (3)** -- `see-version-workflow-run`,
`see-workflow-workflow-run`, `stop-workflow-run`
## Summary
- **Merge queue optimization**: Created a dedicated
`ci-merge-queue.yaml` workflow that only runs Playwright E2E tests on
`ubuntu-latest-8-cores`. Removed `merge_group` trigger from all 7
existing CI workflows (front, server, shared, website, sdk, zapier,
docker-compose). The merge queue goes from ~30+ parallel jobs to a
single focused E2E job.
- **Label-based merge queue simulation**: Added `run-merge-queue` label
support so developers can trigger the exact merge queue E2E pipeline on
any open PR before it enters the queue.
- **Prettier in lint**: Chained `prettier --check` into `lint` and
`prettier --write` into `lint --configuration=fix` across `nx.json`
defaults, `twenty-front`, and `twenty-server`. Prettier formatting
errors are now caught by `lint` and fixed by `lint:fix` /
`lint:diff-with-main --configuration=fix`.
## After merge (manual repo settings)
Update GitHub branch protection required status checks:
1. Remove old per-workflow merge queue checks (`ci-front-status-check`,
`ci-e2e-status-check`, `ci-server-status-check`, etc.)
2. Add `ci-merge-queue-status-check` as the required check for the merge
queue
## Summary
- **Refactor frontend metadata loading architecture**: Split the
monolithic `EagerMetadataLoadEffect` into focused provider effects
(`UserMetadataProviderEffect`, `ObjectMetadataProviderEffect`,
`ViewMetadataProviderEffect`) orchestrated by `MetadataProviderEffects`.
Replaced `UserProvider` + `ObjectMetadataItemsProvider` with a single
`MetadataGater` that gates rendering on `isAppMetadataReadyState`. The
metadata store now validates view-object consistency before promoting
views, and `updateDraft` skips no-op updates via deep equality checks.
- **SDK CLI improvements**: Added `app:typecheck` command, improved
error handling in API sync (extracts GraphQL error messages), added
`serializeError` utility for human-readable error output, added `error`
file status to dev mode orchestrator with UI support, and fixed
ClickHouse migration/seed commands to use `transpile-only`.
## Summary
Fixes `EMFILE: too many open files, watch` crash that most of the team
is hitting on macOS when running `yarn start` or `npx nx start
twenty-server`.
Adds `rimraf dist` before `nest start --watch` in the `start` and
`start:debug` targets, so the watcher starts with a clean output
directory.
## Root cause
The NestJS SWC compiler (`@nestjs/cli@11`) creates **three overlapping
chokidar watchers** when `nest start --watch` runs:
| Watcher | Watches | Purpose | Handles |
|---|---|---|---|
| SWC CLI (`@swc/cli`) | `src/` | Detects file changes → recompiles |
~1,730 |
| NestJS `watchFilesInSrcDir` | `src/` | Workaround: SWC misses new
files | shared with above |
| NestJS `watchFilesInOutDir` | **`dist/`** | Detects compiled `.js` →
restarts server | **~3,548** |
`@nestjs/cli@11` ships with **chokidar v4**, which dropped macOS
`fsevents` support and uses `fs.watch()` instead — creating **one file
descriptor per directory**. Chokidar v3 used a single `fsevents` kernel
subscription per directory tree.
Total: **~5,000+ `fs.watch()` handles**, far exceeding the default macOS
`ulimit -n` of ~2,560.
### Why it broke now
PR #17851 (`15fc850212`) changed the `start` target from `dependsOn:
["build"]` to `dependsOn: ["^build"]`, removing the `rimraf dist && nest
build` pre-step. Without that cleanup, `dist/` accumulated stale
directories from code reorganizations (e.g. `application-layer/` →
`application/` rename), growing to ~3,548 directories vs ~1,730 in a
clean build.
## What this PR does
Adds `rimraf dist &&` before `nest start --watch` in the `start` and
`start:debug` commands. This ensures `dist/` starts empty and only
contains directories matching the current `src/` structure (~1,730),
keeping watcher count in the ~3,400 range.
We still get the startup speed improvement from #17851 (no redundant
full SWC build), since `rimraf dist` is ~instant while the removed `nest
build` step took 30-60s.
## Future considerations
As the codebase grows, even a clean `dist/` will eventually approach the
macOS default `ulimit -n` (~2,560). Options to consider if that happens:
1. **Yarn resolution to force chokidar 3.6.0** — restores `fsevents`,
reducing watcher count from ~5,000 to ~3-5. This is what Vite 7 does
internally. Simple and effective, but pins to an older major version.
2. **Patch `@nestjs/cli`** to skip the `dist/` watcher — the
`watchFilesInOutDir` watcher accounts for ~65% of all handles and only
exists because NestJS doesn't have a direct hook into SWC's
compilation-complete event. Could be removed via `yarn patch`.
3. **Replace `nest start --watch` entirely** — use `node
--watch-path=src` (Node 22+) with `@swc-node/register` for on-the-fly
compilation. Uses a single native watcher regardless of directory count.
Requires rethinking asset copying (`watchAssets` in `nest-cli.json`).
4. **Wait for upstream fix** — NestJS CLI should either re-add
`fsevents` support or use Node's recursive `fs.watch()` option
(available since Node 20) instead of per-directory watchers.
## Test plan
- [ ] Run `npx nx start twenty-server` on macOS — server starts without
EMFILE error
- [ ] Run `npx nx start:debug twenty-server` — debug mode starts without
EMFILE error
- [ ] Edit a `.ts` file while server is running — hot reload still works
- [ ] Run `yarn start` (frontend + backend + worker) — no crashes
Made with [Cursor](https://cursor.com)
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
- Changed 6 Nx targets (`start`, `start:debug`, `typeorm`, `ts-node`,
`database:migrate`, `database:migrate:revert`) from `dependsOn:
["build"]` to `dependsOn: ["^build"]` to eliminate a redundant full SWC
compilation step (~30-60s per invocation).
- These targets all use `nest start --watch`, `ts-node`, or TypeORM's
CLI (which runs under ts-node), so they already compile TypeScript
themselves. The `build` dependency was causing `rimraf dist && nest
build` to run first, only for the target's own command to recompile
everything again.
- Fixed `start:debug` to call `nest start --watch --debug` directly
instead of routing through `nx start --debug`, which would trigger yet
another build cycle.
Note: targets that run pre-compiled code from `dist/` (like
`database:reset`) intentionally keep `dependsOn: ["build"]`.
## Test plan
- [ ] Run `npx nx start twenty-server` and verify the server starts with
only one SWC compilation pass instead of two
- [ ] Run `npx nx start:debug twenty-server` and verify the debugger
attaches correctly
- [ ] Run `npx nx run twenty-server:database:migrate` and verify
migrations run correctly
- [ ] Run `npx nx run twenty-server:database:reset twenty-server` and
verify it still builds before running (unchanged)
Made with [Cursor](https://cursor.com)
Co-authored-by: Cursor <cursoragent@cursor.com>
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1989
1/ Migration, applicationId and universalIdentifier are required on
entity ( save point migration + upgrade command fallback pattern )
2/ Backfill using previous standard ids
## Test
tested prod extract
## Note
Added build to typeorm nx generate migration command so we never forgot
to build server before
Upgraded to Storybook 10. We still use `@storybook/test-runner` for
testing since it appears it'd require more work to move from Jest to
Vitest than I initially anticipated, but I completed this PR to fix
`storybook:serve:dev` - it takes time to load, but it works the way it
used to with Storybook 8.
https://github.com/user-attachments/assets/7afc32c6-4bcf-4b37-b83b-8d00d28dda15
## Summary
The `lint:changed` command was not using the correct ESLint config for
`twenty-server`, causing it to use the root `eslint.config.mjs` instead
of the package-specific one. This PR fixes the issue and renames the
command to `lint:diff-with-main` for clarity.
## Problem
- `twenty-front` correctly specified `--config
packages/twenty-front/eslint.config.mjs`
- `twenty-server` just called `npx eslint` without specifying a config
- This meant `twenty-server` was missing important rules like:
- `@typescript-eslint/no-explicit-any: 'error'`
- `@stylistic/*` rules (linebreak-style, padding, etc.)
- `import/order` with NestJS patterns
- Custom workspace rules (`@nx/workspace-inject-workspace-repository`,
etc.)
## Solution
1. **Renamed** `lint:changed` to `lint:diff-with-main` to be explicit
about what the command does
2. **Centralized** the configuration in `nx.json` targetDefaults:
- Uses `{projectRoot}` interpolation for paths (resolved by Nx at
runtime)
- Each package automatically uses its own ESLint config
- Packages can override specific options (e.g., file extension pattern)
3. **Simplified** `project.json` files by inheriting from defaults
## Usage
```bash
# Lint only files changed vs main branch
npx nx lint:diff-with-main twenty-front
npx nx lint:diff-with-main twenty-server
# Auto-fix files changed vs main
npx nx lint:diff-with-main twenty-front --configuration=fix
```
## Changes
- **nx.json**: Added `lint:diff-with-main` target default with
configurable pattern
- **twenty-front/project.json**: Simplified to inherit from defaults
- **twenty-server/project.json**: Overrides pattern to include `.json`
files
- **CLAUDE.md** and **.cursor/rules**: Updated documentation
## Summary
This PR adds the `lingui/no-unlocalized-strings` ESLint rule to detect
untranslated strings and fixes translation issues across multiple
components.
## Changes
### ESLint Configuration (`eslint.config.react.mjs`)
- Added comprehensive `ignore` patterns for non-translatable strings
(CSS values, HTML attributes, technical identifiers)
- Added `ignoreNames` for props that don't need translation (className,
data-*, aria-*, etc.)
- Added `ignoreFunctions` for console methods, URL APIs, and other
non-user-facing functions
- Disabled rule for debug files, storybook, and test files
### Components Fixed (~19 files)
- Object record components (field inputs, pickers, merge dialogs)
- Settings components (accounts, admin panel)
- Serverless function components
- Record table and title cell components
## Status
🚧 **Work in Progress** - ~124 files remaining to fix
This PR is being submitted as draft to allow progressive fixing of
remaining translation issues.
## Testing
- Run `npx eslint "src/**/*.tsx"` in `packages/twenty-front` to check
remaining issues
In this PR,
- current basic E2E tests are fixed, and some were added, covering some
basic scenarios
- some tests avec been commented out, until we decide whether they are
worth fixing
The next steps are
- evaluate the flakiness of the tests. Once they've proved not to be
flaky, we should add more tests + re-write the current ones not using
aria-label (cf @lucasbordeau indication).
- We will add them back to the development flow
# Introduction
In this pullrequest have been migrated to the flat standard entities:
Role, Agent and RoleTargets.
## What happens
- Removed createStandardMorph tool util in favor of dynamic typing of
`createMorphOrRelationStandardField`
- Implemented a command to remove standard agents and their role that
has been removed in https://github.com/twentyhq/twenty/pull/16513 also
added a default role target to data manipulator role to the only
remaining agent
- Implemented an agent deleteMany service handler
# Introduction
Fixed the build dependency leading to twenty-server start failing before
building,
Removed redundant steps
Make everything run on test db
# Introduction
Defining very first basis of the twenty-cli e2e testing env.
Dynamically generating tests cases based on a list of applications names
that will be matched to stored twenty-apps and run install delete and
reinstall with their configuration on the same instance
We could use a glob pattern with a specific e2e configuration in every
apps but right now overkill
## Notes
- We should define typescript path aliasing to ease import devxp
- parse the config using a zod object
## Some vision on test granularity
Right now we only check that the server sent back success or failure on
below operation. In the future the synchronize will return a report of
what has been installed per entity exactly. We will be able to snapshot
everything in order to detect regressions
We should also be testing the cli directly for init and other stuff in
the end, we could get some inspiration from what's done in preconstruct
e2e tests with an on heap virtual file system
## Conclusion
Any suggestions are more than welcomed !
close https://github.com/twentyhq/core-team-issues/issues/1721
trying to solve build errors when launching commands
```console
> nx run twenty-shared:generateBarrels [existing outputs match the cache, left as is]
> nx run twenty-shared:build [existing outputs match the cache, left as is]
> nx run twenty-emails:build [existing outputs match the cache, left as is]
> nx run twenty-server:typecheck
> tsc -b tsconfig.json --incremental
> nx run twenty-server:build
> rimraf dist
> nest build --path ./tsconfig.build.json
> SWC Running...
[Error: ENOTEMPTY: directory not empty, rmdir '/Users/martinmuller/Desktop/twenty/packages/twenty-server/dist/src/modules'] {
errno: -66,
code: 'ENOTEMPTY',
syscall: 'rmdir',
path: '/Users/martinmuller/Desktop/twenty/packages/twenty-server/dist/src/modules'
}
```
## Context
Now adding serverless code sync within the migration v2 logic itself. To
do that we need to follow the
- Prepare flat input
- Build migration
- Run migration
steps where now the serverless function entity will store in DB and
cache a checksum of its code and the flat input will contain the code
with the checksum. Build will compare checksum and create an update
action containing the code if it has changed and the migration will now
run the corresponding services (instead of calling those in the parent
serverless service exposed in the API) allowing us to keep that logic
functional for other use cases such as import/export, twenty-cli and
twenty upgrades.
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <corentin@twenty.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
BlocknoteJS requires an ESM module where our server is CJS, this forced
us to pin the server-util version, which led us to force the resolution
of several packages, leading to bugs downstream.
From Node 22.12 Node supports requiring ESM modules (available from Node
22.0 with a flag). So I upgrade the module.
I picked Node 22 and not Node 23 or Node 24 because 22 is the LTS and we
don't plan to change node versions frequently.
If you remain on Node 18, things should still mostly work, except if you
edit a Rich Text field.
I also starting changing the default runtime for Serverless Functions
which isn't directly related. This means new serverless functions will
be created on Node 22, but we will still need another PR to migrate
existing serverless functions before September (end of support by AWS).
(In this PR I also remove the upgrade commands from 0.43 since they rely
on Blocknote and I didn't want to have to deal with this)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
No need to audit log workflow runs as it's already a form of audit log.
Add more audit log for other objects
Rename MessagingTelemetry to MessagingMonitoring
Merge Analytics and Audit in one (Audit)
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
# Introduction
Defined `dependsOn` for each nx project's configuration that has a
dependency to another local package ( ui, shared ).
As follows:
```json
"dependsOn": ["^build"]
```
Where the `^` symbol means "all dependencies of this project"
Now on a fresh repo, no built or install deps after install dependencies
you can directly hit `npx nx build PROJECT_NAME`
closes https://github.com/twentyhq/core-team-issues/issues/371
Related what was failing
[run](https://github.com/twentyhq/twenty-infra/actions/runs/13141544809/job/36669643182)
Cancelled before deploy, attested build was correct within the publish
and digest
# Introduction
It seems like that within the `twenty-server` `nx-project` there's no
entry for the `typecheck` command. It also seems to silently fail along
the `start` command even if it depends on it.
Looking at this
[run](https://github.com/twentyhq/twenty/actions/runs/12865329787/job/35865644811?pr=9736)
we can see that even when containing following diff it still does not
fail
```ts
// twenty-server/main.ts
// @ts-expected-error: Manually triggering TSC error should be deleted afterwards
const bootstrap = async () => {
```
By adding the `typecheck` entry in the project the same run will now
[fails](https://github.com/twentyhq/twenty/actions/runs/12865516249/job/35866224325)
# Notes
- There're no existing TSC errors even if wasn't guarded by the CI rn 😎
- The ci will take dozen of more seconds to run, should keep an eye on
// with nx implem checking if it works as expected
The DX is not great when you need to do a lot of database
resets/command.
Should we disable Typescript validation to speed things up? With this
and caching database:reset takes 1min instead of 2 on my machine.
See also: https://github.com/typeorm/typeorm/issues/4136
And #9291 / #9293
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Run the CI integrationin sync mode
and add the option to run it without db reset
cleaning all the useless integration test
---------
Co-authored-by: guillim <guillaume@twenty.com>
### Summary
This PR introduces several integration tests, a mix of manually written
tests and those generated using the `generate-integration-tests` Python
script located in the `scripts` folder.
### Tests Added:
- **Authentication tests**: Validating login, registration, and token
handling.
- **FindMany queries**: Fetching multiple records for all existing
entities that do not require input arguments.
### How the Integration Tests Work:
- A `setupTest` function is called during the Jest test run. This
function initializes a test instance of the application and exposes it
on a dedicated port.
- Since tests are executed in isolated workers, they do not have direct
access to the in-memory app instance. Instead, the tests query the
application through the exposed port.
- A static accessToken is used, this one as a big expiration time so it
will never expire (365 years)
- The queries are executed, and the results are validated against
expected outcomes.
### Current State and Next Steps:
- These tests currently run using the existing development seed data. We
plan to introduce more comprehensive test data using `faker` to improve
coverage.
- At the moment, the only mutation tests implemented are for
authentication. Future updates should include broader mutation testing
for other entities.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
- removing ugprade-0.24 commands as we are releasing 0.30
- introducing cache:flush command
- refactoring upgrade command and sync-metadata command to use the
ActiveWorkspacesCommand so they consistently run on all workspaces or
selected workspaces
Fixes:
- clear localStorage on sign out
- fix missing workspaceMember in verify resolver
- do not throw on datasource already destroyed exception which can
happen with race condition when several resolvers are resolving in
parallel
I have investigated the performance of our frontend vite build:
`npx nx run twenty:start` of `npx nx run twenty:build`
RAM usage:
- 160Mb: vite serve
- background typescript checker: 2.5GB
- background eslint checker: 3.5GB
I'm introducing two environment variables in FE .env to disable these
checkers on lower configuration (and to disable them from CD build):
```
# VITE_DISABLE_TYPESCRIPT_CHECKER=true
# VITE_DISABLE_ESLINT_CHECKER=true
```
An ACTIVE workspace is a workspace that has a complete workspaceSchema
and is authorized to be browsed by users.
In this PR, I'm:
- introducing a new activationStatus: PENDING_CREATION (existing ACTIVE
/ INACTIVE)
- removing workspaceService.isWorkspaceActivated (based on
workspaceSchema existence which is not robust and checking
activationStatus.ACTIVE instead)
- removing dynamic activationStatus field on worksapce resolver (we can
use the postgres column directly now that data has been migrated)
- on user sign up creating the workspace in PENDING_CREATION, and on
workspace activation setting it to ACTIVE
- only re-activating a workspace if the current activationStatus is
INACTIVE through billing webhooks (a PENDING_CREATION should stay
PENDING and ACTIVE should stay ACTIVE)
# This PR
- Fix#5021
- Migrates `passwordResetToken` and `passwordResetTokenExpiresAt` fields
from `core.users` to `core.appToken`
- Marks those fields as `deprecated` so we can remove them later if we
are happy with the transition -- I took this decision on my own,
@FellipeMTX let me know what you think about it, we can also remove them
straight away if you think it's better
- Fixed the `database:migration` script from the `twenty-server` to:
```json
"database:migrate": {
"executor": "nx:run-commands",
"dependsOn": ["build"], // added this line
"options": {
"cwd": "packages/twenty-server",
"commands": [
"nx typeorm -- migration:run -d src/database/typeorm/metadata/metadata.datasource",
"nx typeorm -- migration:run -d src/database/typeorm/core/core.datasource"
],
"parallel": false
}
},
```
The migration script wasn't running because the builds were not executed
- [x] Added unit tests for the token.service file's changes
Looking forward to hearing feedback from you
cc: @charlesBochet
---------
Co-authored-by: Weiko <corentin@twenty.com>
Closes#5097
- Uses "nx affected" to detect what projects need to be checked in the
current PR (for now, `ci-front` and `ci-server` workflows only).
- Caches results of certain tasks (`lint`, `typecheck`, `test`,
`storybook:build`) when a PR pipeline runs. The next runs of the same
PR's pipeline will then be able to reuse the PR's task cache to execute
tasks faster.
- Caches Yarn's cache folder to install dependencies faster in CI jobs.
- Rewrites the node modules cache/install steps as a custom, reusable
Github action.
- Distributes `ci-front` jobs with a "matrix" strategy.
- Sets common tasks config at the root `nx.json`. For instance, to
activate the `typecheck` task in a project, add `typecheck: {}` to its
`project.json` and it'll use the default config set in `nx.json` for the
`typecheck` task. Options can be overridden in each individual
`project.json` if needed.
- Adds "scope" tags to some projects: `scope:frontend`, `scope:backend`,
`scope:shared`. An eslint rule ensures that `scope:frontend` only
depends on `scope:frontent` or `scope:shared` projects, same for
`scope:backend`. These tags are used by `nx affected` to filter projects
by scope and generates different task cache keys according to the
requested scope.
- Enables checks for twenty-emails in the `ci-server` workflow.
