c0fef0be08567fff930bd2d2a5ded3e12e3c35b5
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Paul RastoinandGitHub
|
2413af0ba9 |
[run-instance-commands] Preserve fast slow sequentiality (#19757)
# Introduction The command was wrongly running all fast and then all slow ignoring instance commands segment Leading to ```ts 1.23.0_AddGlobalObjectContextToCommandMenuItemAvailabilityTypeFastInstanceCommand_1776090711153 executed successfully [Nest] 32679 - 04/16/2026, 1:21:07 PM LOG [InstanceCommandRunnerService] 1.23.0_DropWorkspaceVersionColumnFastInstanceCommand_1785000000000 executed successfully [Nest] 32679 - 04/16/2026, 1:21:07 PM LOG [InstanceCommandRunnerService] 1.22.0_BackfillWorkspaceIdOnIndirectEntitiesSlowInstanceCommand_1775758621018 executed successfully [Nest] 32679 - 04/16/2026, 1:21:07 PM LOG [RunInstanceCommandsCommand] Instance commands completed ``` No prod/self host impact as 1.23 hasn't been released yet |
||
|
Paul RastoinandGitHub
|
a4cc7fb9c5 |
[Upgrade] Fix workspace creation cursor (#19701)
## Summary ### Problem The upgrade migration system required new workspaces to always start from a workspace command, which was too rigid. When the system was mid-upgrade within an instance command (IC) segment, workspace creation would fail or produce inconsistent state. ### Solution #### Workspace-scoped instance command rows Instance commands now write upgrade migration rows for **all active/suspended workspaces** alongside the global row. This means every workspace has a complete migration history, including instance command records. - `InstanceCommandRunnerService` reloads `activeOrSuspendedWorkspaceIds` immediately before writing records (both success and failure paths) to mitigate race conditions with concurrent workspace creation. - `recordUpgradeMigration` in `UpgradeMigrationService` accepts a discriminated union over `status`, handles `error: unknown` formatting internally, and writes global + workspace rows in batch. #### Flexible initial cursor for new workspaces `getInitialCursorForNewWorkspace` now accepts the last **attempted** (not just completed) instance command with its status: - If the IC is `completed` and the next step is a workspace segment → cursor is set to the last WC of that segment (existing behavior). - If the IC is `failed` or not the last of its segment → cursor is set to that IC itself, preserving its status. This allows workspaces to be created at any point during the upgrade lifecycle, including mid-IC-segment and after IC failure. #### Relaxed workspace segment validation `validateWorkspaceCursorsAreInWorkspaceSegment` accepts workspaces whose cursor is: 1. Within the current workspace segment, OR 2. At the immediately preceding instance command with `completed` status (handles the `-w` single-workspace upgrade scenario). Workspaces with cursors in a previous segment, ahead of the current segment, or at a preceding IC with `failed` status are rejected. ### Test plan created empty workspaces to allow testing upgrade with several active workspaces |
||
|
Paul RastoinandGitHub
|
21142d98fe |
Implement cross version upgrade (#19559)
# Introduction Refactoring the upgrade engine to handle cross version upgrade, completely getting rid of the semver `version` at db and runtime level It remains a visual a listing indicator for or CD process but also during devenv in order to prepare next release Will write a release process runbook documentation on how to handle upgrade step patch, command insertion etc as it needs to be cascaded across all the involved supported version **The upgrade sequence model:** The sequence is a flat, ordered array of upgrade steps (`UpgradeStep[]`), built from the registry by chaining all versions in order, each version contributing its fast-instance → slow-instance → workspace commands sorted by timestamp. Version is metadata for logging, not used in the algorithm. **Segments:** The sequence naturally splits into alternating segments of contiguous instance steps and contiguous workspace steps. The runner processes segments in order: - **Instance segment:** Run sequentially from the instance cursor. Each step runs once globally. - **Workspace segment:** Each workspace independently walks from its own cursor through the end of the segment. Workspaces are independent within a segment — they can be at different positions. - **Synchronization (workspace → instance):** The runner blocks before entering an instance segment. All active/suspended workspaces must have completed the last workspace step of the preceding workspace segment. If any workspace failed, abort. This is the only explicit synchronization point. - Instance → workspace ordering is implicit — the runner processes segments sequentially, so the instance segment naturally completes before the workspace segment begins. full docs https://gist.github.com/prastoin/e62106d455fd72d6b6ebada8351e5492 ## Version constants & type-level deprecation Version management is split into three atomic constants: `TWENTY_PREVIOUS_VERSIONS`, `TWENTY_CURRENT_VERSION`, and `TWENTY_NEXT_VERSIONS`. Two derived constants compose them: `CROSS_UPGRADE_SUPPORTED_VERSIONS` (previous + current — what the engine runs) and `ALL_TWENTY_VERSIONS` (the full ordered tuple including next). The registry service validates at module init that no version is duplicated across constants and that at least one previous version exists. A `DeprecatedSinceVersion<RemoveAtVersion, T>` type utility resolves to `T` while `TWENTY_CURRENT_VERSION` is below `RemoveAtVersion`, and to `never` once it reaches it — turning deprecation into a compile-time guarantee via `IndexOf` and `IsGreaterOrEqual` generics in `twenty-shared`. ### `workspace.version` column deprecation The column is replaced by cursor-based state inference from `UpgradeMigration` records, but cannot be dropped in 1.22: workspaces activated during 1.21 predate the cursor system and need their initial cursor backfilled first (`backfillWorkspaceCreatedIn1_21_0Cursors`). This backfill itself depends on a new `isInitial` column on `UpgradeMigration`, bootstrapped via a targeted TypeORM migration before the upgrade sequence runs. Both functions and the entity field are typed with `DeprecatedSinceVersion<'1.23.0', ...>`. When `TWENTY_CURRENT_VERSION` reaches `1.23.0`, compile errors force their removal — and the pre-declared `DropWorkspaceVersionColumnFastInstanceCommand` takes over to drop the column. ## What's next - ci cross version upgrade ( wip ) - banner asking to contact twenty administrator if workspace is outdated - upgrade healthcheck cli ## New unit/integ test pattern Create a dedicated `createNestApp` that consumes a real database in order not to have to mack any database interaction to the `upgradeMigrations` allowing full coverage of the whole `upgradeRunnerService.run` core logic |
||
|
Paul RastoinandGitHub
|
85be463487 |
Slow instance commands (#19431)
# Introduction
This PR introduces the slow instance commands pattern, that allow
migrating data in prior of the schema migration, that would fail if not.
Slow instance commands runs after the fast instance commands and before
the workspace commands.
On twenty instance that do not has any active or suspended workspace the
data migration part is skipped but the migration still runs, especially
for fresh installs
We were previously hacking through typeorm transaction system to gain
such granularity using save points:
```ts
export class AddPayloadToCommandMenuItem1775129635528
implements MigrationInterface
{
name = 'AddPayloadToCommandMenuItem1775129635528';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(
`ALTER TABLE "core"."commandMenuItem" ADD "payload" jsonb`,
);
const savepointName =
'sp_add_payload_check_constraint_to_command_menu_item';
try {
await queryRunner.query(`SAVEPOINT ${savepointName}`);
await addPayloadCheckConstraintToCommandMenuItem(queryRunner);
await queryRunner.query(`RELEASE SAVEPOINT ${savepointName}`);
} catch (e) {
try {
await queryRunner.query(`ROLLBACK TO SAVEPOINT ${savepointName}`);
await queryRunner.query(`RELEASE SAVEPOINT ${savepointName}`);
} catch (rollbackError) {
// oxlint-disable-next-line no-console
console.error(
'Failed to rollback to savepoint in AddPayloadToCommandMenuItem1775129635528',
rollbackError,
);
throw rollbackError;
}
// oxlint-disable-next-line no-console
console.error(
'Swallowing AddPayloadToCommandMenuItem1775129635528 error',
e,
);
}
}
```
It was afterwards re-applied within an workspace commands, it was hacky
and missleading for the self host having false positive in logs
## New pattern
Generate the slow instance command
```
npx nx database:migrate:generate twenty-server -- --name add-foo-bar-columns --type slow
```
```ts
import { DataSource, QueryRunner } from 'typeorm';
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
import { SlowInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/slow-instance-command.interface';
@RegisteredInstanceCommand('1.21.0', 1775640902366, { type: 'slow' })
export class AddPrastoinColToWorkspaceSlowInstanceCommand implements SlowInstanceCommand {
async runDataMigration(dataSource: DataSource): Promise<void> {
// TODO: implement data backfill before the DDL migration
}
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query('ALTER TABLE "core"."workspace" ADD "prastoin" character varying');
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query('ALTER TABLE "core"."workspace" DROP COLUMN "prastoin"');
}
}
```
### Why `run-instance-commands` and `upgrade` remain separate commands
These two commands serve fundamentally different purposes with
incompatible scoping semantics:
The run-instance-commands iterates over all the legacy typeorm and the
instance commands of all versions, used for database init and so on. we
could be centralizing both but the readability tradeoff isn't worth it
In the future thanks to the cross-version pattern we will be able to
centralize them but not right now
Please note that by default the `run-instance-commands` only run the
fast instance commands which is expected for our cloud prod CD
|
||
|
Paul RastoinandGitHub
|
c0eacedfec |
Workspace command decorators (#19397)
# Introduction Migrating the workspace commands to the decorator version + timestamp listing as for the instance commands We've now been able to remove the upgrade command abstraction where we needed to import all modules and order them Now they're dynamically retrieved at upgrade runtime, sorted by timestamp ## Instance and workspace commands name The name is computed from the command metadata `version` `className` and `timestamp` we have a duplicate validation at module init from the unified registry |
||
|
Paul RastoinandGitHub
|
23874848a4 |
Instance commands and upgrade_migrations table (#19356)
# Introduction Now only using typeorm to generate migrations up and down statement We handle and maintain our own migration table history ## What's new Now all the instance commands will live within the same module and folder than the upgrade commands Sequentiality comes from the timestamp located in the filename Same sequentiality also applies to the workspace commands in the future, for the moment still expected a as code explicit declaration ( below screen is an example see below section ) <img width="1382" height="634" alt="image" src="https://github.com/user-attachments/assets/5610a246-4eae-485e-99f4-98fb89ad5ac8" /> ## Existing 1.21 migrations We won't start following this pattern in 1.21 yet at least not with the migration that has already been released as typeorm migrations in cloud production as they would rerun ## Small duplication Duplicating the legacy typeorm and instance commands run in the `run-instance-commands` to avoid any merge of interest for the moment ## Concurrency Not handling any run in parrallel of the upgrade for the moment |