Commit Graph
22 Commits
Author SHA1 Message Date
8fa3962e1c feat: add resumable stream support for agent chat (#19107)
## Overview
Add resumable stream support for agent chat to allow clients to
reconnect and resume streaming responses if the connection is
interrupted (e.g., during page refresh).

## Changes

### Backend (Twenty Server)
- Add `activeStreamId` column to `AgentChatThreadEntity` to track
ongoing streams
- Create `AgentChatResumableStreamService` to manage Redis-backed
resumable streams using the `resumable-stream` library with ioredis
- Extend `AgentChatController` with:
  - `GET /:threadId/stream` endpoint to resume an existing stream
  - `DELETE /:threadId/stream` endpoint to stop an active stream
- Update `AgentChatStreamingService` to store streams in Redis and track
active stream IDs
- Add `resumable-stream@^2.2.12` dependency to package.json

### Frontend (Twenty Front)
- Update `useAgentChat` hook to:
- Use a persistent transport with `prepareReconnectToStreamRequest` for
resumable streams
  - Export `resumeStream` function from useChat
  - Add `handleStop` callback to clear active stream on DELETE endpoint
- Use thread ID as stable message ID instead of including message count
- Add stream resumption logic in `AgentChatAiSdkStreamEffect` component
to automatically call `resumeStream()` when switching threads

## Database Migration
New migration `1774003611071-add-active-stream-id-to-agent-chat-thread`
adds the `activeStreamId` column to store the current resumable stream
identifier.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:19:25 +02:00
Charles BochetandGitHub 191a277ddf fix: invalidate rolesPermissions cache + add Docker Hub auth to CI (#19044)
## Summary

### Cache invalidation fix
- After migrating object/field permissions to syncable entities (#18609,
#18751, #18567), changes to `flatObjectPermissionMaps`,
`flatFieldPermissionMaps`, or `flatPermissionFlagMaps` no longer
triggered `rolesPermissions` cache invalidation
- This caused stale permission data to be served, leading to flaky
`permissions-on-relations` integration tests and potentially incorrect
permission enforcement in production after object permission upserts
- Adds the three permission-related flat map keys to the condition that
triggers `rolesPermissions` cache recomputation in
`WorkspaceMigrationRunnerService.getLegacyCacheInvalidationPromises`
- Clears memoizer after recomputation to prevent concurrent
`getOrRecompute` calls from caching stale data

### Docker Hub rate limit fix
- CI service containers (postgres, redis, clickhouse) and `docker
run`/`docker build` steps were pulling from Docker Hub
**unauthenticated**, hitting the 100-pull-per-6-hour rate limit on
shared GitHub-hosted runner IPs
- Adds `credentials` blocks to all service container definitions and
`docker/login-action` steps before `docker run`/`docker compose`
commands
- Uses `vars.DOCKERHUB_USERNAME` + `secrets.DOCKERHUB_PASSWORD`
(matching the existing twenty-infra convention)
- Affected workflows: ci-server, ci-merge-queue, ci-breaking-changes,
ci-zapier, ci-sdk, ci-create-app-e2e, ci-website,
ci-test-docker-compose, preview-env-keepalive, spawn-twenty-docker-image
action
2026-03-27 17:32:53 +01:00
Paul RastoinandGitHub 281bb6d783 Guard yarn database:migrate:prod (#19008)
## Motivations
A lot of self hosters hands up using the `yarn database:migrated:prod`
either manually or through AI assisted debug while they try to upgrade
an instance while their workspace is still blocked in a previous one
Leading to their whole database permanent corruption

## What happened
Replaced the direct call the the typeorm cli to a command calling it
programmatically, adding a layer of security in case a workspace seems
to be blocked in a previous version than the one just before the one
being installed ( e.g 1.0 when you try to upgrade from 1.1 to 1.2 )

For our cloud we still need a way to bypass this security explaining the
-f flag

## Remark
Centralized this logic and refactored creating new services
`WorkspaceVersionService` and `CoreEngineVersionService` that will
become useful for the upcoming upgrade refactor

Related to https://github.com/twentyhq/twenty-infra/pull/529
2026-03-27 14:39:18 +00:00
Charles BochetandGitHub 9d57bc39e5 Migrate from ESLint to OxLint (#18443)
## Summary

Fully replaces ESLint with OxLint across the entire monorepo:

- **Replaced all ESLint configs** (`eslint.config.mjs`) with OxLint
configs (`.oxlintrc.json`) for every package: `twenty-front`,
`twenty-server`, `twenty-emails`, `twenty-ui`, `twenty-shared`,
`twenty-sdk`, `twenty-zapier`, `twenty-docs`, `twenty-website`,
`twenty-apps/*`, `create-twenty-app`
- **Migrated custom lint rules** from ESLint plugin format to OxLint JS
plugin system (`@oxlint/plugins`), including
`styled-components-prefixed-with-styled`, `no-hardcoded-colors`,
`sort-css-properties-alphabetically`,
`graphql-resolvers-should-be-guarded`,
`rest-api-methods-should-be-guarded`, `max-consts-per-file`, and
Jotai-related rules
- **Migrated custom rule tests** from ESLint `RuleTester` + Jest to
`oxlint/plugins-dev` `RuleTester` + Vitest
- **Removed all ESLint dependencies** from `package.json` files and
regenerated lockfiles
- **Updated Nx targets** (`lint`, `lint:diff-with-main`, `fmt`) in
`nx.json` and per-project `project.json` to use `oxlint` commands with
proper `dependsOn` for plugin builds
- **Updated CI workflows** (`.github/workflows/ci-*.yaml`) — no more
ESLint executor
- **Updated IDE setup**: replaced `dbaeumer.vscode-eslint` with
`oxc.oxc-vscode` extension, configured `source.fixAll.oxc` and
format-on-save with Prettier
- **Replaced all `eslint-disable` comments** with `oxlint-disable`
equivalents across the codebase
- **Updated docs** (`twenty-docs`) to reference OxLint instead of ESLint
- **Renamed** `twenty-eslint-rules` package to `twenty-oxlint-rules`

### Temporarily disabled rules (tracked in `OXLINT_MIGRATION_TODO.md`)

| Rule | Package | Violations | Auto-fixable |
|------|---------|-----------|-------------|
| `twenty/sort-css-properties-alphabetically` | twenty-front | 578 | Yes
|
| `typescript/consistent-type-imports` | twenty-server | 3814 | Yes |
| `twenty/max-consts-per-file` | twenty-server | 94 | No |

### Dropped plugins (no OxLint equivalent)

`eslint-plugin-project-structure`, `lingui/*`, `@stylistic/*`,
`import/order`, `prefer-arrow/prefer-arrow-functions`,
`eslint-plugin-mdx`, `@next/eslint-plugin-next`,
`eslint-plugin-storybook`, `eslint-plugin-react-refresh`. Partial
coverage for `jsx-a11y` and `unused-imports`.

### Additional fixes (pre-existing issues exposed by merge)

- Fixed `EmailThreadPreview.tsx` broken import from main rename
(`useOpenEmailThreadInSidePanel`)
- Restored truthiness guard in `getActivityTargetObjectRecords.ts`
- Fixed `AgentTurnResolver` return types to match entity (virtual
`fileMediaType`/`fileUrl` are resolved via `@ResolveField()`)

## Test plan

- [x] `npx nx lint twenty-front` passes
- [x] `npx nx lint twenty-server` passes
- [x] `npx nx lint twenty-docs` passes
- [x] Custom oxlint rules validated with Vitest: `npx nx test
twenty-oxlint-rules`
- [x] `npx nx typecheck twenty-front` passes
- [x] `npx nx typecheck twenty-server` passes
- [x] CI workflows trigger correctly with `dependsOn:
["twenty-oxlint-rules:build"]`
- [x] IDE linting works with `oxc.oxc-vscode` extension
2026-03-06 01:03:50 +01:00
Charles BochetandGitHub d48c58640c Migrate CI runners from Depot back to GitHub-hosted runners (#18347)
## Summary
- Replaces all `depot-ubuntu-24.04` runners with `ubuntu-latest`
- Replaces all `depot-ubuntu-24.04-8` runners with
`ubuntu-latest-8-cores`
- Updates storybook build cache keys in ci-front.yaml to reflect the
runner name change

Reverts the temporary Depot migration introduced in #18163 / #18179
across all 23 workflow files.
2026-03-03 14:14:27 +01:00
Félix MalfaitandGitHub 0223975bbd Harden GitHub Actions: fix injections, isolate privileged operations to ci-privileged repo (#18318)
## Summary

- Fix expression injection vulnerabilities in composite actions
(`restore-cache`, `nx-affected`) and workflow files (`claude.yml`)
- Reduce overly broad permissions in `ci-utils.yaml` (Danger.js) and
`ci-breaking-changes.yaml`
- Restructure `preview-env-dispatch.yaml`: auto-trigger for members,
opt-in for contributor PRs via `preview-app` label (safe because
keepalive has no write tokens)
- Isolate all write-access operations (PR comments, cross-repo posting)
to a new dedicated
[`twentyhq/ci-privileged`](https://github.com/twentyhq/ci-privileged)
repo via `repository_dispatch`, so that workflows in twenty that execute
contributor code never have write tokens
- Create `post-ci-comments.yaml` (`workflow_run` bridge) to dispatch
breaking changes results to ci-privileged, solving the [fork PR comment
issue](https://github.com/twentyhq/twenty/pull/13713#issuecomment-3168999083)
- Delete 5 unused secrets and broken `i18n-qa-report` workflow
- Remove `TWENTY_DISPATCH_TOKEN` from twenty (moved to ci-privileged as
`CORE_TEAM_ISSUES_COMMENT_TOKEN`)
- Use `toJSON()` for all `client-payload` values to prevent JSON
injection

## Security model after this PR

| Workflow | Executes fork code? | Write tokens available? |
|----------|---------------------|------------------------|
| preview-env-keepalive | Yes | None (contents: read only) |
| preview-env-dispatch | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN
only |
| ci-breaking-changes | Yes | None (contents: read only) |
| post-ci-comments (workflow_run) | No (default branch) |
CI_PRIVILEGED_DISPATCH_TOKEN only |
| claude.yml | No (base branch) | CI_PRIVILEGED_DISPATCH_TOKEN,
CLAUDE_CODE_OAUTH_TOKEN |
| ci-utils (Danger.js) | No (base branch) | GITHUB_TOKEN (scoped) |

All actual write tokens (`TWENTY_PR_COMMENT_TOKEN`,
`CORE_TEAM_ISSUES_COMMENT_TOKEN`) live in `twentyhq/ci-privileged` with
strict CODEOWNERS review and branch protection.

## Test plan

- [ ] Verify preview environment comments still appear on member PRs
- [ ] Verify adding `preview-app` label triggers preview for contributor
PRs
- [ ] Verify breaking changes reports still post on PRs (including fork
PRs)
- [ ] Verify Claude cross-repo responses still post on core-team-issues
- [ ] Confirm ci-privileged branch protection is enforced
2026-03-02 10:57:14 +01:00
Charles BochetandGitHub 1fa55bfd02 Fix bugs tied to jotai migration (#18227)
Fixing a few bugs:
- CommandMenu not reactive
- Filtering on index view infinite loop
2026-02-25 17:11:12 +01:00
Charles BochetandGitHub 129d1ede86 Change runners temp (#18163)
Temporarily moving all ubuntu-latest 1 core to depot except ci-website
2026-02-23 10:53:31 +01:00
Félix MalfaitandGitHub e6491d6a80 feat(i18n): fix translation QA issues and add automation (#16756)
## Summary

This PR fixes translation QA issues and adds automation to prevent
future issues.

### Translation Fixes
- Fixed **escaped Unicode sequences** in translations (e.g.,
`\u62db\u5f85` → `招待`)
- Removed **corrupted control characters** from .po files (null bytes,
invalid characters)
- Fixed **missing/incorrect placeholders** in various languages
- Deleted **35 problematic translations** via Crowdin API that had
variable mismatches

### New Scripts (in `packages/twenty-utils/`)
- `fix-crowdin-translations.ts` - Auto-fixes encoding issues and syncs
to Crowdin
- `fix-qa-issues.ts` - Fixes specific QA issues via Crowdin API
- `translation-qa-report.ts` - Generates weekly QA report from Crowdin
API

### New Workflow
- `i18n-qa-report.yaml` - Weekly workflow that creates a PR with
translation QA issues for review

### Other Changes
- Moved GitHub Actions from `.github/workflows/actions/` to
`.github/actions/`
- Fixed `date-utils.ts` to avoid nested `t` macros in plural expressions
(root cause of confusing placeholders)

### QA Status After Fixes
| Category | Count | Status |
|----------|-------|--------|
| variables | 0  | Fixed |
| tags | 1 | Minor |
| empty | 0  | Fixed |
| spaces | 127 | Low priority |
| numbers | 246 | Locale-specific |
| special_symbols | 268 | Locale-specific |
2025-12-22 17:30:46 +01:00
Félix MalfaitandGitHub 3d7f332f80 Change runner for breaking change CI (#15205)
As per title
2025-10-20 13:55:12 +02:00
Félix MalfaitandGitHub 5375a478db Fix: Make CI .env manipulation robust against missing trailing newlines (#15189)
## Problem

CI workflow started timing out on October 14, 2025 after commit
`d750df7fff` removed the trailing newline from `.env.example`.

## Root Cause

When `.env.example` lacks a trailing newline:
```bash
# Last line without newline
# CLICKHOUSE_URL=...twenty
```

And CI runs:
```bash
echo "NODE_PORT=3002" >> .env
```

Result:
```bash
# CLICKHOUSE_URL=...twentyNODE_PORT=3002  ← Commented out!
```

Server starts on default port 3000 instead of 3002, health check fails.

## Fix

1. **Restore trailing newline** to `.env.example`
2. **Make all CI `.env` operations robust** by adding `echo "" >> .env`
before appending
3. **Simplified `set_env_var`** function to always add newline first

Now works regardless of whether template files have trailing newlines.

## Files Changed

- 6 CI workflow files
- 1 .env.example file
2025-10-18 13:46:56 +02:00
Félix MalfaitandGitHub d76abefdee Fix CI concurrency: prevent test cancellation on main branch (#15188)
## Problem
The concurrency rules in CI workflows were cancelling in-progress test
runs even on the main branch. This caused inconsistent check counts when
multiple commits were pushed in quick succession.

## Solution
Updated `cancel-in-progress` in all CI workflows to be conditional:
- **On main branch**: Tests run to completion (no cancellation)
- **On feature branches**: Tests are cancelled when new commits are
pushed (saves CI resources)

## Changes
Modified 11 workflow files to use:
```yaml
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
```

This ensures every commit to main gets fully tested while maintaining
efficiency on feature branches.
2025-10-18 10:50:54 +02:00
EtienneandGitHub c693c4d9cf Common API - create common find one query (#14720)
closes https://github.com/twentyhq/core-team-issues/issues/1418

Tested : 
- findOne on Rest and Gql

### Vision
#### Common
- Common is kind of renamed Gql base resolver
- Common handles args (filter, values, ...) validation
- Common accepts depth or raw gql selected fields to compute
selectedFields
- Common is directly called by each CommonQueries (findOne, ...)
service, which extend CommonBaseQuery service
- Common sequence : 
| - Parse & Validate args (args-handlers, to create)
| - Build query (query-parsers : currently in gql-query-parsers, to
move)
| - Execute query
| - Fetch relation + format
#### Rest
- Simple parsing (without metadata validation)
- Calling Common API
- Simple rest response formatting
#### Gql
- Calling Common API
2025-10-08 10:18:42 +00:00
216d72b5d7 AI SDK v5 migration (#14549)
Co-authored-by: Félix Malfait <felix@twenty.com>
2025-09-22 22:13:43 +02:00
Charles BochetandGitHub a21daa2670 Optimize CI runner cost (#14628) 2025-09-20 12:14:29 +02:00
Félix MalfaitandGitHub 30a2164980 First Application POC (#14382)
Quick proof of concept for twenty-apps + twenty-cli, with local
development / hot reload

Let's discuss it!



https://github.com/user-attachments/assets/c6789936-cd5f-4110-a265-863a6ac1af2d
2025-09-10 15:12:38 +02:00
Charles BochetandGitHub 953f1e7207 Fix CI cache for storybook (#14038)
In this PR:
- Migrate twenty-ui to ES2020 (removing undesired type errors during
build)
- Make CI cache work again between `sb-build` and `sb-test` steps

(Note: also fixing a minor bug on Notes)

Total CI time: ~8min
<img width="1088" height="531" alt="image"
src="https://github.com/user-attachments/assets/0bd0a99a-c69e-491d-91b2-9ddf6622464a"
/>

Both node_modules and sb-build caches are being hit now
<img width="1154" height="722" alt="image"
src="https://github.com/user-attachments/assets/a5addcee-51d2-4f51-9ff6-d02cd25da49a"
/>
2025-08-22 09:34:35 +02:00
nitinandGitHub a3322745b5 revert ci changes on ci-breaking-changes (#13682)
<img width="990" height="513" alt="Screenshot 2025-08-06 at 16 27 36"
src="https://github.com/user-attachments/assets/23a99dab-09cf-4978-8c10-5d8a2f7a84c9"
/>
2025-08-06 16:30:00 +05:30
nitinandGitHub 60257a9701 fix api mismatch on rest metadata (#13680)
## Context

This PR fixes a REST API metadata schema mismatch that occurred after
the webhook and apiKey migration from workspace entities to metadata
level.

## The Issue

After PR #13576 deleted the webhook and apiKey workspace entities, the
metadata OpenAPI specification ended up in a broken state:
- The `/metadata` endpoints still had paths for `/webhooks` and
`/apiKeys`
- But the schema definitions were missing (they were previously pulled
from workspace entities)
- This created dangling references in the OpenAPI document

## The Fix

Added proper schema definitions for webhook and apiKey directly in
`computeMetadataSchemaComponents`:
- `Webhook`, `WebhookForUpdate`, `WebhookForResponse`
- `ApiKey`, `ApiKeyForUpdate`, `ApiKeyForResponse`

## Why the CI is Failing

The CI breaking changes detection is comparing:
- **Main branch**: Has a broken OpenAPI document with endpoints that
reference non-existent schemas
- **This branch**: Has the fixed OpenAPI document with proper schemas

The OpenAPI diff tool can't even parse the main branch's document due to
the missing schema references, hence the error -- tested locally
2025-08-06 16:15:05 +05:30
Félix MalfaitandGitHub d59604c92d Fix small CI escape issue (#13331)
Fixes #13298
2025-07-22 12:13:53 +02:00
Félix MalfaitandGitHub d744ef129e Rebase main before applying breaking change CI (#12992)
Avoid false positives when a breaking change was recently merged and the
PR branch isn't up to date
2025-07-02 07:39:53 +02:00
Félix MalfaitandGitHub 21f9db1f41 Improve breaking change CI (#12816)
It used to post a comment if the API schema changed, even if there's no
breaking change. I thought this could be OK as an FYI. But it is not
since now we generate the examples dynamically with Faker the OpenAPI
schema is always different
2025-06-24 14:53:14 +02:00