Commit Graph
508 Commits
Author SHA1 Message Date
martmullandGitHub 9880f192a5 Move composite types to twenty-shared (#15741) 2025-11-10 22:11:52 +01:00
WeikoandGitHub 02fda92a93 Global workspace datasource poc (#15744)
## Context
This PR introduces a Global Workspace DataSource that consolidates
workspace-specific database access through a single TypeORM DataSource
instance with AsyncLocalStorage-based context management instead of N
workspace datasources.

- Created GlobalWorkspaceDataSource extending TypeORM's DataSource to
manage multiple workspaces with entity metadata caching (1-hour TTL)
- Implemented AsyncLocalStorage for workspace context propagation
(WorkspaceContextForStorage) containing workspace ID, metadata,
permissions, and feature flags
- Modified query execution flow to wrap operations in workspace context
via GlobalWorkspaceOrmManager.executeInWorkspaceContext()
- Added schema name to entity schemas for proper multi-tenant database
separation

Next: 
- use the new global workspace datasource everywhere and deprecate
workspace datasource factory
- improve metadata caching using a short TTL to avoid multiple calls to
redis
- Leverage the new WorkspaceContextALS and put it higher in the request
hierarchy to have access to permission, metadata and featureflag
everywhere --- build it manually for commands --- find a way to
propagate it in jobs?
- Remove PG_POOL patch once we have a unique datasource and increase
global datasource pool size

## Implementation
Why ALS:

1. Automatic Per-Request Isolation

With schema-based multi-tenancy, each workspace has its own PostgreSQL
schema
(e.g. workspace_20202020-1c25-4d02-bf25-6aeccf7ea419).
The critical challenge is ensuring that concurrent requests from
different tenants don't interfere with each other.
```typescript
// Request A (Workspace 1) and Request B (Workspace 2) executing concurrently

// Without ALS: Race condition — they'd share the same global state!
// With ALS: Each request has isolated context ✓
```
ALS automatically isolates context per async execution chain, so:
- Request from Tenant A → ALS stores workspaceId: "tenant-a" → Queries
hit workspace_tenant_a schema
- Request from Tenant B → ALS stores workspaceId: "tenant-b" → Queries
hit workspace_tenant_b schema

 No interference, even when executing simultaneously on the same
Node.js event loop.

2. No Manual Context Passing

Before ALS, you'd need to pass workspaceId through every function call:
```typescript
//  Without ALS - Context threading nightmare
getRepository(workspaceId, entity)
  → createEntityManager(workspaceId)
    → getMetadata(workspaceId, target)
      → findInCache(workspaceId, cacheKey)
```
With ALS:
```typescript
//  With ALS - Clean, implicit context
getRepository(entity)          // Reads workspaceId from ALS
  → createEntityManager()      // Reads workspaceId from ALS
    → getMetadata(target)      // Reads workspaceId from ALS
      → findInCache(cacheKey)  // Reads workspaceId from ALS
```
example
```typescript
override findMetadata(target: EntityTarget<ObjectLiteral>): EntityMetadata | undefined {
  const context = getWorkspaceContext(); // 👈 Automatically gets the right workspace!
  const { workspaceId, metadataVersion } = context;
  const cacheKey = `${workspaceId}-${metadataVersion}`;
  // ... returns metadata for THIS workspace's schema
}
```

3. Async Chain Propagation

Node.js operations are heavily async. ALS automatically propagates
context through:

- async/await chains
- Promise chains
- Callbacks

```typescript
executeInWorkspaceContext(workspaceId, async () => {
  await prepareContext();           // Has context ✓
  const results = await run();      // Has context ✓
  await enrichResults();            // Has context ✓

  // Even nested async operations maintain context!
  await Promise.all([
    saveToCache(),   // Has context ✓
    emitEvent(),     // Has context ✓
    logMetrics(),    // Has context ✓
  ]);
});
```
5. Schema-Specific Metadata Caching

The implementation caches entity metadata per workspace + version:
```typescript
// Cache key format: "workspaceId-metadataVersion"
const cacheKey = `${workspaceId}-${metadataVersion}`;
```
Why this matters with schemas:

- Each workspace has different table structures (custom fields, objects)
- EntitySchema includes schema: "workspace_xxx" property
- Each cached metadata points to the correct schema

ALS ensures getWorkspaceContext() returns the right workspaceId,
so you always get the correct schema's metadata from cache.

6. Single DataSource for All Tenants

The key change here:
```typescript
//  Old approach: One DataSource per tenant
const dataSourceTenantA = new DataSource({ schema: 'workspace_a' });
const dataSourceTenantB = new DataSource({ schema: 'workspace_b' });
// Problem: Hundreds of DB connection pools!
```
```typescript
//  New approach: One shared DataSource + ALS context
const globalDataSource = new GlobalWorkspaceDataSource();

// ALS determines which schema to use at runtime
```
When you call:
```typescript
globalDataSource.getRepository('person');
```
It internally does:
```typescript
const context = getWorkspaceContext();  // Gets current tenant from ALS
const metadata = this.findMetadata('person'); // Finds metadata for THIS tenant's schema
// EntityMetadata includes: schema: "workspace_20202020-1c25..."
// TypeORM automatically queries: SELECT * FROM "workspace_20202020-1c25...".person
```
7. Request Lifecycle Example

```typescript
// 1. GraphQL request arrives: "query people { ... }"
// 2. Middleware extracts authContext.workspace.id = "tenant-a"
// 3. Query runner wraps execution in ALS:
executeInWorkspaceContext("tenant-a", async () => {
  // 4. Everything inside has access to workspace context:
  const repo = getRepository('person');      // ALS → tenant-a
  const metadata = getMetadata('person');    // ALS → tenant-a → cache["tenant-a-v5"]

  // 5. TypeORM builds query with correct schema:
  // SELECT * FROM "workspace_tenant_a"."person" WHERE ...

  // 6. Even nested calls work:
  await saveAuditLog();  // ALS → tenant-a → correct audit schema
  await emitWebhook();   // ALS → tenant-a → correct tenant webhook
});
// 7. Request completes, ALS context automatically cleaned up
```

8. Safety & Error Prevention

```typescript
// If you forget to set context:
const context = getWorkspaceContext();
//  Throws: "Workspace context not set..."
// Fails fast rather than querying wrong schema!

// Can't accidentally query wrong tenant:
// Context is immutable within execution scope
```
2025-11-10 17:32:49 +01:00
Félix MalfaitandGitHub 9a80164cf3 Add comprehensive permission guard coverage across GraphQL and REST endpoints (#15739)
This PR enhances our security model by ensuring all GraphQL resolvers
and REST API endpoints have appropriate permission guards.

## Changes

### ESLint Rules
- Enhanced `graphql-resolvers-should-be-guarded` to require permission
guards on all resolvers (Query, Mutation, Subscription), not just
mutations
- Enhanced `rest-api-methods-should-be-guarded` to require permission
guards on all REST endpoints (GET, POST, PUT, PATCH, DELETE), not just
mutating methods
- Both rules now enforce consistent security: authentication guards +
permission guards for all endpoints

### Permission Guards Added

**Public Endpoints** - Added `NoPermissionGuard`:
- Auth-related queries (checkUserExists, findWorkspaceFromInviteHash,
validatePasswordResetToken)
- Billing webhooks (Stripe callbacks)
- SSO callbacks (SAML authentication)
- Workflow webhooks
- Cloudflare webhooks
- Route trigger endpoints
- GraphQL subscriptions
- Current workspace queries
- Geo-map address autocomplete
- View-related read operations (view-field, view-filter, view-group,
view-sort, view-filter-group)

**Settings Permission Guards** - Added `SettingsPermissionGuard`:
- API Keys management: `PermissionFlagType.API_KEYS_AND_WEBHOOKS`
- Webhooks management: `PermissionFlagType.API_KEYS_AND_WEBHOOKS`
- Page Layouts (write operations): `PermissionFlagType.LAYOUTS`
- REST Metadata API: `PermissionFlagType.DATA_MODEL`
- Agent operations: `PermissionFlagType.AI`
- Remote servers: `PermissionFlagType.DATA_MODEL`
- Remote tables: `PermissionFlagType.DATA_MODEL`
- Serverless functions: `PermissionFlagType.WORKFLOWS`

**Custom Permission Guards** - Added `CustomPermissionGuard`:
- REST Core API (permissions checked at query execution layer)
- Timeline calendar events (permission checks in service layer)
- Timeline messaging (permission checks in service layer)
- Search operations (permission checks in service layer)
- View operations (permission checks via dedicated view permission
guards)

### View Permission Guards
- Created dedicated `FindManyViewsPermissionGuard` and
`FindOneViewPermissionGuard` for reading views
- Created `CreateViewPermissionGuard` for view creation with
visibility-based permission checks
- All view child entities (view-field, view-filter, view-sort,
view-group, view-filter-group) use `NoPermissionGuard` for reads
- Write operations on view child entities use dedicated permission
guards that check parent view access

### Page Layout Permissions
- Read operations (GET/Query) now use `NoPermissionGuard` - users can
view layouts without LAYOUTS permission
- Write operations (POST/PATCH/DELETE/Mutation) require
`SettingsPermissionGuard(PermissionFlagType.LAYOUTS)`
- Applied consistently across page-layout, page-layout-tab, and
page-layout-widget endpoints

## Security Model
All endpoints now follow a consistent pattern:
1. **Authentication**: `UserAuthGuard`, `WorkspaceAuthGuard`, or
`PublicEndpointGuard`
2. **Authorization**: One of:
- `SettingsPermissionGuard(PermissionFlagType.XXX)` - for settings/admin
operations
- `CustomPermissionGuard` - when permissions are checked in service/data
layer
   - `NoPermissionGuard` - for public or non-sensitive read operations

The ESLint rules automatically enforce this pattern going forward.

## Stats
- 47 files changed
- 603 insertions, 163 deletions
- 3 new guard files created
2025-11-10 12:17:38 +01:00
Félix MalfaitandGitHub cff17db6cb Enhance role-check system with stricter checks (#15392)
## Overview

This PR strengthens our permission system by introducing more granular
role-based access control across the platform.

## Changes

### New Permissions Added
- **Applications** - Control who can install and manage applications
- **Layouts** - Control who can customize page layouts and UI structure
- **AI** - Control access to AI features and agents
- **Upload File** - Separate permission for file uploads
- **Download File** - Separate permission for file downloads (frontend
visibility)

### Security Enhancements
- Implemented whitelist-based validation for workspace field updates
- Added explicit permission guards to core entity resolvers
- Enhanced ESLint rule to enforce permission checks on all mutations
- Created `CustomPermissionGuard` and `NoPermissionGuard` for better
code documentation

### Affected Components
- Core entity resolvers: webhooks, files, domains, applications,
layouts, postgres credentials
- Workspace update mutations now use whitelist validation
- Settings UI updated with new permission controls

### Developer Experience
- ESLint now catches missing permission guards during development
- Explicit guard markers make permission requirements clear in code
review
- Comprehensive test coverage for new permission logic

## Testing
-  All TypeScript type checks pass
-  ESLint validation passes
-  New permission guards properly enforced
-  Frontend UI displays new permissions correctly

## Migration Notes
Existing workspaces will need to assign the new permissions to roles as
needed. By default, all new permissions are set to `false` for non-admin
roles.
2025-11-07 15:37:17 +01:00
GuillimandGitHub 2274a937bd first step (#15687)
TODO : 

-> for test integreation for this connect 
-> add validation in common API

Fixes https://github.com/twentyhq/core-team-issues/issues/1278
2025-11-07 13:49:57 +00:00
MarieandGitHub 4ce93aee52 Fix user deletion flows (#15614)
**Before**
- any user with workpace_members permission was able to remove a user
from their workspace. This triggered the deletion of workspaceMember +
of userWorkspace, but did not delete the user (even if they had no
workspace left) nor the roleTarget (acts as junction between role and
userWorkspace) which was left with a userWorkspaceId pointing to
nothing. This is because roleTarget points to userWorkspaceId but the
foreign key constraint was not implemented
- any user could delete their own account. This triggered the deletion
of all their workspaceMembers, but not of their userWorkspace nor their
user nor the roleTarget --> we have orphaned userWorkspace, not
technically but product wise - a userWorkspace without a workspaceMember
does not make sense

So the problems are
- we have some roleTargets pointing to non-existing userWorkspaceId
(which caused https://github.com/twentyhq/twenty/issues/14608 )
- we have userWorkspaces that should not exist and that have no
workspaceMember counterpart
- it is not possible for a user to leave a workspace by themselves, they
can only leave all workspaces at once, except if they are being removed
from the workspace by another user

**Now**
- if a user has multiple workspaces, they are given the possibility to
leave one workspace while remaining in the others (we show two buttons:
Leave workspace and Delete account buttons). if a user has just one
workspace, they only see Delete account
- when a user leaves a workspace, we delete their workspaceMember,
userWorkspace and roleTarget. If they don't belong to any other
workspace we also soft-delete their user
- soft-deleted users get hard deleted after 30 days thanks to a cron
- we have two commands to clean the orphans roleTarget and userWorkspace
(TODO: query db to see how many must be run)

**Next**
- once the commands have been run, we can implement and introduce the
foreign key constraint on roleTarget


Fixes https://github.com/twentyhq/twenty/issues/14608
2025-11-06 18:29:12 +00:00
EtienneandGitHub 037663ddcd RichText editor fixes (#15678)
closes https://github.com/twentyhq/twenty/issues/15474
closes https://github.com/twentyhq/twenty/issues/15677
2025-11-06 16:21:57 +01:00
martmullandGitHub abde3c04ac 1630 extensibility twenty cli ability to create edit and delete fields (#15501)
As title

- adds decorators in twenty-sdk
- update twenty-cli load-manifest to it gets @FieldMetadata infos +
testing
- update twenty-server so it CRUD fields properly, using
universalIdentifier
- Fix UI so we can update managed objects records
- move FieldMetadata items from twenty-server to twenty-shared
2025-11-05 17:50:06 +01:00
003b04e9ae fix: removeuseMergeRecordRelatationship and simplify dry run response (#15486)
## Description

- This PR addresses issue
https://github.com/twentyhq/twenty/issues/15201
- Removed
[useMergeRecordRelationships.ts](https://github.com/twentyhq/twenty/compare/main...harshit078:fix-merge-frontend?expand=1#diff-5d0366f814ff0c0e8961f40120690206a7acd3f7b66135cfb8c4ad8d3f6bc681)
- now backend returns populated result and frontend just shows it
without refetching relations.
- to be merged after PR- https://github.com/twentyhq/twenty/pull/15484


## Visual Appearance

 


https://github.com/user-attachments/assets/09d7548a-74a9-4742-af49-e98b4174f68c

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
2025-11-05 10:33:24 +01:00
9e1c854b27 fix: updated dryrun to return correct populated record (#15484)
## Description

- This PR approaches to solve
https://github.com/twentyhq/twenty/issues/15201
- updated `createDryRunResponse` to return fully populated merged record
- this way the frontend can render the populated data it as-is without
recomputing relations
- Dry-run now uses the same nested-relations population path as the
non-dryRun flow

---------

Co-authored-by: Etienne <45695613+etiennejouan@users.noreply.github.com>
2025-11-03 13:50:14 +01:00
afeb505eed [Breaking Change] Implement reliable date picker utils to handle all timezone combinations (#15377)
This PR implements the necessary tools to have `react-datepicker`
calendar and our date picker components work reliably no matter the
timezone difference between the user execution environment and the user
application timezone.

Fixes https://github.com/twentyhq/core-team-issues/issues/1781

This PR won't cover everything needed to have Twenty handle timezone
properly, here is the follow-up issue :
https://github.com/twentyhq/core-team-issues/issues/1807

# Features in this PR

This PR brings a lot of features that have to be merged together.

- DATE field type is now handled as string only, because it shouldn't
involve timezone nor the JS Date object at all, since it is a day like a
birthday date, and not an absolute point in time.
- DATE_TIME field wasn't properly handled when the user settings
timezone was different from the system one
- A timezone abbreviation suffix has been added to most DATE_TIME
display component, only when the timezone is different from the system
one in the settings.
- A lot of bugs, small features and improvements have been made here :
https://github.com/twentyhq/core-team-issues/issues/1781

# Handling of timezones

## Essential concepts

This topic is so complex and easy to misunderstand that it is necessary
to define the precise terms and concepts first. It resembles character
encoding and should be treated with the same care.

- Wall-clock time : the time expressed in the timezone of a user, it is
distinct from the absolute point in time it points to, much like a
pointer being a different value than the value that it points to.
- Absolute time : a point in time, regardless of the timezone, it is an
objective point in time, of course it has to be expressed in a given
timezone, because we have to talk about when it is located in time
between humans, but it is in fact distinct from any wall clock time, it
exists in itself without any clock running on earth. However, by
convention the low-level way to store an absolute point in time is in
UTC, which is a timezone, because there is no way to store an absolute
point in time without a referential, much like a point in space cannot
be stored without a referential.
- DST : Daylight Save Time, makes the timezone shift in a specific
period every year in a given timezone, to make better use of longer days
for various reasons, not all timezones have DST. DST can be 1 hour or 30
min, 45 min, which makes computation difficult.
- UTC : It is NOT an “absolute timezone”, it is the wall-clock time at
0° longitude without DST, which is an arbitrary and shared human
convention. UTC is often used as the standard reference wall-clock time
for talking about absolute point in time without having to do timezone
and DST arithmetic. PostgreSQL stores everything in UTC by convention,
but outputs everything in the server’s SESSION TIMEZONE.

## How should an absolute point in time be stored ?

Since an absolute point in time is essentially distinct from its
timezone it could be stored in an absolute way, but in practice it is
impossible to store an absolute point in time without a referential. We
have to say that a rocket launched at X given time, in UTC, EST, CET,
etc. And of course, someone in China will say that it launched at 10:30,
while in San Francisco it will have launched at 19:30, but it is THE
SAME absolute point in time.

Let’s take a related example in computer science with character
encoding. If a text is stored without the associated encoding table, the
correct meaning associated to the bits stored in memory can be lost
forever. It can become impossible for a program to guess what encoding
table should be used for a given text stored as bits, thus the glitches
that appeared a lot back in the early days of internet and document
processing.

The same can happen with date time storing, if we don’t have the
timezone associated with the absolute point in time, the information of
when it absolutely happened is lost.

It is NOT necessary to store an absolute point in time in UTC, it is
more of a standard and practical wall-clock time to be associated with
an absolute point in time. But an absolute point in time MUST be store
with a timezone, with its time referential, otherwise the information of
when it absolutely happened is lost.

For example, it is easier to pass around a date as a string in UTC, like
`2024-01-02T00:00:00Z` because it allows front-end and back-end code to
“talk” in the same standard and DST-free wall-clock time, BUT it is not
necessary. Because we have date libraries that operate on the standard
ISO timezone tables, we can talk in different timezone and let the
libraries handle the conversion internally.

It is false to say that UTC is an absolute timezone or an absolute point
in time, it is just the standard, conventional time referential, because
one can perfectly store every absolute points in time in UTC+10 with a
complex DST table and have the exactly correct absolute points in time,
without any loss of information, without having any UTC+0 dates
involved.

Thus storing an absolute point in time without a timezone associated,
for example with `timestamp` PostgreSQL data type, is equivalent to
storing a wall-clock time and then throwing away voluntarily the
information that allows to know when it absolutely happened, which is a
voluntary data-loss if the code that stores and retrieves those
wall-clock points in time don’t store the associated timezone somewhere.

This is why we use `timestamptz` type in PostgreSQL, so that we make
sure that the correct absolute point in time is stored at the exact time
we send it to PostgreSQL server, no matter the front-end, back-end and
SQL server's timezone differences.

## The JavaScript Date object

The native JavaScript Date object is now officially considered legacy
([source](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date)),
the Date object stores an absolute point in time BUT it forces the
storage to use its execution environment timezone, and one CANNOT modify
this timezone, this is a legacy behavior.

To obtain the desired result and store an absolute point in time with an
arbitrary timezone there are several options :

- The new Temporal API that is the successor of the legacy Date object.
- Moment / Luxon / @date-fns/tz that expose objects that allow to use
any timezone to store an absolute point in time.

## How PostgreSQL stores absolute point in times

PostgreSQL stores absolute points in time internally in UTC
([source](https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME-INPUT-TIME-STAMPS)),
but the output date is expressed in the server’s session timezone
([source](https://www.postgresql.org/docs/current/sql-set.html)) which
can be different from UTC.

Example with the object companies in Twenty seed database, on a local
instance, with a new “datetime” custom column :

<img width="374" height="554" alt="image"
src="https://github.com/user-attachments/assets/4394cb43-d97e-4479-801d-ca068f800e39"
/>

<img width="516" height="524" alt="image"
src="https://github.com/user-attachments/assets/b652f36a-d2e2-47a4-8950-647ca688cbbd"
/>

## Why can’t I just use the JavaScript native Date object with some
manual logic ?

Because the JavaScript Date object does not allow to change its internal
timezone, the libraries that are based on it will behave on the
execution environment timezone, thus leading to bugs that appear only on
the computers of users in a timezone but not for other in another
timezone.

In our case the `react-datepicker` library forces to use the `Date`
object, thus forcing the calendar to behave in the execution environment
system timezone, which causes a lot of problems when we decide to
display the Twenty application DATE_TIME values in another timezone than
the user system one, the bugs that appear will be of the off-by-one date
class, for example clicking on 23 will select 24, thus creating an
unreliable feature for some system / application timezone combinations.

A solution could be to manually compute the difference of minutes
between the application user and the system timezones, but that’s not
reliable because of DST which makes this computation unreliable when DST
are applied at different period of the year for the two timezones.

## Why can’t I compute the timezone difference manually ?

Because of DST, the work to compute the timezone difference reliably,
not just for the usual happy path, is equivalent to developing the
internal mechanism of a date timezone library, which is equivalent to
use a library that handles timezones.

## Using `@date-fns/tz` to solve this problem

We could have used `luxon` but it has a heavy bundle size, so instead we
rely here on `@date-fns/tz` (~1kB) which gives us a `TZDate` object that
allows to use any given timezone to store an absolute point-in-time.

The solution here is to trick `react-datepicker` by shifting a Date
object by the difference of timezone between the user application
timezone and the system timezone.

Let’s take a concerte example.

System timezone : Midway,  ⇒ UTC-11:00, has no DST.

User application timezone : Auckland, NZ ⇒ UTC+13:00, has a DST.

We’ll take the NZ daylight time, so that will make a timezone difference
of 24 hours !

Let’s take an error-prone date : `2025-01-01T00:00:00` . This date is
usually a good test-case because it can generate three classes of bugs :
off-by-one day bugs, off-by-one month bugs and off-by-one year bugs, at
the same time.

Here is the absolute point in time we take expressed in the different
wall-clock time points we manipulate


Case | In system timezone ⇒ UTC-11 | In UTC | In user application
timezone ⇒ UTC+13
-- | -- | -- | --
Original date | `2024-12-31T00:00:00-11:00` | `2024-12-31T11:00:00Z` |
`2025-01-01T00:00:00+13:00`
Date shifted for react-datepicker | `2025-01-01T00:00:00-11:00` |
`2025-01-01T11:00:00Z` | `2025-01-02T00:00:00+13:00`

We can see with this table that we have the number part of the date that
is the same (`2025-01-01T00:00:00`) but with a different timezone to
“trick” `react-datepicker` and have it display the correct day in its
calendar.

You can find the code in the hooks
`useTurnPointInTimeIntoReactDatePickerShiftedDate` and
`useTurnReactDatePickerShiftedDateBackIntoPointInTime` that contain the
logic that produces the above table internally.

## Miscellaneous

Removed FormDateFieldInput and FormDateTimeFieldInput stories as they do
not behave the same depending of the execution environment and it would
be easier to put them back after having refactored FormDateFieldInput
and FormDateTimeFieldInput

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-11-01 18:31:24 +01:00
MarieandGitHub bcdc07273e Fix orderBy, groupBy, and orderByForRecords on foreignKey (#15480)
For the variables orderBy (for findMany and groupBy), groupBy (for
groupBy) and orderByForRecords (for groupBy), we were wrongfully adding
the foreign key field (eg: pointOfContact) by its joinColumnName (eg:
pointOfContactId) as the variable key (eg. `orderBy: { pointOfContactId:
"AscNullsFirst" } }`. That broke because then this key is used to
identify the field in the parsers.

This went unnoticed because this order / group option is not very
interesting as it is limited to the id for now, but it s still better to
have it work than crash!

before (on findMany)
<img width="1841" height="674" alt="flawn_order_by_pocId"
src="https://github.com/user-attachments/assets/3ccd7604-041d-4b7e-8b6a-c1d268440a45"
/>

after (on findMany)
<img width="1182" height="805" alt="image"
src="https://github.com/user-attachments/assets/e9253683-bcbe-429e-bbef-e334c7cc76af"
/>
2025-10-31 15:13:49 +01:00
EtienneandGitHub 2461296158 Common - fixes (#15463)
Fixes https://github.com/twentyhq/twenty/issues/15435
Fixes https://github.com/twentyhq/private-issues/issues/338
Fixes https://github.com/twentyhq/twenty/issues/15457
2025-10-30 14:29:57 +01:00
MarieandGitHub 1ad8c05fbc groupBy fix + typeMapper fix (#15433)
groupBy fix: when a group's dimension value is NULL, we need to adapt
the raw sql (stage: NULL -> stage IS NULL)

typeMapper fix: a graphql type should be made non-nullable if was
indicated so + does not have a default value. our check on not having a
default value was limited to having a null defaultValue instead of
having a null or undefined defaultValue. This is a breaking change, but
all the queries that were providing a null value for these args were not
functioning anyway, and luckily in the FE we declared all queries adding
a `!` already.
2025-10-30 10:14:56 +00:00
MarieandGitHub 473efb5dc5 [groupBy] Order by within records (#15404)
Closes https://github.com/twentyhq/core-team-issues/issues/1727

<img width="1415" height="625" alt="image"
src="https://github.com/user-attachments/assets/cd132582-4a95-463c-8946-87b77670e023"
/>
2025-10-29 14:49:10 +01:00
EtienneandGitHub 0849483d2e Common - Remove feature flag (#15371)
To be done in an other PR, move graphql-query-runner handlers in
common-api-query-runner folder (+ renaming + typing improvments)


Fixes : 
- totalCount on findMany (Rest)
- getAllSelectedFields did not handle composite field (Rest)
- issue with endCursor (Rest)
- args processing for updateOne and createOne (Rest + Gql)
- fix findDuplicates (Gql)
2025-10-29 10:01:52 +00:00
EtienneandGitHub 5ae7674b9d Common - Throttle update + Metrics (#15413) 2025-10-28 19:15:18 +01:00
EtienneandGitHub 7df43d8fe8 Common - Update many fix (#15412) 2025-10-28 18:45:08 +01:00
1590751131 Set formatted before instead of after (#15401)
Soft deletion events use before in event properties. But we set the
after instead, that only contains a few data.

Fixes https://github.com/twentyhq/twenty/issues/15120

Debugging raised an additional issue. UpdateMany, RestoreMany and
SoftDeleteMany were broken for custom objects. The `where` clause was
using "_objectName.id" instead of "objectName.id" during select. That's
why we need to use different where clause between the select of the
existing value and the actual mutation.

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-28 17:13:39 +01:00
EtienneandGitHub ee6b86b1a8 Common - many fixes (#15395) 2025-10-28 13:33:41 +01:00
MarieandGitHub 0e7d18546d [groupBy] Handle relations on groupBy with records (#15379)
Following PR https://github.com/twentyhq/twenty/pull/15307
2025-10-27 15:14:02 +00:00
EtienneandGitHub 93ab1c4118 Throttling in common api (#15338) 2025-10-24 17:32:51 +02:00
MarieandGitHub d7bda9576f [groupBy] Load record within groups (without relations) (#15307)
First step of https://github.com/twentyhq/core-team-issues/issues/1726.
I will handle relations in another PR.
Another ticket is planned to allow for sorting among the records.

When querying records we have set the number of maximum groups to 50,
and of maximum records par group to 10.

<img width="1283" height="798" alt="image"
src="https://github.com/user-attachments/assets/7ffc9805-d715-4017-8030-4f3521e6f741"
/>
2025-10-24 15:30:57 +00:00
e37b8b61ba message channel change 2 (#15269)
Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-24 16:39:22 +02:00
EtienneandGitHub 10f4f1592f Common - Update one/many, restore one/many, findDuplicates & mergeMany (#15279)
closes https://github.com/twentyhq/core-team-issues/issues/1739
2025-10-24 12:41:17 +00:00
MarieandGitHub 389433a6b9 Remove groupBy feature flag (#15281) 2025-10-23 13:35:16 +02:00
EtienneandGitHub dc05f18310 Common api - Destroy and delete queries (#15177)
Done ⬇️ 
Gql : move delete and destroy logic to common
Rest : 
- rename 'delete' handler to 'destroy'
- create soft delete handlers (named 'delete') : one and many
- create destroy many handler
- update doc

--> Rest api gains NEW soft delete one/many + destroy many capabilities

closes : https://github.com/twentyhq/core-team-issues/issues/1579
2025-10-22 16:35:17 +02:00
Raphaël BosiandGitHub 1c27206b41 Use aggregate operations in the widget configuration instead of extended aggregate operations (#15248)
- Use aggregate operations in the widget configuration instead of
extended aggregate operations
- Use aggregate operation from generated graphql in the frontend
2025-10-22 14:19:02 +02:00
226cc9eb74 [Dashboards] - Min Max range on secondary axis bar charts (#15118)
video QA


https://github.com/user-attachments/assets/70c37188-2398-43de-bbf6-5882bb79940a

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-22 10:51:01 +02:00
c5564d9bd0 [BREAKING CHANGE] refactor: Add Entity suffix to TypeORM entity classes (#15239)
## Summary

This PR refactors all TypeORM entity classes in the Twenty codebase to
include an 'Entity' suffix (e.g., User → UserEntity, Workspace →
WorkspaceEntity) to improve code clarity and follow TypeORM naming
conventions.

## Changes

### Entity Renaming
-  Renamed **57 core TypeORM entities** with 'Entity' suffix
-  Updated all related imports, decorators, and type references
-  Fixed Repository<T>, @InjectRepository(), and
TypeOrmModule.forFeature() patterns
-  Fixed @ManyToOne/@OneToMany/@OneToOne decorator references

### Backward Compatibility
-  Preserved GraphQL schema names using @ObjectType('OriginalName')
decorators
-  **No breaking changes** to GraphQL API
-  **No database migrations** required
-  File names unchanged (user.entity.ts remains as-is)

### Code Quality
-  Fixed **497 TypeScript errors** (82% reduction from 606 to 109)
-  **All linter checks passing**
-  Improved type safety across the codebase

## Entities Renamed

```
User → UserEntity
Workspace → WorkspaceEntity
ApiKey → ApiKeyEntity
AppToken → AppTokenEntity
UserWorkspace → UserWorkspaceEntity
Webhook → WebhookEntity
FeatureFlag → FeatureFlagEntity
ApprovedAccessDomain → ApprovedAccessDomainEntity
TwoFactorAuthenticationMethod → TwoFactorAuthenticationMethodEntity
WorkspaceSSOIdentityProvider → WorkspaceSSOIdentityProviderEntity
EmailingDomain → EmailingDomainEntity
KeyValuePair → KeyValuePairEntity
PublicDomain → PublicDomainEntity
PostgresCredentials → PostgresCredentialsEntity
...and 43 more entities
```

## Impact

### Files Changed
- **400 files** modified
- **2,575 insertions**, **2,191 deletions**

### Progress
-  **82% complete** (497/606 errors fixed)
- ⚠️ **109 TypeScript errors** remain (18% of original)

## Remaining Work

The 109 remaining TypeScript errors are primarily:

1. **Function signature mismatches** (~15 errors) - Test mocks with
incorrect parameter counts
2. **Entity type mismatches** (~25 errors) - UserEntity vs
UserWorkspaceEntity confusion
3. **Pre-existing issues** (~50 errors) - Null safety and DTO
compatibility (unrelated to refactoring)
4. **Import type issues** (~10 errors) - Entities imported with 'import
type' but used as values
5. **Minor decorator issues** (~9 errors) - onDelete property
configurations

These can be addressed in follow-up PRs without blocking this
refactoring.

## Testing Checklist

- [x] Linter passing
- [ ] Unit tests should be run (CI will verify)
- [ ] Integration tests should be run (CI will verify)
- [ ] Manual testing recommended for critical user flows

## Breaking Changes

**None** - This is a pure refactoring with full backward compatibility:
- GraphQL API unchanged (uses original entity names)
- Database schema unchanged
- External APIs unchanged

## Notes

- Created comprehensive `REFACTORING_STATUS.md` documenting the entire
process
- All temporary scripts have been cleaned up
- Branch: `refactor/add-entity-suffix-to-typeorm-entities`

## Reviewers

Please review especially:
- Entity renaming patterns
- GraphQL backward compatibility
- Any areas where entity types are confused (UserEntity vs
UserWorkspaceEntity)

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-22 09:55:20 +02:00
26e432d8e2 fix: Add reserved subdomains constant and update validation on generateSubdomain (#15217)
## Description

Fixes #15160 

- Moved the reserved subdomains to a separate shared constant file:
`packages/twenty-server/src/engine/core-modules/workspace/constants/reserved-subdomains.constant.ts`
- Updated the validation while generating subdomain to check if the
extracted subdomain (from email or display name) is reserved
- When a reserved subdomain is detected, the server will automatically
fall back to a random subdomain

---------

Co-authored-by: Naineel Soyantar <naineelsoyantar@gmail.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-21 16:53:48 +02:00
neo773andGitHub 139b6dd29b Preserve all emails, phones, and links when merging records (#15224) 2025-10-21 16:16:22 +02:00
Félix MalfaitandGitHub f7421c5fc0 Add queue management dashboard (#15202)
Adds a comprehensive queue management interface to the admin panel for
viewing and managing background jobs.

**Features:**
- Queue detail pages showing paginated job lists (50 per page)
- Filter jobs by state: completed, failed, active, waiting, delayed,
paused
- Checkbox selection with bulk actions (delete jobs, retry failed jobs)
- Per-job dropdown menu for individual retry/delete
- Expandable rows showing error messages, stack traces, and job data
- Relative timestamps with hover tooltips
- Display attempt counts on failed jobs
- Dynamic retention policy info from backend

**Changes:**
- Backend: New AdminPanelQueueService with GraphQL endpoints for job
listing, retry, and delete
- Frontend: Queue detail page with QueueJobsTable component
- Updated retention policy: completed jobs kept 4 hours, failed jobs
kept 7 days (max 1000 each)
- Added JobState enum for type safety

<img width="634" height="696" alt="Screenshot_2025-10-20_at_11 45 25"
src="https://github.com/user-attachments/assets/c67bcd27-26cf-47f5-9575-3cd5684d006b"
/>
<img width="484" height="680" alt="Screenshot_2025-10-20_at_11 45 14"
src="https://github.com/user-attachments/assets/68725cc6-b3ec-4098-99ca-f9a717d6f8f1"
/>
<img width="490" height="643" alt="Screenshot_2025-10-20_at_11 45 05"
src="https://github.com/user-attachments/assets/b68a5809-33ff-4452-b48b-741aff7f1dd6"
/>



<img width="685" height="662" alt="Screenshot 2025-10-20 at 13 15 01"
src="https://github.com/user-attachments/assets/eeb5207b-de5c-4b18-bdde-392892053dab"
/>
2025-10-21 16:02:50 +02:00
793dc3d6fc Remove dependency on lodash.pick. (#15213)
Fixes [Dependabot Alert
85](https://github.com/twentyhq/twenty/security/dependabot/85) -
prototype pollution in lodash.

Added a shared pick helper (with unit tests) in twenty-shared and
refactored front-end/server code to import { pick } from the shared
barrel instead of lodash.pick.

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: martmull <martmull@hotmail.fr>
2025-10-21 08:19:49 +00:00
MarieandGitHub 2903e45bef Fix Group by actor field (#15215) 2025-10-20 19:07:35 +02:00
EtienneandGitHub dea08a3a41 Common api - Group by query (#15108)
closes https://github.com/twentyhq/core-team-issues/issues/1626
2025-10-20 17:51:01 +02:00
3bec43696f feat: multi role permission intersection (#15150)
Implements permission intersection (AND logic) to prevent permission
escalation when agents act on behalf of users.

### Changes:
- **Permission Intersection**: Operations requiring both user AND agent
permissions
- **RoleContext Type**: Unified type supporting single `roleId` or
multiple `roleIds` for intersection
- **CRUD Services**: Updated to accept `roleContext` for granular
permission control
- **Agent Integration**: Chat agents now use user + agent role
intersection for all operations
- **ORM Layer**: Enhanced `getRepository` to support multi-role
permission checks

### Related:
- Part 2 of ["Acting on behalf of user" concept
PR](https://github.com/twentyhq/twenty/pull/15103)

[Closes #1661](https://github.com/twentyhq/core-team-issues/issues/1661)

---------

Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
2025-10-19 09:30:05 +02:00
9cd957844e Log invalid uuids (#15099)
I had an issue with invalid UUIDs, and I think it would be easier to
find the offending one if the UUID were included in the error message.

This way a database dump can be easily searched.

---------

Co-authored-by: prastoin <paul@twenty.com>
2025-10-17 14:33:33 +02:00
246543912d [Dashboards] - Omit zero values (#15112)
video QA


https://github.com/user-attachments/assets/6391afdd-0714-4202-8db0-f67388fb7582

---------

Co-authored-by: Marie <51697796+ijreilly@users.noreply.github.com>
2025-10-16 15:00:28 +05:30
MarieandGitHub 9aa44ef1e8 Fix objectRecord imports (#15113) 2025-10-15 15:33:08 +02:00
EtienneandGitHub 4ae299973b Common Api - createOne/Many (#15083)
closes https://github.com/twentyhq/core-team-issues/issues/1578
2025-10-15 12:13:07 +00:00
f65783f900 [GroupBy] Allow sorting in bar chart (#15097)
Closes https://github.com/twentyhq/core-team-issues/issues/1628

From a technical perspective, we can add more ordering options, such as
the ability to combine two sorts on the X axis, e.g. sort by Close date
ASC and then by Sum ASC, which will sort groups that have the same close
date between themselves depending on their sum ASC. @Bonapara could you
provide design if you want this to be implemented (quite short on our
hand i think - maybe in V2 though)?


https://github.com/user-attachments/assets/6ef21fe1-9d8f-43c0-bfa2-f6fc6341cacf

---------

Co-authored-by: ehconitin <nitinkoche03@gmail.com>
2025-10-15 17:41:14 +05:30
EtienneandGitHub fc7304bc5a Common api - findMany query (#15004)
Done in this PR : 
- simplify rest input parsing by removing metadata validation (for
filter and orderBy - to add in common) (1st commit)
- simplify result getter handlers signature (array of objectRecord only
for input)


closes https://github.com/twentyhq/core-team-issues/issues/1614
closes https://github.com/twentyhq/core-team-issues/issues/1615
closes https://github.com/twentyhq/core-team-issues/issues/1616
2025-10-13 12:48:50 +00:00
MarieandGitHub 4c7e8444cf Inter-groups orderBy on groupByDimension values (#15024)
Closes https://github.com/twentyhq/core-team-issues/issues/1563.

We now have two inter-group orderBy criteria: one on the aggregate
values, the other on the dimension values.
Ex: I am grouping companies by city and querying the average number of
employees for each group. I can either order the groups by the city
name, or by the average number of employees, or by one then the other.
I could actually also order groups by an aggregated value that I did not
ask for (ex: average ARR), but I cannot order groups by a field value I
did not group records by (ex: country).
[See discord
discussion](https://discord.com/channels/1130383047699738754/1425438213555753050/1425438223097921659)

An example of query variables: 
```
{
  "groupBy": [
    {
      "createdAt": {
        "granularity": "QUARTER_OF_THE_YEAR"
      }
    },
    {"city": true}
  ],
  "orderBy": [
    { 
      "aggregate": {
        "avgEmployees": "DescNullsLast"
      }
    },
    { 
      "aggregate": {
        "percentyEmptyEmployees": "DescNullsLast"
      }
    },
    {
      "city": "AscNullsLast"
    },
    {
      "createdAt": {
        "orderBy": "AscNullsLast",
        "granularity": "QUARTER_OF_THE_YEAR",
      }
    }
  ]
}
```

The aggregate orderBy criteria had already been implemented, but I
updated the implementation to add an "aggregate" key to prefix them, in
order to avoid confusion between the two + for the schema generation not
to break (otherwise we would have an issue when generating
OrderByWithGroupByInput if a user has created a field that has the same
name as an aggregate field, such as avgEmployees).
2025-10-13 09:51:42 +00:00
560e24fa4b Update unique fields on standard field - include soft deleted records (#14562)
Done : 
- migrate standard unique fields (add : -old)
- deactivate defaultWhereClause
- restore soft deleted if upserted (same for contact creation in mailing
sync)

fixes https://github.com/twentyhq/twenty/issues/14443

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-11 15:50:29 +02:00
c681fb7fb6 1658 post mortem 0710 send batch events in webhook (#15022)
Use concurrent calls

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-11 12:48:40 +02:00
68c86871dd 🦣🦣🦣 Table virtualization (#14743)
This big PR implements table virtualization with an offset paging,
allowing a way more fluid UX.

It is a v1 that should be improved in the future with partial data
loading and optimization of the browser display performance of a row.

But with this PR we have the solid enough technical foundation, both
frontend and backend, to get to a smooth table UX.

Fixes and improvements after first successful round of development
(needed to have main clean) :

- [x] Delete should refresh virtualized portion only and reset all table
- [x] Fix add new : top and bottom
- [x] Table empty shouldn’t show when first loading
- [x] Fix d&d
- [x]  Fix sorts
- [x] Fix drag when scrolling after a full virtual page (it throws an
error)
- [x] Si update mais qu’on a un sort ou filter, alors il faut trigger le
refresh
- [x]  Reset scroll position between tables
- [x]  Reset scroll shadows between tables
- [x] Setup d&n for virtual list :
https://github.com/hello-pangea/dnd/blob/main/docs/patterns/virtual-lists.md
- [x]  Full table re-render when entering edit mode
- [x] Clean code and prepare for merge

Fixes https://github.com/twentyhq/core-team-issues/issues/1613 that
contains other bugs to be fixed before merge

---------

Co-authored-by: Charles Bochet <charles@twenty.com>
2025-10-11 12:08:10 +02:00
Félix MalfaitandGitHub e577c2d746 Update user friendly errors for translations (#15000)
Force msg typing instead of string for user friendly errors
2025-10-09 18:17:32 +02:00
Paul RastoinandGitHub 59fbe35a8c Move view in metadata-modules/ and create atomic folder + module for each view entity (#14990)
# Introduction
Preparing view-filter and view-group introduction in v2 core engine
Moving view from `core-modules` to `metadata-modules`

## What happened

### Created dedicated modules for each view entity:
- ViewFieldModule
- ViewFilterModule
- ViewFilterGroupModule
- ViewGroupModule
- ViewSortModule

### Each module is now completely independent with its own:
- Controller
- Resolver
- Service
- Entity

### Created dedicated abstraction metadata module folder for:
- flat-view-field
- flat-view

### Dependencies 
- Eleminated circular dep on ViewModule to all others ones
- Granular import not importing the whole viewModule anymore everywhere


close https://github.com/twentyhq/core-team-issues/issues/1703
2025-10-09 15:18:15 +02:00
875dc1a6b8 Connect the bar chart to the group by resolver (#14885)
Closes https://github.com/twentyhq/core-team-issues/issues/1535

Video QA:


https://github.com/user-attachments/assets/153fa3f1-08d9-4952-9456-635c602e1f57



https://github.com/user-attachments/assets/d3111afb-4c84-4f57-8a56-5cf666748c86

There are still some formatting and design issues (the labels which are
on top of each other and the values which should be formatted) that will
be fixed in a future PR

---------

Co-authored-by: ehconitin <nitinkoche03@gmail.com>
2025-10-09 14:54:33 +02:00