In this previous work aiming at eliminating the creation of viewGroups
being separated from the views, I removed the creation of viewGroups
from the FE at view creation, but forgot to do it at view update when
the user chooses "change Layout" option (from Table/Kanban/Calendar to
another).
In this PR
- I removed creation, deletion and destroy of viewGroups from
usePersistViewGroupRecords. I left update because it useful for
visibility and position
- since usePersistViewGroups record was also handling optimistic, I
moved optimistic logic to a new hook useViewsSideEffectsOnViewGroups. I
realized that so far we optimistic is only useful for creation. For the
update though we do need to compute the view groups that will be created
to be able to call loadRecordIndexState, so I created a function for
that, but I did not seek to perform real optimistic, with writing and
deleting groups from the cache as it was a bit heavier and not in use
for now and I want to merge this asap as it fixes the creation of
viewGroups.
In this PR:
- add more logs to troubleshoot issues in production
- use messageChannelSyncStatus in messageSync service instead of making
a manual update
There are three identified scenarios.
## First scenario
The user uses a desktop. They click on a company, which is opened in the
side panel. They click on a tab, like "Tasks". The user chooses to open
the record in fullscreen. The "Tasks" tab is opened by default in the
fullscreen record page.
If the user refreshes the page, the default tab is shown: "Timeline".
**This was the behavior on the show pages, and I kept it.** If we wanted
to show the "Tasks" tab here, we would have to put the id of the "Tasks"
tab in the URL hash when we open the record in fullscreen and an active
tab id is already set.
https://github.com/user-attachments/assets/205776ac-9ad7-4e79-af9a-6b44360a5d77
## Second scenario
The user uses a desktop or a mobile device and interacts with a company
record. They click on a tab. If they refresh, the selected tab will be
displayed by default.
### Desktop
https://github.com/user-attachments/assets/d4e1908b-a591-42f6-bb82-5aec592e2778
### Mobile
https://github.com/user-attachments/assets/7bab73cc-a2ff-4ce8-9bc6-325efef2d500
## Third scenario
The user uses a desktop. They click on a dashboard, which is opened in
fullscreen mode by default. They select tabs. If they refresh, the last
selected tab is opened by default.
When the user turns edit mode on, **the url hash is removed**. The last
selected tab remains selected. If the user selects another tab, the url
hash isn't set, but the newly selected tab is displayed correctly. If
the user saves their changes, the selected tab remains selected.
However, if they refresh the page, the default tab replaces the
previously selected tab.
Not setting the url hash when editing a page layout simplifies the code.
I'm okay with this tradeoff as the feature is primarily meant to let
users share specific tabs of their records.
**This behavior will also be used for record page layout once edit mode
is supported.**
https://github.com/user-attachments/assets/869657a1-6895-4ade-816c-972c77aaab9e
Closes https://github.com/twentyhq/core-team-issues/issues/1784
First PR to implement application tokens
- add new application role in twenty-server
- move duplicated constants and types to twenty-shared
- will add role configuration utils into twenty-sdk in another PR
## Summary
This PR makes the Clickhouse audit logging in the impersonation flow
**non-blocking** (fire-and-forget).
## Problem
When the Clickhouse server is unavailable or slow, the impersonation
feature becomes blocked because all audit logging calls use `await`,
causing the entire impersonation flow to hang.
## Solution
Remove the `await` keyword from all audit logging calls in the
impersonation flow. The `ClickHouseService.insert()` method already
handles errors internally (catches and logs them), so there's no risk of
unhandled promise rejections.
## Changes
- **`impersonation.service.ts`**: 4 audit calls made non-blocking
- **`auth.resolver.ts`**: 5 audit calls made non-blocking
- **`auth.service.ts`**: 2 audit calls made non-blocking
## Testing
- Impersonation flow continues to work when Clickhouse is available
- Impersonation flow no longer blocks when Clickhouse is unavailable
# Introduction
Making columns nullable instead of required + metadata on the fly
migration in typeorm migration that could affect other breaking change
migrations to be run
## Summary
Consolidates the AI tool provider architecture by creating a single
`ToolProviderService` as the entry point for all tool generation. This
removes multiple intermediate services and simplifies the codebase.
## Changes
### New Architecture
- **`ToolProviderService`**: Single service for all tool generation
with:
- `getTools(spec)` - Get tools by category with permissions
- `getToolByType(type)` - Get specific tool for workflow execution
- **`ToolCategory` enum**: Declarative specification of tool types:
- `DATABASE_CRUD` - Record CRUD operations
- `ACTION` - HTTP requests, email sending, article search
- `WORKFLOW` - Workflow management tools
- `METADATA` - Object/field metadata tools
- `NATIVE_MODEL` - Model-specific tools (e.g., web search)
- **`ToolSpecification` type**: Clean API for requesting tools with
permissions
### Removed
- `AiToolsModule` - No longer needed
- `ToolService` - Logic inlined into ToolProviderService
- `ToolAdapterService` - Logic inlined into ToolProviderService
- `ToolRegistryService` - Logic inlined into ToolProviderService
### Updated
- All consumers (agents, chat, MCP, workflows) now use
`ToolProviderService`
- Test files updated accordingly
## Stats
- **547 insertions, 1146 deletions** (net ~600 lines removed)
- 4 services deleted
- 1 module deleted
## Testing
- [x] Typecheck passes
- [x] Lint passes
# Introduction
Refactored the api `validateBuildAndRunWorkspaceMigration` to be require
less configuration but to infer required args dynamically depending on
provided metadata maps to compare
## `inferDeletionFromMissingEntities`
Is not dynamically computed avoiding any miss configuration issue and
any missleading devxp
## Maps computation
Making only one call to redis to build both dependency and to be
compared entity maps. It does not matter to avoid passing a about to
compared flat entity maps to could also be a depedency, it's handled
directly in the builder setup optimistic cache logic
Please note that the flat maps used for the service input transpilation
might differ from the one that we will dynamically compute and inject in
the builder. Leading to do 2 redis calls but also race condition prone
validation error
We prefer that this occurs at the builder rather than at the runner
level as the pg instance is not cache and reflect the real state of a
given workspace
In a nutshell, there's a possible race condition between cache
invalidation and computation in both service input transpilers and
builder but we're totally ok with that
Should be merged once https://github.com/twentyhq/twenty/pull/16206 has
been released + command run to prod
In this PR
- Remove usage of viewGroup.fieldMetadataId, both in BE and FE states.
- But we still need to properly populate it until we fully remove
viewGroup.fieldMetadataId from db and ORM entity (upcoming 3rd PR out of
3). fieldMetadataId was removed from CoreViewGroup type and
CreateViewGroupInput and is determined BE-side based on the associated
view's mainGroupByFieldMetadataId. **I expect this means a downtime on
viewGroup creation, until both FE and BE are deployed and cache is
flushed.** This seems acceptable to me as it only regards viewGroup
creation.
- this information is replaced by view.mainGroupByFieldMetadataID
- Handle view group creation, update and deletion in the BE as a
side-effect of a view creation, update or deletion. Optimistic effects
are still used
- Add validation at view creation or update regarding
mainGroupByFieldMetadata
Left to do in 3rd PR
- Remove viewGroup.fieldMetadataId from db and ORM entity
- Restore feature allowing to update an existing grouped view's group by
field (already OK on BE side but need to rebuild FE optimistic)
Mainly clarifying naming (scheduleMessageImport was actually tagging as
PENDING)
+ tagging as SCHEDULED in case of direct import (we skip PENDING state)
In this PR:
- change messaging / calendar stale duration check to 30minutes (cron is
running every 1h, duration check was 1h, so evaluation was flaky)
- when temporary error (throttling), preserve syncStageStartedAt as this
is necessary to assess exponential throttling
## Summary
This PR adds a new **Metadata Builder** AI agent that specializes in
managing the workspace data model (creating objects, adding fields,
etc.).
## Changes
### New Files
- `data-model-manager-role.ts` - New standard role with `DATA_MODEL`
permission flag
- `metadata-builder-agent.ts` - New standard agent for data model
management
### Modified Files
- **ChatToolsProviderService**: Refactored to consolidate all
permission-based tools into a single `getChatTools()` method. Now
injects both workflow tools and metadata tools based on permissions.
- **AgentChatRoutingService**: Updated to use the new consolidated
`getChatTools()` method
- **AiChatModule**: Added imports for `ObjectMetadataModule` and
`FieldMetadataModule`
- **Router system prompt**: Added metadata-builder agent selection rules
with clear distinction between schema operations vs data operations
- **Metadata tools factories**: Improved error messages to show detailed
validation errors instead of generic messages
### Refactoring
- Renamed `index.ts` files to `standard-agent-definitions.ts` and
`standard-role-definitions.ts` to follow naming conventions
- Renamed exports from `standardAgentDefinitions` to
`STANDARD_AGENT_DEFINITIONS` (SCREAMING_SNAKE_CASE)
## Key Features
1. **Metadata Builder Agent** can:
- Create new custom objects
- Add fields to existing objects
- Update object and field properties
- Create relations between objects
2. **Permission-based tool injection**: Tools are automatically injected
based on the `DATA_MODEL` permission flag
3. **Improved routing**: The router now correctly distinguishes between:
- "Create an object called Project" → metadata-builder (schema)
- "Create a company called Acme" → data-manipulator (data)
4. **Better error messages**: Validation errors now show detailed
messages like:
```
Validation errors:
[objectMetadata] Name must be in camelCase format
[objectMetadata] Label is required
```
# Introduction
Related to https://github.com/twentyhq/core-team-issues/issues/1995
This PR introduces the basis of the `twentyStandard` application as code
on demand, it's highly tied to `ids` where it will becomes workspace
agnostic following the builder and runner `universalIdentifier` refactor
later.
The goal here to allow computing the `allFlatEntityMaps` `to` of the
`twentyStandard` application on a empty workspace ( workspace creation
). Allowing installing the twenty standard app through a workspace
migration instead of passing by the sync metadata
Nothing done will be run in production for the moment if it's not the
small validation refactor we've introduced
Please note that everything introduced here will be replaced at some
point by a twenty app instance when the twenty sdk is mature enough to
handle of the edge cases we need here
## How we've proceeded
We've been iterating over every workspace entity both objects and their
fields, and transpiled them to flatEntity.
Being sure we migrate the defaultValue, settings and so on accordingly.
We've also compute all the ids in prior of the whole entities
computation so we don't face any hoisting issue.
## Current state
At the moment only handling all of the 29 standard objects and their
fields
Settings a unique universalIdentifier for all of them
Will come views, agent role targets and so on later
## `workspace:compute-twenty-standard-migration` command
This command allow generating a workspace migration that will result in
installing the twenty standard app in an empty workspace
It's temporary and aims to allow debugging for the moment we might not
keep it in the future as it is right now
It contains debug writeFileSync which is expected no worries greptile
## `LabelFieldMetadataIdentifierId`
Small refactor allowing defining the label identifier field metadata id
of a uuid field metadata type for system object, as some of our standard
object don't have a name field and don't aim to
Also please note that we might remove this build options later in the
sake of the currently installed universal identifier application that we
could compare with the deterministic twenty standard one
## `runFlatFieldMetadataValidators`
Deprecated this pattern which was redundant and not v2 friendly pattern
## Current errors that will address in upcoming PR
Current standard objects and fields metadata does not pass the
validation that we have in place, as historically the sync metadata
would directly consume the repositories and would just ignore the
validation. This is about to change.
Will handle the below errors in dedicated PRs as they will required
upgrade commands in order to migrate the data, or will handle that from
the sync metadata instead still to be determined but nothing critical
here
- camel case field metadata name
- options label invalid format
```json
{
"status": "fail",
"report": {
"fieldMetadata": [
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Name should be in camelCase",
"userFriendlyMessage": {
"id": "P+jdmX",
"message": "Name should be in camelCase"
},
"value": "iCalUID"
}
],
"flatEntityMinimalInformation": {
"id": "68dd83cd-92c8-4233-bb28-47939bab6124",
"name": "iCalUID",
"objectMetadataId": "11c16ab6-9176-439e-a2db-a12c5a58a524"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"email\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "email"
}
},
"value": "email"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"sms\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "sms"
}
},
"value": "sms"
}
],
"flatEntityMinimalInformation": {
"id": "e3caaf2a-e07d-4146-8dfc-9eef904e82c9",
"name": "type",
"objectMetadataId": "4b777de5-4c7b-4af4-9b92-655c0f87512b"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"incoming\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "incoming"
}
},
"value": "incoming"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"outgoing\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "outgoing"
}
},
"value": "outgoing"
}
],
"flatEntityMinimalInformation": {
"id": "d96233a4-93be-45ea-9548-3b50f3c700cf",
"name": "direction",
"objectMetadataId": "480a648a-d2e5-482a-992f-ef053e1b4bb0"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"from\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "from"
}
},
"value": "from"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"to\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "to"
}
},
"value": "to"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"cc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "cc"
}
},
"value": "cc"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Value must be in UPPER_CASE and follow snake_case \"bcc\"",
"userFriendlyMessage": {
"id": "UBPzFQ",
"message": "Value must be in UPPER_CASE and follow snake_case \"{sanitizedValue}\"",
"values": {
"sanitizedValue": "bcc"
}
},
"value": "bcc"
}
],
"flatEntityMinimalInformation": {
"id": "961c598e-67c3-452d-8bb2-b92c0bc64404",
"name": "role",
"objectMetadataId": "8af8a13c-ff97-4cd3-b70d-52a7dc2924b4"
},
"type": "create_field"
},
{
"status": "fail",
"errors": [
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Commas and dot (1,234.56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Spaces and comma (1 234,56)"
},
{
"code": "INVALID_FIELD_INPUT",
"message": "Label must not contain a comma",
"userFriendlyMessage": {
"id": "k731jp",
"message": "Label must not contain a comma"
},
"value": "Dots and comma (1.234,56)"
}
],
"flatEntityMinimalInformation": {
"id": "7fa20caf-2597-42e3-84e5-15a91b125b9b",
"name": "numberFormat",
"objectMetadataId": "a6974302-9e72-461c-aa09-9390f4ff16fc"
},
"type": "create_field"
}
],
"objectMetadata": [],
"view": [],
"viewField": [],
"viewGroup": [],
"index": [],
"serverlessFunction": [],
"cronTrigger": [],
"databaseEventTrigger": [],
"routeTrigger": [],
"viewFilter": [],
"role": [],
"roleTarget": [],
"agent": []
}
}
```
# Introduction
Migrating `pageLayoutTab` to the v2 engine
- Types and constants
- Builder and validate
- Runner
Introduced a new `StrictSyncableEntity` that enforces that
`universalIdentifier` and `applicationId` are defined
As these entities are brand new we could enforce this rule already
This still requires a migration command to associate the existing
entities to custom workspace application instance and define
universalIdentifier
Handled retro-comp of the migration through a migration as upgrade
command fallback
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
Validations :
- relations count (in common api)
- oneToMany relation nested count (in common)
- requested fields count (in gql)
- root resolver count (in gql)
- root resolver duplicates (in gql)
- specific complexity for metadata / nesting count (in gql)
Fixes a regression on seeding introduced by
https://github.com/twentyhq/twenty/pull/16296.
In the seeding process an async function was used without awaiting. This
caused all the seeded configs to be empty.
Note: The need to await is because `transformRichTextV2Value` is async
due to `const { ServerBlockNoteEditor } = await
import('@blocknote/server-util');`. @etiennejouan do you know why is it
done this way? Is it to reduce bundle size? Would it be possible to
create another util which is not async and which imports
`@blocknote/server-util` at the top?
## Context
Call to incrementMetadataVersion is only done in sync-metadata (to be
deprecated) and migration runner v2. They both already invalidate the
cache so we were doing it twice which is quite heavy with all the
object/field/view/etc maps in it.
Timeline activities are fetched using the `useTimelineActivities` hook,
which relies on `useFindManyRecords`. Previously, it also fetched linked
object titles via `useLinkedObjectsTitle`.
When the user scrolled to the bottom, we triggered `fetchMore` from
`useFindManyRecords` to load additional data. This operation correctly
handled in-place loading and did not set the main `loading` flag to
`true`.
However, after `useFindManyRecords` returned updated data, new variables
were passed to `useLinkedObjectsTitle`, which didn’t leverage the
`fetchMore` pattern. As a result, the `loading` state of
`useLinkedObjectsTitle` was set to `true`, even when only partial data
changed.
Our logic for displaying the skeleton loader was:
```ts
const loading = loadingTimelineActivities || loadingLinkedObjectsTitle;
```
When `loadingLinkedObjectsTitle` turned `true`, this triggered the
entire list to unmount and remount, causing repeated unnecessary
reloads.
To resolve this, I no longer delay rendering the list while loading
linked objects title.
## Demo
The skeleton is only shown at the beginning of the navigation.
https://github.com/user-attachments/assets/cb79f91d-5151-4dc1-8e6b-a322e56a48c4
Closes https://github.com/twentyhq/twenty/issues/16210
pipeline.exec() is a a blocking operation which means if we want to
write a large chunk of data (or many maps in our case) it will block
write operations and impact performances.
We prefer to simply call set multiple times in a promise.all, this is an
acceptable tradeoff
