This PR is a follow-up of https://github.com/twentyhq/twenty/pull/16966
and implements a new mechanism to handle SSE events.
It creates only one event stream per browser tab, then use mutations to
tell the backend which query to listen to, without re-mounting the event
stream connexion.
Then each event that comes from this unique subscription is then
dispatched in a new JavaScript CustomEvent, per queryId, on which
specific hooks add an event listener.
This PR introduces the generic tooling as well as the handling of update
events on table.
Fixes : https://github.com/twentyhq/twenty/issues/17035
There was a bad pattern which risked introduce this bug in
`turnRecordFilterIntoRecordGqlOperationFilter` after recent refactor of
date logic and date filters, which didn't create any bug during dev time
because it wasn't tested with value combinations that existed before the
refactor.
Code has been re-organized to make it resilient to any wrong value
combination.
Also fixed a small bug that appeared during QA with `DATE` filter type,
which was blocking the save button from disappearing after a save.
## Description
SELECT fields have a defined option order that users expect to see
reflected in charts.
This PR allows sorting by that position and also enables custom manual
ordering.
## Video QA
### Reordering on primary axis
https://github.com/user-attachments/assets/994f515e-19cb-4a5e-b745-e8c77e92ae0b
### Reordering on secondary axis
https://github.com/user-attachments/assets/444c16f2-1920-4dc4-8b42-312d520ab43b
Note: The colors in the graph will match the colors of the select
options, but this will be done in another PR
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> Introduces new sort modes and UI for chart groupings, with full FE/BE
support and updated GraphQL schema.
>
> - Extend `GraphOrderBy` with `FIELD_POSITION_ASC/DESC` and `MANUAL`;
add corresponding fields in configs: `primaryAxisManualSortOrder`,
`secondaryAxisManualSortOrder`, and `manualSortOrder` (pie)
> - New UI: dropdown options filtered by field type, icons, and a
draggable submenu (`ChartManualSortSubMenuContent`) to reorder select
options; integrates with widget edit flow
> - Sorting logic added/refactored: `sortChartData`,
`sortByManualOrder`, `sortBySelectOptionPosition`,
`sortLineChartSeries`, plus updates to bar/line/pie transformers to
honor new modes and manual orders
> - Default behaviors: select fields default to `FIELD_POSITION_ASC`;
query variable builders skip `orderBy` when using manual/position sorts
> - Update GraphQL generated types/fragments/queries and backend
DTOs/schemas to persist new fields; add tests for sorting utilities and
snapshots; add sorting icons in `twenty-ui`
>
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
78c9b56c0f. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Fixes https://github.com/twentyhq/twenty/issues/16110
This PR implements Temporal to replace the legacy Date object, in all
features that are time zone sensitive. (around 80% of the app)
Here we define a few utils to handle Temporal primitives and obtain an
easier DX for timezone manipulation, front end and back end.
This PR deactivates the usage of timezone from the graph configuration,
because for now it's always UTC and is not really relevant, let's handle
that later.
Workflows code and backend only code that don't take user input are
using UTC time zone, the affected utils have not been refactored yet
because this PR is big enough.
# New way of filtering on date intervals
As we'll progressively rollup Temporal everywhere in the codebase and
remove `Date` JS object everywhere possible, we'll use the way to filter
that is recommended by Temporal.
This way of filtering on date intervals involves half-open intervals,
and is the preferred way to avoid edge-cases with DST and smallest time
increment edge-case.
## Filtering endOfX with DST edge-cases
Some day-light save time shifts involve having no existing hour, or even
day on certain days, for example Samoa Islands have no 30th of December
2011 : https://www.timeanddate.com/news/time/samoa-dateline.html, it
jumps from 29th to 31st, so filtering on `< next period start` makes it
easier to let the date library handle the strict inferior comparison,
than filtering on `≤ end of period` and trying to compute manually the
end of the period.
For example for Samoa Islands, is end of day `2011-12-29T23:59:59.999`
or is it `2011-12-30T23:59:59.999` ? If you say I don't need to know and
compute it, because I want everything strictly before
`2011-12-29T00:00:00 + start of next day (according to the library which
knows those edge-cases)`, then you have a 100% deterministic way of
computing date intervals in any timezone, for any day of any year.
Of course the Samoa example is an extreme one, but more common ones
involve DST shifts of 1 hour, which are still problematic on certain
days of the year.
## Computing the exact _end of period_
Having an open interval filtering, with `[included - included]` instead
of half-open `[included - excluded)`, forces to compute the open end of
an interval, which often involves taking an arbitrary unit like minute,
second, microsecond or nanosecond, which will lead to edge-case of
unhandled values.
For example, let's say my code computes endOfDay by setting the time to
`23:59:59.999`, if another library, API, or anything else, ends up
giving me a date-time with another time precision `23:59:59.999999999`
(down to the nanosecond), then this date-time will be filtered out,
while it should not.
The good deterministic way to avoid 100% of those complex bugs is to
create a half-open filter :
`≥ start of period` to `< start of next period`
For example :
`≥ 2025-01-01T00:00:00` to `< 2025-01-02T00:00:00` instead of `≥
2025-01-01T00:00:00` to `≤ 2025-01-01T23:59:59.999`
Because, `2025-01-01T00:00:00` = `2025-01-01T00:00:00.000` =
`2025-01-01T00:00:00.000000` = `2025-01-01T00:00:00.000000000` => no
risk of error in computing start of period
But `2025-01-01T23:59:59` ≠ `2025-01-01T23:59:59.999` ≠
`2025-01-01T23:59:59.999999` ≠ `2025-01-01T23:59:59.999999999` =>
existing risk of error in computing end of period
This is why an half-open interval has no risk of error in computing a
date-time interval filter.
Here is a link to this debate :
https://github.com/tc39/proposal-temporal/issues/2568
> For this reason, we recommend not calculating the exact nanosecond at
the end of the day if it's not absolutely necessary. For example, if
it's needed for <= comparisons, we recommend just changing the
comparison code. So instead of <= zdtEndOfDay your code could be <
zdtStartOfNextDay which is easier to calculate and not subject to the
issue of not knowing which unit is the right one.
>
> [Justin Grant](https://github.com/justingrant), top contributor of
Temporal
## Application to our codebase
Applying this half-open filtering paradigm to our codebase means we
would have to rename `IS_AFTER` to `IS_AFTER_OR_EQUAL` and to keep
`IS_BEFORE` (or even `IS_STRICTLY_BEFORE`) to make this half-open
interval self-explanatory everywhere in the codebase, this will avoid
any confusion.
See the relevant issue :
https://github.com/twentyhq/core-team-issues/issues/2010
In the mean time, we'll keep this operand and add this semantic in the
naming everywhere possible.
## Example with a different user timezone
Example on a graph grouped by week in timezone Pacific/Samoa, on a
computer running on Europe/Paris :
<img width="342" height="511" alt="image"
src="https://github.com/user-attachments/assets/9e7d5121-ecc4-4233-835b-f59293fbd8c8"
/>
Then the associated data in the table view, with our **half-open
date-time filter** :
<img width="804" height="262" alt="image"
src="https://github.com/user-attachments/assets/28efe1d7-d2fc-4aec-b521-bada7f980447"
/>
And the associated SQL query result to see how DATE_TRUNC in Postgres
applies its internal start of week logic :
<img width="709" height="220" alt="image"
src="https://github.com/user-attachments/assets/4d0542e1-eaae-4b4b-afa9-5005f48ffdca"
/>
The associated SQL query without parameters to test in your SQL client :
```SQL
SELECT "opportunity"."closeDate" as "close_date", TO_CHAR(DATE_TRUNC('week', "opportunity"."closeDate", 'Pacific/Samoa') AT TIME ZONE 'Pacific/Samoa', 'YYYY-MM-DD') AS "DATE_TRUNC by week start in timezone Pacific/Samoa", "opportunity"."name" FROM "workspace_1wgvd1injqtife6y4rvfbu3h5"."opportunity" "opportunity" ORDER BY "opportunity"."closeDate" ASC NULLS LAST
```
# Date picker simplification (not in this PR)
Our DatePicker component, which is wrapping `react-datepicker` library
component, is now exposing plain dates as string instead of Date object.
The Date object is still used internally to manage the library
component, but since the date picker calendar is only manipulating plain
dates, there is no need to add timezone management to it, and no need to
expose a handleChange with Date object.
The timezone management relies on date time inputs now.
The modification has been made in a previous PR :
https://github.com/twentyhq/twenty/issues/15377 but it's good to
reference it here.
# Calendar feature refactor
Calendar feature has been refactored to rely on Temporal.PlainDate as
much as possible, while leaving some date-fns utils to avoid re-coding
them.
Since the trick is to use utils to convert back and from Date object in
exec env reliably, we can do it everywhere we need to interface legacy
Date object utils and Temporal related code.
## TimeZone is now shown on Calendar :
<img width="894" height="958" alt="image"
src="https://github.com/user-attachments/assets/231f8107-fad6-4786-b532-456692c20f1d"
/>
## Month picker has been refactored
<img width="503" height="266" alt="image"
src="https://github.com/user-attachments/assets/cb90bc34-6c4d-436d-93bc-4b6fb00de7f5"
/>
Since the days weren't useful, the picker has been refactored to remove
the days.
# Miscellaneous
- Fixed a bug with drag and drop edge-case with 2 items in a list.
# Improvements
## Lots of chained operations
It would be nice to create small utils to avoid repeated chained
operations, but that is how Temporal is designed, a very small set of
primitive operations that allow to compose everything needed. Maybe
we'll have wrappers on top of Temporal in the coming years.
## Creation of Temporal objects is throwing errors
If the input is badly formatted Temporal will throw, we might want to
adopt a global strategy to avoid that.
Example :
```ts
const newPlainDate = Temporal.PlainDate.from('bad-string'); // Will throw
```
Since we now store rich text value in blocknote rather than markdown,
variables need to be resolved accordingly.
Replacing the variable tag pattern
`{"type":"variableTag","attrs":\{"variable":"(\{\{[^{}]+\}\})"\}\}` by a
blocknote text `{"type":"text","text":"${escapedText}"}`
Fixes https://github.com/twentyhq/twenty/issues/16583
To test :
- build a workflow that creates a note/ sends an email with a variable
in the body
- make sure the result is properly formatted once run
---------
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Fixes#16636
Added useCloseDropdown() hook and set onEnter prop to
onEnter={closeDropDown()} using dropdownID
EDIT from @charlesBochet after refactoring:
- ObjectDropdownFilters are used in 3 places: Main Filter menu,
EditableChip, AdvancedFilters
- deprecate vectorSearch in view filter area, we are not using them, we
are doing a anyField filter now. While refactoring the points below, I
did not want to maintain vectorSearch as it was not used anymore
- stop confusing the dropdownId (which is an id to interact with a
specific dropdown) and componentInstanceIds (which is used to scope
component states) for EditableFilter case
- I haven't fixed the confusion for MainFilter case
- It was already handled for AdvancedFilter case
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Title: "feat: Add second, minute & hour resolution options to relative
date Filter action"
---
## Summary
This PR enables support for smaller time units — **Seconds, Minutes, and
Hours** — in the *Relative Date* filter used in workflows, rather than
being limited to days only.
---
## What Changed
This PR extends the relative date filter to include support for the
following units:
✔️ `SECOND`
✔️ `MINUTE`
✔️ `HOUR`
✔️ (Existing: `DAY`, `WEEK`, `MONTH`, etc.)
Changes include:
- Adding `SECOND`, `MINUTE`, and `HOUR` options to the internal relative
date unit enum/constant.
- Updating utility functions and parsers to correctly interpret and
evaluate these new units.
- Enhancing existing tests and adding new tests to cover second, minute,
and hour relative filters.
---
## Testing
New and updated tests include:
- Unit tests for serialization of relative filter values including
seconds, minutes, and hours.
- Workflow filter evaluation tests that verify minute/hour resolution
behaves correctly.
Tests are included in the changeset.
---
## Backward Compatibility
This change is fully backward compatible:
- All existing relative date filters using days or larger units behave
exactly as before.
- Adding finer units does not alter existing stored data or workflow
definitions.
---
## Issue Reference
Fixes: **twentyhq/twenty#15525**
<img width="1909" height="896" alt="image"
src="https://github.com/user-attachments/assets/328d03dc-ca0b-4c3f-84e5-58961c178398"
/>
---------
Co-authored-by: Guillim <guillim@users.noreply.github.com>
Co-authored-by: guillim <guigloo@msn.com>
**What this fixes:**
- Addresses a CodeQL security finding: the regex used to find variables
in workflow strings could be slow on malicious inputs (ReDoS).
- Two alerts: [Code Scanning
181](https://github.com/twentyhq/twenty/security/code-scanning/181) and
[Code Scanning
182](https://github.com/twentyhq/twenty/security/code-scanning/182)
**Context:**
- Our workflow system lets users insert variables like `{{user.name}}`
or `{{trigger.properties.after.name}}` into strings and JSON (HTTP
request bodies, record field values, etc.).
- The `variable-resolver.ts` module scans these strings and replaces
variables with actual values.
- Our validation (`isValidVariable`) already enforces that variables
contain no `{` or `}` inside them (only simple property paths like
`user.name`).
**The change:**
- Updated the regex from `/\{\{(.*?)\}\}/g` to `/\{\{([^{}]+)\}\}/g` to
match our validation pattern.
- This removes the ReDoS risk and aligns the resolver with the
validation contract.
**Why this is safe:**
- All supported workflow usage (simple variable paths) continues to
work.
- Both `match` and `replace` behave the same for valid variables.
- Only unsupported patterns with nested braces (e.g., `{{foo {bar}}}`)
would stop matching, which isn't part of our supported syntax anyway.
- moves applicationRoleId to application entity
- add new `APPLICATION` FieldActorSource and `APPLICATION`
JwtTokenTypeEnum value
- create a new token with applicationId when executing a function
- when applicationId is in token, check for application.defaultRole
permissions
-use twenty-shared types in `twenty-sdk/application`
- create a new import from generate called "Twenty" that you can use
directly without having to set TWENTY_API_KEY AND TWENTY_API_URL (keep
metadata or core parameter only)
- provide to serverless unique one time BEARER TOKEN to run it
Result
<img width="977" height="566" alt="image"
src="https://github.com/user-attachments/assets/e78428a0-5b13-4975-aa13-58ee3b32450c"
/>
<img width="910" height="596" alt="image"
src="https://github.com/user-attachments/assets/6ec72bf5-7655-4093-a45e-ad269595a324"
/>
<img width="741" height="568" alt="image"
src="https://github.com/user-attachments/assets/7683944c-fd79-4417-8fb2-8e4815cc112f"
/>
Closes#16557
Tiptap Editor (which the Send Email Node uses) , creates a content json
with type 'hardBreak' for line breaks.
The was no rederer defined for this `hardBreak` node type, so the
`renderNode` function was ignoring that node (returning null).
**Fix :** Added a renderer for `hardBreak` node type.
# Fix: URL Encoding Bug & Code Refactor
## Issue
**Bug**: URLs with encoded characters (e.g., `%20` for spaces) were
being double-encoded or incorrectly processed in
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2),
causing URL mismatches and potential data integrity issues.
**Build Error**: `TS2307: Cannot find module
'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util'`
## Root Cause Analysis
### 1. Missing URL Decoding
The
[lowercaseUrlOriginAndRemoveTrailingSlash](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:4:0-16:2)
function was processing URLs without properly decoding URI components.
When URLs contained encoded characters like `%20`, `%2F`, etc., they
weren't being normalized correctly.
### 2. Invalid Cross-Package Import
The fix attempted to import
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
from `twenty-server`:
```typescript
import { safeDecodeURIComponent } from 'src/modules/messaging/message-import-manager/drivers/imap/utils/safe-decode-uri-component.util';
```
This failed because:
- The file resides in `twenty-shared`, a separate package
- TypeScript cannot resolve internal paths from another package
- The monorepo uses package exports (`twenty-shared/utils`) for
cross-package imports, not direct file paths
## Solution
### 1. Bug Fix: Added Safe URI Decoding
Updated
[lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
to properly decode URL components:
```typescript
export const lowercaseUrlOriginAndRemoveTrailingSlash = (rawUrl: string) => {
const url = getURLSafely(rawUrl);
if (!isDefined(url)) {
return rawUrl;
}
const lowercaseOrigin = url.origin.toLowerCase();
const path =
safeDecodeURIComponent(url.pathname) +
safeDecodeURIComponent(url.search) +
url.hash;
return (lowercaseOrigin + path).replace(/\/$/, '');
};
```
The
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
wrapper handles malformed URI sequences gracefully by returning the
original string if decoding fails, preventing runtime crashes.
### 2. Refactor: Consolidated Shared Utility
**Before**: Duplicate utility existed in `twenty-server`
```
twenty-server/src/modules/messaging/.../utils/safe-decode-uri-component.util.ts
```
**After**: Single source of truth in `twenty-shared`
```
twenty-shared/src/utils/url/safeDecodeURIComponent.ts
```
This follows the established pattern in the codebase where shared
utilities live in `twenty-shared` and are imported via subpath exports:
```typescript
// In twenty-server
import { safeDecodeURIComponent } from 'twenty-shared/utils';
// In twenty-shared (local import)
import { safeDecodeURIComponent } from './safeDecodeURIComponent';
```
## Files Changed
| File | Action | Description |
|------|--------|-------------|
|
[twenty-shared/src/utils/url/safeDecodeURIComponent.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-0:0)
| **Created** | New shared utility (moved from twenty-server) |
|
[twenty-shared/src/utils/url/index.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/index.ts:0:0-0:0)
| **Modified** | Added export for
[safeDecodeURIComponent](cci:1://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/safeDecodeURIComponent.ts:0:0-6:2)
|
|
[twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts](cci:7://file:///Users/apple/WebstormProjects/twenty/packages/twenty-shared/src/utils/url/lowercaseUrlOriginAndRemoveTrailingSlash.ts:0:0-0:0)
| **Modified** | Fixed import path, now uses local relative import |
| `twenty-server/.../imap-message-text-extractor.service.ts` |
**Modified** | Updated import to use `twenty-shared/utils` |
| `twenty-server/.../safe-decode-uri-component.util.ts` | **Deleted** |
Removed duplicate utility |
## The Utility
```typescript
// safeDecodeURIComponent.ts
export const safeDecodeURIComponent = (text: string): string => {
try {
return decodeURIComponent(text);
} catch {
return text;
}
};
```
This wrapper is necessary because `decodeURIComponent()` throws a
`URIError` on malformed sequences (e.g., `%E0%A4%A`). The safe version
returns the original string instead of crashing.
## Testing
- **342 tests passed** in `twenty-shared`
- `lowercaseUrlOriginAndRemoveTrailingSlash.test.ts` validates URL
normalization behavior
- No regressions in existing functionality
## Impact
- **Bug Fixed**: URLs with encoded characters are now properly
normalized
- **Code Quality**: Eliminated code duplication between packages
- **Maintainability**: Single source of truth for URI decoding utility
- **Build**: Resolved TS2307 compilation error
---------
Co-authored-by: Joker <apple@Apples-MacBook-Pro.local>
Using deepEqual, average over 3 calls:
- version trigger 1.53ms
- version steps 38.3ms
- run state 372.8 ms
Using stringified hash, average over 3 calls:
- version trigger 0.07ms
- version steps 0.74 ms
- run state 0.521ms
Short fastDeepEqual
- version trigger 0.028ms
- version steps 0.197ms
- run state 0.214ms
Lib fastDeepEqual
- version trigger 0.038ms
- version steps 0.187ms
- run state 0.230ms
closes https://github.com/twentyhq/core-team-issues/issues/1629
To do before requesting review :
- filter update
Migration to come in an other PR
Strat :
1/ Null transformation
- [x] Transform NULL equivalent value to NULL in field validation in
common api - pre-query - with feature flag
- [ ] Same logic in ORM (Not done, complex to handle feature flag here)
- [x] Transform NULL value to equivalent in data formatting in ORM -
post-query
2/ Migration (in other PR) for fieldMetadata not nullable with default
defaultValue (empty string, ...)
- [ ] Remove NOT NULL db constraint
- [ ] Update record value to NULL
- [ ] Update field metadata : isNullable:true
- [ ] Update uniqueIndex whereClause (also for standard uniqueIndex)
- [ ] Activate feature flag
3/ Update metadata creation
- [x] No more default default value
- [x] Update standard field nullability
- [x] Remove index default whereClause for standard field
4/ Update filter
- [x] When filtering on NULL or empty string, be sure all records are
returned (the one with NULL + the one with "")
5/ Test
- [ ] Strat. to do
## Summary
- add a regression test that reproduces the workflow "Search Records →
ID is <UUID>" failure (issue #15067 / #15746)
- adjust `turnRecordFilterIntoRecordGqlOperationFilter` so UUID filters
fall back to the literal value when no `recordIdsForUuid` context is
provided, always emitting an `in` clause
## Testing
- npx nx test twenty-shared --
--testPathPattern=computeRecordGqlOperationFilter.test.ts
--coverage=false
- Manual: workflow "Search Records" with filter "ID is <UUID>" now
returns the correct record
Fixes#15067Fixes#15746
---------
Co-authored-by: remi <remi@labox-apps.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
## Introduction
Since we've moved some class validator instances from twenty-server to
twenty-shared tests are red because they do not know how to parse
decorators declarations
We've fixed this by explicitly installing `class-validator` in
`twenty-shared` and configuring jest swc accordingly
## Twenty-server class validator patch
I don't even know if that's something we need anymore
Seems to be a patch either fixing or introducing credit card and phone
number validation
Would prefer discussing the need or not to either before merging this as
it could introduce regression at runtime:
- Centralize the patch to be consumed in both `twenty-server` and
`twenty-shared`
- Remove the patch
We should also document every patch motivations we do as it's quite
though to iterate over a such huge one
This PR fixes the filter on not empty phones.
We can end up with phone numbers that only have a country code, but it
makes no sense to consider this special edge case "not empty".
If we have no phone number, then it's empty, so this PR applies this
particular use case for NOT EMPTY filter on phone fields.
Fixes https://github.com/twentyhq/twenty/issues/15638
This PR fixes a bug which could happen if two people were editing a page
layout at the same time.
If one person deletes a widget or a tab and saves first, and the second
person saves after but doesn't delete this widget or tab, an error is
raised.
This is because, in the backend, a diff is computed to know which tab or
widget to create, update or delete. But the logic was maid without
considering soft deletion. So, when the second person saves, the diff
tries to create the widget or tab which had been soft deleted by the
first use. Since they have the same id, a duplicate primary key error is
raised.
Now we always consider that the last user to save has the truth. So we
restore the tab/widget and update it.
This PR implements the necessary tools to have `react-datepicker`
calendar and our date picker components work reliably no matter the
timezone difference between the user execution environment and the user
application timezone.
Fixes https://github.com/twentyhq/core-team-issues/issues/1781
This PR won't cover everything needed to have Twenty handle timezone
properly, here is the follow-up issue :
https://github.com/twentyhq/core-team-issues/issues/1807
# Features in this PR
This PR brings a lot of features that have to be merged together.
- DATE field type is now handled as string only, because it shouldn't
involve timezone nor the JS Date object at all, since it is a day like a
birthday date, and not an absolute point in time.
- DATE_TIME field wasn't properly handled when the user settings
timezone was different from the system one
- A timezone abbreviation suffix has been added to most DATE_TIME
display component, only when the timezone is different from the system
one in the settings.
- A lot of bugs, small features and improvements have been made here :
https://github.com/twentyhq/core-team-issues/issues/1781
# Handling of timezones
## Essential concepts
This topic is so complex and easy to misunderstand that it is necessary
to define the precise terms and concepts first. It resembles character
encoding and should be treated with the same care.
- Wall-clock time : the time expressed in the timezone of a user, it is
distinct from the absolute point in time it points to, much like a
pointer being a different value than the value that it points to.
- Absolute time : a point in time, regardless of the timezone, it is an
objective point in time, of course it has to be expressed in a given
timezone, because we have to talk about when it is located in time
between humans, but it is in fact distinct from any wall clock time, it
exists in itself without any clock running on earth. However, by
convention the low-level way to store an absolute point in time is in
UTC, which is a timezone, because there is no way to store an absolute
point in time without a referential, much like a point in space cannot
be stored without a referential.
- DST : Daylight Save Time, makes the timezone shift in a specific
period every year in a given timezone, to make better use of longer days
for various reasons, not all timezones have DST. DST can be 1 hour or 30
min, 45 min, which makes computation difficult.
- UTC : It is NOT an “absolute timezone”, it is the wall-clock time at
0° longitude without DST, which is an arbitrary and shared human
convention. UTC is often used as the standard reference wall-clock time
for talking about absolute point in time without having to do timezone
and DST arithmetic. PostgreSQL stores everything in UTC by convention,
but outputs everything in the server’s SESSION TIMEZONE.
## How should an absolute point in time be stored ?
Since an absolute point in time is essentially distinct from its
timezone it could be stored in an absolute way, but in practice it is
impossible to store an absolute point in time without a referential. We
have to say that a rocket launched at X given time, in UTC, EST, CET,
etc. And of course, someone in China will say that it launched at 10:30,
while in San Francisco it will have launched at 19:30, but it is THE
SAME absolute point in time.
Let’s take a related example in computer science with character
encoding. If a text is stored without the associated encoding table, the
correct meaning associated to the bits stored in memory can be lost
forever. It can become impossible for a program to guess what encoding
table should be used for a given text stored as bits, thus the glitches
that appeared a lot back in the early days of internet and document
processing.
The same can happen with date time storing, if we don’t have the
timezone associated with the absolute point in time, the information of
when it absolutely happened is lost.
It is NOT necessary to store an absolute point in time in UTC, it is
more of a standard and practical wall-clock time to be associated with
an absolute point in time. But an absolute point in time MUST be store
with a timezone, with its time referential, otherwise the information of
when it absolutely happened is lost.
For example, it is easier to pass around a date as a string in UTC, like
`2024-01-02T00:00:00Z` because it allows front-end and back-end code to
“talk” in the same standard and DST-free wall-clock time, BUT it is not
necessary. Because we have date libraries that operate on the standard
ISO timezone tables, we can talk in different timezone and let the
libraries handle the conversion internally.
It is false to say that UTC is an absolute timezone or an absolute point
in time, it is just the standard, conventional time referential, because
one can perfectly store every absolute points in time in UTC+10 with a
complex DST table and have the exactly correct absolute points in time,
without any loss of information, without having any UTC+0 dates
involved.
Thus storing an absolute point in time without a timezone associated,
for example with `timestamp` PostgreSQL data type, is equivalent to
storing a wall-clock time and then throwing away voluntarily the
information that allows to know when it absolutely happened, which is a
voluntary data-loss if the code that stores and retrieves those
wall-clock points in time don’t store the associated timezone somewhere.
This is why we use `timestamptz` type in PostgreSQL, so that we make
sure that the correct absolute point in time is stored at the exact time
we send it to PostgreSQL server, no matter the front-end, back-end and
SQL server's timezone differences.
## The JavaScript Date object
The native JavaScript Date object is now officially considered legacy
([source](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date)),
the Date object stores an absolute point in time BUT it forces the
storage to use its execution environment timezone, and one CANNOT modify
this timezone, this is a legacy behavior.
To obtain the desired result and store an absolute point in time with an
arbitrary timezone there are several options :
- The new Temporal API that is the successor of the legacy Date object.
- Moment / Luxon / @date-fns/tz that expose objects that allow to use
any timezone to store an absolute point in time.
## How PostgreSQL stores absolute point in times
PostgreSQL stores absolute points in time internally in UTC
([source](https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-DATETIME-INPUT-TIME-STAMPS)),
but the output date is expressed in the server’s session timezone
([source](https://www.postgresql.org/docs/current/sql-set.html)) which
can be different from UTC.
Example with the object companies in Twenty seed database, on a local
instance, with a new “datetime” custom column :
<img width="374" height="554" alt="image"
src="https://github.com/user-attachments/assets/4394cb43-d97e-4479-801d-ca068f800e39"
/>
<img width="516" height="524" alt="image"
src="https://github.com/user-attachments/assets/b652f36a-d2e2-47a4-8950-647ca688cbbd"
/>
## Why can’t I just use the JavaScript native Date object with some
manual logic ?
Because the JavaScript Date object does not allow to change its internal
timezone, the libraries that are based on it will behave on the
execution environment timezone, thus leading to bugs that appear only on
the computers of users in a timezone but not for other in another
timezone.
In our case the `react-datepicker` library forces to use the `Date`
object, thus forcing the calendar to behave in the execution environment
system timezone, which causes a lot of problems when we decide to
display the Twenty application DATE_TIME values in another timezone than
the user system one, the bugs that appear will be of the off-by-one date
class, for example clicking on 23 will select 24, thus creating an
unreliable feature for some system / application timezone combinations.
A solution could be to manually compute the difference of minutes
between the application user and the system timezones, but that’s not
reliable because of DST which makes this computation unreliable when DST
are applied at different period of the year for the two timezones.
## Why can’t I compute the timezone difference manually ?
Because of DST, the work to compute the timezone difference reliably,
not just for the usual happy path, is equivalent to developing the
internal mechanism of a date timezone library, which is equivalent to
use a library that handles timezones.
## Using `@date-fns/tz` to solve this problem
We could have used `luxon` but it has a heavy bundle size, so instead we
rely here on `@date-fns/tz` (~1kB) which gives us a `TZDate` object that
allows to use any given timezone to store an absolute point-in-time.
The solution here is to trick `react-datepicker` by shifting a Date
object by the difference of timezone between the user application
timezone and the system timezone.
Let’s take a concerte example.
System timezone : Midway, ⇒ UTC-11:00, has no DST.
User application timezone : Auckland, NZ ⇒ UTC+13:00, has a DST.
We’ll take the NZ daylight time, so that will make a timezone difference
of 24 hours !
Let’s take an error-prone date : `2025-01-01T00:00:00` . This date is
usually a good test-case because it can generate three classes of bugs :
off-by-one day bugs, off-by-one month bugs and off-by-one year bugs, at
the same time.
Here is the absolute point in time we take expressed in the different
wall-clock time points we manipulate
Case | In system timezone ⇒ UTC-11 | In UTC | In user application
timezone ⇒ UTC+13
-- | -- | -- | --
Original date | `2024-12-31T00:00:00-11:00` | `2024-12-31T11:00:00Z` |
`2025-01-01T00:00:00+13:00`
Date shifted for react-datepicker | `2025-01-01T00:00:00-11:00` |
`2025-01-01T11:00:00Z` | `2025-01-02T00:00:00+13:00`
We can see with this table that we have the number part of the date that
is the same (`2025-01-01T00:00:00`) but with a different timezone to
“trick” `react-datepicker` and have it display the correct day in its
calendar.
You can find the code in the hooks
`useTurnPointInTimeIntoReactDatePickerShiftedDate` and
`useTurnReactDatePickerShiftedDateBackIntoPointInTime` that contain the
logic that produces the above table internally.
## Miscellaneous
Removed FormDateFieldInput and FormDateTimeFieldInput stories as they do
not behave the same depending of the execution environment and it would
be easier to put them back after having refactored FormDateFieldInput
and FormDateTimeFieldInput
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
## Description
- This PR address issue -
https://github.com/twentyhq/core-team-issues/issues/1768
- Added listkit bundle from tiptap which includes BulletList,
orderedList, ListItem and ListKeymap in one single import
- This bundle also includes keyboards shortcuts - `Cmd + Shift + 7` and
`Cmd + Shift + 8` for ordered and bullet list
## Visual Appearance
https://github.com/user-attachments/assets/7eff1233-8503-4854-bad2-2521898bc568
## Why this Approach
- our current version of tiptap is 3.4.2 while the latest is 3.8.0 hence
installing these versions manually would install the latest version of
3.8.0. The issue when downgrading to 3.4.2 was that Version 3.8.0 of
`@tiptap/extension-list` requires `renderNestedMarkdownContent` from
@tiptap/core
but our `@tiptap/core` version 3.4.2 doesn't export this function.
Fixes - https://github.com/twentyhq/twenty/issues/15263
- Replaced `useLoadSelectedRecordsInContextStore` with
`useLoadMergeRecords` in `useOpenMergeRecordsPageInCommandMenu` for
improved functionality.
- Updated `useMergePreview`, `useMergeRecordsActions`, and
`useMergeRecordsSettings` to utilize `mergeRecordsState` instead of the
deprecated context store hook.
- Cleaned up imports and ensured consistency across merge-related hooks.
https://github.com/user-attachments/assets/453539c9-7f2b-4e8c-bfa1-3ceebca07081
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This big PR implements table virtualization with an offset paging,
allowing a way more fluid UX.
It is a v1 that should be improved in the future with partial data
loading and optimization of the browser display performance of a row.
But with this PR we have the solid enough technical foundation, both
frontend and backend, to get to a smooth table UX.
Fixes and improvements after first successful round of development
(needed to have main clean) :
- [x] Delete should refresh virtualized portion only and reset all table
- [x] Fix add new : top and bottom
- [x] Table empty shouldn’t show when first loading
- [x] Fix d&d
- [x] Fix sorts
- [x] Fix drag when scrolling after a full virtual page (it throws an
error)
- [x] Si update mais qu’on a un sort ou filter, alors il faut trigger le
refresh
- [x] Reset scroll position between tables
- [x] Reset scroll shadows between tables
- [x] Setup d&n for virtual list :
https://github.com/hello-pangea/dnd/blob/main/docs/patterns/virtual-lists.md
- [x] Full table re-render when entering edit mode
- [x] Clean code and prepare for merge
Fixes https://github.com/twentyhq/core-team-issues/issues/1613 that
contains other bugs to be fixed before merge
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
Migrating `viewFilter` to v2 in order to migrate later the field update
side effect on view to v2 too
## What's done
- Created flat-view-filter
- flat view filter runner
- flat view filter builder
- create view filter service v2 and input transpilers
- refactor the existing view filter resolver to fix standard (
BREAKING_CHANGE on graphql api update especially ) REST stays the same
- refactored the front to consume the mutations autogenerated
## New generic tools
### Compare two flat entity
Introducing a new util to compare two flat entity, it's strictly typed
and will be added to the generic builder in a following PR
This will ease flat entity addition as won't required to create a
specific abstraction for comparison
Generic builder will expect specific constant: properties to compare and
properties to stringify
### Transform flat entity for comparison
Forked and refactor the initial existing method for flat entity business
scope and type safety
## Coverage
Migrated existing integration tests to fit new contract API
This PR does not add strong coverage on validation exceptions
Deadlines are too short
close https://github.com/twentyhq/core-team-issues/issues/1666
Fixing
<img width="708" height="271" alt="Capture d’écran 2025-10-08 à 12 27
12"
src="https://github.com/user-attachments/assets/2f459e2f-146b-4452-8517-59f58b8a33a4"
/>
The circular-chunk warning came from computeRecordGqlOperationFilter.ts
importing from the barrel @/utils, which reexports
turnRecordFilterIntoRecordGqlOperationFilter and friends, creating a
re-export cycle across chunks.
This PR follows the multiSelect PR merged previously. It will enable
morph relation Many to One to be handled from the table, using a
singleSelect picker
Main point : I decided to change the singleSelect API to take an array
of **objectMetadataName** instead of only one to deal with both our
usecases.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
This PR is the first part of the creation of the Chart editor.
https://github.com/user-attachments/assets/8b0af8ea-be41-4506-84cb-e40f5521cbf0
Done:
- Bar chart settings (except filters)
- Line chart settings (except filters)
In progress:
- Pie chart settings
- Number chart settings
- Gauge chart settings
Left to do:
- Loosen the backend validation to allow the user to save a partial
configuration, validate the graph configuration in the frontend and
display a error friendly message in the graph if the config is not
completed yet
- Implement the filter edition
- Finish the other graph types settings
In this PR https://github.com/twentyhq/twenty/pull/14785 we deprecated
viewFilterOperand (from camelCase to capital snakeCase values), but we
had not migrated the existing workflow steps values. Hence they cannot
be executed!
Let's fix that by dealing with both operands, new and deprecated, to
mitigate the issue. Then we will add a command to migrate the values.
Following https://github.com/twentyhq/twenty/pull/14762
In this PR
- moved logic around any field filter formatting to twenty-shared
- added any field filter to the filters applied to groupBy when viewId
is defined
- added integration test
In the BE we have stored filter operand as IS_EMPTY, IS_NOT_EMPTY, etc.
For some reason in the FE we were manipulating IsEmpty, IsNotEmpty, etc.
(maybe because they were used before in Views before they were moved to
core)
So we were converting the operands in the FE from IS to Is
(convertViewFilterOperandFromCore) to read and manipulate viewFilters,
and then back to BE version to send mutations etc., from Is to IS
(convertViewFilterOperandToCore).
The migration is now over, so we can remove and simplify that code.
(In the 1-5:migrate-views-to-core command we still do the migration from
Is format to IS format.)
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
This PR should fix optimistic rendering issues on step updates:
- compute a diff for trigger and steps on mutations
- build an util that applies that diff (built a more robust version from
https://github.com/AsyncBanana/micropatch)
- apply diff in cache
Add a new graphql mutation in the pageLayout resolver to handle page
layout update with tabs and widgets.
Later we will add validation inside the service to check if the page
layout tab structure is correct (for instance check widgets don't
overlap or aren't out of the grid, or to check that they have a correct
configuration ...).
This mutation will be plugged to the save action of the dashboards in
the frontend.
## Problem
**CRITICAL:** Two PRs were accidentally reverted when PR #14347 "Prevent
csv export injections" was merged:
1. **PR #14348** "[Page Layout] - Review Refactor" - ✅ **RESTORED**
2. **PR #14352** "Fix wrong path used by backend" - ✅ **RESTORED**
## Root Cause Analysis
During the merge of PR #14347, there was a complex merge conflict with
PR #14352 "Fix wrong path used by backend". The merge commit
`324d7204bb` in the PR #14347 branch brought in changes from PR #14352,
but during the conflict resolution, **BOTH PR #14348 and PR #14352's
changes were accidentally overwritten**.
## What This PR Restores
This PR restores **BOTH** PRs by cherry-picking their commits:
### ✅ PR #14348 Changes Restored:
- `GraphWidgetRenderer.tsx` - was deleted, now restored
- `WidgetRenderer.tsx` - was missing, now restored
- `SettingsPageLayoutTabsInstanceId.ts` - was deleted, now restored
- `useUpdatePageLayoutWidget.ts` - was renamed back, now restored with
correct name
- Multiple test files that were deleted
- Several hook files that were renamed/reverted
- File renames: `usePageLayoutWidgetUpdate.ts` →
`useUpdatePageLayoutWidget.ts`
- Hook refactoring and test file organization
- Page layout component improvements
### ✅ PR #14352 Changes Restored:
- **Types moved to twenty-shared:**
- `packages/twenty-shared/src/types/AppBasePath.ts` ✅ RESTORED
- `packages/twenty-shared/src/types/AppPath.ts` ✅ RESTORED
- `packages/twenty-shared/src/types/SettingsPath.ts` ✅ RESTORED
- **Navigation utilities moved to twenty-shared:**
- `packages/twenty-shared/src/utils/navigation/getAppPath.ts` ✅ RESTORED
- `packages/twenty-shared/src/utils/navigation/getSettingsPath.ts` ✅
RESTORED
- **200+ import statements updated** across the codebase to use
twenty-shared
- **Old type files deleted** from twenty-front/src/modules/types/
## Evidence of Complete Restoration
**Before (reverted state):**
- ❌ Types were in `packages/twenty-front/src/modules/types/`
- ❌ Page layout files missing
- ❌ Hook files incorrectly named
**After (this PR):**
- ✅ Types correctly in `packages/twenty-shared/src/types/`
- ✅ All page layout files restored
- ✅ Hook files correctly named
- ✅ All import statements updated
## Verification
**Total changes:**
- PR #14348: 36 files changed, 863 insertions(+), 442 deletions(-)
- PR #14352: 243 files changed, 492 insertions(+), 461 deletions(-)
- **Combined: 279 files changed, 1355 insertions(+), 903 deletions(-)**
## Impact
This completely restores both PRs that were accidentally lost, ensuring:
1. Page layout refactoring work is back
2. Type organization and path utilities are correctly in twenty-shared
3. Backend email paths work correctly again
4. No functionality is lost
Fixes the reversion caused by the merge conflict in PR #14347.
---------
Co-authored-by: nitin <142569587+ehconitin@users.noreply.github.com>
**Small Security Issue:** CSV exports were vulnerable to formula
injection attacks when users entered values starting with =, +, -, or @.
(only happens if a logged-in user injects corrupted data)
Solution:
- Added ZWJ (Zero-Width Joiner) protection that prefixes dangerous
values with invisible Unicode character
- This is the best way to preserve original data while preventing Excel
from executing formulas
- Added import cleanup to restore original values when re-importing
Changes:
- New sanitizeValueForCSVExport() function for security
- Updated all CSV export paths to use both security + formatting
functions
- Added comprehensive tests covering attack vectors and international
characters
- Also added cursor rules for better code consistency
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
