# Introduction
Honestly this implem is a mess, discussing a potential side effect
handler with @weiko before the build and run that would handle each side
effect per entity and operation
Handling:
- [ ] unique index is generated when a field is updated with the
`isUnique`
- [x] an index is generated when a relation is created
- [x] search vector index creation on custom object creation
- [x] renaming a field metadata or an object should re-create all
related indexes which are composed by their namings
- [x] delete object should remove any related indexes
- [x] delete field should update related indexes ( if index ends up
empty it should be removed )
- [ ] on object renaming that contains morph fields -> triggers update
field -> trigger index recompute
- [x] on update name renaming should recompute all related indexes
## TODO
- [x] Integration testing
- [ ] Refactor the index maps cache to be storing a
`idsByObjectMetadataId`
- [x] Refactor deterministic name to use order sorting
- [x] Remove flat index from flat object
## What's next
Will handle morph indexes in a new dedicated PR for the moment will
stick to this
Same for the cache improvement and uniqueness
- Removed buffer to string conversion on message and we instead pass the
direct Buffer
- Removed redundant flattening of results
- Cached `JSDOM` and `DOMPurify` Instance
- MailParser added `skipTextToHtml` along with some other options, which
reduced parsing time from 50ms to 35ms
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
## Context
We are not exactly sure how we want to handle side effects with
workspace migration v2, in the meantime I'm re-introducing this part in
the caller (object v2 service)
## Summary
Enhanced the search resolver integration tests to comprehensively
validate bidirectional accent-insensitive search functionality. Users
can now search with or without accents and find all relevant records
regardless of how the data was originally stored.
## Changes Made
### Test Data Enhancements
- Added 2 new person IDs: `TEST_PERSON_6_ID`, `TEST_PERSON_7_ID`
- Expanded corpus from 5 to 13 records:
- 7 persons
- 2 companies
- 4 pets
- Refactored test data into single arrays with content-driven variable
naming for better maintainability
### Test Coverage Improvements
- Added 6 comprehensive bidirectional tests covering French, Spanish,
and German characters
- Validated multiple fields:
- `firstName`, `lastName`, `jobTitle`, `email`, `company name`, `pet
name`
- Cross-object search testing across Person, Company, and Pet
- Updated all ranking values to account for the enlarged test corpus (5
→ 13 records)
## Test Cases Added/Updated
- [x] should find both **"José"** and **"Jose"** when searching for
`"jose"` (bidirectional accent-insensitive)
- [x] should find both **"García"** and **"Garcia"** when searching for
`"garcia"` (bidirectional accent-insensitive)
- [x] should find both accented and non-accented **"Café"** / **"Cafe"**
records when searching for `"cafe"` (bidirectional accent-insensitive)
- [x] should find both accented and non-accented **"Naïve"** /
**"Naive"** records when searching for `"naive"` (bidirectional
accent-insensitive)
- [x] should find both **"Müller"** and **"Muller"** when searching for
`"muller"` (bidirectional accent-insensitive)
- [x] should find both **"François"** and **"Francois"** when searching
for `"francois"` (bidirectional accent-insensitive)
## Summary
Implements the foundation for dynamic search field configuration from
[issue #1428](https://github.com/twentyhq/core-team-issues/issues/1428).
## Changes
- Add `SearchFieldMetadataEntity` junction table for storing searchable
field configurations
- Add `SearchFieldMetadataService` with core CRUD operations
- Add `SearchFieldMetadataModule` following existing patterns
- Add `IS_DYNAMIC_SEARCH_FIELDS_ENABLED` feature flag (defaults to
`false`)
- Database migration with proper indexes and foreign keys
## Architecture
Uses a junction table where **record existence = field is searchable**:
**Table: `searchFieldMetadata(objectMetadataId, fieldMetadataId,
workspaceId)`**
- Unique constraint on `(objectMetadataId, fieldMetadataId)`
- `ON DELETE CASCADE` on foreign keys
## Testing
- [x] Migration runs successfully
- [x] Table created with correct schema
- [x] Feature flag seeded properly
- [x] Database reset works correctly
## Next Steps
Future PRs will handle:
- Data migration from existing hardcoded search field configs
- Integration with search services
✅ No breaking changes — fully backward compatible.
## Context
While refactoring messaging, I forgot that workspaces that have been
created before july do not have 'CALENDAR_EVENT_LIST_FETCH_PENDING' and
'MESSAGE_LIST_FETCH_PENDING' enum values (as we don't sync-metadata the
enum values, we would need a migrate command that has not been written
yet)
In the current implementation CALENDAR_EVENT_LIST_FETCH_PENDING =
FULL_CALENDAR_EVENT_LIST_FETCH_PENDING, so I'm putting it back as it was
before
## Context
1. We are debugging the experience on messaging and calendar sync. I'm
adding two scripts to relaunch messaging and calendar channels
2. We are seeing errors in Sentry regarding microsoft drivers errors. I
was about to fix them but I'm refactoring / simplifying a bit the logic
first. It will be a betters starting point
Fixes https://github.com/twentyhq/twenty/issues/14442
Issues were
1. Table headers always used label metadata identifier (or record text)
as first column, while table body followed viewFields positions
2. Label metadata identifier should always be visible and in first
position for the table views to work as expected, while when updating
the label metadata identifier for an object, no changes were brought to
the viewFields
To fix that
1. In the BE:
- a new logic is implemented to i) update label identifier's viewFields
position and visibility when an object's label identifier is updated to
a new field; ii) add validation on the update of a viewField's position
and visibility to make sure the label identifier's viewField always has
the lowest position + is visible
- a command was added to check all existing views and viewfields
3. In the FE: at first I tried to replicate the logic of the headers
(based on the label identifier rather than the positions) on the body,
but it was too complex and error-prone as in multiple places we are
based on the positions. It also feels more right to have only one source
of truth which is the viewField position. @lucasbordeau if that does not
suit you, we can throw an error if the field with the lowest position is
not the label metadata identifier, as you said the table view will be
very buggy / wont work if the label identifier is not in the first
position. but now it should never be the case thanks to the validation
implemented in the BE
This should be migrated to viewFieldService V2 when relevant @Weiko
@prastoin
Fixes:
- relation one to many cannot be set (we have deprecated limit: keyword
in graphql at it was not use, we should only use first or last)
- https://github.com/twentyhq/twenty/issues/14556
It was in setup-db.ts but that's not enough for instances that are
already live
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
## Introduction
https://github.com/twentyhq/twenty/pull/14363
In this PR we're creating builder and runner for index metadata
Removing its previous integration within object builder
## Next:
- implem index impact on field and object metadata api
- Add integration testing coverage add integration test on index
- Implement uniqueness toggle in field metadata api
- add integration test on uniqueness
related to https://github.com/twentyhq/core-team-issues/issues/1344
Tested :
- workspace needs to flush cache after activating feature flag
- workspace does not need to flush cache if ff false and coming from
previous version (before gql schema gen updates)
## ✨ Add accent-insensitive search functionality
### 🎯 Overview
Implements accent-insensitive search across all searchable fields in
Twenty CRM.
Users can now search for "jose" to find "José", "muller" to find
"Müller", "cafe" to find "café", etc.
### 🔍 Problem
Twenty's search functionality was accent-sensitive, requiring users to
type exact accented characters to find records.
This created a poor user experience, especially for international names
and content.
### 💡 Solution
Added PostgreSQL `unaccent` extension with a custom immutable wrapper
function to enable accent-insensitive full-text search across all
searchable field types.
### 📋 Changes Made
**Modified Files:**
- `packages/twenty-server/scripts/setup-db.ts`
-
`packages/twenty-server/src/engine/api/graphql/graphql-query-runner/utils/compute-where-condition-parts.ts`
-
`packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/utils/get-ts-vector-column-expression.util.ts`
### 🗄️ Database Setup (`setup-db.ts`)
```sql
-- Added unaccent extension
CREATE EXTENSION IF NOT EXISTS "unaccent";
-- Created immutable wrapper function
CREATE OR REPLACE FUNCTION unaccent_immutable(text) RETURNS text AS $$
SELECT public.unaccent($1)
$$ LANGUAGE sql IMMUTABLE;
```
### 🔍 Search Vector Generation
(`get-ts-vector-column-expression.util.ts`)
Applied `public.unaccent_immutable()` to all searchable field types:
- TEXT fields (job titles, names, etc.)
- FULL_NAME fields (first/last names)
- EMAILS fields (both email address and domain)
- ADDRESS fields
- LINKS fields
- RICH_TEXT and RICH_TEXT_V2 fields
### 🔎 Query Processing (`compute-where-condition-parts.ts`)
Enhanced search queries to use `public.unaccent_immutable()` for both:
- Full-text search (`@@` operator with `to_tsquery`)
- Pattern matching (`ILIKE` operator)
### 🧠 Technical Rationale: Why the Wrapper Function?
**The Challenge:**
PostgreSQL's built-in `unaccent()` is marked as **STABLE**, but
`GENERATED ALWAYS AS` expressions (used for search vector columns)
require **IMMUTABLE** functions.
**The Solution:**
Created an IMMUTABLE wrapper function that calls the underlying
`unaccent()` function:
- ✅ Satisfies PostgreSQL's immutability requirements for generated
columns
- ✅ Maintains the exact same functionality as the original `unaccent()`
- ✅ Uses fully qualified `public.unaccent_immutable()` to ensure
function resolution from workspace schemas
**Alternative Approaches Considered:**
- ❌ Modifying `search_path`: would affect workspace isolation
- ❌ Computing unaccent at query time: would hurt performance
- ❌ Using triggers: would complicate data consistency
### 🎯 Impact
For **Person** records, accent-insensitive search now works on:
- Name (first/last name): `"jose garcia"` finds `"José García"`
- Email: `"jose@cafe.com"` finds `"josé@café.com"`
- Job Title: `"manager"` finds `"Managér"` or `"Gerente de Café"`
Applies to all searchable standard objects:
- Companies, People, Opportunities, Notes, Tasks, etc.
- Any custom fields of searchable types (TEXT, EMAILS, etc.)
### ✅ Testing
- Database reset completes successfully
- Workspace seeding works without errors
- Search vectors generate with unaccent functionality
- All searchable field types properly handle accented characters
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
# Introduction
Initially this PR was about introducing integration coverage for view
field v2 tests
But feature itself wasn't finished, so ended up in a TDD style
devlopment
## The new orchestrator
Orchestrator will organize each flat entity diffing inferring either
validation errors or actions sequentially.
A new builder has been introduced when extended requires to define flat
entity validation utils and action generation for `delete` `update`
`create` operations. It's highly typed using generics
## View field integration testing
Introduce coverage on failing basic tests cases for view field
operations, successfull create is also covered. We could still add more
coverage, will be done later in following PRs
## Remaining tasks for upcoming PRs:
- [ ] rename builder methods names to add generate action in their names
- [x] refactor view validation to handle field and view uniqueness index
combination
- [ ] implem strict update validator on view and view field
- [ ] dynamic cache invalidation post run
- [ ] Add coverage to successful and failing view field operations
- [ ] error formatting summary computation
- [ ] extract types out of the workspace entity builder
## Some vision
Currently flat field metadata maps is nested in the flat object metadata
maps, making its build nested too. ( not extending the new generic
builder )
We will refactor this part for both to be stored extra flat such as the
other existing flat entities.
It will require a small refactor to the generic builder that will not
only have impact on the currently built flat entity optimistic cache but
to an other one ( for example when building a create object action it
will also have an impact flat field maps )
