## Introduction
After enabling flag by default got following errors:
```ts
Test Suites: 48 failed, 1 skipped, 97 passed, 145 of 146 total
Tests: 499 failed, 1 skipped, 644 passed, 1144 total
Snapshots: 61 failed, 133 passed, 194 total
Time: 363.226 s
Ran all test suites.
```
## From
<img width="2952" height="1510" alt="image"
src="https://github.com/user-attachments/assets/7e3b20c6-2552-40a7-90bb-2d7b3002c895"
/>
## To
<img width="3134" height="1510" alt="image"
src="https://github.com/user-attachments/assets/4fc9ada4-3c14-4333-a1db-11daf87db8d6"
/>
There's a huge test bundle in the latest shard that we could split up
## Notes
- Set as failing morph relation field rename as for the moment we do not
handle relation field mutation
- fixed the object update and creation validation adding label
identifier field metadata id checks
- and more
Some integrations tests are still on the v1 ( they have before and after
all disabling and re-enabling the flat ) but mainly we now have more
coverage on the v2 than the v1.
Mainly related records, uniqueness have to be migrated the v2 and so
tests too
## ✨ Add accent-insensitive search functionality
### 🎯 Overview
Implements accent-insensitive search across all searchable fields in
Twenty CRM.
Users can now search for "jose" to find "José", "muller" to find
"Müller", "cafe" to find "café", etc.
### 🔍 Problem
Twenty's search functionality was accent-sensitive, requiring users to
type exact accented characters to find records.
This created a poor user experience, especially for international names
and content.
### 💡 Solution
Added PostgreSQL `unaccent` extension with a custom immutable wrapper
function to enable accent-insensitive full-text search across all
searchable field types.
### 📋 Changes Made
**Modified Files:**
- `packages/twenty-server/scripts/setup-db.ts`
-
`packages/twenty-server/src/engine/api/graphql/graphql-query-runner/utils/compute-where-condition-parts.ts`
-
`packages/twenty-server/src/engine/workspace-manager/workspace-sync-metadata/utils/get-ts-vector-column-expression.util.ts`
### 🗄️ Database Setup (`setup-db.ts`)
```sql
-- Added unaccent extension
CREATE EXTENSION IF NOT EXISTS "unaccent";
-- Created immutable wrapper function
CREATE OR REPLACE FUNCTION unaccent_immutable(text) RETURNS text AS $$
SELECT public.unaccent($1)
$$ LANGUAGE sql IMMUTABLE;
```
### 🔍 Search Vector Generation
(`get-ts-vector-column-expression.util.ts`)
Applied `public.unaccent_immutable()` to all searchable field types:
- TEXT fields (job titles, names, etc.)
- FULL_NAME fields (first/last names)
- EMAILS fields (both email address and domain)
- ADDRESS fields
- LINKS fields
- RICH_TEXT and RICH_TEXT_V2 fields
### 🔎 Query Processing (`compute-where-condition-parts.ts`)
Enhanced search queries to use `public.unaccent_immutable()` for both:
- Full-text search (`@@` operator with `to_tsquery`)
- Pattern matching (`ILIKE` operator)
### 🧠 Technical Rationale: Why the Wrapper Function?
**The Challenge:**
PostgreSQL's built-in `unaccent()` is marked as **STABLE**, but
`GENERATED ALWAYS AS` expressions (used for search vector columns)
require **IMMUTABLE** functions.
**The Solution:**
Created an IMMUTABLE wrapper function that calls the underlying
`unaccent()` function:
- ✅ Satisfies PostgreSQL's immutability requirements for generated
columns
- ✅ Maintains the exact same functionality as the original `unaccent()`
- ✅ Uses fully qualified `public.unaccent_immutable()` to ensure
function resolution from workspace schemas
**Alternative Approaches Considered:**
- ❌ Modifying `search_path`: would affect workspace isolation
- ❌ Computing unaccent at query time: would hurt performance
- ❌ Using triggers: would complicate data consistency
### 🎯 Impact
For **Person** records, accent-insensitive search now works on:
- Name (first/last name): `"jose garcia"` finds `"José García"`
- Email: `"[email protected]"` finds `"josé@café.com"`
- Job Title: `"manager"` finds `"Managér"` or `"Gerente de Café"`
Applies to all searchable standard objects:
- Companies, People, Opportunities, Notes, Tasks, etc.
- Any custom fields of searchable types (TEXT, EMAILS, etc.)
### ✅ Testing
- Database reset completes successfully
- Workspace seeding works without errors
- Search vectors generate with unaccent functionality
- All searchable field types properly handle accented characters
---------
Co-authored-by: Félix Malfait <[email protected]>
# What
Fully deprecate old relations because we have one bug tied to it and it
make the codebase complex
# How I've made this PR:
1. remove metadata datasource (we only keep 'core') => this was causing
extra complexity in the refactor + flaky reset
2. merge dev and demo datasets => as I needed to update the tests which
is very painful, I don't want to do it twice
3. remove all code tied to RELATION_METADATA /
relation-metadata.resolver, or anything tied to the old relation system
4. Remove ONE_TO_ONE and MANY_TO_MANY that are not supported
5. fix impacts on the different areas : see functional testing below
# Functional testing
## Functional testing from the front-end:
1. Database Reset ✅
2. Sign In ✅
3. Workspace sign-up ✅
5. Browsing table / kanban / show ✅
6. Assigning a record in a one to many / in a many to one ✅
7. Deleting a record involved in a relation ✅ => broken but not tied to
this PR
8. "Add new" from relation picker ✅ => broken but not tied to this PR
9. Creating a Task / Note, Updating a Task / Note relations, Deleting a
Task / Note (from table, show page, right drawer) ✅ => broken but not
tied to this PR
10. creating a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
11. updating a relation from settings should not be possible ✅
12. deleting a relation from settings (custom / standard x oneToMany /
manyToOne) ✅
13. Make sure timeline activity still work (relation were involved
there), espacially with Task / Note => to be double checked ✅ => Cannot
convert undefined or null to object
14. Workspace deletion / User deletion ✅
15. CSV Import should keep working ✅
16. Permissions: I have tested without permissions V2 as it's still hard
to test v2 work and it's not in prod yet ✅
17. Workflows global test ✅
## From the API:
1. Review open-api documentation (REST) ✅
2. Make sure REST Api are still able to fetch relations ==> won't do, we
have a coupling Get/Update/Create there, this requires refactoring
3. Make sure REST Api is still able to update / remove relation => won't
do same
## Automated tests
1. lint + typescript ✅
2. front unit tests: ✅
3. server unit tests 2 ✅
4. front stories: ✅
5. server integration: ✅
6. chromatic check : expected 0
7. e2e check : expected no more that current failures
## Remove // Todos
1. All are captured by functional tests above, nothing additional to do
## (Un)related regressions
1. Table loading state is not working anymore, we see the empty state
before table content
2. Filtering by Creator Tim Ap return empty results
3. Not possible to add Tasks / Notes / Files from show page
# Result
## New seeds that can be easily extended
<img width="1920" alt="image"
src="https://github.com/user-attachments/assets/d290d130-2a5f-44e6-b419-7e42a89eec4b"
/>
## -5k lines of code
## No more 'metadata' dataSource (we only have 'core)
## No more relationMetadata (I haven't drop the table yet it's not
referenced in the code anymore)
## We are ready to fix the 6 months lag between current API results and
our mocked tests
## No more bug on relation creation / deletion
---------
Co-authored-by: Weiko <[email protected]>
Co-authored-by: Félix Malfait <[email protected]>
The DX is not great when you need to do a lot of database
resets/command.
Should we disable Typescript validation to speed things up? With this
and caching database:reset takes 1min instead of 2 on my machine.
See also: https://github.com/typeorm/typeorm/issues/4136
And #9291 / #9293
---------
Co-authored-by: Lucas Bordeau <[email protected]>
We will remove the `twenty-postgres` image that was used for local
development and only use `twenty-postgres-pilo` (which we use in prod),
bringing the development environment closer to prod and avoiding having
to maintain 2 images.
Instead of provisioning the super user after the db initialization, we
directly rely on the superuser provided by Spilo for simplicity. We also
introduce a change that tries to create the right database (`default` or
`test`) based on the context.
How to test:
```
docker build -t twentycrm/twenty-postgres-spilo:latest -f ./packages/twenty-docker/twenty-postgres-spilo/Dockerfile .
docker images --no-trunc | grep twenty-postgres-spilo
postgres-on-docker:
docker run \
--name twenty_pg \
-e PGUSER_SUPERUSER=twenty \
-e PGPASSWORD_SUPERUSER=twenty \
-e ALLOW_NOSSL=true \
-v twenty_db_data:/home/postgres/pgdata \
-p 5432:5432 \
REPLACE_WITH_IMAGE_ID
```
## Context
This PR removes pg_graphql from the setup. It also updates the local
setup documentation accordingly.
Note: We removed local setup scripts to align with redis installation,
the setup should be much simpler since we don't rely on pg_graphql
anymore.
## Test
tested locally with docker + mac (brew) setup
## Context
We recently enabled the option to bypass SSL certificate authority
validation when establishing a connection to PostgreSQL. Previously, if
this validation failed, the server would revert to unencrypted traffic.
Now, it maintains encryption even if the SSL certificate check fails. In
the process, we overlooked a few DataSource setups, prompting a review
of DataSource creation within our code.
## Current State
Our DataSource initialization is distributed as follows:
- **Database folder**: Contains 'core', 'metadata', and 'raw'
DataSources. The 'core' and 'metadata' DataSources manage migrations and
static resolver calls to the database. The 'raw' DataSource is utilized
in scripts and commands that require handling both aspects.
- **typeorm.service.ts script**: These DataSources facilitate
multi-schema connections.
## Vision for Discussion
- **SystemSchema (formerly core) DataSource**: Manages system schema
migrations and system resolvers/repos. The 'core' schema will be renamed
to 'system' as the Core API will include parts of the system and
workspace schemas.
- **MetadataSchema DataSource**: Handles metadata schema migrations and
metadata API resolvers/repos.
- **(Dynamic) WorkspaceSchema DataSource**: Will be used in the Twenty
ORM to access a specific workspace schema.
We currently do not support cross-schema joins, so maintaining these
DataSources separately should be feasible. Core API resolvers will
select the appropriate DataSource based on the field context.
- **To be discussed**: The potential need for an AdminDataSource (akin
to 'Raw'), which would be used in commands, setup scripts, and the admin
panel to connect to any database schema without loading any model. This
DataSource should be reserved for cases where utilizing metadata,
system, or workspace entities is impractical.
## In This PR
- Ensuring all existing DataSources are compliant with the SSL update.
- Introducing RawDataSource to eliminate the need for declaring new
DataSource() instances in commands.
