# Introduction
Following https://github.com/twentyhq/twenty/pull/13264, this PR
introduces several `fieldMetadataEntity` typing enhancement suggestions.
Mainly any nullable field metadata entity properties are now either
nullable or defined.
Or never if field is dynamically required or not depending on the field
metadata type
This enhance DevX
## Standards
- field enum ( `MULTI_SELECT`, `SELECT`, `RATING` ) will never have
`options` set to `NULL` in db
- field `RELATION` or `MORH_RELATION` won't ever have its relation
fields set to `NULL` in db
- field of any type `settings`, even if possibly defined, can still be
`NULL` in db
- field of any type `defaultValue`, even if possibly defined, can still
be `NULL` in db
It coud be interesting to guard these standards by adding dedicated pg
constraints on each field
## TypesScript type tests
added coverage for each `settings`, `defaultValue`, and `options`
depending on the current `fieldMetadata`
Honestly I don' know if this typescript assertions test file is not
overkill, but regarding metadata staticness it might be very interesting
to have this guard
## Possible improvements
- We could type as `unknown` instead of "all" on `FieldMetadataType`
inferrance
- We still need to deprecate remaining duplicated entities such as
`Index/Field/MetadataInterface` etc not a huge refactor neither urgent
# Introduction
Following `FieldMetadataInterface` deprecation in
https://github.com/twentyhq/twenty/pull/13264
As for the previous PR will rename and remove all the file in a
secondary PR to avoid conflicts and over loading this one
## Improvements
Removed optional properties from the `objectMetadataEntity` model and
added utils to retrieve test data
## Notes
By touching to `ObjectMetadataDTO` I would have expected a twenty-front
codegenerated types mutation, but it does not seem to be granular enough
to null/undefined coercion
# Introduction
From the moment replaced the FieldMetadataInterface definition to:
```ts
import { FieldMetadataType } from 'twenty-shared/types';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
export type FieldMetadataInterface<
T extends FieldMetadataType = FieldMetadataType,
> = FieldMetadataEntity<T>;
```
After this PR merge will create a new one removing the type and
replacing it to `FieldMetadataEntity`.
Did not renamed it here to avoid conflicts on naming + type issues fixs
within the same PR
## Field metadata entity RELATION or MORPH
Relations fields cannot be null for those field metadata entity instance
anymore, but are never for the others see
`packages/twenty-server/src/engine/metadata-modules/field-metadata/types/field-metadata-entity-test.type.ts`
( introduced TypeScript tests )
## Concerns
- TS_VECTOR is the most at risk with the `generatedType` and
`asExpression` removal from interface
## What's next
- `FielMetadataInterface` removal and rename ( see introduction )
- Depcrecating `ObjectMetadataInterface`
- Refactor `FieldMetadataEntity` optional fiels to be nullable only
- TO DIG `never` occurences on settings, defaultValue etc
- Some interfaces will be replaced by the `FlatFieldMetadata` when
deprecating the current sync and comparators tools
## Context
Add an eventEmitter instance to twenty datasources so we can emit DB
events.
Add input and output formatting to twenty orm (formatData, formatResult)
Those 2 elements simplified existing logic when we interact with the
ORM, input will be formatted by the ORM so we can directly use
field-like structure instead of column-like. The output will be
formatted, for builder queries it will be in `result.generatedMaps`
where `result.raw` preserves the previous column-like structure.
Important change: We now have an authContext that we can pass when we
get a repository, this will be used for the different events emitted in
the ORM. We also removed the caching for repositories as it was not
scaling well and not necessary imho
Note: An upcoming PR should handle the onDelete: cascade behavior where
we send DESTROY events in cascade when there is an onDelete: CASCADE on
the FK.
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
In this PR:
## Improve recompute metadata cache performance. We are aiming for
~100ms
Deleting relationMetadata table and FKs pointing on it
Fetching indexMetadata and indexFieldMetadata in a separate query as
typeorm is suboptimizing
## Remove caching lock
As recomputing the metadata cache is lighter, we try to stop preventing
multiple concurrent computations. This also simplifies interfaces
## Introduce self recovery mecanisms to recompute cache automatically if
corrupted
Aka getFreshObjectMetadataMaps
## custom object resolver performance improvement: 1sec to 200ms
Double check queries and indexes used while creating a custom object
Remove the queries to db to use the cached objectMetadataMap
## reduce objectMetadataMaps to 500kb
<img width="222" alt="image"
src="https://github.com/user-attachments/assets/2370dc80-49b6-4b63-8d5e-30c5ebdaa062"
/>
We used to stored 3 fieldMetadataMaps (byId, byName, byJoinColumnName).
While this is great for devXP, this is not great for performances.
Using the same mecanisme as for objectMetadataMap: we only keep byIdMap
and introduce two otherMaps to idByName, idByJoinColumnName to make the
bridge
## Add dataloader on IndexMetadata (aka indexMetadataList in the API)
## Improve field resolver performances too
## Deprecate ClientConfig
# Fix cursor-based pagination with lexicographic ordering for composite
fields
## Bug
The existing cursor-based pagination implementation had a bug when
handling composite fields.
When paginating through results sorted by composite fields (like
`fullName` with sub-properties `firstName` and`lastName`), the WHERE
conditions generated for cursor positioning were incorrect, leading to
records being skipped.
The previous implementation was generating wrong WHERE conditions:
For example, when paginating with a cursor like `{ firstName: 'John',
lastName: 'Doe' }`, it would generate:
```sql
WHERE firstName > 'John' AND lastName > 'Doe'
```
This is incorrect because it would miss records like `{ firstName:
'John', lastName: 'Smith' }` which should be included in forward
pagination.
## Fix
Create a new util to use proper lexicographic order when sorting a
composite field.
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
Added a no-explicit-any rule to the twenty-server, not applicable to
tests and integration tests folder
Related to https://github.com/twentyhq/core-team-issues/issues/975
Discussed with Charles
## In case of conflicts
Until this is approved I won't rebased and handle conflict, just need to
drop two latest commits and re run the scripts etc
## Legacy
We decided not to handle the existing lint error occurrences and
programmatically ignored them through a disable next line rule comment
## Open question
We might wanna activate the
[no-explicit-any](https://typescript-eslint.io/rules/no-explicit-any/)
`ignoreRestArgs` for our use case ?
```
ignoreRestArgs?: boolean;
```
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
