# Implement Morph Frontend Settings
Possiblity to create a morh relation from the Settings !
## Overview
Morph object system with new frontend settings UI for configuring morph
relations.
## What Changed
- **New Components**: `SettingsDataModelFieldMorphRelationForm`,
`SettingsMorphRelationMultiSelect`
- **Form System**: React Hook Form + Zod validation for morph relation
configuration
- **Preview System**: Live preview of field configurations with
validation
- **State Management**: Enhanced component state handling for complex
form interactions
## Technical Details
- **Architecture**: Restructured field input/display components for
better maintainability
- **Types**: Added comprehensive TypeScript types for morph relations
(`isFieldMorphRelation`, etc.)
- **Validation**: Dynamic validation rules based on field types and
relation patterns
- **Performance**: Optimized rendering with proper state management
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
When the target is not in the direct next step ids of the source (step
or trigger), check if there is a filter between both. If yes, delete the
filter and remove it from source next step ids.
Also refactored the existing.
Unless I'm mistaken the project does not run with node `24.0.0`
Switching to node `24.5.0` ( as defined in vscode node runtime
requirements in https://github.com/twentyhq/twenty/pull/13730 ) seems to
fix the issue
```ts
Successfully compiled: 2897 files with swc (188.32ms)
(node:77006) [DEP0190] DeprecationWarning: Passing args to a child process with shell option true can lead to security vulnerabilities, as the arguments are not escaped, only concatenated.
(Use `node --trace-deprecation ...` to show where the warning was created)
Watching for file changes.
/Users/paulrastoin/ws/twenty/node_modules/buffer-equal-constant-time/index.js:37
var origSlowBufEqual = SlowBuffer.prototype.equal;
^
TypeError: Cannot read properties of undefined (reading 'prototype')
```
Updating engines so local constraint suggest a functional node version
In this PR
- Introduction of readableFields and updatableFields in
objectMetadataItem selector to ease filtering from a developer
experience perspective ( + to help developers think to do it). In
discussion @lucasbordeau @charlesBochet
- Remove non-updatable field from CSV import process (@etiennejouan)
- QA fix / Non-readable fields should not show on show page
- QA fix / It should not be offered to create a kanban view on a
non-readable field
- QA fix / It should not be offered to create view groups on a
non-readable field
- QA fix / Rating field should have a readonly mode
---------
Co-authored-by: Charles Bochet <charles@twenty.com>
# Introduction
The workspace migration runner v2 now computes the next
flatObjectMetadataMaps post current ws migration action has been run
So the following one have an up to date informations
## Still TODO
~~Remove any contextual information from workspace migration v2 and
consume the optimistic cache to retrieve them~~
## Next steps
- Create flat-object-metadata-maps testing toolbox and flat-object/field
testing toolbox
- Implement strong coverage on every created utils/transpiler and
applyWorkspaceMigrationAction method
# Introduction
- Retrieveing flat-object-metadata-maps for input transpilation, we
decided not to introduce map merge for remaining validation operation
for the moment
- Refactored the field metadata service v2 to sequentially
optimistically render on each iteration
# Introduction
The cache is not accurately typed regarding the `fieldMetadataMaps` they
do not contain relations
In this way we:
- Created dedicated transpilation tools for cache manipulation
- Avoided recursivity in transpilation tools
- Fix and finalized the createMany fields metadata service v2
- Refactor the validator to handle validation :)
## What's next:
- Implem integration tests to make things run 🙃
- migrate existing settings valdiation
- Finish the create object metadata service
- Handle update input transpilation and validation
# Introduction
- FlatFieldMetadataType validators hashmap
- Do not fail fast on validation but aggregate errors
- Implemented `enum` validation
- Plugged the new v2 dynamic call in the field metadata service v2
## What's next:
- Implem integration tests to make things run 🙃
- migrate existing settings valdiation
- Finish the create object metadata service
- Handle update input transpilation and validation
## Open question
Should we implement, not covered validation ?, adding strictness now or
never.
This will be required by the import anw
## Discovered issue with cache
Currently the cache is not accurately typed, `fieldsById` map are not
storing any relations.
Which means the current transpilation tools are hitting undefined at
runtime
In the best of the world we will refactor the cache to be storing
`FlatObjectMetadata` and `FlatFieldMetadata` so we don't even have to
transpile them for validation and so on
But it would require to refactor the loaders that returns the cache to
the front on hit as FieldMetadataEntity, so we might land on a lighter
solution to rather add a new `getExistingFlatCache` that handles the
transpilation itself
About to do that in an other PR to be discussed with Coco
# Introduction
Following https://github.com/twentyhq/twenty/pull/13420
What has been done:
- `CreateFieldInput` transpilation to `FlatFieldMetadata`
- `FlatFieldMetadata` validator service
- A lof of transpilation utils from `input` to `flatObject` or
`flatField`
- Created dedicated v2 api metadata services
- Introducing `inferDeletionFromMissingObjectFieldIndex` in the builder,
to avoid diffing every object and field of the current workspace we
allow only generating create/update migration operations, usefull when
passing by the api metadata
## We still need to in another PR:
- Implement a strong unit test coverage and critical functions and
services
- Finalize flat field metadata validation exception for `options`
`defaultValue` `settings` and `relations`
- Finalize `flatObjectMetadata` validation and v2 service refactor
- Plug the new service when feature flag is enabled
# Introduction
In this PR we create basic transpilation methods and utils to handle
input to flat, entity to flat, object maps to flat. In order to
transpile everything into a common validation that will be implemented
in another PR
## FieldMetadataEntity typing
Added `never | null` to fields that should never be in order to ease
general abstracted method to pass null, as anw it's what is in the
database
## Todo
- ~~Create a feature flag~~
- Integration test for object creation through metadata api + pg col
introspection and snapshoting
- add to step output schema the information that field is a composite
sub field
- from output schema, when selecting the variable, copy all info
required to stepFilter
- from stepFilter, when selecting a value, display a special component
for composites
Context :
Large PR with 600+ test files. Enable connect and disconnect logic in
createMany (upsert true) / updateOne / updateMany resolvers
- Add disconnect logic
- Gather disconnect and connect logic -> called relation nested queries
- Move logic to query builder (insert and update one) with a preparation
step in .set/.values and an execution step in .execute
- Add integration tests
Test :
- Test API call on updateMany, updateOne, createMany (upsert:true) with
connect/disconnect
Implementation is very simple
Established authentication dynamic is intercepted at
getAuthTokensFromLoginToken. If 2FA is required, a pattern similar to
EmailVerification is executed. That is, getAuthTokensFromLoginToken
mutation fails with either of the following errors:
1. TWO_FACTOR_AUTHENTICATION_VERIFICATION_REQUIRED
2. TWO_FACTOR_AUTHENTICATION_PROVISION_REQUIRED
UI knows how to respond accordingly.
2FA provisioning occurs at the 2FA resolver.
2FA verification, currently only OTP, is handled by auth.resolver's
getAuthTokensFromOTP
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@twenty.com>
Co-authored-by: Jean-Baptiste Ronssin <65334819+jbronssin@users.noreply.github.com>
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
- add fieldMetadataId to step output schema
- use it to display FormFieldInput in Filter input
- few fixes for a few fields
Next step:
- Handle composite fields
- Design review
# Introduction
Following https://github.com/twentyhq/twenty/pull/13264, this PR
introduces several `fieldMetadataEntity` typing enhancement suggestions.
Mainly any nullable field metadata entity properties are now either
nullable or defined.
Or never if field is dynamically required or not depending on the field
metadata type
This enhance DevX
## Standards
- field enum ( `MULTI_SELECT`, `SELECT`, `RATING` ) will never have
`options` set to `NULL` in db
- field `RELATION` or `MORH_RELATION` won't ever have its relation
fields set to `NULL` in db
- field of any type `settings`, even if possibly defined, can still be
`NULL` in db
- field of any type `defaultValue`, even if possibly defined, can still
be `NULL` in db
It coud be interesting to guard these standards by adding dedicated pg
constraints on each field
## TypesScript type tests
added coverage for each `settings`, `defaultValue`, and `options`
depending on the current `fieldMetadata`
Honestly I don' know if this typescript assertions test file is not
overkill, but regarding metadata staticness it might be very interesting
to have this guard
## Possible improvements
- We could type as `unknown` instead of "all" on `FieldMetadataType`
inferrance
- We still need to deprecate remaining duplicated entities such as
`Index/Field/MetadataInterface` etc not a huge refactor neither urgent
# Introduction
From the moment replaced the FieldMetadataInterface definition to:
```ts
import { FieldMetadataType } from 'twenty-shared/types';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
export type FieldMetadataInterface<
T extends FieldMetadataType = FieldMetadataType,
> = FieldMetadataEntity<T>;
```
After this PR merge will create a new one removing the type and
replacing it to `FieldMetadataEntity`.
Did not renamed it here to avoid conflicts on naming + type issues fixs
within the same PR
## Field metadata entity RELATION or MORPH
Relations fields cannot be null for those field metadata entity instance
anymore, but are never for the others see
`packages/twenty-server/src/engine/metadata-modules/field-metadata/types/field-metadata-entity-test.type.ts`
( introduced TypeScript tests )
## Concerns
- TS_VECTOR is the most at risk with the `generatedType` and
`asExpression` removal from interface
## What's next
- `FielMetadataInterface` removal and rename ( see introduction )
- Depcrecating `ObjectMetadataInterface`
- Refactor `FieldMetadataEntity` optional fiels to be nullable only
- TO DIG `never` occurences on settings, defaultValue etc
- Some interfaces will be replaced by the `FlatFieldMetadata` when
deprecating the current sync and comparators tools
This PR does not produce any functional changes for our users. It
prepares the branches for workflows by:
- decommissioning `output` and `context` fields or `workflowRun` records
and use newly created `state` field from front-end and back-end
- use `stepStatus` computed by `back-end` in `front-end`
- add utils and types in `twenty-shared/workflow` (not completed, a
follow-up is scheduled
https://github.com/twentyhq/core-team-issues/issues/1211)
- add concurrency to `workflowQueue` message queue to avoid weird branch
execution when using forms in workflow branches
- add a WithLock decorator for better dev experience of
`CacheLockService.withLock` usage
Here is an example of such a workflow running (front branch display is
not yet done that's why it looks ugly) ->
https://discord.com/channels/1130383047699738754/1258024460238192691/1392897615171158098
# Introduction
Introduced `EachTesting` pattern for the builder unit tests.
As always any suggestions are more than welcomed !
Still need to:
- [x] implem basic tests for field
- [x] create `get-flat-index-field-metadata.mock.ts`
- [x] Implement basic tests for index and index-fields
- [ ] Implem standard edges cases tests TDD style
## Misc
- was https://github.com/twentyhq/twenty/pull/13132 closed due to mess
to rebase on main
Step operand will more or less be the same as view filter operand.
This PR:
- moves `ViewFilterOperand` to twenty-shared
- use it as step operand
- check what operand should be available based on the selected field
type in filter action
- rewrite the function that evaluates filters so it uses
ViewFilterOperand instead
ViewFilterOperand may be renamed in a future PR.
I rebuilt the advanced filters used in views and workflow search for a
specific filter step.
Components structure remains the same, using `stepFilterGroups` and
`stepFilters`. But those filters are directly sent to backend.
Also re-using the same kind of states we use for advanced filters to
share the current filters used. And a context to share what's coming
from workflow props (function to update step settings and readonly)
⚠️ this PR only focusses on the content of the step. There is still a
lot to do on the filter icon behavior in the workflow
https://github.com/user-attachments/assets/8a6a76f0-11fa-444a-82b9-71fc96b18af4
In this PR
- introduction of fieldPermission entity
- addition of upsertFieldPermission in role resolver
- computing of permissions taking fieldPermission into account. In order
to limit what is stored in Redis we only store fields restrictions. For
instance for objectMetadata with id XXX with a restriction on field with
id YYY we store:
`"XXX":{"canRead":true,"canUpdate":false,"canSoftDelete":false,"canDestroy":false,"restrictedFields":{"YYY":{"canRead":false,"canUpdate":null}}}`
---------
Co-authored-by: Charles Bochet <charlesBochet@users.noreply.github.com>
Context :
- Phones import is a bit complex if not all subfields are provided.
- Phones subfield validation are absent or different from BE validation.
Solution :
- Normalize callingCode and countryCode validation (BE/FE)
- Ease phone import if only phoneNumber is provided
### Added IMAP integration
This PR adds support for connecting email accounts via IMAP protocol,
allowing users to sync their emails without OAuth.
#### DB Changes:
- Added customConnectionParams and connectionType fields to
ConnectedAccountWorkspaceEntity
#### UI:
- Added settings pages for creating and editing IMAP connections with
proper validation and connection testing.
- Implemented reconnection flows for handling permission issues.
#### Backend:
- Built ImapConnectionModule with corresponding resolver and service for
managing IMAP connections.
- Created MessagingIMAPDriverModule to handle IMAP client operations,
message fetching/parsing, and error handling.
#### Dependencies:
Integrated `imapflow` and `mailparser` libraries with their type
definitions to handle the IMAP protocol communication.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Context :
- IndexFieldMetadata was no longer available on 'objects' gql query
([since this PR](https://github.com/twentyhq/twenty/pull/12785)). Then,
unicity checks on import do not work anymore.
Fix :
- Add a dataloader logic in indexFieldMetadata
- Add extra check in unicity hook on import
This PR aims at improving readability in sentry and user experience with
runtime errors.
**GraphQL errors (and ApolloError)**
1. In sentry we have a lot of "Object captured as exception with keys:
extensions, message" errors (2k over the last 90d), on which we have
zero information. This is because in apollo-factory we were passing on
GraphQL errors to sentry directly why sentry expects the structure of a
JS Error. We are now changing that, rebuilding an Error object and
attempting to help grouping by creating a fingerPrint based on error
code and truncated operationName (same as we do in the back for 500
graphql errors).
2. In sentry we have a lot of ApolloError, who actually correspond to
errors that should not be logged in sentry (Forbidden errors such as
"Email is not verified"), or errors that are already tracked by back-end
(Postgres errors such as "column xxx does not exist"). This is because
ApolloErrors become unhandled rejections errors if they are not caught
and automatically sent to sentry through the basic config. To change
that we are now filtering out ApolloErrors created from GraphQL Errors
before sending error to sentry:
<img width="524" alt="image"
src="https://github.com/user-attachments/assets/02974829-26d9-4a9e-8c4c-cfe70155e4ab"
/>
**Runtime errors**
4. Runtime errors were all caught by sentry with the name "Error",
making them not easy to differentiate on sentry (they were not grouped
together but all appeared in the list as "Error"). We are replacing the
"Error" name with the error message, or the error code if present. We
are introducing a CustomError class that allows errors whose message
contain dynamic text (an id for instance) to be identified on sentry
with a common code. _(TODO: if this approach is validated then I have
yet to replace Error with dynamic error messages with CustomError)_
5. Runtime error messages contain technical details that do not mean
anything to users (for instance, "Invalid folder ID: ${droppableId}",
"ObjectMetadataItem not found", etc.). Let's replace them with "Please
refresh the page." to users and keep the message error for sentry and
our dev experience (they will still show in the console as uncaught
errors).
# Introduction
close https://github.com/twentyhq/twenty/issues/12343
Adding a transform step for any field phone in order to infer country
code and calling code from the number if they're provided
## Edges cases
```ts
RecordTransformerExceptionCode.INVALID_PHONE_NUMBER:
RecordTransformerExceptionCode.INVALID_PHONE_COUNTRY_CODE:
RecordTransformerExceptionCode.CONFLICTING_PHONE_COUNTRY_CODE:
RecordTransformerExceptionCode.CONFLICTING_PHONE_CALLING_CODE:
RecordTransformerExceptionCode.CONFLICTING_PHONE_CALLING_CODE_AND_COUNTRY_CODE:
RecordTransformerExceptionCode.INVALID_PHONE_CALLING_CODE:
RecordTransformerExceptionCode.INVALID_URL:
```
## Coverage
Note: Will handle REST api integration testing pivot and UPDATE
operation later in the afternoon, critical bug appeared that I prefer
handling before improving this PR coverage, also would be too many
updates
Note2: Haven't fuzzed all of the string inputs, would seem overkill for
such a use case, to be debated
```ts
PASS test/integration/metadata/suites/field-metadata/phone/create-one-field-metadata-phone.integration-spec.ts (23.609 s)
Phone field metadata tests suite
✓ It should succeed create primary phone field (1397 ms)
✓ It should succeed create primary phone field with number and other information (930 ms)
✓ It should succeed create primary phone field with full international format and other information (893 ms)
✓ It should succeed create primary phone field with full international and infer other information from it but not the countryCode as its shared (825 ms)
✓ It should succeed create primary phone field with full international and infer other information from it (818 ms)
✓ It should succeed create primary phone field with empty payload (827 ms)
✓ It should succeed create additional phone field with number and other information (894 ms)
✓ It should succeed create additional phone field with full international format and other information (1024 ms)
✓ It should succeed create additional phone field with full international and infer other information from it but not the countryCode as its shared (808 ms)
✓ It should succeed create additional phone field with full international and infer other information from it (751 ms)
✓ It should succeed create additional phone field with empty payload (739 ms)
✓ It should fail to create primary phone field without country or calling code at all (776 ms)
✓ It should fail to create primary phone field with invalid country code (782 ms)
✓ It should fail to create primary phone field with invalid calling code (858 ms)
✓ It should fail to create primary phone field with conflicting country code and calling code (872 ms)
✓ It should fail to create primary phone field with invalid phone number format (1489 ms)
✓ It should fail to create primary phone field with conflicting phone number country code (1425 ms)
✓ It should fail to create primary phone field with conflicting phone number calling code (1553 ms)
✓ It should fail to create primary phone field without country or calling code at all (814 ms)
✓ It should fail to create primary phone field with invalid country code (813 ms)
✓ It should fail to create primary phone field with invalid calling code (742 ms)
✓ It should fail to create primary phone field with conflicting country code and calling code (783 ms)
✓ It should fail to create primary phone field with invalid phone number format (731 ms)
✓ It should fail to create primary phone field with conflicting phone number country code (947 ms)
✓ It should fail to create primary phone field with conflicting phone number calling code (822 ms)
Test Suites: 1 passed, 1 total
Tests: 25 passed, 25 total
Snapshots: 14 passed, 14 total
Time: 23.627 s
```
Fix inconsistent domain URL formats : removing the last / that was
caused by URL method
Standardize URL formatting to ensure consistent links storage and
retrieval of domain URLs across the application. Will improve the
dedpulicates in the links
Note: there is another temporary issue from google that was solved on
the 13th of june https://groups.google.com/g/adwords-api/c/tRSQMRZrJYM
but we consider this out of this scope
Fixes#12621
