This PR is the first part of the creation of the Chart editor.
https://github.com/user-attachments/assets/8b0af8ea-be41-4506-84cb-e40f5521cbf0
Done:
- Bar chart settings (except filters)
- Line chart settings (except filters)
In progress:
- Pie chart settings
- Number chart settings
- Gauge chart settings
Left to do:
- Loosen the backend validation to allow the user to save a partial
configuration, validate the graph configuration in the frontend and
display a error friendly message in the graph if the config is not
completed yet
- Implement the filter edition
- Finish the other graph types settings
## Context
Add routes to migration V2
- Resolvers
- Service v2
- Builder
- Validator
- Action runner
Next PR: Add to twenty-cli to sync routes with serverless
## Context
We are experiencing bad performance on Twenty. One of the root cause
hypothesis is that computing `currentUser.currentWorkspace.views` is CPU
consuming. Without views, the GetCurrentUser response is ~1000 lines.
With its ~10000 lines.
As graphql is going through all fields recursively this can be quite
heavy on CPU. We had a similar issues on ObjectMetadataItems 2 years ago
and came with storing the response in redis.
Note: I thought there was also a cache in RAM but this is not the case,
so to invalidate the cache we can just empty redis.
## How
- Extract getting all views from GetCurrentUser and update frontend to
perform both queries
- Add views to cached graphql operations
- invalidate the cache manually on view or related core entities update
/ create / delete / destroy
## Tests
I have tested a lot on v1
# Introduction
Extracting the legacy fields build and dispatch out of the object one to
follow the generic flat entity build, also update caches entries for
object
## Main tasks
- flat field map cache
- flat field builder
- refactored the dispatch matrix to return flat entity maps instead of
flat entity arrays
- orchestrator aggregator
- removing legacy code
- making universal identifier aka standardId of standard field for
custom object deterministically dynamic
- perfs debug logs for v2
## TODO
- [x] Refactor the generic entity builder to be dependency flat maps in
order to main foreign keys list in flat parent
- [x] Refactor the flat object metadata to contain the array of related
fields and avoid costy find object fields
- [ ] Improve the create field handler to handle multiple field at once
- [ ] Refactor the dispatch to embbed the comparison
- [ ] Improve perf by extracting from elements out of existing
- [ ] Fix the labelIdentifierId validators on object before field
creation ( integ tests are in failing mode )
## Debug logs snippet
```ts
[EntityBuilder fieldMetadata] matrix computation: 0.027ms
[EntityBuilder fieldMetadata] creation validation: 0.001ms
[EntityBuilder fieldMetadata] deletion validation: 0.293ms
[EntityBuilder fieldMetadata] update validation: 0.006ms
[EntityBuilder fieldMetadata] entity processing: 0.363ms
[EntityBuilder fieldMetadata] validateAndBuild: 0.455ms
[EntityBuilder index] matrix computation: 0.005ms
[EntityBuilder index] creation validation: 0.001ms
[EntityBuilder index] deletion validation: 0.146ms
[EntityBuilder index] update validation: 0.004ms
[EntityBuilder index] entity processing: 0.199ms
[EntityBuilder index] validateAndBuild: 0.228ms
[Runner] Initial cache retrieval: 0.549ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForWorkspaceSchema: 11.665ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForMetadata: 12.864ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 1.476ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 6.816ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 0.062ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 0.889ms
[Runner] Transaction execution: 23.434ms
[Runner] Cache invalidation: 316.662ms
[Runner] Total execution: 340.767ms
```
As you can see cache invalidation is way to long, we could replace the
cache by the optimistic in the end
twenty-cli serverless triggers follow up. Fixes:
- eventName don't support wildcard
- universalIdentifier not used to create or update trigger : update does
not work properly (does deletion then creation)
- add a base project in twenty-cli that is copied when creating a new
app
In this PR https://github.com/twentyhq/twenty/pull/14785 we deprecated
viewFilterOperand (from camelCase to capital snakeCase values), but we
had not migrated the existing workflow steps values. Hence they cannot
be executed!
Let's fix that by dealing with both operands, new and deprecated, to
mitigate the issue. Then we will add a command to migrate the values.
Tested :
- server level ok
- workspace level ok
- server level without canImpersonate or without allow impersonation -
fail ok
- workspace level without permission - fail ok
- add serverlessFunction schema in twenty-cli
- add trigger schema in twenty-cli
- update serverless function code save and get
- sync serverless function
Legacy from early v2 implementation, we don't want to store relation ids
in the cache as the new implementation handle that with flat entities
properly already.
This causes some issues because when we return the updated view, it
contains an array of viewField objects with only the id in it so gql
complains because some fields are non-nullable in the output and the
resolver can't resolve viewField directly because viewField is in the
output (not complete though) already so it just ignores it.
Following https://github.com/twentyhq/twenty/pull/14762
In this PR
- moved logic around any field filter formatting to twenty-shared
- added any field filter to the filters applied to groupBy when viewId
is defined
- added integration test
In the BE we have stored filter operand as IS_EMPTY, IS_NOT_EMPTY, etc.
For some reason in the FE we were manipulating IsEmpty, IsNotEmpty, etc.
(maybe because they were used before in Views before they were moved to
core)
So we were converting the operands in the FE from IS to Is
(convertViewFilterOperandFromCore) to read and manipulate viewFilters,
and then back to BE version to send mutations etc., from Is to IS
(convertViewFilterOperandToCore).
The migration is now over, so we can remove and simplify that code.
(In the 1-5:migrate-views-to-core command we still do the migration from
Is format to IS format.)
When all items have been processed, iterator should:
- store last iteration in history
- return the final result saying iteration have been successful. So it
gets stored by the executor
It was not doing the first step, so we were losing the last iteration.
Splitting into separated function + adding tests
Closes https://github.com/twentyhq/core-team-issues/issues/1560
If viewId is defined in a groupBy query, we want to apply all filters of
the view to the query.
This required to move a lot of code from twenty-front to twenty-shared
to convert the filters as stored in the db into graphql filters,
applying the right combinations between filters etc., which was
previously only done in the FE.
This PR does not handle any field filters, it will be done in a later pr
## Introduction
After enabling flag by default got following errors:
```ts
Test Suites: 48 failed, 1 skipped, 97 passed, 145 of 146 total
Tests: 499 failed, 1 skipped, 644 passed, 1144 total
Snapshots: 61 failed, 133 passed, 194 total
Time: 363.226 s
Ran all test suites.
```
## From
<img width="2952" height="1510" alt="image"
src="https://github.com/user-attachments/assets/7e3b20c6-2552-40a7-90bb-2d7b3002c895"
/>
## To
<img width="3134" height="1510" alt="image"
src="https://github.com/user-attachments/assets/4fc9ada4-3c14-4333-a1db-11daf87db8d6"
/>
There's a huge test bundle in the latest shard that we could split up
## Notes
- Set as failing morph relation field rename as for the moment we do not
handle relation field mutation
- fixed the object update and creation validation adding label
identifier field metadata id checks
- and more
Some integrations tests are still on the v1 ( they have before and after
all disabling and re-enabling the flat ) but mainly we now have more
coverage on the v2 than the v1.
Mainly related records, uniqueness have to be migrated the v2 and so
tests too
https://github.com/user-attachments/assets/d6c565eb-9a29-4830-9396-5f979c8caa7b
- Added a new component for manual trigger (mostly duplicated from
previous one). Will remove the old one once all data are migrated
- Updated schema output so the current item of the iterator can be typed
Todo left:
- migrate old triggers
- add an util to search iterator output. Today current item fields will
be displayed as not found
- set new manual triggers for workflow runs
## Context
Now adding serverless code sync within the migration v2 logic itself. To
do that we need to follow the
- Prepare flat input
- Build migration
- Run migration
steps where now the serverless function entity will store in DB and
cache a checksum of its code and the flat input will contain the code
with the checksum. Build will compare checksum and create an update
action containing the code if it has changed and the migration will now
run the corresponding services (instead of calling those in the parent
serverless service exposed in the API) allowing us to keep that logic
functional for other use cases such as import/export, twenty-cli and
twenty upgrades.
