Extract field metadata build out of object metadata build (#14763)

# Introduction

Extracting the legacy fields build and dispatch out of the object one to
follow the generic flat entity build, also update caches entries for
object

## Main tasks
- flat field map cache
- flat field builder
- refactored the dispatch matrix to return flat entity maps instead of
flat entity arrays
- orchestrator aggregator
- removing legacy code
- making universal identifier aka standardId of standard field for
custom object deterministically dynamic
- perfs debug logs for v2

## TODO
- [x] Refactor the generic entity builder to be dependency flat maps in
order to main foreign keys list in flat parent
- [x] Refactor the flat object metadata to contain the array of related
fields and avoid costy find object fields
- [ ] Improve the create field handler to handle multiple field at once
- [ ] Refactor the dispatch to embbed the comparison
- [ ] Improve perf by extracting from elements out of existing
- [ ] Fix the labelIdentifierId validators on object before field
creation ( integ tests are in failing mode )

## Debug logs snippet

```ts
[EntityBuilder fieldMetadata] matrix computation: 0.027ms
[EntityBuilder fieldMetadata] creation validation: 0.001ms
[EntityBuilder fieldMetadata] deletion validation: 0.293ms
[EntityBuilder fieldMetadata] update validation: 0.006ms
[EntityBuilder fieldMetadata] entity processing: 0.363ms
[EntityBuilder fieldMetadata] validateAndBuild: 0.455ms
[EntityBuilder index] matrix computation: 0.005ms
[EntityBuilder index] creation validation: 0.001ms
[EntityBuilder index] deletion validation: 0.146ms
[EntityBuilder index] update validation: 0.004ms
[EntityBuilder index] entity processing: 0.199ms
[EntityBuilder index] validateAndBuild: 0.228ms
[Runner] Initial cache retrieval: 0.549ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForWorkspaceSchema: 11.665ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_index executeForMetadata: 12.864ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 1.476ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 6.816ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForWorkspaceSchema: 0.062ms
[BaseWorkspaceMigrationRunnerActionHandlerService] delete_field executeForMetadata: 0.889ms
[Runner] Transaction execution: 23.434ms
[Runner] Cache invalidation: 316.662ms
[Runner] Total execution: 340.767ms
```
As you can see cache invalidation is way to long, we could replace the
cache by the optimistic in the end
This commit is contained in:
Paul Rastoin
2025-10-02 16:07:34 +00:00
committed by GitHub
parent d4b83df3f5
commit 170f6d27c5
240 changed files with 11385 additions and 17744 deletions
@@ -6,9 +6,12 @@ import { ViewFieldEntity } from 'src/engine/core-modules/view/entities/view-fiel
import { ViewEntity } from 'src/engine/core-modules/view/entities/view.entity';
import { WorkspaceFlatViewFieldMapCacheService } from 'src/engine/core-modules/view/flat-view/services/workspace-flat-view-field-map-cache.service';
import { WorkspaceFlatViewMapCacheService } from 'src/engine/core-modules/view/flat-view/services/workspace-flat-view-map-cache.service';
import { FieldMetadataEntity } from 'src/engine/metadata-modules/field-metadata/field-metadata.entity';
import { WorkspaceFlatFieldMetadataMapCacheService } from 'src/engine/metadata-modules/flat-field-metadata/services/workspace-flat-field-metadata-map-cache.service';
import { WorkspaceFlatIndexMapCacheService } from 'src/engine/metadata-modules/flat-index-metadata/services/workspace-flat-index-map-cache.service';
import { WorkspaceFlatObjectMetadataMapCacheService } from 'src/engine/metadata-modules/flat-object-metadata/services/workspace-flat-object-metadata-map-cache.service';
import { IndexMetadataEntity } from 'src/engine/metadata-modules/index-metadata/index-metadata.entity';
import { ObjectMetadataEntity } from 'src/engine/metadata-modules/object-metadata/object-metadata.entity';
import { WorkspaceMetadataCacheModule } from 'src/engine/metadata-modules/workspace-metadata-cache/workspace-metadata-cache.module';
import { WorkspaceMetadataVersionModule } from 'src/engine/metadata-modules/workspace-metadata-version/workspace-metadata-version.module';
import { WorkspacePermissionsCacheModule } from 'src/engine/metadata-modules/workspace-permissions-cache/workspace-permissions-cache.module';
@@ -24,6 +27,8 @@ import { WorkspaceFlatMapCacheRegistryService } from 'src/engine/workspace-flat-
ViewEntity,
ViewFieldEntity,
IndexMetadataEntity,
FieldMetadataEntity,
ObjectMetadataEntity,
]),
],
providers: [
@@ -32,6 +37,7 @@ import { WorkspaceFlatMapCacheRegistryService } from 'src/engine/workspace-flat-
WorkspaceFlatViewMapCacheService,
WorkspaceFlatViewFieldMapCacheService,
WorkspaceFlatIndexMapCacheService,
WorkspaceFlatFieldMetadataMapCacheService,
],
exports: [
WorkspaceFlatMapCacheRegistryService,
@@ -39,6 +45,7 @@ import { WorkspaceFlatMapCacheRegistryService } from 'src/engine/workspace-flat-
WorkspaceFlatViewMapCacheService,
WorkspaceFlatViewFieldMapCacheService,
WorkspaceFlatIndexMapCacheService,
WorkspaceFlatFieldMetadataMapCacheService,
],
})
export class WorkspaceFlatMapCacheModule {}