# Introduction
Migrating `pageLayoutTab` to the v2 engine
- Types and constants
- Builder and validate
- Runner
Introduced a new `StrictSyncableEntity` that enforces that
`universalIdentifier` and `applicationId` are defined
As these entities are brand new we could enforce this rule already
This still requires a migration command to associate the existing
entities to custom workspace application instance and define
universalIdentifier
Handled retro-comp of the migration through a migration as upgrade
command fallback
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
## Problem
buildEntityMetadatas in GlobalWorkspaceOrmManager is computationally
expensive and was running on every executeInWorkspaceContext call. This
method uses
TypeORM's EntitySchemaTransformer and EntityMetadataBuilder to build
metadata for all workspace entities (30-50+ objects with many fields
each).
The resulting EntityMetadata[] is not serialisable which means it cannot
be cached in Redis because they contain:
- Circular references
- Functions/methods
- References to the DataSource instance
## Solution
Extended the workspace cache system to support local-only caching, then
created a cache provider for entityMetadatas.
## Implementation details
Updated @WorkspaceCache decorator (workspace-cache.decorator.ts)
- Added localOnly?: boolean option to skip Redis storage for
non-serializable data
Created WorkspaceEntityMetadatasCacheService
- Computes entity metadatas from DB to avoid race condition, this is
acceptable
Simplified GlobalWorkspaceOrmManager
- Now fetches entityMetadatas from cache instead of rebuilding on every
call
Updated Workspace migration runner - the only entry point where metadata
can change
- Now invalidate the new 'entityMetadata' local cache when
shouldIncrementMetadataGraphqlSchemaVersion is true (== field/object
mutations)
# Introduction
Closes https://github.com/twentyhq/core-team-issues/issues/1980
In this PR we migrate the agent from v1 to v2.
## New FlatRoleTargetByAgentIdMaps
Derivated the `flatRoleTargetMaps` to be building a
`flatRoleTargetByAgentIdMaps` to ease retrieving a roleId to associate
to an agent
## Coverage
Added strong coverage on both failing and successful CRU agents
operations
---------
Co-authored-by: Weiko <corentin@twenty.com>
## Context
We've recently introduced a new workspace cache service which now acts
as a cache access and local storage for all workspace related data,
deprecating the individual specific services.
- Better performance through multiple caching/fetching strategies
- Consistent data access patterns across the codebase
- Reduced redis queries through MGET/MSET/PIPELINE with multiple cache
keys
## Context
Deprecating the old objectMetadataMap type in favour of split flat
entities to match with our new caching.
In the long run, trying to achieve:
- Better performance through caching
- Consistent data access patterns across the codebase
- Reduced database queries
Now that everything is based on flat entities, which are cached, we can
finish the refactoring of workspace context cache which should already
improve performances.
Then the last step will be to consume that new cache in the new global
datasource to get rid of the many workspace datasources stored in the
server
## Context
Implementing a single service managing all the cache scoped to a
workspace, this will be dynamically injected as a WorkspaceContext in
the app during a request lifetime (ingested by the future global
datasource for example).
Usage:
```typescript
this.globalWorkspaceOrmManager.executeInWorkspaceContext(
authContext,
async () => {
// Everything here will have access to a workspaceContext, containing all the cache data + a ready to use datasource with workspace scoped metadata, permissions, feature flags, etc...
// Internally will call loadWorkspaceContext
}
```
Note: executeInWorkspaceContext will probably be owned by a higher level
service later and not only the ORM.
```typescript
private async loadWorkspaceContext(
authContext: WorkspaceAuthContext,
): Promise<WorkspaceContext> {
const workspaceId = authContext.workspace.id;
const cache =
await this.workspaceContextCacheService.get<WorkspaceContextData>(
workspaceId,
[
'objectMetadataMaps',
'metadataVersion',
'featureFlagsMap',
'permissionsPerRoleId',
],
);
return {
authContext,
objectMetadataMaps: cache.objectMetadataMaps,
metadataVersion: cache.metadataVersion,
featureFlagsMap: cache.featureFlagsMap,
permissionsPerRoleId: cache.permissionsPerRoleId,
};
}
```
The cache retrieval strategy is as followed:
```
- Check if there is an ongoing promise fetching data from the cache =>
return the promise.
- Check in the local cache entry if lastCheckedAt has expired. If not,
return as it is without querying redis.
- Check in redis the cache entry hash and compare with local cache entry
hash, if they are the same return the local cache entry data
- Check in redis the cache entry data, if it's there return it and store
it into the local cache entry data and update local cache entry hash. If
it's not there recompute the data by querying the DB and update both
redis and local cache
```
