Done :
- move metrics and health cache services from health module to metrics
module
- refactor metrics counter from specific method to set up from enum keys
- add OpenTelemetry (Otel) instrumentation for metrics
- set up Otel SDK to send metrics to Otel collector
To do later :
- implement Otel instrumentation for traces + plug Sentry on top
# Introduction
In this PR we've migrated `twenty-shared` from a `vite` app
[libary-mode](https://vite.dev/guide/build#library-mode) to a
[preconstruct](https://preconstruct.tools/) "atomic" application ( in
the future would like to introduce preconstruct to handle of all our
atomic dependencies such as `twenty-emails` `twenty-ui` etc it will be
integrated at the monorepo's root directly, would be to invasive in the
first, starting incremental via `twenty-shared`)
For more information regarding the motivations please refer to nor:
- https://github.com/twentyhq/core-team-issues/issues/587
-
https://github.com/twentyhq/core-team-issues/issues/281#issuecomment-2630949682
close https://github.com/twentyhq/core-team-issues/issues/589
close https://github.com/twentyhq/core-team-issues/issues/590
## How to test
In order to ease the review this PR will ship all the codegen at the
very end, the actual meaning full diff is `+2,411 −114`
In order to migrate existing dependent packages to `twenty-shared` multi
barrel new arch you need to run in local:
```sh
yarn tsx packages/twenty-shared/scripts/migrateFromSingleToMultiBarrelImport.ts && \
npx nx run-many -t lint --fix -p twenty-front twenty-ui twenty-server twenty-emails twenty-shared twenty-zapier
```
Note that `migrateFromSingleToMultiBarrelImport` is idempotent, it's atm
included in the PR but should not be merged. ( such as codegen will be
added before merging this script will be removed )
## Misc
- related opened issue preconstruct
https://github.com/preconstruct/preconstruct/issues/617
## Closed related PR
- https://github.com/twentyhq/twenty/pull/11028
- https://github.com/twentyhq/twenty/pull/10993
- https://github.com/twentyhq/twenty/pull/10960
## Upcoming enhancement: ( in others dedicated PRs )
- 1/ refactor generate barrel to export atomic module instead of `*`
- 2/ generate barrel own package with several files and tests
- 3/ Migration twenty-ui the same way
- 4/ Use `preconstruct` at monorepo global level
## Conclusion
As always any suggestions are welcomed !
# Introduction
Under the hood class-validator isSemver uses
https://github.com/validatorjs/validator.js/blob/master/src/lib/isSemVer.js
which does not cover all semVer use cases
## Even tho
Had a discussion with @charles was about to store in db ws version as
`vx.y.z`. We felt like we wanted it to be stored as `x.y.z`, in my
opinion `APP_VERSION` should reflect the tag used to be build the
instance and not be updated
But we could extract only `x.y.z` from it at runtime
Also handling the `v` extraction in CD is IMO not the most reliable
## Env var logging refactor
Now not stopping on first error log
```ts
Successfully compiled: 2128 files with swc (185.34ms)
Watching for file changes.
[Nest] 52686 - 03/14/2025, 6:28:33 PM ERROR PG_DATABASE_URL should not be null or undefined
PG_DATABASE_URL must be a URL address
[Nest] 52686 - 03/14/2025, 6:28:33 PM ERROR APP_VERSION must be a valid semantic version (e.g., 1.0.0)
/Users/paulrastoin/ws/twenty/packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts:1019
throw new Error("Environment variables validation failed")
^
Error: Environment variables validation failed
at Object.validate (/Users/paulrastoin/ws/twenty/packages/twenty-server/src/engine/core-modules/environment/environment-variables.ts:1019:11)
at Function.forRoot (/Users/paulrastoin/ws/twenty/node_modules/@nestjs/config/dist/config.module.js:67:45)
at Object.<anonymous> (/Users/paulrastoin/ws/twenty/packages/twenty-server/src/engine/core-modules/environment/environment.module.ts:11:18)
at Module._compile (node:internal/modules/cjs/loader:1256:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
at Module.load (node:internal/modules/cjs/loader:1119:32)
at Function.Module._load (node:internal/modules/cjs/loader:960:12)
at Module.require (node:internal/modules/cjs/loader:1143:19)
at require (node:internal/modules/cjs/helpers:121:18)
at Object.<anonymous> (/Users/paulrastoin/ws/twenty/packages/twenty-server/dist/src/database/typeorm/typeorm.module.js:14:28)
```
# Introduction
This PR contains a big test file and few snapshots
Related to https://github.com/twentyhq/core-team-issues/issues/487
## New env var `APP_VERSION`
Now will be injected directly in a built docker image the twenty's built
version. Inferred from the build git tag name.
Which mean on main or other `not a tag version` built APP_VERSION will
be `null`
## New upgrade-commander-runner
Refactored the upgrade command to be more strict regarding:
- Version management
- Sync metadata command always run
- Added failing workspaces aggregator + logs on cleanup
From now on the `upgrade` command will compare the `WORKSPACE_VERSION`
to the `APP_VERSION` in order to bypass any workspace version != than
the upgrade version `fromVersion`
## Existing commands
Note that the version validation will be done only when passing by the
`upgrade` command.
Which means that running the following command
`upgrade:x.y-some-specific-command` won't result in workspace version
mutation
This is to enforce that all an upgrade commands + sync-metadata has been
run on a workspace
## Will do in other PR but related
### New workspace
New workspace will now be inserted with version equal to the APP_VERSION
they've been created by
### Old workspace
Will create a command that should be ran outside of any `upgrade-runner`
extending command, the command will have to be ran on every workspace
before making the next release upgrade
This command iterates over any active and suspended workspace that has
`version` to `NULL` in order to update it `APP_VERSION` -1 minor
### SENTRY_RELEASE
- Either deprecate SENTRY_RELEASE in favor of `APP_VERSION` => What
about main with null version ? or create a new env var that would be
`APP_COMMIT_SHA` instead of SENTRY third party ref
### Update CD to inject APP_VERSION from branch name
### Update docs and release logs
Adding documentation for `APP_VERSION`
## Related PRs:
https://github.com/twentyhq/twenty-infra/pull/181
This update provides the flexibility for users to configure SMTP for
local instances where encryption is not required. For environments using
an unencrypted SMTP relay or similar local instances that are not open
to the public, the end user can now choose to disable encryption. This
ensures flexibility in the configuration while still maintaining the
option for secure SMTP connections in other environments.
### From [NodeMailer](https://nodemailer.com/smtp/#tls-options)
**secure** – if true the connection will use TLS when connecting to
server. <mark>If false (the default) then TLS is used if server supports
the STARTTLS extension. In most cases set this value to true if you are
connecting to port 465. For port 587 or 25 keep it false.</mark>
**ignoreTLS** – <mark>if this is true and secure is false then TLS is
not used even if the server supports STARTTLS extension.</mark>
---------
Co-authored-by: Félix Malfait <felix.malfait@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
### Context
For calendar and message sync job health monitoring, we used to
increment a counter in redis cache which could lead to concurrency
issue.
### Solution
- Update to a set structure in place of counter + use sAdd redis method
which is atomic
- Each minute another counter was incremented on a new cache key ->
Update to a 15s window
- Remove ONGOING status not needed. We only need status at job end (or
fail).
### Potential improvements
- Check for cache key existence before fetching data to avoid useless
call to redis ?
closes https://github.com/twentyhq/twenty/issues/10070
Closes https://github.com/twentyhq/core-team-issues/issues/469 and
https://github.com/twentyhq/core-team-issues/issues/500
In this PR
1. stop conditioning permission initialization for a workspace to env
variable value. Instead we want to create and assign permissions and
roles in all new workspaces. For now that will be totally silent.
2. temporarily, the default role is set to the admin role for new
workspaces. it will also be the case for existing workspaces through the
backfill command. Member role is still being created though. (when we
will do the final roll-out we will update this so that future workspaces
have the member role as default role. our goal here is not to break any
current behaviour for users, that today have all have the equivalent of
admin rights).
Replaced multiple environment variables for frontend URL construction
with a single FRONTEND_URL variable. This change reduces complexity and
improves clarity by consolidating frontend URL handling into one source.
Updated relevant validations and removed unused variables like
FRONT_PROTOCOL and FRONT_PORT.
Fix#10016
Updated the environment variable name from PORT to NODE_PORT for more
clarity and alignment with node server conventions. Adjusted references
in configuration and application bootstrap accordingly.
Fix#10017
Introduce isCustomDomainEnabled field in Workspace entity to manage
custom domain activation. Update related services, types, and logic to
validate and toggle the custom domain's status dynamically based on its
current state. This ensures accurate domain configurations are reflected
across the system.
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
addressing >
There are two patterns to avoid:
Creating functions that return JSX like renderThing() -> this was taken
already addressed in https://github.com/twentyhq/twenty/pull/10011
Making a hook that "stores" all the logic of a component - > this PR is
addressing this particular pattern
In essence, handlers should remain in the component and be connected to
their events.
And everything in a handler can be abstracted into its dedicated hook.
For example:
const { myReactiveState } =
useRecoilValue(myReactiveStateComponentState);
const { removeThingFromOtherThing } = useRemoveThingFromOtherThing();
const handleClick = () => {
if (isDefined(myReactiveState)) {
removeThingFromOtherThing();
}
}
Broadly speaking, this is how you can split large components into
several sub-hooks.
---------
Co-authored-by: Lucas Bordeau <bordeau.lucas@gmail.com>
Context :
We want to implement some counters to monitor server health. First
counters will track : messageChannel sync status during job execution
and invalid captcha.
How :
Counters are stored in cache and grouped by one-minute windows.
Controllers are created for each metric, aggregating counter over a
five-minutes window.
Endpoints are public and will be queried by Prometheus.
closes https://github.com/twentyhq/core-team-issues/issues/55
# In this PR
- Allow to register a custom domain
- Refacto subdomain generation
# In other PRs
- Add UI to deal with a custom domain
- Add logic to work with custom domain
### Context
- Update /plan-required page to let users get free trial without credit
card plan
- Update usePageChangeEffectNavigateLocation to redirect paused and
canceled subscription (suspended workspace) to /settings/billing page
### To do
- [x] Update usePageChangeEffectNavigateLocation test
- [x] Update ChooseYourPlan sb test
closes#9520
---------
Co-authored-by: etiennejouan <jouan.etienne@gmail.com>
Closestwentyhq/twenty#8240
This PR introduces email verification for non-Microsoft/Google Emails:
## Email Verification SignInUp Flow:
https://github.com/user-attachments/assets/740e9714-5413-4fd8-b02e-ace728ea47ef
The email verification link is sent as part of the
`SignInUpStep.EmailVerification`. The email verification token
validation is handled on a separate page (`AppPath.VerifyEmail`). A
verification email resend can be triggered from both pages.
## Email Verification Flow Screenshots (In Order):
## Sent Email Details (Subject & Template):
### Successful Email Verification Redirect:
### Unsuccessful Email Verification (invalid token, invalid email, token
expired, user does not exist, etc.):
### Force Sign In When Email Not Verified:
# TODOs:
## Sign Up Process
- [x] Introduce server-level environment variable
IS_EMAIL_VERIFICATION_REQUIRED (defaults to false)
- [x] Ensure users joining an existing workspace through an invite are
not required to validate their email
- [x] Generate an email verification token
- [x] Store the token in appToken
- [x] Send email containing the verification link
- [x] Create new email template for email verification
- [x] Create a frontend page to handle verification requests
## Sign In Process
- [x] After verifying user credentials, check if user's email is
verified and prompt to to verify
- [x] Show an option to resend the verification email
## Database
- [x] Rename the `emailVerified` colum on `user` to to `isEmailVerified`
for consistency
## During Deployment
- [x] Run a script/sql query to set `isEmailVerified` to `true` for all
users with a Google/Microsoft email and all users that show an
indication of a valid subscription (e.g. linked credit card)
- I have created a draft migration file below that shows one possible
approach to implementing this change:
```typescript
import { MigrationInterface, QueryRunner } from 'typeorm';
export class UpdateEmailVerifiedForActiveUsers1733318043628
implements MigrationInterface
{
name = 'UpdateEmailVerifiedForActiveUsers1733318043628';
public async up(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
CREATE TABLE core."user_email_verified_backup" AS
SELECT id, email, "isEmailVerified"
FROM core."user"
WHERE "deletedAt" IS NULL;
`);
await queryRunner.query(`
-- Update isEmailVerified for users who have been part of workspaces with active subscriptions
UPDATE core."user" u
SET "isEmailVerified" = true
WHERE EXISTS (
-- Check if user has been part of a workspace through userWorkspace table
SELECT 1
FROM core."userWorkspace" uw
JOIN core."workspace" w ON uw."workspaceId" = w.id
WHERE uw."userId" = u.id
-- Check for valid subscription indicators
AND (
w."activationStatus" = 'ACTIVE'
-- Add any other subscription-related conditions here
)
)
AND u."deletedAt" IS NULL;
`);
}
public async down(queryRunner: QueryRunner): Promise<void> {
await queryRunner.query(`
UPDATE core."user" u
SET "isEmailVerified" = b."isEmailVerified"
FROM core."user_email_verified_backup" b
WHERE u.id = b.id;
`);
await queryRunner.query(`DROP TABLE core."user_email_verified_backup";`);
}
}
```
---------
Co-authored-by: Antoine Moreaux <moreaux.antoine@gmail.com>
Co-authored-by: Félix Malfait <felix@twenty.com>
Eliminated all references to `isSSOEnabled` across the frontend,
backend, and configuration files. This change simplifies the codebase by
removing unnecessary feature flag checks, associated logic, and
environment variables. The SSO feature remains available without
reliance on this flag.
We want to avoid infinite loops using workflows. Adding a throttler with
a limit of 10 executions / sec by default for each workflow.
We were not emitting events on workflow actions so loops could not
happen. Since throttler is there we can now and these.
Adding an error message so the user knows when it happens.
<img width="1284" alt="Capture d’écran 2024-12-27 à 17 05 20"
src="https://github.com/user-attachments/assets/dafa837b-5b4c-48be-8207-c90f5c71a236"
/>
Beforehand, the name of the branch is not representative of the work
that has been done in this PR
**TLDR:**
Solves https://github.com/twentyhq/private-issues/issues/192
Add 3 tables BillingCustomer, BillingProduct and BillingPrice and
BillingMeter to core, inspired by the Stripe implementation. Separates
migration, between common and billing on order to not populate the db of
the self-hosting instances with unused tables.
**In order to test:**
Run the command:
npx nx typeorm -- migration:run -d
src/database/typeorm/core/core.datasource.ts
**Considerations:**
I only put the information we should use right now in the Billing
module, for instance columns like meter or agreggation formula where
omitted in the creation of the tables.
These columns and other ones who fall on the same spectrum will be added
as we need them.
If you want to add more information to the table, I'll leave some
utility links down bellow:
- BillingPrices: https://docs.stripe.com/api/prices/object
- BillingCustomer: https://docs.stripe.com/api/customers/object
- BillingProduct: https://docs.stripe.com/api/products/object
**Next Steps**
Use the Stripe Webhook in order to update the tables accordingly
---------
Co-authored-by: Félix Malfait <felix@twenty.com>
## Summary
Add support for multi-workspace feature and adjust configurations and
states accordingly.
- Introduced new state isMultiWorkspaceEnabledState.
- Updated ClientConfigProviderEffect component to handle
multi-workspace.
- Modified GraphQL schema and queries to include multi-workspace related
configurations.
- Adjusted server environment variables and their respective
documentation to support multi-workspace toggle.
- Updated server-side logic to handle new multi-workspace configurations
and conditions.
- [ ] Catch this specific `500` error
- [ ] Make sure catched `500` errors are sent to sentry for the Cloud
version
- [ ] Hide the option to sync email with google in this situation where
the according env var is missing
- [x] Add Worskpace information to all catched errors for better
debugging
fix#8607
Previously, SSL_KEY_PATH and SSL_CERT_PATH were required if the
SERVER_URL started with 'https'. This change makes these variables
optional, allowing greater flexibility in dealing with SSL
configurations. This update helps in scenarios where SSL paths are
managed differently or not needed.
Introduced a custom decorator 'WarningIf' to log warnings for specific
environment variable conditions. Implemented this for
SESSION_STORE_SECRET to ensure users change it from the default value.
Implemented:
* Account Connect
* Calendar sync via delta ids then requesting single events
I think I would split the messaging part into a second pr - that's a
step more complex then the calendar :)
---------
Co-authored-by: bosiraphael <raphael.bosi@gmail.com>
