Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0216a13f3e | ||
|
|
347669dcaa | ||
|
|
e350c391fd | ||
|
|
c78e45aa04 | ||
|
|
0273c1a8d5 |
@@ -0,0 +1,263 @@
|
||||
# Email-Group (SES): Per-Workspace Suppression, Events, Rate Limits, Enterprise Gate
|
||||
|
||||
## Problem
|
||||
|
||||
SES suppression list is **account-level only** — one shared list per AWS account/region. Config-set
|
||||
`SuppressionOptions.SuppressedReasons` only toggles whether a reason feeds/honors that one shared list.
|
||||
So a hard bounce in workspace A silently blocks sends to the same address in workspace B. SES "tenants"
|
||||
isolate reputation and sending-pause, **not** suppression.
|
||||
|
||||
Second defect: SES allows only **one contact list per account** (max 20 topics). The current
|
||||
per-workspace `CreateContactListCommand` works for workspace #1 and fails for #2. Unsubscribe must be
|
||||
app-owned, not SES-list-owned.
|
||||
|
||||
## Scope (v1)
|
||||
|
||||
- Per-workspace suppression, **GLOBAL scope only**. No campaign/topic-level scope — no campaign entity
|
||||
exists and transactional-vs-marketing split is coming via a workspace transactional channel type, not
|
||||
via suppression scope.
|
||||
- Three actionable SES event types: `HARD_BOUNCE` (Permanent bounce only) + `COMPLAINT` → suppress;
|
||||
`SUBSCRIPTION` → unsubscribe. All other 7 event types are telemetry.
|
||||
- Soft bounces arrive as `DeliveryDelay`, not `Bounce` — never suppressed.
|
||||
|
||||
---
|
||||
|
||||
## 1. Per-workspace suppression
|
||||
|
||||
### Entity
|
||||
|
||||
`EmailGroupSuppressedRecipientEntity` → table `core.emailGroupSuppressedRecipient`. Schema was
|
||||
revised after a durability simulation — see
|
||||
`EMAILING_DOMAIN_SUPPRESSION_DESIGN_REVIEW.md` for why the naive single-row design fails (22.8% of
|
||||
the event space) and why this one is verified correct (0%).
|
||||
|
||||
| column | type | notes |
|
||||
|---|---|---|
|
||||
| `id` | uuid | `@PrimaryGeneratedColumn('uuid')` |
|
||||
| `workspaceId` | uuid | FK → workspace, `ON DELETE CASCADE` (via `WorkspaceRelatedEntity`) |
|
||||
| `emailAddress` | varchar | normalized (trim + lowercase) |
|
||||
| `scope` | enum | `GLOBAL` (bounce/complaint, all channels) \| `MARKETING` (unsubscribe, reversible) |
|
||||
| `reason` | enum | `HARD_BOUNCE \| COMPLAINT \| UNSUBSCRIBE` |
|
||||
| `status` | enum | `ACTIVE \| RELEASED` — resubscribe sets `RELEASED`, never deletes |
|
||||
| `providerEventId` | varchar null | SES feedbackId/messageId for event-grain dedupe |
|
||||
| `createdBySource` | enum | `FieldActorSource` — `WEBHOOK` for SES events, `API` for self-serve unsubscribe |
|
||||
| `createdAt` | timestamptz | `@CreateDateColumn` |
|
||||
| `updatedAt` | timestamptz | `@UpdateDateColumn` |
|
||||
|
||||
Constraint: `UNIQUE(workspaceId, emailAddress, scope)`.
|
||||
|
||||
Extend `WorkspaceRelatedEntity` (gives `workspaceId` + cascade `@ManyToOne` to `WorkspaceEntity`),
|
||||
matching `EmailingDomainEntity`. Enum columns match the local `EmailingDomainTenantStatus` pattern
|
||||
(TypeORM `enum` type), not string-literal columns.
|
||||
|
||||
Write rules (in `EmailGroupSuppressionService`):
|
||||
- `suppress`: scope derived from reason (`UNSUBSCRIBE → MARKETING`, else `GLOBAL`); reactivating
|
||||
upsert on `(workspaceId, emailAddress, scope)` with `status = ACTIVE`. Scope separation keeps
|
||||
bounce/complaint and unsubscribe in distinct rows, so they never overwrite each other.
|
||||
- `releaseMarketingSuppression`: sets `status = RELEASED` on the `MARKETING` row only; `GLOBAL`
|
||||
(permanent bounce/complaint) is never touched by a resubscribe.
|
||||
- Read path (`getSuppressedAddresses`): returns addresses with any `ACTIVE` row. v1 is
|
||||
channel-agnostic (any active scope blocks); add a scope filter when the transactional channel
|
||||
type lands.
|
||||
|
||||
### Migration
|
||||
|
||||
Generate via CLI, never hand-write:
|
||||
|
||||
```
|
||||
npx nx run twenty-server:database:migrate:generate --name addEmailGroupSuppressedRecipient --type fast
|
||||
```
|
||||
|
||||
Auto-registers in `instance-commands.constant.ts`.
|
||||
|
||||
### Opt out of account-level list
|
||||
|
||||
In `aws-ses-register-domain.service.ts`, `CreateConfigurationSetCommand` currently sends
|
||||
`SuppressionOptions: { SuppressedReasons: ['BOUNCE', 'COMPLAINT'] }`. Change to
|
||||
`SuppressedReasons: []` so this config set neither feeds nor honors the shared account list. Suppression
|
||||
becomes entirely app-owned.
|
||||
|
||||
Existing provisioned config sets need a one-time `PutConfigurationSetSuppressionOptionsCommand` backfill —
|
||||
fold into the same instance command (workspace-scoped) or a follow-up.
|
||||
|
||||
### Enforce at send time
|
||||
|
||||
In `EmailingDomainService.sendEmail` (after tenant-status check, before driver send): query
|
||||
`emailGroupSuppressedRecipient` for `workspaceId` where `emailAddress IN (lowercased to/cc/bcc)`. Filter
|
||||
suppressed addresses out of each list. If **all primary `to` recipients** are suppressed → throw
|
||||
(`ALL_RECIPIENTS_SUPPRESSED`). Partial suppression → send to survivors.
|
||||
|
||||
---
|
||||
|
||||
## 2. Handle actionable events
|
||||
|
||||
### Rip out broken contact-list code
|
||||
|
||||
- Remove `CreateContactListCommand` from `aws-ses-register-domain.service.ts` (breaks at workspace #2).
|
||||
- Remove `ListManagementOptions: { ContactListName, TopicName }` from `SendEmailCommand` in
|
||||
`aws-ses-send-email.service.ts`.
|
||||
- Drop the `AWS_SES_MARKETING_TOPIC_NAME` contact-list plumbing.
|
||||
|
||||
### Routing (extend existing path — do NOT build a new webhook)
|
||||
|
||||
EventBridge → SNS → `POST /webhooks/messaging/ses/outbound` → `SesOutboundWebhookRouterService` already
|
||||
exists and consumes `SesEventBridgeNotification`, switching on `detail-type`. Today it handles only
|
||||
`'Sending Status Enabled' | 'Sending Status Disabled'` → tenant pause.
|
||||
|
||||
As built:
|
||||
1. Widened `SesEventBridgeNotification['detail-type']` to include `'Email Bounced'` and
|
||||
`'Email Complaint Received'` (the actual EventBridge envelope strings — verified against AWS docs;
|
||||
the SNS-style `detail.eventType` is `'Bounce'`/`'Complaint'`) plus the `detail.mail` / `detail.bounce`
|
||||
/ `detail.complaint` payload fields.
|
||||
2. Router dispatches by `detail-type`: sending-status → `SesOutboundSendingStateHandlerService`;
|
||||
`'Email Bounced'`/`'Email Complaint Received'` → new `SesSuppressionEventHandlerService`.
|
||||
3. Handler suppresses via the Item-1 reactivating upsert. Suppress **only** `bounce.bounceType ===
|
||||
'Permanent'` (HARD_BOUNCE) and all complaints (COMPLAINT). `feedbackId` stored in `providerEventId`.
|
||||
4. Workspace ID resolved from the `workspace` EmailTag (`detail.mail.tags.workspace[0]`, the exact id we
|
||||
set at send time), falling back to parsing the resource ARN.
|
||||
5. `Subscription` events are moot once SES list management is removed (we no longer own an SES list);
|
||||
unsubscribe is handled entirely by the app endpoint below.
|
||||
|
||||
### twenty-infra reconciliation
|
||||
|
||||
Dangling uncommitted files (`ses-suppression-events.tf`, `sns-ses-suppression-events.tf`, per-route
|
||||
`SES_SNS_TOPIC_*_ARN` env vars in `charts/*/values.yaml`) point at a route that never existed. Since
|
||||
events ride the existing outbound topic/route, **delete** them rather than wire them up — unless the
|
||||
existing outbound topic does not also carry Bounce/Complaint (verify the EventBridge rule → SNS topic
|
||||
fan-out in twenty-infra before deleting).
|
||||
|
||||
### Unsubscribe (app-owned) — as built
|
||||
|
||||
SES list management is unusable (fact: one list/account). Own it:
|
||||
- `EmailGroupUnsubscribeService` builds `List-Unsubscribe` + `List-Unsubscribe-Post:
|
||||
List-Unsubscribe=One-Click` (RFC 8058) headers, attached via `Content.Simple.Headers`
|
||||
(`MessageHeader[]` — supported in `@aws-sdk/client-sesv2@3.1001.0`; **no Raw MIME**).
|
||||
- Token is a self-contained HMAC-SHA256 over `APP_SECRET` of `{workspaceId, emailAddress}`
|
||||
(base64url payload + signature, constant-time compare). No JWT key/token-type coupling, stateless
|
||||
verify. Never trusts raw query params.
|
||||
- `EmailingDomainUnsubscribeController` exposes public `POST /emailing/unsubscribe` (one-click) and
|
||||
`GET /emailing/unsubscribe` (browser fallback); a valid token writes a `MARKETING`/`UNSUBSCRIBE`
|
||||
suppression row (`FieldActorSource.API`). Always returns 200 (no recipient-existence leak).
|
||||
- Headers are attached **only** when the caller sets `includeUnsubscribe: true` **and** the message
|
||||
resolves to exactly one deliverable recipient (one-click targets a single address). v1 read path is
|
||||
channel-agnostic, so an unsubscribe blocks all sends for that address until the transactional channel
|
||||
type lands — transactional callers must leave `includeUnsubscribe` unset.
|
||||
|
||||
---
|
||||
|
||||
## 3. Rate limits + billing units
|
||||
|
||||
### Input caps (`send-email-via-domain.input.ts`)
|
||||
|
||||
Add `class-validator` decorators:
|
||||
- `@MaxLength` on `subject`, `text`, `html`.
|
||||
- `@ArrayMaxSize` on `to`, `cc`, `bcc`, `replyTo` (existing: only `@ArrayMinSize(1)` on `to`).
|
||||
|
||||
### Throttle — as built
|
||||
|
||||
- Reuses the existing `ThrottlerService.tokenBucketThrottleOrThrow`, keyed
|
||||
`emailing-domain-send:<workspaceId>`, consuming one token per deliverable recipient
|
||||
(`EMAIL_SEND_THROTTLE_MAX_RECIPIENTS` per `EMAIL_SEND_THROTTLE_WINDOW_MS`). Checked after the
|
||||
suppression filter, before the driver send.
|
||||
- Hard per-message recipient cap (`EMAIL_MAX_TOTAL_RECIPIENTS`, SES's 50-recipient limit) on the
|
||||
deliverable total; DTO also caps each field (`@ArrayMaxSize`) and body/subject lengths
|
||||
(`@MaxLength`).
|
||||
|
||||
### Credit metering (Felix)
|
||||
|
||||
Charge ~**3× the AWS cost**. The existing billing system already defines **1 credit = $1**:
|
||||
`DOLLAR_TO_CREDIT_MULTIPLIER = 1_000_000` stores micro-credits, so
|
||||
`creditsUsedMicro = dollarCost × 1_000_000`. No new credit unit needed.
|
||||
|
||||
Cost basis:
|
||||
- AWS SES outbound = **$0.10 / 1000 emails = $0.0001 per recipient** (SES bills per recipient, each
|
||||
To/Cc/Bcc counts).
|
||||
- 3× markup → **$0.0003 per recipient** → `creditsUsedMicro = 300` per recipient.
|
||||
|
||||
Define both pieces as constants so the markup is auditable and AWS repricing is one edit:
|
||||
|
||||
```
|
||||
SES_AWS_COST_PER_RECIPIENT_USD = 0.0001
|
||||
EMAIL_SEND_CREDIT_MARKUP = 3
|
||||
```
|
||||
|
||||
Meter **per recipient, post-suppression-filter** (charge only what is actually accepted for sending —
|
||||
matches how SES itself bills). Bcc/Cc count.
|
||||
|
||||
Integration (existing path):
|
||||
1. Add `EMAIL_SEND` to `UsageOperationType`
|
||||
(`src/engine/core-modules/usage/enums/usage-operation-type.enum.ts`).
|
||||
2. Create/map a Stripe meter with eventName `'EMAIL_SEND'` (mirror `WORKFLOW_NODE_RUN`).
|
||||
3. After a successful driver send, emit:
|
||||
```
|
||||
workspaceEventEmitter.emitCustomBatchEvent(USAGE_RECORDED, [{
|
||||
resourceType: UsageResourceType.API,
|
||||
operationType: UsageOperationType.EMAIL_SEND,
|
||||
quantity: acceptedRecipientCount,
|
||||
creditsUsedMicro: SES_AWS_COST_PER_RECIPIENT_USD * EMAIL_SEND_CREDIT_MARKUP
|
||||
* DOLLAR_TO_CREDIT_MULTIPLIER * acceptedRecipientCount,
|
||||
unit: UsageUnit.<email unit>,
|
||||
periodStart: subscriptionPeriodStart,
|
||||
}], workspaceId)
|
||||
```
|
||||
4. Pre-send gate: `BillingUsageService.canFeatureBeUsed(workspaceId)` (rejects when the workspace has
|
||||
no active subscription); when billing is disabled it returns true so dev/self-host still sends.
|
||||
Order in `sendEmail`: verified → tenant-status → from-domain → suppression filter → recipient cap →
|
||||
billing gate → throttle → send → meter.
|
||||
|
||||
As built: metering emits via the **global `WorkspaceEventEmitter`** only (no `BillingModule` coupling
|
||||
for the meter itself); the existing `USAGE_RECORDED` → ClickHouse → cap pipeline performs credit
|
||||
**enforcement** asynchronously (pausing the workspace at the cap), matching how
|
||||
`WORKFLOW_EXECUTION` is metered. A hard pre-send balance reject (vs the coarse subscription gate) can
|
||||
layer on later via `BillingUsageService` if product wants it. Adding `EMAIL_SEND` to
|
||||
`UsageOperationType` also required an entry in the exhaustive
|
||||
`USAGE_UNIT_BY_OPERATION_TYPE` map (`app-billing.service.ts`).
|
||||
|
||||
Attachment data ($0.12/GB on SES) not metered in v1 — note as follow-up. A Stripe meter with
|
||||
eventName `EMAIL_SEND` must be provisioned for revenue to land (no reverse op-type→meter mapping
|
||||
exists in code yet).
|
||||
|
||||
---
|
||||
|
||||
## 4. Enterprise gate / paywall — as built
|
||||
|
||||
Intent: the whole feature is paywalled, especially emailing-domain provisioning.
|
||||
|
||||
Twenty has **no `entitlement` primitive**; billed features (workflows, AI) are paywalled with
|
||||
`BillingUsageService.canFeatureBeUsed` (active, non-canceled subscription) plus the resource-credit
|
||||
cap. This feature follows that same pattern:
|
||||
|
||||
- **Visibility / admin gate:** every GraphQL operation already carries
|
||||
`@RequireFeatureFlag(FeatureFlagKey.IS_EMAIL_GROUP_ENABLED)` + `FeatureFlagGuard`.
|
||||
- **Paywall gate:** `assertBillingAllowsEmailGroup` (`canFeatureBeUsed`) now guards **both**
|
||||
`createEmailingDomain` (provisioning — the costly AWS step) **and** `sendEmail`. A workspace cannot
|
||||
provision a domain or send without an active plan, even if an admin flips the feature flag.
|
||||
`canFeatureBeUsed` returns true when billing is disabled, so dev/self-host is unaffected.
|
||||
- The public SES webhook + unsubscribe endpoints are intentionally not flag-gated: they are
|
||||
event/token-driven and only become reachable after a (now paywalled) provisioning.
|
||||
|
||||
Follow-ups for a true per-plan paywall (billing config, not code): provision a billing product/price
|
||||
for the email feature and scope `IS_EMAIL_GROUP_ENABLED` to paid plans; a frontend upsell for the
|
||||
blocked state.
|
||||
|
||||
---
|
||||
|
||||
## Conventions
|
||||
|
||||
- No code comments — self-documenting names.
|
||||
- Swallow expected SDK errors via `.send(cmd).catch((e) => { if (!(e instanceof X)) throw e; })`, not
|
||||
try/catch or helper wrappers.
|
||||
- `isDefined` from `twenty-shared/utils`; `isNonEmptyString`/`isNonEmptyArray` from `@sniptt/guards`.
|
||||
- String-literal unions over enums. Named exports. Functional. No `any`.
|
||||
- After changes: `npx nx typecheck twenty-server`; `oxlint --type-aware` on changed files; jest scoped
|
||||
to the file/test, never the full package suite.
|
||||
|
||||
## Verified facts (do not re-litigate)
|
||||
|
||||
- SES suppression list is account-level only; config-set options only toggle feed/honor of that one list.
|
||||
- SES tenants isolate reputation + sending-pause, not suppression.
|
||||
- One contact list per account (max 20 topics) → per-workspace `CreateContactList` is broken.
|
||||
- Only 3 of 10 SES event types actionable: Permanent Bounce + Complaint (suppress), Subscription
|
||||
(unsubscribe). Soft bounce = `DeliveryDelay`.
|
||||
- `@aws-sdk/client-sesv2@3.1001.0` `Content.Simple` (`Message`) supports `Headers?: MessageHeader[]`.
|
||||
- Outbound webhook route + `SesOutboundWebhookRouterService` already consume EventBridge→SNS; extend it.
|
||||
@@ -0,0 +1,129 @@
|
||||
# Suppression Entity — Durability Review (simulation-backed)
|
||||
|
||||
## Question
|
||||
|
||||
Is `EmailGroupSuppressedRecipientEntity` as shipped in Item 1 the final design, or does it
|
||||
encode mistakes that bite later?
|
||||
|
||||
**Answer: it is not final.** A simulation over the full event space shows the single-row schema
|
||||
produces incorrect send decisions in **22.8%** of cases, of which the dominant class is
|
||||
**resending to dead-mailbox / complained addresses** (140 critical cells), plus silently dropping
|
||||
**transactional** mail on a marketing unsubscribe (15 cells).
|
||||
|
||||
## Model
|
||||
|
||||
Each `(workspaceId, emailAddress)` receives a sequence of events
|
||||
`{B = hard bounce, C = complaint, U = unsubscribe, R = resubscribe}` followed by a `SEND` on a
|
||||
channel `{T = transactional, M = marketing}`.
|
||||
|
||||
Correct spec (the lattice we actually need):
|
||||
|
||||
| reason | strength | scope | reversible |
|
||||
|---|---|---|---|
|
||||
| COMPLAINT | 3 | all channels | no (legal record) |
|
||||
| HARD_BOUNCE | 2 | all channels | no (dead mailbox) |
|
||||
| UNSUBSCRIBE | 1 | marketing only | yes (resubscribe) |
|
||||
|
||||
`R` releases an active `U` only; it must never revive `B`/`C`. Transactional mail is exempt from
|
||||
unsubscribe (CAN-SPAM / GDPR both treat it separately).
|
||||
|
||||
Current design under test: one row, `UNIQUE(workspaceId, emailAddress)`, insert-then-DO-NOTHING,
|
||||
no scope, and the natural resubscribe = delete the `(workspaceId, emailAddress)` row.
|
||||
|
||||
## Results (sequences length 1..4 × 2 channels = 680 cells)
|
||||
|
||||
- Total divergence from correct: **155 / 680 = 22.8%**
|
||||
- FALSE_SEND (we send to a suppressed address): **140**, all critical (B/C involved)
|
||||
- FALSE_BLOCK (we drop legitimate mail): **15**
|
||||
|
||||
Minimal witnesses:
|
||||
|
||||
| events | send | current | correct | failure |
|
||||
|---|---|---|---|---|
|
||||
| `B R` | T or M | ALLOW | BLOCK | resubscribe deletes the bounce row → mail to a dead mailbox |
|
||||
| `C R` | T or M | ALLOW | BLOCK | resubscribe deletes the complaint row → resend to a complainer (legal) |
|
||||
| `U` | T | BLOCK | ALLOW | marketing unsubscribe blocks a transactional send (password reset) |
|
||||
|
||||
Root cause of the critical class: a single row cannot distinguish reasons, so any resubscribe keyed
|
||||
by `(workspaceId, emailAddress)` removes whatever reason happens to be stored — including permanent
|
||||
suppressions. The schema, not the code, forces this.
|
||||
|
||||
## The `reason` column is decided by a race
|
||||
|
||||
Closed form: with `n` distinct-precedence events arriving in uniformly random order (SNS delivers
|
||||
unordered, at-least-once), insert-DO-NOTHING keeps the **first** arrival, but the correct stored
|
||||
reason is the **strongest**:
|
||||
|
||||
```
|
||||
P(stored reason wrong) = 1 - 1/n
|
||||
n=2 -> 50.0% n=3 -> 66.7%
|
||||
```
|
||||
|
||||
Empirically over all multi-event orderings: **58.3%** of cases store the wrong reason. The `reason`
|
||||
column is therefore unreliable for any reason-aware logic (reporting, reason-aware resubscribe,
|
||||
audit) — its value reflects SNS arrival order, not severity.
|
||||
|
||||
## Three mistakes that bite later
|
||||
|
||||
1. **`UNIQUE(workspaceId, emailAddress)` collapses multi-reason history.** One address can be
|
||||
unsubscribed *and* later hard-bounce. One row + DO-NOTHING loses the second fact. Reason-aware
|
||||
resubscribe becomes impossible; resubscribe un-suppresses permanent reasons (the 140 cells).
|
||||
|
||||
2. **No scope dimension.** Unsubscribe is marketing-only and reversible; bounce/complaint are
|
||||
all-channel and permanent. Global-only over-blocks transactional mail today, and the
|
||||
transactional/marketing channel split is already on the roadmap — adding scope later is a
|
||||
`UNIQUE`-key change (index rebuild + data backfill), the most expensive migration kind.
|
||||
|
||||
3. **Hard-delete resubscribe destroys the compliance trail and races the webhook.** GDPR wants
|
||||
proof-of-consent history and complaint records retained; deleting rows erases them, and a delete
|
||||
racing an SNS re-delivery can resurrect or drop a row nondeterministically.
|
||||
|
||||
## Recommended revision (durable, still pragmatic)
|
||||
|
||||
Keep a single projection table but fix the key and lifecycle. No event-sourcing required.
|
||||
|
||||
```
|
||||
EmailGroupSuppressedRecipientEntity
|
||||
id uuid pk
|
||||
workspaceId uuid FK CASCADE
|
||||
emailAddress varchar -- normalized (see below)
|
||||
scope enum GLOBAL | MARKETING -- B/C => GLOBAL, U => MARKETING
|
||||
reason enum HARD_BOUNCE | COMPLAINT | UNSUBSCRIBE
|
||||
status enum ACTIVE | RELEASED -- resubscribe sets RELEASED, never deletes
|
||||
providerEventId varchar null -- SES feedbackId/messageId, event-grain dedupe
|
||||
createdBySource FieldActorSource
|
||||
createdAt timestamptz
|
||||
updatedAt timestamptz
|
||||
UNIQUE(workspaceId, emailAddress, scope)
|
||||
```
|
||||
|
||||
Write rules:
|
||||
- Suppress: upsert on `(workspaceId, emailAddress, scope)`. On conflict, keep the **higher-strength**
|
||||
reason and `status=ACTIVE` (precedence-max, not first-wins). Permanent reasons land in `GLOBAL`.
|
||||
- Resubscribe: set `status=RELEASED` on the `MARKETING` row only. `GLOBAL` (B/C) is never touched.
|
||||
- Send decision: blocked iff an `ACTIVE` row exists whose scope covers the channel
|
||||
(`GLOBAL` covers all; `MARKETING` covers marketing only).
|
||||
- Idempotency: dedupe on `providerEventId` when present; the unique key still backstops replays.
|
||||
|
||||
Running the same simulation against these rules yields **0 divergences** — false-sends and
|
||||
false-blocks both go to zero.
|
||||
|
||||
If full auditability is later required (dispute handling, consent proof at event grain), promote to
|
||||
an append-only `EmailGroupSuppressionEventEntity` log with this table as a materialized projection.
|
||||
The schema above is forward-compatible with that move.
|
||||
|
||||
## Orthogonal: address normalization
|
||||
|
||||
Lowercasing is necessary but not sufficient. Same inbox, different stored strings → a bounce on one
|
||||
form won't suppress the other:
|
||||
- Gmail dot-folding (`a.b@gmail.com` ≡ `ab@gmail.com`) and `+tag` sub-addressing.
|
||||
- IDN/Unicode domains; trim/whitespace; display-name leakage.
|
||||
|
||||
Normalize to a canonical form before storing and before the send-time lookup, or accept a known
|
||||
miss rate. Decide explicitly; do not leave it implicit.
|
||||
|
||||
## Cost note
|
||||
|
||||
Item 1's migration is **not yet committed**, so adopting the revised schema now is a regenerate, not
|
||||
a data migration. Done after release, fix #2 alone is a `UNIQUE`-key change over live suppression
|
||||
data. Cheapest moment to get the key right is before the first commit.
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
import { QueryRunner } from 'typeorm';
|
||||
|
||||
import { RegisteredInstanceCommand } from 'src/engine/core-modules/upgrade/decorators/registered-instance-command.decorator';
|
||||
import { FastInstanceCommand } from 'src/engine/core-modules/upgrade/interfaces/fast-instance-command.interface';
|
||||
|
||||
@RegisteredInstanceCommand('2.9.0', 1780000902115)
|
||||
export class AddEmailGroupSuppressedRecipientFastInstanceCommand implements FastInstanceCommand {
|
||||
public async up(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query('CREATE TYPE "core"."emailGroupSuppressedRecipient_scope_enum" AS ENUM(\'GLOBAL\', \'MARKETING\')');
|
||||
await queryRunner.query('CREATE TYPE "core"."emailGroupSuppressedRecipient_reason_enum" AS ENUM(\'HARD_BOUNCE\', \'COMPLAINT\', \'UNSUBSCRIBE\')');
|
||||
await queryRunner.query('CREATE TYPE "core"."emailGroupSuppressedRecipient_status_enum" AS ENUM(\'ACTIVE\', \'RELEASED\')');
|
||||
await queryRunner.query('CREATE TYPE "core"."emailGroupSuppressedRecipient_createdbysource_enum" AS ENUM(\'EMAIL\', \'CALENDAR\', \'WORKFLOW\', \'AGENT\', \'API\', \'IMPORT\', \'MANUAL\', \'SYSTEM\', \'WEBHOOK\', \'APPLICATION\')');
|
||||
await queryRunner.query('CREATE TABLE "core"."emailGroupSuppressedRecipient" ("workspaceId" uuid NOT NULL, "id" uuid NOT NULL DEFAULT uuid_generate_v4(), "emailAddress" character varying NOT NULL, "scope" "core"."emailGroupSuppressedRecipient_scope_enum" NOT NULL, "reason" "core"."emailGroupSuppressedRecipient_reason_enum" NOT NULL, "status" "core"."emailGroupSuppressedRecipient_status_enum" NOT NULL DEFAULT \'ACTIVE\', "providerEventId" character varying, "createdBySource" "core"."emailGroupSuppressedRecipient_createdbysource_enum" NOT NULL, "createdAt" TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(), "updatedAt" TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT now(), CONSTRAINT "IDX_EMAIL_GROUP_SUPPRESSED_RECIPIENT_WORKSPACE_EMAIL_SCOPE_UNIQUE" UNIQUE ("workspaceId", "emailAddress", "scope"), CONSTRAINT "PK_55b0607e539d7941cbaedf1328a" PRIMARY KEY ("id"))');
|
||||
await queryRunner.query('ALTER TABLE "core"."emailGroupSuppressedRecipient" ADD CONSTRAINT "FK_866066f1e73b748f917fd6fcd80" FOREIGN KEY ("workspaceId") REFERENCES "core"."workspace"("id") ON DELETE CASCADE ON UPDATE NO ACTION');
|
||||
}
|
||||
|
||||
public async down(queryRunner: QueryRunner): Promise<void> {
|
||||
await queryRunner.query('ALTER TABLE "core"."emailGroupSuppressedRecipient" DROP CONSTRAINT "FK_866066f1e73b748f917fd6fcd80"');
|
||||
await queryRunner.query('DROP TABLE "core"."emailGroupSuppressedRecipient"');
|
||||
await queryRunner.query('DROP TYPE "core"."emailGroupSuppressedRecipient_createdbysource_enum"');
|
||||
await queryRunner.query('DROP TYPE "core"."emailGroupSuppressedRecipient_status_enum"');
|
||||
await queryRunner.query('DROP TYPE "core"."emailGroupSuppressedRecipient_reason_enum"');
|
||||
await queryRunner.query('DROP TYPE "core"."emailGroupSuppressedRecipient_scope_enum"');
|
||||
}
|
||||
}
|
||||
+2
@@ -58,6 +58,7 @@ import { DropFieldMetadataIsUniqueColumnFastInstanceCommand } from 'src/database
|
||||
import { EmailingDomainTenantStatusAndGlobalUniquenessFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-fast-1799000020000-emailing-domain-tenant-status-and-global-uniqueness';
|
||||
import { EncryptNonSecretApplicationVariableSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1798400000000-encrypt-non-secret-application-variable';
|
||||
import { MigrateAiModelPreferencesSlowInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-slow-1799000010000-migrate-ai-model-preferences';
|
||||
import { AddEmailGroupSuppressedRecipientFastInstanceCommand } from 'src/database/commands/upgrade-version-command/2-9/2-9-instance-command-fast-1780000902115-addEmailGroupSuppressedRecipient';
|
||||
|
||||
export const INSTANCE_COMMANDS = [
|
||||
AddViewFieldGroupIdIndexOnViewFieldFastInstanceCommand,
|
||||
@@ -118,4 +119,5 @@ export const INSTANCE_COMMANDS = [
|
||||
DropFieldMetadataIsUniqueColumnFastInstanceCommand,
|
||||
MigrateAiModelPreferencesSlowInstanceCommand,
|
||||
EncryptNonSecretApplicationVariableSlowInstanceCommand,
|
||||
AddEmailGroupSuppressedRecipientFastInstanceCommand,
|
||||
];
|
||||
|
||||
+1
@@ -20,6 +20,7 @@ const USAGE_UNIT_BY_OPERATION_TYPE: Record<UsageOperationType, UsageUnit> = {
|
||||
[UsageOperationType.WORKFLOW_EXECUTION]: UsageUnit.INVOCATION,
|
||||
[UsageOperationType.CODE_EXECUTION]: UsageUnit.INVOCATION,
|
||||
[UsageOperationType.WEB_SEARCH]: UsageUnit.INVOCATION,
|
||||
[UsageOperationType.EMAIL_SEND]: UsageUnit.INVOCATION,
|
||||
};
|
||||
|
||||
// `workspaceId` + `applicationId` come from the application-access token,
|
||||
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
export const EMAIL_SUBJECT_MAX_LENGTH = 998;
|
||||
export const EMAIL_BODY_MAX_LENGTH = 1_000_000;
|
||||
export const EMAIL_MAX_RECIPIENTS_PER_FIELD = 50;
|
||||
export const EMAIL_MAX_REPLY_TO = 10;
|
||||
export const EMAIL_MAX_TOTAL_RECIPIENTS = 50;
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
export const EMAIL_SEND_AWS_COST_PER_RECIPIENT_USD = 0.0001;
|
||||
export const EMAIL_SEND_CREDIT_MARKUP = 3;
|
||||
|
||||
export const EMAIL_SEND_THROTTLE_MAX_RECIPIENTS = 500;
|
||||
export const EMAIL_SEND_THROTTLE_WINDOW_MS = 60_000;
|
||||
-1
@@ -1 +0,0 @@
|
||||
export const AWS_SES_MARKETING_TOPIC_NAME = 'marketing';
|
||||
+1
@@ -0,0 +1 @@
|
||||
export const AWS_SES_WORKSPACE_TAG_NAME = 'workspace';
|
||||
+31
-44
@@ -1,10 +1,8 @@
|
||||
import {
|
||||
AlreadyExistsException,
|
||||
CreateConfigurationSetCommand,
|
||||
CreateConfigurationSetEventDestinationCommand,
|
||||
CreateContactListCommand,
|
||||
CreateTenantResourceAssociationCommand,
|
||||
GetConfigurationSetCommand,
|
||||
NotFoundException,
|
||||
PutEmailIdentityMailFromAttributesCommand,
|
||||
} from '@aws-sdk/client-sesv2';
|
||||
|
||||
@@ -23,13 +21,12 @@ describe('AwsSesRegisterDomainService', () => {
|
||||
const provisionInput = {
|
||||
tenantName: 'twenty-workspace-ws1',
|
||||
configurationSetName: 'twenty-workspace-ws1',
|
||||
contactListName: 'twenty-workspace-ws1',
|
||||
};
|
||||
|
||||
const buildNotFound = () =>
|
||||
new NotFoundException({
|
||||
$metadata: { httpStatusCode: 404 },
|
||||
message: 'Configuration set not found.',
|
||||
const buildAlreadyExists = () =>
|
||||
new AlreadyExistsException({
|
||||
$metadata: { httpStatusCode: 409 },
|
||||
message: 'Resource already exists.',
|
||||
});
|
||||
|
||||
const setUp = () => {
|
||||
@@ -43,33 +40,7 @@ describe('AwsSesRegisterDomainService', () => {
|
||||
};
|
||||
|
||||
describe('provisionWorkspaceResources', () => {
|
||||
it('creates every workspace-scoped resource when the configuration set does not yet exist', async () => {
|
||||
const { service, send } = setUp();
|
||||
|
||||
send.mockImplementation(async (command) => {
|
||||
if (command instanceof GetConfigurationSetCommand) {
|
||||
throw buildNotFound();
|
||||
}
|
||||
|
||||
return {};
|
||||
});
|
||||
|
||||
await service.provisionWorkspaceResources(provisionInput, config);
|
||||
|
||||
const commandTypes = send.mock.calls.map(
|
||||
([command]) => command.constructor.name,
|
||||
);
|
||||
|
||||
expect(commandTypes).toEqual([
|
||||
GetConfigurationSetCommand.name,
|
||||
CreateConfigurationSetCommand.name,
|
||||
CreateConfigurationSetEventDestinationCommand.name,
|
||||
CreateContactListCommand.name,
|
||||
CreateTenantResourceAssociationCommand.name,
|
||||
]);
|
||||
});
|
||||
|
||||
it('issues no creates when the configuration set already exists', async () => {
|
||||
it('creates every workspace-scoped resource', async () => {
|
||||
const { service, send } = setUp();
|
||||
|
||||
send.mockResolvedValue({});
|
||||
@@ -80,20 +51,36 @@ describe('AwsSesRegisterDomainService', () => {
|
||||
([command]) => command.constructor.name,
|
||||
);
|
||||
|
||||
expect(commandTypes).toEqual([GetConfigurationSetCommand.name]);
|
||||
expect(commandTypes).toEqual([
|
||||
CreateConfigurationSetCommand.name,
|
||||
CreateConfigurationSetEventDestinationCommand.name,
|
||||
CreateTenantResourceAssociationCommand.name,
|
||||
]);
|
||||
});
|
||||
|
||||
it('propagates non-NotFound AWS errors raised by the existence probe', async () => {
|
||||
it('ignores AlreadyExistsException per resource so a retry re-runs every step', async () => {
|
||||
const { service, send } = setUp();
|
||||
|
||||
send.mockRejectedValue(buildAlreadyExists());
|
||||
|
||||
await service.provisionWorkspaceResources(provisionInput, config);
|
||||
|
||||
const commandTypes = send.mock.calls.map(
|
||||
([command]) => command.constructor.name,
|
||||
);
|
||||
|
||||
expect(commandTypes).toEqual([
|
||||
CreateConfigurationSetCommand.name,
|
||||
CreateConfigurationSetEventDestinationCommand.name,
|
||||
CreateTenantResourceAssociationCommand.name,
|
||||
]);
|
||||
});
|
||||
|
||||
it('propagates AWS errors that are not AlreadyExistsException', async () => {
|
||||
const { service, send } = setUp();
|
||||
const fatalError = new Error('Boom');
|
||||
|
||||
send.mockImplementation(async (command) => {
|
||||
if (command instanceof GetConfigurationSetCommand) {
|
||||
throw fatalError;
|
||||
}
|
||||
|
||||
return {};
|
||||
});
|
||||
send.mockRejectedValue(fatalError);
|
||||
|
||||
await expect(
|
||||
service.provisionWorkspaceResources(provisionInput, config),
|
||||
|
||||
+2
-6
@@ -21,7 +21,6 @@ describe('AwsSesSendEmailService', () => {
|
||||
const baseContext = {
|
||||
tenantName: 'twenty-workspace-ws1',
|
||||
configurationSetName: 'twenty-workspace-ws1',
|
||||
contactListName: 'twenty-workspace-ws1',
|
||||
};
|
||||
|
||||
const setUp = () => {
|
||||
@@ -42,7 +41,7 @@ describe('AwsSesSendEmailService', () => {
|
||||
return { service, send, handleErrorService };
|
||||
};
|
||||
|
||||
it('should call SendEmail with tenant, config set, and list management options', async () => {
|
||||
it('should call SendEmail with tenant and config set, without SES list management', async () => {
|
||||
const { service, send } = setUp();
|
||||
|
||||
send.mockResolvedValue({ MessageId: 'msg-1' });
|
||||
@@ -59,11 +58,8 @@ describe('AwsSesSendEmailService', () => {
|
||||
Destination: { ToAddresses: ['user@example.com'] },
|
||||
ConfigurationSetName: 'twenty-workspace-ws1',
|
||||
TenantName: 'twenty-workspace-ws1',
|
||||
ListManagementOptions: {
|
||||
ContactListName: 'twenty-workspace-ws1',
|
||||
TopicName: 'marketing',
|
||||
},
|
||||
});
|
||||
expect(command.input.ListManagementOptions).toBeUndefined();
|
||||
expect(command.input.EmailTags).toEqual(
|
||||
expect.arrayContaining([
|
||||
{ Name: 'workspace', Value: 'ws1' },
|
||||
|
||||
-14
@@ -6,7 +6,6 @@ import {
|
||||
CreateTenantCommand,
|
||||
CreateTenantResourceAssociationCommand,
|
||||
DeleteConfigurationSetCommand,
|
||||
DeleteContactListCommand,
|
||||
DeleteEmailIdentityCommand,
|
||||
DeleteTenantCommand,
|
||||
DeleteTenantResourceAssociationCommand,
|
||||
@@ -120,7 +119,6 @@ export class AwsSesDriver implements EmailingDomainDriverInterface {
|
||||
{
|
||||
tenantName,
|
||||
configurationSetName: this.buildConfigurationSetName(workspaceId),
|
||||
contactListName: this.buildContactListName(workspaceId),
|
||||
},
|
||||
this.config,
|
||||
);
|
||||
@@ -136,7 +134,6 @@ export class AwsSesDriver implements EmailingDomainDriverInterface {
|
||||
return this.awsSesSendEmailService.sendEmail(input, {
|
||||
tenantName: this.buildTenantName(input.workspaceId),
|
||||
configurationSetName: this.buildConfigurationSetName(input.workspaceId),
|
||||
contactListName: this.buildContactListName(input.workspaceId),
|
||||
});
|
||||
}
|
||||
|
||||
@@ -167,7 +164,6 @@ export class AwsSesDriver implements EmailingDomainDriverInterface {
|
||||
const sesClient = this.awsSesClientProvider.getSESClient();
|
||||
const tenantName = this.buildTenantName(workspaceId);
|
||||
const configurationSetName = this.buildConfigurationSetName(workspaceId);
|
||||
const contactListName = this.buildContactListName(workspaceId);
|
||||
const configurationSetArn = `arn:aws:ses:${this.config.region}:${this.config.accountId}:configuration-set/${configurationSetName}`;
|
||||
|
||||
await sesClient
|
||||
@@ -191,12 +187,6 @@ export class AwsSesDriver implements EmailingDomainDriverInterface {
|
||||
if (!(error instanceof NotFoundException)) throw error;
|
||||
});
|
||||
|
||||
await sesClient
|
||||
.send(new DeleteContactListCommand({ ContactListName: contactListName }))
|
||||
.catch((error) => {
|
||||
if (!(error instanceof NotFoundException)) throw error;
|
||||
});
|
||||
|
||||
await sesClient
|
||||
.send(new DeleteTenantCommand({ TenantName: tenantName }))
|
||||
.catch((error) => {
|
||||
@@ -212,10 +202,6 @@ export class AwsSesDriver implements EmailingDomainDriverInterface {
|
||||
return `${AWS_SES_RESOURCE_NAME_PREFIX}-${workspaceId}`;
|
||||
}
|
||||
|
||||
private buildContactListName(workspaceId: string): string {
|
||||
return `${AWS_SES_RESOURCE_NAME_PREFIX}-${workspaceId}`;
|
||||
}
|
||||
|
||||
private async ensureTenantExists(tenantName: string): Promise<void> {
|
||||
const sesClient = this.awsSesClientProvider.getSESClient();
|
||||
|
||||
|
||||
+53
-83
@@ -1,26 +1,21 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import {
|
||||
AlreadyExistsException,
|
||||
CreateConfigurationSetCommand,
|
||||
CreateConfigurationSetEventDestinationCommand,
|
||||
CreateContactListCommand,
|
||||
CreateTenantResourceAssociationCommand,
|
||||
GetConfigurationSetCommand,
|
||||
NotFoundException,
|
||||
PutEmailIdentityMailFromAttributesCommand,
|
||||
type SESv2Client,
|
||||
} from '@aws-sdk/client-sesv2';
|
||||
import { type AwsSesDriverConfig } from 'src/engine/core-modules/emailing-domain/drivers/interfaces/driver-config.interface';
|
||||
|
||||
import { AWS_SES_EVENT_BUS_NAME } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-event-bus-name.constant';
|
||||
import { AWS_SES_MAIL_FROM_SUBDOMAIN } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-mail-from-subdomain.constant';
|
||||
import { AWS_SES_MARKETING_TOPIC_NAME } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-marketing-topic-name.constant';
|
||||
import { AwsSesClientProvider } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/providers/aws-ses-client.provider';
|
||||
|
||||
type ProvisionWorkspaceInput = {
|
||||
tenantName: string;
|
||||
configurationSetName: string;
|
||||
contactListName: string;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
@@ -35,69 +30,64 @@ export class AwsSesRegisterDomainService {
|
||||
): Promise<void> {
|
||||
const sesClient = this.awsSesClientProvider.getSESClient();
|
||||
|
||||
const isAlreadyProvisioned = await this.isWorkspaceProvisioned(
|
||||
sesClient,
|
||||
input.configurationSetName,
|
||||
);
|
||||
|
||||
if (isAlreadyProvisioned) {
|
||||
return;
|
||||
}
|
||||
|
||||
const eventBusArn = `arn:aws:events:${config.region}:${config.accountId}:event-bus/${AWS_SES_EVENT_BUS_NAME}`;
|
||||
const configurationSetArn = `arn:aws:ses:${config.region}:${config.accountId}:configuration-set/${input.configurationSetName}`;
|
||||
|
||||
await sesClient.send(
|
||||
new CreateConfigurationSetCommand({
|
||||
ConfigurationSetName: input.configurationSetName,
|
||||
ReputationOptions: { ReputationMetricsEnabled: true },
|
||||
SendingOptions: { SendingEnabled: true },
|
||||
SuppressionOptions: { SuppressedReasons: ['BOUNCE', 'COMPLAINT'] },
|
||||
Tags: [{ Key: 'managed-by', Value: 'twenty' }],
|
||||
}),
|
||||
);
|
||||
await sesClient
|
||||
.send(
|
||||
new CreateConfigurationSetCommand({
|
||||
ConfigurationSetName: input.configurationSetName,
|
||||
ReputationOptions: { ReputationMetricsEnabled: true },
|
||||
SendingOptions: { SendingEnabled: true },
|
||||
SuppressionOptions: { SuppressedReasons: [] },
|
||||
Tags: [{ Key: 'managed-by', Value: 'twenty' }],
|
||||
}),
|
||||
)
|
||||
.catch((error) => {
|
||||
if (!(error instanceof AlreadyExistsException)) {
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
|
||||
await sesClient.send(
|
||||
new CreateConfigurationSetEventDestinationCommand({
|
||||
ConfigurationSetName: input.configurationSetName,
|
||||
EventDestinationName: 'twenty-eventbridge',
|
||||
EventDestination: {
|
||||
Enabled: true,
|
||||
MatchingEventTypes: [
|
||||
'SEND',
|
||||
'DELIVERY',
|
||||
'BOUNCE',
|
||||
'COMPLAINT',
|
||||
'REJECT',
|
||||
'RENDERING_FAILURE',
|
||||
'DELIVERY_DELAY',
|
||||
'SUBSCRIPTION',
|
||||
],
|
||||
EventBridgeDestination: { EventBusArn: eventBusArn },
|
||||
},
|
||||
}),
|
||||
);
|
||||
|
||||
await sesClient.send(
|
||||
new CreateContactListCommand({
|
||||
ContactListName: input.contactListName,
|
||||
Topics: [
|
||||
{
|
||||
TopicName: AWS_SES_MARKETING_TOPIC_NAME,
|
||||
DisplayName: 'Marketing',
|
||||
DefaultSubscriptionStatus: 'OPT_IN',
|
||||
await sesClient
|
||||
.send(
|
||||
new CreateConfigurationSetEventDestinationCommand({
|
||||
ConfigurationSetName: input.configurationSetName,
|
||||
EventDestinationName: 'twenty-eventbridge',
|
||||
EventDestination: {
|
||||
Enabled: true,
|
||||
MatchingEventTypes: [
|
||||
'SEND',
|
||||
'DELIVERY',
|
||||
'BOUNCE',
|
||||
'COMPLAINT',
|
||||
'REJECT',
|
||||
'RENDERING_FAILURE',
|
||||
'DELIVERY_DELAY',
|
||||
'SUBSCRIPTION',
|
||||
],
|
||||
EventBridgeDestination: { EventBusArn: eventBusArn },
|
||||
},
|
||||
],
|
||||
Tags: [{ Key: 'managed-by', Value: 'twenty' }],
|
||||
}),
|
||||
);
|
||||
}),
|
||||
)
|
||||
.catch((error) => {
|
||||
if (!(error instanceof AlreadyExistsException)) {
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
|
||||
await sesClient.send(
|
||||
new CreateTenantResourceAssociationCommand({
|
||||
TenantName: input.tenantName,
|
||||
ResourceArn: configurationSetArn,
|
||||
}),
|
||||
);
|
||||
await sesClient
|
||||
.send(
|
||||
new CreateTenantResourceAssociationCommand({
|
||||
TenantName: input.tenantName,
|
||||
ResourceArn: configurationSetArn,
|
||||
}),
|
||||
)
|
||||
.catch((error) => {
|
||||
if (!(error instanceof AlreadyExistsException)) {
|
||||
throw error;
|
||||
}
|
||||
});
|
||||
|
||||
this.logger.log(
|
||||
`Provisioned workspace resources for tenant ${input.tenantName}`,
|
||||
@@ -117,24 +107,4 @@ export class AwsSesRegisterDomainService {
|
||||
|
||||
this.logger.log(`Registered MAIL FROM for domain ${domain}`);
|
||||
}
|
||||
|
||||
private async isWorkspaceProvisioned(
|
||||
sesClient: SESv2Client,
|
||||
configurationSetName: string,
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
await sesClient.send(
|
||||
new GetConfigurationSetCommand({
|
||||
ConfigurationSetName: configurationSetName,
|
||||
}),
|
||||
);
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (error instanceof NotFoundException) {
|
||||
return false;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+8
-7
@@ -8,7 +8,7 @@ import {
|
||||
type EmailingDomainSendEmailResult,
|
||||
} from 'src/engine/core-modules/emailing-domain/drivers/types/send-email';
|
||||
|
||||
import { AWS_SES_MARKETING_TOPIC_NAME } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-marketing-topic-name.constant';
|
||||
import { AWS_SES_WORKSPACE_TAG_NAME } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-workspace-tag-name.constant';
|
||||
import { AwsSesClientProvider } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/providers/aws-ses-client.provider';
|
||||
import { AwsSesHandleErrorService } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/services/aws-ses-handle-error.service';
|
||||
import {
|
||||
@@ -19,7 +19,6 @@ import {
|
||||
type SendEmailContext = {
|
||||
tenantName: string;
|
||||
configurationSetName: string;
|
||||
contactListName: string;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
@@ -56,6 +55,12 @@ export class AwsSesSendEmailService {
|
||||
ReplyToAddresses: input.replyTo,
|
||||
Content: {
|
||||
Simple: {
|
||||
Headers: isNonEmptyArray(input.headers)
|
||||
? input.headers.map((header) => ({
|
||||
Name: header.name,
|
||||
Value: header.value,
|
||||
}))
|
||||
: undefined,
|
||||
Subject: { Data: input.subject, Charset: 'UTF-8' },
|
||||
Body: {
|
||||
Text: { Data: input.text, Charset: 'UTF-8' },
|
||||
@@ -75,12 +80,8 @@ export class AwsSesSendEmailService {
|
||||
},
|
||||
ConfigurationSetName: context.configurationSetName,
|
||||
TenantName: context.tenantName,
|
||||
ListManagementOptions: {
|
||||
ContactListName: context.contactListName,
|
||||
TopicName: AWS_SES_MARKETING_TOPIC_NAME,
|
||||
},
|
||||
EmailTags: [
|
||||
{ Name: 'workspace', Value: input.workspaceId },
|
||||
{ Name: AWS_SES_WORKSPACE_TAG_NAME, Value: input.workspaceId },
|
||||
{ Name: 'domain', Value: input.domain },
|
||||
],
|
||||
}),
|
||||
|
||||
+3
@@ -11,6 +11,7 @@ export enum EmailingDomainDriverExceptionCode {
|
||||
INSUFFICIENT_PERMISSIONS = 'INSUFFICIENT_PERMISSIONS',
|
||||
CONFIGURATION_ERROR = 'CONFIGURATION_ERROR',
|
||||
SENDING_SUSPENDED = 'SENDING_SUSPENDED',
|
||||
ALL_RECIPIENTS_SUPPRESSED = 'ALL_RECIPIENTS_SUPPRESSED',
|
||||
UNKNOWN = 'UNKNOWN',
|
||||
}
|
||||
|
||||
@@ -26,6 +27,8 @@ const getEmailingDomainDriverExceptionUserFriendlyMessage = (
|
||||
return msg`Email domain configuration error.`;
|
||||
case EmailingDomainDriverExceptionCode.SENDING_SUSPENDED:
|
||||
return msg`Sending is currently suspended for this email domain.`;
|
||||
case EmailingDomainDriverExceptionCode.ALL_RECIPIENTS_SUPPRESSED:
|
||||
return msg`All recipients are suppressed for this email domain.`;
|
||||
case EmailingDomainDriverExceptionCode.TEMPORARY_ERROR:
|
||||
case EmailingDomainDriverExceptionCode.UNKNOWN:
|
||||
return STANDARD_ERROR_MESSAGE;
|
||||
|
||||
+7
@@ -4,6 +4,11 @@ export type EmailingDomainAttachment = {
|
||||
contentType: string;
|
||||
};
|
||||
|
||||
export type EmailingDomainEmailHeader = {
|
||||
name: string;
|
||||
value: string;
|
||||
};
|
||||
|
||||
export type EmailingDomainEmailContent = {
|
||||
from: string;
|
||||
to: string[];
|
||||
@@ -14,6 +19,8 @@ export type EmailingDomainEmailContent = {
|
||||
html?: string;
|
||||
replyTo?: string[];
|
||||
attachments?: EmailingDomainAttachment[];
|
||||
headers?: EmailingDomainEmailHeader[];
|
||||
includeUnsubscribe?: boolean;
|
||||
};
|
||||
|
||||
export type EmailingDomainSendEmailInput = EmailingDomainEmailContent & {
|
||||
|
||||
+16
@@ -1,14 +1,23 @@
|
||||
import { Field, InputType } from '@nestjs/graphql';
|
||||
|
||||
import {
|
||||
ArrayMaxSize,
|
||||
ArrayMinSize,
|
||||
IsArray,
|
||||
IsEmail,
|
||||
IsOptional,
|
||||
IsString,
|
||||
MaxLength,
|
||||
MinLength,
|
||||
} from 'class-validator';
|
||||
|
||||
import {
|
||||
EMAIL_BODY_MAX_LENGTH,
|
||||
EMAIL_MAX_RECIPIENTS_PER_FIELD,
|
||||
EMAIL_MAX_REPLY_TO,
|
||||
EMAIL_SUBJECT_MAX_LENGTH,
|
||||
} from 'src/engine/core-modules/emailing-domain/constants/email-limits.constant';
|
||||
|
||||
@InputType()
|
||||
export class SendEmailViaDomainInput {
|
||||
@Field(() => String)
|
||||
@@ -18,33 +27,39 @@ export class SendEmailViaDomainInput {
|
||||
@Field(() => [String])
|
||||
@IsArray()
|
||||
@ArrayMinSize(1)
|
||||
@ArrayMaxSize(EMAIL_MAX_RECIPIENTS_PER_FIELD)
|
||||
@IsEmail({}, { each: true })
|
||||
to: string[];
|
||||
|
||||
@Field(() => [String], { nullable: true })
|
||||
@IsOptional()
|
||||
@IsArray()
|
||||
@ArrayMaxSize(EMAIL_MAX_RECIPIENTS_PER_FIELD)
|
||||
@IsEmail({}, { each: true })
|
||||
cc?: string[];
|
||||
|
||||
@Field(() => [String], { nullable: true })
|
||||
@IsOptional()
|
||||
@IsArray()
|
||||
@ArrayMaxSize(EMAIL_MAX_RECIPIENTS_PER_FIELD)
|
||||
@IsEmail({}, { each: true })
|
||||
bcc?: string[];
|
||||
|
||||
@Field(() => String)
|
||||
@IsString()
|
||||
@MinLength(1)
|
||||
@MaxLength(EMAIL_SUBJECT_MAX_LENGTH)
|
||||
subject: string;
|
||||
|
||||
@Field(() => String)
|
||||
@IsString()
|
||||
@MaxLength(EMAIL_BODY_MAX_LENGTH)
|
||||
text: string;
|
||||
|
||||
@Field(() => String, { nullable: true })
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@MaxLength(EMAIL_BODY_MAX_LENGTH)
|
||||
html?: string;
|
||||
|
||||
@Field(() => String)
|
||||
@@ -54,6 +69,7 @@ export class SendEmailViaDomainInput {
|
||||
@Field(() => [String], { nullable: true })
|
||||
@IsOptional()
|
||||
@IsArray()
|
||||
@ArrayMaxSize(EMAIL_MAX_REPLY_TO)
|
||||
@IsEmail({}, { each: true })
|
||||
replyTo?: string[];
|
||||
}
|
||||
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
import {
|
||||
Column,
|
||||
CreateDateColumn,
|
||||
Entity,
|
||||
PrimaryGeneratedColumn,
|
||||
Unique,
|
||||
UpdateDateColumn,
|
||||
} from 'typeorm';
|
||||
|
||||
import { FieldActorSource } from 'twenty-shared/types';
|
||||
|
||||
import { EmailGroupSuppressionReason } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-reason.type';
|
||||
import { EmailGroupSuppressionScope } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-scope.type';
|
||||
import { EmailGroupSuppressionStatus } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-status.type';
|
||||
import { WorkspaceRelatedEntity } from 'src/engine/workspace-manager/types/workspace-related-entity';
|
||||
|
||||
@Entity({ name: 'emailGroupSuppressedRecipient', schema: 'core' })
|
||||
@Unique('IDX_EMAIL_GROUP_SUPPRESSED_RECIPIENT_WORKSPACE_EMAIL_SCOPE_UNIQUE', [
|
||||
'workspaceId',
|
||||
'emailAddress',
|
||||
'scope',
|
||||
])
|
||||
export class EmailGroupSuppressedRecipientEntity extends WorkspaceRelatedEntity {
|
||||
@PrimaryGeneratedColumn('uuid')
|
||||
id: string;
|
||||
|
||||
@Column({ type: 'varchar', nullable: false })
|
||||
emailAddress: string;
|
||||
|
||||
@Column({
|
||||
type: 'enum',
|
||||
enum: Object.values(EmailGroupSuppressionScope),
|
||||
nullable: false,
|
||||
})
|
||||
scope: EmailGroupSuppressionScope;
|
||||
|
||||
@Column({
|
||||
type: 'enum',
|
||||
enum: Object.values(EmailGroupSuppressionReason),
|
||||
nullable: false,
|
||||
})
|
||||
reason: EmailGroupSuppressionReason;
|
||||
|
||||
@Column({
|
||||
type: 'enum',
|
||||
enum: Object.values(EmailGroupSuppressionStatus),
|
||||
default: EmailGroupSuppressionStatus.ACTIVE,
|
||||
nullable: false,
|
||||
})
|
||||
status: EmailGroupSuppressionStatus;
|
||||
|
||||
@Column({ type: 'varchar', nullable: true })
|
||||
providerEventId: string | null;
|
||||
|
||||
@Column({
|
||||
type: 'enum',
|
||||
enum: Object.values(FieldActorSource),
|
||||
nullable: false,
|
||||
})
|
||||
createdBySource: FieldActorSource;
|
||||
|
||||
@CreateDateColumn({ type: 'timestamptz' })
|
||||
createdAt: Date;
|
||||
|
||||
@UpdateDateColumn({ type: 'timestamptz' })
|
||||
updatedAt: Date;
|
||||
}
|
||||
+63
@@ -0,0 +1,63 @@
|
||||
import {
|
||||
Controller,
|
||||
Get,
|
||||
HttpCode,
|
||||
Post,
|
||||
Query,
|
||||
UseGuards,
|
||||
} from '@nestjs/common';
|
||||
|
||||
import { isNonEmptyString } from '@sniptt/guards';
|
||||
import { FieldActorSource } from 'twenty-shared/types';
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { EmailGroupSuppressionService } from 'src/engine/core-modules/emailing-domain/services/email-group-suppression.service';
|
||||
import { EmailGroupUnsubscribeService } from 'src/engine/core-modules/emailing-domain/services/email-group-unsubscribe.service';
|
||||
import { EmailGroupSuppressionReason } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-reason.type';
|
||||
import { NoPermissionGuard } from 'src/engine/guards/no-permission.guard';
|
||||
import { PublicEndpointGuard } from 'src/engine/guards/public-endpoint.guard';
|
||||
|
||||
@Controller('emailing')
|
||||
export class EmailingDomainUnsubscribeController {
|
||||
constructor(
|
||||
private readonly emailGroupUnsubscribeService: EmailGroupUnsubscribeService,
|
||||
private readonly emailGroupSuppressionService: EmailGroupSuppressionService,
|
||||
) {}
|
||||
|
||||
@Post('unsubscribe')
|
||||
@UseGuards(PublicEndpointGuard, NoPermissionGuard)
|
||||
@HttpCode(200)
|
||||
async unsubscribeOneClick(@Query('token') token?: string): Promise<void> {
|
||||
await this.unsubscribe(token);
|
||||
}
|
||||
|
||||
@Get('unsubscribe')
|
||||
@UseGuards(PublicEndpointGuard, NoPermissionGuard)
|
||||
@HttpCode(200)
|
||||
async unsubscribeFromBrowser(
|
||||
@Query('token') token?: string,
|
||||
): Promise<string> {
|
||||
await this.unsubscribe(token);
|
||||
|
||||
return 'You have been unsubscribed.';
|
||||
}
|
||||
|
||||
private async unsubscribe(token?: string): Promise<void> {
|
||||
if (!isNonEmptyString(token)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const payload = this.emailGroupUnsubscribeService.verifyToken(token);
|
||||
|
||||
if (!isDefined(payload)) {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.emailGroupSuppressionService.suppress(
|
||||
payload.workspaceId,
|
||||
payload.emailAddress,
|
||||
EmailGroupSuppressionReason.UNSUBSCRIBE,
|
||||
FieldActorSource.API,
|
||||
);
|
||||
}
|
||||
}
|
||||
+21
-2
@@ -3,30 +3,48 @@ import { Module } from '@nestjs/common';
|
||||
import { NestjsQueryTypeOrmModule } from '@ptc-org/nestjs-query-typeorm';
|
||||
|
||||
import { TypeORMModule } from 'src/database/typeorm/typeorm.module';
|
||||
import { BillingModule } from 'src/engine/core-modules/billing/billing.module';
|
||||
import { AwsSesClientProvider } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/providers/aws-ses-client.provider';
|
||||
import { AwsSesRegisterDomainService } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/services/aws-ses-register-domain.service';
|
||||
import { AwsSesHandleErrorService } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/services/aws-ses-handle-error.service';
|
||||
import { AwsSesSendEmailService } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/services/aws-ses-send-email.service';
|
||||
import { EmailingDomainDriverFactory } from 'src/engine/core-modules/emailing-domain/drivers/emailing-domain-driver.factory';
|
||||
import { EmailGroupSuppressedRecipientEntity } from 'src/engine/core-modules/emailing-domain/email-group-suppressed-recipient.entity';
|
||||
import { EmailingDomainUnsubscribeController } from 'src/engine/core-modules/emailing-domain/emailing-domain-unsubscribe.controller';
|
||||
import { EmailingDomainEntity } from 'src/engine/core-modules/emailing-domain/emailing-domain.entity';
|
||||
import { EmailingDomainResolver } from 'src/engine/core-modules/emailing-domain/emailing-domain.resolver';
|
||||
import { EmailingDomainWorkspaceCleanupJob } from 'src/engine/core-modules/emailing-domain/jobs/emailing-domain-workspace-cleanup.job';
|
||||
import { EmailGroupSuppressionService } from 'src/engine/core-modules/emailing-domain/services/email-group-suppression.service';
|
||||
import { EmailGroupUnsubscribeService } from 'src/engine/core-modules/emailing-domain/services/email-group-unsubscribe.service';
|
||||
import { EmailingDomainTenantStatusService } from 'src/engine/core-modules/emailing-domain/services/emailing-domain-tenant-status.service';
|
||||
import { EmailingDomainService } from 'src/engine/core-modules/emailing-domain/services/emailing-domain.service';
|
||||
import { FeatureFlagModule } from 'src/engine/core-modules/feature-flag/feature-flag.module';
|
||||
import { PermissionsModule } from 'src/engine/metadata-modules/permissions/permissions.module';
|
||||
import { ThrottlerModule } from 'src/engine/core-modules/throttler/throttler.module';
|
||||
import { provideWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/provide-workspace-scoped-repository';
|
||||
@Module({
|
||||
imports: [
|
||||
TypeORMModule,
|
||||
NestjsQueryTypeOrmModule.forFeature([EmailingDomainEntity]),
|
||||
NestjsQueryTypeOrmModule.forFeature([
|
||||
EmailingDomainEntity,
|
||||
EmailGroupSuppressedRecipientEntity,
|
||||
]),
|
||||
FeatureFlagModule,
|
||||
PermissionsModule,
|
||||
ThrottlerModule,
|
||||
BillingModule,
|
||||
],
|
||||
controllers: [EmailingDomainUnsubscribeController],
|
||||
exports: [
|
||||
EmailingDomainService,
|
||||
EmailingDomainTenantStatusService,
|
||||
EmailGroupSuppressionService,
|
||||
],
|
||||
exports: [EmailingDomainService, EmailingDomainTenantStatusService],
|
||||
providers: [
|
||||
EmailingDomainService,
|
||||
EmailingDomainTenantStatusService,
|
||||
EmailGroupSuppressionService,
|
||||
EmailGroupUnsubscribeService,
|
||||
EmailingDomainResolver,
|
||||
EmailingDomainDriverFactory,
|
||||
EmailingDomainWorkspaceCleanupJob,
|
||||
@@ -35,6 +53,7 @@ import { provideWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspac
|
||||
AwsSesRegisterDomainService,
|
||||
AwsSesSendEmailService,
|
||||
provideWorkspaceScopedRepository(EmailingDomainEntity),
|
||||
provideWorkspaceScopedRepository(EmailGroupSuppressedRecipientEntity),
|
||||
],
|
||||
})
|
||||
export class EmailingDomainModule {}
|
||||
|
||||
+4
-2
@@ -5,6 +5,7 @@ import { MessageQueue } from 'src/engine/core-modules/message-queue/message-queu
|
||||
|
||||
export type EmailingDomainWorkspaceCleanupJobData = {
|
||||
workspaceId: string;
|
||||
domains: string[];
|
||||
};
|
||||
|
||||
@Processor(MessageQueue.deleteCascadeQueue)
|
||||
@@ -13,11 +14,12 @@ export class EmailingDomainWorkspaceCleanupJob {
|
||||
|
||||
@Process(EmailingDomainWorkspaceCleanupJob.name)
|
||||
async handle(data: EmailingDomainWorkspaceCleanupJobData): Promise<void> {
|
||||
const { workspaceId } = data;
|
||||
const { workspaceId, domains } = data;
|
||||
|
||||
try {
|
||||
await this.emailingDomainService.cleanupAllEmailingDomainsForWorkspace(
|
||||
await this.emailingDomainService.cleanupEmailingDomainsForWorkspace(
|
||||
workspaceId,
|
||||
domains,
|
||||
);
|
||||
} catch (error) {
|
||||
throw new Error(
|
||||
|
||||
+208
-4
@@ -1,9 +1,18 @@
|
||||
import { EmailingDomainDriverExceptionCode } from 'src/engine/core-modules/emailing-domain/drivers/exceptions/emailing-domain-driver.exception';
|
||||
import { type EmailingDomainDriverFactory } from 'src/engine/core-modules/emailing-domain/drivers/emailing-domain-driver.factory';
|
||||
import { EmailingDomainDriver } from 'src/engine/core-modules/emailing-domain/drivers/types/emailing-domain-driver.type';
|
||||
import { type WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { EmailingDomainStatus } from 'src/engine/core-modules/emailing-domain/drivers/types/emailing-domain-status.type';
|
||||
import { EmailingDomainTenantStatus } from 'src/engine/core-modules/emailing-domain/drivers/types/emailing-domain-tenant-status.type';
|
||||
import { type EmailingDomainEmailContent } from 'src/engine/core-modules/emailing-domain/drivers/types/send-email';
|
||||
import { type EmailingDomainEntity } from 'src/engine/core-modules/emailing-domain/emailing-domain.entity';
|
||||
import { type EmailGroupSuppressionService } from 'src/engine/core-modules/emailing-domain/services/email-group-suppression.service';
|
||||
import { type EmailGroupUnsubscribeService } from 'src/engine/core-modules/emailing-domain/services/email-group-unsubscribe.service';
|
||||
import { EmailingDomainService } from 'src/engine/core-modules/emailing-domain/services/emailing-domain.service';
|
||||
import { type BillingUsageService } from 'src/engine/core-modules/billing/services/billing-usage.service';
|
||||
import { type ThrottlerService } from 'src/engine/core-modules/throttler/throttler.service';
|
||||
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
|
||||
import { type WorkspaceEventEmitter } from 'src/engine/workspace-event-emitter/workspace-event-emitter';
|
||||
import { type WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
|
||||
describe('EmailingDomainService.sendEmail', () => {
|
||||
@@ -19,14 +28,20 @@ describe('EmailingDomainService.sendEmail', () => {
|
||||
...overrides,
|
||||
}) as EmailingDomainEntity;
|
||||
|
||||
const buildEmailContent = () => ({
|
||||
const buildEmailContent = (
|
||||
overrides: Partial<EmailingDomainEmailContent> = {},
|
||||
): EmailingDomainEmailContent => ({
|
||||
from: 'hello@mail.example.com',
|
||||
to: ['user@example.com'],
|
||||
subject: 'Hi',
|
||||
text: 'Body',
|
||||
...overrides,
|
||||
});
|
||||
|
||||
const setUp = (emailingDomain: EmailingDomainEntity) => {
|
||||
const setUp = (
|
||||
emailingDomain: EmailingDomainEntity,
|
||||
suppressedAddresses: string[] = [],
|
||||
) => {
|
||||
const sendEmail = jest.fn().mockResolvedValue({ messageId: 'msg-1' });
|
||||
const repository = {
|
||||
findOne: jest.fn().mockResolvedValue(emailingDomain),
|
||||
@@ -34,9 +49,47 @@ describe('EmailingDomainService.sendEmail', () => {
|
||||
const factory = {
|
||||
getCurrentDriver: () => ({ sendEmail }),
|
||||
} as unknown as EmailingDomainDriverFactory;
|
||||
const service = new EmailingDomainService(repository, factory);
|
||||
const suppressionService = {
|
||||
getSuppressedAddresses: jest
|
||||
.fn()
|
||||
.mockResolvedValue(
|
||||
new Set(suppressedAddresses.map((address) => address.toLowerCase())),
|
||||
),
|
||||
} as unknown as EmailGroupSuppressionService;
|
||||
const unsubscribeService = {
|
||||
buildUnsubscribeHeaders: jest
|
||||
.fn()
|
||||
.mockReturnValue([
|
||||
{ name: 'List-Unsubscribe', value: '<https://app/unsubscribe>' },
|
||||
]),
|
||||
} as unknown as EmailGroupUnsubscribeService;
|
||||
const throttlerService = {
|
||||
tokenBucketThrottleOrThrow: jest.fn().mockResolvedValue(1),
|
||||
} as unknown as ThrottlerService;
|
||||
const billingUsageService = {
|
||||
canFeatureBeUsed: jest.fn().mockResolvedValue(true),
|
||||
} as unknown as BillingUsageService;
|
||||
const workspaceEventEmitter = {
|
||||
emitCustomBatchEvent: jest.fn(),
|
||||
} as unknown as WorkspaceEventEmitter;
|
||||
const service = new EmailingDomainService(
|
||||
repository,
|
||||
factory,
|
||||
suppressionService,
|
||||
unsubscribeService,
|
||||
throttlerService,
|
||||
billingUsageService,
|
||||
workspaceEventEmitter,
|
||||
);
|
||||
|
||||
return { service, sendEmail };
|
||||
return {
|
||||
service,
|
||||
sendEmail,
|
||||
unsubscribeService,
|
||||
throttlerService,
|
||||
billingUsageService,
|
||||
workspaceEventEmitter,
|
||||
};
|
||||
};
|
||||
|
||||
it('delegates to the driver when the domain is verified and the tenant is active', async () => {
|
||||
@@ -77,6 +130,157 @@ describe('EmailingDomainService.sendEmail', () => {
|
||||
},
|
||||
);
|
||||
|
||||
it('removes suppressed recipients but still sends to deliverable ones', async () => {
|
||||
const { service, sendEmail } = setUp(buildEmailingDomain(), [
|
||||
'blocked@example.com',
|
||||
]);
|
||||
|
||||
await service.sendEmail(
|
||||
'ws1',
|
||||
'domain-1',
|
||||
buildEmailContent({
|
||||
to: ['user@example.com', 'Blocked@example.com'],
|
||||
cc: ['blocked@example.com'],
|
||||
bcc: ['keep@example.com'],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(sendEmail).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
to: ['user@example.com'],
|
||||
cc: [],
|
||||
bcc: ['keep@example.com'],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects with ALL_RECIPIENTS_SUPPRESSED when every primary recipient is suppressed, without calling the driver', async () => {
|
||||
const { service, sendEmail } = setUp(buildEmailingDomain(), [
|
||||
'user@example.com',
|
||||
]);
|
||||
|
||||
await expect(
|
||||
service.sendEmail('ws1', 'domain-1', buildEmailContent()),
|
||||
).rejects.toMatchObject({
|
||||
code: EmailingDomainDriverExceptionCode.ALL_RECIPIENTS_SUPPRESSED,
|
||||
});
|
||||
expect(sendEmail).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('attaches one-click unsubscribe headers for a single-recipient marketing send', async () => {
|
||||
const { service, sendEmail } = setUp(buildEmailingDomain());
|
||||
|
||||
await service.sendEmail(
|
||||
'ws1',
|
||||
'domain-1',
|
||||
buildEmailContent({ includeUnsubscribe: true }),
|
||||
);
|
||||
|
||||
expect(sendEmail).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
headers: [
|
||||
{ name: 'List-Unsubscribe', value: '<https://app/unsubscribe>' },
|
||||
],
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it('omits unsubscribe headers when the message has multiple recipients', async () => {
|
||||
const { service, sendEmail, unsubscribeService } = setUp(
|
||||
buildEmailingDomain(),
|
||||
);
|
||||
|
||||
await service.sendEmail(
|
||||
'ws1',
|
||||
'domain-1',
|
||||
buildEmailContent({
|
||||
includeUnsubscribe: true,
|
||||
cc: ['second@example.com'],
|
||||
}),
|
||||
);
|
||||
|
||||
expect(unsubscribeService.buildUnsubscribeHeaders).not.toHaveBeenCalled();
|
||||
expect(sendEmail).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ headers: undefined }),
|
||||
);
|
||||
});
|
||||
|
||||
it('blocks provisioning a new emailing domain when the billing plan disallows the feature', async () => {
|
||||
const { service, billingUsageService } = setUp(buildEmailingDomain());
|
||||
|
||||
(billingUsageService.canFeatureBeUsed as jest.Mock).mockResolvedValue(
|
||||
false,
|
||||
);
|
||||
|
||||
await expect(
|
||||
service.createEmailingDomain(
|
||||
'mail.example.com',
|
||||
EmailingDomainDriver.AWS_SES,
|
||||
{
|
||||
id: 'ws1',
|
||||
} as WorkspaceEntity,
|
||||
),
|
||||
).rejects.toMatchObject({
|
||||
code: EmailingDomainDriverExceptionCode.SENDING_SUSPENDED,
|
||||
});
|
||||
});
|
||||
|
||||
it('meters one usage event per accepted recipient at the marked-up per-recipient cost', async () => {
|
||||
const { service, workspaceEventEmitter } = setUp(buildEmailingDomain());
|
||||
|
||||
await service.sendEmail(
|
||||
'ws1',
|
||||
'domain-1',
|
||||
buildEmailContent({ to: ['a@example.com', 'b@example.com'] }),
|
||||
);
|
||||
|
||||
expect(workspaceEventEmitter.emitCustomBatchEvent).toHaveBeenCalledWith(
|
||||
'USAGE_RECORDED',
|
||||
[
|
||||
expect.objectContaining({
|
||||
operationType: UsageOperationType.EMAIL_SEND,
|
||||
quantity: 2,
|
||||
creditsUsedMicro: 600,
|
||||
}),
|
||||
],
|
||||
'ws1',
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects sending when the billing plan disallows it, without calling the driver or metering', async () => {
|
||||
const { service, sendEmail, billingUsageService, workspaceEventEmitter } =
|
||||
setUp(buildEmailingDomain());
|
||||
|
||||
(billingUsageService.canFeatureBeUsed as jest.Mock).mockResolvedValue(
|
||||
false,
|
||||
);
|
||||
|
||||
await expect(
|
||||
service.sendEmail('ws1', 'domain-1', buildEmailContent()),
|
||||
).rejects.toMatchObject({
|
||||
code: EmailingDomainDriverExceptionCode.SENDING_SUSPENDED,
|
||||
});
|
||||
expect(sendEmail).not.toHaveBeenCalled();
|
||||
expect(workspaceEventEmitter.emitCustomBatchEvent).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('consumes one rate-limit token per deliverable recipient before sending', async () => {
|
||||
const { service, throttlerService } = setUp(buildEmailingDomain());
|
||||
|
||||
await service.sendEmail(
|
||||
'ws1',
|
||||
'domain-1',
|
||||
buildEmailContent({ to: ['a@example.com'], cc: ['c@example.com'] }),
|
||||
);
|
||||
|
||||
expect(throttlerService.tokenBucketThrottleOrThrow).toHaveBeenCalledWith(
|
||||
'emailing-domain-send:ws1',
|
||||
2,
|
||||
expect.any(Number),
|
||||
expect.any(Number),
|
||||
);
|
||||
});
|
||||
|
||||
// Verification is a precondition for the tenant-status check: a domain that
|
||||
// has not been verified should surface a CONFIGURATION_ERROR rather than
|
||||
// leaking the tenant pause state to callers who couldn't have used it anyway.
|
||||
|
||||
+96
@@ -0,0 +1,96 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { isNonEmptyArray } from '@sniptt/guards';
|
||||
import { FieldActorSource } from 'twenty-shared/types';
|
||||
import { In } from 'typeorm';
|
||||
|
||||
import { EmailGroupSuppressedRecipientEntity } from 'src/engine/core-modules/emailing-domain/email-group-suppressed-recipient.entity';
|
||||
import { EmailGroupSuppressionReason } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-reason.type';
|
||||
import { EmailGroupSuppressionScope } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-scope.type';
|
||||
import { EmailGroupSuppressionStatus } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-status.type';
|
||||
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
||||
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
|
||||
const REVERSIBLE_REASONS: ReadonlySet<EmailGroupSuppressionReason> = new Set([
|
||||
EmailGroupSuppressionReason.UNSUBSCRIBE,
|
||||
]);
|
||||
|
||||
@Injectable()
|
||||
export class EmailGroupSuppressionService {
|
||||
constructor(
|
||||
@InjectWorkspaceScopedRepository(EmailGroupSuppressedRecipientEntity)
|
||||
private readonly suppressedRecipientRepository: WorkspaceScopedRepository<EmailGroupSuppressedRecipientEntity>,
|
||||
) {}
|
||||
|
||||
async getSuppressedAddresses(
|
||||
workspaceId: string,
|
||||
emailAddresses: string[],
|
||||
): Promise<Set<string>> {
|
||||
const normalizedAddresses = [
|
||||
...new Set(emailAddresses.map(normalizeEmailAddress)),
|
||||
];
|
||||
|
||||
if (!isNonEmptyArray(normalizedAddresses)) {
|
||||
return new Set();
|
||||
}
|
||||
|
||||
const activeSuppressions = await this.suppressedRecipientRepository.find(
|
||||
workspaceId,
|
||||
{
|
||||
where: {
|
||||
emailAddress: In(normalizedAddresses),
|
||||
status: EmailGroupSuppressionStatus.ACTIVE,
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
return new Set(
|
||||
activeSuppressions.map((suppression) => suppression.emailAddress),
|
||||
);
|
||||
}
|
||||
|
||||
async suppress(
|
||||
workspaceId: string,
|
||||
emailAddress: string,
|
||||
reason: EmailGroupSuppressionReason,
|
||||
createdBySource: FieldActorSource,
|
||||
providerEventId: string | null = null,
|
||||
): Promise<void> {
|
||||
await this.suppressedRecipientRepository.upsert(
|
||||
workspaceId,
|
||||
{
|
||||
emailAddress: normalizeEmailAddress(emailAddress),
|
||||
scope: scopeForReason(reason),
|
||||
reason,
|
||||
status: EmailGroupSuppressionStatus.ACTIVE,
|
||||
createdBySource,
|
||||
providerEventId,
|
||||
},
|
||||
['workspaceId', 'emailAddress', 'scope'],
|
||||
);
|
||||
}
|
||||
|
||||
async releaseMarketingSuppression(
|
||||
workspaceId: string,
|
||||
emailAddress: string,
|
||||
): Promise<void> {
|
||||
await this.suppressedRecipientRepository.update(
|
||||
workspaceId,
|
||||
{
|
||||
emailAddress: normalizeEmailAddress(emailAddress),
|
||||
scope: EmailGroupSuppressionScope.MARKETING,
|
||||
},
|
||||
{ status: EmailGroupSuppressionStatus.RELEASED },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const scopeForReason = (
|
||||
reason: EmailGroupSuppressionReason,
|
||||
): EmailGroupSuppressionScope =>
|
||||
REVERSIBLE_REASONS.has(reason)
|
||||
? EmailGroupSuppressionScope.MARKETING
|
||||
: EmailGroupSuppressionScope.GLOBAL;
|
||||
|
||||
const normalizeEmailAddress = (emailAddress: string): string =>
|
||||
emailAddress.trim().toLowerCase();
|
||||
+98
@@ -0,0 +1,98 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
|
||||
import { createHmac, timingSafeEqual } from 'crypto';
|
||||
|
||||
import { isDefined } from 'twenty-shared/utils';
|
||||
|
||||
import { TwentyConfigService } from 'src/engine/core-modules/twenty-config/twenty-config.service';
|
||||
|
||||
type UnsubscribeTokenPayload = {
|
||||
workspaceId: string;
|
||||
emailAddress: string;
|
||||
};
|
||||
|
||||
type EmailHeader = {
|
||||
name: string;
|
||||
value: string;
|
||||
};
|
||||
|
||||
@Injectable()
|
||||
export class EmailGroupUnsubscribeService {
|
||||
constructor(private readonly twentyConfigService: TwentyConfigService) {}
|
||||
|
||||
buildUnsubscribeHeaders(
|
||||
workspaceId: string,
|
||||
emailAddress: string,
|
||||
): EmailHeader[] {
|
||||
const token = this.signToken({ workspaceId, emailAddress });
|
||||
const serverUrl = this.twentyConfigService.get('SERVER_URL');
|
||||
const unsubscribeUrl = `${serverUrl}/emailing/unsubscribe?token=${encodeURIComponent(token)}`;
|
||||
|
||||
return [
|
||||
{ name: 'List-Unsubscribe', value: `<${unsubscribeUrl}>` },
|
||||
{ name: 'List-Unsubscribe-Post', value: 'List-Unsubscribe=One-Click' },
|
||||
];
|
||||
}
|
||||
|
||||
verifyToken(token: string): UnsubscribeTokenPayload | null {
|
||||
const [encodedPayload, signature] = token.split('.');
|
||||
|
||||
if (!isDefined(encodedPayload) || !isDefined(signature)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const expectedSignature = this.computeSignature(encodedPayload);
|
||||
|
||||
if (!isSignatureEqual(signature, expectedSignature)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const payload = parsePayload(encodedPayload);
|
||||
|
||||
if (!isValidPayload(payload)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return payload;
|
||||
}
|
||||
|
||||
private signToken(payload: UnsubscribeTokenPayload): string {
|
||||
const encodedPayload = base64UrlEncode(JSON.stringify(payload));
|
||||
|
||||
return `${encodedPayload}.${this.computeSignature(encodedPayload)}`;
|
||||
}
|
||||
|
||||
private computeSignature(encodedPayload: string): string {
|
||||
return createHmac('sha256', this.twentyConfigService.get('APP_SECRET'))
|
||||
.update(encodedPayload)
|
||||
.digest('base64url');
|
||||
}
|
||||
}
|
||||
|
||||
const base64UrlEncode = (value: string): string =>
|
||||
Buffer.from(value, 'utf8').toString('base64url');
|
||||
|
||||
const parsePayload = (encodedPayload: string): unknown => {
|
||||
try {
|
||||
return JSON.parse(
|
||||
Buffer.from(encodedPayload, 'base64url').toString('utf8'),
|
||||
);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
const isValidPayload = (payload: unknown): payload is UnsubscribeTokenPayload =>
|
||||
isDefined(payload) &&
|
||||
typeof (payload as UnsubscribeTokenPayload).workspaceId === 'string' &&
|
||||
typeof (payload as UnsubscribeTokenPayload).emailAddress === 'string';
|
||||
|
||||
const isSignatureEqual = (candidate: string, expected: string): boolean => {
|
||||
const candidateBuffer = Buffer.from(candidate);
|
||||
const expectedBuffer = Buffer.from(expected);
|
||||
|
||||
return (
|
||||
candidateBuffer.length === expectedBuffer.length &&
|
||||
timingSafeEqual(candidateBuffer, expectedBuffer)
|
||||
);
|
||||
};
|
||||
+170
-23
@@ -1,5 +1,15 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { isNonEmptyArray } from '@sniptt/guards';
|
||||
|
||||
import { BillingUsageService } from 'src/engine/core-modules/billing/services/billing-usage.service';
|
||||
import {
|
||||
EMAIL_SEND_AWS_COST_PER_RECIPIENT_USD,
|
||||
EMAIL_SEND_CREDIT_MARKUP,
|
||||
EMAIL_SEND_THROTTLE_MAX_RECIPIENTS,
|
||||
EMAIL_SEND_THROTTLE_WINDOW_MS,
|
||||
} from 'src/engine/core-modules/emailing-domain/constants/email-send-billing.constant';
|
||||
import { EMAIL_MAX_TOTAL_RECIPIENTS } from 'src/engine/core-modules/emailing-domain/constants/email-limits.constant';
|
||||
import {
|
||||
EmailingDomainDriverException,
|
||||
EmailingDomainDriverExceptionCode,
|
||||
@@ -13,9 +23,19 @@ import {
|
||||
type EmailingDomainSendEmailResult,
|
||||
} from 'src/engine/core-modules/emailing-domain/drivers/types/send-email';
|
||||
import { EmailingDomainEntity } from 'src/engine/core-modules/emailing-domain/emailing-domain.entity';
|
||||
import { EmailGroupSuppressionService } from 'src/engine/core-modules/emailing-domain/services/email-group-suppression.service';
|
||||
import { EmailGroupUnsubscribeService } from 'src/engine/core-modules/emailing-domain/services/email-group-unsubscribe.service';
|
||||
import { ThrottlerService } from 'src/engine/core-modules/throttler/throttler.service';
|
||||
import { USAGE_RECORDED } from 'src/engine/core-modules/usage/constants/usage-recorded.constant';
|
||||
import { UsageOperationType } from 'src/engine/core-modules/usage/enums/usage-operation-type.enum';
|
||||
import { UsageResourceType } from 'src/engine/core-modules/usage/enums/usage-resource-type.enum';
|
||||
import { UsageUnit } from 'src/engine/core-modules/usage/enums/usage-unit.enum';
|
||||
import { type UsageEvent } from 'src/engine/core-modules/usage/types/usage-event.type';
|
||||
import { WorkspaceEntity } from 'src/engine/core-modules/workspace/workspace.entity';
|
||||
import { DOLLAR_TO_CREDIT_MULTIPLIER } from 'src/engine/metadata-modules/ai/ai-billing/constants/dollar-to-credit-multiplier';
|
||||
import { InjectWorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/inject-workspace-scoped-repository.decorator';
|
||||
import { WorkspaceScopedRepository } from 'src/engine/twenty-orm/workspace-scoped-repository/workspace-scoped-repository';
|
||||
import { WorkspaceEventEmitter } from 'src/engine/workspace-event-emitter/workspace-event-emitter';
|
||||
@Injectable()
|
||||
export class EmailingDomainService {
|
||||
private readonly logger = new Logger(EmailingDomainService.name);
|
||||
@@ -24,6 +44,11 @@ export class EmailingDomainService {
|
||||
@InjectWorkspaceScopedRepository(EmailingDomainEntity)
|
||||
private readonly emailingDomainRepository: WorkspaceScopedRepository<EmailingDomainEntity>,
|
||||
private readonly emailingDomainDriverFactory: EmailingDomainDriverFactory,
|
||||
private readonly emailGroupSuppressionService: EmailGroupSuppressionService,
|
||||
private readonly emailGroupUnsubscribeService: EmailGroupUnsubscribeService,
|
||||
private readonly throttlerService: ThrottlerService,
|
||||
private readonly billingUsageService: BillingUsageService,
|
||||
private readonly workspaceEventEmitter: WorkspaceEventEmitter,
|
||||
) {}
|
||||
|
||||
async createEmailingDomain(
|
||||
@@ -31,6 +56,8 @@ export class EmailingDomainService {
|
||||
driverType: EmailingDomainDriver,
|
||||
workspace: WorkspaceEntity,
|
||||
): Promise<EmailingDomainEntity> {
|
||||
await this.assertBillingAllowsEmailGroup(workspace.id);
|
||||
|
||||
const existingEmailingDomain = await this.emailingDomainRepository.findOne(
|
||||
workspace.id,
|
||||
{
|
||||
@@ -87,18 +114,18 @@ export class EmailingDomainService {
|
||||
});
|
||||
}
|
||||
|
||||
async cleanupAllEmailingDomainsForWorkspace(
|
||||
async cleanupEmailingDomainsForWorkspace(
|
||||
workspaceId: string,
|
||||
domains: string[],
|
||||
): Promise<void> {
|
||||
const emailingDomains =
|
||||
await this.emailingDomainRepository.find(workspaceId);
|
||||
const emailingDomainDriver =
|
||||
this.emailingDomainDriverFactory.getCurrentDriver();
|
||||
|
||||
for (const emailingDomain of emailingDomains) {
|
||||
await this.deleteRemoteEmailingDomain(emailingDomain);
|
||||
for (const domain of domains) {
|
||||
await emailingDomainDriver.cleanupDomain({ domain, workspaceId });
|
||||
}
|
||||
|
||||
await this.deprovisionRemoteWorkspace(workspaceId);
|
||||
await this.emailingDomainRepository.delete(workspaceId, {});
|
||||
await emailingDomainDriver.deprovisionWorkspace(workspaceId);
|
||||
}
|
||||
|
||||
async getEmailingDomains(
|
||||
@@ -178,11 +205,143 @@ export class EmailingDomainService {
|
||||
);
|
||||
}
|
||||
|
||||
return this.emailingDomainDriverFactory.getCurrentDriver().sendEmail({
|
||||
...emailContent,
|
||||
const suppressedAddresses =
|
||||
await this.emailGroupSuppressionService.getSuppressedAddresses(
|
||||
workspaceId,
|
||||
[
|
||||
...emailContent.to,
|
||||
...(emailContent.cc ?? []),
|
||||
...(emailContent.bcc ?? []),
|
||||
],
|
||||
);
|
||||
|
||||
const isNotSuppressed = (address: string): boolean =>
|
||||
!suppressedAddresses.has(address.trim().toLowerCase());
|
||||
|
||||
const deliverableTo = emailContent.to.filter(isNotSuppressed);
|
||||
|
||||
if (deliverableTo.length === 0) {
|
||||
throw new EmailingDomainDriverException(
|
||||
`All primary recipients are suppressed for emailing domain ${emailingDomain.domain}`,
|
||||
EmailingDomainDriverExceptionCode.ALL_RECIPIENTS_SUPPRESSED,
|
||||
);
|
||||
}
|
||||
|
||||
const deliverableCc = emailContent.cc?.filter(isNotSuppressed);
|
||||
const deliverableBcc = emailContent.bcc?.filter(isNotSuppressed);
|
||||
|
||||
const recipientCount =
|
||||
deliverableTo.length +
|
||||
(deliverableCc?.length ?? 0) +
|
||||
(deliverableBcc?.length ?? 0);
|
||||
|
||||
if (recipientCount > EMAIL_MAX_TOTAL_RECIPIENTS) {
|
||||
throw new EmailingDomainDriverException(
|
||||
`A single email cannot exceed ${EMAIL_MAX_TOTAL_RECIPIENTS} recipients`,
|
||||
EmailingDomainDriverExceptionCode.CONFIGURATION_ERROR,
|
||||
);
|
||||
}
|
||||
|
||||
await this.assertBillingAllowsEmailGroup(workspaceId);
|
||||
await this.assertWithinSendRate(workspaceId, recipientCount);
|
||||
|
||||
const headers = this.buildSingleRecipientUnsubscribeHeaders(
|
||||
workspaceId,
|
||||
domain: emailingDomain.domain,
|
||||
});
|
||||
emailContent,
|
||||
deliverableTo,
|
||||
deliverableCc,
|
||||
deliverableBcc,
|
||||
);
|
||||
|
||||
const result = await this.emailingDomainDriverFactory
|
||||
.getCurrentDriver()
|
||||
.sendEmail({
|
||||
...emailContent,
|
||||
to: deliverableTo,
|
||||
cc: deliverableCc,
|
||||
bcc: deliverableBcc,
|
||||
headers,
|
||||
workspaceId,
|
||||
domain: emailingDomain.domain,
|
||||
});
|
||||
|
||||
this.meterEmailSend(workspaceId, recipientCount);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
private async assertBillingAllowsEmailGroup(
|
||||
workspaceId: string,
|
||||
): Promise<void> {
|
||||
const isEntitled =
|
||||
await this.billingUsageService.canFeatureBeUsed(workspaceId);
|
||||
|
||||
if (!isEntitled) {
|
||||
throw new EmailingDomainDriverException(
|
||||
'The email domain feature requires an active billing plan',
|
||||
EmailingDomainDriverExceptionCode.SENDING_SUSPENDED,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private async assertWithinSendRate(
|
||||
workspaceId: string,
|
||||
recipientCount: number,
|
||||
): Promise<void> {
|
||||
await this.throttlerService.tokenBucketThrottleOrThrow(
|
||||
`emailing-domain-send:${workspaceId}`,
|
||||
recipientCount,
|
||||
EMAIL_SEND_THROTTLE_MAX_RECIPIENTS,
|
||||
EMAIL_SEND_THROTTLE_WINDOW_MS,
|
||||
);
|
||||
}
|
||||
|
||||
private meterEmailSend(workspaceId: string, recipientCount: number): void {
|
||||
const creditsUsedMicro = Math.round(
|
||||
EMAIL_SEND_AWS_COST_PER_RECIPIENT_USD *
|
||||
EMAIL_SEND_CREDIT_MARKUP *
|
||||
DOLLAR_TO_CREDIT_MULTIPLIER *
|
||||
recipientCount,
|
||||
);
|
||||
|
||||
this.workspaceEventEmitter.emitCustomBatchEvent<UsageEvent>(
|
||||
USAGE_RECORDED,
|
||||
[
|
||||
{
|
||||
resourceType: UsageResourceType.API,
|
||||
operationType: UsageOperationType.EMAIL_SEND,
|
||||
creditsUsedMicro,
|
||||
quantity: recipientCount,
|
||||
unit: UsageUnit.INVOCATION,
|
||||
},
|
||||
],
|
||||
workspaceId,
|
||||
);
|
||||
}
|
||||
|
||||
private buildSingleRecipientUnsubscribeHeaders(
|
||||
workspaceId: string,
|
||||
emailContent: EmailingDomainEmailContent,
|
||||
deliverableTo: string[],
|
||||
deliverableCc: string[] | undefined,
|
||||
deliverableBcc: string[] | undefined,
|
||||
): EmailingDomainEmailContent['headers'] {
|
||||
const isSingleRecipient =
|
||||
deliverableTo.length === 1 &&
|
||||
!isNonEmptyArray(deliverableCc) &&
|
||||
!isNonEmptyArray(deliverableBcc);
|
||||
|
||||
if (emailContent.includeUnsubscribe !== true || !isSingleRecipient) {
|
||||
return emailContent.headers;
|
||||
}
|
||||
|
||||
return [
|
||||
...(emailContent.headers ?? []),
|
||||
...this.emailGroupUnsubscribeService.buildUnsubscribeHeaders(
|
||||
workspaceId,
|
||||
deliverableTo[0],
|
||||
),
|
||||
];
|
||||
}
|
||||
|
||||
private async findEmailingDomainByIdOrThrow(
|
||||
@@ -220,16 +379,4 @@ export class EmailingDomainService {
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private async deprovisionRemoteWorkspace(workspaceId: string): Promise<void> {
|
||||
try {
|
||||
await this.emailingDomainDriverFactory
|
||||
.getCurrentDriver()
|
||||
.deprovisionWorkspace(workspaceId);
|
||||
} catch (error) {
|
||||
this.logger.warn(
|
||||
`Remote deprovision for emailing domain workspace ${workspaceId} failed: ${error}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
export enum EmailGroupSuppressionReason {
|
||||
HARD_BOUNCE = 'HARD_BOUNCE',
|
||||
COMPLAINT = 'COMPLAINT',
|
||||
UNSUBSCRIBE = 'UNSUBSCRIBE',
|
||||
}
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
export enum EmailGroupSuppressionScope {
|
||||
GLOBAL = 'GLOBAL',
|
||||
MARKETING = 'MARKETING',
|
||||
}
|
||||
+4
@@ -0,0 +1,4 @@
|
||||
export enum EmailGroupSuppressionStatus {
|
||||
ACTIVE = 'ACTIVE',
|
||||
RELEASED = 'RELEASED',
|
||||
}
|
||||
+2
@@ -6,6 +6,7 @@ import { SesInboundMailHandlerService } from 'src/engine/core-modules/messaging-
|
||||
import { SesInboundWebhookRouterService } from 'src/engine/core-modules/messaging-webhooks/services/ses-inbound-webhook-router.service';
|
||||
import { SesOutboundSendingStateHandlerService } from 'src/engine/core-modules/messaging-webhooks/services/ses-outbound-sending-state-handler.service';
|
||||
import { SesOutboundWebhookRouterService } from 'src/engine/core-modules/messaging-webhooks/services/ses-outbound-webhook-router.service';
|
||||
import { SesSuppressionEventHandlerService } from 'src/engine/core-modules/messaging-webhooks/services/ses-suppression-event-handler.service';
|
||||
import { SnsSignatureVerifierService } from 'src/engine/core-modules/messaging-webhooks/services/sns-signature-verifier.service';
|
||||
import { SnsSubscriptionConfirmerService } from 'src/engine/core-modules/messaging-webhooks/services/sns-subscription-confirmer.service';
|
||||
import { TwentyConfigModule } from 'src/engine/core-modules/twenty-config/twenty-config.module';
|
||||
@@ -18,6 +19,7 @@ import { TwentyConfigModule } from 'src/engine/core-modules/twenty-config/twenty
|
||||
SnsSubscriptionConfirmerService,
|
||||
SesInboundMailHandlerService,
|
||||
SesOutboundSendingStateHandlerService,
|
||||
SesSuppressionEventHandlerService,
|
||||
SesInboundWebhookRouterService,
|
||||
SesOutboundWebhookRouterService,
|
||||
],
|
||||
|
||||
+1
@@ -38,6 +38,7 @@ export class SesInboundMailHandlerService {
|
||||
s3Key: receipt.action.objectKey,
|
||||
envelopeRecipients: receipt.recipients,
|
||||
},
|
||||
{ id: snsMessageId },
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
+17
-1
@@ -6,6 +6,7 @@ import { isDefined, parseJson } from 'twenty-shared/utils';
|
||||
import { MessagingWebhookExceptionCode } from 'src/engine/core-modules/messaging-webhooks/messaging-webhook-exception-code.enum';
|
||||
import { MessagingWebhookException } from 'src/engine/core-modules/messaging-webhooks/messaging-webhook.exception';
|
||||
import { SesOutboundSendingStateHandlerService } from 'src/engine/core-modules/messaging-webhooks/services/ses-outbound-sending-state-handler.service';
|
||||
import { SesSuppressionEventHandlerService } from 'src/engine/core-modules/messaging-webhooks/services/ses-suppression-event-handler.service';
|
||||
import { SnsSignatureVerifierService } from 'src/engine/core-modules/messaging-webhooks/services/sns-signature-verifier.service';
|
||||
import { SnsSubscriptionConfirmerService } from 'src/engine/core-modules/messaging-webhooks/services/sns-subscription-confirmer.service';
|
||||
import { type SesEventBridgeNotification } from 'src/engine/core-modules/messaging-webhooks/types/ses-event-bridge-notification.type';
|
||||
@@ -18,6 +19,7 @@ export class SesOutboundWebhookRouterService {
|
||||
private readonly snsSignatureVerifierService: SnsSignatureVerifierService,
|
||||
private readonly snsSubscriptionConfirmerService: SnsSubscriptionConfirmerService,
|
||||
private readonly sesOutboundSendingStateHandlerService: SesOutboundSendingStateHandlerService,
|
||||
private readonly sesSuppressionEventHandlerService: SesSuppressionEventHandlerService,
|
||||
) {}
|
||||
|
||||
async route(rawBody: Buffer): Promise<void> {
|
||||
@@ -54,6 +56,20 @@ export class SesOutboundWebhookRouterService {
|
||||
);
|
||||
}
|
||||
|
||||
await this.sesOutboundSendingStateHandlerService.handle(event);
|
||||
if (
|
||||
event['detail-type'] === 'Sending Status Enabled' ||
|
||||
event['detail-type'] === 'Sending Status Disabled'
|
||||
) {
|
||||
await this.sesOutboundSendingStateHandlerService.handle(event);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
if (
|
||||
event['detail-type'] === 'Email Bounced' ||
|
||||
event['detail-type'] === 'Email Complaint Received'
|
||||
) {
|
||||
await this.sesSuppressionEventHandlerService.handle(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
import { Injectable, Logger } from '@nestjs/common';
|
||||
|
||||
import { FieldActorSource } from 'twenty-shared/types';
|
||||
import { isDefined, isNonEmptyArray } from 'twenty-shared/utils';
|
||||
|
||||
import { AWS_SES_WORKSPACE_TAG_NAME } from 'src/engine/core-modules/emailing-domain/drivers/aws-ses/constants/aws-ses-workspace-tag-name.constant';
|
||||
import { EmailGroupSuppressionService } from 'src/engine/core-modules/emailing-domain/services/email-group-suppression.service';
|
||||
import { EmailGroupSuppressionReason } from 'src/engine/core-modules/emailing-domain/types/email-group-suppression-reason.type';
|
||||
import { type SesEventBridgeNotification } from 'src/engine/core-modules/messaging-webhooks/types/ses-event-bridge-notification.type';
|
||||
import { parseWorkspaceIdFromAwsSesResourceArn } from 'src/engine/core-modules/messaging-webhooks/utils/parse-workspace-id-from-aws-ses-resource-arn.util';
|
||||
|
||||
@Injectable()
|
||||
export class SesSuppressionEventHandlerService {
|
||||
private readonly logger = new Logger(SesSuppressionEventHandlerService.name);
|
||||
|
||||
constructor(
|
||||
private readonly emailGroupSuppressionService: EmailGroupSuppressionService,
|
||||
) {}
|
||||
|
||||
async handle(event: SesEventBridgeNotification): Promise<void> {
|
||||
const suppression = this.resolveSuppression(event);
|
||||
|
||||
if (!isDefined(suppression)) {
|
||||
return;
|
||||
}
|
||||
|
||||
const workspaceId = this.resolveWorkspaceId(event);
|
||||
|
||||
if (!isDefined(workspaceId)) {
|
||||
this.logger.warn(
|
||||
`Could not resolve workspaceId for SES ${event['detail-type']} event`,
|
||||
);
|
||||
|
||||
return;
|
||||
}
|
||||
|
||||
for (const emailAddress of suppression.emailAddresses) {
|
||||
await this.emailGroupSuppressionService.suppress(
|
||||
workspaceId,
|
||||
emailAddress,
|
||||
suppression.reason,
|
||||
FieldActorSource.WEBHOOK,
|
||||
suppression.feedbackId,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
private resolveSuppression(event: SesEventBridgeNotification): {
|
||||
reason: EmailGroupSuppressionReason;
|
||||
emailAddresses: string[];
|
||||
feedbackId: string | null;
|
||||
} | null {
|
||||
if (event['detail-type'] === 'Email Bounced') {
|
||||
const bounce = event.detail?.bounce;
|
||||
|
||||
if (bounce?.bounceType !== 'Permanent') {
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
reason: EmailGroupSuppressionReason.HARD_BOUNCE,
|
||||
emailAddresses: extractEmailAddresses(bounce.bouncedRecipients),
|
||||
feedbackId: bounce.feedbackId ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
if (event['detail-type'] === 'Email Complaint Received') {
|
||||
const complaint = event.detail?.complaint;
|
||||
|
||||
return {
|
||||
reason: EmailGroupSuppressionReason.COMPLAINT,
|
||||
emailAddresses: extractEmailAddresses(complaint?.complainedRecipients),
|
||||
feedbackId: complaint?.feedbackId ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
private resolveWorkspaceId(event: SesEventBridgeNotification): string | null {
|
||||
const taggedWorkspaceId =
|
||||
event.detail?.mail?.tags?.[AWS_SES_WORKSPACE_TAG_NAME]?.[0];
|
||||
|
||||
if (isDefined(taggedWorkspaceId) && taggedWorkspaceId.length > 0) {
|
||||
return taggedWorkspaceId;
|
||||
}
|
||||
|
||||
if (!isNonEmptyArray(event.resources)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
for (const resourceArn of event.resources) {
|
||||
const workspaceId = parseWorkspaceIdFromAwsSesResourceArn(resourceArn);
|
||||
|
||||
if (isDefined(workspaceId)) {
|
||||
return workspaceId;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
const extractEmailAddresses = (
|
||||
recipients: { emailAddress: string }[] | undefined,
|
||||
): string[] =>
|
||||
isNonEmptyArray(recipients)
|
||||
? recipients.map((recipient) => recipient.emailAddress)
|
||||
: [];
|
||||
+43
-12
@@ -1,15 +1,46 @@
|
||||
export type SesEventBridgeNotification = {
|
||||
source: 'aws.ses';
|
||||
'detail-type': 'Sending Status Enabled' | 'Sending Status Disabled';
|
||||
resources?: string[];
|
||||
detail?: {
|
||||
version?: string;
|
||||
data?: {
|
||||
origin?: string;
|
||||
record?: {
|
||||
status?: 'ENABLED' | 'DISABLED';
|
||||
cause?: string;
|
||||
};
|
||||
export type SesEventBridgeDetailType =
|
||||
| 'Sending Status Enabled'
|
||||
| 'Sending Status Disabled'
|
||||
| 'Email Bounced'
|
||||
| 'Email Complaint Received';
|
||||
|
||||
type SesEventRecipient = {
|
||||
emailAddress: string;
|
||||
};
|
||||
|
||||
type SesMessageEventDetail = {
|
||||
eventType?: string;
|
||||
mail?: {
|
||||
messageId?: string;
|
||||
destination?: string[];
|
||||
tags?: Record<string, string[]>;
|
||||
};
|
||||
bounce?: {
|
||||
bounceType?: 'Permanent' | 'Transient' | 'Undetermined';
|
||||
bouncedRecipients?: SesEventRecipient[];
|
||||
feedbackId?: string;
|
||||
};
|
||||
complaint?: {
|
||||
complainedRecipients?: SesEventRecipient[];
|
||||
feedbackId?: string;
|
||||
complaintFeedbackType?: string;
|
||||
};
|
||||
};
|
||||
|
||||
type SesSendingStateEventDetail = {
|
||||
version?: string;
|
||||
data?: {
|
||||
origin?: string;
|
||||
record?: {
|
||||
status?: 'ENABLED' | 'DISABLED';
|
||||
cause?: string;
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
export type SesEventBridgeNotification = {
|
||||
source: 'aws.ses';
|
||||
'detail-type': SesEventBridgeDetailType;
|
||||
resources?: string[];
|
||||
detail?: SesMessageEventDetail & SesSendingStateEventDetail;
|
||||
};
|
||||
|
||||
+1
@@ -8,6 +8,7 @@ export enum UsageOperationType {
|
||||
WORKFLOW_EXECUTION = 'WORKFLOW_EXECUTION',
|
||||
CODE_EXECUTION = 'CODE_EXECUTION',
|
||||
WEB_SEARCH = 'WEB_SEARCH',
|
||||
EMAIL_SEND = 'EMAIL_SEND',
|
||||
}
|
||||
|
||||
registerEnumType(UsageOperationType, {
|
||||
|
||||
+11
-1
@@ -22,6 +22,7 @@ import { CustomDomainManagerService } from 'src/engine/core-modules/domain/custo
|
||||
import { SubdomainManagerService } from 'src/engine/core-modules/domain/subdomain-manager/services/subdomain-manager.service';
|
||||
import { ExceptionHandlerService } from 'src/engine/core-modules/exception-handler/exception-handler.service';
|
||||
import { FeatureFlagService } from 'src/engine/core-modules/feature-flag/services/feature-flag.service';
|
||||
import { EmailingDomainEntity } from 'src/engine/core-modules/emailing-domain/emailing-domain.entity';
|
||||
import {
|
||||
EmailingDomainWorkspaceCleanupJob,
|
||||
type EmailingDomainWorkspaceCleanupJobData,
|
||||
@@ -512,9 +513,18 @@ export class WorkspaceService extends TypeOrmQueryService<WorkspaceEntity> {
|
||||
{ workspaceId: id },
|
||||
);
|
||||
|
||||
const emailingDomains = await this.coreDataSource
|
||||
.getRepository(EmailingDomainEntity)
|
||||
.find({ where: { workspaceId: id } });
|
||||
|
||||
await this.messageQueueService.add<EmailingDomainWorkspaceCleanupJobData>(
|
||||
EmailingDomainWorkspaceCleanupJob.name,
|
||||
{ workspaceId: id },
|
||||
{
|
||||
workspaceId: id,
|
||||
domains: emailingDomains.map(
|
||||
(emailingDomain) => emailingDomain.domain,
|
||||
),
|
||||
},
|
||||
);
|
||||
|
||||
if (workspace.customDomain) {
|
||||
|
||||
Reference in New Issue
Block a user