Commit Graph
214 Commits
Author SHA1 Message Date
Dries Augustyns 6ab4d77ca9 feat(SecurityService): enhance phishing detection by verifying sender domains and institutional TLDs 2026-05-25 18:16:07 +02:00
Dries Augustyns edfc399061 feat(SecurityService): enhance phishing detection by verifying sender domains and institutional TLDs 2026-05-25 18:04:52 +02:00
Dries AugustynsandGitHub 4a145f3488 Merge pull request #391 from andygrunwald/andygrunwald/csv-import-coerce-custom-field-types
fix: coerce boolean and numeric values in custom CSV columns
2026-05-25 07:30:50 +02:00
Andy Grunwald 868bdee615 import-processor: Reworked comments for coerceCustomValue 2026-05-24 16:15:57 +02:00
Andy GrunwaldandClaude Opus 4.7 81315eac8e docs(env): add wiki-documented vars to apps/api/.env.example
Brings the dev template into parity with the env vars wiki for variables a
developer might toggle locally:

- PORT (in Environment & Security)
- PLUNK_FROM_ADDRESS (in Plunk API)
- New sections: Notifications (NTFY_URL), Attachments, User Management
  (DISABLE_SIGNUPS, VERIFY_EMAIL_ON_SIGNUP), Phishing Detection

All additions are commented so default local behaviour is unchanged. Vars that
only make sense in the bundled Docker stack (SMTP relay ports, Minio internals,
NEXT_PUBLIC_*, NGINX_PORT) are intentionally omitted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:55:52 +02:00
Andy GrunwaldandClaude Opus 4.7 d59f4a10cd Treat "0" and "1" as numbers, not booleans, in custom CSV columns
The previous coercion treated "1"/"0" as booleans for parity with the
reserved `subscribed` column parser, but in custom columns those values
are far more often counts, quantities, or ids than true/false flags.
Coercing them to booleans hid the numeric segment-filter operators
(`gt`, `lt`) for fields that semantically are numbers, and miscategorised
them in `ContactService.getAvailableFields()` via `jsonb_typeof()`.

Drop "1"/"0" from the truthy/falsy keyword sets in `coerceCustomValue`.
With those entries gone the existing `NUMERIC_RE` branch picks both up
unchanged (the pattern already matches single-digit `0` and `1`), so
they land in `Contact.data` as JSON numbers. Boolean keyword coverage
remains for `true`/`false`/`yes`/`no` (case-insensitive, trimmed).

The reserved `subscribed` column parser at the top of the worker keeps
its own inline keyword set and continues to accept "1"/"0" — that
column is explicitly boolean by contract, so the asymmetry is
intentional.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:34 +02:00
Andy GrunwaldandClaude Opus 4.7 51ac88b9f5 Coerce custom CSV column values to number during import
Extends the helper introduced for boolean coercion so numeric values
(`42`, `3.14`) become JSON numbers in `Contact.data` instead of strings.
With the right JSON primitive in place, post-import inference via
`jsonb_typeof()` classifies the field as `number`, the dashboard's
segment-filter UI renders a numeric input, and comparison operators
(`gt`, `lt`) become available.

Use a strict integer-or-decimal regex rather than `Number()` to
preserve string-shaped numerics that users intend as identifiers:
leading-zero IDs (`"01234"`), zip codes, phone numbers, and signed or
scientific-notation forms (`"+42"`, `"1e10"`, `".5"`, `"42."`) are left
as strings. Boolean coercion still takes precedence, so `"1"` and `"0"`
remain booleans for consistency with how `subscribed` is parsed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:34 +02:00
Andy GrunwaldandClaude Opus 4.7 523bcad602 Coerce custom CSV column values to boolean during import
Custom CSV columns with values like "true"/"false" were stored as
strings, which made `ContactService.getAvailableFields()` report them
as string fields and lost the boolean toggle in the dashboard's
segment-filter UI. Only the reserved `subscribed` column was coerced.

Mirror the truthy keyword set already used for `subscribed`
(`true`/`1`/`yes`) and add its symmetric falsy counterpart
(`false`/`0`/`no`) for custom columns. Values outside that keyword set
are returned unchanged so names, IDs, or arbitrary strings are not
corrupted into `false`. Coercion runs over `Object.entries(customData)`
right before the `upsert()` call; no other code path needs to change
because `ContactService.upsert()` already accepts
`Record<string, unknown>` and `mergeContactData()` accepts mixed JSON
primitives. Post-import inference via `jsonb_typeof()` then classifies
the stored JSON boolean as type `boolean` automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:33 +02:00
Dries Augustyns 71e2277643 refactor(database): increase Prisma connection pool limits for improved test performance 2026-05-22 20:43:49 +02:00
Dries Augustyns 079e1879b3 test(SecurityService): update test case for complaint count thresholds to reflect new ceiling values 2026-05-22 20:20:45 +02:00
Dries Augustyns 4de40f40fa refactor(SecurityService): update absolute count ceilings for new projects to improve spam detection 2026-05-22 20:12:25 +02:00
Dries Augustyns 94ceadbbe4 feat: add disabledReason field to projects for better tracking of disable reasons 2026-05-22 13:03:32 +02:00
jaschaio ba3813e242 feat: make detectCustomHtmlPatterns aware of TipTap's actual capabilities
The previous detection tripped on ANY inline `style=` attribute and on
`<span>` elements specifically, which forced templates into HTML-only
editing mode whenever the user had used Visual mode features like text
color. TipTap's TextStyle + Color + Link extensions (configured in
EmailEditor.tsx) natively round-trip exactly that markup -- TipTap emits
`<span style="color: rgb(...)">…</span>` itself when you change a text
color, then the detection rejected it as "custom HTML" on the very next
load. Empirically about 21% of a 156-template corpus tripped this purely
on TipTap-export artifacts (`background-color: initial`, color spans).

This rewrite permits what TipTap can represent and rejects only what it
can't:

* Drop the broad inline-style check entirely (TextStyle/Color/Link
  preserve inline styles on spans and links).
* Remove `<span>` from the custom-elements list (TextStyle handles it).
* Expand the custom-elements list to explicitly cover everything TipTap
  has no extension for: `<div>`, `<section>`, `<article>`, `<header>`,
  `<footer>`, `<nav>`, `<aside>`, `<main>`, full table family
  (`<table>`, `<tr>`, `<td>`, `<th>`, `<tbody>`, `<thead>`, `<tfoot>`,
  `<colgroup>`, `<col>` -- no Table extension is loaded), form/embed/
  media/interactive (`<form>`, `<input>`, `<button>`, `<select>`,
  `<textarea>`, `<iframe>`, `<video>`, `<audio>`, `<svg>`, `<object>`,
  `<embed>`, `<details>`, `<summary>`, `<dialog>`). Single-table is now
  enough to opt out (previously needed nested tables -- harmless
  tightening, single `<table>` already isn't TipTap content).
* Tighten the custom-attributes regex with a leading `[\s"']` boundary so
  query strings like `<a href="…?id=…">` no longer false-match as an
  HTML `id=` attribute.
* `<style>` tags, `@media` queries, and the class allowlist (`prose`,
  `variable-`, `email-image`, `ProseMirror`, `resizable-image`,
  `selected`, `resize-handle`) are unchanged.

Mirrors the same logic in apps/api/src/services/EmailService.ts so the
server-side wrap decision in `EmailService.compile()` stays in lockstep
with the client-side editor-mode decision.

Side-effect on `wrapEmailWithStyles` / `EmailService.compile`: templates
that previously kept their own (unwrapped) shell because they contained
a colored `<span>` or an inline-styled `<a>` will now flow through the
prose wrapper. This is the correct behavior -- those templates ARE
visual-editor output and SHOULD get the same wrapper the preview modal
applies.

Tests: new vitest suite at apps/web/src/lib/__tests__/emailStyles.test.ts
covers 24 cases including the TipTap-export artifacts above, the
href-URL-with-id false-match, and the rejected-element set.
2026-05-17 15:57:37 +02:00
Dries AugustynsandGitHub d2496bc51d Merge pull request #368 from ReylanLugo/feat/domains-api-key-auth
feat(api): allow API key authentication for domain endpoints
2026-05-16 21:42:43 +02:00
taniasanz7andTania Sanz c484da88ab feat: render template variables in WEBHOOK step url, headers and body 2026-05-16 08:11:35 +02:00
Dries Augustyns a27d564e1a fix: implement mergeContactData method for efficient contact data updates 2026-05-13 19:19:11 +02:00
Dries AugustynsandGitHub 463301b5db Merge pull request #373 from taniasanz7/patch-18-configurable-mail-from-subdomain
feat: make MAIL FROM subdomain configurable via MAIL_FROM_SUBDOMAIN env
2026-05-12 22:00:21 +02:00
Dries Augustyns 6195b39f3d feat: add SwitchOffer component to promote switching from competitors for enhanced user engagement 2026-05-12 21:48:35 +02:00
Tania Sanz e0bf0f628a feat: make MAIL FROM subdomain configurable via MAIL_FROM_SUBDOMAIN env var 2026-05-12 20:35:53 +02:00
ReylanLugo 6aee5db588 refactor(api): rely on auth middleware for domain endpoint permissions
Replace per-route apiKey/jwt branching with auth.projectId from middleware,
matching the contacts controller pattern. Preserve JWT admin gating on
POST/DELETE; API keys are project-scoped by design and skip the role check.
Cross-project domain access by ID now returns 404 instead of 403 to avoid
leaking existence.
2026-05-10 22:44:28 -04:00
Dries Augustyns f22da4add1 feat: implement caching for recent activity count to optimize performance and reduce database load 2026-05-10 11:37:31 +02:00
Dries Augustyns de6335e999 feat: implement bulk contact action selector for improved flexibility in bulk operations 2026-05-10 10:40:40 +02:00
ReylanLugo 3f30a48c40 feat(api): allow API key authentication for domain endpoints
Switch /domains controller from `isAuthenticated` (cookie-only) to
`requireAuth` (cookie OR API key), matching the pattern used by other
project-scoped API endpoints (/v1/send, /contacts, etc.).

API keys are project-scoped credentials with full access; for write
operations the projectId in the request must equal the API key's
projectId. JWT (dashboard) auth retains role-based checks
(requireAdminAccess for POST/DELETE).

Also: improve UX when a domain is already linked to the same project
by returning a clear error instead of the generic "linked to another
project" message.

Refactor DomainService.checkDomainOwnership to make `userId` optional
(needed for API key path) while preserving its existing return shape
and adding `projectId` to the result.
2026-05-09 22:25:17 -04:00
Dries Augustyns c6340a1dc7 feat: add workflow duplication functionality with API endpoint and UI button 2026-05-09 16:51:36 +02:00
Dries Augustyns d11495061d feat: add search functionality to campaigns list with debounce effect 2026-05-09 09:10:17 +02:00
Dries Augustyns fadc19d139 feat: Ability to change subscription status in workflows 2026-05-09 08:43:05 +02:00
Dries Augustyns 2529c9b428 fix: enhance campaign finalization process to handle pending emails and ensure accurate status updates 2026-05-06 21:48:54 +02:00
Dries Augustyns 4ddafdc041 docs: expand documentation with new sections on importing contacts, unsubscribe pages, and API key management 2026-05-06 21:37:00 +02:00
Dries Augustyns 735acff454 feat: add sanitize-html for improved email content sanitization 2026-05-06 11:06:50 +02:00
Dries Augustyns 1193d6c7e6 log: remove detailed logging of full SES notification for inbound email 2026-05-06 10:46:47 +02:00
Dries Augustyns 62661ae45f log: improve logging for inbound email notifications and enhance email content parsing 2026-05-06 10:37:39 +02:00
Dries Augustyns 4587e9d670 fix: enhance email content parsing and logging for better debugging 2026-05-05 20:35:36 +02:00
Dries Augustyns a126563542 fix: update segment filter logic to retain value and unit, enhance activity name mapping 2026-05-04 19:35:29 +02:00
Dries Augustyns a3cc62213f feat: add segment membership operators and enhance segment filter functionality 2026-05-04 19:07:04 +02:00
Dries Augustyns 0f00ca1b8c feat: add 'notTriggeredWithin' operator to segment filters for enhanced event tracking 2026-05-04 18:04:24 +02:00
Dries Augustyns 48425d1df1 feat: implement early fraud warning handling in webhooks 2026-05-01 21:55:32 +02:00
Dries Augustyns bb72911fa2 feat: enhance contact addition with bulk email lookup and subscription options 2026-05-01 19:44:48 +02:00
Matthias Endler 11c428d4ff fix(ses): emit List-Unsubscribe inside the header section, not the body
The raw MIME template in sendRawEmail produced a blank line between Content-Type and List-Unsubscribe whenever no custom headers were passed, because the ternary for custom headers expanded to an empty string surrounded by newlines.

Per RFC 5322 §2.1 a blank line terminates the header section, so List-Unsubscribe ended up as the first line of the body. Lenient clients (Gmail, Outlook) recover; strict clients (Thunderbird) do not, breaking one-click unsubscribe and degrading Gmail/Yahoo bulk-sender deliverability signals.

Collect optional headers (custom + List-Unsubscribe) into an array, filter empties, and append them to the Content-Type line with a single newline separator. No codepath can now produce a blank line inside the header block.
2026-04-30 15:20:06 +02:00
Dries Augustyns 2eb407993b feat: add configurable attachment limits for email service
Closes #358
2026-04-29 08:48:03 +02:00
Dries Augustyns 10946511b0 feat: enhance contact data handling by filtering empty strings and allowing null to delete fields 2026-04-27 16:59:18 +02:00
Dries Augustyns d7eb85ffdd fix: update project disabled messages for clarity and consistency 2026-04-24 19:21:07 +02:00
Dries Augustyns c910d20bb7 feat: Enhance ease of use of workflow editor 2026-04-24 11:56:43 +02:00
Dries Augustyns 1637d54e1b fix: add project name and sender email parameters to phishing content check 2026-04-22 22:03:57 +02:00
Dries Augustyns 15294a40d8 fix: add project name and sender email parameters to phishing content check 2026-04-22 22:03:53 +02:00
Dries Augustyns bb78c86c40 fix: update phishing confidence threshold to 95% 2026-04-22 21:59:34 +02:00
Dries Augustyns 208c809a90 fix: Add in-memory cache for lower-confidence phishing checks 2026-04-22 07:40:23 +02:00
Dries Augustyns b7eb2549d4 fix: turn warning into success log 2026-04-21 22:08:07 +02:00
Dries Augustyns 91d0d2d297 fix: update response format for phishing analysis and improve JSON parsing comment 2026-04-21 21:53:16 +02:00
Dries Augustyns 862875428d fix: log message for projects passing phishing checks 2026-04-21 21:49:57 +02:00
Dries Augustyns 2da8b065d0 feat: implement phishing detection using OpenRouter API with configurable sampling rate 2026-04-21 21:28:31 +02:00