Compare commits

..
Author SHA1 Message Date
github-actions[bot]andGitHub ca06f56a6d chore(next): release 0.12.0 2026-05-27 15:54:15 +00:00
Dries Augustyns 80beb2bb99 feat(EmailService): add worker concurrency settings and improve email queue prioritization 2026-05-27 17:53:30 +02:00
Dries Augustyns 6ab4d77ca9 feat(SecurityService): enhance phishing detection by verifying sender domains and institutional TLDs 2026-05-25 18:16:07 +02:00
Dries Augustyns edfc399061 feat(SecurityService): enhance phishing detection by verifying sender domains and institutional TLDs 2026-05-25 18:04:52 +02:00
Dries AugustynsandGitHub 4a145f3488 Merge pull request #391 from andygrunwald/andygrunwald/csv-import-coerce-custom-field-types
fix: coerce boolean and numeric values in custom CSV columns
2026-05-25 07:30:50 +02:00
Andy Grunwald 868bdee615 import-processor: Reworked comments for coerceCustomValue 2026-05-24 16:15:57 +02:00
Dries AugustynsandGitHub 6ebbb50f68 Merge pull request #393 from andygrunwald/andygrunwald/sync-env-vars-and-add-process-rule
docs(env): sync env example files, fix CLAUDE.md drift, add process rule
2026-05-24 16:07:11 +02:00
Andy Grunwald 87eb56ff36 Revert "docs(env): sync .env.self-host.example with missing variables"
This reverts commit 1c1c95d332.
2026-05-24 16:05:25 +02:00
Andy GrunwaldandClaude Opus 4.7 a348d37c21 docs: add env-var sync rule to CLAUDE.md
Plunk has three sources of truth for environment variables:

1. apps/api/.env.example — local development defaults
2. .env.self-host.example — self-hosting/production template
3. apps/wiki/content/docs/self-hosting/environment-variables.mdx — user-facing reference

They have drifted repeatedly when new vars land in one location only. Codify
the requirement to update all three in the same change as part of CLAUDE.md so
future Claude-assisted (and human) contributions don't reintroduce the drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:56:23 +02:00
Andy GrunwaldandClaude Opus 4.7 5c166797b5 docs: correct PHISHING_CONFIDENCE_THRESHOLD default in CLAUDE.md
CLAUDE.md listed the default as 85, but the source of truth
(apps/api/src/app/constants.ts:137) and the env vars wiki both use 95. Aligning
CLAUDE.md so future contributors don't propagate the stale value into new code
or docs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:56:15 +02:00
Andy GrunwaldandClaude Opus 4.7 971b98a4cc docs(wiki): document MAIL_FROM_SUBDOMAIN and NGINX_PORT env vars
Both variables exist in .env.self-host.example but were missing from the
user-facing env vars reference, so self-hosters had to read the example file's
inline comments to discover them.

- MAIL_FROM_SUBDOMAIN: added a row to the AWS SES section explaining how the
  prefix combines with a verified domain to construct the MAIL FROM hostname,
  and when to override the default.
- NGINX_PORT: added a new "Advanced" section for variables that almost never
  need tuning, with the host-port override use-case spelled out.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:56:00 +02:00
Andy GrunwaldandClaude Opus 4.7 81315eac8e docs(env): add wiki-documented vars to apps/api/.env.example
Brings the dev template into parity with the env vars wiki for variables a
developer might toggle locally:

- PORT (in Environment & Security)
- PLUNK_FROM_ADDRESS (in Plunk API)
- New sections: Notifications (NTFY_URL), Attachments, User Management
  (DISABLE_SIGNUPS, VERIFY_EMAIL_ON_SIGNUP), Phishing Detection

All additions are commented so default local behaviour is unchanged. Vars that
only make sense in the bundled Docker stack (SMTP relay ports, Minio internals,
NEXT_PUBLIC_*, NGINX_PORT) are intentionally omitted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:55:52 +02:00
Andy GrunwaldandClaude Opus 4.7 1c1c95d332 docs(env): sync .env.self-host.example with missing variables
Brings the self-hosting template into parity with apps/api/.env.example and the
env vars wiki. Adds (mostly commented for visibility without changing default
behaviour):

- NODE_ENV, DATABASE_URL, DIRECT_DATABASE_URL, REDIS_URL, PORT
  (Docker auto-configures these via DB_PASSWORD; the lines document overrides)
- Stripe billing block (STRIPE_SK, STRIPE_WEBHOOK_SECRET, STRIPE_PRICE_*,
  STRIPE_METER_EVENT_NAME)
- Attachments (MAX_ATTACHMENT_SIZE_MB, MAX_ATTACHMENTS_COUNT)
- SMTP_ENABLED in the SMTP Server block
- VERIFY_EMAIL_ON_SIGNUP in User Management
- Phishing Detection block (OPENROUTER_*, PHISHING_*)

Each was either present in apps/api/.env.example or documented in the wiki but
missing from the production template, forcing self-hosters to read source or
the wiki to discover them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 12:55:41 +02:00
Andy GrunwaldandClaude Opus 4.7 844be42151 Document boolean and numeric CSV value typing
The preceding commits taught the import worker to coerce custom CSV
column values into JSON booleans and numbers via `coerceCustomValue`
in `apps/api/src/jobs/import-processor.ts`. Without a corresponding
docs update, users can't predict whether a cell like `01234` lands as
a string or as the number `1234`, or which segment-filter operators a
field will expose after import.

Add two bullets to the existing "Rules and limits" list in the
contact-import guide, adjacent to the **Date columns** bullet that
already documents value typing for ISO 8601 dates. The bullets mirror
that style and brevity: one names the boolean keyword set and the
toggle it unlocks in segment filters, the other names the numeric
pattern, the `gt`/`lt` operators it unlocks, and the deliberately
preserved-as-string forms (leading zeros, `+`-prefixed, scientific
notation) so users keep their IDs, zip codes, and phone numbers
intact.

No other content is touched. Closes the documentation gap for
useplunk/plunk#390.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 11:26:52 +02:00
Andy GrunwaldandClaude Opus 4.7 d59f4a10cd Treat "0" and "1" as numbers, not booleans, in custom CSV columns
The previous coercion treated "1"/"0" as booleans for parity with the
reserved `subscribed` column parser, but in custom columns those values
are far more often counts, quantities, or ids than true/false flags.
Coercing them to booleans hid the numeric segment-filter operators
(`gt`, `lt`) for fields that semantically are numbers, and miscategorised
them in `ContactService.getAvailableFields()` via `jsonb_typeof()`.

Drop "1"/"0" from the truthy/falsy keyword sets in `coerceCustomValue`.
With those entries gone the existing `NUMERIC_RE` branch picks both up
unchanged (the pattern already matches single-digit `0` and `1`), so
they land in `Contact.data` as JSON numbers. Boolean keyword coverage
remains for `true`/`false`/`yes`/`no` (case-insensitive, trimmed).

The reserved `subscribed` column parser at the top of the worker keeps
its own inline keyword set and continues to accept "1"/"0" — that
column is explicitly boolean by contract, so the asymmetry is
intentional.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:34 +02:00
Andy GrunwaldandClaude Opus 4.7 51ac88b9f5 Coerce custom CSV column values to number during import
Extends the helper introduced for boolean coercion so numeric values
(`42`, `3.14`) become JSON numbers in `Contact.data` instead of strings.
With the right JSON primitive in place, post-import inference via
`jsonb_typeof()` classifies the field as `number`, the dashboard's
segment-filter UI renders a numeric input, and comparison operators
(`gt`, `lt`) become available.

Use a strict integer-or-decimal regex rather than `Number()` to
preserve string-shaped numerics that users intend as identifiers:
leading-zero IDs (`"01234"`), zip codes, phone numbers, and signed or
scientific-notation forms (`"+42"`, `"1e10"`, `".5"`, `"42."`) are left
as strings. Boolean coercion still takes precedence, so `"1"` and `"0"`
remain booleans for consistency with how `subscribed` is parsed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:34 +02:00
Andy GrunwaldandClaude Opus 4.7 523bcad602 Coerce custom CSV column values to boolean during import
Custom CSV columns with values like "true"/"false" were stored as
strings, which made `ContactService.getAvailableFields()` report them
as string fields and lost the boolean toggle in the dashboard's
segment-filter UI. Only the reserved `subscribed` column was coerced.

Mirror the truthy keyword set already used for `subscribed`
(`true`/`1`/`yes`) and add its symmetric falsy counterpart
(`false`/`0`/`no`) for custom columns. Values outside that keyword set
are returned unchanged so names, IDs, or arbitrary strings are not
corrupted into `false`. Coercion runs over `Object.entries(customData)`
right before the `upsert()` call; no other code path needs to change
because `ContactService.upsert()` already accepts
`Record<string, unknown>` and `mergeContactData()` accepts mixed JSON
primitives. Post-import inference via `jsonb_typeof()` then classifies
the stored JSON boolean as type `boolean` automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 10:16:33 +02:00
17 changed files with 344 additions and 41 deletions
+13
View File
@@ -163,6 +163,19 @@ SMTP_DOMAIN=smtp.example.com
# Default: unset (auto-detect, falls back to 14)
# EMAIL_RATE_LIMIT_PER_SECOND=1
# Number of emails the worker processes in parallel. When unset, concurrency is
# derived from the effective rate limit (~ rate * 0.5, min 5, capped by
# EMAIL_WORKER_MAX_CONCURRENCY) so a higher SES quota translates into higher
# throughput automatically. Pin this only when the Prisma pool or memory is the
# binding constraint.
# Default: unset (auto-derived)
# EMAIL_WORKER_CONCURRENCY=10
# Upper bound applied to the auto-derived concurrency. Raise this when your SES
# quota is high AND the Prisma connection pool has been sized for it.
# Default: 50
# EMAIL_WORKER_MAX_CONCURRENCY=50
# ========================================
# ADVANCED (rarely needed)
# ========================================
+1 -1
View File
@@ -1,3 +1,3 @@
{
".": "0.11.0"
".": "0.12.0"
}
+42
View File
@@ -1,5 +1,47 @@
# Changelog
## [0.12.0](https://github.com/useplunk/plunk/compare/v0.11.0...v0.12.0) (2026-05-27)
### Features
* add disabledReason field to projects for better tracking of disable reasons ([94ceadb](https://github.com/useplunk/plunk/commit/94ceadbbe417f0cb3ab72c66bfad9428bdce2d11))
* **api:** allow API key authentication for domain endpoints ([d2496bc](https://github.com/useplunk/plunk/commit/d2496bc51d17657380160dbfc20e099d3f4eadba))
* **contacts:** make email cell a link to the contact detail page ([6759bff](https://github.com/useplunk/plunk/commit/6759bffd2a1f0bfb09b754bb0efdfe5c347ea2b3))
* **contacts:** make email cell a link to the contact detail page ([6d98d51](https://github.com/useplunk/plunk/commit/6d98d512222ffb422bc2a4bef4284a644558b39d))
* **EmailService:** add worker concurrency settings and improve email queue prioritization ([80beb2b](https://github.com/useplunk/plunk/commit/80beb2bb9937101d1723f2d54648060fdcbe6cef))
* make detectCustomHtmlPatterns aware of TipTap's actual capabilities ([9797aed](https://github.com/useplunk/plunk/commit/9797aed47f520aba51bec4e09c9bc4b7762e02f8))
* make detectCustomHtmlPatterns aware of TipTap's actual capabilities ([ba3813e](https://github.com/useplunk/plunk/commit/ba3813e2422d9fe4e252a90242393cada65644dc))
* render template variables in WEBHOOK step url, headers and body ([bfecf04](https://github.com/useplunk/plunk/commit/bfecf04fa38309804ee19e99b9908caf51c5e039))
* render template variables in WEBHOOK step url, headers and body ([c484da8](https://github.com/useplunk/plunk/commit/c484da88ab7b4635987868e2e3c6fdded68ade8c))
* **SecurityService:** enhance phishing detection by verifying sender domains and institutional TLDs ([6ab4d77](https://github.com/useplunk/plunk/commit/6ab4d77ca9bebd01f75d8799dad23fdec7768b52))
* **SecurityService:** enhance phishing detection by verifying sender domains and institutional TLDs ([edfc399](https://github.com/useplunk/plunk/commit/edfc399061cb7fd079b7f11167a2796df766d8ac))
* **tests:** enhance test database setup and cleanup for improved isolation and performance ([32dd7bb](https://github.com/useplunk/plunk/commit/32dd7bba462a6f4a13d59b8fb03708f971d1eff0))
### Bug Fixes
* coerce boolean and numeric values in custom CSV columns ([4a145f3](https://github.com/useplunk/plunk/commit/4a145f3488bee63c2d82a4a477f6dfaaaaede64b))
* **filters:** land templates/workflows/campaigns search inputs at 32px to match filter buttons ([283f402](https://github.com/useplunk/plunk/commit/283f40239dcc81e78d35b39cb9b31f97b435b8d7))
* make email templates, campaigns and workflow search inputs same height as the rest of the app ([8b3657d](https://github.com/useplunk/plunk/commit/8b3657d056938b17f0634c0fe8d664ae1e07cd73))
### Code Refactoring
* **database:** increase Prisma connection pool limits for improved test performance ([71e2277](https://github.com/useplunk/plunk/commit/71e227764319e6fc732aefdebd11f3e844257ef6))
* **SecurityService:** update absolute count ceilings for new projects to improve spam detection ([4de40f4](https://github.com/useplunk/plunk/commit/4de40f40fa300c9e9ed2ed35f367ca4cd601f309))
### Documentation
* add env-var sync rule to CLAUDE.md ([a348d37](https://github.com/useplunk/plunk/commit/a348d37c21ecf1d57845f31210f66a8113fadb8d))
* add new recipe pages for waitlist and sync unsubscribes ([01ec34a](https://github.com/useplunk/plunk/commit/01ec34a8cbbe0f4a158398c0eb8300d847bc29ab))
* correct PHISHING_CONFIDENCE_THRESHOLD default in CLAUDE.md ([5c16679](https://github.com/useplunk/plunk/commit/5c166797b57f2e7b714957824ee1acdcb22c05f2))
* **env:** add wiki-documented vars to apps/api/.env.example ([81315ea](https://github.com/useplunk/plunk/commit/81315eac8eb32a7d2e61617c6a84f349afaf7860))
* **env:** sync .env.self-host.example with missing variables ([1c1c95d](https://github.com/useplunk/plunk/commit/1c1c95d332fb9ab6c2f926f6d51071cfb245ed86))
* **env:** sync env example files, fix CLAUDE.md drift, add process rule ([6ebbb50](https://github.com/useplunk/plunk/commit/6ebbb50f6817f1354f26cc7734631d3b6ded32ed))
* **wiki:** document MAIL_FROM_SUBDOMAIN and NGINX_PORT env vars ([971b98a](https://github.com/useplunk/plunk/commit/971b98a4cc6145571204f5a4ad794aa3d09ea71c))
## [0.11.0](https://github.com/useplunk/plunk/compare/v0.10.0...v0.11.0) (2026-05-13)
+16 -1
View File
@@ -161,7 +161,7 @@ Required for builds and deployment (see turbo.json and .env.example):
- `OPENROUTER_API_KEY` - API key for OpenRouter (enables phishing detection)
- `OPENROUTER_MODEL` (default: anthropic/claude-3-haiku) - LLM model to use for content analysis
- `PHISHING_DETECTION_SAMPLE_RATE` (default: 0.1) - Percentage of emails to check (0.0-1.0, e.g., 0.1 = 10%)
- `PHISHING_CONFIDENCE_THRESHOLD` (default: 85) - Minimum confidence percentage (0-100) to auto-disable project for single detection
- `PHISHING_CONFIDENCE_THRESHOLD` (default: 95) - Minimum confidence percentage (0-100) to auto-disable project for single detection
- `PHISHING_CUMULATIVE_THRESHOLD` (default: 3) - Number of phishing detections within time window to trigger auto-disable
- `PHISHING_CUMULATIVE_WINDOW_MS` (default: 3600000) - Time window in milliseconds for cumulative tracking (default 1 hour)
@@ -174,6 +174,21 @@ Required for builds and deployment (see turbo.json and .env.example):
- **Frontend Variables**: Next.js apps use `NEXT_PUBLIC_*` prefixed variables that are embedded at build time for
client-side access
## Environment Variable Changes
When you add, rename, remove, or change the default/behaviour of any environment variable, you MUST update all THREE of the following in the same change:
1. `apps/api/.env.example` — local development defaults
2. `.env.self-host.example` — self-hosting / production template
3. `apps/wiki/content/docs/self-hosting/environment-variables.mdx` — user-facing reference
Rules:
- If the variable already exists in any file, **modify** its line/row/description — do not duplicate or leave a stale entry.
- Keep section/category names consistent across all three files (e.g. "AWS SES", "Phishing Detection").
- For dev-only or self-host-only variables, still mention them in the wiki and note the scope; only skip the example file where the variable is genuinely never applicable.
- When in doubt about whether a variable belongs in `apps/api/.env.example` (development), include it commented out with a short note.
## Plugins
There are two plugins installed for you to use.
+58
View File
@@ -8,6 +8,8 @@
# ==============================================================================
NODE_ENV=development
JWT_SECRET=hBx9Xh8J6KOMAGAsSjvcZJBT5TWyIkFX
# Port the API server listens on (default: 8080)
# PORT=8080
# ==============================================================================
# Application URLs
@@ -25,6 +27,8 @@ LANDING_URI=http://localhost:4000
# ==============================================================================
# API key for authenticating with the Plunk API (obtained from dashboard)
PLUNK_API_KEY=
# From address used for platform notification emails (project disabled, billing limits, etc.)
# PLUNK_FROM_ADDRESS=
# ==============================================================================
# Database & Redis
@@ -61,6 +65,17 @@ SES_CONFIGURATION_SET_NO_TRACKING=plunk-configuration-set-no-tracking # Optiona
# Default: Fetched from AWS (typically 14 for sandbox, higher for production accounts)
# EMAIL_RATE_LIMIT_PER_SECOND=14
# Email worker concurrency (number of emails processed in parallel)
# If not set, derived from the effective rate limit (~ rate * 0.5, min 5, capped
# by EMAIL_WORKER_MAX_CONCURRENCY). Set this to pin a fixed value when the
# Prisma connection pool or memory is the binding constraint.
# EMAIL_WORKER_CONCURRENCY=10
# Upper bound for auto-derived worker concurrency
# Raise this when your SES quota is high AND the Prisma pool has been sized for it.
# Default: 50
# EMAIL_WORKER_MAX_CONCURRENCY=50
# ==============================================================================
# OAuth (Optional - for social login)
# ==============================================================================
@@ -85,3 +100,46 @@ STRIPE_METER_EVENT_NAME=emails # Meter event name (API key from your Stripe mete
# Set to 'false' to disable automatic project suspension (useful for self-hosters who manage manually)
# Default: true (automatic project disabling enabled)
# AUTO_PROJECT_DISABLE=true
# ==============================================================================
# Notifications (Optional - system notifications via ntfy)
# ==============================================================================
# ntfy topic URL for internal system notifications (e.g. project disabled, billing limits).
# When unset, ntfy notifications are disabled.
# Examples:
# - Public ntfy.sh: https://ntfy.sh/your-unique-topic-name
# - Self-hosted: https://your-ntfy-server.com/your-topic
# NTFY_URL=
# ==============================================================================
# Attachments (Optional)
# ==============================================================================
# Limits applied to attachments on transactional emails.
# AWS SES caps total message size at 40 MB; the defaults below leave headroom.
# MAX_ATTACHMENT_SIZE_MB=10
# MAX_ATTACHMENTS_COUNT=10
# ==============================================================================
# User Management (Optional)
# ==============================================================================
# When 'true', the signup endpoint rejects new user registrations.
# Default: false
# DISABLE_SIGNUPS=false
# When 'true', validates emails on signup (disposable domains, plus-addressing,
# domain existence, MX records).
# Default: false
# VERIFY_EMAIL_ON_SIGNUP=false
# ==============================================================================
# Phishing Detection (Optional - AI-powered phishing scan via OpenRouter)
# ==============================================================================
# When OPENROUTER_API_KEY is set, a random sample of outgoing emails is
# analyzed by an LLM. Projects can be auto-disabled if a single email exceeds
# PHISHING_CONFIDENCE_THRESHOLD, or if PHISHING_CUMULATIVE_THRESHOLD emails are
# flagged within PHISHING_CUMULATIVE_WINDOW_MS.
# OPENROUTER_API_KEY=
# OPENROUTER_MODEL=anthropic/claude-3-haiku
# PHISHING_DETECTION_SAMPLE_RATE=0.1
# PHISHING_CONFIDENCE_THRESHOLD=95
# PHISHING_CUMULATIVE_THRESHOLD=3
# PHISHING_CUMULATIVE_WINDOW_MS=3600000
+1 -1
View File
@@ -39,7 +39,7 @@
"mailparser": "^3.9.8",
"morgan": "^1.10.0",
"multer": "^2.1.1",
"sanitize-html": "^2.17.4",
"sanitize-html": "^2.17.3",
"signale": "^1.4.0",
"stripe": "^20.0.0"
},
+14
View File
@@ -58,6 +58,20 @@ export const EMAIL_RATE_LIMIT_PER_SECOND = process.env.EMAIL_RATE_LIMIT_PER_SECO
? Number(process.env.EMAIL_RATE_LIMIT_PER_SECOND)
: undefined;
// Email Worker Concurrency (optional override)
// If not set, concurrency is derived from the effective rate limit so a higher
// SES quota actually translates into higher throughput. Set this to pin a fixed
// value (useful when Prisma pool size or memory is the binding constraint).
export const EMAIL_WORKER_CONCURRENCY = process.env.EMAIL_WORKER_CONCURRENCY
? Number(process.env.EMAIL_WORKER_CONCURRENCY)
: undefined;
// Upper bound for auto-derived concurrency. Raise this if you have a large SES
// quota AND have sized the Prisma connection pool accordingly.
export const EMAIL_WORKER_MAX_CONCURRENCY = process.env.EMAIL_WORKER_MAX_CONCURRENCY
? Number(process.env.EMAIL_WORKER_MAX_CONCURRENCY)
: 50;
// Storage
export const REDIS_URL = validateEnv('REDIS_URL');
export const DATABASE_URL = validateEnv('DATABASE_URL');
@@ -1,6 +1,7 @@
import {beforeEach, describe, expect, it} from 'vitest';
import {factories, getPrismaClient} from '../../../../../test/helpers';
import {ContactService} from '../../services/ContactService.js';
import {coerceCustomValue} from '../import-processor.js';
/**
* Tests for Contact Import Processor - Subscription Status Preservation
@@ -279,3 +280,51 @@ describe('Contact Import - Subscription Status Preservation', () => {
});
});
});
describe('coerceCustomValue', () => {
describe('boolean coercion', () => {
it.each(['true', 'TRUE', 'True', ' true ', 'yes', 'YES', 'Yes'])('coerces %j to true', value => {
expect(coerceCustomValue(value)).toBe(true);
});
it.each(['false', 'FALSE', 'False', ' false ', 'no', 'NO', 'No'])('coerces %j to false', value => {
expect(coerceCustomValue(value)).toBe(false);
});
});
describe('number coercion', () => {
it.each([
['42', 42],
['-7', -7],
['3.14', 3.14],
[' 42 ', 42],
['0.5', 0.5],
['-0.25', -0.25],
['0', 0],
['1', 1],
])('coerces %j to %j', (value, expected) => {
expect(coerceCustomValue(value)).toBe(expected);
});
it.each(['01234', '+42', '.5', '42.', '1e10', 'NaN', 'Infinity', '1.2.3', '4-2'])(
'leaves %j as a string (preserves IDs / rejects loose formats)',
value => {
expect(coerceCustomValue(value)).toBe(value);
},
);
it('"1.0" is a number (does not match the boolean truthy set)', () => {
expect(coerceCustomValue('1.0')).toBe(1);
});
});
describe('passthrough', () => {
it.each(['Alice', 'true!', 'yesno', 'maybe'])('leaves %j as a string', value => {
expect(coerceCustomValue(value)).toBe(value);
});
it('leaves empty string as empty string', () => {
expect(coerceCustomValue('')).toBe('');
});
});
});
+29 -2
View File
@@ -8,7 +8,12 @@ import type {SendEmailJobData} from '@plunk/types';
import {type Job, Worker} from 'bullmq';
import signale from 'signale';
import {DASHBOARD_URI, EMAIL_RATE_LIMIT_PER_SECOND} from '../app/constants.js';
import {
DASHBOARD_URI,
EMAIL_RATE_LIMIT_PER_SECOND,
EMAIL_WORKER_CONCURRENCY,
EMAIL_WORKER_MAX_CONCURRENCY,
} from '../app/constants.js';
import {prisma} from '../database/prisma.js';
import {CampaignService} from '../services/CampaignService.js';
import {EmailService} from '../services/EmailService.js';
@@ -47,9 +52,31 @@ async function getEmailRateLimit(): Promise<number> {
return DEFAULT_RATE_LIMIT;
}
/**
* Derive worker concurrency from the rate limit so a higher SES quota actually
* translates into higher throughput. The mean job duration is ~0.5s (Prisma
* reads + HTML compile + SES call + writes), so `rate * 0.5` gives ~2× headroom
* over the per-second cap. Clamped to keep sandbox accounts useful and to
* protect the Prisma pool on very large quotas.
*/
function deriveWorkerConcurrency(rateLimit: number): number {
if (EMAIL_WORKER_CONCURRENCY !== undefined) {
return EMAIL_WORKER_CONCURRENCY;
}
const TARGET_JOB_SECONDS = 0.5;
const MIN_CONCURRENCY = 5;
const derived = Math.ceil(rateLimit * TARGET_JOB_SECONDS);
return Math.max(MIN_CONCURRENCY, Math.min(derived, EMAIL_WORKER_MAX_CONCURRENCY));
}
export async function createEmailWorker() {
// Fetch the rate limit (from env, AWS, or default)
const rateLimit = await getEmailRateLimit();
const concurrency = deriveWorkerConcurrency(rateLimit);
signale.info(
`[EMAIL-PROCESSOR] Worker concurrency: ${concurrency} (rate limit: ${rateLimit}/s)`,
);
const worker = new Worker<SendEmailJobData>(
emailQueue.name,
async (job: Job<SendEmailJobData>) => {
@@ -253,7 +280,7 @@ export async function createEmailWorker() {
},
{
connection: emailQueue.opts.connection,
concurrency: 10, // Process up to 10 emails concurrently
concurrency,
limiter: {
max: rateLimit, // Max emails per second (from env, AWS SES quota, or default)
duration: 1000,
+32 -1
View File
@@ -123,7 +123,11 @@ export function createImportWorker() {
// Extract custom data (all fields except email and subscribed)
const {email: _, subscribed: __, ...customData} = record;
const data = Object.keys(customData).length > 0 ? customData : undefined;
const customEntries = Object.entries(customData);
const data =
customEntries.length > 0
? Object.fromEntries(customEntries.map(([k, v]) => [k, coerceCustomValue(v)]))
: undefined;
// Check if contact exists before upserting
const existingContact = await ContactService.findByEmail(projectId, email);
@@ -216,3 +220,30 @@ function isValidEmail(email: string): boolean {
const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
return emailRegex.test(email);
}
// Values considered as boolean during import.
// Numbers (0, 1) are intentionally absent.
const BOOLEAN_TRUE = new Set(['true', 'yes']);
const BOOLEAN_FALSE = new Set(['false', 'no']);
// Strict integer-or-decimal number detection pattern.
// Valid: 0, 42, -42, 3.14
// Rejected: 007, +42, 1.2.3, 1e5
const NUMERIC_RE = /^-?(0|[1-9]\d*)(\.\d+)?$/;
/**
* Coerces a raw string into its most natural primitive type: `boolean`,
* `number`, or `string`. Values that match neither are
* returned unchanged.
*
* @param value The raw string to coerce.
* @returns The coerced value as `boolean`, `number`, or `string`.
*/
export function coerceCustomValue(value: string): string | boolean | number {
const trimmed = value.trim();
const lower = trimmed.toLowerCase();
if (BOOLEAN_TRUE.has(lower)) return true;
if (BOOLEAN_FALSE.has(lower)) return false;
if (NUMERIC_RE.test(trimmed)) return Number(trimmed);
return value;
}
+5 -5
View File
@@ -108,7 +108,7 @@ export class EmailService {
await BillingLimitService.incrementUsage(params.projectId, EmailSourceType.TRANSACTIONAL);
// Queue email for sending
await this.queueEmail(email.id);
await this.queueEmail(email.id, EmailSourceType.TRANSACTIONAL);
return email;
}
@@ -172,7 +172,7 @@ export class EmailService {
await BillingLimitService.incrementUsage(params.projectId, sourceType);
// Queue email for sending
await this.queueEmail(email.id);
await this.queueEmail(email.id, sourceType);
return email;
}
@@ -278,7 +278,7 @@ export class EmailService {
await BillingLimitService.incrementUsage(params.projectId, sourceType);
// Queue email for sending
await this.queueEmail(email.id);
await this.queueEmail(email.id, sourceType);
return email;
}
@@ -1137,7 +1137,7 @@ export class EmailService {
* Queue an email for sending
* Adds email to the BullMQ queue for processing by workers
*/
private static async queueEmail(emailId: string, delay?: number): Promise<void> {
await QueueService.queueEmail(emailId, delay);
private static async queueEmail(emailId: string, sourceType: EmailSourceType, delay?: number): Promise<void> {
await QueueService.queueEmail(emailId, sourceType, delay);
}
}
+28 -5
View File
@@ -1,4 +1,4 @@
import {CampaignStatus, EmailStatus} from '@plunk/db';
import {CampaignStatus, EmailSourceType, EmailStatus} from '@plunk/db';
import {type Job, Queue} from 'bullmq';
import type {RedisOptions} from 'ioredis';
import signale from 'signale';
@@ -174,20 +174,43 @@ export const meterQueue = new Queue<MeterEventJobData>('meter', {
},
});
function emailPriorityFor(sourceType: EmailSourceType): number {
switch (sourceType) {
case EmailSourceType.TRANSACTIONAL:
return 1;
case EmailSourceType.WORKFLOW:
return 5;
case EmailSourceType.CAMPAIGN:
return 10;
default:
return 5;
}
}
/**
* Queue Service - Centralized queue management
*/
export class QueueService {
/**
* Add email to queue for sending
* Add email to queue for sending.
*
* Transactional emails jump the queue ahead of workflow and campaign sends
* via BullMQ's priority (lower number = higher precedence). This prevents
* latency-sensitive sends (login codes, password resets) from queuing behind
* large campaign bursts on the shared `email` queue.
*/
public static async queueEmail(emailId: string, delay?: number): Promise<Job<SendEmailJobData>> {
public static async queueEmail(
emailId: string,
sourceType: EmailSourceType,
delay?: number,
): Promise<Job<SendEmailJobData>> {
return emailQueue.add(
'send-email',
{emailId},
{
delay, // Optional delay in milliseconds
jobId: `email-${emailId}`, // Prevent duplicate jobs
delay,
jobId: `email-${emailId}`,
priority: emailPriorityFor(sourceType),
},
);
}
+37 -2
View File
@@ -765,7 +765,36 @@ export class SecurityService {
const uniqueUrls = [...new Set(urlMatches.map(u => u.replace(/[.,;)]+$/, '')))].slice(0, 20);
// Extract sender domain for context
const senderDomain = fromEmail.includes('@') ? fromEmail.split('@')[1] : fromEmail;
const senderDomain = (fromEmail.split('@')[1] ?? fromEmail).toLowerCase();
// Check whether this domain is verified by the project. A verified
// domain means the sender proved DNS/DKIM control — strong evidence of
// legitimacy, especially for institutional TLDs like .gov, .edu, .mil.
const verifiedDomain = await prisma.domain.findFirst({
where: {projectId, domain: senderDomain, verified: true},
select: {domain: true},
});
const isDomainVerified = verifiedDomain !== null;
// Institutional TLDs that imply a vetted, real-world entity behind the
// domain (government, military, accredited education). When combined
// with DKIM verification these effectively cannot be phishing senders.
const institutionalTldPattern =
/\.(gov|mil|edu)(\.[a-z]{2,})?$|\.gc\.ca$|\.gouv\.fr$|\.gov\.uk$|\.ac\.[a-z]{2,}$/i;
const isInstitutionalDomain = institutionalTldPattern.test(senderDomain);
// Skip the LLM check entirely when the sender is a verified institutional
// domain (e.g. a .gov customer). These TLDs are gated by registries that
// verify the real-world entity, and DKIM verification proves the project
// controls the domain — together they make phishing effectively
// impossible from this sender. Avoids paying for an LLM call that
// sometimes false-positives on official government communications.
if (isDomainVerified && isInstitutionalDomain) {
signale.info(
`[PHISHING] Skipping check for project ${projectId} — verified institutional domain (${senderDomain})`,
);
return safeResponse;
}
// Call OpenRouter API
const response = await fetch('https://openrouter.ai/api/v1/chat/completions', {
@@ -798,7 +827,11 @@ Criteria for phishing/dangerous content:
- Requests for sensitive personal information
IMPORTANT - Use sender and project context when evaluating:
- The sender project name and domain are provided. Links to the sender's own domain(s) are expected and NOT suspicious.
- The sender project name and domain are provided, along with whether the domain has been verified (DKIM/DNS) by this project.
- A VERIFIED sender domain means the sender proved ownership of the domain via DNS records. This is strong evidence of legitimacy.
- If the verified sender domain is an institutional domain (e.g. .gov, .gov.uk, .gouv.fr, .gc.ca, .mil, .edu, .ac.*), treat the email as legitimate institutional communication. Government, military, and accredited education domains cannot be obtained by phishers — do NOT flag these as impersonation of government/banks/etc. just because the content mentions official topics, taxes, benefits, court notices, etc.
- Impersonation rules only apply when the sender domain does NOT match the brand being referenced. A verified bank domain sending a banking email is not impersonating itself.
- Links to the sender's own domain(s) are expected and NOT suspicious.
- URLs that match or are clearly related to the project name or sender domain add credibility.
- Only flag a URL as suspicious if it is unrelated to or impersonates a different known brand.
- Lack of recognizable brand does NOT make an email phishing — many legitimate businesses are not famous.
@@ -811,6 +844,8 @@ Set confidence to 100 only if you are absolutely certain it's phishing.`,
role: 'user',
content: `Sender project name: ${projectName}
Sender domain: ${senderDomain}
Sender domain verified (DKIM/DNS confirmed by project): ${isDomainVerified ? 'yes' : 'no'}
Sender domain is an institutional TLD (gov/mil/edu/ac/etc.): ${isInstitutionalDomain ? 'yes' : 'no'}
${uniqueUrls.length > 0 ? `URLs found in email: ${uniqueUrls.join(', ')}` : ''}
Subject: ${subject}
@@ -37,6 +37,8 @@ In this example, every imported contact ends up with `data.firstName`, `data.pla
- **Email column**: must be present and valid. Rows with missing or invalid emails are reported back as errors.
- **Reserved column names**: `id`, `subscribed`, `createdAt`, `updatedAt`, and the auto-generated URL variables (`unsubscribeUrl`, etc.) are silently filtered out. Don't include them as columns.
- **Date columns**: use ISO 8601 (`2026-05-06T12:00:00Z`) so they're typed as dates and become usable with `within` / `olderThan` segment operators.
- **Boolean columns**: `true`, `false`, `yes`, `no` (case-insensitive) are stored as booleans and get the boolean toggle in segment filters.
- **Numeric columns**: plain integers and decimals (`42`, `3.14`) are stored as numbers and become usable with `gt` / `lt` segment operators. Leading-zero values (`01234`), `+`-prefixed numbers, and scientific notation stay strings so IDs, zip codes, and phone numbers aren't corrupted.
- **Existing contacts**: if a row's email matches an existing contact, the import **updates** the contact (merging the CSV's columns into `data`). It doesn't create a duplicate or overwrite the whole record.
## Importing your CSV
@@ -36,6 +36,7 @@ Set your subdomains here. The application automatically derives all internal and
| `AWS_SES_SECRET_ACCESS_KEY` | Yes | AWS secret access key for SES. | `wJalr...` |
| `SES_CONFIGURATION_SET` | No | SES configuration set name used for open/click tracking. | `plunk-configuration-set` (default) |
| `SES_CONFIGURATION_SET_NO_TRACKING` | No | A second SES configuration set without tracking. When set, projects can toggle email tracking on/off. If omitted, the tracking toggle is hidden. | `plunk-no-tracking-configuration-set` (default) |
| `MAIL_FROM_SUBDOMAIN` | No | Subdomain prefix used when constructing the MAIL FROM hostname for a verified domain (e.g. with default `plunk` and domain `yourdomain.com`, the MAIL FROM is `plunk.yourdomain.com`). Override when the default subdomain is already in use (e.g. by an R2/CDN custom domain), since the MAIL FROM hostname needs MX + TXT records that can't coexist with a CNAME. | `plunk` |
## Storage (Minio)
@@ -129,6 +130,16 @@ Plunk bundles a self-hosted [ntfy](https://ntfy.sh) server for internal system n
| ----------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- |
| `AUTO_PROJECT_DISABLE` | No | When `true`, projects are automatically suspended when bounce or complaint rate thresholds are exceeded. Set to `false` to manage project status manually. | `true` |
| `EMAIL_RATE_LIMIT_PER_SECOND` | No | Override the email sending rate limit. If not set, Plunk automatically fetches the quota from your AWS SES account. | — |
| `EMAIL_WORKER_CONCURRENCY` | No | Number of emails the worker processes in parallel. When unset, derived from the effective rate limit so a higher SES quota scales throughput automatically. | — |
| `EMAIL_WORKER_MAX_CONCURRENCY`| No | Upper bound applied to the auto-derived worker concurrency. Raise this only after sizing the Prisma connection pool accordingly. | `50` |
## Advanced
Variables for unusual deployments. The defaults work for the standard Docker Compose setup — only change these if you know you need to.
| Variable | Required | Description | Default |
| ------------ | -------- | ------------------------------------------------------------------------------------------------------------------------ | ------- |
| `NGINX_PORT` | No | Host port the bundled Nginx reverse proxy binds to. Override when port 80/443 is already in use on the host (e.g. running behind another reverse proxy that forwards to a different port). | `80` |
## Phishing Detection
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "plunk",
"version": "0.11.0",
"version": "0.12.0",
"private": true,
"scripts": {
"build": "turbo build",
+5 -22
View File
@@ -8101,7 +8101,7 @@ __metadata:
mailparser: "npm:^3.9.8"
morgan: "npm:^1.10.0"
multer: "npm:^2.1.1"
sanitize-html: "npm:^2.17.4"
sanitize-html: "npm:^2.17.3"
signale: "npm:^1.4.0"
stripe: "npm:^20.0.0"
tsx: "npm:^4.20.6"
@@ -9522,13 +9522,6 @@ __metadata:
languageName: node
linkType: hard
"dayjs@npm:^1.11.7":
version: 1.11.20
resolution: "dayjs@npm:1.11.20"
checksum: 10c0/8af525e2aa100c8db9923d706c42b2b2d30579faf89456619413a5c10916efc92c2b166e193c27c02eb3174b30aa440ee1e7b72b0a2876b3da651d204db848a0
languageName: node
linkType: hard
"debounce-fn@npm:^6.0.0":
version: 6.0.0
resolution: "debounce-fn@npm:6.0.0"
@@ -13298,15 +13291,6 @@ __metadata:
languageName: node
linkType: hard
"launder@npm:^1.7.1":
version: 1.7.1
resolution: "launder@npm:1.7.1"
dependencies:
dayjs: "npm:^1.11.7"
checksum: 10c0/c4884c08cc5a1a19cbec840aac7fa97db4928c25fc99ea2981a0482df3ebdbf1cf6605226a3c968e3281025126ff10055686e81f428ecc0e8f8666ca05bae8cc
languageName: node
linkType: hard
"leac@npm:^0.6.0":
version: 0.6.0
resolution: "leac@npm:0.6.0"
@@ -17231,18 +17215,17 @@ __metadata:
languageName: node
linkType: hard
"sanitize-html@npm:^2.17.4":
version: 2.17.4
resolution: "sanitize-html@npm:2.17.4"
"sanitize-html@npm:^2.17.3":
version: 2.17.3
resolution: "sanitize-html@npm:2.17.3"
dependencies:
deepmerge: "npm:^4.2.2"
escape-string-regexp: "npm:^4.0.0"
htmlparser2: "npm:^10.1.0"
is-plain-object: "npm:^5.0.0"
launder: "npm:^1.7.1"
parse-srcset: "npm:^1.0.2"
postcss: "npm:^8.3.11"
checksum: 10c0/5c352376a44bf8a70644f6d4421684000a982f6bda59beac051693d8fc08acbe48dc6358f5c8eb8ae4a815746260167926747a858e6a6e2daf01ccfb775100dd
checksum: 10c0/8afa59bed125b38bf4b437f9b5a3289a4307f42d720e45105de5a0b3d665be70e27d1722d223121993be2e54a2b99304cd9c54317fb2d251fd7f4abf06b68d27
languageName: node
linkType: hard