Initial push of Plunk Next

This commit is contained in:
Dries Augustyns
2025-12-01 09:56:56 +01:00
parent 07cea20262
commit ff1876d580
566 changed files with 89036 additions and 28423 deletions
+289
View File
@@ -0,0 +1,289 @@
# Plunk SMTP Relay Server
A production-ready SMTP relay server that accepts emails via SMTP protocol and forwards them to the Plunk API's `/v1/send` endpoint.
## Features
- **Secure Authentication**: API key-based authentication using project secrets
- **Domain Verification**: Validates sender domains are verified before accepting emails
- **TLS Support**: Supports both implicit TLS (port 465) and STARTTLS (port 587)
- **Flexible Certificate Handling**: Works with Traefik's acme.json or standard PEM files
- **Email Parsing**: Full email parsing with support for HTML and plain text
- **Rate Limiting**: Configurable recipient limits per email
- **Production Ready**: Built with TypeScript, error handling, and logging
## Architecture
```
SMTP Client → SMTP Server → Email Parser → API /v1/send → AWS SES
```
The SMTP server acts as a relay:
1. Accepts SMTP connections with authentication
2. Validates sender domains against the database
3. Parses incoming emails
4. Forwards to the Plunk API
5. Returns success/error to the SMTP client
## Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `API_URI` | `http://localhost:3000` | Plunk API base URL |
| `SMTP_DOMAIN` | *(empty)* | SMTP domain - required when using Traefik acme.json with multiple certificates |
| `PORT_SECURE` | `465` | SMTPS port (implicit TLS) |
| `PORT_SUBMISSION` | `587` | SMTP submission port (STARTTLS) |
| `MAX_RECIPIENTS` | `5` | Maximum recipients per email |
| `CERT_PATH` | `/certs` | Path to certificate files |
| `ACME_JSON_PATH` | `/certs/acme.json` | Path to Traefik acme.json file |
See `.env.self-host.example` in the repository root for full configuration options.
## Development
### Prerequisites
- Node.js 20+
- PostgreSQL database (running via `yarn services:up`)
- Redis (running via `yarn services:up`)
### Running Locally
```bash
# Install dependencies
yarn install
# Start development server (with hot reload)
yarn workspace smtp dev
```
The SMTP server will start on ports 465 and 587. For local testing without TLS certificates, it will run in plaintext mode on port 587.
### Testing with Telnet
```bash
# Connect to SMTP server
telnet localhost 587
# Example SMTP session:
EHLO localhost
AUTH LOGIN
cGx1bms= # base64("plunk")
<your-api-key> # your project secret (plain text)
MAIL FROM:<[email protected]>
RCPT TO:<[email protected]>
DATA
Subject: Test Email
From: [email protected]
To: [email protected]
This is a test email.
.
QUIT
```
### Testing with Mail Clients
Configure your email client with:
- **SMTP Server**: `localhost` (or your domain in production)
- **Port**: 587 (STARTTLS) or 465 (SSL/TLS)
- **Username**: `plunk`
- **Password**: Your Plunk API key (project secret)
- **From Address**: Must use a verified domain in your Plunk project
## Production Deployment
### TLS Certificate Support
The SMTP server supports TLS certificates through two methods:
1. **Traefik acme.json** (recommended for Traefik/Dokploy users)
- Mount your Traefik acme.json file to `/certs/acme.json`
- Set `SMTP_DOMAIN` environment variable to select the correct certificate
- The server will automatically use the certificate for your domain
2. **PEM Files** (standard certificate files)
- Mount `privkey.pem` and `fullchain.pem` to `/certs/`
- These are standard Let's Encrypt/Certbot filenames
- `SMTP_DOMAIN` is optional when using PEM files
If no certificates are mounted, the server will run without TLS (not recommended for production).
### Docker Deployment
The SMTP server is included in the main Plunk Docker image:
**Option 1: With Traefik acme.json**
```bash
docker run -d \
-p 465:465 \
-p 587:587 \
-e SERVICE=all \
-e API_URI=https://api.yourdomain.com \
-e SMTP_DOMAIN=smtp.yourdomain.com \
-e DATABASE_URL=postgresql://... \
-e REDIS_URL=redis://... \
-v /path/to/acme.json:/certs/acme.json:ro \
plunk:latest
```
**Option 2: With PEM files**
```bash
docker run -d \
-p 465:465 \
-p 587:587 \
-e SERVICE=all \
-e API_URI=https://api.yourdomain.com \
-e DATABASE_URL=postgresql://... \
-e REDIS_URL=redis://... \
-v /etc/letsencrypt/live/smtp.yourdomain.com/privkey.pem:/certs/privkey.pem:ro \
-v /etc/letsencrypt/live/smtp.yourdomain.com/fullchain.pem:/certs/fullchain.pem:ro \
plunk:latest
```
**Option 3: Without TLS (Development Only)**
```bash
docker run -d \
-p 587:587 \
-e SERVICE=all \
-e API_URI=http://api.yourdomain.com \
-e DATABASE_URL=postgresql://... \
-e REDIS_URL=redis://... \
plunk:latest
```
**⚠️ Running without TLS is not recommended for production!**
### DNS Configuration
Add an A record pointing to your server:
```
smtp.example.com. A 1.2.3.4
```
Optionally, add MX records if receiving email:
```
example.com. MX 10 smtp.example.com.
```
## Security
### Authentication
- Username must be `plunk` (case-insensitive)
- Password is the project secret (API key)
- Invalid credentials are rejected with proper SMTP error codes
### Domain Validation
- Sender domain must be added to the project
- Sender domain must be verified (DNS records validated)
- Unverified domains are rejected with clear error messages
### Rate Limiting
- Maximum 5 recipients per email by default (configurable)
- 10MB maximum message size
- Proper error handling for oversized messages
## Monitoring
The SMTP server logs all operations using `signale`:
```
✅ SMTP server listening on port 465 (secure=true)
✅ SMTP server listening on port 587 (secure=false)
✅ Email relayed: [email protected][email protected]
❌ Sender domain is not verified or not associated with your account
```
Use PM2 or similar process managers to monitor the service in production.
## Troubleshooting
### TLS Certificate Issues
If TLS is not working:
1. Verify certificates are mounted correctly:
```bash
docker exec <container> ls -la /certs/
```
2. Check certificate permissions (should be readable)
3. Review logs for certificate loading errors:
```bash
docker logs <container> 2>&1 | grep -i "cert"
```
4. If using Traefik acme.json, ensure `SMTP_DOMAIN` is set correctly
5. If using PEM files, ensure both `privkey.pem` and `fullchain.pem` are present
### Connection Refused
If clients cannot connect:
1. Verify ports 465 and 587 are exposed and not blocked by firewall
2. Check if the service is running: `pm2 list`
3. Review logs: `pm2 logs smtp`
### Authentication Failures
If authentication fails:
1. Verify username is exactly `plunk`
2. Verify password is the project secret, not public key
3. Check database connectivity
### Domain Verification Errors
If emails are rejected with domain errors:
1. Verify domain is added to your project
2. Check domain verification status in Plunk dashboard
3. Ensure DNS records are properly configured
## Performance Considerations
The SMTP server is designed for high-scale email sending:
- **Async Processing**: All database and API calls are non-blocking
- **Connection Pooling**: Prisma handles database connection pooling
- **Minimal Memory**: Streams email data without buffering entire messages
- **Fast Authentication**: Single database query per connection
For high-volume sending:
- Deploy multiple SMTP server instances behind a load balancer
- Use connection pooling in your SMTP clients
- Monitor API rate limits and adjust accordingly
## API Integration
The SMTP server forwards emails to the API endpoint:
```
POST /v1/send
Authorization: Bearer {project_secret}
{
"from": "[email protected]",
"name": "Sender Name",
"to": ["[email protected]"],
"subject": "Email Subject",
"body": "<html>...</html>"
}
```
The API response is translated to SMTP status codes:
- `200 OK` → `250 Message accepted`
- `4xx/5xx` → `554 Transaction failed`
## Contributing
When modifying the SMTP server:
1. Follow the existing code patterns
2. Add proper error handling
3. Update TypeScript types
4. Test with real SMTP clients
5. Update this README if adding features
## License
Part of the Plunk project - see repository root for license information.
+22
View File
@@ -0,0 +1,22 @@
import js from '@eslint/js';
import tseslint from 'typescript-eslint';
export default tseslint.config(
js.configs.recommended,
...tseslint.configs.recommended,
{
ignores: ['node_modules/**', 'dist/**', '.turbo/**'],
},
{
rules: {
'@typescript-eslint/no-unused-vars': [
'warn',
{
argsIgnorePattern: '^_',
varsIgnorePattern: '^_',
},
],
'@typescript-eslint/no-explicit-any': 'warn',
},
},
);
+27
View File
@@ -0,0 +1,27 @@
{
"name": "smtp",
"version": "1.0.0",
"private": true,
"type": "module",
"scripts": {
"dev": "tsx watch src/server.ts",
"build": "tsc",
"start": "node dist/server.js",
"lint": "eslint .",
"clean": "rimraf node_modules dist .turbo"
},
"dependencies": {
"@plunk/db": "*",
"dotenv": "^17.2.3",
"mailparser": "^3.7.1",
"signale": "^1.4.0",
"smtp-server": "^3.13.4"
},
"devDependencies": {
"@types/mailparser": "^3.4.4",
"@types/node": "^22.0.0",
"@types/signale": "^1.4.7",
"@types/smtp-server": "^3.5.10",
"tsx": "^4.20.6"
}
}
+3
View File
@@ -0,0 +1,3 @@
import {PrismaClient} from '@plunk/db';
export const prisma = new PrismaClient();
+509
View File
@@ -0,0 +1,509 @@
import 'dotenv/config';
import {simpleParser} from 'mailparser';
import signale from 'signale';
import {
SMTPServer,
type SMTPServerAddress,
type SMTPServerAuthentication,
type SMTPServerAuthenticationResponse,
type SMTPServerDataStream,
type SMTPServerSession,
type SMTPServerOptions,
} from 'smtp-server';
import fs from 'fs';
import path from 'path';
import {prisma} from './database/prisma.js';
// Environment variables
const API_URI = process.env.API_URI ?? 'http://localhost:3000';
const SMTP_DOMAIN = process.env.SMTP_DOMAIN ?? '';
const MAX_RECIPIENTS = parseInt(process.env.MAX_RECIPIENTS ?? '5', 10);
const PORT_SECURE = parseInt(process.env.PORT_SECURE ?? '465', 10);
const PORT_SUBMISSION = parseInt(process.env.PORT_SUBMISSION ?? '587', 10);
const CERT_PATH = process.env.CERT_PATH ?? '/certs';
const ACME_JSON_PATH = process.env.ACME_JSON_PATH ?? path.join(CERT_PATH, 'acme.json');
// Extended session interface to store authenticated project
interface ExtendedSession extends SMTPServerSession {
user?: string;
address?: string;
envelope: {
mailFrom: SMTPServerAddress | false;
rcptTo: SMTPServerAddress[];
};
}
/**
* Load certificates from Traefik's acme.json file
* Requires SMTP_DOMAIN to be set to select the correct certificate
*/
function loadFromTraefikAcme(): {key: Buffer; cert: Buffer} | null {
try {
if (!fs.existsSync(ACME_JSON_PATH)) {
return null;
}
if (!SMTP_DOMAIN) {
signale.warn('SMTP_DOMAIN not set, cannot load certificate from acme.json');
return null;
}
signale.info(`Loading certificates from Traefik acme.json: ${ACME_JSON_PATH}`);
const acmeData = JSON.parse(fs.readFileSync(ACME_JSON_PATH, 'utf8'));
// Traefik acme.json structure: {letsencrypt: {Certificates: [{domain: {main: ...}, certificate: ..., key: ...}]}}
const certificates = acmeData?.letsencrypt?.Certificates || acmeData?.Certificates || [];
interface TraefikCert {
domain?: string | { main?: string };
certificate: string;
key: string;
}
const certData = certificates.find((cert: TraefikCert) =>
(typeof cert.domain === 'object' && cert.domain?.main === SMTP_DOMAIN) || cert.domain === SMTP_DOMAIN
);
if (!certData) {
signale.warn(`Certificate for domain ${SMTP_DOMAIN} not found in acme.json`);
return null;
}
// Decode base64 encoded certificate and key
const cert = Buffer.from(certData.certificate, 'base64');
const key = Buffer.from(certData.key, 'base64');
signale.success(`Loaded certificate for ${SMTP_DOMAIN} from Traefik acme.json`);
return {key, cert};
} catch (error) {
signale.error('Failed to load Traefik acme.json:', error);
return null;
}
}
/**
* Load certificates from PEM files
* Looks for standard filenames: privkey.pem and fullchain.pem
*/
function loadFromPemFiles(): {key: Buffer; cert: Buffer} | null {
try {
const keyPath = path.join(CERT_PATH, 'privkey.pem');
const certPath = path.join(CERT_PATH, 'fullchain.pem');
if (!fs.existsSync(keyPath) || !fs.existsSync(certPath)) {
return null;
}
signale.info(`Loading certificates from ${keyPath} and ${certPath}`);
const key = fs.readFileSync(keyPath);
const cert = fs.readFileSync(certPath);
signale.success('Loaded certificates from PEM files');
return {key, cert};
} catch (error) {
signale.error('Failed to load PEM files:', error);
return null;
}
}
/**
* Load TLS certificates
* Priority:
* 1. Mounted Traefik acme.json
* 2. Mounted PEM files
* 3. No TLS
*/
async function getCertificates(): Promise<{key: Buffer; cert: Buffer} | null> {
try {
signale.info('Checking for mounted certificates...');
// Try loading from Traefik acme.json
let certs = loadFromTraefikAcme();
if (certs) {
return certs;
}
// Try loading from PEM files
certs = loadFromPemFiles();
if (certs) {
return certs;
}
signale.warn('No certificates found. SMTP server will run without TLS.');
signale.info('To use TLS, mount certificates to one of:');
signale.info(` 1. Traefik acme.json: ${ACME_JSON_PATH}`);
signale.info(` 2. PEM files: ${CERT_PATH}/privkey.pem and ${CERT_PATH}/fullchain.pem`);
return null;
} catch (error) {
signale.error('Failed to load certificates:', error);
return null;
}
}
/**
* Authentication handler
* Validates API key against the database
*/
async function handleAuth(
auth: SMTPServerAuthentication,
session: ExtendedSession,
callback: (err: Error | null | undefined, response?: SMTPServerAuthenticationResponse) => void,
): Promise<void> {
try {
// Username must be 'plunk'
if (auth.username?.toLowerCase() !== 'plunk') {
return callback(new Error('Invalid username'));
}
// Validate API key (project secret)
const project = await prisma.project.findUnique({
where: {
secret: auth.password,
},
select: {
id: true,
secret: true,
},
});
if (!project) {
return callback(new Error('Invalid API key'));
}
// Store the API key in the session for later use
callback(null, {user: project.secret});
} catch (error) {
signale.error('Authentication error:', error);
callback(new Error('Authentication failed'));
}
}
/**
* MAIL FROM handler
* Validates that the sender domain belongs to the authenticated project and is verified
*/
async function handleMailFrom(
address: SMTPServerAddress,
session: ExtendedSession,
callback: (err?: Error | null) => void,
): Promise<void> {
try {
const senderDomain = address.address.split('@')[1];
// First, get the authenticated project
const project = await prisma.project.findUnique({
where: {
secret: session.user,
},
select: {
id: true,
},
});
if (!project) {
return callback(new Error('Invalid session'));
}
// Check if this domain belongs to the project and is verified
const domain = await prisma.domain.findFirst({
where: {
projectId: project.id,
domain: senderDomain,
verified: true,
},
});
if (!domain) {
return callback(new Error('Sender domain is not verified or not associated with your account'));
}
// Store the sender address for later use
session.address = address.address;
callback();
} catch (error) {
signale.error('MAIL FROM error:', error);
callback(new Error('Failed to validate sender'));
}
}
/**
* RCPT TO handler
* Validates recipient limit
*/
function handleRcptTo(
address: SMTPServerAddress,
session: ExtendedSession,
callback: (err?: Error | null) => void,
): void {
if (session.envelope.rcptTo.length >= MAX_RECIPIENTS) {
return callback(new Error(`Maximum ${MAX_RECIPIENTS} recipients allowed`));
}
callback();
}
/**
* DATA handler
* Parses the email and relays it to the API
*/
function handleData(
stream: SMTPServerDataStream,
session: ExtendedSession,
callback: (err?: Error | null) => void,
): void {
simpleParser(stream, async (err, parsed) => {
if (err) {
signale.error('Email parsing error:', err);
return callback(new Error('Failed to parse email'));
}
// Validate sender
if (!parsed.from && !session.address) {
return callback(new Error('Email must have a sender'));
}
// Validate recipients
if (!session.envelope.rcptTo || session.envelope.rcptTo.length === 0) {
return callback(new Error('Email must have at least one recipient'));
}
// Validate subject
if (!parsed.subject || parsed.subject.trim() === '') {
return callback(new Error('Email must have a subject'));
}
// Validate subject length (RFC 5322 recommends < 78 chars per line, but max is 998)
if (parsed.subject.length > 998) {
return callback(new Error('Subject line is too long (max 998 characters)'));
}
// Validate body content
const bodyContent = parsed.html || parsed.text;
if (!bodyContent || (typeof bodyContent === 'string' && bodyContent.trim() === '')) {
return callback(new Error('Email must have a body (HTML or text content)'));
}
const fromAddress = parsed.from?.value[0]?.address ?? session.address;
const fromName = parsed.from?.value[0]?.name;
const recipients = session.envelope.rcptTo.map((to: SMTPServerAddress) => to.address);
// Parse attachments
const attachments = parsed.attachments
? parsed.attachments.map(attachment => ({
filename: attachment.filename || 'attachment',
content: attachment.content.toString('base64'),
contentType: attachment.contentType,
}))
: undefined;
// Extract custom headers from the parsed email
// Filter out standard headers that are automatically set
const standardHeaders = new Set([
'from',
'to',
'cc',
'bcc',
'subject',
'date',
'message-id',
'mime-version',
'content-type',
'content-transfer-encoding',
'reply-to',
'return-path',
'received',
'dkim-signature',
]);
// RFC 5322 header name validation: printable ASCII except colon and space
const headerNameRegex = /^[!-9;-~]+$/;
const MAX_CUSTOM_HEADERS = 50;
const customHeaders: Record<string, string> = {};
if (parsed.headers) {
for (const [key, value] of parsed.headers) {
const normalizedKey = key.toLowerCase();
// Skip standard headers
if (standardHeaders.has(normalizedKey)) {
continue;
}
// Validate header name (RFC 5322 compliance)
if (!headerNameRegex.test(key)) {
return callback(new Error(`Invalid header name: ${key}`));
}
// Limit number of custom headers to prevent abuse
if (Object.keys(customHeaders).length >= MAX_CUSTOM_HEADERS) {
return callback(new Error(`Too many custom headers (max ${MAX_CUSTOM_HEADERS})`));
}
// Get string value from header
const headerValue = Array.isArray(value) ? value.join(', ') : String(value);
// Validate header value length (RFC 5322 recommends max 998 chars per line)
if (headerValue.length > 998) {
return callback(new Error(`Header value too long for ${key} (max 998 characters)`));
}
customHeaders[key] = headerValue;
}
}
try {
// Relay to API
const response = await fetch(`${API_URI}/v1/send`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${session.user}`,
},
body: JSON.stringify({
from: fromAddress,
name: fromName,
to: recipients,
subject: parsed.subject,
body: bodyContent,
headers: Object.keys(customHeaders).length > 0 ? customHeaders : undefined,
attachments: attachments,
}),
});
if (!response.ok) {
const errorText = await response.text();
signale.error(`API error (${response.status}):`, errorText);
return callback(new Error(`Failed to send email: ${response.statusText}`));
}
signale.success(`Email relayed: ${fromAddress}${recipients.join(', ')}`);
callback();
} catch (error) {
signale.error('Relay error:', error);
callback(new Error('Failed to relay email to API'));
}
});
}
/**
* Create and start an SMTP server
*/
function createSMTPServer(options: {secure: boolean; port: number; key?: Buffer; cert?: Buffer}): void {
const {secure, port, key, cert} = options;
const serverOptions: SMTPServerOptions = {
onAuth: handleAuth,
onMailFrom: handleMailFrom,
onRcptTo: handleRcptTo,
onData: handleData,
banner: 'Plunk SMTP Relay',
size: 10 * 1024 * 1024, // 10MB max message size
authOptional: false, // Require authentication
};
// Add TLS if certificates are available
if (key && cert) {
serverOptions.key = key;
serverOptions.cert = cert;
serverOptions.secure = secure;
} else if (secure) {
signale.warn(`Cannot start secure server on port ${port} without TLS certificates`);
return;
}
const server = new SMTPServer(serverOptions);
server.on('error', (err: Error) => {
signale.error(`SMTP server error on port ${port}:`, err.message);
});
server.listen(port, '0.0.0.0', () => {
signale.success(`SMTP server listening on port ${port} (secure=${secure})`);
});
}
/**
* Validate environment configuration
*/
function validateEnvironment(): void {
// Validate MAX_RECIPIENTS
if (MAX_RECIPIENTS < 1 || MAX_RECIPIENTS > 100) {
signale.fatal('MAX_RECIPIENTS must be between 1 and 100');
process.exit(1);
}
// Validate ports
if (PORT_SECURE < 1 || PORT_SECURE > 65535) {
signale.fatal('PORT_SECURE must be a valid port number (1-65535)');
process.exit(1);
}
if (PORT_SUBMISSION < 1 || PORT_SUBMISSION > 65535) {
signale.fatal('PORT_SUBMISSION must be a valid port number (1-65535)');
process.exit(1);
}
if (PORT_SECURE === PORT_SUBMISSION) {
signale.fatal('PORT_SECURE and PORT_SUBMISSION must be different');
process.exit(1);
}
// Validate API_URI
try {
new URL(API_URI);
} catch {
signale.fatal('API_URI must be a valid URL');
process.exit(1);
}
signale.success('Environment configuration validated');
}
/**
* Main entry point
*/
async function main(): Promise<void> {
signale.info('Starting Plunk SMTP Relay...');
// Validate environment configuration
validateEnvironment();
signale.info(`API endpoint: ${API_URI}`);
signale.info(`Max recipients per email: ${MAX_RECIPIENTS}`);
if (SMTP_DOMAIN) {
signale.info(`SMTP domain: ${SMTP_DOMAIN}`);
}
const certs = await getCertificates();
// Start secure SMTP (port 465) - only if TLS is available
if (certs) {
createSMTPServer({
secure: true,
port: PORT_SECURE,
key: certs.key,
cert: certs.cert,
});
}
// Start submission SMTP (port 587) - STARTTLS if certs available, plaintext otherwise
createSMTPServer({
secure: false,
port: PORT_SUBMISSION,
key: certs?.key,
cert: certs?.cert,
});
signale.success('SMTP relay started successfully');
if (!certs) {
signale.warn('⚠️ Running without TLS encryption. Mount certificates to enable TLS.');
}
}
// Start the server
main().catch(error => {
signale.fatal('Failed to start SMTP server:', error);
process.exit(1);
});
+8
View File
@@ -0,0 +1,8 @@
{
"extends": "@plunk/typescript-config/base.json",
"compilerOptions": {
"rootDir": "src",
"outDir": "dist"
},
"exclude": ["node_modules", "dist"]
}