vynte-infra/twenty
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity
Files
e09b67158ec9fae9efcb3c4926c19081fdb0967d
twenty/packages/twenty-apps/webmetic/yarn.lock
T
Yannik SüßandGitHub ad80a50354 feat: add Webmetic Visitor Intelligence (#15551) 
# Webmetic Visitor Intelligence

Automatically sync B2B website visitor data into Twenty CRM. Identify
anonymous companies visiting your website and track their engagement
without forms or manual entry. Every hour, Webmetic enriches your CRM
with actionable sales intelligence about who's researching your product
before they ever fill out a contact form.

## Features

- 🔄 **Hourly Automatic Sync**: Fetches visitor data every hour via cron
trigger
- 🏢 **Company Enrichment**: Creates and updates company records with
visitor intelligence
- 📊 **Website Lead Tracking**: Records individual visits with detailed
engagement metrics
- 📈 **Engagement Scoring**: Webmetic's proprietary scoring algorithm
(0-100) identifies warm leads
- 🎯 **Sales Intelligence**: See which companies are actively researching
your product
- 🌍 **Geographic Data**: Captures visitor city and country information
- 🔗 **UTM Parameter Tracking**: Full campaign attribution with
utm_source, utm_medium, utm_campaign, utm_term, and utm_content
- 📄 **Page Journey Mapping**: Records complete navigation paths and
scroll depth
- ⚡ **Smart Deduplication**: Prevents duplicate records using
session-based identification
- 🔐 **Production-Ready**: Built with rate limiting, error handling, and
idempotent operations

## Requirements

- `twenty-cli` — `npm install -g twenty-cli`
- A Twenty API key (create one at
`https://twenty.com/settings/api-webhooks` and name it **"Webmetic"**)
- A Webmetic account with API access. Sign up at
[webmetic.de](https://webmetic.de)
- Node 18+ (for local development)

## Metadata prerequisites

The app automatically creates the `websiteLead` custom object with all
required fields on first run. No manual field provisioning is needed.

**Created automatically:**
- `websiteLead` object with 14 custom fields (TEXT, NUMBER, DATE_TIME
types)
- Company relation field (Many-to-One from websiteLead to Company)
- All fields are idempotent — safe to re-run without errors

## Quick start

### 1. Deploy the app

```bash
twenty auth login
cd packages/twenty-apps/webmetic
twenty app sync
```

### 2. Configure environment variables

- **First, create a Twenty API key**:
  - Go to **Settings → API & Webhooks → API Keys**
  - Click **+ Create API Key**
  - Name it **"Webmetic"**
  - Copy the generated key
- **Then configure the app**:
- Open **Settings → Apps → Webmetic Visitor Intelligence →
Configuration**
  - Provide values for the required keys:
- `TWENTY_API_KEY` (required secret; paste the API key you just created)
- `TWENTY_API_URL` (optional; defaults to `http://localhost:3000` for
local dev, set to your production URL)
- `WEBMETIC_API_KEY` (required secret; get from
[app.webmetic.de/?menu=api_details](https://app.webmetic.de/?menu=api_details))
- `WEBMETIC_DOMAIN` (required; your website domain to track, e.g.,
`example.com`)
  - Save the configuration

### 3. Test the function

- On the app page, select **Test your function**
- Click **Run**
- You should see a success summary showing companies and leads created
- Check **Settings → Data Model → Website Leads** to verify the custom
object was created
- Navigate to **Website Leads** from the sidebar to view synced visitor
data

### 4. Automatic hourly sync begins

The cron trigger (`0 * * * *`) runs every hour on the hour, continuously
syncing new visitor data.

## How it works

### Data flow

```
Webmetic API ─[hourly]→ sync-visitor-data ─[create/update]→ Twenty CRM
                              │
                              ├─→ Company records (with enrichment data)
                              └─→ Website Lead records (linked to companies)
```

### Sync process

1. **Cron Trigger**: Every hour at :00 (e.g., 1:00, 2:00, 3:00)
2. **Schema Validation**: Ensures `websiteLead` object and all fields
exist (creates if missing)
3. **Fetch Visitors**: Queries Webmetic API `/company-sessions` endpoint
for last hour
4. **Process Companies**: For each visitor's company:
   - Searches for existing company by domain
- Creates new company or updates existing with latest data from Webmetic
   - Extracts employee count from ranges (e.g., "11-50" → 50)
5. **Create Website Leads**: For each session:
   - Checks for duplicate (by name: "Company - Date")
   - Creates lead record with engagement metrics
   - Links to company via relation field
6. **Rate Limiting**: 800ms delay between API calls (75 requests/minute
max)

### Data captured

**Company enrichment (from Webmetic):**
- Name, domain, address (street, city, postcode, country)
- Employee count (parsed from ranges)
- LinkedIn URL (if available)
- Tagline/short description

**Website Lead tracking:**
- Visit date and session duration
- Page views count and pages visited (full navigation path)
- Traffic source (Direct, or utm_source/utm_medium combination)
- UTM campaign parameters (campaign, term, content)
- Visitor location (city, country)
- Engagement score (Webmetic's 0-100 scoring)
- Average scroll depth percentage
- Total user interaction events (clicks, etc.)

## Configuration reference

| Variable | Required | Description |
|----------|----------|-------------|
| `TWENTY_API_KEY` | ✅ Yes | Your Twenty API key for authentication |
| `TWENTY_API_URL` | ❌ No | Base URL of your Twenty instance (defaults
to `http://localhost:3000`) |
| `WEBMETIC_API_KEY` | ✅ Yes | Your Webmetic API key from
[app.webmetic.de/?menu=api_details](https://app.webmetic.de/?menu=api_details)
|
| `WEBMETIC_DOMAIN` | ✅ Yes | Website domain to track (e.g.,
`example.com` without protocol) |

## API integration

This app uses multiple Twenty CRM APIs:

**REST API** (data operations):
- `GET /rest/metadata/objects` — Fetch object metadata with fields
- `GET /rest/companies` — Find existing companies by domain
- `POST /rest/companies` — Create new companies
- `PATCH /rest/companies/:id` — Update company data
- `POST /rest/websiteLeads` — Create website lead records

**GraphQL Metadata API** (schema management):
- `createOneObject` mutation — Creates custom objects (if needed)
- `createOneField` mutation — Creates custom fields and relations

## Website Lead object structure

The app creates a custom `websiteLead` object with the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `name` | TEXT | Lead identifier (Company Name - Date) |
| `company` | RELATION | Many-to-One relation to Company object |
| `visitDate` | DATE_TIME | When the visit occurred |
| `pageViews` | NUMBER | Number of pages viewed during session |
| `sessionDuration` | NUMBER | Visit length in seconds |
| `trafficSource` | TEXT | Where visitor came from
(utm_source/utm_medium or Direct) |
| `pagesVisited` | TEXT | List of page URLs visited (→ separated, max
1000 chars) |
| `utmCampaign` | TEXT | UTM campaign parameter |
| `utmTerm` | TEXT | UTM term parameter (keywords for paid search) |
| `utmContent` | TEXT | UTM content parameter (for A/B testing) |
| `visitorCity` | TEXT | Geographic city of visitor |
| `visitorCountry` | TEXT | Geographic country of visitor |
| `visitCount` | NUMBER | Total number of visits from this company |
| `engagementScore` | NUMBER | Webmetic engagement score (0-100) |
| `averageScrollDepth` | NUMBER | Average scroll percentage (0-100) |
| `totalUserEvents` | NUMBER | Total count of user interactions (clicks,
etc.) |

## Troubleshooting

**Issue**: No data syncing after setup
- **Solution**: Run "Test your function" to manually trigger a sync and
check logs. Verify your `WEBMETIC_API_KEY` and `WEBMETIC_DOMAIN` are
correct.

**Issue**: "Duplicate Domain Name" error
- **Solution**: This occurs if you previously deleted a company. Twenty
maintains unique constraints on soft-deleted records. Either restore the
company from trash or contact support.

**Issue**: Missing fields on websiteLead object
- **Solution**: The sync function recreates missing fields
automatically. Run "Test your function" once to repair the schema.

**Issue**: Empty linkedinLink on companies
- **Solution**: Webmetic doesn't have LinkedIn data for that specific
company. The mapping is working correctly; data availability depends on
Webmetic's enrichment coverage.

**Issue**: Employee count not matching Webmetic
- **Solution**: Webmetic returns ranges (e.g., "11-50"). The app uses
the maximum value (50) to better represent company size.

**Issue**: Test shows "No new visitors in the last hour"
- **Solution**: Normal if you have no traffic in the last 60 minutes.
Wait for actual traffic or manually adjust the time range in code for
testing.

## Rate limiting and performance

- **Webmetic API**: No pagination used; fetches all visitors from last
hour
- **Twenty API**: 800ms delay between requests (75 requests/minute)
- **Processing**: Handles 14+ companies with full enrichment in under 30
seconds
- **Cron schedule**: `0 * * * *` (every hour on the hour)
- **Duplicate prevention**: Checks existing leads by name before
creating

## Development

### Local testing

```bash
cd packages/twenty-apps/webmetic
yarn install

# Set up .env file
cp .env.example .env
# Edit .env with your credentials

# Sync to local Twenty instance
npx twenty-cli app sync

# Watch for changes
npx twenty-cli app dev
```

### Manual trigger

Use the Twenty UI test panel or trigger via API:

```bash
curl -X POST http://localhost:3000/functions/sync-visitor-data \
  -H "Authorization: Bearer YOUR_API_KEY"
```

## Architecture notes

- **100% programmatic schema**: Fields created via GraphQL Metadata API,
not manifests
- **Idempotent operations**: Safe to re-run without duplicates or errors
- **Smart domain matching**: Normalizes domains (strips www, protocols)
for matching
- **Error resilience**: Individual company failures don't stop the
entire sync
- **Detailed logging**: Returns full execution log in response for
debugging

## Contributing

Built with 🍺 and ❤️ in Munich by [Team Webmetic](https://webmetic.de)
for Twenty CRM Hacktoberfest 2025.

For issues or questions:
- Webmetic API: [webmetic.de](https://webmetic.de)
- Twenty CRM: [twenty.com/developers](https://twenty.com/developers)

## License

MIT
2025-11-04 12:10:11 +01:00

274 lines
9.5 KiB
YAML
Raw Blame History

# This file is generated by running "yarn install" inside your project.
# Manual changes might be lost - proceed with caution!
__metadata:
version: 8
cacheKey: 10c0
"@types/node@npm:^24.7.2":
version: 24.9.1
resolution: "@types/node@npm:24.9.1"
dependencies:
undici-types: "npm:~7.16.0"
checksum: 10c0/c52f8168080ef9a7c3dc23d8ac6061fab5371aad89231a0f6f4c075869bc3de7e89b075b1f3e3171d9e5143d0dda1807c3dab8e32eac6d68f02e7480e7e78576
languageName: node
linkType: hard
"async-function@npm:^1.0.0":
version: 1.0.0
resolution: "async-function@npm:1.0.0"
checksum: 10c0/669a32c2cb7e45091330c680e92eaeb791bc1d4132d827591e499cd1f776ff5a873e77e5f92d0ce795a8d60f10761dec9ddfe7225a5de680f5d357f67b1aac73
languageName: node
linkType: hard
"async-generator-function@npm:^1.0.0":
version: 1.0.0
resolution: "async-generator-function@npm:1.0.0"
checksum: 10c0/2c50ef856c543ad500d8d8777d347e3c1ba623b93e99c9263ecc5f965c1b12d2a140e2ab6e43c3d0b85366110696f28114649411cbcd10b452a92a2318394186
languageName: node
linkType: hard
"asynckit@npm:^0.4.0":
version: 0.4.0
resolution: "asynckit@npm:0.4.0"
checksum: 10c0/d73e2ddf20c4eb9337e1b3df1a0f6159481050a5de457c55b14ea2e5cb6d90bb69e004c9af54737a5ee0917fcf2c9e25de67777bbe58261847846066ba75bc9d
languageName: node
linkType: hard
"axios@npm:^1.12.2":
version: 1.12.2
resolution: "axios@npm:1.12.2"
dependencies:
follow-redirects: "npm:^1.15.6"
form-data: "npm:^4.0.4"
proxy-from-env: "npm:^1.1.0"
checksum: 10c0/80b063e318cf05cd33a4d991cea0162f3573481946f9129efb7766f38fde4c061c34f41a93a9f9521f02b7c9565ccbc197c099b0186543ac84a24580017adfed
languageName: node
linkType: hard
"call-bind-apply-helpers@npm:^1.0.1, call-bind-apply-helpers@npm:^1.0.2":
version: 1.0.2
resolution: "call-bind-apply-helpers@npm:1.0.2"
dependencies:
es-errors: "npm:^1.3.0"
function-bind: "npm:^1.1.2"
checksum: 10c0/47bd9901d57b857590431243fea704ff18078b16890a6b3e021e12d279bbf211d039155e27d7566b374d49ee1f8189344bac9833dec7a20cdec370506361c938
languageName: node
linkType: hard
"combined-stream@npm:^1.0.8":
version: 1.0.8
resolution: "combined-stream@npm:1.0.8"
dependencies:
delayed-stream: "npm:~1.0.0"
checksum: 10c0/0dbb829577e1b1e839fa82b40c07ffaf7de8a09b935cadd355a73652ae70a88b4320db322f6634a4ad93424292fa80973ac6480986247f1734a1137debf271d5
languageName: node
linkType: hard
"delayed-stream@npm:~1.0.0":
version: 1.0.0
resolution: "delayed-stream@npm:1.0.0"
checksum: 10c0/d758899da03392e6712f042bec80aa293bbe9e9ff1b2634baae6a360113e708b91326594c8a486d475c69d6259afb7efacdc3537bfcda1c6c648e390ce601b19
languageName: node
linkType: hard
"dunder-proto@npm:^1.0.1":
version: 1.0.1
resolution: "dunder-proto@npm:1.0.1"
dependencies:
call-bind-apply-helpers: "npm:^1.0.1"
es-errors: "npm:^1.3.0"
gopd: "npm:^1.2.0"
checksum: 10c0/199f2a0c1c16593ca0a145dbf76a962f8033ce3129f01284d48c45ed4e14fea9bbacd7b3610b6cdc33486cef20385ac054948fefc6272fcce645c09468f93031
languageName: node
linkType: hard
"es-define-property@npm:^1.0.1":
version: 1.0.1
resolution: "es-define-property@npm:1.0.1"
checksum: 10c0/3f54eb49c16c18707949ff25a1456728c883e81259f045003499efba399c08bad00deebf65cccde8c0e07908c1a225c9d472b7107e558f2a48e28d530e34527c
languageName: node
linkType: hard
"es-errors@npm:^1.3.0":
version: 1.3.0
resolution: "es-errors@npm:1.3.0"
checksum: 10c0/0a61325670072f98d8ae3b914edab3559b6caa980f08054a3b872052640d91da01d38df55df797fcc916389d77fc92b8d5906cf028f4db46d7e3003abecbca85
languageName: node
linkType: hard
"es-object-atoms@npm:^1.0.0, es-object-atoms@npm:^1.1.1":
version: 1.1.1
resolution: "es-object-atoms@npm:1.1.1"
dependencies:
es-errors: "npm:^1.3.0"
checksum: 10c0/65364812ca4daf48eb76e2a3b7a89b3f6a2e62a1c420766ce9f692665a29d94fe41fe88b65f24106f449859549711e4b40d9fb8002d862dfd7eb1c512d10be0c
languageName: node
linkType: hard
"es-set-tostringtag@npm:^2.1.0":
version: 2.1.0
resolution: "es-set-tostringtag@npm:2.1.0"
dependencies:
es-errors: "npm:^1.3.0"
get-intrinsic: "npm:^1.2.6"
has-tostringtag: "npm:^1.0.2"
hasown: "npm:^2.0.2"
checksum: 10c0/ef2ca9ce49afe3931cb32e35da4dcb6d86ab02592cfc2ce3e49ced199d9d0bb5085fc7e73e06312213765f5efa47cc1df553a6a5154584b21448e9fb8355b1af
languageName: node
linkType: hard
"follow-redirects@npm:^1.15.6":
version: 1.15.11
resolution: "follow-redirects@npm:1.15.11"
peerDependenciesMeta:
debug:
optional: true
checksum: 10c0/d301f430542520a54058d4aeeb453233c564aaccac835d29d15e050beb33f339ad67d9bddbce01739c5dc46a6716dbe3d9d0d5134b1ca203effa11a7ef092343
languageName: node
linkType: hard
"form-data@npm:^4.0.4":
version: 4.0.4
resolution: "form-data@npm:4.0.4"
dependencies:
asynckit: "npm:^0.4.0"
combined-stream: "npm:^1.0.8"
es-set-tostringtag: "npm:^2.1.0"
hasown: "npm:^2.0.2"
mime-types: "npm:^2.1.12"
checksum: 10c0/373525a9a034b9d57073e55eab79e501a714ffac02e7a9b01be1c820780652b16e4101819785e1e18f8d98f0aee866cc654d660a435c378e16a72f2e7cac9695
languageName: node
linkType: hard
"function-bind@npm:^1.1.2":
version: 1.1.2
resolution: "function-bind@npm:1.1.2"
checksum: 10c0/d8680ee1e5fcd4c197e4ac33b2b4dce03c71f4d91717292785703db200f5c21f977c568d28061226f9b5900cbcd2c84463646134fd5337e7925e0942bc3f46d5
languageName: node
linkType: hard
"generator-function@npm:^2.0.0":
version: 2.0.1
resolution: "generator-function@npm:2.0.1"
checksum: 10c0/8a9f59df0f01cfefafdb3b451b80555e5cf6d76487095db91ac461a0e682e4ff7a9dbce15f4ecec191e53586d59eece01949e05a4b4492879600bbbe8e28d6b8
languageName: node
linkType: hard
"get-intrinsic@npm:^1.2.6":
version: 1.3.1
resolution: "get-intrinsic@npm:1.3.1"
dependencies:
async-function: "npm:^1.0.0"
async-generator-function: "npm:^1.0.0"
call-bind-apply-helpers: "npm:^1.0.2"
es-define-property: "npm:^1.0.1"
es-errors: "npm:^1.3.0"
es-object-atoms: "npm:^1.1.1"
function-bind: "npm:^1.1.2"
generator-function: "npm:^2.0.0"
get-proto: "npm:^1.0.1"
gopd: "npm:^1.2.0"
has-symbols: "npm:^1.1.0"
hasown: "npm:^2.0.2"
math-intrinsics: "npm:^1.1.0"
checksum: 10c0/9f4ab0cf7efe0fd2c8185f52e6f637e708f3a112610c88869f8f041bb9ecc2ce44bf285dfdbdc6f4f7c277a5b88d8e94a432374d97cca22f3de7fc63795deb5d
languageName: node
linkType: hard
"get-proto@npm:^1.0.1":
version: 1.0.1
resolution: "get-proto@npm:1.0.1"
dependencies:
dunder-proto: "npm:^1.0.1"
es-object-atoms: "npm:^1.0.0"
checksum: 10c0/9224acb44603c5526955e83510b9da41baf6ae73f7398875fba50edc5e944223a89c4a72b070fcd78beb5f7bdda58ecb6294adc28f7acfc0da05f76a2399643c
languageName: node
linkType: hard
"gopd@npm:^1.2.0":
version: 1.2.0
resolution: "gopd@npm:1.2.0"
checksum: 10c0/50fff1e04ba2b7737c097358534eacadad1e68d24cccee3272e04e007bed008e68d2614f3987788428fd192a5ae3889d08fb2331417e4fc4a9ab366b2043cead
languageName: node
linkType: hard
"has-symbols@npm:^1.0.3, has-symbols@npm:^1.1.0":
version: 1.1.0
resolution: "has-symbols@npm:1.1.0"
checksum: 10c0/dde0a734b17ae51e84b10986e651c664379018d10b91b6b0e9b293eddb32f0f069688c841fb40f19e9611546130153e0a2a48fd7f512891fb000ddfa36f5a20e
languageName: node
linkType: hard
"has-tostringtag@npm:^1.0.2":
version: 1.0.2
resolution: "has-tostringtag@npm:1.0.2"
dependencies:
has-symbols: "npm:^1.0.3"
checksum: 10c0/a8b166462192bafe3d9b6e420a1d581d93dd867adb61be223a17a8d6dad147aa77a8be32c961bb2f27b3ef893cae8d36f564ab651f5e9b7938ae86f74027c48c
languageName: node
linkType: hard
"hasown@npm:^2.0.2":
version: 2.0.2
resolution: "hasown@npm:2.0.2"
dependencies:
function-bind: "npm:^1.1.2"
checksum: 10c0/3769d434703b8ac66b209a4cca0737519925bbdb61dd887f93a16372b14694c63ff4e797686d87c90f08168e81082248b9b028bad60d4da9e0d1148766f56eb9
languageName: node
linkType: hard
"math-intrinsics@npm:^1.1.0":
version: 1.1.0
resolution: "math-intrinsics@npm:1.1.0"
checksum: 10c0/7579ff94e899e2f76ab64491d76cf606274c874d8f2af4a442c016bd85688927fcfca157ba6bf74b08e9439dc010b248ce05b96cc7c126a354c3bae7fcb48b7f
languageName: node
linkType: hard
"mime-db@npm:1.52.0":
version: 1.52.0
resolution: "mime-db@npm:1.52.0"
checksum: 10c0/0557a01deebf45ac5f5777fe7740b2a5c309c6d62d40ceab4e23da9f821899ce7a900b7ac8157d4548ddbb7beffe9abc621250e6d182b0397ec7f10c7b91a5aa
languageName: node
linkType: hard
"mime-types@npm:^2.1.12":
version: 2.1.35
resolution: "mime-types@npm:2.1.35"
dependencies:
mime-db: "npm:1.52.0"
checksum: 10c0/82fb07ec56d8ff1fc999a84f2f217aa46cb6ed1033fefaabd5785b9a974ed225c90dc72fff460259e66b95b73648596dbcc50d51ed69cdf464af2d237d3149b2
languageName: node
linkType: hard
"proxy-from-env@npm:^1.1.0":
version: 1.1.0
resolution: "proxy-from-env@npm:1.1.0"
checksum: 10c0/fe7dd8b1bdbbbea18d1459107729c3e4a2243ca870d26d34c2c1bcd3e4425b7bcc5112362df2d93cc7fb9746f6142b5e272fd1cc5c86ddf8580175186f6ad42b
languageName: node
linkType: hard
"twenty-sdk@npm:0.0.3":
version: 0.0.3
resolution: "twenty-sdk@npm:0.0.3"
checksum: 10c0/0a3c85c27edb22fb50f7eb0da4f9770e85729fce05e9e0118ad0cdfc36e42425c93340a6cd1c276daf30aeeaa612db0cd905831c0a8287a31bff3da5be9b0562
languageName: node
linkType: hard
"undici-types@npm:~7.16.0":
version: 7.16.0
resolution: "undici-types@npm:7.16.0"
checksum: 10c0/3033e2f2b5c9f1504bdc5934646cb54e37ecaca0f9249c983f7b1fc2e87c6d18399ebb05dc7fd5419e02b2e915f734d872a65da2e3eeed1813951c427d33cc9a
languageName: node
linkType: hard
"webmetic@workspace:.":
version: 0.0.0-use.local
resolution: "webmetic@workspace:."
dependencies:
"@types/node": "npm:^24.7.2"
axios: "npm:^1.12.2"
twenty-sdk: "npm:0.0.3"
languageName: unknown
linkType: soft
Reference in New Issue View Git Blame Copy Permalink