Zum Inhalt

Email Activity Import from EML Files

This guide explains how to import archived .eml files from the filesystem into Newton as email activities and activity mails.

Overview

The EML import command scans a directory structure such as:

./data/eml-files/
├── 22532010/
│   ├── first-email.eml
│   ├── reply-1.eml
│   └── ...
└── 22540012/
    └── kickoff-notes.eml

Each top-level folder name must match an existing projectNumber. Every .eml file inside that folder is parsed and stored as:

  • An Activity of type Email, linked to the project
  • An ActivityMail row containing headers, recipients, timestamps, and the full EML content

Duplicate detection is handled via the internetMessageId header. If an email with the same ID already exists, it is skipped automatically.

Command

nx run api:import-emails <path-to-eml-root> [options]

Options

Option Description Example
--dry-run Parses emails and reports statistics without writing to the database --dry-run
--project=... Restrict import to one or more project numbers (comma-separated or repeat) --project=22532010
--limit=number Stop after importing the specified number of new emails (ignores dry-run) --limit=50

Multiple --project flags can be provided:

nx run api:import-emails ./data/eml-files --project=22532010 --project=22540012

Typical Workflow

  1. Prepare filesystem so each project has its own directory containing .eml files.
  2. If you start with Outlook .msg archives, run scripts/convert-msg-to-eml.ps1 first: powershell pwsh ./scripts/convert-msg-to-eml.ps1 -SourceRoot "\\192.168.200.146\mails\projekte" -Overwrite This keeps the same project/year folder layout and produces .eml siblings.
  3. Run a dry run to preview statistics and detect missing projects:

bash nx run api:import-emails ./data/eml-files --dry-run

  1. Review the summary for duplicates, missing projects, or parse errors.
  2. Execute the real import (omit --dry-run) once the preview looks correct:

bash nx run api:import-emails ./data/eml-files

Summary Output

During execution the command prints a summary similar to:

=== EML IMPORT SUMMARY ===
Resolved root: C:/workspace/newton/data/eml-files
Projects discovered: 12
Projects processed: 10
Total EML files found: 642
Imported emails: 618
Dry-run emails: 24
Skipped existing emails: 4
Skipped emails without internetMessageId: 0
Emails skipped due to missing projects: 20

Errors (if any) are listed with the project number and relative file path to make cleanup easier.

Data Handling Details

  • Project lookup: Folder name must match an existing Project.projectNumber. Missing projects are reported and their emails are skipped.
  • Activity defaults:
  • Type: Email
  • Visibility: Management, BackOffice, ProjectManager, Employee
  • Title: Email subject (truncated to 200 chars)
  • Description: Body preview (first 2,000 characters)
  • Occurred at: The email Date header (fallback to current time)
  • ActivityMail fields are populated from standard headers (From, To, Cc, Bcc, Message-ID, Date).
  • Raw EML files are uploaded to the configured S3-compatible object storage. The resulting key is stored in the ActivityMail.emlStorageKey column for later retrieval (e.g. download links or reprocessing).
  • Duplicate protection: The importer checks internetMessageId before creating new records.

Troubleshooting

Issue Cause / Fix
Project not found warnings Create the project first or move the folder to the correct project number.
Failed to import ... errors Inspect the referenced EML file (invalid encoding, missing headers, etc.).
Emails skipped as “existing” The Message-ID already exists in the database; no action needed.
Emails skipped without internetMessageId File lacks a Message-ID header; add one manually or accept the skip.
Missing configuration for ... at startup Ensure all EML_STORAGE_* variables are defined in .env.

Object Storage Configuration

The importer and runtime API both rely on an S3-compatible bucket (MinIO by default) for persisting raw EML files. Set the following variables in .env (see .env.example for defaults):

EML_STORAGE_ENDPOINT=http://localhost:9000
EML_STORAGE_REGION=us-east-1
EML_STORAGE_BUCKET=newton-eml
EML_STORAGE_ACCESS_KEY=minioadmin
EML_STORAGE_SECRET_KEY=minioadmin
EML_STORAGE_BASE_URL=http://localhost:9000/newton-eml
EML_STORAGE_PREFIX=
EML_STORAGE_FORCE_PATH_STYLE=true

The optional EML_STORAGE_BASE_URL is used to build browser-accessible links (e.g. when serving download URLs). Leave it empty if the bucket should only be accessed via signed URLs or internal tooling.

For additional support, contact the Newton backend team.