Add email system planning doc for transactional and mass email

Provider abstraction (SMTP, SendGrid, SES, Postmark), email templates with merge fields, BullMQ queue for async sending, mass email campaigns with recipient filtering, CAN-SPAM unsubscribe management, email logging.
2026-03-28 15:38:54 -05:00
parent 6adce51e6c
commit 5aadd68128
1 changed files with 327 additions and 0 deletions
--- a/planning/24_Email_System.md
+++ b/planning/24_Email_System.md
@@ -0,0 +1,327 @@
+Music Store Management Platform
+
+Email System — Transactional & Mass Email
+
+Version 1.0  |  Draft
+
+
+
+# 1. Overview
+
+The email system handles all outbound email from the platform — both automated transactional messages (receipts, reminders, notifications) and bulk mass emails (announcements, promotions, newsletters). Like file storage, the email provider is abstracted behind an interface to support multiple backends.
+
+
+
+# 2. Email Provider Interface
+
+## 2.1 Providers
+
+Provider | Backend | Best For
+smtp | Any SMTP server | Self-hosted stores, existing email infrastructure
+sendgrid | SendGrid API | Deliverability, templates, analytics
+ses | AWS SES | Cost-effective high volume, SaaS deployment
+postmark | Postmark API | Transactional focus, fast delivery
+
+## 2.2 Configuration
+
+Variable | Description | Default
+EMAIL_PROVIDER | smtp, sendgrid, ses, postmark | smtp
+EMAIL_FROM_ADDRESS | Default sender address | noreply@{company domain}
+EMAIL_FROM_NAME | Default sender name | {Company Name}
+SMTP_HOST | SMTP server hostname | (required if smtp)
+SMTP_PORT | SMTP port | 587
+SMTP_USER | SMTP username | (required if smtp)
+SMTP_PASS | SMTP password | (required if smtp)
+SMTP_SECURE | Use TLS | true
+SENDGRID_API_KEY | SendGrid API key | (required if sendgrid)
+AWS_SES_REGION | SES region | (required if ses)
+AWS_SES_ACCESS_KEY | SES access key | (required if ses)
+AWS_SES_SECRET_KEY | SES secret key | (required if ses)
+POSTMARK_API_KEY | Postmark server token | (required if postmark)
+
+## 2.3 Interface
+
+```typescript
+interface EmailProvider {
+  send(message: EmailMessage): Promise<EmailResult>
+  sendBatch(messages: EmailMessage[]): Promise<EmailResult[]>
+}
+
+interface EmailMessage {
+  to: string | string[]
+  from?: string       // overrides default
+  replyTo?: string
+  subject: string
+  html: string
+  text?: string       // plain text fallback
+  tags?: string[]     // for tracking/categorization
+  metadata?: Record<string, string>  // provider-specific metadata
+}
+
+interface EmailResult {
+  success: boolean
+  messageId?: string
+  error?: string
+}
+```
+
+
+
+# 3. Email Types
+
+## 3.1 Transactional (System-Triggered)
+
+These fire automatically based on business events. Each has a template and trigger condition.
+
+Email | Trigger | Recipients | Template
+Receipt | Sale completed | Account email | Transaction summary, line items, payment method
+Rental Agreement | Rental activated | Account email | Agreement PDF attached, signing link
+Rental Reminder | 3 days before billing | Account email | Upcoming charge, amount, payment method
+Payment Failed | Charge declined | Account email | Failed amount, retry schedule, update card link
+Payment Received | Recurring charge success | Account email | Amount, date, next billing date
+Repair Ready | Repair status → ready | Account email or customer phone | Instrument ready for pickup
+Repair Estimate | Estimate created | Account email | Estimate details, approve/decline link
+Lesson Reminder | 24 hours before lesson | Member email or account email | Time, instructor, location
+Lesson Cancelled | Lesson cancelled | Member email | Reason, reschedule options
+Tax Exemption Expiring | 30 days before expiry | Account email | Certificate expiry warning
+Maintenance Due | Maintenance reminder due | Account email | Instrument, service needed, schedule link
+Welcome | Account created | Account email | Getting started, portal access
+Password Reset | Reset requested | User email | Reset link (24h expiry)
+
+## 3.2 Mass Email (Staff-Initiated)
+
+Staff can send bulk emails to filtered recipient lists. These are NOT marketing automation — they're one-off announcements.
+
+Use Cases:
+- Store sale or promotion announcement
+- Schedule change notification (holiday hours)
+- Rental program updates
+- Back-to-school instrument availability
+- Concert or event invitations
+- Policy changes
+
+## 3.3 Recipient Lists
+
+Mass emails target filtered lists built from account/member data:
+
+Filter | Description
+All accounts | Every account with an email
+Active rentals | Accounts with active rental contracts
+Active lessons | Members with active lesson enrollments
+By instrument type | Members renting/owning specific instruments
+By school | Accounts linked to a specific school
+By category | Accounts that purchased from a product category
+By location | Accounts at a specific store location
+Tax exempt | Tax-exempt accounts only
+Custom | Staff-built filter with multiple criteria
+
+
+
+# 4. Database Schema
+
+## 4.1 email_template
+
+Store-customizable email templates. System provides defaults, stores can modify.
+
+Column | Type | Notes
+id | uuid PK |
+company_id | uuid FK | Tenant scoping
+slug | varchar | Unique identifier: receipt, rental_reminder, repair_ready, etc.
+name | varchar | Display name
+subject | varchar | Email subject line (supports merge fields)
+html_body | text | HTML body (supports merge fields)
+text_body | text | Plain text fallback
+is_system | boolean | Default false. System templates seeded on company creation.
+is_active | boolean | Inactive templates don't send
+created_at | timestamptz |
+updated_at | timestamptz |
+
+## 4.2 email_log
+
+Append-only log of every email sent. Used for troubleshooting, compliance, and deliverability monitoring.
+
+Column | Type | Notes
+id | uuid PK |
+company_id | uuid FK |
+template_id | uuid FK | Nullable — mass emails may not use a template
+to_address | varchar | Recipient email
+from_address | varchar | Sender address used
+subject | varchar | Actual subject sent
+message_id | varchar | Provider's message ID for tracking
+status | varchar | sent, failed, bounced, delivered, opened, clicked
+error | text | Error message if failed
+entity_type | varchar | Nullable — what triggered this: transaction, rental, repair_ticket, etc.
+entity_id | uuid | Nullable — ID of the triggering record
+campaign_id | uuid FK | Nullable — if part of a mass email campaign
+sent_at | timestamptz |
+created_at | timestamptz |
+
+## 4.3 email_campaign
+
+Mass email campaigns.
+
+Column | Type | Notes
+id | uuid PK |
+company_id | uuid FK |
+name | varchar | Campaign name for internal reference
+subject | varchar | Email subject
+html_body | text | Email content
+text_body | text | Plain text fallback
+status | varchar | draft, scheduled, sending, sent, cancelled
+filter_criteria | jsonb | Recipient filter criteria (serialized query)
+recipient_count | integer | Total recipients at send time
+sent_count | integer | Successfully sent
+failed_count | integer | Failed to send
+scheduled_at | timestamptz | Nullable — when to send (null = send immediately)
+sent_at | timestamptz | When sending started
+completed_at | timestamptz | When sending finished
+created_by | uuid FK | Employee who created
+created_at | timestamptz |
+updated_at | timestamptz |
+
+## 4.4 email_unsubscribe
+
+Opt-out tracking. Respects CAN-SPAM.
+
+Column | Type | Notes
+id | uuid PK |
+company_id | uuid FK |
+email | varchar | The unsubscribed email address
+account_id | uuid FK | Nullable
+reason | text | Nullable — why they unsubscribed
+unsubscribed_at | timestamptz |
+created_at | timestamptz |
+
+
+
+# 5. Merge Fields
+
+Templates support merge fields that resolve at send time:
+
+Field | Resolves To
+{{account_name}} | account.name
+{{account_email}} | account.email
+{{member_name}} | member.firstName + member.lastName
+{{company_name}} | company.name
+{{company_phone}} | company.phone
+{{company_email}} | company.email
+{{amount}} | Transaction/payment amount (formatted)
+{{date}} | Relevant date (formatted)
+{{instrument}} | Instrument description
+{{portal_url}} | Customer portal URL
+{{unsubscribe_url}} | One-click unsubscribe link (required for mass email)
+
+Custom merge fields per template type (e.g. repair estimate has {{estimate_total}}, {{repair_description}}).
+
+
+
+# 6. Email Queue
+
+Emails are not sent synchronously from API routes. They're queued via BullMQ and processed by a worker.
+
+## 6.1 Queue Flow
+
+1. Business event fires (e.g. repair status changes to "ready")
+2. Event handler creates an email job: `emailQueue.add('send', { templateSlug, recipientAccountId, entityType, entityId, data })`
+3. Worker picks up the job
+4. Worker resolves template + merge fields
+5. Worker calls `emailProvider.send()`
+6. Result logged to email_log
+7. On failure: retry with exponential backoff (3 attempts over 1 hour)
+
+## 6.2 Mass Email Flow
+
+1. Staff creates campaign with content and filter criteria
+2. Staff clicks "Send" (or schedules)
+3. System resolves recipient list from filter criteria
+4. Creates one job per recipient (batched, throttled)
+5. Worker processes queue with rate limiting (avoid provider throttling)
+6. Campaign status updated as batches complete
+7. Unsubscribed addresses skipped automatically
+
+## 6.3 Rate Limiting
+
+Provider | Default Rate
+smtp | 10/second
+sendgrid | 100/second
+ses | 14/second (sandbox), higher with production access
+postmark | 50/second
+
+
+
+# 7. Unsubscribe Management
+
+## 7.1 Requirements (CAN-SPAM)
+
+- Every mass email includes an unsubscribe link
+- Unsubscribe is one-click (no login required)
+- Unsubscribe processed within 10 business days (we do it instantly)
+- Unsubscribed addresses never receive mass email
+- Transactional emails (receipts, billing) are exempt from unsubscribe
+
+## 7.2 Unsubscribe Endpoint
+
+GET /unsubscribe?token={signed_token}
+
+- Token encodes email + company_id, signed with JWT secret
+- No authentication required
+- Shows confirmation page
+- Inserts into email_unsubscribe table
+- Returns "You've been unsubscribed" page
+
+
+
+# 8. Admin UI
+
+## 8.1 Email Templates Page
+
+- List all templates with status (active/inactive)
+- Edit template: subject, HTML body, text body
+- Preview with sample data
+- Reset to system default
+
+## 8.2 Mass Email Page
+
+- Create campaign: subject, body (rich text editor), recipient filter
+- Preview recipient count before sending
+- Schedule or send immediately
+- Campaign history with sent/failed counts
+- View individual send status
+
+## 8.3 Email Log
+
+- Searchable log of all sent emails
+- Filter by template, status, date, recipient
+- View email content as sent
+- Resend failed emails
+
+
+
+# 9. Implementation Order
+
+1. EmailProvider interface with SmtpProvider
+2. email_template table + seed system templates
+3. email_log table
+4. BullMQ email queue + worker
+5. Transactional triggers (start with receipt + repair ready)
+6. Template editor in admin UI
+7. email_campaign + email_unsubscribe tables
+8. Mass email UI (create, filter, send)
+9. Additional providers (SendGrid, SES, Postmark)
+10. Email log viewer in admin UI
+
+
+
+# 10. Business Rules
+
+- Emails are always queued, never sent synchronously from API routes
+- Failed emails retry 3 times with exponential backoff
+- Unsubscribed addresses are never sent mass email — checked at send time
+- Transactional emails always send regardless of unsubscribe status
+- All emails logged with full content for compliance
+- Email templates are company-scoped — each store customizes their own
+- Mass emails require at least one unsubscribe link in the body
+- Email provider is set at startup via env var — same as storage provider
+- BullMQ uses the same Valkey instance as the rest of the application
+- Mass email campaigns are rate-limited per provider to avoid throttling
+- Empty recipient lists block campaign send with clear error