TEGI — Software Analysis

1. Language Strategy

Deliberate two-language strategy

Based on a review of 2025 best practices for TypeScript monorepos with Python ML workloads. The decision boundary is clear: use the library that is native to the task domain.

Layer	Language	Rationale
Next.js App	TypeScript	Core application — single-language stack for build velocity and end-to-end type safety
tRPC API Layer	TypeScript	Type safety between frontend and backend is the core value proposition of tRPC
BullMQ Workers	TypeScript	Email, i18n, notification workers stay in TS — simple queue consumers with no ML dependency
File Clerk Pipeline	Python	ML/NLP ecosystem superiority: unstructured, LlamaIndex, sentence-transformers, PyMuPDF. Communicates via BullMQ/Redis — no cross-language function calls
Doc Generation Scripts	Python	python-docx and python-pptx are more mature than JS equivalents. Used for one-off tooling only — not in the application runtime
Future: LoRA Fine-tuning	Python	Phase 6 fine-tuning pipeline will extend the File Clerk microservice — stays in Python

Key principle: Python is added only where the ML/NLP ecosystem provides a genuine advantage. All Python services communicate with the TypeScript core via BullMQ/Redis queues — never via direct function calls or HTTP internal APIs. This keeps the boundary clean and each runtime independently deployable.

2. Breaking Changes — Stack Versions

Critical version migrations

Three major dependencies have breaking changes not yet reflected in most tutorials. These are captured in docs/STACK_VERSIONS.md.

Next.js 15 → 16.1

middleware.ts replaced by proxy.ts
Async params in page/layout components
Turbopack is now the default bundler
Caching is opt-in (not opt-out as in v15)

Zod v3 → v4

Same import path (no breaking import changes)
Significantly faster parse performance
New standalone validators: z.email(), z.url(), z.uuid()
Some edge case behaviour changes in union resolution

Tailwind v3 → v4

tailwind.config.ts is removed entirely
Configuration moves to CSS via @theme directive
Utility class generation is now JIT-only by default
Some v3 plugin APIs changed

3. Monorepo Structure

TypeScript monorepo with pnpm workspaces

apps/web // Next.js 16 application — pages, components, tRPC client

packages/api // tRPC router definitions, Zod schemas, procedure implementations

packages/db // Prisma schema, migrations, typed DB client

packages/workers // BullMQ TypeScript workers: email, i18n, notification, graph-enrichment

services/file-clerk // Python 3.12 microservice — FastAPI, unstructured, LlamaIndex, sentence-transformers

packages/ui // Shared Tailwind v4 component library

packages/config // Shared ESLint, TypeScript, and environment config

docs/ // STACK_VERSIONS.md, ADRs, API reference

docker-compose.dev.yml // Infrastructure only: Postgres (5432), Redis, Qdrant, LibreTranslate

pm2.config.js // Process config for app + all TS workers in dev and production

4. Testing Mandate

Five-command definition of done

Hard rule: No task is declared complete without passing all five commands in sequence.

type-check → lint → test → build → test:e2e:stable

Type	Tool	Scope
Unit Tests	Vitest	All business logic, Zod schemas, utility functions, tRPC procedure logic
Integration Tests	Vitest + real DB	docker-compose.dev.yml must be running (port 5432 local, 5433 CI). Silently skipping due to port mismatch is a known gotcha.
E2E Tests	Playwright	Run on port 3001 (not 3002 dev). Use waitForURL instead of immediate assertions to eliminate flakiness.
IDOR Tests	Vitest	Required for every tRPC procedure — verify that entity A cannot access entity B's data via forged IDs.
i18n Check	pnpm i18n:check	All user-facing strings must pass. Enforced before task sign-off.
Python Tests	pytest	File Clerk microservice: unit tests for each pipeline stage, integration tests against real Qdrant and Postgres.

5. Security Model

Security by default at every layer

🔐 Authentication

NextAuth with OAuth providers (GitHub, LinkedIn, Google) + email magic link. Session tokens stored in Redis.

🛡️ Authorisation

tRPC middleware: every procedure checks entity ownership before data access. IDOR tests required per procedure.

🤖 AI Disclosure

is_ai_agent=TRUE is mandatory for all agent entities. Display banner on all surfaces. Cannot be overridden by entity owner post-creation.

🏛️ Trust Tier Gates

Unclaimed entities cannot post. Unverified entities display a disclaimer banner. Trust tier changes are logged in an immutable audit table.

🔒 Data Isolation

Context profiles are per user↔entity pair. Cross-entity leakage is architecturally prevented — no shared context store.

📦 File Clerk Sandbox

Ingestion pipeline runs in an isolated Python process. File parsing is sandboxed via resource limits. No filesystem access outside designated upload directory.