Overview
The insights system provides AI-powered feedback processing with template inheritance. Programs define InsightType templates, projects enable them via ProjectInsightType links, and customization creates local copies to preserve source templates.
What it is
A template inheritance system where programs define insight types and projects can enable, customize, or inherit them. AI pipeline processes feedback into structured insights.
Why we use it
Consistent insight templates across projects with project-level customization. AI-powered classification, deduplication, and impact scoring.
When to use
Processing user feedback, managing insight types, calculating impact scores. Always use ServiceContext for auth and audit.
Template Inheritance Model
Program Project
┌───────────────┐ ┌────────────────────┐
│ InsightType │ ──────▶ │ ProjectInsightType │
│ (template) │ │ (link) │
└───────────────┘ └────────────────────┘
│
┌────────┴────────┐
│ │
source mode local mode
(inherit) (customize)
│ │
▼ ▼
use parent's localInsightType
definition (project's copy)AI Pipeline Steps
| Step | Description |
|---|---|
| EXTRACT | Parse feedback into discrete insights |
| CLASSIFY | Match to InsightTypes |
| DEDUPLICATE | Find similar existing insights |
| ENRICH | Fill structured fields from context |
Deduplication Confidence Thresholds
| Confidence | Range | Action |
|---|---|---|
| High | >90% | Auto-merge |
| Medium | 70-90% | User choice |
| Low | <70% | Create new |
Quick Start
Template Access
Access templates through the ProjectInsightType link.
// Access templates through ProjectInsightType link const config = await projectInsightTypeRepository.getProjectInsightType(id); // Get effective type (local copy or inherited source) const effectiveType = config.localInsightType ?? config.sourceInsightType;
Patterns
Customization Flow
Create local copy before modifying.
// Customization flow - create local copy first await projectInsightTypeServices.customizeInsightType(ctx, configId); // Then update the local copy await projectInsightTypeServices.updateCustomizedBlocks(ctx, configId, blocks);
ServiceContext Pattern
Required for auth and audit trail.
// ServiceContext pattern - required for all mutations
interface ServiceContext {
userId: string;
projectId: string;
programId: string;
isProjectOwner: boolean;
isProgramOwner: boolean;
}
// Pass context to all service calls
await enableInsightType(ctx, sourceId);
await projectInsightTypeServices.customizeInsightType(ctx, configId);AI Feedback Processing
Process feedback with AI classification and deduplication.
// AI feedback processing
import { processFeedback, type ProcessingContext } from '@/lib/ai/insights';
const context: ProcessingContext = {
availableTypes: [...], // InsightTypes for classification
existingInsights: [...], // For deduplication
};
const result = await processFeedback(
{ content: feedback, projectId, userId },
context
);
if (result.requiresUserAction) {
// Duplicates need user confirmation
}Background Processing
Process feedback asynchronously via Inngest.
// Background feedback processing via Inngest
await inngest.send({
name: 'insights/feedback.process',
data: {
feedbackId: crypto.randomUUID(),
projectId,
userId,
content,
autoFinalize: true, // Auto-merge high-confidence duplicates
},
});Impact Scoring
Calculate and recalculate impact scores.
// Impact score calculation
import { calculateInsightScore } from '@/features/insights';
const result = await calculateInsightScore(ctx, insightId);
// Returns: { impactScore, confidenceScore, breakdown, calculatedAt }
// Bulk recalculation via Inngest
await inngest.send({
name: 'insights/scores.recalculate',
data: { projectId, reason: 'weights_updated' },
});UDE Context
Build context for AI processing.
// Build UDE context for AI processing
import { buildUDEInsightContext } from '@/features/insights';
const context = await buildUDEInsightContext(projectId, insightIds);
// Returns enriched insight data for AI analysisWatch Out
Access templates through ProjectInsightType, never directly
Don't
// WRONG - Skip ProjectInsightType link
const insightType = await prisma.formDefinition.findUnique({
where: { id: templateId },
});Do
// RIGHT - Go through ProjectInsightType const config = await projectInsightTypeRepository.getProjectInsightType(id); const effectiveType = config.localInsightType ?? config.sourceInsightType;
Create local copy before modifying templates
Don't
// WRONG - Modify program template directly
await prisma.formBlock.update({
where: { id: sourceBlockId },
data: { title: 'New Title' },
});Do
// RIGHT - Create local copy first await projectInsightTypeServices.customizeInsightType(ctx, configId); await projectInsightTypeServices.updateCustomizedBlocks(ctx, configId, blocks);
- Always pass ServiceContext for auth and audit
- Audit log all mutations with category, action, userId, scopeType, scopeId
Scoring Factors
| Factor | Default Weight | Description |
|---|---|---|
| submitterCount | 30 | Number of unique submitters |
| severity | 25 | Issue severity rating |
| engagement | 15 | User interaction level |
| voteScore | 15 | Community votes |
| evidenceDepth | 10 | Supporting evidence quality |
| recency | 5 | How recent the insight is |