products

🧠 Designing Full-Stack Instrumentation for an AI Chat Application

From a click on a ShadCN button → to a user question → retrieval → generation → citations → feedback → product analytics → distributed traces.

Most teams start observability for an AI application with questions such as:

• How many questions were asked?

• How many users visited?

• How many answers failed?

• What is our average latency?

Those are useful questions.

But they barely scratch the surface.

Imagine a user arrives at Pensieve, opens a chat, asks:

“How do I submit a new servicing request?”

Pensieve retrieves three relevant pages, generates an answer, renders two citations and a form link.

The user:

1. reads the answer,

2. opens the second citation,

3. returns,

4. copies a command from the answer,

5. clicks 👍,

6. asks a follow-up question,

7. eventually opens the form.

Was that one interaction?

From the user’s perspective, yes.

From the instrumentation perspective, it may involve:

flowchart LR

A["🌐 Session"] --> B["💬 Chat"]
B --> C["❓ Turn"]

C --> D["📨 Request"]
D --> E["🔎 Retrieval"]
E --> F["🤖 Generation"]
F --> G["📄 Response"]

G --> H["🔗 Citation Impression"]
G --> I["📋 Copy"]
G --> J["👍 Feedback"]
G --> K["🔄 Regenerate"]
G --> L["📝 Form Link"]

H --> M["🔗 Citation Click"]
L --> N["✅ Business Action"]

And underneath those product interactions there may be dozens of technical operations:

flowchart LR

UI["React UI"]
API["Chat API"]
RAG["Retrieval"]
DB["Vector / Search"]
LLM["LLM"]
FMT["Response Formatter"]
MDB["MongoDB"]
OTEL["OTel Collector"]

UI --> API
API --> RAG
RAG --> DB
API --> LLM
LLM --> FMT
FMT --> UI

UI -. telemetry .-> OTEL
API -. telemetry .-> OTEL
RAG -. telemetry .-> OTEL
LLM -. telemetry .-> OTEL

UI -. product events .-> MDB
API -. product events .-> MDB

The challenge therefore is not simply:

“How do we track clicks?”

The real problem is:

How do we create a coherent event and identity model that allows Product, Frontend, Backend, ML, AI, Data and Platform teams to reconstruct what happened without every team inventing its own telemetry language?

That is the problem Pensieve Instrumentation solves.

🌟 1. Pensieve as a Case Study

Pensieve is an AI question-answering application.

A user can:

This creates several completely different observability questions.

👩‍💼 Product Owner

Are people actually getting value from Pensieve?

🎨 Frontend Engineer

Did users see the citation component before we say that they ignored it?

⚙️ Backend Engineer

Which service caused this request to take 11 seconds?

🤖 AI Engineer

Was the poor answer caused by retrieval, reranking, prompting or generation?

📊 Data Analyst

What percentage of first-time users who ask one question return within seven days?

🏗️ Platform Engineer

Are telemetry pipelines themselves dropping events?

Pensieve therefore needs multiple telemetry planes sharing one correlation model.

🧭 2. Observability Is Not the Same as Product Analytics

One of the most important architectural decisions is separating these concepts.

OpenTelemetry is excellent for generating and correlating traces, metrics and logs. The OpenTelemetry Collector then provides vendor-neutral receiving, processing and exporting, including batching and retry behavior.

But consider this event:

response.liked

Should that disappear because your production environment samples only 10% of traces?

Absolutely not.

Similarly:

citation.clicked

is primarily a business/product fact, not merely a debugging artifact.

This leads us to the Pensieve dual-plane architecture.

flowchart TB

USER["👤 User"]
UI["⚛️ React + ShadCN"]

USER --> UI

subgraph PRODUCT["📊 PRODUCT EVENT PLANE"]
EVENTSDK["Pensieve Event SDK"]
INGEST["Event Ingestion API"]
QUEUE["Async Event Queue"]
WRITER["Batch Writer"]
MONGO[("MongoDB<br/>Analytics Event Store")]

EVENTSDK --> INGEST --> QUEUE --> WRITER --> MONGO
end

subgraph OBS["🔭 OBSERVABILITY PLANE"]
WEBOTEL["OpenTelemetry Web SDK"]
BACKEND["Backend OTel SDK"]
COLLECTOR["OpenTelemetry Collector"]
OBSDB["Trace / Metric / Log Backend"]

WEBOTEL --> COLLECTOR
BACKEND --> COLLECTOR
COLLECTOR --> OBSDB
end

UI --> EVENTSDK
UI --> WEBOTEL

subgraph APP["🤖 APPLICATION PLANE"]
CHATAPI["Chat API"]
RAG["RAG / Search"]
LLM["LLM"]
end

UI --> CHATAPI
CHATAPI --> RAG
CHATAPI --> LLM
CHATAPI --> BACKEND

EVENTSDK -. "trace_id + IDs" .-> WEBOTEL
CHATAPI -. "domain events" .-> INGEST

The crucial point is:

Both planes share the same identifiers.

Therefore Product can discover:

Response resp_5827 received a dislike.

Engineering can then discover:

Response resp_5827 belonged to trace 9be....

AI engineering can discover:

Retrieval returned poorly ranked documents.

Suddenly telemetry becomes investigative infrastructure, rather than dashboards.

🪪 3. Start With Identity, Not Events

This is probably the single most important section of the architecture.

If the identifiers are wrong, no amount of instrumentation added later will save the analytics model.

For Pensieve I would use the following hierarchy.

flowchart TD

V["👤 visitor_id<br/>Persistent anonymous browser identity"]

V --> S1["🌐 session_id"]
V --> S2["🌐 session_id"]
V --> S3["🌐 session_id"]

S1 --> C1["💬 chat_id"]
S1 --> C2["💬 chat_id"]

C1 --> T1["🔁 turn_id"]
C1 --> T2["🔁 turn_id"]

T1 --> RQ1["📨 request_id<br/>Attempt 1"]
T1 --> RQ2["📨 request_id<br/>Regeneration"]

RQ1 --> RS1["🤖 response_id"]
RQ2 --> RS2["🤖 response_id"]

RS1 --> E1["⚡ event_id"]
RS1 --> E2["⚡ event_id"]
RS1 --> E3["⚡ event_id"]

RQ1 -.-> TRACE["🔭 trace_id"]
TRACE --> SP1["span_id"]
TRACE --> SP2["span_id"]
TRACE --> SP3["span_id"]

ID Contract

🔥 4. Why turn_id Matters

Suppose the user asks:

“What is the eligibility requirement?”

Pensieve generates answer A.

The user presses Regenerate.

Pensieve generates answer B.

These are not two questions.

They represent:

flowchart LR

Q["User Question<br/>turn_id = T100"]

Q --> A["request_id = R101<br/>response_id = A101"]

Q --> B["request_id = R102<br/>response_id = A102"]

Q --> C["request_id = R103<br/>response_id = A103"]

Without turn_id, analytics might incorrectly report:

3 questions asked

when in reality:

1 question
3 generation attempts

That difference affects:

• question volume,

• user engagement,

• model quality,

• regeneration rate,

• token cost,

• request load,

• answer satisfaction.

🌐 5. Session Identity Needs Another Identity Above It

If every browser session receives a new session_id, how do we calculate:

Returning Users
D7 Retention
Sessions per User
MAU
User Stickiness

We cannot.

Therefore Pensieve needs:

visitor_id
    ↓
session_id

For anonymous users, visitor_id can be a pseudonymous identifier persisted in first-party browser storage.

For authenticated users:

visitor_id + user_id

can be retained independently.

OpenTelemetry itself now defines developing client-session semantic conventions including session.id, session.previous_id, session.start and session.end.

A useful browser policy

localStorage
    visitor_id

sessionStorage
    session_id

runtime
    tab_id

If Pensieve specifically defines every newly opened tab/window as a new analytics session, generate session_id at application bootstrap rather than trying to infer conventional 30-minute analytics sessions.

Alternatively use:

visitor
   └── session
        └── tab

if you eventually want multiple tabs grouped into a single browsing session.

⚡ 6. What Exactly Is an Event?

An event should represent a meaningful occurrence.

OpenTelemetry’s event guidance makes an important distinction:

• something having a meaningful duration → span

• a point-in-time occurrence/state change → event

• an operation-wide characteristic → span attribute

Consider:

response.generate

It has duration.

Use a span.

But:

response.like_clicked

occurs at a moment.

Use an event.

🧱 7. The Pensieve Event Envelope

Every product event should conform to a consistent envelope.

{
  "event_id": "0199f58e-...",
  "event_name": "response.liked",
  "event_version": 1,

  "event_time": "2026-10-04T22:15:31.442Z",
  "received_at": "2026-10-04T22:15:31.517Z",

  "actor": {
    "visitor_id": "vis_...",
    "user_id": null
  },

  "context": {
    "session_id": "ses_...",
    "tab_id": "tab_...",
    "chat_id": "chat_...",
    "turn_id": "turn_...",
    "request_id": "req_...",
    "response_id": "resp_..."
  },

  "trace": {
    "trace_id": "...",
    "span_id": "..."
  },

  "application": {
    "version": "2.4.1",
    "environment": "production"
  },

  "page": {
    "route": "/chat",
    "referrer": "/home"
  },

  "component": {
    "id": "answer-feedback",
    "type": "button"
  },

  "properties": {
    "feedback": "positive"
  }
}

🏷️ 8. Event Naming

Use names such as:

session.started
chat.created
question.submitted
response.completed
response.liked
citation.clicked
form.submitted

Do not create:

response_liked_resp_123
citation_clicked_document_782
button_clicked_blue_submit

Dynamic values belong in attributes.

OpenTelemetry similarly recommends event names remain stable and low-cardinality, with dynamic values represented as event attributes.

🧩 9. Do Not Instrument Every React Component

This is another common mistake.

Imagine automatically instrumenting every:

<Card>
<Button>
<Dialog>
<Popover>
<Tooltip>
<Separator>
<Avatar>

in ShadCN.

You quickly generate millions of events that nobody can interpret.

Instead distinguish:

flowchart TD

COMP["React Component"]

COMP --> M["Mounted"]
COMP --> I["Visible / Impression"]
COMP --> C["Interacted"]

M -->|"Usually engineering"| E["Frontend diagnostics"]

I -->|"Meaningful product surface"| P["Product analytics"]

C -->|"User action"| P

For product analytics, visible is more important than mounted.

A citation component may technically mount below the fold.

That does not mean the user actually saw it.

For impressions use:

IntersectionObserver

rather than React mount alone.

🧾 10. Pensieve Event Taxonomy

Application + Session Events

💬 11. Chat Events

🤖 12. Response Events

🔗 13. Citation Events

For grounded systems, citations deserve their own instrumentation model.

Important attributes could include:

citation_id
document_id
document_type
rank
retrieval_score
reranker_score
position
domain
response_id

Now we can answer:

Are users more likely to click citation #1 than citation #4?

or:

Do documents retrieved at ranks 4–6 ever drive downstream actions?

That is enormously valuable feedback for the RAG system.

📝 14. Form and Action Events

For Pensieve, URLs and forms may represent the closest thing to BOFU conversion.

Instrument:

This lets Product distinguish:

ANSWER SHOWN

from:

ANSWER CAUSED ACTION

🔎 15. RAG-Specific Events and Spans

Some interactions belong primarily on the backend.

sequenceDiagram

participant API as Chat API
participant Q as Query Processor
participant R as Retriever
participant RR as Reranker
participant L as LLM
participant E as Evaluator

API->>Q: Normalize query
Q->>R: Retrieve candidates
R-->>Q: Top-K documents

Q->>RR: Rerank
RR-->>Q: Ranked documents

Q->>L: Prompt + context
L-->>API: Streaming answer

API->>E: Async evaluation
E-->>API: Grounding / relevance scores

Recommended spans:

chat.request
query.rewrite
retrieval.search
retrieval.rerank
context.build
prompt.render
llm.generate
response.format
response.stream
evaluation.faithfulness
evaluation.relevance

Possible attributes:

retrieval.top_k
retrieval.returned_count
retrieval.index_version
reranker.model
prompt.version
model.name
model.version
context.token_count
input.token_count
output.token_count
finish_reason

OpenTelemetry’s GenAI semantic conventions already cover concepts such as conversation identifiers, provider, model responses, retrieval documents and GenAI evaluation attributes. They also explicitly warn that input/output messages and retrieval queries may contain sensitive information.

🏗️ 16. Full Pensieve Architecture

flowchart TB

subgraph BROWSER["🌐 Browser"]
REACT["React Application"]
SHADCN["ShadCN Components"]
EVENT["Pensieve Analytics SDK"]
OTELWEB["OTel Web SDK"]
BUFFER["In-Memory Event Buffer"]
ID["Identity Context"]

SHADCN --> REACT
ID --> EVENT
ID --> OTELWEB
REACT --> EVENT
REACT --> OTELWEB
EVENT --> BUFFER
end

subgraph EDGE["🚪 Application Edge"]
CHAT["Chat API"]
INGEST["Telemetry Ingest API"]
OTELPROXY["OTLP / Telemetry Proxy"]
end

subgraph AI["🤖 AI Runtime"]
QUERY["Query Processing"]
SEARCH["Retriever"]
RERANK["Reranker"]
MODEL["LLM"]
EVAL["Evaluation"]
end

subgraph EVENTS["⚡ Product Event Pipeline"]
QUEUE["Queue / Stream"]
WORKER["Event Writer"]
MONGO[("MongoDB<br/>Product Analytics")]
AGG["Aggregation Jobs"]
MART[("Metrics / Product Mart")]
end

subgraph OBS["🔭 Observability"]
COLLECTOR["OTel Collector"]
TRACE["Trace Backend"]
METRICS["Metrics Backend"]
LOGS["Logs Backend"]
end

BUFFER -->|"batched HTTPS"| INGEST
INGEST --> QUEUE
QUEUE --> WORKER
WORKER --> MONGO
MONGO --> AGG --> MART

REACT --> CHAT
CHAT --> QUERY
QUERY --> SEARCH
QUERY --> RERANK
QUERY --> MODEL
MODEL --> EVAL

OTELWEB --> OTELPROXY --> COLLECTOR
CHAT -. OTLP .-> COLLECTOR
QUERY -. OTLP .-> COLLECTOR
SEARCH -. OTLP .-> COLLECTOR
MODEL -. OTLP .-> COLLECTOR

COLLECTOR --> TRACE
COLLECTOR --> METRICS
COLLECTOR --> LOGS

CHAT -. "domain events" .-> QUEUE

🔄 17. What Happens When Someone Asks a Question?

sequenceDiagram

actor User
participant UI as React UI
participant T as Pensieve Telemetry
participant API as Chat API
participant RAG as RAG Pipeline
participant LLM as LLM
participant Q as Event Queue
participant DB as MongoDB
participant OTel as OTel Collector

User->>UI: Submit question

UI->>T: question.submitted
T-->>UI: non-blocking

UI->>API: POST /chat
Note over UI,API: request_id + chat_id + turn_id + traceparent

API->>OTel: Start server trace
API->>Q: question.accepted

API->>RAG: Retrieve context
RAG->>OTel: retrieval span

RAG-->>API: Documents

API->>LLM: Generate answer
LLM->>OTel: generation span

LLM-->>API: First token
API-->>UI: Stream token

UI->>T: response.first_token

LLM-->>API: Generation complete
API->>Q: response.completed

API-->>UI: Final response

UI->>T: response.rendered

T->>Q: Batch UI events
Q->>DB: Bulk insert

User->>UI: Click citation
UI->>T: citation.clicked

User->>UI: Click Like
UI->>T: response.liked

Notice something important:

The telemetry path never blocks the answer path.

🚫 18. Never Let Analytics Block Chat

This is one of the key engineering rules for Pensieve.

Bad:

flowchart LR

USER --> EVENT["Write Analytics"]
EVENT --> MONGO
MONGO --> CHAT["Call Chat API"]
CHAT --> USER

Good:

flowchart LR

USER --> CHAT["Chat Request"]
CHAT --> USER

USER -.-> EVENT["Telemetry Buffer"]
EVENT -.-> QUEUE["Async Queue"]
QUEUE -.-> MONGO

The user experience must not depend upon analytics persistence.

🪶 19. Browser-Side Batching

Do not make an HTTP request for every event.

A single answer could create:

citation.impression x 5
response.rendered
citation.clicked
response.copied
response.liked

Instead:

flowchart LR

E1["Event"]
E2["Event"]
E3["Event"]
E4["Event"]

E1 --> B["Browser Buffer"]
E2 --> B
E3 --> B
E4 --> B

B -->|"10 events OR 2 seconds"| API["/telemetry/events"]

A sensible browser policy might be:

Flush if:
    batch >= 10–25 events
OR
    2–5 seconds elapsed
OR
    visibility becomes hidden
OR
    page is exiting

Use:

fetch(..., { keepalive: true })

and where appropriate:

navigator.sendBeacon()

during page termination.

For truly important unsent events, IndexedDB can provide a small browser-side durable spool.

📦 20. JavaScript Packages

Browser

OpenTelemetry’s current browser documentation specifically includes document-load, XHR and user-interaction instrumentation. Browser instrumentation is still described as experimental/mostly unspecified, even though JavaScript traces and metrics themselves are stable.

That is another reason Pensieve should maintain a thin domain telemetry SDK above OTel.

⚙️ 21. Backend Packages

For Node.js:

OpenTelemetry provides Node and Web meta-packages, although its documentation notes that selecting individual instrumentations can reduce dependency footprint if you already know exactly what you need.

⚛️ 22. Instrumenting ShadCN Without Polluting the Components

Instead of modifying every ShadCN primitive, build semantic wrappers.

For example:

<TrackedButton
  telemetryId="answer-like"
  event="response.liked"
  context={{
    chatId,
    turnId,
    responseId
  }}
>
  <ThumbsUp />
</TrackedButton>

Internally:

function TrackedButton({
  telemetryId,
  event,
  context,
  children,
  ...props
}) {
  const telemetry = useTelemetry();

  return (
    <Button
      {...props}
      data-telemetry-id={telemetryId}
      onClick={(e) => {
        telemetry.emit(event, {
          component_id: telemetryId,
          ...context
        });

        props.onClick?.(e);
      }}
    >
      {children}
    </Button>
  );
}

The UI remains:

presentation
+
behavior
+
semantic telemetry

without hard-coding MongoDB or OpenTelemetry everywhere.

🧠 23. Create a Pensieve Telemetry Context

Conceptually:

type TelemetryContext = {
  visitorId: string;
  sessionId: string;
  tabId: string;

  chatId?: string;
  turnId?: string;
  requestId?: string;
  responseId?: string;

  traceId?: string;
};

Then components should not repeatedly discover these identifiers.

flowchart TD

ROOT["TelemetryProvider"]

ROOT --> NAV["Navigation"]
ROOT --> CHAT["Chat"]
ROOT --> RESPONSE["Response"]
ROOT --> CIT["Citations"]
ROOT --> FORM["Forms"]

CHAT --> TURN["Turn Context"]
TURN --> RESPONSE
RESPONSE --> CIT

This gives us hierarchical context propagation resembling distributed tracing itself.

📊 24. MongoDB Event Model

I would avoid creating one collection per event type.

Prefer one append-oriented event collection:

analytics_events

Example:

{
  "_id": "...",
  "event_id": "...",
  "event_name": "citation.clicked",
  "event_version": 1,

  "event_time": "...",
  "received_at": "...",

  "visitor_id": "...",
  "session_id": "...",
  "chat_id": "...",
  "turn_id": "...",
  "request_id": "...",
  "response_id": "...",

  "trace_id": "...",

  "component_id": "answer-citation",

  "properties": {
    "citation_id": "...",
    "document_id": "...",
    "rank": 2
  }
}

🗂️ 25. Recommended Collections

erDiagram

VISITOR ||--o{ SESSION : creates
SESSION ||--o{ CHAT : contains
CHAT ||--o{ TURN : contains
TURN ||--o{ REQUEST : generates
REQUEST ||--|| RESPONSE : produces
RESPONSE ||--o{ FEEDBACK : receives

VISITOR ||--o{ EVENT : generates
SESSION ||--o{ EVENT : groups
CHAT ||--o{ EVENT : correlates
TURN ||--o{ EVENT : correlates
REQUEST ||--o{ EVENT : correlates
RESPONSE ||--o{ EVENT : correlates

VISITOR {
    string visitor_id
    datetime first_seen
    datetime last_seen
}

SESSION {
    string session_id
    string visitor_id
    datetime started_at
    datetime last_activity
}

CHAT {
    string chat_id
    string session_id
    datetime created_at
}

TURN {
    string turn_id
    string chat_id
    string user_message_id
}

REQUEST {
    string request_id
    string turn_id
    string trace_id
    string model
}

RESPONSE {
    string response_id
    string request_id
    int generation_attempt
}

FEEDBACK {
    string feedback_id
    string response_id
    string rating
}

EVENT {
    string event_id
    string event_name
    int event_version
    datetime event_time
    string visitor_id
    string session_id
}

Conceptually I would have:

Do not duplicate everything unnecessarily.

analytics_events should remain the reconstructable behavioral ledger.

🗃️ 26. MongoDB Indexes

Likely high-value indexes include:

{ event_id: 1 }                 // unique
{ event_name: 1, event_time: -1 }
{ visitor_id: 1, event_time: -1 }
{ session_id: 1, event_time: 1 }
{ chat_id: 1, event_time: 1 }
{ response_id: 1, event_time: 1 }
{ trace_id: 1 }

For old low-value telemetry, TTL policies can automatically expire documents; MongoDB explicitly identifies machine-generated events, logs and session-like data as common TTL use cases.

A useful strategy might be:

Raw high-volume interaction telemetry     → 90–180 days
Conversation/product facts                → longer
Daily aggregate metrics                   → long term
Technical raw traces                      → shorter

⏱️ 27. Should We Use MongoDB Time-Series Collections?

Maybe—but selectively.

MongoDB time-series collections are optimized for high-volume time-oriented measurements and can reduce storage and I/O requirements.

They are excellent for:

latency measurements
collector health
token usage
hourly counters
system metrics

But for a behavioral ledger where we want:

unique event_id
rich variable properties
deduplication
occasional updates

a regular collection is often easier.

Therefore:

analytics_events             regular collection
service_metrics              time-series collection
daily_product_metrics        regular/materialized collection

is a reasonable design.

🚚 28. Queue Architecture

There are several maturity levels.

🪄 29. The Practical Starting Architecture

For Pensieve I would initially choose:

flowchart LR

B["Browser Event Buffer"]

B --> API["Telemetry API"]

API --> Q["Bounded Backend Queue"]

Q --> W["Batch Writer"]

W -->|"insertMany / bulkWrite"| M[("MongoDB")]

MongoDB’s Node driver supports insertMany() and bulkWrite(), and bulk operations can reduce network round trips.

Flush perhaps:

100–500 events
OR
100–500 ms

depending upon volume.

🏭 30. When Pensieve Gets Bigger

Then evolve to:

flowchart LR

UI["React"]

UI --> API["Telemetry Ingest"]

API --> K["Kafka / Redis Stream"]

K --> W1["Mongo Writer"]
K --> W2["Realtime Metrics"]
K --> W3["Quality Evaluation"]
K --> W4["Experiment Pipeline"]

W1 --> M["MongoDB"]
W2 --> D["Dashboards"]
W3 --> Q["Evaluation Store"]
W4 --> X["Experiment Mart"]

This is where an event bus becomes strategically useful:

one event → multiple consumers.

🔌 31. OpenTelemetry Collector + MongoDB

There is an important current ecosystem nuance.

As of 2026, the standard OpenTelemetry Collector contrib distribution does not provide a native MongoDB exporter for arbitrary traces/logs/metrics; a proposed MongoDB exporter remained an outstanding/unsupported capability rather than a standard path.

Therefore I would not design:

OTel Collector → MongoDB

as though it were a turnkey standard exporter.

A scalable route is instead:

flowchart LR

OTEL["OTel Collector"]

OTEL --> K["Kafka"]

K --> CONNECT["MongoDB Kafka Sink Connector"]

CONNECT --> M[("MongoDB")]

The official MongoDB Kafka Connector supports consuming Kafka topics as a sink and persisting them into MongoDB.

But unless Pensieve already operates Kafka, introducing Kafka only to move telemetry into Mongo may be needless complexity.

For most implementations:

Product events → MongoDB

OTel signals → observability backend

is cleaner.

🔭 32. What Should OpenTelemetry Capture?

Traces

browser interaction
   ↓
POST /chat
   ↓
authentication
   ↓
query rewrite
   ↓
retrieval
   ↓
reranking
   ↓
context construction
   ↓
LLM
   ↓
stream
   ↓
response persistence

Metrics

Examples:

chat_requests_total
chat_failures_total
retrieval_duration
llm_duration
time_to_first_token
response_duration
tokens_input
tokens_output
retrieval_document_count
event_queue_depth
event_drop_count
mongo_write_duration

Logs

Examples:

generation timeout
retrieval exception
malformed citation
queue overflow
Mongo write failure

⚠️ 33. Never Put User IDs Into Metric Labels

Imagine:

chat_requests_total{
   visitor_id="..."
}

That creates enormous cardinality.

Instead:

Metrics
    → environment
    → service
    → model
    → endpoint
    → result

Events / traces
    → request_id
    → session_id
    → response_id

Keep business identifiers in queryable event records, not dimensions of continuously aggregated infrastructure metrics.

🧵 34. Trace Propagation

Browser requests should propagate:

traceparent
tracestate

so the trace can cross:

flowchart LR

B["Browser"]
A["API Gateway"]
C["Chat Service"]
R["Retriever"]
L["LLM Client"]
D["Database"]

B --> A --> C
C --> R
C --> L
C --> D

B -. same trace .-> D

OpenTelemetry recommends connecting client telemetry to backend traces through propagated trace context.

But:

trace_id does not replace request_id.

A business request identifier must remain meaningful regardless of telemetry sampling or tracing implementation.

📡 35. Do Not Expose the Collector Carelessly

Browser-based OTLP has several constraints.

Browsers cannot use the normal gRPC exporter directly; OTLP/HTTP is used, and CSP/CORS configuration matters. OpenTelemetry also cautions against simply exposing a Collector directly to the public Internet and recommends placing an appropriate proxy/front door in front of it.

Prefer:

flowchart LR

B["Browser"]

B --> EDGE["Same-Origin<br/>Telemetry Endpoint"]

EDGE --> COL["OTel Collector"]

COL --> BACK["Observability Backend"]

instead of:

Browser
   ↓
public unauthenticated collector

🛡️ 36. Privacy by Design

AI telemetry is unusually dangerous because it is tempting to log:

prompt
conversation
retrieved document
generated answer
user information

everywhere.

Do not.

Create explicit telemetry levels.

The OpenTelemetry GenAI conventions specifically warn that prompt/message/retrieval content can contain sensitive or personally identifiable information.

📉 37. Sampling: Analytics and Tracing Need Different Policies

Suppose Pensieve gets:

10,000,000 traces/day

We may sample technical traces.

For example:

normal requests           5–10%
errors                    100%
slow traces               100%
evaluation failures       100%

The JS SDK supports ratio-based trace sampling.

But do not apply trace sampling blindly to:

question.submitted
response.completed
response.liked
response.disliked
citation.clicked
form.submitted

Those are product facts.

📐 38. Now We Can Calculate Product Metrics

Once this foundation exists, Pensieve becomes analytically powerful.

🔻 39. TOFU → MOFU → BOFU for an AI Assistant

Traditional marketing terminology can be adapted into an AI product journey.

flowchart TD

T["🌍 TOFU<br/>Discovery"]

M["🧠 MOFU<br/>Value Exploration"]

B["🎯 BOFU<br/>Resolution / Action"]

T -->|"Opens chat"| M
M -->|"Gets sufficient value"| B

T --> T1["Visitor"]
T --> T2["Page View"]
T --> T3["Chat Open"]

M --> M1["Question"]
M --> M2["Answer"]
M --> M3["Follow-up"]
M --> M4["Citation"]
M --> M5["Copy"]

B --> B1["Positive Feedback"]
B --> B2["Form Click"]
B --> B3["Form Completion"]
B --> B4["Task Completed"]

🌍 40. TOFU Metrics — Discovery & Activation

TOFU tells us whether users are discovering Pencieve and crossing the first threshold into actual usage.

The important transition is:

flowchart LR

VISIT["🌍 Visitor"]
--> CHAT["💬 Opens Chat"]
--> QUESTION["❓ Submits Question"]
--> ACTIVATED["✅ Activated User"]

🧠 41. MOFU Metrics — Engagement & Value Exploration

MOFU asks whether the user begins receiving enough value to continue interacting with the system.

🎯 42. BOFU Metrics — Resolution & Business Action

For Pencieve, BOFU is not necessarily a purchase.

It is the moment where an AI answer enables the user to complete the task they originally came for.

👍 43. Feedback & Answer Quality Metrics

One important distinction is that rating participation and rating sentiment are different metrics.

Why the denominator matters

For example:

These are three different questions and should not be mixed.

🔄 44. Regeneration Metrics

Regeneration is particularly interesting because it can be both a quality failure signal and a legitimate exploration behavior.

Possible causes:

📋 45. Copy & Reuse Metrics

For knowledge assistants, copy behavior can be a stronger signal than explicit Likes.

🔗 46. Citation & Grounding Metrics

Citation analytics give Pencieve something traditional chat analytics cannot provide:

visibility into whether users actually trust and inspect the evidence behind an answer.

Track rank independently:

This can directly influence future retrieval and reranking optimization.

📅 47. Retention Metrics

visitor_id now becomes essential because sessions alone cannot determine whether somebody returned.

Cohort View

Cohorts allow us to ask:

Is the product becoming more useful for newly acquired users over time?

🧲 48. Stickiness Metrics

For knowledge assistants, daily use is not always the ideal benchmark.

A user might legitimately need Pencieve only twice a week.

For an enterprise application, WAU/MAU may tell us considerably more than DAU/MAU.

🔁 49. Returning-User Metrics

Returning users deserve their own analysis because retention alone can obscure depth.

💬 50. Engagement Metrics

🧠 51. Defining a Meaningful Session

Simple page views inflate engagement.

Instead Pencieve can define:

Therefore:

Meaningful Session = Completed Answer + At Least One Value Signal

And:

This becomes much more useful than measuring raw session duration alone.

⭐ 52. Pencieve North-Star Metric

Weekly Successful Answer Users — WSAU

A user qualifies when:

Possible Value Signals

Then WSAU becomes the outcome of several upstream systems:

Instead of optimizing:

How many questions did Pencieve answer?

we optimize:

How many people successfully accomplished something using Pencieve this week?

⚡ 53. Performance Metrics

Streaming AI systems require more than a single response-time number.

For every important latency measure report:

Averages alone hide tail pain.

🧮 54. AI Cost Metrics

Once token, model, request and outcome telemetry share the same identifiers, economics become measurable.

The question changes from:

“Which model has the best benchmark score?”

to:

Which architecture produces the greatest number of successful outcomes per dollar?

🔬 55. Product Evaluation Framework

Pencieve can be evaluated across interconnected product layers.

🔍 56. Product Analytics + Trace Analytics

This is where Pencieve’s instrumentation architecture becomes especially powerful.

Example investigation:

The important discovery becomes:

The LLM didn’t fail. Retrieval did.

Without correlation IDs, those two worlds would never meet.

📏 57. Instrumentation Quality Metrics

Instrumentation itself is production infrastructure and must therefore also be observable.

Event Delivery Rate

Otherwise a dashboard decline could mean either:

Those are completely different situations.

🧯 58. Queue Backpressure & Event Priority

Not every telemetry event deserves the same durability.

Queue policy:

🧪 59. Event Schema Versioning

Schemas will evolve.

Treat event definitions as contracts.

Example evolution:

The rule is:

Never silently redefine the meaning of an existing event.

🧪 60. Instrumentation Testing Matrix

Analytics code should be tested with the same discipline as application code.

🚧 61. Instrumentation Anti-Patterns

🪜 62. Pencieve Implementation Roadmap

Rather than introducing hundreds of events immediately, build the system layer-by-layer.

Phase 2 — Minimum Recommended Events

🏁 63. The Pencieve Mental Model

Pencieve’s instrumentation system can ultimately be understood through seven layers.

And the identifier hierarchy answers complementary questions:

🌌 64. From Telemetry to Product Memory

The final outcome is not simply a larger analytics dashboard.

The loop becomes:

flowchart LR

OBSERVE["👀 Observe"]
--> UNDERSTAND["🧠 Understand"]
--> EVALUATE["📊 Evaluate"]
--> IMPROVE["🛠 Improve"]
--> MEASURE["📈 Measure Again"]
--> OBSERVE

This is the larger vision of Pencieve:

Instrumentation becomes the product’s memory of how humans interact with intelligence.

Pencieve remembers not merely what it answered.

It remembers whether the answer was:

And that is what transforms instrumentation from a collection of frontend events into the nervous system of an intelligent product.

🧰 65. Tool Selection: Do We Really Need to Build All of This Ourselves?

Up to this point, we have designed:

• the event taxonomy,

• identity hierarchy,

• OpenTelemetry correlation model,

• MongoDB event store,

• product metrics,

• funnels,

• feedback signals,

• RAG instrumentation,

• and telemetry pipelines.

The next obvious question becomes:

Do we really need to build every instrumentation capability ourselves?

Fortunately, no.

The JavaScript ecosystem already contains several libraries capable of giving us a familiar analytics interface such as:

analytics.page(...)
analytics.track(...)
analytics.identify(...)
analytics.reset(...)

very similar conceptually to:

Google Analytics
Segment
Snowplow
Mixpanel
Amplitude

But there is an important architectural decision.

We don’t necessarily want Pencieve’s React components to know that we use:

Snowplow
PostHog
MongoDB
ClickHouse
OpenTelemetry
Jitsu

Instead, we want:

flowchart LR

UI["⚛️ React / ShadCN"]

SDK["🧠 Pencieve Telemetry SDK"]

UI --> SDK

SDK --> PRODUCT["📊 Product Events"]
SDK --> OTEL["🔭 OpenTelemetry"]
SDK --> EXP["🧪 Experiments"]

PRODUCT --> API["Event API"]

API --> STORE["Event Store"]

OTEL --> COL["OTel Collector"]

EXP --> FLAGS["Feature Flags / Experiments"]

The UI understands only one interface:

telemetry.track("response.liked", {...})

Everything behind that interface can change.

This turns telemetry into an architectural capability, rather than a vendor dependency.

🧱 66. There Are Actually Several Different Tooling Problems

One of the reasons analytics implementations become confusing is that teams compare tools that solve completely different problems.

Snowplow, OpenTelemetry, PostHog, MongoDB and ClickHouse are not interchangeable.

They occupy different layers.

The mistake would be trying to find:

one product that does all thirteen perfectly.

Instead Pencieve should construct a composable analytics stack.

🥇 67. Open-Source JavaScript Instrumentation Options

The first requirement is deceptively simple:

Can we instrument our existing React + ShadCN UI similarly to Google Analytics without paying a vendor license?

Yes.

There are several options.

A particularly interesting lightweight option is the analytics npm package. It exposes track, page, identify, plugin lifecycle hooks, offline behavior and provider plugins while explicitly supporting your own backend. The package is MIT licensed.

This is unusually well aligned with Pencieve because we don’t actually need another analytics vendor.

We need a clean frontend abstraction.

🧠 68. My Preferred Pencieve Architecture

Instead of:

flowchart LR

BUTTON["Button"]
--> POSTHOG["PostHog"]

BUTTON --> OTEL["OTel"]

BUTTON --> MONGO["Mongo"]

BUTTON --> GA["GA"]

BUTTON --> LOG["Logger"]

which creates vendor-specific instrumentation everywhere, build:

flowchart LR

COMP["ShadCN Component"]

COMP --> SDK["Pencieve Telemetry SDK"]

SDK --> VALIDATE["Schema Validation"]

VALIDATE --> BUFFER["Event Buffer"]

BUFFER --> API["Pencieve Event API"]

API --> QUEUE["Queue"]

QUEUE --> MONGO["MongoDB Raw Events"]

SDK -. technical telemetry .-> OTEL["OpenTelemetry"]

MONGO --> MARTS["Product Marts"]

MARTS --> DASH["Dashboards"]

The frontend sees:

telemetry.track(...)
telemetry.page(...)
telemetry.identify(...)
telemetry.impression(...)
telemetry.error(...)

not:

posthog.capture(...)
snowplow(...)
rudderanalytics.track(...)
mongo.insert(...)

This gives us:

📦 69. Using the analytics Library as the Pencieve Frontend Kernel

One attractive option is:

npm install analytics

The library already gives us:

analytics.track()
analytics.page()
analytics.identify()
analytics.reset()

and a plugin system.

It is specifically designed to work with either third-party analytics services or your own backend.

Therefore Pencieve could create a custom plugin:

const pencievePlugin = {
  name: "pencieve",

  track: ({ payload }) => {
    eventBuffer.enqueue(
      buildPencieveEvent(
        payload.event,
        payload.properties
      )
    );
  },

  page: ({ payload }) => {
    eventBuffer.enqueue(
      buildPencieveEvent(
        "page.viewed",
        payload
      )
    );
  },

  identify: ({ payload }) => {
    identityManager.identify(
      payload.userId
    );
  }
};

Then initialize:

import Analytics from "analytics";

export const analytics = Analytics({
  app: "pencieve",
  version: APP_VERSION,

  plugins: [
    pencievePlugin
  ]
});

React code remains extremely small:

analytics.track(
  "response.liked",
  {
    response_id: responseId
  }
);

But our plugin can expand this into the complete standard envelope:

{
  "event_name": "response.liked",
  "event_version": 1,

  "event_id": "evt_...",
  "event_time": "...",

  "visitor_id": "...",
  "session_id": "...",
  "tab_id": "...",

  "chat_id": "...",
  "turn_id": "...",
  "request_id": "...",
  "response_id": "...",

  "trace_id": "...",

  "application_version": "...",

  "properties": {}
}

That is exactly the separation we want.

🧬 70. Keep a Pencieve API Above the Analytics Library

I would go one step further.

Do not even expose the generic library directly to the application.

Instead create:

pencieve.track()

which internally calls whichever library we choose.

flowchart TB

APP["Application"]

SDK["Pencieve SDK"]

ANALYTICS["analytics library"]

PLUGIN["Pencieve Plugin"]

API["Event API"]

APP --> SDK
SDK --> ANALYTICS
ANALYTICS --> PLUGIN
PLUGIN --> API

For example:

export interface PencieveTelemetry {

  track<T>(
    event: EventName,
    properties?: T
  ): void;

  page(
    page: PageContext
  ): void;

  identify(
    userId: string
  ): void;

  impression(
    component: ComponentContext
  ): void;
}

Then:

telemetry.track(
  "citation.clicked",
  {
    citation_id,
    document_id,
    rank
  }
);

This prevents even the analytics package from becoming part of our domain API.

That distinction becomes very valuable five years later.

🧾 71. Schema Validation: Recreating the Best Part of Iglu

The strongest concept from Snowplow/Iglu was never merely event collection.

It was this:

Events are contracts.

Snowplow’s schema model associates events with self-describing JSON schemas and validates them against those schemas before they become trusted behavioral data. Iglu acts as the registry containing those schemas.

For Pencieve we can reproduce the same philosophy using completely independent technologies.

flowchart LR

EVENT["Event"]

SCHEMA["JSON Schema"]

VALID["Validator"]

EVENT --> VALID
SCHEMA --> VALID

VALID -->|"Valid"| PIPE["Event Pipeline"]
VALID -->|"Invalid"| DLQ["Dead Letter / Reject"]

A schema could look like:

{
  "$id": "pencieve/response-liked/1-0-0",

  "type": "object",

  "required": [
    "event_id",
    "event_name",
    "session_id",
    "response_id"
  ],

  "properties": {

    "event_name": {
      "const": "response.liked"
    },

    "response_id": {
      "type": "string"
    },

    "feedback_source": {
      "enum": [
        "inline",
        "feedback_dialog"
      ]
    }
  },

  "additionalProperties": false
}

📚 72. Do We Still Need Iglu?

Conceptually: yes.

Technically: not necessarily.

There is an important 2026 licensing nuance.

Snowplow’s JavaScript tracker remains BSD-3-Clause licensed, and Snowplow trackers continue to have permissive open-source licenses. However, newer core pipeline applications and current Iglu Server releases moved under Snowplow’s Limited Use License. Snowplow’s own licensing documentation distinguishes the tracker SDKs from the newer pipeline apps.

Therefore Pencieve could choose:

Option A — Git-Based JSON Schema Registry

/events
    /response.liked
        /1-0-0.json

    /response.disliked
        /1-0-0.json

    /citation.clicked
        /1-0-0.json

Validation:

Ajv
+
JSON Schema
+
Git
+
CI validation

This is probably sufficient initially.

Option B — Apicurio Registry

If we want a true schema registry, Apicurio Registry is a compelling modern alternative.

It supports:

• JSON Schema,

• Avro,

• Protobuf,

• AsyncAPI,

• OpenAPI,

• compatibility rules,

• artifact versioning,

• schema evolution,

• REST APIs,

and is Apache-2.0 licensed.

That gives Pencieve:

flowchart LR

DEV["Developer"]

REG["Apicurio Registry"]

CI["CI/CD"]

APP["Pencieve"]

COL["Collector"]

DEV --> REG
REG --> CI

REG --> APP
REG --> COL

For a larger organization, this is probably the closest architectural replacement for the Iglu experience while remaining on a permissive open-source base.

🛡️ 73. Zod vs Ajv vs Schema Registry

These tools solve slightly different problems.

A nice Pencieve pattern is:

flowchart LR

TS["TypeScript"]

ZOD["Zod"]

JSON["JSON Schema"]

REG["Registry"]

AJV["Ajv"]

TS --> ZOD
ZOD --> JSON
JSON --> REG
REG --> AJV
AJV --> INGEST["Ingestion API"]

Or simply maintain JSON Schema as the canonical definition and generate types.

The important thing is not which validation library wins.

The important thing is:

event contracts stop being tribal knowledge.

🛣️ 74. Jitsu: An Interesting Open-Source Segment-Style Alternative

Jitsu deserves particular attention for Pencieve.

Its API follows the familiar Analytics.js model:

jitsu.page()
jitsu.identify()
jitsu.track()
jitsu.group()

and the JavaScript SDK is available as:

@jitsu/js

Jitsu’s API is explicitly based on Analytics.js conventions.

More importantly, Jitsu’s current open-source architecture is MIT licensed and self-hostable. Its architecture uses:

Go ingestion
        ↓
Kafka / Redpanda
        ↓
TypeScript routing
        ↓
Warehouse bulk loader

with

ClickHouse → live events / metrics
MongoDB   → profiles
Postgres  → configuration

according to its current repository documentation.

That architecture is itself educational for Pencieve.

flowchart LR

JS["Browser SDK"]

INGEST["Event Ingest"]

STREAM["Kafka / Redpanda"]

ROUTER["Router"]

CH["ClickHouse"]

PROFILE["Mongo Profiles"]

WAREHOUSE["Warehouse"]

JS --> INGEST --> STREAM --> ROUTER

ROUTER --> CH
ROUTER --> PROFILE
ROUTER --> WAREHOUSE

Jitsu becomes attractive if Pencieve eventually needs:

• multiple destinations,

• warehouse delivery,

• event transformations,

• reusable ingestion infrastructure,

• Kafka/Redpanda,

• and analytics beyond MongoDB.

For a single application, however, this may be more infrastructure than necessary.

🪶 75. RudderStack

RudderStack provides an excellent SDK experience.

Its JavaScript SDK supports familiar analytics semantics and is MIT licensed.

It can act as a routing layer:

flowchart LR

APP["Pencieve"]

R["RudderStack"]

GA["GA"]
WH["Warehouse"]
PH["PostHog"]
CUSTOM["Custom Destination"]

APP --> R

R --> GA
R --> WH
R --> PH
R --> CUSTOM

Its current server licensing deserves attention, however.

The rudder-server repository currently uses Elastic License 2.0, which permits internal use but includes restrictions such as offering the software as a hosted/managed service.

Therefore:

For a corporate internal application this may still be perfectly acceptable.

But it is not the same thing as an MIT or Apache-2 stack.

🦔 76. PostHog

PostHog provides perhaps the richest product analytics experience among these tools.

It supports:

• custom events,

• autocapture,

• pageviews,

• identity,

• funnels,

• paths,

• retention,

• cohorts,

• experimentation,

• session replay.

Its frontend autocapture can automatically detect interactions with elements including buttons, links, forms and standard form controls.

That sounds perfect.

But there is an important distinction:

PostHog can be a useful analytics consumer, but I would hesitate to make PostHog’s event format the Pencieve domain contract.

Why?

Because Pencieve has richer requirements:

session_id
chat_id
turn_id
request_id
response_id
trace_id
retrieval metadata
citation rank
model information
experiment context

Keep those contracts independent.

Also, PostHog currently documents self-hosting as available but officially unsupported, with a free Docker Compose deployment rather than a supported production offering.

So PostHog is excellent for:

exploration
funnels
retention
product dashboards
experimentation

but it does not need to own the canonical event model.

🍵 77. Umami, Matomo and Plausible

These tools are attractive when the requirement is closer to:

“Give me an open-source Google Analytics replacement.”

rather than:

“Build a deeply instrumented agentic product telemetry platform.”

Umami

Umami is MIT licensed and supports:

umami.track(...)
umami.identify(...)

as well as declarative HTML attributes:

<button
  data-umami-event="response-like"
>

Custom event properties are also supported.

This makes Umami excellent for simple website analytics.

Matomo

Matomo remains a mature self-hostable analytics platform under GPLv3 and offers both JavaScript event tracking and Tag Manager-based instrumentation.

It is especially useful for organizations wanting a traditional analytics experience.

Plausible

Plausible Community Edition is self-hostable and AGPLv3 licensed; its browser tracker is separately MIT licensed. It focuses strongly on privacy-oriented web analytics, goals and conversions.

Fit Comparison

🎥 78. Session Replay Is a Different Signal

Sometimes:

response.disliked

tells us what happened.

But not why.

Suppose we see:

citation.clicked
citation.failed
response.disliked

Was the answer wrong?

Or was the citation link obscured by a UI overlay?

That is where session replay becomes valuable.

OpenReplay is an open-source self-hosted experience platform providing:

• session replay,

• technical debugging context,

• product analytics,

• co-browsing,

• privacy masking.

Most of its codebase is AGPLv3 with some MIT and enterprise-licensed areas.

Pencieve could therefore correlate:

session_id
        ↓
product events
        ↓
trace
        ↓
session replay

giving us four perspectives:

That becomes extremely powerful during debugging.

🧪 79. Experimentation: GrowthBook

Once Pencieve has reliable instrumentation, the natural next stage is experimentation.

Examples:

Does showing citations inline increase trust?

Does a shorter answer reduce regeneration?

Does Model B improve successful answer rate?

Does suggested follow-up UI increase engagement?

Does streaming improve perceived latency?

GrowthBook provides open-source experimentation and feature flagging capabilities; much of the project is MIT licensed, while enterprise components use a separate commercial license.

Add experiment context to every Pencieve event:

{
  "experiment": {
    "id": "citation-layout-v2",
    "variant": "B"
  }
}

Then calculate:

[
Lift =
Metric_B - Metric_A
]

for metrics such as:

Positive Rating Rate
Citation CTR
Regeneration Rate
Meaningful Session Rate
Answer-to-Action Conversion

🎯 80. Autocapture vs Semantic Instrumentation

One of the largest architecture choices is:

Should Pencieve automatically capture every button click?

Tools such as PostHog make that easy.

But easy capture does not automatically create useful analytics.

Consider:

<button>
  👍
</button>

Autocapture may produce:

button_clicked

But Pencieve needs:

response.liked

with:

response_id
turn_id
request_id
chat_id
model_version
prompt_version
experiment_variant

These are not equivalent.

🧠 81. The Three Levels of Instrumentation

I would use three instrumentation levels.

flowchart TD

AUTO["Automatic"]

SEMIAUTO["Semi-Automatic"]

SEMANTIC["Semantic"]

AUTO --> TECH["Technical Observability"]
SEMIAUTO --> UX["UX Analytics"]
SEMANTIC --> PRODUCT["Product Metrics"]

For Pencieve:

L3 semantic events are authoritative.

Autocapture is supplementary.

⚛️ 82. Instrumenting ShadCN Components Cleanly

We do not want this:

<Button
  onClick={() => {
    posthog.capture(...);
    snowplow(...);
    fetch("/events", ...);
    likeResponse();
  }}
>

Instead build reusable telemetry-aware primitives.

TrackedButton

<TrackedButton
  telemetry={{
    event: "response.liked",

    properties: {
      response_id: responseId
    }
  }}

  onClick={likeResponse}
>
  <ThumbsUp />
</TrackedButton>

Internally:

function TrackedButton({
  telemetry,
  onClick,
  ...props
}) {

  const { track } =
    usePencieveTelemetry();

  return (
    <Button
      {...props}

      onClick={(event) => {

        track(
          telemetry.event,
          telemetry.properties
        );

        onClick?.(event);
      }}
    />
  );
}

🧩 83. Instrumentation Can Also Be Declarative

Another pattern is:

<Button

  data-telemetry-event="response.liked"

  data-response-id={responseId}

>

A delegated event listener at the application root can listen for:

click
submit
change

and inspect:

data-telemetry-event

This resembles the declarative approach offered by tools such as Umami, which can track custom element events through data-* attributes.

⚖️ 84. Wrapper vs Hook vs Data Attribute

Use all four selectively.

My preference:

Business-critical event
        → explicit hook/wrapper

Generic UI action
        → data attribute

Diagnostic discovery
        → autocapture

Technical operation
        → OpenTelemetry

🧬 85. Automatically Attach Context

Developers should not repeatedly write:

telemetry.track(
  "response.liked",
  {
    visitor_id,
    session_id,
    chat_id,
    turn_id,
    request_id,
    response_id,
    application_version,
    route,
    browser,
    trace_id
  }
);

That guarantees inconsistency.

Instead:

telemetry.track(
  "response.liked",
  {
    feedback_source: "inline"
  }
);

The SDK enriches automatically:

flowchart LR

EVENT["response.liked"]

IDENTITY["Identity Context"]

CHAT["Chat Context"]

TRACE["Trace Context"]

APP["App Context"]

EXP["Experiment Context"]

EVENT --> BUILD["Event Builder"]

IDENTITY --> BUILD
CHAT --> BUILD
TRACE --> BUILD
APP --> BUILD
EXP --> BUILD

BUILD --> FINAL["Canonical Pencieve Event"]

This dramatically reduces frontend instrumentation burden.

🪶 86. Browser Performance: Keep the SDK Tiny

Telemetry should not become a new performance problem.

The ideal browser path is:

flowchart LR

CLICK["User Action"]

MEM["Memory Buffer"]

BATCH["Batch"]

HTTP["Single HTTP Request"]

CLICK --> MEM --> BATCH --> HTTP

Not:

flowchart LR

CLICK["Click"]

HTTP1["Request"]

HTTP2["Request"]

HTTP3["Request"]

HTTP4["Request"]

CLICK --> HTTP1
CLICK --> HTTP2
CLICK --> HTTP3
CLICK --> HTTP4

Recommended design:

🚌 87. Collection Layer Selection

Once the browser emits a canonical event, what receives it?

We have several choices.

For Pencieve I would start with:

React
 ↓
Pencieve SDK
 ↓
/api/events
 ↓
validation
 ↓
queue
 ↓
Mongo