Employee Event Streams

Summary

This chapter explores the rich sources of organizational data hidden in everyday digital tools. Students learn about event logs from email, chat, devices, calendars, and business processes. The chapter covers how to capture, timestamp, normalize, and enrich these events to prepare them for graph-based analysis, including an introduction to business process mining.

Concepts Covered

This chapter covers the following 17 concepts from the learning graph:

1. Employee Event Streams
2. Event Logs
3. Universal Timestamps
4. Event Normalization
5. Event Enrichment
6. Email Event Streams
7. Chat Event Streams
8. Device Activity Logs
9. Desktop Activity
10. Mobile Device Events
11. Software Application Logs
12. Calendar Events
13. Meeting Patterns
14. Login and Logout Events
15. Business Process Mining
16. Process Discovery
17. Process Conformance

Prerequisites

This chapter builds on concepts from:

• Chapter 1: Introduction to Organizational Analytics

---

Following the Pheromone Trail

"Every interaction leaves a trace. In my colony, it's a pheromone trail. In your organization, it's an event log. Follow the trail — the data always leads somewhere." — Aria

Let's dig into this! In Chapter 1, you learned that organizational analytics goes beyond the org chart to reveal how work actually happens. In Chapter 2, you explored the graph data structures that will hold all those rich insights. But nodes and edges don't materialize out of thin air. Before you can build a graph of your organization, you need raw material — the digital footprints that people leave behind as they go about their daily work.

That raw material is what we call employee event streams, and your organization is already generating them by the millions. Every email sent, every chat message typed, every meeting accepted, every login recorded — each one is a discrete, timestamped event that tells a small part of a much larger story. Taken individually, a single event is unremarkable. Taken together, these streams of events reveal the living, breathing communication network that makes your organization function.

Think of it this way: if the graph database is the map, event streams are the surveyor's field notes. This chapter teaches you how to collect those notes, make sense of them, and prepare them for the graph loading that comes in Chapter 4.

What Is an Employee Event Stream?

An employee event stream is a chronological sequence of discrete actions or interactions generated by an employee as they use organizational tools and systems. Each event in the stream captures a single moment — a message sent, a file opened, a badge swiped, a meeting started — along with metadata that describes the who, what, when, and where of that action.

The key properties that distinguish event streams from static HR records:

• Temporal — Every event has a timestamp; order matters
• Continuous — Events are generated constantly, creating an ongoing flow of data
• High-volume — A single employee can generate hundreds of events per day
• Multi-source — Events come from many different systems (email, chat, calendar, devices)
• Relational — Most events involve connections between people, or between people and organizational artifacts

In an ant colony, you'd call these pheromone trails — chemical signals deposited at specific times and places that collectively encode the colony's communication patterns. The parallel is striking: just as an entomologist can reconstruct a colony's foraging routes by tracing pheromone deposits, an organizational analyst can reconstruct communication networks by tracing event streams.

Here's a sample of what one employee's event stream might look like over a single morning:

Time	Source System	Event Type	Details
08:01	Badge System	Building entry	Front entrance, Badge #4471
08:04	Laptop	Login	Windows authentication
08:05	Email	Receive	From: j.park@company.com, Subject: "Q3 roadmap"
08:12	Email	Send	To: a.patel@company.com, Subject: "Re: Sprint review"
08:15	Slack	Message sent	Channel: #engineering, 42 characters
08:30	Calendar	Meeting start	"Daily standup", 6 attendees, Room 301
08:45	Calendar	Meeting end	"Daily standup", duration: 15 min
08:47	Jira	Ticket update	PROJ-1234, status: In Progress
09:02	Slack	Direct message	To: c.rivera@company.com, 118 characters
09:15	Email	Send	To: l.wei@company.com, CC: j.park@company.com

Ten events in about seventy-five minutes. Multiply that across a full workday, then across hundreds or thousands of employees, and you begin to see the scale of data available. A mid-sized organization of 5,000 employees can easily generate two to five million events per day.

Event Logs: The Foundation

An event log is the structured record that captures event stream data. While the event stream is the conceptual flow, the event log is the concrete, stored artifact — typically a file, database table, or message queue entry that contains the event data in a processable format.

Every well-formed event log entry contains at least these fields:

• Timestamp — When the event occurred
• Actor — Who performed the action (usually an employee identifier)
• Action — What happened (send, receive, login, join, update)
• Target — What or who the action was directed at
• Source system — Which tool or platform generated the event
• Event ID — A unique identifier for deduplication and tracing

Many event logs also include additional context: IP addresses, device identifiers, session IDs, content length, channel names, or attachment counts. This metadata becomes valuable during the enrichment phase we'll cover later in this chapter.

Diagram: Event Log Anatomy

Event Log Anatomy

Type: infographic

Bloom Taxonomy: Understand (L2) Bloom Verb: describe Learning Objective: Students will describe the core fields of an event log entry and explain why each is necessary for organizational analytics.

Purpose: Visualize the anatomy of a single event log record, highlighting required and optional fields.

Layout: A single large "card" representing one event log entry, with labeled fields arranged vertically. Required fields (timestamp, actor, action, target, source system, event ID) shown in indigo (#303F9F) with a solid border. Optional metadata fields (IP address, device ID, session ID, content length, channel, attachments) shown in amber (#D4880F) with a dashed border.

Interactive elements:

• Hover over any field to see a tooltip explaining its purpose and an example value
• Toggle button to switch between "Email Event" and "Chat Event" examples, showing how the same structure applies to different sources

Visual style: Clean card layout with Aria color scheme. Field names in bold, example values in monospace font.

Responsive design: Card scales to container width; on narrow screens, fields stack single-column.

Implementation: p5.js with canvas-based rendering and hover detection

Universal Timestamps: Making Time Consistent

When event data arrives from a dozen different systems, one of the first problems you'll encounter is time. Email servers record time in UTC. Chat platforms might use the user's local timezone. Badge systems log in the building's timezone. Calendar applications store times in the organizer's timezone but display them in each attendee's local timezone.

If you don't resolve these differences, your event streams become unreliable. Did Maria send that email before or after the meeting started? If the email server uses UTC and the calendar uses Eastern Time, you might get the wrong answer — and in organizational analytics, sequence matters enormously.

Universal timestamps solve this by converting all event times to a single, unambiguous format. The standard choice is ISO 8601 in UTC (Coordinated Universal Time):

2026-03-15T14:32:07Z

The T separates date from time, and the trailing Z indicates UTC (sometimes called "Zulu time"). Every event, regardless of its source system, gets converted to this format during ingestion. This ensures that when you sort events chronologically or calculate the time gap between two interactions, the results are accurate.

Aria's Insight

Here's a mistake I see all the time: analysts skip the timestamp normalization step because "everything looks fine" during development with a small dataset. Then they go to production with users across five time zones, and suddenly meetings appear to end before they start. Always, always normalize your timestamps to UTC before doing anything else. Trust me — I once mapped my colony's shift changes using three different sundials, and the results were... let's just say the night shift got very confused.

Key considerations for timestamp handling:

• Precision — Some systems log to the second, others to the millisecond. Standardize on millisecond precision when possible.
• Timezone metadata — Store the original timezone alongside the UTC conversion so you can reconstruct local time for reporting.
• Clock drift — Physical devices (badge readers, IoT sensors) may have clocks that drift. Account for synchronization errors.
• Daylight saving time — UTC doesn't observe DST, which is precisely why it's the right choice for a universal standard.

The Stream Types: Where Events Come From

Now that you understand what event streams are and how to timestamp them consistently, let's explore the major categories of organizational event data. Each source system generates its own type of stream with unique characteristics, volumes, and analytical value.

Email Event Streams

Email event streams are among the richest sources of organizational communication data. Every email generates multiple events: the sender creates a SEND event, each recipient generates a RECEIVE event, and subsequent actions like REPLY, FORWARD, and OPEN create additional events.

Critically, organizational analytics works with email metadata, not message content. You don't need to read anyone's emails to extract powerful insights. The metadata alone — sender, recipients, CC/BCC lists, timestamps, subject line, attachment count, thread ID — reveals communication patterns, frequency, and network structure.

A single email might generate this set of event records:

Field	Value
Event ID	EMAIL-2026-0315-0847-A1
Timestamp	2026-03-15T13:47:22Z
Actor	m.chen@company.com
Action	SEND
Recipients	a.patel@company.com, l.wei@company.com
CC	j.park@company.com
Subject Hash	SHA256(subject)
Thread ID	THR-88421
Attachment Count	2
Size (bytes)	34,200

Notice the subject hash rather than the raw subject line. Hashing the subject preserves the ability to detect email threads (same subject = same hash) without storing potentially sensitive content. This is a common privacy-preserving technique in organizational analytics.

Email event streams are particularly valuable for:

• Mapping communication networks — Who emails whom, and how often?
• Detecting cross-departmental bridges — Which employees connect otherwise siloed teams?
• Identifying response patterns — How quickly do people respond, and does it vary by sender?
• Measuring information flow — How many hops does it take for information to reach from leadership to front-line teams?

Chat Event Streams

Chat event streams capture interactions from platforms like Slack, Microsoft Teams, Google Chat, and other messaging tools. Chat data differs from email in important ways: it's typically faster-paced, more informal, and often takes place in shared channels rather than private exchanges.

Chat events include:

• Direct messages — One-to-one conversations, similar to email but more immediate
• Channel messages — Posts to shared spaces, visible to all channel members
• Reactions — Emoji reactions to messages (a lightweight form of engagement)
• Thread replies — Responses within a specific message thread
• Mentions — Tagging another user in a message (a signal of directed attention)
• File shares — Sharing documents, images, or links in channels

The analytical value of chat streams lies in their real-time, informal nature. Where email captures deliberate, structured communication, chat captures the spontaneous, fast-moving interactions that often drive day-to-day collaboration. An employee might send five emails in a day but exchange fifty chat messages.

Channel membership data adds another layer. Knowing that Maria is a member of #engineering, #cross-functional-planning, and #hackathon-2026 tells you about her organizational reach even before you analyze any messages.

Device Activity Logs

Device activity logs capture the digital footprint of hardware and system usage. These logs come from a range of sources and can be broken into three subcategories: desktop activity, mobile device events, and software application logs.

Desktop activity includes events generated by workstation operating systems and management agents:

• Boot and shutdown events
• Screen lock and unlock times
• Active window tracking (which application is in the foreground)
• File access events (opening, saving, closing documents)
• USB device connections
• Print jobs

Mobile device events are generated by company-managed smartphones and tablets through Mobile Device Management (MDM) platforms:

• App installation and removal
• Location check-ins (for field workers, with consent)
• Push notification interactions
• Mobile email and calendar synchronization
• Device compliance status (encryption, OS version)

Software application logs are generated by the business applications themselves — CRM systems, project management tools, HR platforms, document editors, and analytics dashboards:

• Record creation and modification events
• Report generation
• Dashboard views
• Workflow approvals
• Data exports

Device activity logs are sensitive by nature. They can reveal detailed patterns about how individuals spend their time, and they must be handled with care. The goal is never individual surveillance — it's aggregate pattern recognition. You're looking for things like: "Do teams that use the collaboration platform more frequently also have higher project completion rates?" not "How many minutes did a specific employee spend on non-work applications?"

Diagram: Event Source Taxonomy

Event Source Taxonomy

Type: infographic

Bloom Taxonomy: Analyze (L4) Bloom Verb: categorize Learning Objective: Students will categorize the major types of organizational event sources and identify the kinds of events each produces.

Purpose: Show the hierarchical taxonomy of event sources — from high-level categories (Communication, Device, Business Process) down to specific source systems and event types.

Layout: Tree diagram rooted at "Employee Event Streams" at the top. Three main branches:

1. "Communication Streams" (indigo #303F9F)
2. Email Events: SEND, RECEIVE, REPLY, FORWARD
3. Chat Events: MESSAGE, REACTION, MENTION, THREAD_REPLY

4. Calendar Events: MEETING_CREATE, ACCEPT, DECLINE, ATTEND

5. "Device & Application Streams" (amber #D4880F)

6. Desktop Activity: LOGIN, LOGOUT, APP_FOCUS, FILE_ACCESS
7. Mobile Events: APP_INSTALL, LOCATION, SYNC

8. Software Logs: RECORD_CREATE, APPROVAL, EXPORT

9. "Business Process Streams" (gold #FFD700)

10. Process Events: TASK_START, TASK_COMPLETE, HANDOFF, ESCALATION
11. Compliance Events: APPROVAL, REVIEW, AUDIT_LOG

Interactive elements:

• Click any branch to expand/collapse its children
• Hover over a leaf node (specific event type) to see an example log entry in a tooltip
• Color-coded by category with subtle connecting lines

Visual style: Clean hierarchical tree with rounded nodes. Aria color scheme. White background.

Responsive design: On narrow screens, tree collapses to an expandable accordion view.

Implementation: p5.js with canvas-based tree layout and click/hover interactions

Calendar Events and Meeting Patterns

Calendar events provide a structured view of how people spend their collaborative time. Unlike email and chat, which capture ad hoc communication, calendar data reveals planned interactions — the meetings, workshops, one-on-ones, and all-hands that shape the weekly rhythm of organizational life.

Calendar event data typically includes:

• Organizer — Who created the meeting
• Attendees — Who was invited, and their response (accepted, declined, tentative)
• Time and duration — When the meeting occurred and how long it lasted
• Recurrence — Whether it's a one-time or recurring event
• Location — Physical room or virtual meeting link
• Subject — Meeting title (handle with the same privacy care as email subjects)

From calendar data, you can extract meeting patterns — the structural rhythms that define how groups collaborate:

• Meeting load — How many hours per week does a team spend in meetings? Is it sustainable?
• Meeting overlap — Which teams regularly share meeting attendees, suggesting strong cross-functional ties?
• One-on-one frequency — How often do managers meet individually with their direct reports?
• Recurring vs. ad hoc ratio — A high ratio of recurring meetings might signal rigid processes; a high ratio of ad hoc meetings might signal reactive firefighting.
• Declined meeting rate — Are certain meetings consistently declined? That's a signal worth investigating.
• Large meeting concentration — Are decisions being made in meetings of twenty people when five would suffice?

Meeting patterns are especially powerful when combined with email and chat data. If two teams never meet together (calendar) but exchange frequent emails (email stream) and have active cross-team channels (chat stream), that tells a very different story than if they share none of those signals.

Login and Logout Events

Login and logout events mark the boundaries of active work sessions. They come from operating system authentication, VPN connections, single sign-on (SSO) platforms, and application-specific authentication systems.

These events are deceptively simple — just a user ID, a timestamp, and a direction (in or out). But in aggregate, they reveal important patterns:

• Work hour distribution — When does actual work happen? Are people logging in at 6 AM and staying until midnight?
• Session duration — How long are typical work sessions? Are there frequent short sessions suggesting interruptions?
• Off-hours access — Who regularly works outside standard hours? This can signal dedication, but also potential burnout.
• Location patterns — VPN logins from unusual locations (with privacy safeguards) can indicate remote work patterns.
• System access breadth — How many different applications does an employee access? Broad access might indicate a cross-functional role; narrow access might signal specialization.

Login/logout events also serve as a framework for anchoring other event types. When you know that Maria's work session ran from 8:04 AM to 5:47 PM, you can contextualize all the emails, chats, and meetings that occurred within that window.

Event Normalization: Creating a Common Language

You've now seen five major categories of event sources, each with its own format, naming conventions, and level of detail. The challenge is this: how do you combine data from Outlook, Slack, Active Directory, Jira, and Google Calendar into a single, coherent event stream that can be loaded into a graph?

The answer is event normalization — the process of transforming raw event data from diverse sources into a consistent, standardized format. Normalization ensures that every event, regardless of its origin, speaks the same language.

Normalization involves several transformations:

1. Field mapping — Standardizing field names across sources. One system calls it "sender," another calls it "from," a third calls it "originator." After normalization, they're all "actor."

2. Timestamp conversion — Converting all timestamps to UTC in ISO 8601 format (as discussed earlier).

3. Action vocabulary — Creating a controlled vocabulary of action types. Slack's "message_posted" and Teams' "chatMessageSent" both become "CHAT_SEND."

4. Identity resolution — Mapping different user identifiers to a single canonical ID. The same person might be "m.chen" in Active Directory, "maria.chen@company.com" in email, and "Maria C." in Slack.

5. Schema alignment — Ensuring every normalized event has the same base fields, with source-specific details stored in an extensible metadata block.

Here's what normalization looks like in practice:

Before normalization (raw Slack event):

{
  "type": "message",
  "user": "U03B7K9QP",
  "text": "Can we sync on the API changes?",
  "ts": "1742047622.003400",
  "channel": "C01ENGINEERING"
}

After normalization:

{
  "event_id": "CHAT-2026-0315-1347-B2",
  "timestamp": "2026-03-15T13:47:02Z",
  "actor": "EMP-00147",
  "action": "CHAT_SEND",
  "target": "CHANNEL-engineering",
  "source_system": "slack",
  "metadata": {
    "content_length": 39,
    "channel_name": "engineering",
    "thread_id": null
  }
}

Notice that the message text has been replaced with a content length — another privacy-preserving transformation. The Slack-specific user ID has been resolved to a canonical employee ID. The Unix timestamp has been converted to ISO 8601 UTC. And the action type has been standardized to a controlled vocabulary term.

The controlled action vocabulary is a critical design decision. Here's a sample mapping:

Source System	Raw Action	Normalized Action
Outlook	MessageSent	EMAIL_SEND
Outlook	MessageReceived	EMAIL_RECEIVE
Gmail	messages.send	EMAIL_SEND
Slack	message_posted	CHAT_SEND
Teams	chatMessageSent	CHAT_SEND
Slack	reaction_added	CHAT_REACT
Calendar	event.created	MEETING_CREATE
Calendar	attendee.accepted	MEETING_ACCEPT
Active Directory	UserLogon	SESSION_LOGIN
Active Directory	UserLogoff	SESSION_LOGOUT
Jira	issue_updated	TASK_UPDATE
Badge System	door_access	FACILITY_ENTRY

Event Enrichment: Adding Context

Once events are normalized, the next step is event enrichment — augmenting each event with contextual information drawn from other data sources. Enrichment transforms a flat log entry into a richly contextualized record that's ready for graph construction.

Common enrichment operations include:

• Organizational context — Adding the actor's department, team, manager, office location, and job level from the HR system. This enables queries like "How do communication patterns differ between Engineering and Sales?"

• Temporal context — Tagging events with derived time attributes: day of week, business hours vs. off-hours, fiscal quarter, time since hire date. This supports meeting pattern analysis and work rhythm detection.

• Relationship context — Annotating whether the actor and target are in the same department, the same management chain, or the same project team. Cross-departmental communication is analytically different from within-team communication.

• Interaction history — Adding cumulative counters: "This is the 47th email between these two employees this month." Frequency and trend data transform individual events into relationship strength signals.

• Content signals — For communication events, adding NLP-derived features like sentiment score, detected topic, urgency classification, or language. (We'll cover NLP in depth in Chapter 9, but basic enrichment can happen here.)

Here's a before-and-after example of enrichment:

Normalized event (pre-enrichment):

{
  "event_id": "EMAIL-2026-0315-0847-A1",
  "timestamp": "2026-03-15T13:47:22Z",
  "actor": "EMP-00147",
  "action": "EMAIL_SEND",
  "target": "EMP-00203",
  "source_system": "outlook"
}

Enriched event:

{
  "event_id": "EMAIL-2026-0315-0847-A1",
  "timestamp": "2026-03-15T13:47:22Z",
  "actor": "EMP-00147",
  "action": "EMAIL_SEND",
  "target": "EMP-00203",
  "source_system": "outlook",
  "enrichment": {
    "actor_department": "Engineering",
    "target_department": "Product",
    "cross_departmental": true,
    "actor_tenure_days": 847,
    "day_of_week": "Monday",
    "business_hours": true,
    "interaction_count_30d": 23,
    "same_manager": false,
    "actor_job_level": "IC3",
    "target_job_level": "M2"
  }
}

That enriched record tells a far richer story than the original. It's not just "someone emailed someone" — it's "a mid-level engineer with over two years of tenure reached out to a product manager in a different department, during business hours, continuing an active cross-departmental communication pattern." When this event becomes an edge in your graph, it carries all the context needed for meaningful analysis.

Diagram: Normalization and Enrichment Pipeline

Normalization and Enrichment Pipeline

Type: workflow

Bloom Taxonomy: Apply (L3) Bloom Verb: implement Learning Objective: Students will trace the steps of event normalization and enrichment and explain how raw events are transformed into graph-ready records.

Purpose: Show the multi-stage pipeline that transforms raw events from diverse sources into normalized, enriched records ready for graph loading.

Layout: Horizontal flow diagram with four stages, left to right:

1. "Raw Sources" (left) — Multiple input icons representing Email, Chat, Calendar, Devices, and Applications, each with a different raw format snippet
2. "Normalization" (center-left) — A processing block showing field mapping, timestamp conversion, action vocabulary mapping, and identity resolution
3. "Enrichment" (center-right) — A processing block showing organizational context, temporal context, relationship context, and interaction history being added
4. "Graph-Ready Events" (right) — A single unified format flowing toward a graph database icon

Arrows connect each stage. Below each stage, show a sample JSON snippet (abbreviated) illustrating the data at that point.

Interactive elements:

• Click any stage to expand it and see detailed sub-steps
• Hover over the sample data at each stage to see the full JSON record
• Animate a single event flowing through the pipeline when a "Play" button is clicked

Visual style: Clean workflow with rounded processing blocks. Inputs in amber (#D4880F), processing stages in indigo (#303F9F), output in gold (#FFD700). White background.

Responsive design: On narrow screens, stages stack vertically.

Implementation: p5.js with canvas-based layout, click/hover interactions, and simple animation

Business Process Mining: Discovering How Work Really Happens

Everything we've covered so far — email streams, chat logs, device activity, calendar events — feeds into a powerful analytical discipline called business process mining. Process mining uses event log data to reconstruct, visualize, and analyze the actual workflows that operate within an organization, as opposed to the workflows that are documented or assumed.

The gap between how processes are supposed to work and how they actually work is one of the most consequential blind spots in organizational management. A procurement process might be documented as a five-step approval chain, but event logs reveal that 40% of purchase orders skip step three, 15% get routed through an unofficial approver, and the average cycle time is three times longer than the documented target.

"In my colony, we thought the leaf-processing pipeline was simple: cut, carry, clean, cultivate. Then I mapped the actual event logs and discovered that 30% of the leaves were being rerouted through an unofficial quality-check tunnel run by a retired forager named Beatrice. The process documentation said five steps. Reality had seven. Beatrice was the most important node nobody knew about." — Aria

Process Discovery

Process discovery is the automated reconstruction of a business process model directly from event log data. Rather than interviewing stakeholders or reviewing documentation (both of which are colored by assumption and wishful thinking), process discovery algorithms analyze the actual sequence of events to build a model of what really happens.

The fundamental input for process discovery is a set of event logs where each event is tagged with:

• A case ID — which process instance this event belongs to (e.g., a specific purchase order, a specific employee onboarding)
• An activity name — what happened (e.g., "Submit Request," "Manager Approval," "Finance Review")
• A timestamp — when it happened

From these three fields, process discovery algorithms can reconstruct the typical flow of a process, identify variations, detect bottlenecks, and surface exceptional paths that deviate from the norm.

Common process discovery techniques include:

• Alpha algorithm — Constructs a process model (specifically a Petri net) from the ordering relationships between activities in the log
• Heuristic mining — More robust to noise than the Alpha algorithm; uses frequency-based thresholds to determine which activity sequences represent real process patterns
• Inductive mining — Guarantees a sound process model and handles complex process structures like concurrency and loops

The output of process discovery is typically a process map — a visual model showing activities as nodes, transitions as edges, and annotations for frequency, duration, and variance. This map becomes a powerful tool for understanding how work actually flows through the organization.

Diagram: Process Discovery Flow

Process Discovery Flow

Type: microsim

Bloom Taxonomy: Analyze (L4) Bloom Verb: analyze Learning Objective: Students will analyze event log data to discover the actual flow of a business process and compare it to the documented process.

Purpose: Interactive simulation showing how process discovery transforms raw event logs into a visual process map, highlighting deviations from the expected process.

Layout: Two-panel layout:

• Left panel: "Event Log" — A scrollable table of events with columns for Case ID, Activity, Timestamp, and Actor. Pre-loaded with 15-20 events across 4-5 case instances of a "Hiring Process" (Post Job, Screen Applications, Schedule Interview, Conduct Interview, Decision, Offer, Onboard).
• Right panel: "Discovered Process" — A process map that builds as events are analyzed, showing activities as rounded rectangles (indigo #303F9F) connected by directed edges (amber #D4880F) with frequency labels.

Interactive elements:

• "Discover" button: Animates the construction of the process map from the event log, highlighting each event as it's processed
• "Show Ideal Process" toggle: Overlays the documented/expected process in gray, so students can see deviations
• Hover over any edge to see the number of cases that followed that path and the average time between activities
• Hover over any activity node to see average duration and which actors performed it most

Data: Include realistic variations — most cases follow the standard path, but 2-3 cases skip the screening step, one case loops back from Decision to Schedule Interview, and one case has an unusually long delay at the Offer stage.

Visual style: Process map with rounded activity nodes in indigo, edges in amber with thickness proportional to frequency. Deviation edges shown in coral/red. Aria color scheme throughout.

Responsive design: On narrow screens, panels stack vertically.

Implementation: p5.js with canvas-based process map rendering, animation, and hover interactions

Process Conformance

Once you've discovered how a process actually works, the natural next question is: how well does reality match the design? This is the domain of process conformance — the systematic comparison of actual process execution (as revealed by event logs) against a reference model (the intended or documented process).

Conformance analysis identifies four types of deviations:

Deviation Type	Description	Example
Skipped activities	Steps in the reference model that were not executed	Manager approval bypassed
Inserted activities	Steps that occurred but aren't in the reference model	Unofficial peer review added
Wrong sequence	Activities performed in a different order than specified	Testing done before development complete
Wrong resource	The correct activity was performed by an unauthorized person	Intern approving purchase orders

Conformance checking doesn't just find problems — it also identifies positive deviations. That unofficial peer review step? Maybe it's reducing defects by 30% and should be formalized. The team that consistently skips a redundant approval step? Maybe the process documentation is the problem, not the team.

The key metrics in conformance analysis:

• Fitness — What proportion of cases in the event log can be replayed on the reference model? High fitness means reality closely matches the design.
• Precision — Does the model allow only the behavior observed in the log, or does it also permit paths that never actually occur? High precision means the model isn't overly permissive.
• Generalization — Will the model hold up for future cases, or is it overfit to the specific log used to build it?
• Simplicity — Is the model as simple as possible while still accurately representing the process?

From Events to Graphs

You might be wondering how process mining connects to graph databases. The connection is natural: a discovered process map is a graph. Activities are nodes, transitions are edges, and properties on those edges (frequency, duration, variance) encode the behavioral patterns. In Chapter 4, you'll learn how to load process mining results directly into your organizational graph, connecting process data with communication data, organizational structure, and more.

Putting It All Together: From Streams to Graph-Ready Data

Let's step back and see the full picture. This chapter has traced a path from raw digital footprints to enriched, normalized, graph-ready event records. Here's the complete flow:

1. Capture — Event streams flow from organizational systems (email, chat, calendar, devices, business applications) into event logs
2. Timestamp — All events are converted to universal timestamps in UTC
3. Normalize — Raw events are transformed to a consistent schema with standardized fields, controlled action vocabulary, and resolved identities
4. Enrich — Organizational, temporal, and relational context is added to each event
5. Prepare — Enriched events are formatted for graph ingestion, with actors becoming nodes and interactions becoming edges

This pipeline doesn't run once — it runs continuously. As new events are generated, they flow through the same normalization and enrichment steps, keeping your organizational graph current. The result is a living, breathing representation of how your organization actually operates, updated in near-real-time.

Diagram: Complete Event Stream Pipeline

Complete Event Stream Pipeline

Type: workflow

Bloom Taxonomy: Evaluate (L5) Bloom Verb: assess Learning Objective: Students will assess the complete event stream pipeline from capture through graph preparation and evaluate the role of each stage.

Purpose: End-to-end visualization of the entire event stream pipeline covered in this chapter, showing how raw events from multiple sources are captured, timestamped, normalized, enriched, and prepared for graph loading.

Layout: A horizontal pipeline with five stages connected by directional arrows:

1. "Capture" — Icons for 5 source types feeding into a collection funnel
2. "Timestamp" — Clock icon showing UTC conversion
3. "Normalize" — Gear icon showing schema alignment and vocabulary mapping
4. "Enrich" — Plus icon showing context being added from HR and organizational data
5. "Graph-Ready" — Graph icon showing nodes and edges emerging from the pipeline

Below the pipeline, a running counter shows: "Events processed: [count]" that increments during animation.

At the bottom, three sample events are shown at their current pipeline stage, with color coding to indicate their source (email = indigo, chat = amber, calendar = gold).

Interactive elements:

• "Start Pipeline" button animates events flowing through each stage
• Click any stage to see a detailed breakdown of what happens at that step
• Speed control slider (1x, 2x, 5x) for animation pace
• Pause/resume button

Visual style: Clean industrial pipeline metaphor with Aria color scheme. Stages are rounded blocks with icons. Event tokens are small colored circles flowing along the pipeline.

Responsive design: On narrow screens, pipeline wraps vertically.

Implementation: p5.js with canvas-based animation and click/hover interactions

Privacy and Ethics: A First Look

Before we leave this chapter, a critical reminder. The event data we've described is extraordinarily revealing. Email patterns expose social networks. Chat metadata reveals informal hierarchies. Device logs show work habits. Calendar data maps power structures. Combined, these streams paint an intimate portrait of organizational life.

This data must be handled with profound respect for the people it represents. Chapter 6 covers ethics, privacy, and security in full depth, but here are the principles that apply specifically to event stream collection:

• Metadata over content — Analyze communication patterns, not message content. You don't need to read emails to map networks.
• Aggregation over identification — Report on team and departmental patterns, not individual behaviors.
• Transparency — Employees should know what data is being collected and how it's being used.
• Purpose limitation — Event data collected for organizational improvement must never be repurposed for performance surveillance or punitive action.
• Data minimization — Collect only the fields you need. If content length is sufficient, don't store content.
• Retention limits — Define how long event data is retained and enforce it automatically.

Chapter Summary

"Look at you — you just mapped the entire landscape of organizational data sources. That's like knowing every tunnel, every chamber, and every pheromone trail in the colony before you've even started the analysis. Not bad at all." — Aria

Let's stash the big ideas before we move on:

• Employee event streams are the chronological sequences of digital actions — emails, chats, meetings, logins — that collectively reveal how an organization actually operates.

• Event logs are the structured records that capture these streams, with each entry containing at minimum a timestamp, actor, action, target, and source system.

• Universal timestamps in UTC (ISO 8601) are essential for combining events from multiple source systems into a single, chronologically accurate stream.

• Email event streams capture communication patterns through metadata (sender, recipients, timestamps, thread IDs) without requiring access to message content.

• Chat event streams reveal fast-paced, informal collaboration patterns across direct messages, channels, reactions, and mentions.

• Device activity logs encompass desktop activity, mobile device events, and software application logs — providing insight into how tools are used and when work happens.

• Calendar events and meeting patterns expose the structured, planned dimension of collaboration — who meets with whom, how often, and whether those meetings are sustainable.

• Login and logout events mark the boundaries of work sessions and reveal patterns in work hours, system access, and location.

• Event normalization transforms raw events from diverse sources into a consistent schema with standardized fields, controlled action vocabulary, and resolved identities.

• Event enrichment adds organizational, temporal, and relational context to normalized events, transforming flat log entries into richly contextualized records ready for graph construction.

• Business process mining uses event logs to reconstruct and analyze actual workflows, revealing how processes really operate versus how they're documented.

• Process discovery automatically builds process models from event log data, surfacing the true flow of activities, including variations and bottlenecks.

• Process conformance compares actual process execution against reference models, identifying skipped steps, inserted activities, wrong sequences, and unauthorized performers.

In Chapter 4, you'll learn how to take these normalized, enriched event streams and load them into a graph database — transforming flat records into the interconnected nodes and edges that make organizational analytics possible.

Six legs, one insight at a time. You've got this.

See Annotated References