To understand how work is performed across the organization, Toptal developed ActionTrace, an AI-powered process intelligence platform that transforms multimodal user activity into canonical process models.
As Toptal expanded its AI initiatives, it needed an operational intelligence layer that could serve as the foundation for its broader AI stack. Traditional process documentation and task mining tools were unable to capture the workflow context in a structured form for AI-assisted querying and analysis. To address this challenge, Toptal needed to build a scalable platform that could automatically discover and model business workflows directly from user activity.
Captured user interactions via a desktop application that records screen activity, application context, mouse movements, keyboard input, and timing information, creating a comprehensive view of how work is performed. The platform is designed to evolve with capabilities such as edge-based privacy protections and human-in-the-loop (HITL) refinement to strengthen data quality while preserving user trust.
Combined operational data from screen recordings, user interactions, and application telemetry to reconstruct business context. The architecture is designed to correlate these signals with external system events, providing a more complete view of downstream business impact. Multimodal large language models interpret visual interfaces alongside contextual signals, enabling the system to understand the business intent behind user actions.
Machine learning and process mining techniques infer higher-level business activities from multimodal interaction data, then reconstruct discovered workflows and generate canonical process models. Activity boundaries are identified using signals such as application switches, terminal UI elements, and time gaps, with labels resolved through a layered annotation system spanning element-, group-, and application-level context. This heuristic path is paired with an LLM-based path that propagates labels across batches for consistent naming at scale, consolidating workflow variations into representations that reveal common paths, variants, and bottlenecks.
Implemented canonical process models to create a machine-readable understanding of business operations. Each model is represented in interoperable semantic and process formats: OWL/Turtle ontologies for semantic querying and Petri nets (PNML) for compatibility with standard process-mining tools. Decision points inferred from process branching explain why workflows diverge, creating a shared operational context that supports process analysis and positions the platform for natural-language querying and intelligent workflow automation.
Built the platform on a cloud-native architecture using Google Cloud technologies to support scalable data processing and AI inference. BigQuery and Google Cloud Storage manage operational data and process artifacts, while Vertex AI supports multimodal semantic inference through batched LLM calls. Designed to evolve alongside discovered workflows, the architecture supports future orchestration and broader AI applications.
ActionTrace operates as an end-to-end platform spanning workflow capture, contextual understanding, process discovery, and visualization. Built on a scalable, cloud-native architecture, the platform combines multimodal AI, process mining, and semantic enrichment to transform user interactions into canonical process models that reflect how work is actually performed. With discovery algorithms continuing to be refined, ActionTrace has established a machine-readable operational foundation for workflow intelligence and automation.
