Looking Back: Building Data Foundations
My work in Data and AI began with a deceptively simple but hard question:
How do you build data platforms that scale with the business?
In 2023, most of my focus was on:
- Designing and operating large-scale data platforms
- Improving data reliability, freshness, and trust
- Enabling analytics, experimentation, and machine learning
- Reducing friction between engineering, product, and data teams
At that stage, success wasn’t measured by how advanced the models were, but by whether data systems were dependable enough to support real decisions.
This period reinforced a belief I still hold strongly:
Strong AI systems are built on strong data platforms.
ELT-Based Big Data Stacks as a Foundation
One of the most impactful architectural decisions was adopting ELT-based big data stacks.
ELT enabled:
- Scalable ingestion of diverse data sources
- Clear separation of data movement from data modeling
- Faster iteration on transformations and schemas
- Better support for analytics and ML workloads
This approach allowed teams to focus on using data, not fighting pipelines.
More importantly, it created a flexible foundation capable of supporting increasingly advanced AI use cases.
High-Level Platform Architecture
The diagram below shows how ELT-based data platforms evolve into AI-native platforms that support agents.
From Predictive Models to Agentic Systems
Traditional ML systems fit neatly into batch processing or request–response APIs.
Agentic AI changes the paradigm.
Agentic systems:
- Reason continuously over time
- Plan and execute multi-step actions
- Use tools and data dynamically
- Learn from outcomes and feedback
This reframes AI from prediction services into systems that think and act.
At this stage, platform design becomes more important than any individual model.
Data Platforms as Cognitive Infrastructure
Agentic AI raises expectations for what data platforms must provide.
Beyond storage and pipelines, platforms now need to support:
- Contextual access to historical and real-time data
- Clear data semantics and lineage
- Durable memory grounded in reliable storage
- Feedback loops tied to real outcomes
- Strong access controls and safety boundaries
Data platforms become:
Cognitive infrastructure that grounds autonomous intelligence in reality
They are what make agent reasoning auditable, explainable, and trustworthy.
Open-Source Observability: Making AI Operable
As systems become more autonomous, observability becomes foundational.
A significant part of my work has involved deploying open-source observability stacks spanning data, ML, and agents.
These stacks monitor:
- Data pipelines and freshness
- Feature generation and serving
- Model performance, drift, and bias
- Agent reasoning, decisions, and tool usage
Observability Across Data, ML, and Agents
Open-source observability provides transparency, extensibility, adherence to regulatory rules, and deep platform integration.
More importantly, it turns AI from a black box into an operable system.
Data, ML, and Agent APIs as Platform Interfaces
At scale, intelligence must be exposed through clear, stable interfaces.
Modern platforms achieve this through a combination of data, ML, and agent APIs that separate implementation from usage while enabling governance, control, and observability.
Platform APIs: Data, ML, and Agent Interfaces
Modern intelligent platforms typically provide:
- Data APIs for features, aggregates, and real-time signals
- ML APIs for predictions, embeddings, and scoring
- Agent APIs for tool access, memory, and controlled actions
These APIs are where governance meets autonomy—ensuring that intelligent systems remain auditable and controllable.
Architecture Overview
APIs decouple intelligence from implementation details and make agent behavior controllable and auditable.
ELT, APIs, and Observability: Enabling Agentic AI
Agentic AI does not emerge from models alone—it is the result of platform design choices.
A reliable foundation is created by combining three core capabilities:
- ELT-based data platforms that support scalable, replayable, and auditable data flows
- Strong Data and ML APIs that expose features, predictions, and embeddings through stable interfaces
- Open-source observability tools that provide visibility into decisions, actions, and outcomes
Together, these components transform AI from isolated predictions into operational intelligence.
What This Enables
A platform built on ELT, APIs, and observability allows teams to:
- Roll out intelligent systems incrementally, starting with human-in-the-loop workflows
- Evaluate human and agent decisions side by side, using the same data and metrics
- Continuously monitor, learn, and improve, based on real-world feedback
- Build trust through visibility and control, rather than blind automation
Scaling Intelligence Responsibly
This approach ensures that intelligence scales in a way that is:
- Powerful, by leveraging modern data and ML capabilities
- Transparent, through observable behavior and decision traces
- Accountable, with clear ownership, governance, and guardrails
This is how agentic systems move from experimentation to production—
not as black boxes, but as trusted, auditable platform capabilities.
Looking Toward 2026: Building AI-Native Platforms
Looking ahead, my focus is shifting toward AI-native platform design—systems where intelligence is not an add-on, but a foundational capability.
Rather than centering on individual models, AI-native platforms are built to support reasoning, action, and learning as continuous processes.
Areas of Focus
I’m particularly interested in advancing platforms that emphasize:
-
Agent-Aware Data Architectures
Data systems designed to support planning, memory, feedback loops, and long-running agent workflows. -
Observability for Reasoning and Actions
Visibility not just into outputs, but into why decisions were made and how actions unfolded over time. -
Data, ML, and Agent APIs as First-Class Citizens
Stable interfaces that make intelligence accessible, governable, and composable across the organization. -
Human-in-the-Loop Autonomy
Systems that blend automation with oversight, enabling agents to act independently while remaining accountable. -
Intelligence as Infrastructure
Platforms that treat reasoning, learning, and decision-making as shared capabilities—just like compute, storage, and networking.
A Platform-First Future
The future of AI will be defined less by individual models and more by the systems that surround them.
AI-native platforms will be the differentiator—enabling organizations to scale intelligence responsibly, adapt continuously, and build trust in autonomous systems.
This is the direction where durable, production-grade AI will be built.