Saket Saurabh:
Hello everyone, and thank you for listening. This is your host, Saket Saurabh, with another episode of Data Innovators and Builders. Today I’m speaking with Mahesh Mishra, Vice President of AI Products at Cloudera. Mahesh, thanks for chatting with me today.
Mahesh Mishra:
Thank you for having me.
Saket Saurabh:
Mahesh, tell us a little bit about your journey in data so far.
Mahesh Mishra:
Yeah, so I have been in this data and analytics space, including AI, for the last twenty years. The first half of my career, I was an engineer doing hands-on coding. I joined AWS a long time ago in 2011 and spent close to fifteen years there. Half of my stint there was primarily building data products, and the second half was building some of the foundational building blocks for customers so that they can host their data platforms.
I hold three patents in the area. One is for distributed data warehouse management, the other is for provisioning data services at scale, and the third is dynamic hierarchy in cataloging. So pretty much very deep into how you do compute management in cataloging.
Recently I joined Cloudera, close to three or four months ago, as VP of AI Products. I lead the Cloudera AI product portfolio. Many of these products are about how customers use data to build and train AI models, host inference endpoints on the Cloudera platform, and use the agentic platform to build agents that can interact with their data and their tools and start working autonomously. That’s who I am.
Saket Saurabh:
Great background. We were talking earlier about how data and AI are closely related, close cousins, as we were saying. Your work has spanned both areas, especially when it comes to AI, and you’re seeing it from the world of enterprises. What kind of use cases are you seeing starting to take root in enterprise deployments?
Mahesh Mishra:
I think at this point, most of the use cases are focused around engineering the stack itself. For example, many customers build text-to-SQL use cases, where they can ask a plain-language business question and agents can autonomously look at the data dictionary, figure out which columns may be relevant, and then generate a query that can answer the question. That’s a very common use case.
Customers who are a bit more advanced do data engineering, where they give a use case or a data pipeline requirement in plain language, and the system looks at the infrastructure and the underlying engine, and if Spark code is needed, generates the Spark code. If there’s additional SQL or DBT code that needs to be generated, it generates that too.
The very small number of customers who are really in production are actually focusing on low-stakes tasks, the mundane tasks that data engineers or operations teams typically handle. For example, going through a log when a job fails and figuring out what happened, then fixing it automatically.
If you’ve worked in a data platform for a long time, you’ll have seen that people don’t communicate well within the enterprise, and somewhere a data type changes, and that causes a cascading failure everywhere. Those trivial changes need people to pay attention, and those kinds of trivial, mundane tasks are being automated by AI. These are the common use cases I see.
There are customers thinking about putting some high-stakes decision-making into agentic workflows, but I haven’t seen a lot of customers who are actually successful at that, given the space is evolving and there’s a lot of risk and regulatory requirement involved with high-stakes decision-making. Does that make sense?
Saket Saurabh:
Yeah, that makes sense. From a business side, customer support has been one area where a lot of companies have built solutions and shown success. I’m seeing more and more tools around go-to-market becoming agentic, not to mention all the vibe coding side. So the business impact is definitely starting to show.
Mahesh Mishra:
Absolutely, yeah. Customer care and customer support use cases are something I come across quite frequently. There are also very interesting use cases in the law enforcement space. If you look at law enforcement officers, half of what they do is basically paperwork. Some incident happens, and they have to fill out the paperwork, even though they’ve called 911 and given all the information there. Many companies are thinking about how to reduce that burden on the law enforcement officer. That’s an interesting area as well.
Saket Saurabh:
That reminds me of similar use cases I’m seeing in healthcare, where doctors have exactly the same thing going on. So definitely a lot of process efficiency and workflow automation there. One thing you mentioned that’s very interesting is that you hold a patent in the catalog space, around multi-level catalogs, and we were talking a bit about knowledge engineering. Tell us a little bit about what knowledge engineering is and how it’s playing a role in better AI outcomes.
Mahesh Mishra:
I think knowledge engineering becomes very relevant at the intersection of data and AI, where you’re using AI to do a lot of data processing or using AI and data to make critical actions or decisions in your enterprise. Everybody talks about prompt engineering, as a layer of instructions to the LLM to do certain tasks. People talk about context engineering, where you provide more information about the problem you’re solving so the agent has an informed decision to make.
But what nobody is talking about is knowledge engineering. The way to describe knowledge engineering in comparison to context engineering is: context engineering is point-in-time. Someone wrote a prompt, or gave a tool, and told the system to read the context from the system, and that’s a point-in-time view of the world. It has a source-of-truth system, and it extracts from there.
Knowledge engineering, think of it like a human baby. As that baby grows, it learns from its own experiences. It does some tasks, makes mistakes, someone tells it “hey, that’s not the right thing to do,” it course-corrects, and evolves. Knowledge engineering is exactly that: bringing that experience layer, learning through experience, to the LLMs.
It has three components. One is knowledge acquisition. There needs to be a strong mechanism to go and gather knowledge from the places where it’s documented or stored. This is a concept that’s been around for thirty years, but given the technology was less mature then, and there wasn’t a lot of understanding of graph-type databases, it didn’t kick off. Now is the time to think about it, because we have LLMs that can understand structured and unstructured data together. Knowledge scouts can go to the knowledge sources and gather knowledge.
Generally there’s a disconnect in the data platform world: decisions are made in Slack messages, in a meeting room with a bunch of people, or on a Zoom call. Once those decisions are made, they either land in a Jira ticket, in GitHub code, or they just stay in the Slack threads. So your data is structured, but your decisions are unstructured. You need to combine them to build a knowledge base that is your true context layer, because it captures not just what the data means, but the reasoning behind why certain decisions were made, when, and who made them, plus a ton of other associated metadata. That’s your knowledge scout and knowledge repository.
Then you need to train your agents to make calls. That’s where two concepts I’m a strong proponent of come in: decision flow language and decision tracing language. A decision flow language tells the LLM what to do, what alternate criteria to evaluate, and when something passes versus when it should go to a human to correct. Tracing is basically, after the fact, when you need to do the postmortem, understanding what the reasoning was behind certain decisions. It’s a language where you store all the traces generated by an LLM call.
Your knowledge base, which is self-curating, plus your decision flow language and decision tracing language layers, need to be fed together: here are the high-stakes decisions we made, here are the decisions where humans were involved, here’s the correction made, and going forward, this is the new decision flow we need to follow. That flywheel has to move so you can build a strong knowledge engineering platform, one that’s not only gathering and storing point-in-time information, but also learning from its own mistakes and building a self-aware layer that evolves its decision-making capability. Does that make sense?
Saket Saurabh:
Yeah, that totally makes sense, especially given that if we want to bring AI into certain workflows and operational aspects of how a team or a function works, we need those traces of decisions and how they were made. One thing I’ve been thinking about is that there are certain decisions that could be high impact but low risk versus high risk, high impact decisions. There are ones you can start to automate because there’s enough context, knowledge, and decision traces around them, and there are ones that are still going to be human-driven, where a lot of judgment is involved.
Mahesh Mishra:
There’s an example. Say you’re a bank doing underwriting. Underwriting is never a black-and-white decision. You have to look at a bunch of metrics: asset-to-liability ratio, income-to-debt ratio, how many times someone has defaulted on a payment, and a ton of other data around their profile. But it’s never black and white.
Sometimes it’s like, okay, there was a default six years ago, that’s not as relevant today. If an LLM is making that decision, every time it sees a default, it’ll say “I’m not going to approve this, I’ll involve a human.” But if a human looks at the data and sees that, while there was a default six years ago, the last three to five years show consistent payments and a good debt-to-income ratio, they’ll approve it.
If that happens three times, it should be fed back to the LLM, and the LLM should autonomously make that decision going forward. I think knowledge engineering is going to be super handy for data-driven workflows: you have context, which is point-in-time information, plus that context evolves through decision flows and traces.
Saket Saurabh:
Yeah, or at least, even if it has to come to a human for review, all that information is laid out in front of the person, showing why it should be a yes or what the factors are for a no. That simplifies the function and reduces the variability that might otherwise be hard to explain. Quick question: when you talk about decision flow language, decision trace language, and the knowledge engineering layer, are there tools for that today? Is it being solved, or is it a collection of disparate capabilities?
Mahesh Mishra:
There are no such tools today. Everything is embedded within a prompt. The decision flow is embedded within a prompt, and decision traces sometimes use OpenTelemetry, but the real traces aren’t really being stored there, to be honest. The blog post I wrote is basically calling on the developer community to think through this problem and come up with an open standard for both decision flow and decision traces. It doesn’t have to be a new prompt language, it has to be a language where an LLM can automatically define a flow, and once that flow is executed, everything is traced and fed back into the context layer to make it more mature.
That’s the proposal. I’m actually writing something as a project myself, and I’ll open source it and see how people use it and get feedback. It’s very early stage, so I don’t want to say it’ll solve world hunger, but it’ll start from a baseline.
Saket Saurabh:
I think that’s a very relevant set of information if you’re able to capture this over time. Then, depending on what operation or question you’re asking, the AI model can retrieve the right pieces of information. You’re right that a lot of decision tracing lives in Slack and call recordings, and now those data sources are available to be ingested and processed. Do you think it also becomes a bit of a context management problem, because you can very quickly generate a lot of noise? The amount of data generated in Slack, calls, and documents is huge.
Mahesh Mishra:
Yeah, there’ll be a lot of noise. There’s always a human-intelligence relationship that decides what’s really knowledge noise versus what’s a fact. For agent workflows, we need to find mechanisms where the system can filter out noise from the actual fact. I don’t know what the solution is, but ideas are welcome in this space.
Saket Saurabh:
You talked about the human learning process, where we learn, get reinforcement, and also forget things, but at the right time we remember the right pieces of information. Maybe there’s a parallel mechanism here. I can imagine this easily becoming terabyte-scale data over time as you capture that, and then you need to be able to pick the right pieces out of it. Fascinating.
Mahesh Mishra:
Yeah, these are some of the areas where we need to start thinking, because slowly we’ll move from low-stakes decision-making into high-stakes decision-making using AI. That’s when an auditor comes in and asks, how did you decide this person was approved for a loan? If you don’t have the traces, you don’t know. You can’t just say the computer approved it and leave it there.
Saket Saurabh:
Earlier in the conversation you also mentioned different stages of AI adoption at different companies. This is a big question for people: going from prototypes to production-level use cases, where are people missing the point? What are the big gaps? Of course, the knowledge layer is part of it too.
Mahesh Mishra:
We’ve talked enough about the knowledge layer, and at this point the audience is probably aware of what it means and how to think about it. There are two things where we tell ourselves a lot of lies. First, every time there’s a new model, people think, “I need a more intelligent model.” Models are not the most difficult part of your AI journey. At least with cloud platforms out there, every intelligent model is just an API call away. Overthinking about models is one of the reasons people don’t move from prototype to production.
The second is chasing a cool demo. The truth is, when you do a demo, all your assumptions are right there, all your assumptions are true. Production is an entirely different ballgame, where you have messy data, lack of context, columns named a certain way where you don’t know that column is deprecated, so an LLM can always hallucinate. There are access control policies and governance you need to adhere to, and you need to figure out what counts as acceptable performance versus what’s too slow. The LLM can’t decide that on its own.
You need to have all the challenges sorted out before you go to production. What people generally do today is a prototype, they like it, and they stay in that cycle of improving the demo or prototype, never thinking about how to take it to production, because production is an entirely different ballgame.
Third, platforms aren’t necessarily ready for read-write agentic workloads. Lakehouse architecture is great, decoupled storage, a catalog that stores table definitions, decoupled engines that query the data. But the challenge is that when a human operates against it, a human makes the decision and is accountable for it. When you offload the same task to a machine, you need slightly different primitives. You need something like GitHub-style branching, where you create a branch on the data, the machine makes commits, and then a second set of agents reads the committed data and confirms it’s accurate before the merge happens.
All these primitives exist in bits and pieces in the Iceberg world, but nobody is thinking through the new primitives that are needed as more and more agents work on this data going forward.
Then, as we talked about with knowledge engineering, the key problem is that your decisions happen elsewhere and your data catalog is elsewhere. Honestly, the data catalog as we know it is a dying story. That’s probably a big statement, but the truth is, you need an army of people to maintain a data catalog. If your agents are actually making decisions and working with the data, your catalog, or whatever that layer is, needs to evolve at that same pace. You need a layer constantly maintaining that metadata with accurate information, otherwise your entire enterprise is getting the wrong results.
That’s an area where people are struggling. The platform isn’t ready, and we’re caught up in this LLM-driven discovery churn, not thinking about all the unglamorous work that needs to be done to get to production.
Saket Saurabh:
That’s fair, there’s a lot of groundwork to be done. When we talk about agents using data versus a human, a human comes with a lot of context, business know-how, and domain knowledge, and agents don’t have that. Your observation about the catalog is very interesting. Do you see the catalog space just becoming automated, where LLMs update the information, or does it not need to exist at all, or take a different form or shape?
Mahesh Mishra:
Think about a knowledge layer where the dots are connected. The catalog is a place where it enriches. Everybody talks about knowledge graphs. If you look at Collibra’s website or Alation’s website, and I’m not saying these are bad products, they’re solving real problems, everybody calls it a knowledge layer without understanding what’s actually needed for AI agents.
A knowledge layer that needs an army of people to maintain it isn’t really a knowledge layer, because it goes stale very quickly, and your agents end up looking at the wrong knowledge to make decisions. So the catalog as a layer needs to rediscover itself. It has to become a semantic knowledge graph. All you need is a physical metadata store and a semantic knowledge graph that connects the dots: how tables are joined, what each column means, what formula generates a given metric, which is a dimension and which is a fact, plus the decision-making flows that happen on top of that data.
When a marketing user asks what their churn rate is, that flow should be documented in the catalog, so next time someone asks the same question, that flow, or an enhanced version, can be reused. So the catalog being just a metadata repository is going to die. It needs to evolve into something like a knowledge graph, with decision flows, decision traces, summaries, and knowledge scouts that can bring knowledge together from different sources and enrich that graph.
Without those three core integration layers, the catalog as a product will slowly become obsolete. I’m not talking about the technical data catalog, like an Iceberg REST catalog, that will always be there because engines depend on it. I’m talking about business data catalog layers that call themselves knowledge layers today.
Saket Saurabh:
Do you feel like the different components of data management, integration, pipelining, cataloging, observability, will probably converge into a single system that feeds the whole cycle continuously? I’m asking because you mentioned DBT earlier, and the Fivetran-DBT merger just closed. What’s your perspective on how that space evolves?
Mahesh Mishra:
This is my strong opinion, and I may be totally wrong here. My thinking has been that every vendor out there is looking at the world in a very narrow view. DBT, for example, is looking at things from the point of view of metadata generation and transformation, and that’s the world they confine themselves to. Similarly, Databricks assumes all the data needs to be in the Databricks platform with Unity Catalog on top, and they build everything around that, which only works if the enterprise uses Databricks as the platform.
The problem is none of the enterprises are that clean today. Any enterprise has some on-prem, something on the cloud, something on Databricks, something on Snowflake. The data platform itself is so messy that having a myopic view while building these primitives isn’t going to work long term. We need a layer that’s open standard, sitting on top of all these data services, whether it’s Unity Catalog, Alation, Collibra, or Cloudera. There has to be one horizontal layer, open-source-driven, with open interfaces for AI agents to interact with, and integrations into all these platforms to extract and store information. That single layer is missing today. It can be the interface for an entire enterprise’s institutional memory that agents can access.
Saket Saurabh:
That’s a good point. The general thinking in enterprises has been “let’s do this, then this, then this, and eventually we’ll get to doing something interesting with data,” but there’s really no time or patience to build all that, and that fragmentation will always exist. So it’s about figuring out how to work across the fragmentation rather than taking a multi-year strategy to centralize something that’s almost never cleanly possible. Talking about that open architecture, and Iceberg, tell us about where you see people making mistakes in how they think about managing data, and where Iceberg is coming in as a potential solution.
Mahesh Mishra:
We need to understand the evolution. If you go back twenty-five or thirty years, people only had Teradata or Oracle, which is where their data warehouse would be hosted. What was simple in that world was that you didn’t need to know where the table data or metadata resided. You just had a bunch of SQL queries you could run to extract information from somewhere, magically.
Then Hadoop came in, where data could be decoupled and stored independently of compute, so you could scale both layers independently. Then HDFS and Hive Metastore came in, and Hive Metastore built a convention for what a table means: some metadata pointing to files, surfaced as a table. The challenge was that you couldn’t do transactions. If you had streaming data changing constantly, updating a record meant reading the whole partition and replacing it with a new one. Those challenges were very painful for customers.
When object storage became popular, it got even worse, because your storage was no longer in HDFS, which you managed, it was stored elsewhere, so you lost direct control of the storage. Then Iceberg came along, this was Netflix, and at that time three open table formats emerged: Delta, Hudi, and Iceberg. What Iceberg did really well is that it’s not just a table format, it’s a specification, and the specification itself is open source.
They built metadata indexes on top of the storage data, the Parquet files, and defined that an Iceberg table should have data files on the object store, plus a manifest list, manifest, and metadata.json, a three-layer metadata structure, to be called an Iceberg table. This opened up a lot: it had a dependency on a catalog for applying commits to the data and doing an atomic switch of metadata, so now both your data and metadata are decoupled from everything, stored in the object store, and your engines can access the data directly at the storage layer, with no other dependencies needed.
In this new agentic world, there are some cool primitives, branching and tagging, already in Iceberg, but they’re not finished yet. They’re half-baked solutions. You can’t create multiple conflicting branches and merge them to mainline; there’s no conflict detection at that level. Some primitives are missing, but the core capabilities exist.
So the thing we were discussing earlier, about agents interacting with data, let me retake this question, because I sidetracked a little.
Saket Saurabh:
No, it’s a good evolution to walk through. I was thinking about lakehouse architecture, there’s a whole spectrum there. My main question was really what people are getting wrong in setting these things up today, but the history lesson is actually very helpful because it shows the evolution and the challenges each stage resolved.
Mahesh Mishra:
Got it, okay. So all these primitives have been built, branching and tagging in Iceberg data, which means you’re ready for agents. Where people generally go wrong is thinking the lakehouse layer itself is a product. It’s actually not a product, it’s a conscious architectural choice. It’s more complex than a simple data warehouse. A data warehouse abstracts out all the complexity of a table, whereas now you have to manage a lot more yourself. And when you move into the agentic world, you also need policies, governance, and tracing, all additional work.
So your “simple” warehouse in this modern world needs to be thought through differently. You need three core layers: storage, a technical catalog for table definitions, and everything decoupled between them. On top of that, you need a policy and governance layer, which doesn’t have a real open-source specification today, so someone has to build that. There’s no notion of a single identity across the board. Go to any catalog, unless it’s a managed one from a vendor, and you won’t find an open-source specification for what an identity is, what a policy definition is, what the policy grammar is, or how to write fine-grained access control.
These are things people don’t think through upfront when getting into modern lakehouse architecture, and eventually they realize there’s a lot to manage, so they try to retrofit everything back into their old world. Going forward, those are the mistakes people make. Then there are operational challenges. In the old data warehouse world, data was automatically optimized because there was a tightly coupled engine with the compute power to do automatic storage optimization. In the lakehouse world, storage is separate, so you need additional maintenance work, compactions, reordering, sorting, to keep the storage layer efficient. People don’t think about this upfront, and they get caught by surprise when their queries run slow or a policy gets violated.
Saket Saurabh:
So a lot of thoughtful decision-making has to go into this whole process. But one advantage with Iceberg is that multiple engines support it at the storage level. As we move into more AI applications, and the foundations need to be in a good place, do you see Iceberg serving that need in a way that the worlds of data lakes, lakehouses, and databases converge together, storing data in Iceberg and accessing it with different engines?
Mahesh Mishra:
Iceberg is definitely useful and important for structured data. But in this agentic world, unstructured data is equally important as structured data. The way I look at it, there needs to be an open format like LanceDB, which brings vectorization to the object store, storing vectors and providing a table-like primitive for unstructured data, while Iceberg handles structured data. Together, Lance and Iceberg complete the whole story for AI adoption.
Saket Saurabh:
That’s a great point. What’s your take on the rapid adoption of tools like Supabase, for example, running on Postgres, which also deal with structured data and embeddings at the same time? Do you see some of these databases resurging and sitting side by side with Iceberg?
Mahesh Mishra:
We’ve gone through this cycle multiple times in the last thirty years, and we know at this point that no single database can solve all use cases. Your online use cases need something like Postgres; your offline use cases need object storage, probably with a performance hit, which is why we call it offline data storage.
I’m pretty sure Supabase will come along and play a super critical role in this space. But you’ll definitely need two types of storage. One is object storage for offline data, because as agents make decisions, think about how much ancillary data, traces and so on, gets created. If you store all of that in something like Supabase, it becomes very expensive. Postgres now has a vector extension, pgvector, and a graph extension in the Apache community for Postgres, so all of this makes sense for online workloads needing low latency and high concurrency, but against a limited data set, probably a few terabytes at most.
For historical analysis, decision traces going back five years, you’d go to an object store, using something like LanceDB for vectorization, Iceberg for tabular data, and something like Apache AGE or a graph engine that’s becoming popular for graph data. All these storage primitives will be needed for historical analysis on the offline layer.
Saket Saurabh:
And I think that becomes an important part of the whole knowledge engineering piece we discussed earlier. Let’s talk a bit about policy and governance. In practice, as people are building this, what does realistic governance look like right now?
Mahesh Mishra:
There are two models in governance here. One is an agent acting as a service, doing work on behalf of a user. The other is an agent becoming a separate user, like another person in your workforce, doing the work itself. Identity is super important here.
If you want to treat an agent as part of the workforce, one more employee in your organization, you need to define these system users as human-like users in your identity providers and use them across the board. Those agents will assume roles and execute work on behalf of that identity.
The second model is: you, as an agent creator, create an agent and publish it for others in the organization to use, but as the invoker of that agent, your own policy has to prevail. That’s a more complicated problem, where the agent does the work but impersonates you. That’s where “on behalf of” primitives are becoming popular, but there isn’t a very hardened system built for this yet, at least not that I’ve seen, though I haven’t seen everything.
This is a super critical area going forward, because you don’t want to give super-privileges to an agent just because everybody in the organization wants to access it. You need to allow an agent to work within a guardrailed sandbox, with a dynamic policy enforced on top, built around who invoked that agent. NeMo Guardrails is one example of similar primitives being built, but those are more for personal agents. We need to think about how enterprise agents will have that “on behalf of” flow, with external injection of policies and controls and gateways to enforce them for whatever action the agent performs. That’s an area that needs investment.
Saket Saurabh:
So agents can be designed by a few people for a particular function, and then everybody in that function can take advantage of it using their own credentials, acting on behalf of that user, keeping with policies rather than service keys, which are much easier to lose track of in terms of how access is managed. That’s certainly an area of active work. When it comes to governance and capabilities like NeMo Guardrails, how do you see the balance between speed of innovation and keeping compliance together? How do you encourage teams to experiment fast but stay compliant?
Mahesh Mishra:
I see compliance and security as accelerators. They don’t get in the way of innovation if you do them right. The way to think about it is you need to think upfront about what primitives are needed, define the guardrails around them, and then set your users and agents free to operate within that guardrail. Think about a soccer game: there are rules, but you don’t control every single move of the players.
The way to think about security and governance is that enterprises serious about agentic workloads need to think through the broader security and compliance controls needed and build integrations for that, rather than controlling every single move the agent makes. From today onward, if we’re serious about AI, and we want to be a differentiated organization in AI over the next five years, we have to treat AI like any other team member. The way we define rules and compliance policies for team members, we need to define those for agents too.
It’s easier said than done, but as companies we need to think through how we want to do it. NeMo Guardrails has some primitives, OpenSSF has some others, and we need to bring them together and build a hardened control layer for our agents. After that, we don’t need to review every single agent flow. Does that make sense?
Saket Saurabh:
Yeah, that totally makes sense. I’d love to ask, given all the research you’ve done in the space, and your work as a thought leader, what question do you think data leaders should be asking you, but aren’t?
Mahesh Mishra:
I think there are three or four questions data leaders should ask. First, there’s a kind of madness where enterprises assume they’re heading toward lakehouse architecture, so Spark, or distributed processing generally, should be the preferred engine. That creates complexity. I have a patent in distributed data processing, and the way I look at it: if you don’t need distributed processing, don’t do it, because you’re signing up for a complexity tax.
The first question data leaders should ask themselves is, is my data really big data? If you seriously ask this of enterprises today, 70% of the data on top of which most critical decisions are made is actually not big data. If you’re a B2B company, you don’t have trillions of customers, you have a handful. If you’re a B2C company, you don’t have trillions of orders every day, maybe a few million at most.
So your most critical data sets aren’t big data. Why sign up for that complexity tax? Go back and look at where 90% of your business decisions are being made, on big data or small data. DuckDB is doing a good job here, recognizing that most companies have tabular, structured data, often monthly or weekly summaries, small data sets of a few hundred gigabytes, where single-node processing, like a library installed on a computer, should just work. That shift needs to happen in the market. We shouldn’t just follow what others are doing; we should ask whether it actually makes sense for our use case.
The second thing is, how big is my lakehouse? When I talk to data leaders and ask this, they’ll say their lakehouse holds ten petabytes of data. That’s not a badge of honor. Nobody’s asking the hard question of whether all ten petabytes are actually relevant, or why they’re spending so much money if they’re not making decisions on most of that data. Every data leader treats a five or ten petabyte lakehouse as a badge of honor. We need to look at that number, go back, ask our engineering teams, and often thirty to forty percent of that data could be deleted, saving thirty percent of storage costs immediately. Those are some hard questions that need to be asked.
The third thing is, is my data platform ready for AI workloads? This distributed tax becomes a problem because when things get expensive, people blame the technology rather than looking at what’s wrong in their own platform. Today, humans interacting with a data platform might generate ten thousand to a hundred thousand queries a day. If you make it incredibly simple for users to ask natural-language questions, and agents start firing those queries, you’ll see a hockey-stick growth in the number of queries hitting your lakehouse. Decoupling storage and compute helps, but if you’re still using a distributed compute layer, you’ll bleed money because you’re generating far more workload than before. You need to think about how to do single-node processing for most of the important decisions on your data platform. That shift needs to happen.
The last one is knowledge engineering, which we’ve already talked about quite a bit. We need a knowledge layer on top of the data platform to inform the agent which data set to use, when to use it, why, when it was last used, and whether the outcome was liked. All of that needs to be curated and stored in a single place so agents can interact with the platform well.
Saket Saurabh:
This is very well put, Mahesh. I agree that companies often get oversold on technologies they don’t necessarily need, and that same complexity you’re describing is what prevents people from taking advantage of AI, because their data isn’t ready, since they’ve built a much more complex system than they needed. It’s very valid advice for every data leader: figure out what part of your data is really impactful, has high ROI, and if you keep that clean and organized, there’s a lot of value to generate. And you’re right, as it gets simpler to ask questions, more people will ask them, more SQL queries will get generated under the hood, and costs will go up. Very wise advice.
Mahesh Mishra:
Thank you.
Saket Saurabh:
So, given everything you’re seeing on the AI side, and your thought leadership, where do you see things three to five years out from a data-platforming-and-AI perspective? You mentioned earlier the automation coming into how we manage data. What’s your vision five years out?
Mahesh Mishra:
I’ll break it into two parts: what will probably happen in the next five years, and, since you didn’t ask but I’ll answer anyway, who are the companies that will win and what do they need to do.
In the next five years, the first year or two will bring a lot of automation: data engineering automation, text-to-SQL automation, visualization automation, troubleshooting, fixing, optimization. There will also be a ton of knowledge engineering work, semantic views, semantic materialized views, semantic layers, knowledge engineering platforms, all of these will come. But take that consciously, and think about what you really need. It’s a lot of noise that can distract you quickly, since everybody’s talking about semantic views, metric views, semantic layers, or knowledge layers right now. If you have good engineers who can write metric views your AI agents can make sense of, go for it. If you think you need a more robust knowledge engineering mechanism, go for that. But don’t try to invest in everything, because if you run in all directions, you won’t get anywhere.
The third set of things happening over the next two to five years is data-driven workflows: high-stakes decision-making and actions. A lot of companies will build agents that take high-stakes actions, not just decisions. For example, going to a doctor with your medical history and a symptom, and asking what they think. I think in the next three to four years, that decision will largely be offloaded to an agent, which looks at your history, current medication, and symptoms, and suggests a prescription, with the doctor mostly serving as the face of it, since human life is involved. Vendor selection in supply chain is another example, high-stakes decisions that will be automated, offloaded to agents.
Where humans will remain central is high-value decision-making: product strategy, go-to-market. Even though many suggestions will come from AI agents, humans will still make those high-stakes calls, because not everything will be known to the agent.
Engineering development cost is going to go significantly lower, because AI agents will be more efficient at writing code and producing better software. There will be AI costs, but actual engineering headcount costs will be lower. LLMs will be commoditized, and their cost will likely go down. That’s my forecast at this point. Last but not least, you’ll see creative infrastructure companies placing data centers in unusual places, mountains, the ocean, even space, because of how much energy, water, and cooling these data centers will need.
On the other side, the companies that build differentiated experiences will be the ones that think now and build the right operating model for AI adoption, organizing their teams to treat AI as a team member rather than a machine sitting outside. They’ll build harnesses to control security and compliance for agents, defining primitives and controlling the high-level outcomes, without micromanaging every single move the agent makes.
Knowledge engineering platforms will be super popular, with integrations into wikis, Jira, Slack, GitHub, and other places where knowledge is documented, bridging structured and unstructured data through that knowledge layer. Last but not least, the successful companies will move fast, fail fast, and course-correct quickly. If they don’t, they won’t build differentiation over the next five years.
Saket Saurabh:
Fascinating, very well said. I really like the point you made earlier about everybody building a semantic layer. As data leaders, one of the important things is not to get sucked into FOMO from LinkedIn or Twitter, where it looks like everybody’s buying the same technology or building the same solution. Your advice is very practical: stay grounded in what you’re doing and what creates ROI for you. That’s a great way to wrap the conversation. Mahesh, how can people stay in touch with you? Where can they follow your insights?
Mahesh Mishra:
I’m on LinkedIn, people can reach out to me there. I write a lot of blog posts, so you can stay connected through those. More recently I started writing on Medium as well. I’m not really a social media person, so there’s not much presence on Facebook or Instagram. LinkedIn and my blog are the best ways to reach me.
Saket Saurabh:
That’s perfect. Well, thank you so much, Mahesh, for joining us today. Fascinating conversation, I appreciate you taking the time.
Mahesh Mishra:
Absolutely, it’s a pleasure. Thank you for having me. We really had a great conversation. Looking forward to more.
