How AI Is Transforming Data Engineering: From Code to Prompts
AI is shifting data engineering from code-heavy ETL to prompt-driven pipelines. Explore where LLMs fit, common pitfalls, and how Nexla makes AI-ready data workflows practical.
Let’s be clear—this isn’t a comparison of IBM and Fivetran in terms of market share or company size. The enterprise world is still dominated by traditional ETL tools, and cloud-based ELT hasn’t reached broad adoption in that space. Fivetran is also still several years away from reaching $1 billion in ARR.
But the question remains: Now that Fivetran acquires Census, is Fivetran shifting its focus toward branding and bundling rather than innovating and integrating?
IBM has long been a master of working in mature markets. It excels at strategic branding, bundling offerings, prioritizing features that resonate with its core customers, and integrating or replacing products over time. Occasionally, it also breaks new ground with true innovation.
Fivetran, on the other hand, seems to have leaned heavily on bundling—without yet following through on the integration part of the equation.
In 2021, Fivetran acquired HVR, aiming to break into more enterprise-grade deals and rebuild the core Fivetran product. But fast forward four years, and not much has changed.
HVR remains a bundled product. Its connectors don’t interoperate with Fivetran’s native connectors within the same pipeline. Real-time capability and robust on-premise support are still limited. Customers are effectively managing two separate products, even if they’re billed under a single edition.
Reverse ETL (R-ETL) is a powerful add-on for teams using the data warehouse as their system of record. Done well, it can save significant engineering effort and cost. But when R-ETL isn’t integrated into the ELT platform, customers end up either building their own solution or using a completely separate tool—with all the extra work of data remapping that entails.
Moreover, the R-ETL term itself is a bit misleading in this context. Fivetran lands raw data into staging tables. dbt then transforms and moves this data to new tables, typically very different from the original source. Without access to all relevant metadata, Fivetran can’t easily support mapping the data back for reverse ETL.
Nexla, in contrast, was built from the ground up to handle ETL, ELT, and R-ETL within a unified architecture. Its bidirectional connectors, metadata-aware architecture, and AI-powered data mapping make integration seamless.
Whether you’re moving data back to the source system or to entirely new destinations, Nexla’s intelligent metadata system auto-generates data products and manages mappings efficiently. This simplifies R-ETL dramatically—no remapping headaches, no duplicate tooling.
If history is any guide, Fivetran’s acquisition of Census would likely follow the same bundling model we’ve seen with HVR. Integration might not follow anytime soon. That’s an important consideration for anyone hoping for a tightly unified ELT and R-ETL stack.
Check out this insightful post by Taresh Multy on Medium to see a deeper dive into how reverse ETL should really work in a modern data architecture.
Read our in-depth guide on reverse ETL to discover practical use cases and examples.
Let’s connect—reach out to see how Nexla can simplify your data integration strategy.
AI is shifting data engineering from code-heavy ETL to prompt-driven pipelines. Explore where LLMs fit, common pitfalls, and how Nexla makes AI-ready data workflows practical.
A research-backed framework for evaluating LLM-generated data transformations. Learn how datasets, sandboxed execution, and automated judging reveal failure patterns and model performance across real-world data engineering tasks.
In this third episode of DatAInnovators & Builders, BigPanda’s Alexander Page shares how his team designs AI agents that internalize corrections, evaluate tool use, and scale reliably in production.