Embedded ETL at Scale | Hotglue cover

Hotglue's Architecture for 38,000 Tenant Integrations

Hotglue Team profile image

by Hotglue Team

Oct 24th 2026

Most teams don't realize their integration layer is a problem until a customer calls to say their sync is broken. At 38,000 active tenants, you can't afford to find out that way. Getting the architecture right early is what separates teams that scale integrations from teams that babysit them.

TLDR:

  • Integrations show up in 60% of B2B SaaS sales cycles, making them a product decision your CPO owns, not IT
  • Building connectors in-house hides the real cost: API drift and schema changes quietly consume sprint capacity over time
  • Scaling past a few hundred tenants breaks job isolation and observability long before it breaks your connector logic
  • On-premise connectors like QuickBooks Desktop and Sage 300 CRE are missing from most vendors' lists, but not all verticals can swap them out
  • Hotglue runs 38,000+ active tenants processing ~10 billion records weekly on AWS ECS via Fargate

What ETL Actually Means in a B2B SaaS Context

ETL covers three steps: extract data from a source, reshape it, and load it somewhere useful. Simple enough in theory.

Most people encounter ETL through the internal analytics lens, where a data engineering team pipes Salesforce records and payment events into a warehouse so a BI tool can build dashboards. Valid use case. Just not the one that keeps a CPO up at night.

In a B2B SaaS context, ETL means something different. Your customers connect their own tools: QuickBooks, Shopify, HubSpot, Paylocity, and expect your product to sync their data automatically. You're moving each customer's data, across thousands of tenants, into your product's data model. Every tenant has slightly different configurations, field mappings, and sync schedules. The reshaping step alone can vary customer by customer.

That's the piece most ETL explainers skip entirely.

Embedded ETL vs. Traditional iPaaS

Traditional iPaaS tools were built for IT and ops teams automating internal workflows: connecting Slack to Jira, syncing HR records, triggering alerts when a form is submitted. They assume one team configures the integration once and owns it forever.

Embedded ETL works differently. Your customers authenticate their own accounts inside your product. Each one gets isolated state, their own sync schedule, and their own field mapping. The connector logic is reusable across thousands of tenants, but execution is per-tenant. Your product team owns the roadmap, not an IT administrator.

That distinction has real commercial weight. According to a 2024 survey of B2B SaaS leaders, 84% said integrations were "very important" or a "key requirement" for customers, and integrations came up in roughly 60% of sales cycles. Picking a tool designed for internal automation and stretching it into a customer-facing product layer is where engineering hours quietly disappear. A decision framework for in-house vs. embedded iPaaS can help clarify the tradeoffs.

Why Integrations Are a Product Decision, Not an IT Decision

When a prospect asks "do you integrate with NetSuite?" during a sales call, that question lands on the CPO's desk, gets escalated to partnerships, and stalls the deal until someone answers it convincingly.

Integrations have become a product feature, one that shows up in sales cycles and determines whether your product grows with a customer's stack. According to a 2024 survey of B2B SaaS leaders, integrations came up in roughly 60% of sales cycles. Customers who connect their existing tools are stickier, and every integration they set up is another reason not to leave.

That's why Heads of Partnerships and CPOs now own the integrations roadmap. Not IT, not a shared services team, and definitely not whoever happens to be on-call that weekend. They see the churn risk and the expansion revenue in the same conversation. IT never had that context. Product does. If your org still routes integration decisions through IT, that's the first thing worth fixing.

The Hidden Cost of Building Customer Integrations In-House

The most common objection we hear: "our engineers already handle integrations." And they probably do. The question is what that actually costs over time.

Third-party APIs change endpoints with little warning. Schema fields get renamed, deprecated, or moved. A Shopify connector that works cleanly for one customer breaks for another the moment they add Loop Subscriptions, because now there's a subscription object your mapping logic never accounted for. Each customer variation is effectively a new edge case, and they accumulate faster than anyone budgets for.

The build cost is visible. The maintenance cost is not. API migrations, schema drift, connector rebuilds after a vendor updates their auth flow: none of these show up as line items, which is exactly why the real cost of maintaining integrations is so easy to underestimate. They show up as sprint capacity that quietly disappears. (Your engineers will describe it as "just a quick fix" right up until Q4.)

That's the gap the build-vs-buy conversation usually misses. Engineering time spent keeping existing connectors alive is time not spent on the product. And unlike a new feature, a broken integration generates immediate customer complaints.

What to Look for in an ETL Solution for B2B SaaS

When choosing an ETL layer for customer-facing integrations, the criteria look nothing like a standard data warehouse assessment. Here's what actually matters.

ETL Evaluation Criteria for B2B SaaS

CriteriaWhat to Look ForRed Flag
Connector breadth & depthRead and write support, on-premise coverage (QuickBooks Desktop, Sage 300 CRE), active maintenance when APIs change upstreamLong logo list with shallow connectors: no write support or on-premise options
Reshaping flexibilityPer-tenant data reshaping via Python-based scripts that run per job, handling customer-specific field mappings without rebuilding the connectorFixed schema that forces all tenants into one mapping
Multi-tenant scalabilityIsolated state, individual sync schedules, and separate job logs per customerShared execution context where one tenant's failure cascades to others
Monitoring & observabilityJob-level health metrics, failure alerts, and per-tenant logs surfaced proactivelyNo alerting: your CS team hears about broken syncs from angry customers
Security & complianceSOC 2 Type II and GDPR compliance; solution processes and delivers data without storing itVendor stores customer data long-term, creating a different risk profile
Deployment flexibilityWhite-labeled widget, API-triggered syncs, and Magic Links for lower-friction onboardingSingle deployment mode that forces tradeoffs your customers will eventually feel

ETL vs. ELT for Customer-Facing SaaS Integrations

For customer-facing integrations, the ETL vs. ELT debate matters far less than vendor comparisons suggest.

ELT loads raw data first, then reshapes it inside the destination via dbt or SQL. That model works when your warehouse is the endpoint and your data team owns the reshaping logic. Fivetran is built around this pattern.

ETL reshapes data before delivery. Your customers don't have a warehouse; they have your product's data model. Their data needs to arrive shaped to what your backend expects, per tenant, per sync. Adding transformation after load brings latency and complexity you don't need.

The distinction does matter if you're routing customer data into a shared analytics layer. But most B2B SaaS teams pulling Shopify orders or QuickBooks invoices into their own database want pre-shaped records on arrival. Hotglue's Python-based transformation layer runs at the pipeline level, before data reaches your backend, which is the right place for customer-facing use cases. It's one of several reasons teams weighing unified API vs. embedded iPaaS land on an ETL-first approach.

Multi-Tenant Scale: What It Actually Takes to Run Thousands of Integration Pipelines

Scaling from 10 tenants to 10,000 doesn't break your connector logic. It breaks everything around it.

A technical architectural diagram illustration showing multiple isolated pipeline channels running in parallel, each represented as a glowing blue data stream flowing through separate contained tubes or lanes, set against a dark background. Each lane is distinct and separated, symbolizing tenant isolation. The streams have nodes or connectors along them, suggesting data processing stages. The overall aesthetic is modern, clean, and technical — like a high-tech infrastructure visualization with subtle grid lines and soft neon blue and teal tones.

The first thing that goes is job isolation. A shared pipeline with per-customer config flags works until one tenant's slow API response blocks the queue for everyone else. Then you need separate execution contexts, per-tenant scheduling, and failure handling that doesn't cascade.

The second thing is observability. At 50 tenants, a broken sync is a support ticket. At 38,000, you need job-level health metrics and proactive alerting before customers notice anything. Manual log review stops being a viable strategy somewhere around month three.

Here's what actually breaks first in a DIY setup:

  • A single slow tenant stalls syncs for unrelated customers, turning one bad actor into an outage for everyone else
  • Partial failures that report success, quietly poisoning downstream data before anyone catches it
  • Retry logic that doesn't preserve state, causing duplicate records on re-run
  • No per-tenant sync history, so debugging requires guesswork instead of evidence

A properly isolated architecture runs each tenant's job independently, tracks state separately, and surfaces failures with enough context to act on. That's not a feature. It's the floor.

On-Premise and Legacy Connectors: The Connectors No One Wants to Talk About

Most embedded iPaaS vendors publish a connector count, then quietly footnote that QuickBooks Desktop isn't included.

It's a reasonable omission from a technical standpoint. On-premise software doesn't have REST APIs. QuickBooks Desktop runs on Windows, requires a local agent, and authenticates through the QuickBooks Web Connector. Sage 300 CRE connects directly to a local database. Neither fits the cloud-first connector model most vendors build around.

A technical illustration showing an on-premise server or desktop computer connected to a cloud infrastructure through a secure tunnel or bridge. The desktop machine sits on one side, represented as a physical tower or workstation with glowing connection ports, while on the other side floats a modern cloud platform with interconnected nodes. The connection between them is depicted as a glowing data conduit or cable crossing from local hardware to cloud, symbolizing a hybrid bridge architecture. Dark background with blue and teal neon accents, clean and minimal technical aesthetic, no labels or text.

The problem is that construction SaaS, nonprofit tech, and SMB accounting are full of customers still running these systems. A facilities management product whose customers use Sage 300 CRE can't substitute QuickBooks Online, a challenge covered in our Sage 300 CRE integration guide. These aren't legacy edge cases; they're the majority of accounts in specific verticals.

Getting on-premise connectors right requires local installation, database-level access, and resilience to software updates that would silently break a cloud-dependent connector. Hotglue's Sage 300 CRE connector installs on-premise, connects directly to the local database, and supports writing data back, including jobs, vendors, and cost codes. The QuickBooks Desktop connector handles multi-user environments through the Web Connector, something cloud-only vendors can't touch. Sage 200, common in UK markets, is supported via a cloud proxy.

If your customers run legacy software, the connector list that matters isn't the one on the marketing page.

MCP and AI Agents: The Next Integration Layer for B2B SaaS

Anthropic launched MCP in November 2024 to solve what engineers call the N×M problem: if N AI models need to talk to M different data sources, you'd previously need a separate connector for every combination. What MCP means for B2B SaaS integrations is worth understanding in more depth before building against it. As of early 2026, MCP has reached 97 million monthly SDK downloads and is supported by major AI vendors, including OpenAI, Google, Microsoft, and AWS.

For B2B SaaS product teams, MCP changes a concrete question: when your customers start using AI agents inside their workflows, how does that agent authenticate with your product's connected systems? This is where MCP auth for embedded iPaaS becomes a product decision as much as a technical one. Without MCP, you'd build point-to-point connectors for every AI tool your customers might use. With a single MCP endpoint, an AI agent can access live business data across whatever systems your customer has already connected.

Hotglue's Composite MCP exposes a single authenticated endpoint per tenant at mcp.hotglue.com/mcp. Tools are registered automatically from whatever connectors that tenant has linked, namespaced per integration. A customer who has connected Notion and QuickBooks gets a Composite MCP endpoint with scoped tools for both, without re-authenticating or re-configuring anything.

How hotglue Runs 38,000 Tenant Integrations Without an Army of Engineers

Hotglue runs 38,000+ active tenants, processes approximately 10 billion records weekly, and serves 70+ B2B SaaS customers. The architecture is a big part of how.

Hotglue runs on AWS ECS via Fargate, and each tenant's sync runs as its own job with its own state. Sync schedules are configurable per connector via cron, so a customer syncing Salesforce hourly and QuickBooks nightly gets both on their own cadence. Connectors are open-source and Singer and Airbyte-compatible, meaning they can be forked, extended, or replaced without touching core infrastructure, which is part of how hotglue keeps its growing connector catalog current. Data reshaping logic runs at the pipeline level in Python, with GluestickAI generating mapping code from plain language when the field logic gets complex.

Chargebee's RevRec team cut their time to implement new customer integrations by roughly 90% after switching to hotglue. That number reflects what reusable connector architecture actually buys you at scale: the tenth tenant connecting QuickBooks Online costs almost nothing to stand up, because the connector already exists and the field-mapping layer handles per-tenant configuration without rebuilding anything.

Pricing follows the same logic. We charge per active tenant in a 30-day rolling window, not per record or per GB moved. As your integration catalog grows, the cost curve stays predictable. A spike in sync volume for one customer does not change your bill for everyone else.

The Composite MCP endpoint fits the same pattern. Re-engineering auth for every AI tool a customer might use is off the table: a single endpoint at mcp.hotglue.com/mcp exposes tools scoped to whatever connectors that tenant has already linked. One connection, live data, no re-authentication. The same multi-tenant isolation principle applied to agent-facing workflows instead of scheduled batch pipelines.

Final Thoughts on Embedded ETL and Multi-Tenant Integration Architecture

Most integration problems are architecture problems in disguise. When each tenant's sync runs as its own job, failures stay contained, debugging becomes manageable, and your CS team stops hearing about broken syncs from customers before you do. Hotglue's combination of per-tenant jobs and state, on-premise connector support, Python-based reshaping, and Composite MCP makes it the strongest embedded ETL option for B2B SaaS, built for product teams at scale and not retrofitted from an internal IT tool. Your customers get reliable syncs, your engineers get their sprints back, and you get an integrations roadmap that actually moves. Talk to the hotglue team to see it firsthand.

FAQ

What does embedded ETL mean, and how is it different from a traditional iPaaS for SaaS products?

Embedded ETL moves each of your customers' data (from sources like QuickBooks, Shopify, or HubSpot) into your product's data model, per tenant, with isolated sync state and per-job transformation logic. Traditional iPaaS tools were built for internal IT workflows where one team configures one integration once; embedded ETL assumes thousands of tenants, each with their own field mappings, sync schedules, and edge cases. Stretching an IT-focused tool into a customer-facing product layer is where engineering hours quietly disappear.

What's the build-vs-buy case for Hotglue versus building native SaaS integrations in-house?

The build cost is visible; the maintenance cost is not. API endpoint changes, schema drift, and connector rebuilds after a vendor updates their auth flow don't show up as line items; they show up as sprint capacity that evaporates. Hotglue's integrations team monitors third-party API changes and migrates connectors automatically, and reusable connector architecture means the tenth tenant connecting QuickBooks Online costs almost nothing to stand up because the field-mapping layer handles per-tenant configuration without rebuilding anything.

How do product teams handle QuickBooks Online and QuickBooks Desktop integrations differently?

QuickBooks Online uses a standard cloud OAuth flow; QuickBooks Desktop runs locally on Windows, requires a Windows agent, and authenticates through the QuickBooks Web Connector. It's a fundamentally different technical path that most embedded iPaaS vendors skip entirely. For B2B SaaS products serving SMB accounting, construction, or nonprofit customers, that gap matters: you can't substitute QuickBooks Online for a customer still running Desktop across a multi-user office environment. Hotglue supports both, including multi-user Desktop environments and writing Purchase Orders back to Desktop.

How does Hotglue run 38,000 tenant integrations without breaking under scale?

Hotglue runs on AWS ECS via Fargate, with each tenant's sync running as its own job, with per-tenant state and sync scheduling. At scale, the things that break first in a DIY setup are job isolation (one slow tenant stalls everyone else), partial failures that silently report success, and retry logic that creates duplicate records. Hotglue guarantees that a partial sync failure never silently succeeds: it either delivers all data or fails the job and preserves state for retry.

Which embedded iPaaS tools support AI agent and MCP authentication workflows?

MCP (Model Context Protocol) lets AI agents authenticate with business systems through a single endpoint, skipping the point-to-point connector required for every tool. Hotglue's Composite MCP exposes a single authenticated endpoint per tenant at mcp.hotglue.com/mcp, with tools scoped automatically to whatever connectors that tenant has already linked. A customer who has connected Notion and QuickBooks gets a Composite MCP endpoint with scoped tools for both, without re-authenticating or reconfiguring anything.