Lakehouse Architecture is a topic the MagnusMinds engineering team writes about from project work: how it is set up, where it breaks, and what we changed in production. This tag page lists all 2 articles tagged Lakehouse Architecture.
Start with “We built a multi-tenant SaaS on Microsoft Fabric with a workspace for every customer” or “From Manual Reporting to Real-Time Insights with Microsoft Fabric and Power BI”. Open a post for the full write-up, or follow another tag or category on the card to a related topic.
Most Microsoft Fabric projects serve one organization. Ours had to serve many. A cloud cost management (FinOps) software provider asked us to build the data platform behind their product: collect each customer's Azure and AI spending, turn it into one standard format, and serve it to dashboards and an AI assistant. Every customer had to see only their own data, onboarding had to be automatic, and the platform had to grow region by region. We built it entirely on Microsoft Fabric, with a dedicated workspace for every customer. This post walks through the design, the parts that made it work in production, and what we would tell any team building SaaS on Fabric. The challenge (pain points in real operations) A single-tenant Fabric design is straightforward. Multi-tenant SaaS raises harder questions, and each one had to be answered before the first customer went live. Isolation that can be proven: Customers trust you with billing data. "We are careful" is not enough. The platform had to show, automatically, that one customer's reports can never read another customer's data. One codebase, many customers: Every customer needed the same pipelines, the same data model and the same app. Hand-built workspaces drift apart within weeks. Onboarding without a project: Each new customer needed groups, storage, pipelines, a semantic model, an app and credentials. Doing that by hand would turn every sale into a mini-project. Capacity cost: Fabric capacity is billed whether it is busy or idle. Sizing it for the heaviest nightly run all day would waste money. Growth by region: Customers in different geographies expect their data to stay in region, and an incident in one region should not affect the others. What we built (tailored solution, delivered end-to-end) We designed the platform around three ideas: a regional deployment holds everything a set of customers needs, every customer gets their own workspace, and nothing is created by hand. 1. Regional deployments (growth and blast radius) Customers are placed on regional deployments, sometimes called deployment stamps. Each deployment has its own Fabric capacity, Azure Key Vault, Azure Function Apps and a core Fabric workspace. The core workspace holds the shared notebooks and pipelines, plus a Fabric SQL database that acts as the customer registry. The registry lists every customer with their workspace and lakehouse IDs, status and settings. Pipelines never hard-code a customer. They read the registry at run time. Adding a region means standing up a new deployment from the same templates. An issue in one deployment stays inside it. 2. A workspace per customer (isolation by design) Every customer gets a dedicated Fabric workspace containing: Bronze, Silver and Gold lakehouses in a Medallion Architecture A Direct Lake semantic model that holds every business measure The customer-facing Fabric App, built with Microsoft's Rayfin SDK A Fabric Data Agent that answers plain-English cost questions from that customer's model only Access is controlled through Microsoft Entra security groups per customer. Each customer's Azure credentials live in Key Vault and are read at run time, never stored in code. The important part is that isolation is tested. An automated check confirms that each customer's semantic model is bound only to that customer's own Gold lakehouse, and a read-only smoke test runs across every active customer on a deployment. 3. One pipeline run for every customer (one codebase) The transformation logic is written once and runs for everyone. A master pipeline reads the active customers from the registry, and a ForEach activity runs the per-customer pipeline for each of them: Converting Azure, OpenAI, Anthropic and Microsoft Copilot spend into FOCUS, the open standard for cloud cost data Cost allocation to teams, budgets, anomaly detection and savings recommendations Reservations, price sheets and Microsoft 365 usage A semantic model refresh at the end Every notebook and pipeline lives in source control. Processing is incremental: a watermark table records what each customer has already processed, so nightly runs only touch new data. Ingestion that needs customer credentials stays in Azure. Azure Durable Functions call the Cost Management API, Azure Resource Graph, Microsoft Graph and AI vendor usage APIs, and write Delta tables straight into each customer's OneLake. Azure cost export files reach Fabric through OneLake shortcuts, so raw files are never copied. 4. Onboarding as a workflow (one run instead of a project) Onboarding a customer is a single automated workflow. It: Creates the Entra groups Adds the registry entry Creates the workspace and the three lakehouses Stores the customer's credentials in Key Vault Deploys the semantic model, the app and the Data Agent Runs the isolation check Offboarding reverses it. One detail saved us more than once. A semantic model fails to deploy if it references a table that has no data yet. We generate a skeleton schema listing every table and column the model expects, create empty tables from it at onboarding, and backfill new tables into existing customers. A CI test also checks that every column a notebook writes exists in the model, so a schema change cannot break every customer overnight. 5. Capacity that follows the workload (cost control) Each deployment has two capacity sizes: a smaller resting size for daytime reading and a larger size for the nightly processing run. The capacity scales up before the run and scales down as soon as the master pipeline finishes, rather than after a fixed duration. The app also caches semantic model query results against a data-version stamp, which removes repeated identical queries and keeps the capacity free for real work. 6. Customer-hosted delivery (data residency) Some enterprise customers cannot allow their reporting data to sit in a supplier's tenant. For them, processing stays in the provider's tenant and only the finished Gold data is published into the customer's own Fabric tenant. The semantic model and app run there on the customer's capacity, and no copy is left on the provider's side. That deserves its own post, and we will cover how we built and tested it separately. Real-world impact (what changed after rollout) Isolation you can show an auditor: An automated check proves each customer's model reads only their own data, on every deployment. Onboarding in one workflow: A new customer goes from contract to live workspace, model, app and AI assistant without hand-built steps. One codebase for every customer: A fix or feature in a notebook, pipeline or model reaches all customers in the next release. Lower capacity cost: Compute is sized up only while the nightly run is actually running. Room to grow: New regions are new deployments from the same templates, with problems contained to one deployment. What we would tell any team building SaaS on Fabric Decide your isolation unit first. For us it was the workspace. Everything else, from access groups to deployment scripts, followed from that choice. Keep a registry and read it at run time. Pipelines that look up their customer list never need editing when a customer joins or leaves. Treat every Fabric item as code. Notebooks, pipelines, lakehouses and the semantic model all live in Git and deploy the same way to every workspace. Test isolation automatically. A check that runs on every release catches a binding mistake before a customer does. If you are planning a multi-tenant product or a large data platform on Microsoft Fabric, we would be glad to share what we learned. Connect with us to talk through your architecture.
From Manual Reporting to Real-Time Insights with Microsoft Fabric and Power BI The Challenge: Fragmented Data and Manual Reporting Many organizations struggle with the same problem Critical business data is spread across HR systems, finance platforms, operational databases, compliance tools, and spreadsheets. Reporting becomes a manual and time-consuming process. Decision-makers often wait days or weeks for actionable insights. KPI definitions vary across departments, creating inconsistencies. Limited visibility and delayed reporting reduce confidence in business decisions. The Solution: A Unified KPI Reporting Platform To address these challenges, at MagnusMinds we implemented a centralized KPI reporting platform using Microsoft Fabric and Power BI. Key Objectives Establish a single source of truth for organizational performance. Consolidate data from multiple business functions. Standardize KPI calculations and reporting processes. Enable scalable and future-ready analytics. Building a Scalable Data Foundation with Microsoft Fabric Using Microsoft Fabric's Lakehouse architecture, data was unified into a centralized analytics platform. Architecture Approach Bronze Layer: Raw data ingestion from source systems. Silver Layer: Data cleansing, validation, and business-ready transformations. Gold Layer: KPI-ready datasets optimized for reporting and analytics. Delivering Actionable Insights with Power BI Power BI was used to create an executive-ready KPI dashboard that provided immediate visibility into organizational performance. Dashboard Capabilities Monitor Actuals vs Targets. Track KPI performance in real time. Analyze historical trends and variances. Drill down into operational details. Improve visibility across business functions. Strengthening Data Governance and KPI Consistency A key focus of the solution was governance and standardization. Governance Improvements Standardized KPI definitions. Centralized calculation logic. Defined KPI ownership and accountability. Consistent reporting structures across the organization. Outcomes Improved reporting accuracy. Reduced ambiguity in KPI interpretation. Increased trust in business data. Enhanced decision-making confidence. Business Impact and Measurable Benefits The impact was significant: Reduced manual reporting effort. Improved data accuracy and consistency. Faster access to business insights. Better executive decision-making. Enhanced cross-functional visibility. Scalable foundation for future analytics initiatives. Final Thoughts Organizations today don't need more reports they need better visibility into the data they already have By combining Microsoft Fabric and Power BI, businesses can move beyond fragmented spreadsheets and disconnected systems to create a modern, automated KPI platform that supports real-time reporting, data governance, and strategic decision-making. The result is not just a dashboard, but a scalable business intelligence solution that turns data into action.
