
Data & AI architecture for modern analytics
Data & AI architecture, learned the practical way
I’m Bipin Pothody, a Data & AI Architect with more than 25 years of experience in technology. For over 15 of those years I’ve been architecting enterprise data platforms, and for the last decade my focus has been on lakehouse architecture, data governance and AI-driven analytics.
This site is where I write down what I’ve learned along the way: the patterns that held up in production, the decisions I’d make again, and the ones I’d make differently next time.
My story so far
Twenty-five years is long enough to see several generations of data technology come and go, and to live with the consequences of design decisions long after the project that made them has finished. That has shaped how I think about data platforms more than any diagram ever has. A design that looks elegant on a whiteboard can be painful to run at 2 a.m. when a pipeline fails, and a slightly “boring” design that a team understands will usually outlast a clever one that only its author can maintain.
Over the last decade, the work has moved from traditional warehouses towards the lakehouse, and now towards platforms that have to serve AI as well as reports. I’ve led large-scale Medallion lakehouse work on Azure Databricks and Unity Catalog at a global professional services firm, where governance, security and cost had to work across many teams at once. Today I’m building an AI-ready data lake on Microsoft Fabric, and designing agentic AI solutions with Model Context Protocol (MCP), retrieval-augmented generation (RAG), Microsoft Foundry (formerly Azure AI Foundry) and Fabric data agents.
I also work with Power BI and the Power Platform, because a data platform only matters if people can actually use what it produces.
What I write about
- Microsoft Fabric: OneLake, lakehouses and warehouses, mirroring and shortcuts, Direct Lake, data agents, and what’s genuinely ready for production.
- Azure Databricks and Unity Catalog: catalog design, access control, Medallion architecture, Lakeflow pipelines, system tables and cost.
- Lakehouse design: open table formats, layering, data modelling and the operational habits that keep a platform healthy.
- Data governance and quality: ownership, classification, lineage, data contracts and quality checks that are designed in, not bolted on.
- Power BI: semantic models people trust, performance, and preparing models for Copilot and agents.
- AI agents, MCP and RAG: getting enterprise data ready for AI, and building agents that give grounded, auditable answers.
What you’ll get from this site
I write for data engineers, analytics engineers, BI developers, architects and technical leads who have to make real decisions on real platforms. In most posts you can expect:
- Practical guidance you can apply on your own platform, not a re-wording of product documentation.
- Honest trade-offs. Where one approach suits some teams and not others, I’ll say so and explain why.
- Worked examples such as naming conventions, checklists, diagrams and code snippets you can adapt.
- Dates and status. Fabric and Databricks change every month, so I note when a post was written and whether a feature was in preview or generally available at the time.
- Learning paths for people who are starting out or moving into data and AI.
How I write
As a practitioner. Everything here comes from hands-on work or from testing things myself. When I’m summarising an announcement rather than reporting on something I’ve run, I’ll make that clear.
Vendor-neutral. I work a lot with Microsoft and Databricks technologies, and I like both, but I’m not here to sell either. I’m not affiliated with the vendors whose products I write about, and recommendations of courses, books or channels are my own, independent choices.
Generalised, never confidential. The lessons I share are drawn from experience, but I never share client names, internal documents or anything confidential. Examples are simplified or made up to illustrate a point.
Plain English. If a concept can be explained without jargon, I’ll try to do that. If I get something wrong, or it goes out of date, I’d rather be corrected and fix it.
What I’ve learned to value
- Start with outcomes, constraints and the reality of the current estate.
- Prefer incremental modernisation to risky “big bang” change.
- Make governance, security, quality and cost part of the design.
- Pair architecture with clear documentation and coaching, so teams can own what they build.
Outside the day job
I enjoy tinkering with ideas in my own time. One of my current personal projects is AssetFabric, a small experiment in asset tracking with an AI twist (“Scan It. Track It. Ask It.”). I may write about what I learn from building it.
Get in touch
This site is a place for knowledge-sharing and community, not a business. If something here helped you, if you see things differently, or if you’d simply like to compare notes on Fabric, Databricks or AI agents, I’d love to hear from you. The best way to reach me is through the Contact Me page or on LinkedIn.
Views on this site are my own and don’t represent my employer. Content is shared for general information and education. See the Disclaimer for details.