In recent weeks I’ve been implementing single-sign on with Entra for a couple of difference clients. In one case this was for connecting to AWS, and in the other, GitHub. One of these solutions was quite easy—the other one was the company Microsoft owns and was a lot more painful. In this post, you are going to learn about a couple of protocols—Security Assertion Markup Language (SAML) and System for Cross-Domain Identity Management (SCIM—they really worked hard for this acronym). We’re going to talk about the benefits of this config and then how you can implement.

One of the challenges historically around single-sign on (SSO) has been the variety of different providers and having their own ecosystem. I remember a very nascent attempt at this from Oracle (good lord, could you imagine the licensing on that) in the early 2000s, that never really got anywhere. Active Directory was the closest thing we had to ubiquitous access control system, but it mostly required line of site to a domain controller (or the complexities of ADFS). Entra, nee Azure Active Directory really made it possible for your identity management system to interact with lots of third party SaaS apps.

The benefits of single sign on should be obvious, but the biggest is that when you deprovision users in your identity management system, those users are then deprovisioned everywhere. The other benefit is that it allows you to use Entra groups to manage permissions across your systems. You can make application admin teams owners of their own groups, so they can manage their own permissions easily within those apps. (You don’t want to assign users directly if you can help it).

There are two main protocols involved here—the first being SAML, which actually handles the authentication step. There are three roles in SAML: the principal (which is typically the human or machine user), the identity provider or IdP, and the service provider (SP). The SP requests and obtains an authentication token from the IdP and then makes an access control decision to allow the principal to come into the system. This happens at login time. In both our AWS and GitHub examples , we define an identity (our AWS or GitHub organization) on the Entra side and then provide AWS with a certificate and redirect URL to our Entra tenant. That’s the authentication side, which is just one part of this.

 The second part is where SCIM comes in—we want to be able to add users and groups to our app in Entra, and then have them show up in AWS.

For SCIM to work on the Entra side, we pass in a Tenant URL and secret token:

What this does is allows users and groups to be added to the application in Entra, and then pass through into the service on the other side. When we add the group AWSLogin it will show up in AWS. This sync process runs every 40 minutes by default but can be forced manually.

I wanted to complain a minute about GitHub here—I set this up on AWS at client in like 15 minutes, and John and I did for DCAC’s AWS account in even less time. In GitHub, the SAML setup is pretty easy as well—but getting the right SKU for single sign-on is really hard. First of all, GitHub is one of those companies that charges an SSO tax (you can only get SSO on the Enteprise tier). Secondly, you have to have a specific SKU of Enterprise (it’s the same price as regular enterprise at least), but you can’t migrate from another tier to it. So if you have a teams or regular enterprise GitHub account to enable “true” SSO, you need to migrate. Which sucks.  Finally, the docs aren’t clear about this, and you can setup SSO for the non managed user tier, but then your users still have to authenticate to GitHub after completing an Entra login. This is all entirely harder than it needs to be

Single-sign on increases security and makes users lives easier. So even though setup can be a pain in the neck, it’s worth in the end.

Microsoft Fabric is software-as-a-service platform for data processing, business intelligence reporting, and even online transaction processing apps. Fabric has been generally available since November, 2023. Building planet scale platforms is hard, and I have a great deal of sympathy for engineers and architects who are dealing with layoffs, constant pressure for new features, and the push to somehow incorporate AI into every piece of every platform.

All that being said, customers buy online services and expect them to be available. One of the reasons a company chooses Fabric, Databricks, or Snowflake is the notion that those platforms for Spark and various data warehousing options will be secured, patched, and better maintained than a non-technology company could do by simply deploying Spark into Kubernetes or VMware. With that, the cloud providers have an obligation to deliver services to their customers, and deliver availability and performance congruent with their pricing.

One of the things I expect from a cloud provider is honest post-mortems when they have an outage, and maintaining a history of their outages. These histories help architects better design systems, as we can better identify weaknesses in various cloud services that we might want to design around. Azure and AWS both do an excellent job of providing detailed information around “what happened” in incidents.

Cloud providers service customers all over the world, and should feel an obligation to provide detailed incident reports. Azure and AWS are also supported by financially backed service level agreements (SLAs), which mean customers get refunded during outages exceeding the SLA of a service. For example, Azure SQL Database has a 99.995% SLA for Business Critical. If you’re Azure SQL DB is unavailable for more than roughly 2 minutes in a month you’re due a refund (this also applies to data loss or slow failover events). This isn’t business continuity insurance—Microsoft doesn’t pay you for loss sales during the outage, but it’s better than nothing.

Fabric on the other hand, doesn’t have a dedicated SLA. While there is a service reliability documents page, customers have extremely limited options for cross-regional services. There isn’t a simple “geo-replicate” my Fabric environment that you can enable for any amount of money. Microsoft also has a separate Fabric status page, which maintains a status history (which seems to go back a week or so) and doesn’t track historical incident reports like the Azure page does. There was a global Fabric outage in May, there’s no public facing page that details the outage. I can still see it in my Microsoft 365 admin center, but that only keeps 30 days of history and it’s about to roll off. Additionally, the status page for Fabric doesn’t seem to retain the history of all of the service health events. Thanks to u/Fabric-Status on Reddit, I was able to capture all of the service degraded events in June.

Date	Services Impacted	Regional Scope
June 23, 2026	Power BI Labeling	Global
June 23, 2026	Data Pipelines	UK South
June 22, 2026	Power BI (Narrow)	Brazil South
June 20, 2026	Copilot/ML	UAE North
June 17, 2026	Power BI Downloads	Global?
June 14, 2026	Fabric Control Plane	East US 2
June 12, 2026	Activator	India Central
June 10, 2026	Copilot/ML	North Central US
June 10, 2026	Spark Jobs, Notebooks, Python/R Visuals	East US 2
June 4, 2026	Power BI Refresh	Global

That’s quite a list, and they aren’t all in the Fabric status portal—they were all one time, but they weren’t archived and we don’t have detailed incident reports on any of them. The May outage was global and users literally couldn’t access Fabric, and all we know is that it was a DDOS attack, but with no other additional details.

I know this comes off as old man yells at cloud. But this is an expensive service that customers use globally, that has had 12 service degradations in the last month, and customers don’t have a root cause on any of them. This is simply unacceptable behavior from any service provider.

There was a time in 2018-2019 where Azure was a mature platform, but at the same time suffering from a lot of service reliability issues. One of these involved a data center being struck by lightning, but there was also a major Azure AD (now Entra), and DNS problems that had widespread impact (including some very limited Azure SQL DB data loss). After all this happened, Microsoft made major investments in improving the reliability of the Azure platform as a whole. Nothing is perfect—there are still Azure outages, but the response, the reporting process makes them fully transparent to customers.

I feel like every time I’m writing about Fabric I’m yelling at the leadership team to make Fabric more like Azure. That’s probably accurate—Azure is a robust mature platform that supports enterprise controls, auditing, policy, reliability and security. There are still parts of Azure that suck (anyone know how monitor usage on a business critical readable secondary?), but overall I have faith in the platform, because of how open Microsoft have been when they have failures.

Fabric needs to be more robust, more transparent, and more reliable. Given the scope of most of these outages, I suspect they relate to the cadence of software releases. New features are cool, but services that offline because the new feature release went bad are really uncool. Not having an outage history, or options to make your Fabric environment more fault tolerant is even less cool. Microsoft should do their customers better. Fixing this include a few things:

• A persisted outage history page with detailed incident reports
• Service Level Agreements for every component in Fabric and their expected availability
• The ability to monitor Fabric from outside of Fabric
• Simply better reliability—the current state is unacceptable

Some of this is easy—the outage history page should happen yesterday. And put it on the Azure page—the fact that the DDOS report was trapped behind an M365 login was frustrating. The SLAs are something customers should demand. The other two things are harder, but every other important system has them. Reddit shouldn’t be your outage reporting system.

Yesterday, I gave a presentation at Microsoft Fabcon in Atlanta titled “Building the Well-Architected Framework for Fabric”. I’ve been a public speaker for nearly 20 years, and I don’t usually get nervous before a talk. Not that I was shaking yesterday, but given this was a new topic for me, and a big crowd (I had over 350 people sign up for my session), I wanted to make sure I nailed the talk. Based on all the feedback and questions I got yesterday, the talk was very well received by the audience.

Let’s take a step back and talk about why I built this session. Like it or not, Microsoft’s intention with Fabric (and Power BI before it) is to make it easier for less-technical business users to build and consume data-driven reports. While I understand this mission, and it has been wildly successful in spreading love for Power BI, despite Fabric’s software-as-a-service branding, it’s actually a fully fledged data engine that needs to be well-managed to ensure data governance, security, and adherence to general best practices. In building my demos, I created a sample workspace with a couple of objects.

Several attendees asked if how I had gotten access to their tenants. As you can see, this has a horrendous workspace name that has no meaning, and I have a notebook that somehow spans four departments and has its own version control. This can happen for a number of reasons, but the big two are that Fabric is mostly built for those “citizen developers” who build things in a browser with a mouse, start with the default names, and run with it from there. The other problem, which I highlighted throughout my talk, is the lack of a policy engine in Fabric. The corollaries to this in Microsoft-land are group policy objects (GPOs) in Active Directory and Azure Policy, both of which allow administrators to manage how and what gets built in those environments. Sorry to harsh the mellow of anyone on the Fabric team, but publicly traded companies have audit requirements (at least while the SEC and FTC in the US are still operational).

While building the talk, I created a GitHub repo (https://github.com/jdanton/FabricWAF) to start putting my thoughts on a naming standard for Fabric. Additionally, I created some Terraform code on the Azure side (Fabric capacities are deployed through Azure), which limits who can deploy a capacity, which regions they can be in, and which users can be specified for Fabric Capacity Admins.

I’ve been working with Azure since it started, and long enough to remember when there were like two roles: Global Admin and Co-Admin (I still can’t, for the life of me, remember how Co-Admin was different from Global Admin). However, as larger corporations began adopting the cloud, Azure had to mature quickly and build a robust security model, along with a policy engine. The other aspect of that maturity was that larger and more security-minded organizations blocked all manual deployments of Azure resources. If you wanted to deploy cloud resources, you had to write Terraform, commit it to a repository, and then the build server would deploy the code, which had either an identity or credentials that allowed it to deploy those resources on your behalf. This ensured your configuration was in source control and also allowed for some code checking against the standard, either using policy or during the build phase of your project.

While prepping for this presentation, I was deploying some Terraform at my current client and noticed their extensive GitHub Actions check process. When I submit code, another GitHub action workflow is triggered that checks my code for security vulnerabilities, secrets, not meeting best practices, etc. I considered whether I could work with Fabric deployments and use GitHub Actions as my policy engine, since Fabric doesn’t currently have one.

This solution is imperfect, but I also think it’s better than anything else I’ve seen for Fabric. Mainly because, for a more perfect solution, Fabric needs more controls, and the big one is a very granular security model. Fabric is somewhat similar to Azure in 2010, with just four roles, which, frankly, in 2026, is unexcusable. There should be a fully baked out role-based access control model in any sort of enterprise software, especially one that is so API driven.

So what can we do to work around those platform limitations? Here are my thoughts:

• No user is designated as an owner or contributor in any Fabric capacity for production.
• We can somewhat enforce that by applying a naming standard for capacities using Azure Policy.
• We grant the build server’s managed identity (in this case, a GitHub runner) Owner access to all of those capacities.
• We use a rules engine and check on the GitHub actions side to enforce things like naming standards (which I’ve written), potentially item types, etc. Non-compliant builds will fail.
• We periodically audit to look for non-compliant resources. In my audit code, I’m comparing against our naming standards and looking for items/workspaces owned by individual users.

The one big warning I’ll give you about this audit code is that I was rate-limited after running this several times. I waited 30 minutes, and on the third attempt, things seemed to be better.

It’s been amazing to be at Fabcon this week—the energy users have around Fabric are evident everywhere. However, IT’s job is to ensure security, availability, and auditability, which sometimes means saying “no,” or, better yet, “we’ll work with you to achieve what you want in a secure manner.”

In starting to build this framework, I’m hoping to draw on my knowledge of how the cloud has evolved and help organizations apply it to make their Fabric deployments more mature. I’ve also been an architect long enough to know that standards that don’t have a programmatic enforcement method, aren’t worth the bits and pixels they consume. (hi, former boss, you were wrong) This is an attempt to remediate that in Fabric.  If anyone at Microsoft is reading, it would also help if you could give us more controls and RBAC. Thanks, Joey.

I recently started a new role, and I wanted to talk about what I did in my last role—it wasn’t top secret or anything, but it was somewhat different than my normal operations. I’d like to thank everyone at Designmind for giving me this opportunity, as it was a very interesting role. I didn’t leave the job because I disliked the project, I mainly didn’t see my skillset as being compatible with a firm like Cognizant, but I wish everyone there the best.

The Project

When I started at Designmind, we weren’t quite billing on the project—my role was to be data architect, I guess, but that really pretty quickly evolved into being the everything architect. While I did some other client work while I was there, nearly all of what I did during my last job was supporting this project. The project itself was a new application development project for client who aimed to build a supply chain resiliency system. We had a small team of a project manager, a data engineer, and two developers. Sadly, early in the project, we had to remove the data engineer as his skills didn’t really align to the project. That left all of the data tasks to myself and my project manager, who was a massive help. We’ll talk more about data later.

The client specified a fairly specific technology stack, AWS, Python/Flask/SQLAlchemy, PostgreSQL, Kubernetes which I was happy to adopt. They also had some specific requests around DevOps and security where we differed, and went in another direction, based on some other business requirements that the application had. While, I’ve been experienced with all of these technologies, I also had to help our developers get up to speed. We had a sample app one of the developers wrote, so at the beginning of the project, I containerized all of that code, and wrote a bunch of scripts to automatically deploy the app, on Windows, Macs, and Linux, as our target would ultimately be Linux.

Getting Started with AWS

I’ve always worked with AWS, just not as much as I’ve worked with Azure. I’ve had the good fortune to work with both vendors and customers in various AWS projects, so I had a good feel for sizing and performance. The first thing I did was to define the network configuration and building a VPN—I built everything on a private network from the beginning of the project. The basic architecture of the app was that we were going to have front-end containers running React and Nginx, connecting to a couple of middle-tier containers running Python/Flask, with a database running Postgres. Given the fairly narrow scope of this deployment, I used Elastic Kubernetes Services and Amazon RDS for PostgreSQL. For local testing and development, I instructed the team to have a local PostgreSQL instance, and Docker with Kubernetes enabled, so we could install the same stack locally, and use similar deployment scripts.

This kind of leads us into DevOps—which I’m not sure why, but fell into my lap. The client wanted us to use Jenkins, however, the ecosystem and community is dying and it was really complicated and proprietary. I have very good bash scripting skills, so using GitHub actions was a more natural fit to me. We faced a couple of challenges in building out our DevOps workflows—the first was that the builds operated differently on AWS as opposed to locally—this is easily taken care of by making a call to 169.254.169.254 which is a cloud metadata endpoint which works on all clouds, but can let you identity where you code is running. This mattered because we were using IAM authentication and other conditional deployment steps based on build location. Not just in our build process, but in our Python middle tier, I implemented conditional logic to decide how to authenticate to the database.

The Data

I’ve written here before about our data flow process and how we used some AI tooling to improve it. Our data flow and engineering process was really confusing to a lot of traditional data engineering pros I talked to about the project. The biggest issue was we didn’t have regular flows of inbound data—we had two data sets coming from the federal government that were published nightly. Those were easy—I built an Amazon glue job that downloaded, wrote them to S3, and did some degree of cleanup. That Glue job, when complete triggered a data ingestion process—Glue has its limitations, but for this data, it was fine.

The rest of our data sources were either what we could find or sporadic. From the earliest days of this project it was very obvious to me that we would have a data sourcing problem. While the federal government did supply some data, we were either going to have to gather, scrap or buy data. We ultimately bought data from a vendor, who was terrible (bad inconsistent data). I asked the vendor at one point if they could provide a delta file (a file of just changes) and they didn’t know what that was. If you know my email, and are in the data market—email me and I’ll tell you who not to buy data from. That vendor data did provide a basis for our webscraping efforts, which was pretty cool. Most of our reference data like geography, congressional districts, etc. was open sourced and downloaded in our environment. My project manager helped a lot here, identifying and vetting potential new data sources and getting us started with getting them integrating them into the app.

What I Learned

At this point in my career, I’ve been functioning as an architect since about 2013. A lot of people try to define what an architect does, and they try to do it in the context of what actual building architects do, or what some business book written by someone who’s never done the job, or written a line of code thinks an architect does. A good architect has to be ahead of the project, to understand where the priorities of both the project and client wants. The architect needs to be flexible and forward thinking. I always make culinary comparisons, but the role is a lot like being an executive chef. It doesn’t matter if your create amazing recipes (or designs), if your line cooks (or developers) don’t have the skill set to execute them. You either have to increase the skills of those workers (best), hire new workers (hardest), or dial back the recipe/design to match the skill of your team. The architect also needs to think about the problems the team is going to have next—mostly from a technical perspective, but also organizationally, or tooling. If you can get in front of those problems, you can help your developers do their jobs better.

It was a cool experience to be working on an app dev project. I was happy to get to push my skill set and help others grow their own. I would have loved to have completed the project, but unfortunately circumstances got in the way.

Ever since the parquet format came out over a decade ago, it became very popular for analytics workloads. Being columnar in format, it allowed for massive scale analytics, while delivering strong and lossless compression. Various engines including Snowflake, Databricks, Synapse, SQL Server, and other databases I’m likely ignoring can all interact with Parquet. In it’s newer incarnations like Delta parquet, you can also update those files.

There is a notion of a transaction log for each Delta parquet file–it exists in the form of JSON, and isn’t as efficient as a singular transaction log, especially for multi-table (or file) transactions. It’s not a replacement for an OLTP database, but for an analytics workload where you have to occasionally update something it works.

What I’m writing about today has nothing to do with analytics, per se. It has everything to do with cloud storage, and the way operations there are priced. Specifically, metadata operations–in the demo code I’ve shared we’re going from five files to one, but you can imagine going from a much larger number of files to much smaller number of files. You may ask–“Joey that sounds dumb, why are you reinventing zip and iso files”. Well, the main reason is that many cloud operations are priced on the number of objects–for example if you had to calculate a checksum across a number of files on S3. (For files/objects that were created before S3 automatically did checksums).

So the notion of this code, that I wanted to play with, was storing files within a parquet file. At first, I loaded 5 text files into a single parquet file. Then I added an index to the parquet file–thinking forward I added a mapping parquet file, in order to support multiple parquet files with five files each. You can see the demo in this GitHub repo. This is pretty basic code, but the notion is clear–if you have a very large number of small files, you need to store in object storage, and want to reduce that number, and potentially reduce the storage volume, you can use parquet to do it.

There’s a lot of crap out there when you read about Artificial Intelligence projects, especially on LinkedIn, where I suspect half of the posts may have been created by AI bots. However, we recently implemented a process that included the use of an LLM, but in a very controlled fashion. The overall implementation process was pretty interesting, and I wanted to talk about some of the decisions I made, and why.

I obviously can’t share all the details on my current project, but at a high level it’s a custom application that we are developing. One of the biggest challenges my team has faced in the project is acquiring data to support the application. We’ve tried to engage with several data vendors, and when why finally landed with one, we weren’t very happy with the quality and depth of the data they provided. I can say we were seeking information about companies in specific sectors. The obvious answer here was “webscraping”—I don’t know if you’ve ever written code to try and scrape websites, but given there is no common standard for websites, and they are developed with a wide array of frameworks, languages and formats, it’s just a mess.

One day during our standup my product manager/data engineer, suggested that instead of using traditional webscraping, we capture images of web pages, and then feed them to an optical character recognition (OCR) model. This immediately piqued my interest—he had tried it as one off, and it seemed effective. This led me to try it out with about 100 sites—I wrote some code on my machine to scrape the 100 sites using some Python code and package called Playwright. I initially ran it through Azure Computer vision, because I have a free account with my MVP. I had the scraping code grab the home page and about us page of each of my targets.

I looked at the output and it was reasonably good—I had a CSV file with a domain name (which effectively acts as our primary key) and  a long text description of the company and what they did. My plan was to feed this to an LLM and get it to summarize the what the company did, and pull some other specific data features we were looking for. I first tried using the Azure Document Services summary tool, and that worked pretty poor. I then used Azure AI Foundry to use one of the OpenAI models to see if I would get better results. I got a lovely summary and my other data features were extracted as I expected. Now I could see this working, I had to make this work in a production environment.

I quickly threw together a script to scrape 200k websites—I decided to get smart and split the load across 8 nodes. But I cheated—I just split the file into 8 parts, and ran the Python script to do the screen scraping, I knew this was a bad idea, but it was late on a Friday and I wanted to get this going. Predictably all of my worker nodes died over the weekend, and I had to start over from square one.

I’ve been working in Linux for a very long time, so the next part of this process was fairly easy to me, but I still learned a few new things. I implemented a package called Supervisor, which let me build a cluster. I wrote some additional code to be able to easily add additional nodes, and to take advantage of using an AWS Simple Queue Services queue, to pull URLs off the queue. This gave me resumability, and scale—and because the queue maintained state, if nodes were rebooted, it didn’t impact my workload. In fact, I added an additional script running as a service on my controller nodes, which checked for unavailable worker nodes—if they were unavailable, we simply rebooted them. I ultimately scaled this scraping cluster to 36 nodes, and we completed our process in about 2.5 days.

I was able to reuse the same cluster to perform the OCR and summarization tasks. Both were much less time consuming than the scraping process. I was able to get away with using eight nodes for both of those processes. The same basic idea applied—publish the data into the queue and let the worker nodes operate on them.

The summarization process is important to us—we wanted to have high quality data and avoid the risk of hallucinations that LLMs can have. I did a couple of things to reduce the risk there—I dropped the temperature parameter to .1, greatly reducing the creativity of the model. I also carefully crafted a system prompt instructing the model to only use it’s input text to create a summary of the site. I ended up using one of the Amazon Nova models—you don’t need a big cutting-edge model to summarize text and extract features. This means the inference costs were extremely low.

AI tools are best used when we tightly control the input data and put tight guardrails around the process. In this post, I wanted to demonstrate how you can take advantage of the benefits of an LLM, at a low cost. I also wanted to walk you through my process of how to scale this process, and make it into a production level process.

This week Microsoft Front Door suffered another major outage. I wrote about the last outage(s) in my column at Redmond just a couple of weeks ago. Microsoft Front Door is a global content delivery network that does a number of other services for websites/APIs/endpoints. One of the challenges around Front Door is that being a global service, when it goes down, there’s no native failover process that you can easily use.

Microsoft has published an initial incident report and there were some interesting details.

Nothing reads too out of the ordinary for a cloud outage–but a couple of things there was around 8.5 hours of downtime for the service. The other notable thing (bolded) is that Microsoft failed the Azure Portal away from Front Door. There was some comments about this in the earlier incident report. So that brings up the question–do you need to have a plan to fail away from Front Door?

Do You To Need to Be Multi-Cloud?

I talked about this in my Redmond column, but implementing a backup solution to Azure Front Door, is inherently a multi-cloud solution. There are a few choices for global WAF solutions–not just from hyperscaler like AWS, Azure and GCP, but also CloudFlare, But if you’re application is global, has a low recovery point objective, and is critical to your business then you need to multi-cloud.

The bigger question is does your entire stack need to multi-cloud? I would argue, that at least in light of our knowledge of cloud failures–probably not. Unless you have an extremely tight SLA–you are greatly increasing the cost and complexity of the network stack. In fact, I would argue most applications don’t need this kind of highly available network stack.

In designing this I took some lessons from what I think the Azure Portal team has done–I suspect they have their servers behind Application Gateways and Front Door interacts with those applications

The basic notion is we use Azure Traffic Manager with priority routing, the Front Door instance pictured here would be the initial fallback. That gives us some degree of protection against Front Door failures, and that approach seemed to work for the most recent outages. However, there was a lot of downstream DNS issues in other Azure services that raised concerns. For example, you could login to the portal, but a symptom was that you could only see Resource Groups, but no other resources.

Cloudflare comes into play here, presuming you can’t make any app updates, or your app gateways go sideways. You could recreate all of your Front Door functionality and have easy failover, in effectively a completely different provider. That doesn’t help if Azure were to go completely down, but we haven’t seen an outage like that, since the great certificate expiration failure of 2013. Generally speaking failures are limited to regions–these Front Door outages are exceptions to that rule, as Front Door is a “global” service and isn’t homed to a single region (which makes the recent outages more infuriating).

Traffic Manager being in Azure is a concern to me–I put it into this architecture because it’s relatively easy to configure, but being in another cloud for DNS could be a good option. Both Google (Google DNS) and AWS (Route 53) have services that allow for multiple IP addresses and failover based on health probes, or you could use a service like DNSMadeEasy to also handle this. DNS is really the ultimate challenge in any sort of a multi-cloud scenario–where do you put it.

There’s a lot more detail here than in a normal blog post, and yet, I held back a lot of detail. There’s a method to my madness, I’ll be publishing a white paper and doing a webinar with my friends from Denny Cherry and Associates Consulting, John Morehouse and Denny Cherry, to discuss pros and cons, and detailed configurations for how to make your applications more resilent when the Front Door closes as they say. Look for more details on that over at dcac.com/ in the next few weeks.

I’ve seen a couple of posts (of course they were chock full of AI slop images) on LinkedIn in the last couple of weeks, talking about how challenging it is to implement Kubernetes. In the most recent post I saw, it stated that “it took 5 months for our CEO to implement Kubernetes for our app”, to which I would ask, why the hell is your CEO configuring your clusters. I designed, and implemented the Kubernetes infrastructure on my current project, and I’ve worked on for a while, so of course, I felt the need to share my opinions on the matter.

If you are trying to build bare metal Kubernetes (are you also compiling your own Linux?), it is probably pretty difficult. If you are like the rest of the word, just use your preferred cloud providers Kubernetes distribution (Azure Kubernetes Service or the Elastic Kubernetes Service on AWS) and run with that. Even if you suck at AWS security like I do, you can get this up and running in a couple of hours. I’d even say you could run it on-prem on VMware, but Broadcom threatened me with a lawsuit for saying that without a license. After that, you really don’t have to think about Kubernetes that much other than deploying containers to run as pods.

Yes, this does mean your developers have to learn YAML (or ask an LLM to make it for them), which they should already know, understand how containers works (which I hope they do already), and learn a few organizing things about security and labeling in K8s. But after that Kubernetes handles auto-scaling (especially if you checked the auto-scale box in your cloud provider), does a lot of heavy lifting for networking in your microservices app, and provides a pretty good level of high availability.

Kubernetes has about the same level of complexity as your average cloud deployment, and the infrastructure as code scenarios are far simpler. If you just think of it as VMware, but for containers, and roll with changes, you’ll have a good time and gain a lot of functionality for a bit of work.