MLOps: Building a CI/CD Pipeline for ML Models on Azure Databricks
Most ML teams are great at training models. Very few are great at shipping them. The gap between a notebook that works and a model that reliably serves producti...
Hi! I'm a software engineer who likes to build things.
Here's my most recent posts or read a random one!
Building Evaluation, Cost Governance, and Observability for a Multi-Agent System in Microsoft Foundry
This closes out the series' capstone: the multi-agent customer support system built across Parts 6-9, now hardened with evaluation, cost governance, and observa...
Real-Time AI Feature Engineering with Spark Structured Streaming and Databricks Feature Store
> Building point-in-time correct, production-grade feature pipelines — from raw Kafka events to online feature serving in milliseconds, using Spark Structured S...
Apache Spark Query Optimization on Databricks: Catalyst, AQE, and Photon Engine
> A deep dive into how Spark transforms your SQL into a physical execution plan — and how Databricks layers Adaptive Query Execution and the Photon vectorized e...
Fine-Tuning LLMs at Scale with Databricks MLflow and Spark
> From raw data to a registered, served fine-tuned model — a production walkthrough using Databricks, Apache Spark, MLflow, and Hugging Face Transformers. --- T...
Subscribe with email!
View posts by tag
#A2A#Agentic Workflows#AgenticAI#AgenticIDE#AGUI#AI#AI Architecture#ai-engineering#AIAgent#AIAgents#AIArchitecture#AIInfra#Airflow#AISafety#Algorithms#AmazonBedrock#Analytics#APIDesign#APISecurity#Arrays#Artificial Intelligence#ArtificialIntelligence#Automation#AWS#Azure#Bedrock#BigLake#BigQuery#BinarySearch#BinaryTrees#BitManipulation#CAPTheorem#CareerGrowth#Careers#CI_CD#CleanCode#Cloud#Cloud Architecture#Cloud Computing#CloudCompute#CloudComputing#CloudCosts#CloudEconomics#CloudMessaging#CloudRun#CloudStrategy#Coding#CodingInterview#Containers#CosmosDB#CostManagement#CostOptimization#Data#DataAnalytics#DataArchitecture#Databases#Databricks#DataEngineering#DataLakes#DataStructures#DeltaLake#DevEx#DevOps#DistributedSystems#DSA#DynamoDB#EKS#EMR#EngineeringLessons#Enterprise#Enterprise Tech#EnterpriseAI#EnterpriseSoftware#ETL#EventDriven#Extension#Fabric#FeatureStore#FinOps#Foundry#GCP#Gemini#Gemini Pro#Gemma#GenAI#Generative AI#GenerativeAI#GitHub#GitHubActions#Glue#Google#GoogleCloud#Governance#Graphs#Graviton#Heaps#HighAvailability#IAM#Iceberg#Identity#Interview#InterviewPrep#Interviews#InterviewTips#Kafka#keyvault#kiro#Knowledge Management#kubernetes#LangGraph#Large Language Models#LeetCode#LegacySystems#LLM#LLM Ops#LLMOps#Machine Learning#MachineLearning#MCP#Messaging#Microservices#Microsoft#MLFlow#MLOps#MLSystems#Monitoring#MultiCloud#MultiTenancy#Networking#NoSQL#Observability#OpenAI#OpenSearch#Orchestration#Payments#Performance#Pipelines#PlatformEngineering#PowerBI#Production#Productivity#PromptFlow#PubSub#Python#Queues#Quick#RAG#RateLimiting#Recursion#Redshift#Refactoring#Reliability#ResponsibleAI#S3#Sagemaker#Scalability#Security#Serverless#Software Architecture#software-engineering#SoftwareEngineering#Spanner#Spark#SRE#Stacks#Storage#Streaming#StreamingData#SystemDesign#Traceability#Trees#Unity#Vectors#VectorSearch#VertexAI#VideoEngineering#VSCode#WebDev#Workflows#ZeroTrust