How to deploy and configure routing table, vnet, nsg, including nsg rules and subnets using IaC for a production ready data platform

#02 - This article is a deep-dive, on how to configure a network layer for your data platform in Azure and how to deploy it using Azure Bicep (IaC).

Before we start diving into the details here is a reason why, creating a network layer is important, don’t let anyone tell you that it’s sufficient to use managed networking built into many services on Azure. By creating a network layer that we have full control of we are preparing for a greater enterprise setup such as Hub and Spoke network architecture. In that context this network layer, housing the platform we want to build, we are going to create is a single spoke in a larger enterprise network landscape.

DISCLAIMER: I am sorry to say but I am commited to writting quality articles for my subscribers. This means there will be alot of detail and I cannot promise any of it beeing easy to setup. This also means these articles are very different from what you will find elsewhere. Many guides aim at merly introducing topics and giving examples which cannot be used in any sort of production environment. This is very different in the articles you will find here.

Infrastructure-as-code (IaC)

The IaC consists of bicep files applied to Azure infrastructure by submitting it to Azure Resource Manager (ARM)

Using DevOps pipelines. While this is the ideal solution, which we will cover, we will skip it for now and run the bicep scripts from our local machine. I want to skip the DevOps automation pipelines for now because that would take focus from the main objective of this article, which is to get an understanding of how to create a network layer infrastructure that can be used in development, testing, and production environments, and how to do that using Azure Bicep.

Create vnet and subnet and nsg

DISCLAIMER: The following commands are for local, manual deployment and should only be used during development. We will go into the DevOps pipelines automation of the IaC in a later Article. But if you are up for it by having the commands that can be run manually you practically have the recipe for that commands a DevOps pipeline should run to automate the deployment process.

Prerequisites

• You need to have access to an Azure Subscription.

• You need to have the Azure CLI installed on your local machine.

• You need to have created a Service Principal with a Secret.

• The Service Principal needs Contributor rights on the Subscription.

Access the subscription

Here is how you can access your subscription from the Azure CLI. Because we want to be able to automate the deployment in the future, we will not authenticate with the Azure CLI using a personal user, instead, let’s authenticate using a Service Principal.