r1153upgraderegionalprereqs.md
markdown_main
rendered_scus/r1153upgraderegionalprereqs.md
Rendered Markdown
610 lines
|**Metadata**|**Description** |
|--|--|
|Doc Title| MVM v3: Upgrade Regional PreReqs|
|Navigation|[WIKI Home Page](https://dev.azure.com/mvmprodeus2/MVM/_wiki/wikis/documentation/1/documents-home#)|
|Tracking| Document Number: VPE-5512-007|
|Author| Graeme Thomson (gt163y) |
| Agreement Number | 24252.S.005 |
***
**Notices**
Copyright © 2025 Metaswitch Networks. All rights reserved.
This manual is Confidential Information of Metaswitch Networks subject to the confidentiality terms
of the Agreement 01019223 as amended between AT&T and Metaswitch Networks.
It is issued on the understanding that no part of the product code or documentation (including this manual)
will be copied or distributed without prior agreement in writing from Metaswitch Networks and Microsoft.
Metaswitch Networks and Microsoft reserve the right to, without notice, modify or revise all or part of
this document and/or change product features or specifications and shall not be responsible for any
loss, cost, or damage, including consequential damage, caused by reliance on these materials.
Metaswitch and the Metaswitch logo are trademarks of Metaswitch Networks. Other brands and
products referenced herein are the trademarks or registered trademarks of their respective holders.
Product(s) and features documented in this manual handle various forms of data relating to your
users. You must comply with all laws and regulations applicable to your deployment, management,
and use of said product(s), and you should take all appropriate technical and organizational
measures to ensure you are handling this data appropriately according to any local legal and
regulatory obligations.
You are responsible for determining whether said product(s) or feature(s) is/are appropriate for
storage and processing of information subject to any specific law or regulation and for using said
product(s) or feature(s) in a manner consistent with your own legal and regulatory obligations. You
are also responsible for responding to any request from a third party regarding your use of said
product(s), such as a request to take down content under the U.S. Digital Millennium Copyright Act
or other applicable laws.
Metaswitch Networks
399 Main Street
Los Altos
CA 94022
<http://www.metaswitch.com>
***
***Table of Contents***
[[_TOC_]]
# 1. Document History
| **Issue** | **Issue Date** | **Author(s)** | **Identification** **of** **Changes** |
|-|-|-|-|
| 1| 06/10/2024| Gthomson| initial draft |
| 2| 08/29/2024| Gthomson| Add South Central specific regional prereqs instructions |
| 3| 09/30/2024| Gthomson| updates based on Ops feedback |
| 4| 11/26/2024| Gthomson| Remove spurious CLOUD_SHELL_LOCATION declaration from 9.2.2 |
| 5| 11/26/2024| Gthomson| Format cloud shell location in monospace format in section 5.2 |
# 2. Versions
| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Gthomson| initial draft |
| 2| Gthomson| Add South Central specific regional prereqs instructions |
| 3| Gthomson| updates based on Ops feedback |
| 4| Gthomson| Remove spurious CLOUD_SHELL_LOCATION declaration from 9.2.2 |
| 5| Gthomson| Format cloud shell location in monospace format in section 5.2 |
# 3. Integrated Solution Approach v1 (ISA v1)
| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Gthomson| initial draft |
| 2| Gthomson| Add South Central specific regional prereqs instructions |
| 3| Gthomson| updates based on Ops feedback |
| 4| Gthomson| Remove spurious CLOUD_SHELL_LOCATION declaration from 9.2.2 |
| 5| Gthomson| Format cloud shell location in monospace format in section 5.2 |
# 4. MOP Impact Scope / General Information
## 4.1 Description
This procedure will upgrade the Azure regional resources in an existing region. These resources include:
- Virtual network. A virtual network with subnets, network security groups and private endpoints to Azure resources. Service Instances are all deployed into a common Virtual Network, allowing them all to be routable from each other.
- Private endpoints to enable secure Azure PrivateLink connection to the per-region Azure Blob Store and the deployment-wide Azure Cassandra API Cosmos DB account.
- Azure Blob Stores. We set up blob stores for use by various components:
- MVM, for audio data
- SIMPL, for Terraform state
- SAS, for resource bundles
- SIMon, for metrics, one per availability zone
- Diags, for storing diagnostic bundles
- Container registry. Contains the binary container images required to spin up MVM containers within the Azure Kubernetes Service clusters.
- Azure Queue Store (AQS). We set up a per-region Azure Queue Storage account to hold the message metadata during a Cosmos DB outage
- Log analytics workspace.
- Azure key Vaults
## 4.2 Site Specific Description
| **Originator** | **Date** | **Time** |
|-|-|-|
| **Deployment Location(s)** | |
| **Description** | This MOP applies to the MVM V3 on Azure deployment, Release R11.5.3 | |
## 4.3 Service Impact
Service impact is not expected during this procedure.
## 4.4 Coordination
This MOP has no interactions outside of the MVM subscription.
# 5. Prerequisite/Dependencies/Entrance Criteria of MOP
This MOP is one of several that need to be run to execute the process to upgrade an existing deployment to an 11.5.3 release/patch.
Please refer to the corresponding *R11.5.3 Release Upgrade Overview* document for guidance on the order in which to run these MOPs
## 5.1 Required parameters
The following parameter values are required to run this MOP
| **Identifier** | **Description** |
|-|-|
| **AZURE_REGION** | This Azure region, e.g. `eastus2`. |
| **CLOUD_SHELL_LOCATION** | The location of the files that were copied to the Cloud Shell account. |
| **DOWNLEVEL_MVM_FILESHARE** | Name of the fileshare containing the current MVM release |
| **DOWNLEVEL_PIPELINE_CONFIGURATION_NAME** | Name of the downlevel pipeline configuration file for the region without the .yml suffix (e.g. `vars_eus2_11400`). |
| **DOWNLEVEL_REGION_TFVARS_NAME** | Name of the downlevel (current) version of the regional pre-requisites configuration file without the tfvars suffix (e.g. `region_eus2_11400`). |
| **GIT_AUTOMATION_REPOSITORY** | Name of the automation Azure DevOps repository. |
| **GIT_AUTOMATION_URL** | URL of the automation git repository.|
| **GIT_PASSWORD** | Password used to access the Azure DevOps repositories if you are using https to manage the local copy of the access the repository. |
| **ORGANIZATION_NAME** | Name of the Azure DevOps organization. |
| **PROJECT** | Name of the Azure DevOps project. |
| **SUBSCRIPTION_ID** | Azure subscription identifier. |
| **UPLEVEL_MVM_FILESHARE** | Name of the fileshare containing the Uplevel Release (**This is specified in the release note**) |
| **UPLEVEL_MVM_VERSION** | The version number of the uplevel release |
| **UPLEVEL_PIPELINE_CONFIGURATION_NAME** | Name of the uplevel pipeline configuration file for the region without the .yml suffix (e.g. `vars_eus2_11500`). |
| **UPLEVEL_REGION_TFVARS_NAME** | Name of the uplevel version of the regional pre-requisites configuration file without the tfvars suffix (e.g. `region_eus2_11500`). |
| **VNET_NAME** | Name of the mvm vnet |
## 5.2 Required files
The following files from the release point `Name of the fileshare containing the MVM release (This is specified in the release note)` must be uploaded to `The location of the files that were copied to the Cloud Shell account.` before starting this MOP:
- The file `SIMPL/terraform/main_region_prerequisites_terraform.tgz`
# 6. Assumptions
The target audience for this procedure is the AT&T Engineer who will be performing the task. They will need to be familiar with Azure and have a working knowledge of the Azure CLI and Linux.
# 7. Material Requirements
## 7.1 Required Documents
## 7.2 Tools
| **Tool** | **Description** | **Quantity** |
|-|-|-|
| Laptop or Desktop PC | PC With at least 1G Memory and a network communications software application such as Procomm, Reflections or PuTTY | 1 |
| Azure connectivity PC | CloudShell Connectivity is required to the azure subscription. This can be accessed via [My Dashboard - Microsoft Azure](https://portal.azure.com/#cloudshell/) | |
# 8. Pre Maintenance Check, Precautions and Preparations
## 8.1 Precautions and Preparation
## 8.2 Precautions
> This procedure may cause a partial outage during implementation. Use executable script files to minimize down time and typing errors. Familiarize yourself with back-out procedures prior to starting the procedure.
| **Ask Yourself Principle** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Do I have the proper ID and appropriate building access permissions for the environment I am about to enter? | | |
| 2. Do I know why I'm doing this work? | | |
| 3. Have I identified and notified everybody - customers and internal groups - who will be directly affected by this work? | | |
| 4. Can I prevent or control service interruption? | | |
| 5. Is this the right time to do this work? | | |
| 6. Am I trained and qualified to do this work? | | |
| 7. Are the work orders, MOPs, and supporting documentation current and error-free? | | |
| 8. Do I have everything I need to quickly and safely restore service if something goes wrong? | | |
| 9. Have I walked through the procedure? | | |
| 10. Have I made sure the procedure includes proper closure including obtaining clearance and release for the appropriate work center? | | |
| **E911 Ask Yourself** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Does this work impact E911? | | |
| 2. Do I know how this work could impact 911/e911? | | |
| 3. Do I know what 911/e911 phase is required? | | |
| 4. Have I identified potential risks to 911/e911 and taken all measures to minimize? | | |
| 5. Does this work affect 15+ sites? | | |
| 6. Can I prevent or control service Interruptions to 911/e911? | | |
| 7. Is this the right time to do the work? | | |
| 8. Is the individual performing the work trained and qualified to do this work? | | |
| 9. Are MOPs and supporting documents current and error free? | | |
| 10. Does the MOP include a 911/e911 test plan? | | |
## 8.3 Pre-Maintenance Check Tools/System
Tier 2 needs to identify which tools they will use. This doesn't necessarily need to be included in the MOP but OPS needs to know which tools they will run.
(NEED TO USE STANDARD TOOLS) TIER 2
## 8.4 Pre-Maintenance Check Manual (Non-Automated Requirements)
These will be identify by the tier 3 MOP developer were required.
(MANDATORY CHECK REQUIRE FOR THE MOP) TIER 3
## 8.5 MOP Certification Environment
Examples: PSL certified. OR This MOP was paper certified by ATS engineers.
## 8.6 ATS Bulletin
**ATS Bulletin Check**
| **Step** | **Action** | **Results/Description** | **Timeline** |
|-|-|-|-|
| 1. | No Applicable bulletins | | |
## 8.7 Emergency Contacts
The following emergency contact numbers are to be used in the event provisioning support is required.
In the event a service interruption is encountered the AT&T Implementation Engineer will:
- Cease all work immediately.
- Notify the AT&T Voicemail TRC.
- Escalate to the next level of support.
| **Organization** | **Contact Name** | **Contact Number** |
|-|-|-|
| Voicemail TRC | SANRC | 877-662-7674, opt 3 |
# 9. Implementation
## 9.1 Preliminary Implementation
Pre-check tasks are completed the night of the cutover at least one hour prior to cutover activities.
1. Connect to the DevOps Portal
1. Start a browser session to <https://dev.azure.com/>. This will be required to manage the pipelines
1. Select the project associated with MVM v3
1. Connect to the Azure Portal
1. Start a browser session to <https://portal.azure.com/>. This will be required to manage Azure resources
and access the log analytics workspace (LAW)
1. If prompted, complete the log in process
1. Connect to Azure Cloud Shell
1. Start a CloudShell session by connecting a browser to <https://shell.azure.com/>
1. If the menu at the top left indicates PowerShell select Bash from the menu and confirm at the prompt
1. Upload any files and directories outlined in section 5.2 to your Cloud Shell account as they will be needed later
## 9.2 Implementation
### 9.2.1 Set the default subscription to the MVM subscription
1. Set the default subscription by running the command:
```
az account set --subscription "Azure subscription identifier for the MVM subscription."
```
### 9.2.2 Prepare the automation git repository
This is the git repository that holds the pipelines, Terraform scripts etc.
These commands are run from the CloudShell session created above
1. Set the following environment variables:
```
BRANCH=Change ID, used as the prefix for any git branch created in the MOPs_The version number of the uplevel release, e.g. 11.5.0+1_update_regional_terraform_file
```
Export the correct form of the URL to access the git repository
- If using https to interact with the git repository
```
GIT_AUTOMATION_URL=https://Name of the Azure DevOps organization.@dev.azure.com/Name of the Azure DevOps organization./Name of the Azure DevOps project./_git/Name of the automation Azure DevOps repository.
```
- If using ssh to interact with the git repository
```
GIT_AUTOMATION_URL=git@ssh.dev.azure.com:v3/Name of the Azure DevOps organization./Name of the Azure DevOps project./Name of the automation Azure DevOps repository.
```
1. Change to an appropriate working directory in Cloud shell. Your git repository will live in a subdirectory off of this path.
```
cd ~
mkdir automation_repo
cd automation_repo
```
1. Clone the existing Azure DevOps git repository with **<GIT_AUTOMATION_URL>**. The repository can be cloned using either ssh or https. In both cases you will run the following command:
```
git clone ${GIT_AUTOMATION_URL} .
```
(note the trailing whitespace and period after the URL)
- If using HTTPS:
- When prompted, input the password, **<GIT_PASSWORD>**, that you specified when the repository was first created
- If using SSH:
- You will not be prompted for a password.
This will create a local copy of the repository in the current working directory.
1. Create a new working branch by running the command
```
git checkout -b ${BRANCH}
```
The branch currently only exists on your local server - it will be pushed to the DevOps repository in a later step
### 9.2.3 Update the regional tfvars file
1. Set the following environment variables:
```
CLOUD_SHELL_LOCATION=The location of the files that were copied to the Cloud Shell account.
DOWNLEVEL_REGION_TFVARS_FILE=Name of the downlevel (current) version of the regional pre-requisites configuration file without the tfvars suffix (e.g. region_eus2_11400)..tfvars
UPLEVEL_MVM_VERSION=The version number of the uplevel release, e.g. 11.5.0+1
UPLEVEL_REGION_TFVARS_FILE=Name of the uplevel version of the regional pre-requisites configuration file without the tfvars suffix (e.g. region_eus2_11500)..tfvars
```
1. Extract the uplevel example tfvars file by running the following command
```
cd pipelines/configuration
tar -zxvf ${CLOUD_SHELL_LOCATION}/main_region_prerequisites_terraform.tgz \
main_region/main.tfvars.example --transform \
"s#main_region/main.tfvars.example#""main_region_${UPLEVEL_MVM_VERSION}.tfvars.example#"
```
1. Create the regional tfvars file for the new release by running the following commands
```
cp ${DOWNLEVEL_REGION_TFVARS_FILE} ${UPLEVEL_REGION_TFVARS_FILE}
```
Update the file with any new variables introduced in this release. New variables are identified in the *Upgrade Overview* document accompanying the release. They can also be determined by diffing the appropriate versions of the example variable file.
If there are no changes then this step is a null-op
1. Add the new files to the repository by running the command
```
git add -A
```
1. Commit the change to the local branch by running the command
```
git commit -a -m "Create new regional tfvars file"
```
1. Push the change to the DevOps repository by running the command:
```
git push --set-upstream origin ${BRANCH}
```
1. Merge the change into the main branch via the 'pull request' mechanism
1. Tidy up by running the command:
```
cd ~
rm -rf automation_repo
```
(We have finished with the local copy of the repository)
### 9.2.4 Update the regional pre-requisites
These commands are run from the DevOps Portal created in section 9.1
1. Select pipelines from the Left hand menu
1. Select pipelines from the sub menu
1. Select "All" from the resultant page
1. Expand pipelines
1. Select the pipeline `mvmupgraderegion`
1. Select Run pipeline
1. In the resultant Run pipeline menu
- Set the name of the release to deploy to `Name of the fileshare containing the MVM release (This is specified in the release note)`
- Set the name of the MVM config file to `Name of the uplevel pipeline configuration file for the region without the .yml suffix (e.g. vars_eus2_11500).`
- Set the name of the Terraform config file to `Name of the uplevel version of the regional pre-requisites configuration file without the tfvars suffix (e.g. region_eus2_11500).`
- Set 'Use a separate MT Subscription for Perimeta related resources' to true, the default
- Leave the override check flag with its default value of `unchecked (false)`
- Select Run
Selecting Run will cause an approval email to be sent to the reviewers list. The pipeline will stall until the reviewers have approved the request. Once approval has been given the pipeline will run.
The progress of the pipeline can be followed by clicking on the jobs list.
Once the pipeline has completed, success, or failure, is reported to screen
1. If the pipeline reports that the upgrade is unsafe
- open the artifact `check_upgrade.txt`
- If the changes are limited to the following:
- Deletion of tags
- Deletion of private_link_access to a storage account to the storageDataScanner resource
- Changes to outputs
- Changing the `private_link_service_network_policies_enabled` value for a subnet which contains no private endpoints (see details below)
It is safe to proceed. You should repeat the previous step, but this time check
the `override check flag` checkbox
You can tell if the subnet contains no private endpoints by finding the vnet (present in the resource ID) on the Azure portal, then going to the private endpoint section (Settings->Private endpoint) and then scanning the list of private endpoints.
- If there are other changes identified in the artifact file, **stop** and contact **Support**
1. Acceptable error when `override check flag` is enabled
If, having enabled the flag, the Terraform apply step in the pipeline fails with
an error recreating the peering between the main vnet and the self hosted agent vnet,
similar to the excerpt shown below:
```
2024-08-08T14:15:44.3029985Z [0m[1mmodule.private_endpoint[0].azurerm_private_dns_zone_virtual_network_link.mvm_retry_queue_dns_link[0]: Creation complete after 32s [id=/subscriptions/***/resourceGroups/ra-mvm-eus2-dev-app-rg/providers/Microsoft.Network/privateDnsZones/privatelink.queue.core.windows.net/virtualNetworkLinks/retry-queue-DNSLink-vnet-l][0m
2024-08-08T14:15:44.8754652Z [31m╷[0m[0m
2024-08-08T14:15:44.8757658Z [31m│[0m [0m[1m[31mError: [0m[0m[1mnetwork.VirtualNetworkPeeringsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="RemotePeeringIsDisconnected" Message="Cannot create or update peering /subscriptions/***/resourceGroups/ra-mvm-eus2-sha-rg/providers/Microsoft.Network/virtualNetworks/ramvmeus2sha-vnet/virtualNetworkPeerings/ramvmeus2sha-vnet_to_ra-mvm-eus2-dev-app-vnet-vnet-pr because remote peering /subscriptions/***/resourceGroups/ra-mvm-eus2-dev-app-rg/providers/Microsoft.Network/virtualNetworks/ra-mvm-eus2-dev-app-vnet/virtualNetworkPeerings/ra-mvm-eus2-dev-app-vnet_to_ramvmeus2sha-vnet-vnet-pr referencing parent virtual network /subscriptions/***/resourceGroups/ra-mvm-eus2-sha-rg/providers/Microsoft.Network/virtualNetworks/ramvmeus2sha-vnet is in Disconnected state. Update or re-create the remote peering to get it back to Initiated state. Peering gets Disconnected when remote vnet or remote peering is deleted and re-created." Details=[][0m
2024-08-08T14:15:44.8759565Z [31m│[0m [0m
2024-08-08T14:15:44.8760294Z [31m│[0m [0m[0m with module.selfhost_vm_vnet_peering[0].azurerm_virtual_network_peering.rmt_to_local["ramvmeus2sha-vnet"],
2024-08-08T14:15:44.8761037Z [31m│[0m [0m on vnet_peering/main.tf line 54, in resource "azurerm_virtual_network_peering" "rmt_to_local":
2024-08-08T14:15:44.8761743Z [31m│[0m [0m 54: resource "azurerm_virtual_network_peering" "rmt_to_local" [4m{[0m[0m
2024-08-08T14:15:44.8762227Z [31m│[0m [0m
2024-08-08T14:15:44.8762651Z [31m╵[0m[0m
```
Then this is acceptable and is remedied as follows:
In the portal:
- Search for the mvm vnet resource `Name of the mvm vnet`
- Expand **Settings** from the Left Hand Side pane
- Select **Peerings**
- Scroll down until you find the peering with a `Disconnected` peering state (see image below).
The name of the peering should match the name of the remote peering in the `Original Error` message above
- Check the check box to the left of the peering name
- Select **Delete** from the top of the page
- In the subsequent dialog window
- Enter `delete` to confirm deletion
- Select **Delete**
Success, or failure is reported to screen
Assuming the peer was successfully deleted, rerun the pipeline with the
`override check flag` enabled. If this still fails then **stop** and contact **Support**
### 9.2.5 Update the region specific sensitive variable group
This is an optional step which adds new variables to the deployment pipeline library.
New variables are identified in the Upgrade overview document accompanying the release.
1. Return to your Azure DevOps project
- Select **Pipelines** on the left sidebar
- Select **Library**
1. Select the variable group `mvm-The short (4-characters maximum) DNS label for the region-vault`
1. In the resultant window, select **+ Add** located at the bottom of the existing variables
1. Add the variables indicated in the release note and select **Ok**
Note that you might have to select the **load More** link at the bottom of the secrets window if the secret you need is not currently shown
1. Select **Save** to commit the changes
## 9.3 Test Plan
There is no test plan associated with this MOP. If it runs to completion you can proceed to the next MOP
## 9.4 Backout Procedure
The backout procedure requires re-running the regional pre-requisites using the down level configuration file and Terraform scripts.
### 9.4.1 Roll back the regional pre-requisites
1. Select pipelines from the Left hand menu
1. Select pipelines from the sub menu
1. Select "All" from the resultant page
1. Expand pipelines
1. Select the pipeline `mvmrollbackregion`
1. Select Run pipeline
1. In the resultant Run pipeline menu
- Set the name of the release to deploy to `Name of the fileshare containing the downlevel (current) MVM release`
- Set the name of the MVM config file to `Name of the downlevel pipeline configuration file for the region without the .yml suffix (e.g. vars_eus2_11400).`
- Set the name of the Terraform config file to `Name of the downlevel (current) version of the regional pre-requisites configuration file without the tfvars suffix (e.g. region_eus2_11400).`
- Set 'Use a separate MT Subscription for Perimeta related resources' to true, the default
- Leave the override check flag with its default value of 'unchecked (false)'
- Select Run
Selecting Run will cause an approval email to be sent to the reviewers list. The pipeline will stall until the reviewers have approved the request. Once approval has been given the pipeline will run.
The progress of the pipeline can be followed by clicking on the jobs list.
Once the pipeline has completed, success, or failure, is reported to screen
1. If the pipeline reports that the upgrade is unsafe
- open the artifact `check_upgrade.txt`
- If the changes are limited to the following:
- Deletion of tags
- Deletion of private_link_access to a storage account to the storageDataScanner resource
- Changes to outputs
It is safe to proceed. You should repeat the previous step, but this time check
the `override check flag` checkbox
- If there are other changes identified in the artifact file, **stop** and contact **Support**
1. Acceptable error when `override check flag` is enabled
If, having enabled the flag, the Terraform apply step in the pipeline fails with
an error recreating the peering between the main vnet and the self hosted agent vnet,
similar to the excerpt shown below:
```
2024-08-08T14:15:44.3029985Z [0m[1mmodule.private_endpoint[0].azurerm_private_dns_zone_virtual_network_link.mvm_retry_queue_dns_link[0]: Creation complete after 32s [id=/subscriptions/***/resourceGroups/ra-mvm-eus2-dev-app-rg/providers/Microsoft.Network/privateDnsZones/privatelink.queue.core.windows.net/virtualNetworkLinks/retry-queue-DNSLink-vnet-l][0m
2024-08-08T14:15:44.8754652Z [31m╷[0m[0m
2024-08-08T14:15:44.8757658Z [31m│[0m [0m[1m[31mError: [0m[0m[1mnetwork.VirtualNetworkPeeringsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="RemotePeeringIsDisconnected" Message="Cannot create or update peering /subscriptions/***/resourceGroups/ra-mvm-eus2-sha-rg/providers/Microsoft.Network/virtualNetworks/ramvmeus2sha-vnet/virtualNetworkPeerings/ramvmeus2sha-vnet_to_ra-mvm-eus2-dev-app-vnet-vnet-pr because remote peering /subscriptions/***/resourceGroups/ra-mvm-eus2-dev-app-rg/providers/Microsoft.Network/virtualNetworks/ra-mvm-eus2-dev-app-vnet/virtualNetworkPeerings/ra-mvm-eus2-dev-app-vnet_to_ramvmeus2sha-vnet-vnet-pr referencing parent virtual network /subscriptions/***/resourceGroups/ra-mvm-eus2-sha-rg/providers/Microsoft.Network/virtualNetworks/ramvmeus2sha-vnet is in Disconnected state. Update or re-create the remote peering to get it back to Initiated state. Peering gets Disconnected when remote vnet or remote peering is deleted and re-created." Details=[][0m
2024-08-08T14:15:44.8759565Z [31m│[0m [0m
2024-08-08T14:15:44.8760294Z [31m│[0m [0m[0m with module.selfhost_vm_vnet_peering[0].azurerm_virtual_network_peering.rmt_to_local["ramvmeus2sha-vnet"],
2024-08-08T14:15:44.8761037Z [31m│[0m [0m on vnet_peering/main.tf line 54, in resource "azurerm_virtual_network_peering" "rmt_to_local":
2024-08-08T14:15:44.8761743Z [31m│[0m [0m 54: resource "azurerm_virtual_network_peering" "rmt_to_local" [4m{[0m[0m
2024-08-08T14:15:44.8762227Z [31m│[0m [0m
2024-08-08T14:15:44.8762651Z [31m╵[0m[0m
```
Then this is acceptable and is remedied as follows:
In the portal:
- Search for the mvm vnet resource `Name of the mvm vnet`
- Expand **Settings** from the Left Hand Side pane
- Select **Peerings**
- Scroll down until you find the peering with a `Disconnected` peering state (see image below).
The name of the peering should match the name of the remote peering in the `Original Error` message above
- Check the check box to the left of the peering name
- Select **Delete** from the top of the page
- In the subsequent dialog window
- Enter `delete` to confirm deletion
- Select **Delete**
Success, or failure is reported to screen
Assuming the peer was successfully deleted, rerun the pipeline with the
`override check flag` enabled. If this still fails then **stop** and contact **Support**
# 10. Post checks
[System healthchecks]
# 11. Risk Assessment Score
1 - TBD
# 12. Execute MOP clean up if required
# 13. End of Document MOP
# 14. Service Assurance/Monitoring
# A. Appendix and Tables
# B. Approvers
# C. Peer Reviewers
# D. References for Other Documents
# E. Additional Appendices (If required)