r1153leaupdateconfigserviceconnection.md
markdown_lea
rendered_eus2/r1153leaupdateconfigserviceconnection.md
Rendered Markdown
610 lines
|**Metadata**|**Description** |
|--|--|
|Doc Title| MVM v3: 11.5.2 - Update the Config Validation Service Connection |
|Navigation|[WIKI Home Page](https://dev.azure.com/mvmprodeus2/MVM/_wiki/wikis/documentation/1/documents-home#)|
|Tracking| Document Number: VPE-5512-524 |
|Author| Microsoft AFO Engineering |
| Agreement Number | 24252.S.005 |
***
**Notices**
Copyright © 2025 Metaswitch Networks. All rights reserved.
This manual is Confidential Information of Metaswitch Networks subject to the confidentiality terms
of the Agreement 01019223 as amended between AT&T and Metaswitch Networks.
It is issued on the understanding that no part of the product code or documentation (including this manual)
will be copied or distributed without prior agreement in writing from Metaswitch Networks and Microsoft.
Metaswitch Networks and Microsoft reserve the right to, without notice, modify or revise all or part of
this document and/or change product features or specifications and shall not be responsible for any
loss, cost, or damage, including consequential damage, caused by reliance on these materials.
Metaswitch and the Metaswitch logo are trademarks of Metaswitch Networks. Other brands and
products referenced herein are the trademarks or registered trademarks of their respective holders.
Product(s) and features documented in this manual handle various forms of data relating to your
users. You must comply with all laws and regulations applicable to your deployment, management,
and use of said product(s), and you should take all appropriate technical and organizational
measures to ensure you are handling this data appropriately according to any local legal and
regulatory obligations.
You are responsible for determining whether said product(s) or feature(s) is/are appropriate for
storage and processing of information subject to any specific law or regulation and for using said
product(s) or feature(s) in a manner consistent with your own legal and regulatory obligations. You
are also responsible for responding to any request from a third party regarding your use of said
product(s), such as a request to take down content under the U.S. Digital Millennium Copyright Act
or other applicable laws.
Metaswitch Networks
399 Main Street
Los Altos
CA 94022
<http://www.metaswitch.com>
***
***Table of Contents***
[[_TOC_]]
# 1. Document History
| **Issue** | **Issue Date** | **Author(s)** | **Identification** **of** **Changes** |
|-|-|-|-|
| 1| 11/20/2024| Microsoft AFO Engineering| Initial Delivery of MOP|
# 2. Versions
| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Microsoft AFO Engineering| Initial Delivery of MOP|
# 3. Integrated Solution Approach v1 (ISA v1)
| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Microsoft AFO Engineering| Initial Delivery of MOP|
# 4. MOP Impact Scope / General Information
## 4.1 Description
Your Azure DevOps config validation pipeline needs to be able to pull artifacts from your ACR in order to validate the contents of the repository. This is achieved through creating a Service Connection linked to Service Principal.
This MOP describes the process for replacing the pre-existing service connection with a new service connection with reduced permissions over your Azure subscription.
This MOP is only required as part of upgrade to R11.5.2.
## 4.2 Site Specific Description
| **Originator** | **Date** | **Time** |
|-|-|-|
| **Deployment Location(s)** | |
| **Description** | This MOP applies to the MVM V3 on Azure deployment, Release R11.5.3 | |
## 4.3 Service Impact
Service impact is not expected during this procedure.
## 4.4 Coordination
This MOP has no interactions outside of the MVM subscription.
# 5. Prerequisite/Dependencies/Entrance Criteria of MOP
This MOP is one of several that need to be run to execute the process to upgrade an existing deployment to an 11.5.3 release/patch.
Please refer to the corresponding *R11.5.3 Release Upgrade Overview* document for guidance on the order in which to run these MOPs
## 5.1 Required parameters
The following parameter values are required to run this MOP
| **Identifier** | **Description** |
|-|-|
| **CLOUD_SHELL_LOCATION** | The location of the files that were copied to the Cloud Shell account. |
| **GIT_CONFIGURATION_REPOSITORY** | Name of the configuration Azure DevOps repository. |
| **GIT_CONFIGURATION_URL** | URL of the configuration git repository. |
| **GIT_PASSWORD** | Password used to access the Azure DevOps repositories if you are using https to manage the local copy of the access the repository. |
| **ORGANIZATION_NAME** | Name of the Azure DevOps organization. |
| **PROJECT** | Name of the Azure DevOps project. |
| **REGION_SHORTNAME** | The short (4-characters maximum) DNS label for the region |
| **SUBSCRIPTION_ID** | Azure subscription identifier. |
| **SUBSCRIPTION_NAME** | Azure subscription name. |
| **TENANT_ID** | Azure tenant identifier. |
| **UPLEVEL_MVM_VERSION** | The version number of this release. |
## 5.2 Required files
No additional files are required to run this MOP.
## 5.3 Required permissions
The following permissions are required to run this MOP
- You must have the `Endpoint Administrator` permission over the Azure DevOps project which hosts the configuration repository.
- You must have permission to create a Microsoft Entra Application and Service Principal as described here: https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#permissions-required-for-registering-an-app.
- You must have `Key Vault Secrets Officer` permissions on the `Name of the Key Vault where secrets and certificates are stored for the entire deployment.` Key Vault.
# 6. Assumptions
The target audience for this procedure is the AT&T Engineer who will be performing the task. They will need to be familiar with Azure and have a working knowledge of the Azure CLI and Linux.
# 7. Material Requirements
## 7.1 Required Documents
## 7.2 Tools
| **Tool** | **Description** | **Quantity** |
|-|-|-|
| Laptop or Desktop PC | PC With at least 1G Memory and a network communications software application such as Procomm, Reflections or PuTTY | 1 |
| Azure connectivity PC | CloudShell Connectivity is required to the azure subscription. This can be accessed via [My Dashboard - Microsoft Azure](https://portal.azure.com/#cloudshell/) | |
# 8. Pre Maintenance Check, Precautions and Preparations
## 8.1 Precautions and Preparation
## 8.2 Precautions
> This procedure may cause a partial outage during implementation. Use executable script files to minimize down time and typing errors. Familiarize yourself with back-out procedures prior to starting the procedure.
| **Ask Yourself Principle** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Do I have the proper ID and appropriate building access permissions for the environment I am about to enter? | | |
| 2. Do I know why I'm doing this work? | | |
| 3. Have I identified and notified everybody - customers and internal groups - who will be directly affected by this work? | | |
| 4. Can I prevent or control service interruption? | | |
| 5. Is this the right time to do this work? | | |
| 6. Am I trained and qualified to do this work? | | |
| 7. Are the work orders, MOPs, and supporting documentation current and error-free? | | |
| 8. Do I have everything I need to quickly and safely restore service if something goes wrong? | | |
| 9. Have I walked through the procedure? | | |
| 10. Have I made sure the procedure includes proper closure including obtaining clearance and release for the appropriate work center? | | |
| **E911 Ask Yourself** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Does this work impact E911? | | |
| 2. Do I know how this work could impact 911/e911? | | |
| 3. Do I know what 911/e911 phase is required? | | |
| 4. Have I identified potential risks to 911/e911 and taken all measures to minimize? | | |
| 5. Does this work affect 15+ sites? | | |
| 6. Can I prevent or control service Interruptions to 911/e911? | | |
| 7. Is this the right time to do the work? | | |
| 8. Is the individual performing the work trained and qualified to do this work? | | |
| 9. Are MOPs and supporting documents current and error free? | | |
| 10. Does the MOP include a 911/e911 test plan? | | |
## 8.3 Pre-Maintenance Check Tools/System
Tier 2 needs to identify which tools they will use. This doesn't necessarily need to be included in the MOP but OPS needs to know which tools they will run.
(NEED TO USE STANDARD TOOLS) TIER 2
## 8.4 Pre-Maintenance Check Manual (Non-Automated Requirements)
These will be identify by the tier 3 MOP developer were required.
(MANDATORY CHECK REQUIRE FOR THE MOP) TIER 3
## 8.5 MOP Certification Environment
Examples: PSL certified. OR This MOP was paper certified by ATS engineers.
## 8.6 ATS Bulletin
**ATS Bulletin Check**
| **Step** | **Action** | **Results/Description** | **Timeline** |
|-|-|-|-|
| 1. | No Applicable bulletins | | |
## 8.7 Emergency Contacts
The following emergency contact numbers are to be used in the event provisioning support is required.
In the event a service interruption is encountered the AT&T Implementation Engineer will:
- Cease all work immediately.
- Notify the AT&T Voicemail TRC.
- Escalate to the next level of support.
| **Organization** | **Contact Name** | **Contact Number** |
|-|-|-|
| Voicemail TRC | SANRC | 877-662-7674, opt 3 |
# 9. Implementation
## 9.1 Preliminary Implementation
Pre-check tasks are completed the night of the cutover at least one hour prior to cutover activities.
1. Connect to the DevOps Portal
1. Start a browser session to <https://dev.azure.com/>. This will be required to manage the pipelines
1. Select the project associated with MVM v3
1. Connect to the Azure Portal
1. Start a browser session to <https://portal.azure.com/>. This will be required to manage Azure resources
and access the log analytics workspace (LAW)
1. If prompted, complete the log in process
1. Connect to Azure Cloud Shell
1. Start a CloudShell session by connecting a browser to <https://shell.azure.com/>
1. If the menu at the top left indicates PowerShell select Bash from the menu and confirm at the prompt
1. Upload any files and directories outlined in section 5.2 to your Cloud Shell account as they will be needed later
## 9.2 Implementation
### 9.2.1 Create the config validation Service Principal
> This step is only required for the first region being upgraded.
>
> The config validation Service Principal is a deployment-wide resource.
1. Register an application with Microsoft Entra and create a service principal, following the instructions in https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal#register-an-application-with-azure-ad-and-create-a-service-principal
- This should be created with a suitable name and the audience "Accounts in this organizational directory only". No Redirect URI is required.
1. Record the name of the Microsoft Entra application as CONFIG_VALIDATION_SP_APP_NAME
### 9.2.2 Assign permissions to the Microsoft Entra application
> This step is only required for the first region being upgraded.
>
> The config validation Service Principal is a deployment-wide resource.
1. Assign the `Reader` role on the subscription `Azure subscription name for the MVM subscription` to the Microsoft Entra application, following the instructions in https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal?tabs=current.
1. Assign the `AcrPull` roles on the subscription `Azure subscription name for the MVM subscription` to the Microsoft Entra application, following the instructions in https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal?tabs=current.
### 9.2.3 Record the Microsoft Entra application Service Principal properties
> This step is only required for the first region being upgraded.
>
> The config validation Service Principal is a deployment-wide resource.
Record the identifiers of the Microsoft Entra application's Service Principal. These are properties of the associated Enterprise application, rather than the Microsoft Entra application itself.
1. Navigate to the Enterprise application:
- From the Overview tab of the Microsoft Entra application, extend the Essentials group
- Click the link corresponding to "Managed application in local directory"
1. From the Overview tab of the Enterprise application, record the "Application ID" as CONFIG_VALIDATION_SP_APP_ID and the "Object ID" as CONFIG_VALIDATION_SP_OBJ_ID
### 9.2.4 Upload the new Service Principal properties to Key Vault
> This step is only required for the first region being upgraded.
>
> The config validation Service Principal is a deployment-wide resource.
1. Set the following environment variables:
```
AGENT_KEY_VAULT_NAME=Name of the Key Vault where secrets and certificates are stored for the entire deployment.
USER_IP=$(curl -s http://ipinfo.io/json | jq -r '.ip')
```
1. Give your user IP address access to the Key Vault
```
az keyvault network-rule add --name $AGENT_KEY_VAULT_NAME --ip-address $USER_IP
```
1. Upload the CONFIG_VALIDATION_SP_APP_ID to the deployment key vault
```
az keyvault secret set --vault-name "$AGENT_KEY_VAULT_NAME" --name config-validation-sp-app-id --value "$CONFIG_VALIDATION_SP_APP_ID"
```
1. Upload the CONFIG_VALIDATION_SP_OBJ_ID to the deployment key vault
```
az keyvault secret set --vault-name "$AGENT_KEY_VAULT_NAME" --name config-validation-sp-obj-id --value "$CONFIG_VALIDATION_SP_OBJ_ID"
```
1. Remove access to the Key Vault for your user IP address
```
az keyvault network-rule remove --name $AGENT_KEY_VAULT_NAME --ip-address $USER_IP
```
### 9.2.5 Create the new config validation service connection
As part of creating this service connection, you will need access to both the Azure Portal, and access to the Azure DevOps portal.
These commands are run from the DevOps Portal created in section 9.1
> Note: In January 2025, AzureDevOps has partially rolled out a user interface change that affects the creation of service connections.
>
> The UI you are offered will determine the options you need to select, and differences are identified in the following steps with bold **If available:** and **Else:** comments.
1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select New service connection.
1. Select Azure Resource Manager.
1. **If available:** Select Workload Identity federation (manual).
1. **Else:** Set Identity type: App registration or managed identity (manual), and set Credential: Workload identity federation.
1. Fill in the "Step 1: Basics" form:
- Set Service connection name to `config-validation`
- **If available:** Ensure Grant access permission to all pipelines is ticked.
- Click Next
1. This will open the "Step 2: App registration details" form, or the "Step 2: Service Principal details" form:
- Note down the issuer as `SC_ISSUER`
- Note down the subject identifier as `SC_SUBJECT_ID`
- You will need these variables in the next step to create the federated credential in the Azure Portal.
1. Select "Keep as draft" - the service connection configuration will be comlpeted after the Azure Portal steps below.
These commands are run from the Azure Portal session created in section 9.1
1. Go to App Registrations, and select your newly created service principal with name CONFIG_VALIDATION_SP_APP_NAME.
- Select "Certificates & secrets" under Manage
- Select "Federated credentials"
- Create a new federated credential
- Under "Federated credential scenario", choose "Other issuer"
1. Under "Connect your account":
- Set Issuer to `SC_ISSUER`
- Leave Type as the default - "Explicit subject identifier"
- Set Subject identifier "Value" to `SC_SUBJECT_ID`
- Set Name to `config-validation-The short (4-characters maximum) DNS label for the region`
1. Verify that you can see the newly created federated credential on your service principal.
These commands are run from the DevOps portal, continuing from where you left above
1. Select the draft service connection called `config-validation`.
1. Select the "Finish setup" button.
1. Fill in rest of the "Step 2: App registration details" form:
- Leave the issuer and Subject identifier fields as their default values
- Set Environment to "Azure Cloud"
- Set Scope Level to "Subscription"
- Set Subscription to `Azure subscription identifier for the MVM subscription.`
- Set Subscription Name to `Azure subscription name for the MVM subscription`
- Set Service Principal Id to `<CONFIG_VALIDATION_SP_APP_ID>`.
- Set Tenant Id to `Azure tenant identifier`
- **If available:** Ensure Grant access permission to all pipelines is ticked.
1. Select Verify and save.
A new Service connection called `config-validation` should now appear in the list.
### 9.2.6 Prepare the configuration Git repository
This is the Git repository that holds the code and configuration for MVM.
These commands are run from the CloudShell session created above
1. Set the following environment variables:
```
CLOUD_SHELL_LOCATION=The location of the files that were copied to the Cloud Shell account.
UPLEVEL_MVM_VERSION=The version number of the uplevel release, e.g. 11.5.0+1
BRANCH=Change ID, used as the prefix for any git branch created in the MOPs_The version number of the uplevel release, e.g. 11.5.0+1_update_service_connection
```
Export the correct form of the URL to access the git repository
- If using https to interact with the git repository
```
GIT_CONFIGURATION_URL=https://Name of the Azure DevOps organization. Note this must be different from the organization used for the Main deployment.@dev.azure.com/Name of the Azure DevOps organization. Note this must be different from the organization used for the Main deployment./Name of the Azure DevOps project./_git/Name of the configuration Azure DevOps repository.
```
- If using ssh to interact with the git repository
```
GIT_CONFIGURATION_URL=git@ssh.dev.azure.com:v3/Name of the Azure DevOps organization. Note this must be different from the organization used for the Main deployment./Name of the Azure DevOps project./Name of the configuration Azure DevOps repository.
```
1. Change to an appropriate working directory in Cloud shell. Your Git repository will live in a subdirectory off of this path.
```
cd ~
mkdir configuration_repo
cd configuration_repo
```
1. Clone the existing Azure DevOps Git repository with **<GIT_CONFIGURATION_URL>**. The repository can be cloned using either ssh or https. In both cases you will run the following command:
```
git clone ${GIT_CONFIGURATION_URL} .
```
(note the trailing whitespace and period after the URL)
- If using HTTPS:
- When prompted, input the password, **<GIT_PASSWORD>**, that you specified when the repository was first created.
- If using SSH:
- You will not be prompted for a password.
This will create a local copy of the repository in the current working directory.
1. Create a new working branch by running the command
```
git checkout -b ${BRANCH}
```
The branch currently only exists on your local server - it will be pushed to the DevOps repository in a later step
### 9.2.7 Apply manual configuration change to use the new service connection
1. Change the name of the service connection to use:
```
sed -i 's/simpl-sp/config-validation/' variables/config-pipeline-vars.yaml
```
1. Commit the change to the local branch by running the command:
```
git commit -a -m "Change service connection name"
```
### 9.2.8 Update the remote repository
1. Push the change to the DevOps repository by running the command:
```
git push --set-upstream origin ${BRANCH}
```
1. Merge the change into the main branch via the 'pull request' mechanism.
1. Tidy up by running the command:
```
cd ~
rm -rf configuration_repo
```
(We have finished with the local copy of the repository)
### 9.2.9 Verify the new service connection
1. Follow the [**Test Plan**](#testplan) to verify the timer functionality.
### <a id=testpass></a> 9.2.10 New service connection successful
Only run this section of the MOP if the new service connection is working correctly.
### 9.2.11 Delete old service connection
These commands are run from the DevOps Portal created in section 9.1
1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the service connection called `simpl-sp-old` (renamed from `simpl-sp` during testing).
1. Select the kebab menu button.
1. Select `Delete` to delete the service connection.
## <a id=testplan></a> 9.3 Test Plan
The Test Plan here will validate that the config validation continues to run successfully after adding the new service connection.
It will then validate that the config validation pipeline is correctly using the new service connection, and is not using the old service connection.
### 9.3.1 Run a pipeline after adding the new service connection
1. Select pipelines from the Left hand menu
1. Select pipelines from the sub menu
1. Select "All" from the resultant page
1. Expand pipelines
1. Select the config validation pipeline
- This pipeline will have the name assigned when it was created, which is likely to be the name of the configuration repository.
- The file used to create this pipeline is Azure-pipelines.yml.
1. Select Run pipeline
- Verify that the pipeline succeeds.
### 9.3.2 Change the name of the old service connection
These commands are run from the DevOps Portal created in section 9.1
1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the service connection called `simpl-sp`
1. Select `Edit`
1. Change the `Service connection name` to be `simpl-sp-old`
1. Click `Verify and save`
### 9.3.3 Re-run a pipeline to validate that the new service connection is being used
1. Select pipelines from the Left hand menu
1. Select pipelines from the sub menu
1. Select "All" from the resultant page
1. Expand pipelines
1. Select the config validation pipeline
- This pipeline will have the name assigned when it was created, which is likely to be the name of the configuration repository.
- The file used to create this pipeline is Azure-pipelines.yml.
1. Select Run pipeline
- Verify that the pipeline succeeds.
If the pipeline succeeds, return to [**New service connection successful**](#testpass) to continue the MOP.
If the validation pieline fails, contact **Support** and follow the backout procedure.
## 9.4 Backout Procedure
### 9.4.1 Revert the service connection changes
These commands are run from the DevOps Portal created in section 9.1
1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the service connection called `simpl-sp-old`
1. Select `Edit`
1. Change the `Service connection name` back to `simpl-sp`
1. Click `Verify and save`
### 9.4.2 Revert the config repository changes
These commands are run from the DevOps Portal created in section 9.1
1. Access your Git repo in ADO using a web browser.
1. Navigate to Repos > Commits
1. Find the second-to-last commit (the one from before this MOP was ever run).
Note the 8-digit hex string referring to that commit; the location of the string is highlighted in the screenshot below.
1. From your Cloud Shell, within the cloned Git repo, run the following, replacing `<commit>` with the string from step 3 above:
```
git revert <commit>
git commit
git push
```
# 10. Post checks
[System healthchecks]
# 11. Risk Assessment Score
1 - TBD
# 12. Execute MOP clean up if required
# 13. End of Document MOP
# 14. Service Assurance/Monitoring
# A. Appendix and Tables
# B. Approvers
# C. Peer Reviewers
# D. References for Other Documents
# E. Additional Appendices (If required)