|**Metadata**|**Description**  |
|--|--|
|Doc Title|  MVM v3: 11.5.2 - Convert the Azure Pipeline Service Connection to use Federated Credentials |
|Navigation|[WIKI Home Page](https://dev.azure.com/mvmprodeus2/MVM/_wiki/wikis/documentation/1/documents-home#)|
|Tracking| Document Number: VPE-5512-522 |
|Author| Microsoft AFO Engineering |
| Agreement Number | 24252.S.005 |

***
**Notices**

Copyright © 2025 Metaswitch Networks.  All rights reserved.

This manual is Confidential Information of Metaswitch Networks subject to the confidentiality terms
of the Agreement 01019223 as amended between AT&T and Metaswitch Networks.

It is issued on the understanding that no part of the product code or documentation (including this manual)
will be copied or distributed without prior agreement in writing from Metaswitch Networks and Microsoft.

Metaswitch Networks and Microsoft reserve the right to, without notice, modify or revise all or part of
this document and/or change product features or specifications and shall not be responsible for any
loss, cost, or damage, including consequential damage, caused by reliance on these materials.

Metaswitch and the Metaswitch logo are trademarks of Metaswitch Networks. Other brands and
products referenced herein are the trademarks or registered trademarks of their respective holders.

Product(s) and features documented in this manual handle various forms of data relating to your
users. You must comply with all laws and regulations applicable to your deployment, management,
and use of said product(s), and you should take all appropriate technical and organizational
measures to ensure you are handling this data appropriately according to any local legal and
regulatory obligations.

You are responsible for determining whether said product(s) or feature(s) is/are appropriate for
storage and processing of information subject to any specific law or regulation and for using said
product(s) or feature(s) in a manner consistent with your own legal and regulatory obligations. You
are also responsible for responding to any request from a third party regarding your use of said
product(s), such as a request to take down content under the U.S. Digital Millennium Copyright Act
or other applicable laws.


Metaswitch Networks
399 Main Street
Los Altos
CA 94022
<http://www.metaswitch.com>


***
***Table of Contents***
[[_TOC_]]

# 1. Document History

| **Issue** | **Issue Date** | **Author(s)** | **Identification** **of** **Changes** |
|-|-|-|-|
| 1| 11/20/2024| Microsoft AFO Engineering| Initial Delivery of MOP|

# 2. Versions

| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Microsoft AFO Engineering| Initial Delivery of MOP|

# 3. Integrated Solution Approach v1 (ISA v1)

| **Version #** | **Editor** | **Comments** |
|-|-|-|
| 1| Microsoft AFO Engineering| Initial Delivery of MOP|

# 4. MOP Impact Scope / General Information

## 4.1 Description

The Azure pipelines service connection is currently configured to use an Azure Service Principal client secret for authentication.

This MOP describes the process for converting the Azure pipelines service connection to use federated credentials, with Azure DevOps as the federated identity provider.

This MOP must be run once for every ADO project in the deployment. This MOP is only required as part of upgrade to R11.5.2.

## 4.2 Site Specific Description

| **Originator** | **Date** | **Time** |
|-|-|-|
| **Deployment Location(s)** | |
| **Description** | This MOP applies to the MVM V3 on Azure deployment, Release R11.5.3 | |

## 4.3 Service Impact

Service impact is not expected during this procedure.

## 4.4 Coordination

This MOP has no interactions outside of the MVM subscription.

# 5. Prerequisite/Dependencies/Entrance Criteria of MOP

This MOP is one of several that need to be run to execute the process to upgrade an existing deployment to an 11.5.3 release/patch.

Please refer to the corresponding *R11.5.3 Release Upgrade Overview* document for guidance on the order in which to run these MOPs

## 5.1 Required parameters

The following parameter values are required to run this MOP

| **Identifier** | **Description** |
|-|-|
| **REGION_SHORTNAME** | The short (4-characters maximum) DNS label for the region |
| **SUBSCRIPTION_NAME** |  Azure subscription name.  |
| **SUBSCRIPTION_ID** | Azure subscription identifier.  |
| **SP_APP_ID** |   Identifier of the Azure Service Principal.  |
| **SP_APP_NAME** |  Name of the Microsoft Entra ID Application.  |
| **TENANT_ID** | Azure tenant identifier. |

## 5.2 Required files

No additional files are required to run this MOP.

## 5.3 Required permissions

The following permissions are required to run this MOP

   - You must have the `Endpoint Administrator` permission over the Azure DevOps project which hosts the configuration repository.
   - You must have Owner permissions over the Microsoft Entra ID App Registration with name `Name of the Microsoft Entra ID Application`.
   - You must have Owner permissions over the Microsoft Entra ID Enterprise Application with name `Name of the Microsoft Entra ID Application`.

# 6. Assumptions

The target audience for this procedure is the AT&T Engineer who will be performing the task. They will need to be familiar with Azure and have a working knowledge of the Azure CLI and Linux.

# 7. Material Requirements

## 7.1 Required Documents

## 7.2 Tools

| **Tool** | **Description** | **Quantity** |
|-|-|-|
| Laptop or Desktop PC | PC With at least 1G Memory and a network communications software application such as Procomm, Reflections or PuTTY | 1 |
| Azure connectivity PC | CloudShell Connectivity is required to the azure subscription. This can be accessed via [My Dashboard - Microsoft Azure](https://portal.azure.com/#cloudshell/) | |

# 8. Pre Maintenance Check, Precautions and Preparations

## 8.1 Precautions and Preparation

## 8.2 Precautions

> This procedure may cause a partial outage during implementation. Use executable script files to minimize down time and typing errors. Familiarize yourself with back-out procedures prior to starting the procedure.

| **Ask Yourself Principle** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Do I have the proper ID and appropriate building access permissions for the environment I am about to enter? | | |
| 2. Do I know why I'm doing this work? | | |
| 3. Have I identified and notified everybody - customers and internal groups - who will be directly affected by this work? | | |
| 4. Can I prevent or control service interruption? | | |
| 5. Is this the right time to do this work? | | |
| 6. Am I trained and qualified to do this work? | | |
| 7. Are the work orders, MOPs, and supporting documentation current and error-free? | | |
| 8. Do I have everything I need to quickly and safely restore service if something goes wrong? | | |
| 9. Have I walked through the procedure? | | |
| 10. Have I made sure the procedure includes proper closure including obtaining clearance and release for the appropriate work center? | | |


| **E911 Ask Yourself** | **Yes** | **No** | **N/A** |
|-|-|-|-|
| 1. Does this work impact E911? | | |
| 2. Do I know how this work could impact 911/e911? | | |
| 3. Do I know what 911/e911 phase is required? | | |
| 4. Have I identified potential risks to 911/e911 and taken all measures to minimize? | | |
| 5. Does this work affect 15+ sites? | | |
| 6. Can I prevent or control service Interruptions to 911/e911? | | |
| 7. Is this the right time to do the work? | | |
| 8. Is the individual performing the work trained and qualified to do this work? | | |
| 9. Are MOPs and supporting documents current and error free? | | |
| 10. Does the MOP include a 911/e911 test plan? | | |
     

## 8.3 Pre-Maintenance Check Tools/System

Tier 2 needs to identify which tools they will use. This doesn't necessarily need to be included in the MOP but OPS needs to know which tools they will run.

(NEED TO USE STANDARD TOOLS) TIER 2


## 8.4 Pre-Maintenance Check Manual (Non-Automated Requirements)

These will be identify by the tier 3 MOP developer were required.

(MANDATORY CHECK REQUIRE FOR THE MOP) TIER 3


## 8.5 MOP Certification Environment

Examples:  PSL certified.  OR This MOP was paper certified by ATS engineers.

## 8.6 ATS Bulletin

**ATS Bulletin Check**
| **Step** | **Action** | **Results/Description** | **Timeline** |
|-|-|-|-|
| 1. | No Applicable bulletins | | |


## 8.7 Emergency Contacts

The following emergency contact numbers are to be used in the event provisioning support is required.

In the event a service interruption is encountered the AT&T Implementation Engineer will:
- Cease all work immediately.
- Notify the AT&T Voicemail TRC.
- Escalate to the next level of support.


| **Organization** | **Contact Name** | **Contact Number** |
|-|-|-|
| Voicemail TRC | SANRC | 877-662-7674, opt 3 |

# 9. Implementation

## 9.1 Preliminary Implementation
Pre-check tasks are completed the night of the cutover at least one hour prior to cutover activities.

1. Connect to the DevOps Portal
   1. Start a browser session to <https://dev.azure.com/>. This will be required to manage the pipelines
   1. Select the project associated with MVM v3
1. Connect to the Azure Portal
   1. Start a browser session to <https://portal.azure.com/>. This will be required to manage Azure resources
      and access the log analytics workspace (LAW)
   1. If prompted, complete the log in process
1. Connect to Azure Cloud Shell
   1. Start a CloudShell session by connecting a browser to <https://shell.azure.com/>
   1. If the menu at the top left indicates PowerShell select Bash from the menu and confirm at the prompt

      ![screenshot](images/powershell.jpg)
1. Upload any files and directories outlined in section 5.2 to your Cloud Shell account as they will be needed later


## 9.2 Implementation

This MOP consists of:
- Initial attempt to convert the existing Azure pipelines service connection to use federated credentials automatically.
   - This is expected to fail. On failure, the error output will provide the details required to setup a new Federated Credential.
- Creation of a new federated credential on the deployment service principal, using the values obtained in the previous step.
- Convert the existing Azure pipelines service connection to use federated credentials automatically.

### 9.2.1 Obtain federated credentials variables by attempting to convert the Azure pipelines service connection

These commands are run from the DevOps Portal created in section 9.1

1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the deployment service connection named `Name of the ADO service connection used in MVM deployment pipelines - recorded as `mvm_service_connection` in the pipeline configuration variables file`. 

The service connection in ADO should show a banner with the following text:

> Manually created service connections use an App Registration that was created by the user. Please add a federated credential to the App Registration with the following details: Issuer: https://vstoken.dev.azure.com/<org id>, Subject identifier: sc://<org>/<project>/<sc name>.

1. Click on the "Convert" button on the right of this banner.
1. This will lead to a pop-up box stating `Converting will delete the secrets associated with this service connection from Azure DevOps in 7 days`.
1. Press "Convert" again on the pop-up box.

This will fail with the following message.

> Automatic authentication conversion failed. Your service connection was not modified. To continue the conversion manually, create a Federated Credential for the underlying Service Principal using the Federation Subject Identifier below and try again.

![Authentication conversion section](images/redacted-mvm-pipelines-conversion.jpg)

A new "Authentication conversion" section should appear beneath this error message.
1.  Note down the issuer as `SC_ISSUER`
1.  Note down the subject identifier as `SC_SUBJECT_ID`

### 9.2.2 Create federated credential on the deployment service principal

These commands are run from the Azure Portal session created in section 9.1

1. Go to App Registrations, and select your deployment service principal with name `Name of the Microsoft Entra ID Application`.
   - Select "Certificates & secrets" under Manage
   - Select "Federated credentials"
   - Create a new federated credential
   - Under "Federated credential scenario", choose "Other issuer"

1. Under "Connect your account":
   - Set Issuer to `SC_ISSUER`
   - Leave Type as the default - "Explicit subject identifier"
   - Set Subject identifier "Value" to `SC_SUBJECT_ID`
   - Set Name to `Name of the ADO service connection used in MVM deployment pipelines - recorded as `mvm_service_connection` in the pipeline configuration variables file`

1. Verify that you can see the newly created federated credential on your service principal.

### 9.2.3 Convert the Azure pipelines service connection to use federated credentials

These commands are run from the DevOps Portal session opened in section 9.1.

1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the `Name of the ADO service connection used in MVM deployment pipelines - recorded as `mvm_service_connection` in the pipeline configuration variables file` service connection
1. The service connection in ADO should show a banner with the following text

> Automatic authentication conversion failed. Your service connection was not modified. To continue the conversion manually, create a Federated Credential for the underlying Service Principal using the Federation Subject Identifier below and try again.

1. Press the "Try again" button, inside the "Automatic authentication conversion failed" warning box.
1. Verify that the conversion succeeds.

## 9.3 Test Plan
### 9.3.1 Run a pipeline using the new service connection

**Wait 30 minutes after creating the new federated credential before running this test.**

1. Select pipelines from the Left hand menu

1. Select pipelines from the sub menu

1. Select "All" from the resultant page

1. Expand pipelines

1. Select the pipeline to validate the configuration
    Choose the `mvmsecretsmonitoring` pipeline

1. Select Run pipeline

   Verify that the pipeline succeeds.

## 9.4 Backout Procedure

The change to the service connection can be reverted back to using a client secret for 7 days after it was converted.

### 9.4.1 Revert the change to using a client secret

These commands are run from the Azure DevOps Portal.

1. Select Project Settings from the Left hand menu.
1. Select Service connections under Pipelines.
1. Select the `Name of the ADO service connection used in MVM deployment pipelines - recorded as `mvm_service_connection` in the pipeline configuration variables file` service connection.
1. Click "Revert conversion to the original scheme". This appears either as a button on the page or in the menu under the three-dot button in the top right of the page.
1. Click "Revert" on the pop-up.

![Revert conversion](images/revert-mvm-pipelines.jpg)

![Revert conversion in three-dot menu](images/revert-mvm-pipelines-2.jpg)

# 10. Post checks

[System healthchecks]

# 11. Risk Assessment Score

1 - TBD

# 12. Execute MOP clean up if required

# 13. End of Document MOP

# 14. Service Assurance/Monitoring

# A. Appendix and Tables

# B. Approvers

# C. Peer Reviewers

# D. References for Other Documents

# E. Additional Appendices (If required)