Tag Archives: api

The woes of implementing M365 Information Barriers – One BIG gotcha and how to undo unexpected Teams team/group chat/meeting membership removals using Graph API

Leave a reply

For the better part of three years we had been working to implement Information Barriers for Microsoft Teams. All throughout that time we ran into countless issues, bugs, and back-end changes forcing us to open a number of tickets with Microsoft and do multiple rounds of extensive testing before we were comfortable implementing. The day finally came when we were ready to implement. Finally one evening we had everything staged and we were ready to flip the switch later that week. To our surprise a number of unexpected changes occurred a day before flipping the switch. A large set of users were removed from teams, group chats, and meetings. There were also reports of users not being able to communicate with each other via chat/calls/meetings. Some of the users affected were not even part of either of the two IB segments that we were implementing policies for. After a little digging it became clear that IB started to go haywire sometime after we activated the IB policies (we did this as part of staging), but before running Start-InformationBarrierPoliciesApplication (this was our final step in activation). We immediately disabled both of the IB policies and ran Start-InformationBarrierPoliciesApplication to force an immediate processing of the policies. After this was done processing, all users were able to communicate again, but we were still left with all of the team/group chat/meeting removals.

I decided to start with addressing the team removals. I decided the quickest way to get a list of these was to do an Azure AD audit search since all team member removals translate to member removals of the associated M365 group. I already had experience with how IB removals show from an Azure AD perspective, so I started by running an audit search for AAD group removals and then filtered this using the three principals that could perform these IB-related removals:

Information Barrier Processor
5214069a-b412-4856-b885-c1db77635b0a

Microsoft Teams Services
47ec8689-d241-46e8-adfa-c7d2144c9848

Microsoft Substrate Management
8f3d0e67-f567-4e23-8e81-2cd5ee520c4c

I then exported this to a CSV, spot checked it for any removals that seemed to not be IB-related, and ran a little PS loop to add the members back to the M365 groups:

$RemovalList = Import-Csv -Path "C:\Temp\TeamsUserReadds.csv"

ForEach ($Removal in $RemovalList) {
	Add-AzureADGroupMember -ObjectId $Removal.GroupId -RefObjectId $Removal.UserId
}

Now for the group chat/meetings removals; these unfortunately were not going to be easy. I started digging into how I would even detect these removals. I first tried M365 audit logs, but the IB-initiated removals did not show in any audit logs. The only place I knew of where you could see a removal was in the chat itself. Whenever a user is added or removed from a group chat/meeting there is an event message generated in the chat. I started digging into the Teams Graph APIs for ways to query these events. Eventually I wrote a pair of scripts to deal with these removals. The first script is GetGroupChatRemovals.ps1. This script crawls through all group chats (and optionally meetings) for a specified set of users over a specified date/time range for removals and creates a CSV of each unique removal. We were able to produce a set of users to scan against because every removal could be traced back to group chat/meeting where there was at least one member who was in one of the IB segments. Sometimes the user who was removed was not in either IB segment, but that didn’t matter because the script would be scanning the whole group chat/meeting for all removals. Since we knew the approximate start and end times of the removals, we had a fairly tight window of ~15 hours to scan. There were multiple challenges with the API, one of them being how it sorts chats when you request them and the inability to properly filter on the last event. I had to make the script walk backwards through chats until it was outside of the specified date/time range. After that list was created it would go through each individual message searching for the deleted system-generated member removed messages. Meetings were even worse. Not only are there many meetings, but for some reason meetings were not being returned in an order based on last event. Because of this you would have to just go through all meetings. For this reason I made scanning meetings optional. This is how the script is executed (I recommend using dot sourcing so you are able to retrieve the output variables after execution in case of a failure or issue):

."C:\Temp\GetGroupChatRemovals.ps1" -CsvPath 'C:\Temp\IBUsers.csv' -TenantId 'aa10484c-ea8b-4d62-a130-54b306384fcc' -AppId '9f3d6a8e-4e1f-436b-9972-a6918e07dbcb' -AppSecret '*PqbUTk,#d;&gK:c@[]4tRHE^TCQ<ZDQ8-NW6Q&9' -StartDateTimeString '2022-12-01T00:00:00.000Z' -EndDateTimeString '2022-12-01T15:00:00.000Z' -OutputCsvPath 'C:\Temp\ChatRemovalsOutput.csv' -ErrorTxtPath 'C:\Temp\ErrorUserUPNs.txt'

The parameters are as follows:

  • CsvPath – A path for a CSV containing all UPNs you are scanning (this will scan for any removals in group chats/meetings that the user was part of)
  • TenantId – Your tenant ID
  • AppId – Your app ID. (You will need to create an app registration with the following permissions: Chat.Read.All, ChatMember.ReadWrite.All. You will also need to request access to the Teams Protected APIs to read chats as an application)
  • AppSecret – The secret created for your application registration
  • StartDateTimeString – The start date/time in UTC ISO 8601 format
  • EndDateTimeString – The end date/time in UTC ISO 8601 format
  • OutputCsvPath – The path to write the output CSV
  • ErrorTxtPath – The path to write the output TXT of UPNs of users the script was unable to retrieve chats for
  • IncludeMeetings – Include this as a switch if you want to scan meetings in addition to group chats. This is optional

Once the script is completed you are left with output like this:

The results this script produced were surprising. IB removed many people (IB segmented and non-segmented) from many group chats. Now that we had this list we used the second script (UndoChatRemovals.ps1) to restore the memberships. This script is fairly simple and you can feed it the exact same CSV that was created from the first script. We actually decided to filter this down to a much smaller list based on the last chat activity column (by excluding stale chats). The four parameters in this script are the same as the first script. This is how the script is executed:

."C:\Temp\UndoChatRemovals.ps1" -CsvPath 'C:\Temp\ChatRemovalsOutput.csv' -TenantId 'aa10484c-ea8b-4d62-a130-54b306384fcc' -AppId '9f3d6a8e-4e1f-436b-9972-a6918e07dbcb' -AppSecret '*PqbUTk,#d;&gK:c@[]4tRHE^TCQ<ZDQ8-NW6Q&9'

Once this script is completed all removed members should have been re-added. There are a few things to note. One is that there will be system messages generated in each chat for each member add event. Another is that there is no way to know if the removed member had limited visibility (added with the ability to only see beyond a certain date) or had access to the entire chat, so we have to add the user back with full visibility.

In conclusion, I will say this all could have been avoided by coupling the two IB activation steps together. Do not activate an IB policy until you are ready to run Start-InformationBarrierPoliciesApplication. The documentation does not mention that any actions happen until you complete the last step and in testing we never saw anything happen until completing the last step. For some reason IB starts to malfunction (seems to take hours for this to happen) if a policy is left active without completing the last step. When running them together it behaves as expected, lesson learned.

Using Application Permissions (and client credentials grant flow) with Hybrid Exchange Graph API

4 Replies

We recently came across an application that uses Graph API and we wanted to start using it for some our on-prem mailboxes. Hybrid Graph API only supports delegated authentication flows and not application authentication flows. Just because something isn’t “supported” doesn’t mean you can’t make it work! There are two things that we’ll need to do to make this work.

First, any internet-facing Exchange server will need to have ‘V1S2SAppOnly‘ OAuth support added. You can do this by adding V1S2SAppOnly to the OAuthHttpModule.Profiles key in the REST web.config (ex. …\Program Files\Microsoft\Exchange Server\V15\FrontEnd\HttpProxy\rest). Once you add this value the key should look like this:

After this has been added either perform an iisreset or restart the ‘MSExchangeRestFrontEndAppPool‘ app pool in IIS for each server where you did this modification.

The next step is to add the appropriate ‘AppOnlyPermissions‘ to the Microsoft Graph partner application in Exchange/AD. First we’ll take a look at our Graph partner application:

Get-PartnerApplication | Where {$_.Name -like '*graph*'} | select *permissions* | fl

AppOnlyPermissions : 
ActAsPermissions   : {Mail.Read, Mail.Write, Mail.Send, Calendars.Read...}

The ‘AppOnlyPermissions‘ value should be blank. We need this to match the ‘ActAsPermissions‘ value. You’d think (and some other articles say) that you could just run Set-PartnerApplication -AppOnlyPermissions… but this was not a supported parameter for me. To set this we’ll have to edit AD directly. You’ll need to fire up something like ADSIEdit, load the AD configuration partition, and drill down to your Exchange org and partner application object. The path should be something like:

CN=Microsoft Graph,CN=Partner Applications,CN=Auth Configuration,CN=YOUR_EXCH_ORG,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=com

Once here you’ll need to open the ‘Microsoft Graph‘ object and copy the ‘msExchConfigurationXML‘ attribute value to the clipboard:

Next, we’ll use Notepad++ and the XML Tools plugin to manipulate this. MAKE SURE YOU BACKUP THIS VALUE. We want to convert this to a nicely formatted XML output so that it is easy to work with. To do this we use the ‘Pretty print’ option in XML Tools.

Once we have this we’ll need to duplicate all of the ‘ActAsPermissions’ lines and then use find and replace to convert those tags to be ‘AppOnlyPermissions’. Doing this will create a set of application permissions based on our delegated permissions (ActAsPermissions).

Once completed we need to linearize the output again so that we can copy it back into AD. We can use the ‘Linearize’ option in XML Tools for this:

Once we have the XML in the proper format we can put it back in the AD object:

Now that we’ve updated the AD object we can verify everything looks good by checking Exchange again (AppOnlyPermissions should have the same values as ActAsPermissions):

Get-PartnerApplication | Where {$_.Name -like '*graph*'} | select *permissions* | fl


AppOnlyPermissions : {Mail.Read, Mail.Write, Mail.Send, Calendars.Read...}
ActAsPermissions   : {Mail.Read, Mail.Write, Mail.Send, Calendars.Read...}

At this point you should be able to access on-prem Exchange resources using the supported Graph API functions with Application Permissions.

NOTE: One limitation of this is that application access policies (set in EXO) DO NOT apply and are ignored when accessing an on-prem mailbox.

AirWatch API – The argument cannot be null

Leave a reply

I recently was creating a PowerShell script that uses the AirWatch REST API to perform mass updates to enrollment users. When testing the process using an API tool (Insomnia) I was receiving the following error when issuing a POST to “/api/V1/system/users/{id}/update“:

<?xml version="1.0" encoding="utf-8"?>
<AirWatchFaultContract
  xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://www.air-watch.com/">
  <ErrorCode>1018</ErrorCode>
  <Message>The argument cannot be null</Message>
  <ActivityId>99311627-d7fd-4fa3-bede-78553fe0ac88</ActivityId>
</AirWatchFaultContract>

I was using an XML body to pass one parameter as per the documentation and the user id was correct. I was unable to find any information on this error and the only thing I was left with was using a JSON body instead of XML for the POST. Once I switched to a JSON body the call was successful. I tested other POST commands using an XML body and they did not produce this error.