Author Archives: Jan Vidar Elven

About Jan Vidar Elven

Microsoft MVP Enterprise Mobility. P-TSP Cloud and Datacenter Management.

Explore Microsoft Graph as a B2B Guest Account

The Microsoft Graph Explorer (https://aka.ms/ge) is always a great learning source and useful tool for querying the Microsoft Graph, especially as you can use your own Azure AD work account or Microsoft account to query for real data. Another advantage with the Graph Explorer is that you can use it without requiring an App Registration in your tenant, something most users are not able to do themselves as they don’t have the administrator rights for registering apps.

However, sometimes working with Microsoft Graph, I find myself in a scenario where I want to use my own work account, but where the Microsoft Graph resources I want to query is in another Organization’s tenant. These kind of scenarios is usually where my account is invited as a B2B guest user to the resource tenant, and currently there are no way to use the Graph Explorer tool to do that.

So I need another tool, and as an IT pro I could easily write myself some PowerShell code, or as a developer I could create a Web app or Console app querying Graph, and run the queries against the other tenant from there.

On the other hand, I want to do this more inline with the Graph Explorer experience, so the most logical choice for me is to use a tool where I can just run the REST API queries I like. And the most popular tool, both by me and many others are the “Postman” client. You can get it yourself for free at https://www.getpostman.com, both Windows, Linux and Mac downloads are supported!

So in this blog post I will show how I use Postman to query Microsoft Graph in a B2B Guest User scenario.

Requirements & Preparation

So, first you will need to get invited to another tenant where the resources you wan’t to query is. You might already have this Azure AD B2B invitation accepted previously. If you aren’t an administrator in that tenant, you will have to ask someone that has the rights to invite to do that for you.

Next, you will need to get assigned permissions to the resources you want to query in the resource tenant. This is all dependent on what kind of queries you want to run, whether it is for reading, writing or deleting resources for example.

Then, you will need an Global Administrator in that resource tenant to create an App Registration. The following instructions and screenshots can be used as a guide:

First, under the Azure AD experience in the Azure Portal, go to App Registrations and create a new Registration, type a name and a redirect URI like shown below:

image

You can type any name you like, for example in this scenario I want to use it for querying identities. I choose to support accounts in this organizational directory only, as this app registration is for members or guests from this tenant. And since I will be using the Postman i specify the redirect URI of “https://www.getpostman.com/oauth2/callback”. This is important as when I authenticate from Postman later the response will be returned to the Postman client.

When accessing Microsoft Graph you have to authenticate using one of the Oauth2 flows, and the most common is using authorization code flow, (https://developer.microsoft.com/en-us/graph/blogs/30daysmsgraph-day-12-authentication-and-authorization-scenarios/), which is exactly this scenario is all about as I will authenticate on behalf of my guest user account in the resource tenant. That means that I will have to add some delegated Graph permissions to the app registration, and in this example I add User.ReadBasic.All, this will make me able to query users via Graph:

image

After granting Admin consent for the permissions, I can verify that the permissions are added correctly:

image

Next I will need to create a client secret to be used in the request to get an access token, go to certificates & secrets and add a client secret of chosen time expiry:

image

Copy and make sure you save the client secret for later next:

image

Then copy the application id and tenant id and save for later:

image

Click on endpoints (see arrow above), and note the Oauth2 authorization and token endpoints (v2), this endpoints contains the tenantid:

image

With these steps the requirements are complete and we can move on to the postman client.

Authenticating and Querying Graph with Postman

There are a lot of features in the Postman client that you should look into when working with REST API’s. You can organize your queries in Collections, save variables in Environments, synchronize your requests across devices and so on. But for now we will start simple and easy.

In the main canvas at your workspace, create a new query, and for example start with https://graph.microsoft.com/v1.0/me:

image

Don’t push Send just yet, we need to authenticate first. Go to the Authorization section under the request, and select Oauth2, then click Get New Access Token:

image

In the following dialog fill in your details as shown below, where:

  • Token Name: You can type what you want here, this is named reference to the Access Token you will aquire.
  • Grant Type: As earlier mentioned, running graph queries as the logged in user (delegated) use the Authorization Code Oauth2 flow.
  • Callback URL: This is the URL that you specified earlier for Redirect URI under the Azure AD App Registration.
  • Auth URL and Access Token URL: These are the URL’s you saw earlier from the Endpoints setting for the Azure AD App Registration (contains the Tenant ID).
  • Client ID: This is the Application ID for the App Registration.
  • Client Secret: This is the secret key you generated under the App Registration.
  • Scope: Provide a default scope, use the default.
  • State: Scope is used for creating application logic that prevents cross use of Access Tokens. You can type anything you want here.
  • Client Authentication: Select to Send client credentials in body when working with Graph requests.
image

When you click request token, you will be taken to the resource tenant for authenticating with your delegated user. Type in your username and click next. This username can either be a normal user that belongs to this resource tenant, or in this case you can log in with your B2B Guest Account:

image

Now dependent on any Conditional Access policies and settings you might be required to approve sign in:

image

After successfully authenticating I should receive a valid Access Token:

image

If you get an error here, please verify your App Registration settings and that the account you logged in as is correct.

Scroll down and click Use Token:

image

You will see that the Access Token is now filled in the Access Token textbox. Next click Preview Request down to the left, this will add the Access Token to the request as a authorization header:

image

If you click on Headers, you can see the Authorization header has been added with the access token as a Bearer Token value:

image

This Access Token is now valid for 1 hour, and you can run as many requests you like as long as you are inside the delegated permissions (for the App Registration) and the logged in users actual permissions. After 1 hour you can request a new Access Token.

PS! Postman will save the authenticated user session in cookies, if you want to log in as a different user clear those cookies, se “cookies” link right below the Send button.

So, now lets run this query, click Send at verify the response, you should get details for your guest user. From the screenshot below you can clearly see that this user is a Guest account looking at the userPrincipalName attribute:

image

Lets try another query, in this query I list all users that has userPrincipalName that starts with “Jan”, and showing only the displayName and userPrincipalName attributes:

image

As you can see from the result above, I have several guest accounts in this tenant (Microsoft Account, Google, Azure AD) as well as a normal user account. You can also see that the Postman client is helpful in specifying my parameters.

Inspecting a B2B Guest Access Token

If you copy the Access Token you got earlier, and paste it into a site like jwt.ms or jwt.io, we can take a look at the access token contents and claims:

image

If I scroll down a little I see the displayname of the App Registration, but the most important info is the mail claim, which for Guest users will be the external e-mail address. Idp is the source authority for the Guest account, in this case another Azure AD tenant with the Tenant Id as shown below:

image

Working with Environments

Chances are that you might work with several environments in Postman, and that where it’s useful to create environment variables. For example create an environment like shown below:

image

That way you can select which environment you want to work with when running queries, and when referencing variables use “{{ .. }}”. For example under Get New Access Token, change to this:

image

Now, lets finalize this blog post by logging in with another guest account, I will choose my Gmail account, I’ve already set up Google Federation and invited this user to my tenant.

First I need to clear the cookies:

image

Next I will click to Get a New Access Token again, and then authenticate as my Google account, which directs me to the Google login page:

image

After successfully authenticating, and using the new Access Token in the Authorization Header, I can run the basic /me query again, this time showing me that I’m now authenticated to Graph with my Gmail user:

image

And looking inside the Access Token again, I can see that the e-mail address is my gmail and the idp is now google.com:

image

If I had logged on with a Microsoft Account the idp value would have been “live.com”.

Next steps

So, now you know how to authenticate to and query Microsoft Graph with an Azure AD B2B Guest User. I really hope this functionality will come to Graph Explorer eventually, but for now Postman is already an awesome free tool for organizing and running your Microsoft Graph queries that I use a lot myself.

The Microsoft Graph Team has also published this source for a lot of useful collections of Graph queries:

https://github.com/microsoftgraph/microsoftgraph-postman-collections

Get and Set Automatic Replies like OOF with Microsoft Graph

Hi, a short blog post this time as Summer Vacation 2019 is here shortly! And on that note, the topic of the post is to show how you can get and set automatic replies with Microsoft Graph. Automatic replies, previously known as Out of Office (OOF) messages is a mailbox setting for each Exchange Online enabled user.

The Microsoft Graph API documentation for mailbox settings is located here: https://docs.microsoft.com/en-us/graph/api/user-get-mailboxsettings?view=graph-rest-1.0&tabs=http, and besides automatic replies you can also get and set locale (language and country/region), time zone, and working hours.

But for now we will only focus on automatic replies, using the automaticRepliesSetting resource type: https://docs.microsoft.com/en-us/graph/api/resources/automaticrepliessetting?view=graph-rest-1.0.

This resource type has the following settings:

{
   "externalAudience": "String",
   "externalReplyMessage": "string",
   "internalReplyMessage": "string",
   "scheduledEndDateTime": {"@odata.type": "microsoft.graph.dateTimeTimeZone"},
   "scheduledStartDateTime": {"@odata.type": "microsoft.graph.dateTimeTimeZone"},
   "status": "String"
 }

Let’s look at some different samples, and I will use Graph Explorer (https://aka.ms/ge). Please note that every end user already have user permission to get or set their own mailbox settings, but you need an Exchange Admin role to get or set the settings for other users in your organization. In addition, if you create your own app registration for Microsoft Graph, you need to make sure the app has either MailboxSettings.Read or MailboxSettings.ReadWrite permission.

In Graph Explorer, after you sign in with your work account, you can modify these permissions if needed:

image

After signing out and in again you will be prompted to consent, if you havent already:

image

Get Current Mailbox Settings

To get your own current settings you can run the following:

GET /me/mailboxSettings

In Graph Explorer this would look like this, and you might have some previous values set here. In my example automatic replies have a status of disabled:

image

To get another users mailbox settings you can run the following (but then you must be an Exchange Admin):

GET /users/{id|userPrincipalName}/mailboxSettings

Simple Update of Status

Lets see how Microsoft Graph can be used to change the status value, there are 3 different settings:

  • disabled. No automatic replies are sent.
  • alwaysEnabled. Automatic replies are sent as specified.
  • scheduled. Automatic replies are sent if between a specific time period.

First, change the method i Graph Explorer to PATCH:

image

Then you need to supply a request body. This sample is just for enabling automatic replies:

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#Me/mailboxSettings",
    "automaticRepliesSetting": {
        "status": "alwaysEnabled",
    }
}

So paste that body to Graph Explorer and then Run Query:

image

You should then get a successful response. Likewise you can set the status to “disabled” again, or to “scheduled”. But using scheduled means that you must set some datetime values as well.

Set Scheduled Automatic Replies

To set scheduled automatic replies, in your request body include the resource types scheduledStartDateTime and scheduledEndDateTime. You can read more about that resource type here, including available time zones: https://docs.microsoft.com/en-us/graph/api/resources/datetimetimezone?view=graph-rest-1.0. This is a sample specifying a scheduled automatic replies:

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#Me/mailboxSettings",
    "automaticRepliesSetting": {
        "status": "scheduled",
        "scheduledStartDateTime": {
            "dateTime": "2019-07-15T08:00:00.0000000",
            "timeZone": "Europe/Berlin"
        },
        "scheduledEndDateTime": {
            "dateTime": "2019-08-09T16:00:00.0000000",
            "timeZone": "Europe/Berlin"
        }
    }
}

Customize internal and external reply messages

The last part is where we put it all together, specifying the following values:

  • internalReplyMessage: Plain text or HTML formatted message sent to all internal users in your organization as the automatic reply.
  • externalReplyMessage: Plain text or HTML formatted message sent to all external users as the automatic reply, but depending on this value:
  • externalAudience: If “none”, no external users will get automatic replies, if “contactsOnly” replies will only be sent to users in your contacts, and if “all” every external user will get a reply.

So this is a working sample of a complete request body:

{
    "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#Me/mailboxSettings",
"automaticRepliesSetting": {
        "status": "scheduled",
        "externalAudience": "contactsOnly",
        "internalReplyMessage": "<html&gt;\n<body&gt;\n<div&gt;</div&gt;\n<div&gt;Hi, I'm enjoying summer vacation 2019. I'm back at work August 12th!</div&gt;\n<div&gt;<br&gt;\n</div&gt;\n<div&gt;Kindly Regards</div&gt;\n<div&gt;Jan Vidar Elven</div&gt;\n<div&gt;</div&gt;\n</body&gt;\n</html&gt;\n",
        "externalReplyMessage": "<html&gt;\n<body&gt;\n<div&gt;</div&gt;\n<div&gt;Hi, I'm enjoying summer vacation 2019. I'm back at work August 12th!</div&gt;\n<div&gt;<br&gt;\n</div&gt;\n<div&gt;I'll only read e-mails intermittently, and rarely respond before I'm back. Please contact management if anything urgent business needs follow up. Contact info on our website.</div&gt;\n<br&gt;\n</div&gt;\n<div&gt;Kindly Regards</div&gt;\n<div&gt;Jan Vidar Elven</div&gt;\n</div&gt;\n<div&gt;</div&gt;\n</body&gt;\n</html&gt;\n",
        "scheduledStartDateTime": {
            "dateTime": "2019-07-15T08:00:00.0000000",
            "timeZone": "Europe/Berlin"
        },
        "scheduledEndDateTime": {
            "dateTime": "2019-08-09T16:00:00.0000000",
            "timeZone": "Europe/Berlin"
        }
    }
}

In Graph Explorer:

image

And I can verify in my Outlook settings:

image

Summary and Usage Scenarios

Beside that it is always fun to learn something new about the Microsoft Graph, and automation, the reality is that for many users they will just click to enable or disable automatic replies directly in their Outlook client, both Office Outlook, Outlook Mobile and Outlook on the Web supoorts this. Finding out how to do it with Graph took me just under 2 hours, including writing this blog post. But then again, I learned something new! And I picked up a couple of more tips and tricks on different JSON Request Body constructs 😉

Anyway, in a bigger picture, Graph API is great for customizing, integrating, reporting and automating, so if your organization maybe have create a vacation calendar, you could use the Graph API to automatically enable or disable out of office replies, this is just one example, many more will exist. Please share with me in the comments if you have done or plan to do something with this or similar.

Smile

But first: Summer Vacation 2019! And I’m all set with automatic replies !

Speaking at Microsoft Ignite – The Tour Stockholm

It is with great pride I can announce that I will speak at Microsoft Ignite – The Tour, in Stockholm April 24-25 2019. This conference, which already is sold out and now only available on waiting list, will be held at Stockholmsmassan, and you can read more about it here: https://www.microsoft.com/sv-se/ignite-the-tour/Stockholm.

Ignite The Tour Stockholm will have over 150 different breakout sessions, theatre sessions, modules and workshops, covering 10 learning paths. Speakers will be from Microsoft, and from the MVP and Regional Director community, which I’m so fortunate and honored to be part of 😉

I will be speaking at the following sessions:

In addition to this I will participate on the following hands-on workshop as a proctor:

You will also find me at the experts area and doing booth duty, I will be at the following demo stations at the Hub and Microsoft Showcase area, covering topics on Azure and Azure AD:

  • Demo Station #5 (Azure): Day 1 1600-1800: “Getting Started with Azure Log Analytics and Azure Monitor using Azure AD Activity Logs
  • Demo Station #6 (Azure): Day 2 0800-1200: “Getting Started with Azure Log Analytics and Azure Monitor using Azure AD Activity Logs

I’m really excited for presenting these sessions and being part of the Ignite The Tour Community! Hope to see you there 🙂

Exploring Azure MFA sign-in failures using Log Analytics

Most IT admins, pros and end users from organizations that use Office 365 and Azure AD will by now have heard about the big Azure MFA outage on Monday November 19. When something like this happens, it is important to get insights on which users that were affected, and in what type of scenarios they most experienced the problem. Microsoft MVP Tony Redmond wrote a useful blog post (https://office365itpros.com/2018/11/21/reporting-mfa-enabled-accounts/) on how to report on possible/potentially affected and MFA enabled users, and how to disable and re-enable those users. But many organizations are now using Conditional Access policies using Azure AD Premium, so this will be of limited help for those.

If I could wish one thing from Microsoft for Christmas this year, it would be to be able to manage MFA and Conditional Access policies with Azure AD PowerShell and Microsoft Graph! Admins could then run “break-the-glass” administrative users (or even “break-the-glass service principals”) to disable/re-enable policies when big MFA outages happens. A good CA policy design, trusting compliant devices and secure locations could also go a long way in mitigating such big outage problems.

Tony’s blog post made me think about the feature I recently activated, on integrating Azure AD Activity Logs to Azure Log Analytics, you can read more about this here: https://gotoguy.blog/2018/11/06/get-started-with-integration-of-azure-ad-activity-logs-to-azure-log-analytics/

By exploring the Sign-in logs in Log Analytics I could get some more insights into how my organization was affected by the MFA outage on November 19. Please see the above blog post on how to get started setting up this integration, the rest of this blog post will show some sample queries for the SigninLogs.

Querying the SigninLogs for failed and interrupted sign-ins

All the queries seen below are shown in screenshot images, but I have listed all them for you to copy at the end of the blog post.

First I can take a look at the SigninLogs for the specific day of 19th November, and the grouping on the result type and description of the sign-in events. For example I can see that there is a high number of event 50074: User did not pass the MFA challenge. Interestingly there is also a relatively high number of invalid username or password, that could be a separate issue but could also be that users that fails MFA sign-ins tries to log in again thinking they had wrong password first time.

image

Changing that query a little, I can exclude the successful sign-ins (ResultType 0), and sort on the most count of failures. Two of the events of most interest here is 50074 and 50076:

image

In this next query I focus on the “50074: User did not pass the MFA challenge” error. By increasing the time range to last 31 days, and adding a bin(TimeGenerated, 1d) to the summarize group, I’ll be able to see the count of this error on each day in the last month. This will give me a baseline, and we can see that on the 19th this number spikes. I have added a render to timechart for graphical display. There are also some other days where this number increases, I can look into more insights for that if I want as well, but for now I will focus on the 19th.

image

Back to the time range of the 19th of November, I can modify the summarize to group by each hour, by using bin(TimeGenerated, 1h). This will show me how the problems evolved during the day. Must errors occurred about 10 am in the morning:

image

Lets look at some queries for how this error affected my environment. First I can group on the Users and how many errors they experience. Some users were really persistent in trying to get through the MFA error. I have masked the real names. We also see some admin accounts but admins quickly recognized that something was wrong, and actively sought information on the outage. By midday most users were notified on the on-going outage and the number of errors slowly decrease during the day.

image

In this next query, I group on the Apps the users tried to reach:

image

And in this following query, what kind of Client App they used. It would be normal that Browser is quite high, as mobile apps and desktop clients are more likely to have valid refresh tokens.

image

In this query I can look into the device operating system the users tried to sign in from:

2018-11-21_22-05-27

In the following query I can look at which network the users tried to log in from, by identifying IP address:

image

And in this query we can get more location details from where users tried to sign in:

2018-11-21_22-07-17

Summary

Querying Log Analytics for Sign-in events as shown above can provide valuable insights into how such an outage can affect users. This can also give me some input on how to design Conditional Access policies. Querying this data over time can also provide a baseline for normal operations in your environment, and make it easier to set alert thresholds if you want to get alerts when number of failures inside a time interval gets higher than usual. Using Azure Monitor and action groups you can be pro-active and be notified if something similar should occur again.

Here are all the queries shown above:

// Look at SigninLogs for a custom date time interval and group by sign in results
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| summarize count() by ResultType, ResultDescription

// Exclude successful signins and format results with sorting
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType != "0" 
| summarize FailedSigninCount = count() by ResultDescription, ResultType 
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error last month to see trend
// and present to line chart group by each 1 day
SigninLogs
| where TimeGenerated >= ago(31d)
| where ResultType == "50074"
| summarize FailedSigninCount = count() by ResultDescription, bin(TimeGenerated, 1d)
| render timechart

// Look at User did not pass the MFA challenge error on the MFA outage day
// and present to line chart group by each 1 hour to see impact during day
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074" 
| summarize FailedSigninCount = count() by ResultDescription, bin(TimeGenerated, 1h)
| render timechart

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on users to see affected users
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by UserDisplayName
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on Apps to see affected applications the users tried to sign in to
SigninLogs
| where TimeGenerated  between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by AppDisplayName
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on client apps to see affected apps the users tried to sign in from
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by ClientAppUsed
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on device operating system to see affected platforms
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by tostring(DeviceDetail.operatingSystem)
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on IP address to see from which network users tried to sign in from
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by IPAddress
| sort by FailedSigninCount desc

// Look at User did not pass the MFA challenge error on the MFA outage day
// and group on users location details to see which country, state and city users tried to sign in from
SigninLogs
| where TimeGenerated between(datetime("2018-11-19 00:00:00") .. datetime("2018-11-19 23:59:59")) 
| where ResultType == "50074"
| summarize FailedSigninCount = count() by tostring(LocationDetails.countryOrRegion), tostring(LocationDetails.state), tostring(LocationDetails.city)
| sort by FailedSigninCount desc

Alert on On-premises Connectivity for Self Service Password Reset using Azure Monitor and Azure AD Activity Logs in Log Analytics

Recently I wrote a blog post on how to get started with integration of Azure AD Activity Logs to Azure Log Analytics. Setting up this is a requirement for the solution in this blog post, so make sure you have set this up first: https://gotoguy.blog/2018/11/06/get-started-with-integration-of-azure-ad-activity-logs-to-azure-log-analytics/.

In this blog post I wanted to show a practical example on how to create an alert for when Azure AD Self Service Password fails in password writeback because of connectivity error to the On-premises environment.

Build the query

If you know the schema, you can write the query directly, but more often than not you will work out these scenarios by exploring your actual log data. In my example we had a concrete example where password resets failed because of On-premises connectivity error. Looking into the Azure Log Analytics logs, I started with this simple query against AuditLogs:

image

After that I looked into the filters, and found that I could filter on Failures:

image

This resultated in some failure results. Exploring the results in the bottom right windows, I found that the failures had a ResultDescription of “OnPremisesConnectivityError”:

image

By clicking on the plus sign above I add that to the query:

image

I want to save my query next, so that I have it available for later:

image

Now that I have the results I want I can proceed to create an Alert Rule. Btw, here is the full query (I have since amended it to include OnPremisesConnectivityFailure in addition):

AuditLogs
| where Category == "Self-service Password Management"
| where ResultType == "Failure"
| where ResultDescription == "OnPremisesConnectivityError" or ResultDescription == "OnPremisesConnectivtyFailure"

Create an Alert Rule

Next, I can create a new Alert Rule for this query, something you can do directly from the query:

image

This next step would bring me over to the Azure Monitor and Rules Management section. The alert target (OMS/Log Analytics Workspace) and target hierarchy (Azure Subscription and Resource Group) should already be specifed:

image

Now I need to configure the alert criteria. Note that currently monthly cost for this alert is $1.50. Click on the criteria to configure the signal logic:

image

Here we see the query from before, as well as we need to set a threshold for number of results, and a period and frequency. Pricing details can be found at: https://azure.microsoft.com/en-us/pricing/details/monitor/. If you look at Alert signals and Log section, you’ll see that alerts with frequency of 5 minutes is $1.5, 10 minutes $1 and 15 minutes $0.5. This is per log monitored. I changed my period and frequency to 15 minutes:

image

After I click done I see that the alert criteria is correctly configured with a price of $0.50:

image

Next I need to specify the alert details, for example like below. You also have the option to supress multiple alerts inside a time window. I configured 60 minutes:

image

Next I need to either select an existing action group or create a new. An action group decides which action to take when an alert occures:

image

I’ll create a new action group for now. In this action group I will select to send an e-mail to a group in my company. As you see in the image below I have several options for action type, examples of use can be:

  • Trigger e-mail, SMS, push or other notifications.
  • Trigger Azure Functions for running some code logic.
  • Trigger Logic App for executing a business flow logic.
  • Trigger a webhook for posting a status, for example to a Microsoft Teams webhook.
  • Send the Alert via ITSM connector to create an incident in your connected ITSM system.
  • Trigger a Runbook in Azure Automation to run your own PowerShell runbooks, or you can use one of the built-in runbooks for restart, stop, remove or scale up/down VM.

image

When selecting Email I need to specify an e-mail address for the user/group I want to notify:

image

After that I’m ready to create the Alert rule:

SNAGHTML50ef585

After I created the rule, the group e-mail address I specified received this e-mail, confirming that it is now part of an action group:

image

If you want to locate and change this alert rule at a later stage, you will find it under Azure Monitor and Alert Rules:

image

Thats it, now we can just wait for future Self-Service Password Reset or Change connectivity errors, and we will get notified.

Testing the Alert rule Notification

For testing, I just wanted to force the error by logging into my Azure AD Connect Server and stop the service.

After that I tried to reset or change my password, resulting in this error message shown to the user:

image

Now in this situation, most users will either just wait and try again later, try one more time and then give up, and if you are lucky they will contact their IT admin and notify of the error. More often than not users just leave it there, and not notify anyone. This is where it is useful to get an alert as we have created here, because then you as an IT admin can proactively analyze and fix the error before it affects more users. This is the alert I received to my specified group:

image

We can directly click on the result to get into details for the error, for example which user was affected, from which IP address and more:

image

I don’t know about you, but I think this is just brilliant 🙂 With the integration of Azure AD Activity Logs in Log Analytics, I can really explore and analyze a lot of the operations going on in my tenant, and using Azure Monitor I can create alert rules that notifies or trigger other actions to handle those alerts.

Thanks for reading, more blog post will follow on this subject of Azure AD and Log Analytics, so stay tuned!

Get started with integration of Azure AD Activity Logs to Azure Log Analytics

Recently Microsoft announced the availability of forwarding the Azure AD Activity Logs to Azure Log Analyctis. You can read the announcement in full here: https://techcommunity.microsoft.com/t5/Azure-Active-Directory-Identity/Azure-Active-Directory-Activity-logs-in-Azure-Log-Analytics-now/ba-p/274843.

By bringing thousands (or even millions depending on your organization size and use of Azure AD), of sign-in and audit log events to Log Analytics you can finally use the power of Log Analytics for query, analyze, visualize and alert on your data.

In this blog post I will show how to get started and provide some useful tips. Most of this is already well documented in the following Microsoft Docs, but I will provide my own perspective and experience and as well let this blog post be an anchor for future detailed blog posts on the subject of analyzing Azure AD sign-in and audit logs in Log Analytics and Azure Monitor:

Set up Diagnostic Settings to Log Analytics

The first action we need to do is to Turn on diagnostics in the Azure AD Portal. You will need to be a Global Administrator or Security Administrator to do this:

image

PS! Another way to get to this setting to Turn on diagnostics is to either go to Sign-ins or Audit logs under Monotoring, and from there click on Export Data Settings:

SNAGHTML591fe33

Next select to Send to Log Analytics, and then select either or both of the AuditLogs or SigninLogs.

image

Note that to be able to export Sign-in data, your organization needs Azure AD Premium P1 or P2 (or EMS E3/E5). This requirement only applies to sign-in logs, not audit logs.

After selecting Log Analytics, and which logs to export, you need to configure which Log Analytics (still named as OMS) workspace to export the data to:

image

Note that this requires access to an Azure Subscription. You can either select an existing OMS workspace or create a new:

image

Important info! Usually you will need to be a Global Administrator or Security Administrator to be able to access the details of Sign-in logs or Audit logs in Azure AD, but by exporting this data to either an existing or a new Log Analytics workspace, potentially a lot more users can access that data. You need to think about if this is something you want to do, and at least control and govern which users can access that Log Analytics workspace.

For this reason alone it would probably be a better idea to create a dedicated Log Analytics workspace for the Azure AD activity logs:

image

Regarding pricing, using a Log Analytics workspace for Azure AD Activity Logs alone should not incur a notable cost in most normal environments. In an environment of less than 100 users I found the following consumption per day, which is way below the amount of free data you get included:

image

If you want to save and use that same query yourself, here it is:

Usage| where TimeGenerated > startofday(ago(31d))| where IsBillable == true
| where (DataType == "SigninLogs" or DataType == "AuditLogs") and Solution == "LogManagement"
| summarize TotalVolumeGB = sum(Quantity) / 1024 by bin(TimeGenerated, 1d), Solution| render barchart

Choosing a pricing tier depends on whether the Subscription was created before April 2, 2018 or not, or whether you have elected to move to a new pricing model. The older pricing model had a choice of free tier, which had a daily cap of 500 MB and a data retention of 7 days. As the diagram above showed, most organizations will be way below the 500 MB daily cap, but a retention of only 7 days will be considered short for most analyzing needs. So under the older pricing model you would consider a standalone per GB model, giving a retention of 1 month by default, but a cost of $2.30 per GB.

The new pricing model after April 2, 2018 has a simplified pricing model. Here the first 5 GB are free and you have a default retention of 31 days. Additional GBs for ingestion are $2.99 per month, and extra retention after the first 31 days is $0.13 per GB per month. Note that this pricing model is on subscription level and affects all your Log Analytics workspaces, so you need to carefully consider any changes to the new pricing model in your subscription.

After you have selected/created a Log Analytics workspace, and provided a name for the Diagnostic settings, you are ready to Save:

image

After about 15 minutes you can start explore the Logs in the Log Analytics workspace.

Start to Analyze Azure AD Activity logs with Log Analytics

To begin analyze the exported Azure AD Activity Logs with Log Analytics, you can either go to the Log Analytics section in your Azure Portal. You can also access the logs directly from Azure Active Directory from under the Monitoring section, which will take you directly to the configured Log Analytics workspace:

image

By default this will open a search query showing sample data from all your Log Analytics workspace.

I find that a good way to start learning about the sign-in and audit logs is to look at the schema. The SigninLogs and AuditLogs schemas should appear right under LogManagement as shown below:

SNAGHTML5fc1b7b

To look at the SigninLogs just add that to the query window and select a time range and click Run:

image

Depending on your sample data you can start filter on the left side, for example to look at only certain app sign ins, or client apps used, location and more..

Similarly for AuditLogs, in the following example I have set a time range of last 7 days:

image

See the links in the beginning of this blog post for some more sample queries and you can also import some sample views.

So now that we have a working diagnostic setting that exports my Azure AD sign-in and audit logs to Azure Log Analytics, I’m ready to explore some interesting scenarios for analyzing this data. This will be a topic for upcoming blog posts, so stay tuned for that!

Thanks for reading so far, I’m really excited for this feature! Smile

Exporting and Importing PowerApps and Flows Package that use a Custom Connector

Just recently I published a blog post on how to use PowerApps and Flow with a custom connector using Microsoft Graph API, to create an app for Azure AD PIM (Privileged Identity Management): https://gotoguy.blog/2018/09/15/create-your-own-azure-ad-pim-app-with-powerapps-and-flow-using-microsoft-graph/.

In this blog post I want to share some instructions and experiences on exporting the PowerApp and Flows to a package, and how you can export the Custom Connector definitions to a swagger file. After that I will show how you in a new environment can import these definitions, and import the PowerApp and Flow package.

Even better, based on the aforementioned blog post on the Azure AD PIM App, I will provide you with download links for the custom connector swagger defininiton for Microsoft Graph, as well as the PowerApp and Flows Package, so you can start from there without having to build all the stuff yourselves Smile!

Export the PowerApps Package

First, start in your Apps gallery of PowerApps, find the Export package (preview) button as shown below:

image

Specify a package name, environment and optionally a description as I have below:

image

Next, review the package content. For the Azure AD PIM App, I’ll change the Import Setup to “Create as new”, the same for the 3 Flows, as shown below:

image

For some of the resources you can select between Create as new or Update, and as I’m planning to import this as a new App with new Flows in the environment, I’ll change this from the default.

image

The other resources (like the connector and connections) I will select during import. This means these will have to be already existing in the environment I want to import the package to.

I can then download the package:

SNAGHTML9abd28

The package is downloaded as a zip file:

image

Inside the zip file there are some manifest json files and the PowerApps and Flows definitions:

image

Export the Custom Connector swagger file

The next thing we want to do is to export the custom connector and its operations. Go to Custom connectors in the menu:

image

Find the “PowerApps Microsoft Graph” connector, and click on the down arrow as shown below. This will download a swagger definition file in JSON format:

SNAGHTMLb16f76

You can open and inspect that JSON file in your favorite JSON editor, here is mine shown in Visual Studio Code:

image

Community Download

Courtesy of gotoguy.blog, I’ll provide you with a download for both the PowerApps/Flows package, as well as the Custom Connector Swagger JSON file. This is helpful if you want to skip right ahead to the next Import section.

These files are placed at my GitHub, in the following repositories:

Import the Custom Connector swagger file

In the new/target environment we will first have to import the swagger file for the Custom Connector. Here you have 2 options:

  1. You can create a new custom connector, and Import from an OpenAPI file/URL:

    image

  2. Or, if you already have a Custom Connector for Microsoft Graph, you can select to Update the existing connector from OpenAPI file/URL:

    image

For sake of education, lets try both variants. The first time you will have to create a new custom connector anyway, but later you will only need to update if there are any changes. I will use OpenAPI URL, as the swagger file is avaiable at my GitHub here: https://raw.githubusercontent.com/skillriver/PowerAppsFlowCustomConnector/master/MicrosoftGraphApi/PowerApps-Microsoft-Graph.swagger.json

PS! Prerequisite

Remember that to be able to use a Custom Connector and Microsoft Graph, you will have to create or use an App Registration in Azure AD in your target enviroment, like I have described in this blog article, under the section “App Registration”: https://gotoguy.blog/2017/12/17/access-microsoft-graph-api-using-custom-connector-in-powerapps-and-flows/.

Take a note of the application ID and secret key:

image

Remember also to give the App the right Microsoft Graph Permissions, and give Admin grant if needed:

image

Import from OpenAPI URL

To create a new custom connector, select to import from OpenAPI URL:

  1. Type a name for the Custom connector, and paste in the URL for the swagger json file:

    image

    Verify the URL and click Continue.

  2. Following that, verify that host is graph.microsoft.com and base URL is “/”, and optionally specify a connector icon, color and description:

    image

  3. On the security page you have to specify the client id which is the app id from the registered app in your target Azure AD environment, as well as client secret and resource URL:

    image
    In my target environment I have pasted in the client id, secret, and the resource URL is https://graph.microsoft.com. Note that the Redirect URL is not available before after the custom connector is saved:

    image

    Click to go to the next Definition page.

  4. At the Definition page, the actions are already in place because they were defined in the OpenAPI swagger file:

    image

    Click “Create connector”.

  5. After the Connector is created and saved, go back to Security, and copy the Redirect URL:

    image

  6. Make sure that the Redirect URL is on the list of the Reply URLs of the Azure AD App Registration:

    image

  7. Back in the Custom Connector, lets test the connector. Go to the Test page and create a connection:

    image

  8. After establishing a connection with your user account, you can go ahead and test one or more of the operations and verify that they run successfully:

    image

After testing this the custom connector is ready to use.

Update from OpenAPI URL

I you want to update an existing custom connector, select to Update from OpenAPI URL:

  1. Provide the URL for the swagger json file:

    image

  2. As with when creating a new custom connector, verify that host is graph.microsoft.com and base URL is “/”, and optionally specify a connector icon, color and description:

    image

  3. When updating an existing connector, you only have to specify the client secret again:

    image

    If you don’t have the original secret stored securely somewhere, you have to go to the App Registration in Azure AD and generate a new one.

  4. Verify that the Operations now has been updated from the imported OpenAPI swagger json file:

    image

    Click Update Connector to save the changes.

  5. After this, go to Test, and either use an existing connection or create a new, and the Test some of the operations to verify:

    image

Now we are ready import the PowerApp and the Flows that will use this custom connector.

Import the PowerApps and Flows Package

We can now import the package we exported earlier, or if you want to use the community download from my GitHub repository, make sure that you download the zip package before this next step.

Start by selecting Import package (preview) from the PowerApps menu:

image

Then browse to the zip packaged to start uploading:

image

When the upload is complete, we can review the package content. We have to select during import the connector and connections, marked as red under here:

image

After selecting the custom connector, and changing the connections to the target environment, we are ready to Import:

image

Note that you also can change the name of the PowerApp and Flows by clicking on the wrench symbol.

Click Import when you are ready, and verify that the import is successful:

image

You can now proceed to open the app for customizations and testing. If prompted, click to Allow the permission request:

image

After opening the Azure AD PIM App, now in the target environment, hold down the ALT key and click Refresh My Roles to test. And you should get the logged on users roles:

image

Obviously, now in the target environment, you would probably start to customize the logo, colors, label texts and language, if you don’t want to proceed with the “Elven” theme 😉

For example something like this from my company:

image

With that I can conclude this blog post, we have been able to export the custom connector definition and the PowerApps package including the Flows, and import these into a new environment. Now all that is left is to publish and share the PowerApp to be used in your organization.

Thanks for reading, hope it has been helpful!