Monitoring Microsoft 365
This best-practices guide is intended for readers who already have some proficiency in ThousandEyes and are ready to delve into more advanced guidance. It assumes a basic understanding of networking concepts and familiarity with the ThousandEyes platform. If you're new to ThousandEyes, we recommend starting with our getting started guides to establish a solid foundation.
This article describes monitoring for a third-party application, and might not apply to all use cases. Because the third-party company updates their application separately from ThousandEyes, the information outlined here might not represent the application's latest architecture. ThousandEyes will provide best-effort support for these solutions.
Audience Prerequisites
To effectively follow this guide, you should be:
Familiar with the three types of ThousandEyes Agents and the test types for Cloud and Enterprise Agents and Endpoint Agents
Familiar with ThousandEyes’ role-based access control settings to ensure your user account has the necessary permissions
Able to deploy Enterprise Agents and/or Endpoint Agents, if they’re not already deployed
Aware of your organization’s available licenses for Endpoint Agents and Internet Insights, and available units for Cloud and Enterprise Agents. For information on your organization’s usage and capacity, see the articles in the Usage-Based Billing section.
Comfortable with networking concepts, such as:
TCP/IP, HTTP, DNS, and how they relate
The difference between DNS resolvers (recursive DNS servers) and authoritative DNS nameservers
Learn more in the ThousandEyes Learning Center
Hub-and-spoke WAN architecture vs direct internet egress
Introduction
This guide will teach you how to monitor your workforce digital experience for Microsoft 365, formerly known as Office 365. To do this, you'll use multiple features within ThousandEyes: ThousandEyes Cloud and Enterprise Agents, ThousandEyes Endpoint Agents, and ThousandEyes Internet Insights. (Microsoft Teams is not included here, but instead will have its own guide published separately.)
Microsoft 365 Architecture
To monitor a SaaS suite like Microsoft 365, it is important to understand its architecture: where and how the services are hosted, and how your users reach those locations.
Microsoft 365 is a composite of multiple services, maintained by different groups within Microsoft. Any of these services can change (network architecture change, new code release) without necessarily affecting the others. This is why it is important to monitor each of these services individually, and not just monitor one of them.
Microsoft 365 includes:
SharePoint Online: a solution for creating websites to share documents and information securely with colleagues and customers. SharePoint Online is hosted on hardware managed by the SharePoint Online business unit.
Office Online: a solution to view and edit documents via web browser. Office Online is hosted on Azure virtual machines managed by the Office Online business unit, while the underlying hardware is managed by Microsoft Azure.
Exchange Online: a hosted email solution for business. Exchange Online is hosted on physical servers managed by the Exchange Online business unit.
These services operate through a vast and distributed global network. User experience, including performance and reliability, relies on connectivity through various service front doors spread across hundreds of Microsoft datacenters in dozens of regions. The design of the data centers prioritizes geographical proximity to customers. Microsoft uses anycast DNS to direct users to the best location. Microsoft also uses software-defined networking (SDN) to proactively and automatically manage network traffic on a large scale. The network architecture automatically redirects traffic in the event of failures.
In most cases, Microsoft recommends routing your user requests to the nearest Microsoft 365 service front door, instead of backhauling through a central location, in order to ensure the best possible user experience. Whenever possible, try to avoid backhauling traffic from remote sites (for example, branch offices) to a centralized Internet egress point (for example, your head office or data center).
The two illustrations below show the two prevailing enterprise network architectures. On the left, branch offices must backhaul through a central Internet egress point at the head office, leading to worse user experience because traffic is not routed to the nearest Microsoft 365 front door for all users. On the right, branch offices have local Internet breakout, and can directly reach the nearest Microsoft 365 service front door.
Your Microsoft 365 Tenant
The sections below describe monitoring configurations that test towards both common/shared Microsoft 365 services as well as services that are unique to your tenant. The tenant here refers to your organization's unique dedicated instance of Microsoft 365 services; you can learn more about Microsoft 365 tenants in Microsoft's documentation. To monitor your organization's Microsoft 365 tenant, it is necessary to identify the unique subdomains, sometimes called “vanity domains”, assigned to your tenant.
To identify your unique SharePoint and OneDrive URLs:
Navigate to https://microsoft365.com in your web browser and sign in, if not already signed in.
Use the Search field at the top of the page to search for SharePoint or OneDrive.
Click the app’s icon in the search results to launch the app.
Look at your web browser’s address bar for your organization’s vanity domain.
Note that the vanity URLs are in the format of https://<tenant>.sharepoint.com and https://<tenant>-my.sharepoint.com.
Monitoring Microsoft 365 with Cloud and Enterprise Agents
To monitor Microsoft 365, use Enterprise Agents from within your own secure networks, and Cloud Agents to monitor from outside of your own networks. This section includes where to locate your ThousandEyes agents, and what to test from each vantage point.
Agent Placement and Selection
Use ThousandEyes Enterprise Agents to proactively monitor your Microsoft 365 digital experience from vantage points within your enterprise WAN. This is known as “inside-out” testing.
In the default template configuration, Enterprise Agents are required to deploy the Microsoft 365 test template. If you have not yet installed any Enterprise Agents, see the Installing section of the Enterprise Agent documentation.
At a minimum, place an Enterprise Agent at each Internet egress point.
For hub-and-spoke network architectures with centralized egress, the recommended best practice is to also test from Enterprise Agents at each “spoke” user location. Spokes are typically branch offices.
Enterprise Agents are used to monitor DNS resolution (either internal resolvers or public resolvers), and to monitor Microsoft 365 web application performance.
Enterprise Agents are required in 10 of the 20 tests included in the Microsoft 365 test template. If you want to deploy the Microsoft 365 test template without any Enterprise Agents, you must:
Exclude all of the "Internal" tests from the template
See the Excluding Tests from the Template Deployment section in Using the Microsoft 365 Test Template later in this article.
Select any Cloud Agent in the Which Enterprise agents should we run tests from? agent selector
The template requires that all user input fields not be blank. If you disable all the "Internal" tests, your selection in this field will not be used in any of the deployed tests, but making a selection is nonetheless required because of user input validation.
Use ThousandEyes Cloud Agents as follows:
Monitor public DNS infrastructure and Microsoft’s authoritative nameservers
Establish a baseline for web application performance outside of the enterprise network.
It is recommended to select at least one Cloud Agent for each region where you have an Internet egress and Enterprise Agent.
Note on Proxies
Web proxy configuration can be applied either at the agent level or at the test level. The tests that are included in the ThousandEyes test template for Microsoft 365 do not specify any proxy configurations at the test level. Instead, these tests default to use the proxy configuration of each individual agent. This allows you to deploy the Microsoft 365 test template using a mixture of agents with and without a proxy; for example, to compare the performance of Microsoft 365 when accessed directly versus through the proxy.
For more information on configuring agent-level proxies, see the Proxy Settings section of the Working with Agent Settings article.
Microsoft 365 Test Template
ThousandEyes provides a single test template for Microsoft 365 that includes all of the Cloud and Enterprise Agent tests that you’re most likely to want to use.
The ThousandEyes Microsoft 365 test template includes each of the test types described below for five critical Microsoft 365 web applications:
Outlook (part of Exchange Online)
SharePoint and OneDrive (part of SharePoint Online)
The Microsoft 365 application portal (part of Office Online)
The Microsoft Online login service (a common shared service)
Included Tests
There are three types of tests included in the Microsoft 365 test template:
HTTP server tests run from both ThousandEyes Enterprise Agents and Cloud Agents. HTTP server tests are used to monitor the performance, reachability, and availability of Microsoft 365 web applications. The test template includes two HTTP server tests for each application: one sourced from the selected Enterprise Agents (“internal”) and one sourced from the selected Cloud Agents (“external”).
DNS server tests run from ThousandEyes Enterprise Agents (“internal”). These tests monitor network connectivity to your DNS recursive resolvers (public or internal) and DNS resolution performance.
DNS trace tests run from ThousandEyes Cloud Agents (“external”). DNS trace tests monitor the DNS from the root nameservers, through the top-level domain (TLD) nameservers, and down to the Microsoft 365 authoritative nameservers.
Both the HTTP server and DNS server test types are multi-layer: they automatically include network tests towards their targets, measuring packet loss, latency, jitter, and identifying the network path, in addition to the application metrics they collect. See Test Type Layering for more information on test layers.
Monitoring DNS in addition to HTTP for application availability and performance is crucial because DNS is critical for translating domain names to IP addresses. Any issues or delays in DNS resolution can significantly impact the accessibility and performance of a web application, even if the HTTP service itself is functioning properly. DNS is especially important for Microsoft 365 due to the large anycast DNS architecture, the use of low time-to-live values (i.e, minimal caching and frequent lookups), and because DNS is used to direct users to the best service front door.
Note that the Microsoft 365 test template includes two HTTP server tests for each target application: one sourced from Enterprise Agents (the “Internal” tests) and one sourced from Cloud Agents (the “External” tests). This means there are 20 total tests in the test template: 5 targets * (1 DNS server test + 1 DNS trace test + 2 HTTP server tests).
This configuration consumes the same amount of units as if the HTTP server tests were combined into a single test from both types of agent. However, it is more flexible in terms of visualizing the data in dashboards and test views and for fine-tuning alert rules. For example, you may decide to disable alerts for the tests which only run from external vantage points and only treat alerts from internal vantage points as actionable.
A high-level diagram is shown for each of these test types below, in the next section. When you deploy the test template, you will have the opportunity to deselect any of the tests you do not want to include in your Microsoft 365 test suite.
HTTP Server Test Deployment
The diagram below shows HTTP server testing from Cloud Agents and Enterprise Agents. As described previously, Enterprise Agents should be placed at each Internet egress point – in the diagram below, this includes the branch office shown at the top, and the head office. If any branch offices do not have direct Internet egress and instead backhaul through an enterprise WAN, Enterprise Agents should also be deployed at the branch office location if possible. The Enterprise Agent HTTP server tests monitor each site’s network path and quality to the Microsoft 365 service front doors along with application performance and availability metrics.
The Cloud Agent HTTP server tests provide a baseline and reference point to complement your own Enterprise Agent tests. The combined visibility of internal and external vantage points allows you to quickly and easily identify the scope of any issues, such as:
Issues solely within your enterprise network or network edge (e.g., one or more Enterprise Agents affected, no Cloud Agents affected)
Issues within regional transit providers (e.g., multiple Enterprise Agents and Cloud Agents affected within a particular region or ISP)
Issues within Microsoft 365 itself (e.g., all Enterprise Agents and Cloud Agents affected, either within a region or globally)
For more details on the metrics collected by HTTP server tests, see the HTTP Server section of the ThousandEyes Metrics article.
DNS Server Test Deployment
The diagram above shows DNS server testing from Enterprise Agents. When your users try to access Microsoft 365, their device must be able to resolve the hostname (e.g, pseudoco.sharepoint.com) to the IP address for the service front door. There are three critical components to this operation:
The device’s configured DNS resolver
The public DNS infrastructure (including root nameservers and top-level domain nameservers)
The authoritative nameservers for Microsoft 365.
The purpose of the DNS server tests from your Enterprise Agents is to monitor your organization’s DNS resolvers.
Your organization may use internal DNS resolvers or public DNS resolvers, or sometimes both. Common public DNS resolvers include Google DNS, Cloudflare DNS, Quad9, and Cisco Umbrella (formerly OpenDNS). To deploy the Microsoft 365 test template, you will need to know the IP addresses or hostnames of the DNS resolvers that your organization uses. If you do not know your organization’s DNS resolvers, you should ask your network team or DNS team, if your organization has one.
In this template, DNS server tests are only run from Enterprise Agents to accommodate use cases where none of your resolvers are public and are therefore are unreachable from Cloud Agents.
If your organization uses public DNS resolvers, you may add Cloud Agents to the DNS server tests from the Cloud & Enterprise Agents > Test Settings page after you deploy the template.
DNS Trace Test Deployment
The diagram above shows DNS trace testing from Cloud Agents. While the DNS server test described earlier monitors your DNS resolvers, the DNS trace test monitors the public DNS infrastructure and Microsoft 365 authoritative nameservers. See DNS Trace Test for information on why this test type is important.
Using the Microsoft 365 Test Template
Now that we've discussed all the tests, agents, and targets in the ThousandEyes Microsoft 365 test template, we are ready to use this template to deploy our tests.
Multiple permissions are required to view and deploy test templates. For the complete list, see the Prerequisites section of the Test Templates article.
To begin deploying a template:
Navigate to the Cloud & Enterprise Agents > Test Settings page
Click Add From Template in the drop-down menu
This will open the Deploy Template dialog at Step 1 of 3 - Select template. At the top of the Deploy Template dialog:
Type “Microsoft 365” in the Search box to filter the list of available templates.
Click the Microsoft 365 template in the list to proceed to configure the tests.
After clicking the Microsoft 365 template, the dialog moves forward to Step 2 of 3 - Configure tests and shows the template’s Global Settings. This test template requires you to:
Select the Enterprise Agents for the internal tests
Select the Cloud Agents for the external tests
Enter your Microsoft 365 tenant name
E.g, if your SharePoint URL is https://pseudoco.sharepoint.com, then your tenant name is
pseudoco. See Your Microsoft 365 Tenant above for more details.
Select a testing interval (recommended: 1 or 2 minutes, no greater than 5 minutes)
Enter your DNS resolvers, see note below
Provide a name for your test suite to easily identify which tests were deployed from this template. This name is arbitrary, but note that:
The name will be prefixed to all of the tests’ names, and long names may be harder to discern or distinguish in dashboards and test views.
A label will be created with this same name and applied to all the tests
Note on Entering DNS Resolvers
By default, the template will automatically look up the authoritative DNS servers for the Microsoft 365 targets. As described above, the DNS server tests are intended to monitor your DNS resolvers, not the authoritative nameservers (DNS trace tests are used for that instead).
To enter your DNS resolvers, first click the “x” at the right side of the DNS resolvers input field to clear the servers that were automatically identified.
Next, place your cursor in the text input field and type the IP address or hostname of your DNS resolver, and press your enter key. Repeat for each of your DNS resolvers.
Excluding Tests from the Template Deployment
Before proceeding to review your template deployment, you may want or need to exclude some of the tests in the template from the deployment. For example, if your organization uses SharePoint Online but not Exchange Online, you should exclude the Outlook tests from the template. Or, if you have not deployed any Enterprise Agents, you must exclude the "Internal" tests. To exclude a test, click the toggle switch next to the test’s name, as shown in the screenshot below.
Once you have provided the necessary details in the template’s Global Settings, you should be able to proceed to review the template deployment by clicking the Review button.
If the test is a DNS trace test, you will see a field labeled “Which DNS servers should we test?” with an empty value. Just by clicking on this individual test and viewing its configuration, the field should be populated automatically. If it is not automatically populated, you may enter any value for this field.
Once the field is no longer empty, repeat for any remaining tests which show the exclamation mark icon next to them.
After clicking the Review button, the dialog moves forward to Step 3 of 3 - Review template and displays a summary of the tests, labels, and dashboards included in the deployment. Review the summary, then click Deploy Now to deploy the monitoring suite. The dialog updates to show the template is being generated and the tests, labels, and dashboards are being built, as shown in the image below.
The deployment process may take a few minutes to complete. When it has finished, the dialog shows a success message and includes two links:
Clicking Go to Test Settings will navigate to the Cloud & Enterprise Agents > Test Settings page, filtered to show only the tests included in this deployment
Clicking Go to Dashboards will navigate to one of the dashboards created by the template
Microsoft 365 Dashboards
This section describes the two dashboards that are included in the Microsoft 365 test template. Both of these dashboards are designed with the highest-level information shown at the top, with increasing granularity in the widgets that follow.
All of the widgets in these dashboards allow you to drill down into the individual tests for complete details. Click on the widget to open the drilldown dialog, then select the tests to view, and click Open in Views.
Microsoft 365 Health Overview - By Service
The first dashboard provides a service-oriented health overview for Microsoft 365. The widgets in this dashboard are primarily grouped by service, highlighting issues that affect one or more of the individual Microsoft 365 services.
The first three rows of the service-oriented dashboard show alerts activity, web app health summary, and network health summary, aggregated for all Microsoft 365 tests in your deployment. This top third of the dashboard, shown in the image above, is intended as a default "NOC view" from which you can easily glean the health status of Microsoft 365 overall.
When Microsoft 365 is healthy, expect to see 'No Alert Activity' in the alerts widget, and green number cards in the web app health and network app health widgets. When problems arise, alerts will show in the alerts widget, and the web app health and network app health widgets will change from green to yellow to orange to red, depending on the scope and severity of the performance degradation.
The middle section of the service-oriented dashboard, shown above, begins to break down the metrics and groups them by Microsoft 365 services, such as Exchange, Office Online, and SharePoint Online. This allows you to easily see if user experience for an individual service is impacted. Additionally, each row in this section contains two columns:
On the left, the widgets are filtered to show data from Enterprise Agents, i.e, tests run from within your environment
On the right, the widgets are filtered to show data from Cloud Agents, i.e, tests run from outside your environment
This allows you to quickly compare the experience from your enterprise network versus the baseline from external vantage points.
The bottom widget of the service-oriented dashboard shows each of your Microsoft 365 tests, along with their current alert status, the most recent test measurements, and the trends of those measurements over the last 12 hours. You can click on any of the tests in this list to open the test view.
Microsoft 365 Health Overview - By Location
The second dashboard provides a location-oriented health overview for Microsoft 365. This dashboard is designed with the highest-level information shown at the top, with increasing granularity in the widgets that follow. The widgets in this dashboard are primarily grouped by agent, highlighting issues that affect one or more of locations within your enterprise network.
The first widget in the location-oriented dashboard, shown above, show Enterprise Agent status so you can quickly verify all of your Enterprise Agents are online.
The middle section of the location-oriented dashboard, shown above, displays summary web application performance and DNS performance grouped by agent. This allows you to quickly identify locations with poor user Microsoft 365 experience relative to the rest of your enterprise network. This section shows both mean and 90th percentile measurements to provide an idea of the distribution of app performance. Note that the map widgets show each agents' metrics aggregated for all Microsoft 365 services, while the color grid widgets show each agents' metrics for each Microsoft 365 service.
The bottom section of the location-oriented dashboard shows more granular web application and DNS performance, grouped by agent. The box and whiskers widgets provide more information about the distribution of test measurements. The Tests list widget shows each of your Microsoft 365 tests, along with their current alert status, the most recent test measurements, and the trends of those measurements over the last 12 hours. You can click on any of the tests in this list to open the test view.
Monitoring Microsoft 365 with Endpoint Agents
In addition to the Cloud and Enterprise Agent test template described above, you can use ThousandEyes Endpoint Agents for real user monitoring of Microsoft 365 combined with scheduled synthetic testing for proactive monitoring.
Synthetic Tests
Synthetic tests are a combination of scheduled and dynamic tests designed for end-user monitoring to gain visibility into the user experience.
The Endpoint Agents run tests at set times without needing user input. This helps with troubleshooting. The dynamic test allows the Endpoint Agent to monitor and identify the changing network connections between a user's Microsoft 365 application and the destination node (host server). This gives detailed network information to help find and fix issues. The tests are based on the destination used for the Microsoft 365 application and are stopped when the session ends.
Configure Synthetic Tests
You must have the Edit endpoint tests permission to configure synthetic tests.
ThousandEyes provides a pre-defined (but customizable) synthetic test template for Microsoft 365 comprising the required tests configured with the options and values making it efficient and easier to deploy.
To configure the synthetic test:
Navigate to the Endpoint Agents > Test Settings > Synthetic Tests tab.
Click the Monitor Application button and select the Microsoft 365 template.
If required, edit the information for both tests under the Global Settings section. To learn more about configuring synthetic tests, see the Configuration Options for Synthetic Tests article.
Ensure the Tenant field is filled before editing the information related to the tests.
Click Review to review the test configuration.
After reviewing the test configuration, click Deploy Now to monitor Microsoft 365.
Real User Tests
Configure real user monitoring for Microsoft 365 by including the most important domains in the monitored domain set for your Endpoint Agents. Real user tests provide you with detailed session information when users navigate their web browser to the associated monitored domain including page load elements, detailed endpoint information, and network statistics. ThousandEyes recommends including the following Microsoft 365 Monitored Domain Set in your Endpoint Agent browser session configuration:
Microsoft 365 Monitored Domain Set:
office.com
office365.com
microsoft365.com
microsoftonline.com
sharepoint.com
office.live.com
office.net
msidentity.com
The Edit endpoint agent monitored domain sets permission is required to configure real user tests.
Configure the real user test Monitored Domain Set as described in the steps below:
Navigate to the Endpoint Agents > Test Settings > Real User Tests tab.
Click the Add New Monitored Domain Set button.
Configure the basic settings for the session.
Add the domains above to the Monitored Domains input field. (Note: You must enter each domain one at a time, you cannot copy and paste the full list)
Agents select All agents, Specific agents or use Agent labels (ThousandEyes recommends selecting All agents unless you have a specific use case)
Click Add New Monitored Domain Set to save so that the selected agents will monitor the domains
The browser plugin must be installed and active for the ThousandEyes Endpoint Agent to capture browser statistics.
To learn more about real user tests, see the Real User Tests article.
Monitoring Microsoft 365 with Internet Insights
When a critical service is disrupted, it's common to wonder if you're the only one affected by the outage or if the issue is larger in scope or scale. Internet Insights collects data from a diverse set of vantage points across the globe to offer visibility into service providers, including Azure and Microsoft 365. Internet Insights is built upon ThousandEyes’ collective data set -- billions of probes across the Internet to websites, apps, and API endpoints every day -- combined with outage detection to provide a macro-scale view into network and application outages. The intelligence derived from this data enables operations teams to quickly identify and resolve issues with providers using concrete Internet telemetry data.
ThousandEyes recommends selecting the following packages in your Internet Insights Catalog Settings configuration to enable you to understand if these services are affected by disruptions.
Package Selection
SaaS package in each region from which your users work (for example, “North America SaaS Providers”), which includes:
Microsoft
Microsoft Online
Microsoft 365
Microsoft SharePoint
IaaS package in each region from which your users work (for example, “North America IaaS Providers”), which includes:
Azure
Internet Insights packages are a pre-set collection of service providers categorized by geographic region and provider type. Here, we recommend the selection of the “SaaS” packages because they identify application outages with Microsoft 365 and its common shared services.
We also recommend the “IaaS” packages, because they identify application and network outages within Azure, upon which some Microsoft 365 services are built and hosted.
If possible, both should be activated. However, if your licensed capacity limits you to choosing only one, we recommend the SaaS packages over the IaaS packages because they are more specific to Microsoft 365.
To activate a package for Internet Insights when you have available licenses for it:
Go to Internet Insights > Catalog Settings screen and click the Packages tab.
Verify that the Available counter shows one or more licenses.
Find the row with the package that you want to add.
In the Included column, click the Active slider to add the package.
To activate a package when you have no available licenses, you can purchase an additional license by contacting your customer success manager. Or, you can first deactivate a package, then activate the desired package in its place. To deactivate an Internet Insights package:
Go to Internet Insights > Catalog Settings screen and click the Packages tab.
Find the row with the package that you want to remove.
In the Included column, click the Active slider to remove the package.
To learn more about Internet Insights, see the Getting Started with Internet Insights article and the Internet Insights section of the documentation.
Last updated
