Open Education Analytics

  • Article

Overview

Open Education Analytics (OEA) is an open source program coordinated by Microsoft Education. We aim to empower education systems around the world to use data, analytics and AI more effectively and ethically. OEA improves learning outcomes by developing shared insights and best practices on the use of advanced analytics. OEA supports open source sharing of education data and analytics tools, and skills and talent development for the education ecosystem.

OEA is a fully open-sourced (Creative Commons and MIT) data integration and analytics framework for the education sector. It's built on Azure Synapse - with Azure Data Lake Storage as the storage backbone, Microsoft Entra ID providing role-based access control, and Azure Purview for data discovery and governance.

OEA technical assets include open source data modules for common education datasets. You can connect your datasets quickly, combine them with other data, and move to data visualization and insights quickly. An example of an OEA module is the Microsoft Insights Premium module. You can use OEA packages to develop models that solve challenges like predicting at risk students.

The GitHub repository for OEA technical infrastructure, modules and packages is located here. It contains a set of assets for setting up and walking through a reference implementation of the OEA architecture.

collect, compute, communicate.

discover and govern azure purview

The underlying Azure platform services are mature and well documented. However, the OEA technical assets are a work in progress and comes with no warranties nor SLAs. Each organization implementing these assets is responsible for adhering to their own data governance framework and ensuring data security and privacy. General guidance is provided through the training resources listed below. The OEA Github repo should be considered as a starting point for the development of your modern education data estate. Once you have your starting point, you should iterate and optimize as you evolve your design and build out your solution. We look forward to growing this set of assets with our customers and partners.

Prerequisites

To set up an environment with OpenEduAnalytics, you'll need:

  • an Azure subscription. If you don't have an Azure subscription, you can set up a free subscription here, or check the current list of Azure offers).

  • role assignment of "Owner" on the Azure subscription you're using

  • your preferred subscription is selected as default

    az account list --query "[].{SubscriptionId:id,IsDefault:isDefault,

    Name:name,TenantId:tenantId}"

    az account set --subscription <SubscriptionId>

Set up

You can set up this functional reference architecture (which includes test data sets for basic examples of usage) in three steps:

  1. Open cloud shell in your Azure subscription (use ctrl+click on the button below to open in a new page) open cloud shell

  2. Download the repo to your Azure Cloud drive.

    cd clouddrive git clone https://github.com/microsoft/OpenEduAnalytics

  3. Run the set up script by substituting "mysuffix" with your preferred suffix. This suffix must be fewer than 13 characters and can only contain letters and numbers. It'll be used as a suffix in the naming of provisioned resources:

    ./OpenEduAnalytics/setup.sh mysuffix

You can refer to this set up video for a walk-through.

More set up options

By default, the set up script provisions Azure resources in the East US region. You can choose other locations as well, for example, "WestUs" or "NorthEurope".

./OpenEduAnalytics/setup.sh mysuffix northeurope

For a list of available locations, you can use the command

az account list-locations

You can choose to have the script create security groups to facilitate the use of "role-based access control" to the data lake.

If you're running the set up for an environment in which you have Global Admin permissions on the tenant, and you want to have security groups provisioned, you can invoke the set up script like this:

./OpenEduAnalytics/setup.sh mysuffix eastus true

By default, the provisioned Azure resources are named according to recommended Azure naming standards. You can directly modify set_names.sh before running the set up if you want to specify an alternative set of resource names.

Provisioned Azure resources are tagged with oea_version. If an Azure policy requires specific tag(s) to be assigned when a resource is created, these can be included via $OEA_ADDITIONAL_TAGS in set_names.sh using the format tagName=tagValue. If multiple tags are required, they need to be space-separated.

More info

You can get more info by reviewing the documentation available in the docs folder. A PowerPoint deck with notes and a PDF version can be viewed in the browser:

For details on the installation and usage of the OEA base architecture and test environment, see Open Edu Analytics Solution Guide.

For an introduction to Azure Synapse Analytics, see Cloud Analytics with Microsoft Azure, which is a free e-book published in January 2021. It is also available in other formats for purchase).

Enabling Sync to Insights Premium

Enabling Insights Premium Export

[]