Team members (from top left to right):
Emeralda Sesari, Khesim Reid, Arzhan Tong, Xinyuan Zhuang, Omar Beyhum, Lydia Tsami, Yanke Zhang, Chon Ng, Elena Aleksieva, Zisen Lin, Yang Fan, Yifei Zhao
Supported by:
Lee Stott, Microsoft
Dean Mohamedally, University College London
Giovani Guizzo, University College London
The Team
We are a group of 12 students at University College London (UCL). UCL's Industry Exchange Network (IXN) programme enabled us to work on this project in collaboration with Microsoft as part of a course within our degree.
Introduction
Attendance and engagement monitoring is an important but exceptionally challenging process universities and colleges; especially during the COVID-19 pandemic in 2020, students attend both online synchronous and asynchronous courses around the world. Predictably, we need a sensible course monitoring system to facilitate teaching staff and learning administrators who set the curriculum to better manage students in their courses, not just during the pandemic. ORCA is a system that was created for this purpose.
ORCA is designed to complement the online learning and collaboration tools of schools and universities, most notably Moodle and Microsoft Teams. In brief, it can generate visual reports based on student attendance and engagement metrics, and then provide them to the relevant teaching staff.
Since ORCA is designed to integrate with both MS Teams and Moodle, our entire development team (12 students in total) is grouped into two small groups (the MS Teams group and the Moodle group). The two groups set up the main functionalities of ORCA together and implemented the integrations with the respective platforms separately.
In short, ORCA is aimed to:
- assess the effectiveness of online learning by gathering data on students’ use of online learning including students’ attendance and engagement from both MS Teams and Moodle;
- keep track of the status of students that may impact their education;
- help teaching staff and learning administrators get actionable insights from data to improve teaching methods.
Design
ORCA allows institutions to gain insights about students' attendance and engagement with online platforms. To accomplish this, the ORCA system mainly integrates with Moodle and Microsoft Graph
Figure 1: ORCA's design
These two external systems send out notifications to ORCA whenever students interact with Moodle and Microsoft Teams respectively. ORCA then standardizes the incoming information into a common format and stores it on Sharepoint lists which the relevant teaching and learning administration staff can access to check attendance.
ORCA also gives the option to store student engagement data in a database which can be exposed to the Power BI dashboarding service and embedded into Sharepoint, allowing staff to visualize how students interact with online content.
Figure 2: Student engagement visualized through Power BI
Implementation
Figure 3: ORCA's architecture
ORCA is a .NET 5 application implemented in C#. Figure 3 shows the main architecture of ORCA. The core of the system, the event aggregator, is where all the information is collected from different sources. The event aggregator interfaces with pluggable API adapters, which allow data (i.e., student events) from a different system to be standardized so that the event aggregator can persist them for future analysis.
So far, there are 2 main API adapters, the MS Graph Adapter which is notified by Microsoft’s Graph API whenever students join lectures, and the Moodle Adapter, which receives information about student activity through a plugin that can be installed on an institution’s Moodle server. The Moodle Adapter also follows the IMS Caliper standard, which allows it to integrate with any Learning Management System implementing the standard, such as Blackboard Learn.
ORCA can be automatically deployed to the Azure cloud or configured to run on a custom (on-premises) Windows, Linux, or macOS machine of an IT Administrator's choosing. Powershell scripts and precompiled binaries are provided in our Github Releases page to automate configuration and deployment, although ORCA's centralized configuration can also be manually changed.
ORCA makes use of Azure Resource Manager (ARM) Templates to define the resources it requires to run on the Azure cloud. By packaging a script which dynamically generates an ARM template based on the IT Administrator's configuration choices, we can automatically provision and configure a webserver and optional database to run ORCA on the cloud within minutes. If a database was selected, the script also generates Power BI dashboards which automatically connect to the provisioned database.
Future Work
While the ORCA project is a functional software product that can be deployed to an institution’s infrastructure, we have yet to run it against real student data during the development stage due to privacy concerns. To comply with privacy regulations, current testing is based on synthetically generated data. We are in the process of piloting ORCA with UCL’s Information Services Division to better assess how the system meets institutions’ needs in a more productionized environment.
We also hope to increase ORCA’s level of customization and allow administrators to configure which activities count towards engagement, and at what point they should be automatically alerted about student inactivity.
ORCA is an open source project which any institution can adopt, contribute to, or simply inspire themselves from to build their own solution.
You can check it out at https://github.com/UCL-ORCA/Orca
Getting started with Microsoft Graph on Microsoft Learn