Scheduling BizTalk processing to run at predetermined time intervals has been a discussion topic in user forums. As was recently announced in Advanced Scheduling in Receive Locations, BizTalk 2020 expanded on the existing built-in functionality at the receive location level.
This article presents by a detailed example an integration pattern that leverages the calendar functionality of Office 365 to schedule workload processing in orchestrations. In the demo scenario, The Office 365 Outlook Calendar adapter is used to communicate with an Office 365 calendar which then acts as a "remote control" of when and how long an orchestration should be running.
The advantages of this pattern are:
- Different schedules can be accommodated without changes to the configuration of receive locations as is required in the receive location scheduling.
- No need for custom adapters or "trigger messages" such as in the BizTalk Scheduled Task Adapter.
- Full-fledged support for every scheduling pattern that Office 365 can offer.
- Schedule is conveyed by a calendar event message, which gives the ability to transmit additional information to the orchestration and customize business processes. A calendar event is more than just a start/end datetimes.
- Concurrent processing of events with different schedules, even when the schedules overlap.
Business Scenario
A school teacher uses BizTalk to conduct timed assignments remotely: an email is sent to the students at start time, with the assignment material attached. Students submit their answers by email within the allotted time and receive a response with their grades. At the end of the assignment time, everyone gets notified by email. A grade report is saved for the teacher.
Assignments are scheduled by creating calendar events in advance in an Office 365 school calendar that only the teacher or the school has access to. For the sake of simplicity in the demo, we assume that the calendar events contain the assignment id in the subject, and the assignment answer in the body. This is illustrated below:
Assignment sheets are placed by the teacher in a location accessible by BizTalk, for example in a local folder. When the assignment is about to start, students receive the following email with the assignment material attached (123.pdf):
Students can then email their answer and receive their grade "on the fly".
| Student sends in plain-text | Student receives |
|
<ns0:Assignment xmlns:ns0="http://OrchestrationScheduling.Assignment"> <AssignmentId>123</AssignmentId> <Answer>ABC</Answer> </ns0:Assignment> |
<ns0:AssignmentGraded xmlns:ns0="http://OrchestrationScheduling.AssignmentGraded"> <AssignmentId>123</AssignmentId> <Pass>true</Pass> </ns0:AssignmentGraded> |
At the end of the assignment period, an email is sent to all students to notify them that submissions are closed. Grades are saved for the teacher in a file that contains the following:
|
<ns0:QuizSummary xmlns:ns0="http://OrchestrationScheduling.QuizSummary"> <AssignmentId>123</AssignmentId> <Summary> Student1 True Student2 True … </Summary> </ns0:QuizSummary> |
Scheduled Orchestration
The BizTalk orchestration consists of three parts:
1. Activation and assignment extraction (schedule and answer).
2. Receive assignment/send grade loop.
3. Post-assignment tasks.
The full orchestration view is shown below. The work being scheduled (ReceiveAssignments/SendGradeToStudent) is encapsulated in a listen shape, but decoupled from the scheduling logic. In the general case, a more complicated business application could be done in one or more separate orchestrations to be called from the calendar-event-activated orchestration.
The schema of the initializing receive shape CalendarEvent is a copy of Office365OutlookCalendarReceive.xsd, which is available in the BizTalk installation folder (C:\Program Files (x86)\Microsoft BizTalk Server\SDK\Schemas). The subject field corresponds to the assignment id, which was used as correlation in order to have the same orchestration instance process all sends and receives for the same assignment. As in the correlations walkthrough, we used a property schema with a common id for the assignment id field of all send and receive shapes. Schemas are shown below.
The orchestration design shows a sequential convoy with different ports for receiving. By having send shapes interleaved with receive shapes, the design does not have the compile time error “in a sequential convoy the ports must be identical”.
Scheduled assignments can be processed concurrently. There is one orchestration instance per scheduled event, which can therefore overlap with others. This extra flexibility to accommodate multiple schedules at once is not readily available with other scheduling alternatives such as receive location scheduling.
Part 1: Orchestration Activation
The logical flow, illustrated in the next picture, can be divided into 6 chronological steps:
- Receive a calendar event, with the subject containing the assignment id, used as orchestration correlation. The receive location is to be configured to listen to a specific non-default calendar. More information on the calendar receive location properties is available at Filtering in Office 365 Outlook Adapters.
- Extract the assignment answer from the body of the event message. The body preview field was used as distinguished field instead of the entire body field, since all we need is the first line of the body, not the entire body with html markup.
- Transform the event message to a message which contains local start and end datetimes. Note that we used a script functoid to map the received start/end event datetimes which were in UTC to local time. In the general case, start and end time zones are available if needed.
- Use a delay shape to wait for the assignment start local datetime extracted in the previous step, before proceeding with the assignment.
- Construct the email to be sent at the start of the assignment period for students notifications and provide the assignment material. In our case, the email also contains an attachment, the assignment material. Email attachments in the mail adapter are presented in greater detail in Attachments in the Office 365 Outlook Email Adapter.
- Send the email using the Office 365 Outlook Mail adapter. In the send port configuration, the default "To:" field was set to the class mail alias. Office 365 Outlook Adapters in Action is a good starting point for background and general information on the various Office 365 Outlook adapters used together.
Part 2: Scheduled Workload
At the heart of the design, the scheduled workload corresponds to a branch of a listen shape with a timeout branch set to the scheduled end date. A loop shape ensures that the orchestration processes all assignments received from students until the end datetime.
The logical flow of each loop iteration can be divided into 5 steps:
1. Receive assignments solutions from students, sent in plain-text email as mentioned earlier. The receive location is configured with Office 365 Outlook Email as transport type and the default XML receive pipeline.
2. Construct the assignment grade email. The "To:" field corresponds to the SenderAddress field of the received assignment.
3. Send the email constructed in step 2. The send port is configured with Office 365 Outlook Email transport type.
4. Record the student's grade by using a GradesInfo helper class defined in an external assembly. The code is presented at the end of this article.
5. Check if the assignment schedule has ended.
The 5 step are indicated in the following picture:
If a student sends her assignment before the start datetime, there are two variations to the scenario:
- If there is no listener for the mail receive port ReceiveAssignments, the student's answer will be suspended until an orchestration instance starts and reaches the start datetime. A teacher could choose this option to accommodate early submissions.
- If there is a listener for the ReceiveAssignments port, for instance a send port with file transport type filtering on ReceiveAssignments, the student's assignment will be already processed (published) by the time the orchestration instance reaches start datetime.
If a student sends her email in html, or without following the relevant XML schema, the email will be suspended - as expected - with for instance:
There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, …" Source: "XML disassembler" Receive Port: "MailReceivePort" URI: "O365Mail://…@outlook.com/Inbox" Reason: Finding the document specification by message type "html" failed. Verify the schema deployed properly.
Part 3: Post-Assignment Tasks
After the assignment deadline, the following tasks happen in parallel:
1. An email is sent again to everyone in the class;
2. A summary is saved locally for instance with a file send port, for the teacher to use at a later time. The GradesInfo helper class provides the grades report.
Helper Class
A serializable helper class was added to the project for saving students' grades. Note that protected constructor and GetObjectData method with sufficient privileges need to be provided for the class to be serializable.
using System;
using System.Collections.Generic;
using System.Runtime.Serialization;
using System.Security.Permissions;
namespace StudentsGrades
{
[Serializable]
public class GradesInfo : Dictionary<string, bool>
{
public GradesInfo() { }
public void AddOrUpdate(string key, bool value)
{
if (!this.ContainsKey(key))
{
this.Add(key, value);
}
else
{
this[key] = value;
}
}
public string GetPrintableSummary()
{
string result = Environment.NewLine;
foreach (var student in this.Keys)
{
result += string.Join(" ", student, this[student], Environment.NewLine);
}
return result;
}
protected GradesInfo(SerializationInfo info, StreamingContext context) : base(info, context) {}
[SecurityPermissionAttribute(SecurityAction.Demand, SerializationFormatter = true)]
public override void GetObjectData(SerializationInfo info, StreamingContext context)
{
base.GetObjectData(info, context);
}
}
}
Concluding Remarks
The objective of the demo scenario is to showcase a reusable way to implement scheduling in orchestrations. More error handling would normally be part of a "real-life" application. For instance, emails sent to the teacher need to be filtered so that only the ones with assignment answers are processed, and the other emails ignored. One way to accomplish that is to use server-side rules to direct emails to a specific folder, and have a receive location poll the folder.
The scheduled events should also come from a non-default calendar rather than the default calendar since we don't want any meeting request sent to the teacher's account to initiate new orchestration instances. More information on filtering in the Office 365 Outlook adapters may be found in the related post Filtering in Office 365 Outlook Adapters.
Attachments to calendar events are currently ignored by the Office 365 Outlook Calendar adapter but they could be supported in future BizTalk releases if there is demand for the feature. In the demo scenario, the teacher would be able to attach assignment material directly to the scheduled event and let the orchestration send it out.
Sample Code
All code used in this article is attached.
When building the BizTalk solution, the following warnings can be safely ignored:
...\OrchestrationScheduling\OrchestrationScheduling\ScheduledOrchestration.odx(678,35): warning X4014: convoy processing will not occur -- check your protocol if you were expecting it
...\OrchestrationScheduling\OrchestrationScheduling\ScheduledOrchestration.odx(647,30): convoy found at 'activate receive(ReceiveScheduledQuiz.Operation_1, TriggerAssignmentTask, initialize AssignmentId)'
...\OrchestrationScheduling\OrchestrationScheduling\ScheduledOrchestration.odx(678,35): and 'receive(ReceiveAssignments.Operation_1, AssignmentSubmitted, AssignmentId)'
References
Office 365 Outlook Calendar adapter
Office 365 Outlook Email adapter
Advanced Scheduling in Receive Locations
Attachments in the Office 365 Outlook Email Adapter
Filtering in Office 365 Outlook Adapters
Office 365 Outlook Adapters in Action
Correlations in BizTalk Orchestrations