Author : Soujanya Akella
The main purpose of this document is to outline the testing strategy used for Performance Testing of BOT in one of the customer engagements.
The Application Under Test is a Mobile App that communicates with BOT which consumes services to call the Knowledge Graph APIs. Load testing was performed at the BOT layer.
Following are the detailed steps involved:
- Implementation of the mock channel to which BOT response would be directed to.
- When a request is sent to BOT, response of BOT is sent back to BOT Connector Service by default. In order to capture the timestamp for BOT response, we create a mock channel to which response is redirected.
- Within this mock channel, methods for testing BOT are defined.
- When BOT’s endpoint receives a request ( message) from user, using the information in that request an Activity object is created for response to user request.
- To reply to a specific message within the conversation, Reply to Activity method is used that is implemented in the mock channel.
- Within this method, BOT Response is captured and saved to Storage Table that can be retrieved later. The time stamp of response received is also captured in the Storage table for that activity.
For details to implement the mock channel, please refer to published IP.
2. Send Request from Visual Studio Web Test to BOT.
Request1: POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
- Obtain an access token from the Bot Framework to authorize the web test to the BOT.
In the below request, replace MICROSOFT-APP-ID and MICROSOFT-APP-PASSWORD with the App ID and Password of the BOT (obtained during BOT Registration).
POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
Host: login.microsoftonline.com
Content-Type: application/x-www-form-urlencoded
Request Body:
grant_type=client_credentials&client_id=MICROSOFT-APP-ID&client_secret=MICROSOFT-APP-PASSWORD&scope=MICROSOFT-APP-ID/.default
Successful request returns a JSON payload with access_token.
Request2: POST https://{{BotEndPoint}}/api/messages
- Use the Bot Connector service to exchange messages between BOT and user:
Create HTTP POST to BOT’s endpoint URL that includes Authorization header where the access_token retrieved in above step is passed.
Requests sent to BOT Endpoint https://{{BotEndPoint}}/api/messages include:
- Authorization token retrieved earlier is passed in the header
POST api/messages
Authorization: Bearer
Content-Type: application/json
- JSON Serialized body contains the ServiceUrl which is the mock channel end point to which BOT sends the response back.
{
"Type": "message",
"Id": "{{ActivityID}}"
"ChannelId": "test",
"Conversation": {
"id": "{{ConversationID}}"
},
"From": {
"id": "1"
},
"Recipient": {
"id": "2"
},
"ServiceUrl": "https://{{mockchannel}}.azurewebsites.net",
"Text": “{Message Sent}"
3. Save the above request body along with request timestamp to the same Storage table account created in Step 1. with ConvID(guid) and ActivityID(guid) as the PartitionKey and RowKey.
- The time stamp of request sent from user is also captured in the Storage table for that activity. This is shown in the code shared below.
- The ServiceUrl property within the user's message indicates that the BOT should send its response to the endpoint https://{{mockchannel}}.azurewebsites.net that will be the base URI for any subsequent requests that the bot issues in the context of this conversation.
- Context parameter e.WebTest.Context["BotConnectorBaseUrl"].ToString() points to https://{{mockchannel}}.azurewebsites.net
Code for design of Storage Table, generation of request body for api/messages and save the request to Storage Table.
- Design a Storage Table with columns mapping the request details such as ConversationID, ActivityID, requestbody, request timestamp etc..
public class ActivityTableEntity : TableEntity
{
public ActivityTableEntity()
{
}
public ActivityTableEntity(string convId)
{
this.PartitionKey = convId;
}
public ActivityTableEntity(string convId, string activityId)
{
this.PartitionKey = convId;
this.RowKey = activityId;
}
public string activityMsg;
public string scenarioName { get; set; }
public string incomingAcvitityMsg { get; set; }
public string botResponseMsg { get; set; }
public DateTime requestTimeStamp { get; set; }
public string responseTimeStamp { get; set; }
}
- Generate request body for api/messages and save the request to Storage Table.
public class SaveRequest : WebTestPlugin
{
private static CloudStorageAccount account = default(CloudStorageAccount);
private static CloudTableClient tableClient;
private static CloudTable botOperationLogTable;
public string azureStorageAccount = HelperMethods.GetAppConfigValues("azureStorageAccount");
public string azureStorageSecret = HelperMethods.GetAppConfigValues("azureStorageSecret");
public string TableName = HelperMethods.GetAppConfigValues("tableName");
ActivityTableEntity activityTableEntity;
private string postData = string.Empty;
private string Text = string.Empty;
private string AuthToken = string.Empty;
private Guid guid = Guid.NewGuid();
private string Id = string.Empty;
public override void PreWebTest(object sender, PreWebTestEventArgs e)
{
if (!string.IsNullOrEmpty(azureStorageAccount) && !string.IsNullOrEmpty(azureStorageSecret))
{
account = new CloudStorageAccount(new StorageCredentials(azureStorageAccount, azureStorageSecret), true);
tableClient = account.CreateCloudTableClient();
botOperationLogTable = tableClient.GetTableReference(TableName);
botOperationLogTable.CreateIfNotExistsAsync();
}
}
/* ToDo Include a check for the request to match with api/messages in order to create the request body only for that request */
public override void PreRequestDataBinding(object sender, PreRequestDataBindingEventArgs e)
{
// Generation of request body
Id = e.WebTest.Context["ActivityID"].ToString();
postData =
@"{
""Type"":""message"",
""Id"":""" + Id +
@""",
""ChannelId"":""test"",
""Conversation"":{
""id"":""" + e.WebTest.Context["ConvID"].ToString() +
@"""
},
""From"":{
""id"":""1""
},
""Recipient"":{
""id"":""2""
},
""ServiceUrl"":""" + e.WebTest.Context["BotConnectorBaseUrl"].ToString() + @""",
""Text"" : ""{\""Query\"":\""" + @"\""}""
}";
e.WebTest.Context.Add("postData", postData);
activityTableEntity = new ActivityTableEntity(e.WebTest.Context["ConvID"].ToString(),Id);
activityTableEntity.incomingAcvitityMsg = postData;
activityTableEntity.requestTimeStamp = DateTime.UtcNow;
activityTableEntity.scenarioName = "WebTest1";
}
/* ToDo Include a check for request to match with api/messages in order to save only for that request */
public override void PostRequest(object sender, PostRequestEventArgs e)
{
// Save the request details to table storage
TableOperation tableOperation = TableOperation.Insert(activityTableEntity);
botOperationLogTable.Execute(tableOperation);
}
}
4. Calculation of response timestamp: Since both request timestamp -requestTimeStamp (from web test) and response timestamp-responseTimeStamp(from mock channel ) are captured in Storage Table, the difference in columns gives the response times.
