How to write an Exchange 2013 transport agent

[The_Exchange_Team]

Update 2/5/2013: We have also uploaded a sample that will work on Exchange 2010 servers, you can find it here.

What is a Transport Agent?

Transport agents allow Microsoft, developers in your organization and third-party vendors to hook into the Exchange transport pipeline with their code to process messages (e.g. an antivirus scanner for incoming email messages). Transport agents can process email messages that pass through the transport pipeline in many ways. An agent is a .Net assembly that has to be installed on the Exchange Client Access or Mailbox server. The agent is then loaded by the Exchange Transport service and invoked in the transport pipeline on the specified event. In Microsoft Exchange Server 2013, the transport pipeline is made of three different processes:

• Front End Transport service:   This service runs on all Client Access servers and acts as a stateless SMTP proxy to route messages to and from the Transport service on a Mailbox server.
• Transport service:   This service runs on all Mailbox servers and is virtually identical to the Hub Transport server role in previous versions of Exchange. Unlike previous versions of Exchange, the Transport service never communicates directly with the mailbox store. That task is now handled by the Mailbox Transport service. The Transport service routes messages between the Mailbox Transport service, the Transport service, and the Front End Transport service.
• Mailbox Transport service:   This service runs on all Mailbox servers and consists of two separate services: Mailbox Transport Submission and Mailbox Transport Delivery. Mailbox Transport Delivery receives SMTP messages from the Transport service, and connects to the mailbox database using an Exchange remote procedure call (RPC) to deliver the message. Mailbox Transport Submission connects to the mailbox database using RPC to retrieve messages, and submits the messages over SMTP to the Transport service.

Like the previous version of Exchange, Exchange 2013 transport provides extensibility that is based on the Microsoft .NET Framework version 4.0 and allows third parties to implement the following predefined classes:

• SmtpReceiveAgent
• RoutingAgent
• DeliveryAgent

Note: This Article will concentrate mainly on how to implement and build a SmtpReceiveAgent. The SmtpReceiveAgentFactory and SmtpReceiveAgent classes provide support for the extension of the Microsoft Exchange Server 2013 Edge Transport behavior. You can use these classes to implement transport agents that are designed to respond to messages coming into and going out of the organization.

The following list explains the requirements for using transport agents in Exchange 2013.

• The Transport service fully supports all the predefined classes in Exchange 2010, which means that any transport agents written for Hub Transport server role in Exchange 2010 should work in the Transport service in Exchange 2013.
• The Front End Transport service only supports the SmtpReceiveAgent class that was available in Exchange 2010, and third party agents can't operate on the OnEndOfData SMTP event.
• You can't use any third party agents in the Mailbox Transport service.

Exchange 2013 updates to Transport Agent Management

Due to the updates to the transport pipeline, the transport agent cmdlets need to distinguish between the Hub Transport service and the Front End Transport service, especially if Client Access server and Mailbox server are installed on the same physical server. All transport agent cmdlets now have the TransportService parameter. The values you can specify are Hub for the Hub Transport service and FrontEnd for the Front End Transport service. For example, to view the manageable transport agents in the Hub Transport service, run the command: Get-TransportAgent -TransportService Hub. To view the manageable transport agents in the Front End Transport service, run the command: Get-TransportAgent -TransportService FrontEnd. The TransportService parameter is available in all transport agent cmdlets:

• Disable-TransportAgent
• Enable-TransportAgent
• Get-TransportAgent
• Install-TransportAgent
• Set-TransportAgent
• Uninstall-TransportAgent

StripIncomingLinkReceiveAgent

This Transport Agent was designed and implemented to illustrate various Exchange 2013 transport agent functionality as well as stripping all the hyperlinks from the message body. This agent will illustrate the following functionality:

1. Setting up the Visual Studio Environment to code and build the agent
2. Adding Agent Event Handlers such as EndOfData and EndOfHeader to strip all the Hyperlink from the incoming message body
3. Enumerate through all the message body part and select the text body part and then stripping the link.
4. Removing any existing attachment from the message
5. Skipping Health messages sent by the Exchange
6. Process MimePart of the body
7. Error Handling
8. Implementing test unit to test various part of agent before installing on Exchange 2013.

Setting up the Environment

Visual Studio can be utilized to build and implement a transport agent. This lesson will show you how to build a transport agent to remove all HTML links from an incoming SMTP messages as well as illustrating on how to use the agent in the Exchange 2013 Front End Transport Service which was introduced in Exchange 2013.

1. In Visual Studio 2012, Create a new project using Templates->Visual C#->Windows->Class Library and name the project StripIncomingLinkAgent

2. Create a folder name Exchange under \StripIncomingLinkAgent\StripIncomingLinkAgent and copy the following DLLs from C:\Program Files\Microsoft\Exchange Server\Public

Microsoft.Exchange.Data.Common.dll
Microsoft.Exchange.Data.Transport.dll

3. In Visual Studio 2012, Go to Solution Explorer, Right click the References and select “Add Reference…”

4. Select Browse from the Reference Manager dialog and navigate to \StripIncomingLinkAgent\StripIncomingLinkAgent\Exchange

5. Select both dll and click add and OK

6. These two DLL will be used to integrate the agent to MSExchangeFrontEndTransport and MSExchangeTransport process.

7. First step is to rename the Class1.cs generated by Visual Studio to a meaningful name (e.g. StripIncomingLinkReceiveAgent)

8. Next Step is to create a ReceiveAgent class (naming the class StrinIncomingLinkReceiveAgent) which will inherit from SmtpReceiveAgent and add the proper references:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Diagnostics;
using System.IO;
using Microsoft.Exchange.Data.Transport;
using Microsoft.Exchange.Data.Transport.Smtp;
using Microsoft.Exchange.Data.Transport.Routing;
using Microsoft.Exchange.Data.Transport.Delivery;
using Microsoft.Exchange.Data.Mime;
using Microsoft.Exchange.Data.Transport.Email;
using StripLink;
using StripLink.Utilities;
using StripIncomingLinkAgent.Configuration; 

9. Before adding the business logic to our agent code and setup the agent call back, we need to setup a logging method to log events to the application event log:

public static void WriteLog(string message, EventLogEntryType entryType,
                                int eventID, string proccessName)
{
    try
    {
        EventLog evtLog = new EventLog();
        evtLog.Log = s_EventLogName;
        evtLog.Source = proccessName;
        if (!EventLog.SourceExists(evtLog.Source))
        {
            EventLog.CreateEventSource(evtLog.Source, evtLog.Log);
        }
        evtLog.WriteEntry(message, entryType, eventID);
    }
    catch (ArgumentException)
    {
    }
    catch (InvalidOperationException)
    {
    }
}

10. Next step is to register our call back with the Transport EventHandlers and create our agent:

public sealed class StripLinkReceiveAgentFactory : SmtpReceiveAgentFactory
{
    public override SmtpReceiveAgent CreateAgent(SmtpServer server)
    {
        IConfigurationProvider config;
        config = new StaticConfigProvider
        {
            ForceSinglePart = true,
            ForceTextPlain = true,
            FilterAuthenticated = true,
            FilterTNEF = true,
            HonorAntiSpamBypass = false,
            SkipInternalMessages = false,
            AlwaysFilterCAFETraffic = false
        };
        // To use the RegistryConfigProvider class - uncomment the line 
        // below and comment above staticConfigProvider
        // config = new RegistryConfigProvider()
        return new StripIncomingLinkReceiveAgent(config);
    }
}
public class StripIncomingLinkReceiveAgent : SmtpReceiveAgent
{
    private static string processName = "ExchagneStripIncomingAgent";
    private string machineName;
    private Process currentProcess;
    private static string FrontEndTransport = "ExchangeStripIncomingLinkFrontEndAgent";
    private static RoutingAddress inboundProxy = new RoutingAddress(
 "inboundproxy@inboundproxy.com");
    private bool m_IsTNEF = false;
    private bool m_IsSummaryTNEF = false;
    /// <summary>
    /// Configuration provider. Should be set by constructor.
    /// </summary>
    private IConfigurationProvider configProvider;
    public static void WriteLog(string message, EventLogEntryType entryType, int eventID,
                               string proccessName)
    {
        StripLinkHelper.WriteLog("ReceiveAgent: " + message, entryType, eventID,
                               proccessName);
    }
    public StripIncomingLinkReceiveAgent(IConfigurationProvider config)
    {
        configProvider = config;
        this.OnEndOfData += new 
               EndOfDataEventHandler(StripIncomingLinkEndOfDataHandler);
        this.OnEndOfHeaders += new
  EndOfHeadersEventHandler(StripIncomingLinkEndOfHeadersHandler);
        currentProcess = Process.GetCurrentProcess();
        machineName = Environment.MachineName;
        if (currentProcess.ProcessName.ToLower().Contains("frontend"))
        {
            processName = FrontEndTransport;
        }
    }

11. Now we can implement the Event Handler logic for End of Header event handler. We are keeping the Header Event handler simple. The code checks for health messages generated by Exchange and skip processing of those messages and it adds a marker to the header to notify the backend agent that header has been processed by frontend agent:

private bool IsHealthMessage(MailItem mailItem)
{
    string domainName = mailItem.FromAddress.DomainPart;
    RoutingAddress adminEmailAddress = new RoutingAddress("Administrator@" + domainName);
    return (mailItem.Recipients.Contains(inboundProxy) ||
                        mailItem.FromAddress.LocalPart.Contains("HealthMailbox") ||
                        mailItem.FromAddress.LocalPart.Contains("inboundproxy"));
}
private void StripIncomingLinkEndOfHeadersHandler(ReceiveMessageEventSource source,
                        EndOfHeadersEventArgs e)
{
    try
    {
        if (IsHealthMessage(e.MailItem) &&
                       !(currentProcess.ProcessName.ToLower().Contains("frontend")))
            return;
        // Add a header indicating that this was processed 
        // by a Front End Server...
        StripLinkHelper.MarkAsProcessedByCAFE(machineName, e.Headers);
    }
    catch (Exception except)
    {
        WriteLog("EndofHeader Exception = " + except.ToString() + processName,
                        EventLogEntryType.Error, 10, processName);
    }
}

12. End of Data Event handler has all the logic to parse the message body, create a single part (based on registry key value), convert the message to plain text (based on registry key value) and finally remove all the hyperlink from the message and save the body using StreamWriter:

private void StripIncomingLinkEndOfDataHandler(ReceiveMessageEventSource source,
                        EndOfDataEventArgs e)
{
    try
    {
        EmailMessage message = e.MailItem.Message;
        Body currentBody = message.Body;
        if(ShouldSkipMessage(e.MailItem, e.SmtpSession))
        {
            // Message skipped, nothing to do.
            return;
        }
        // The following Actions are only valid for
        // pure mime messages (Not TNEF)
        if (!m_IsTNEF)
        {
            // Do we want to make it single part
            // (only Valid is no TNEF)
            // The goal here is to remove all other mime parts
            // (body types, attachments) to minimize 
            // the surface area where hyperlinks can be found.
            // There is no point in removing hyperlinks in the body
            // if there is an HTML attachment in the email. :)
            if (configProvider.ForceSinglePart)
            {
                // Once again, we try to reduce the exposure to hyperlinks.
                // We do this by trying to get the lowest fidelity
                // body, and making that the only mimepart on the message.
                // We hope for Text/Plain body but it could be HTML.
                MimePart mimepartLowFidelity =
                               StripLinkHelper.GetLowerFidelityBodyPart(currentBody);
                if (mimepartLowFidelity != null)
                {
                    // We now remove any branches that do not contain
                    // this node. Another option is to make the
                    // message single part, however, this will
                    // require merging headers of the source
                    // part with the root part.
                    // NOTE: This will break multipart/related messages 
                    // because they rely on the other parts
                    // to store the other components (images, documents, etc).
                    // This should not be an issue
                    // if the target part is always text/plain.
                    StripLinkHelper.MakeSingleBranch(mimepartLowFidelity);
                }
                else
                {
                    // TODO: Decide what to do in this scenario...
                    // Probably route to Admin...
                    // This is the case for TNEF messages as there
                    // is no MimePart associated with the body.
                    WriteLog("Failed to get a low fidelity body.” + 
 “Mime tree will not be simplified. Quarantine Message " +
                            processName, EventLogEntryType.Error, 10, processName);
                    source.Quarantine(e.MailItem.Recipients,
                            "Could not find MimePart for the body.");
                    return;
                }
            }
            // Force Plain text...
            if (configProvider.ForceTextPlain)
            {
                StripLinkHelper.ForcePlainTextIfNeeded(message);
            }
        }
        else if(configProvider.ForceSinglePart)
        {
            // We do honor the ForceSinglePart for TNEF, we
            // assume this means we don't want any attachments
            // so for TNEF we simply remove the message attachments.
            // A seperate option could be added, TNEFRemoveAttachments
            // if this needs to be handle independent of
            // the ForceSinglePart settings.
            message.Attachments.Clear();
        }
        // We now need to process the message.
        StripLinkHelper.ProcessEmailBody(message.Body);
        // TODO: For LegacyTNEF we thought we also needed to
        // filter the text/plain part that is included
        // for non-TNEF clients, but it appears modifying the body
        // also generates a new text/plain part so
        // the code below is not needed.
        //if (m_IsTNEF && !m_IsSummaryTNEF)
        //{
        // FilterTNEFTextPart(message);
        //}
        StripLinkHelper.MarkAsProcessedByFilteringAgent(message.RootPart.Headers,
                             machineName);
    }
    catch (Exception except)
    {
        WriteLog("EndofData Exception = " + except.ToString() + processName,
                            EventLogEntryType.Error, 10, processName);
        source.Quarantine(e.MailItem.Recipients,
                           "StripIncomingLinkAgent - Error occurred: " + except.Message);
    }
}

13. Process Body method uses the regular expression class to remove any hyperlink or website address from the body of the message:

public class TextToTextLinkProcessor : IHyperLinkProcessor
{
private int m_LinkCount = 0;
private const string s_RegExBodyString =
 @"((www\.|(http|https|ftp|news|file)+\:\/\/)
 [&#95;.a-z0-9-]+\.[a-z0-9\/&#95;:@=.+?,##%&~-]*[^.|\'|\# |!|\(|?|,| |>|<|;|\)])";
private bool m_bChanged = false;
private string m_strReplacementText = string.Empty;
public bool WasChanged
{
    get
    {
        return m_bChanged;
    }
}
public void ProcessEmailBody(Body bodyMessage)
{
    m_LinkCount = 0;
    string savedContent = string.Empty;
    Stream memStream;
    Encoding encoding = StripLinkHelper.GetEncodingFromBody(bodyMessage.CharsetName);
    if (bodyMessage.TryGetContentReadStream(out memStream))
    {
        using (StreamReader streamRead = new StreamReader(memStream, encoding))
        {
            // TODO: May also want to decide on size of message and
            // whether or not it should be processed if it is too large.
            savedContent = FilterText(streamRead.ReadToEnd());
        }
        // Now write the new body only if it was changed/filtered.
        if (m_bChanged)
        {
            using (StreamWriter streamWriter = new
  StreamWriter(bodyMessage.GetContentWriteStream(), encoding))
            {
                streamWriter.Write(savedContent);
            }
        }
    }
}
public string FilterText(string strText)
{
    Regex rgx = new Regex(s_RegExBodyString, RegexOptions.IgnoreCase);
    string strFiltered = rgx.Replace(strText, new MatchEvaluator(
                match =>
                {
                    // If we got a match, mark it as changed.
                    m_bChanged = true;
                    m_LinkCount++;
                    return m_strReplacementText;
                }));
    return strFiltered;
}

14. GetLowerFidelityBodyPart enumerates through all the available body type and return the plain text body:

/// <summary>
/// Finds the lowest fidelity MimePart associated to a Body,
/// normally Text/Plain.
/// </summary>
/// <param name="body">The body part that we want
/// to find the lowest fidelity MimePart for.</param>
/// <returns></returns>
public static MimePart GetLowerFidelityBodyPart(Body body)
{
    // Nothing to work with if they are null...
    // Divided here for logging purposes..
    if (body == null)
    {
        return null;
    }
    else if (body.MimePart == null)
    {
        return null;
    }
    MimePart bodyPart = body.MimePart; ;
    // If it is text already, then that's the lowest fidelity...
    if (body.BodyFormat == BodyFormat.Text)
        return body.MimePart;
    // Need to find lower fidelity body type at the same level.
    // If no parent, there are no children, so this would be the only
    // part at this level.
    // If it has a parent but the parent is not multipart then this will also
    // be the only child. so:
    // if (bodyPart.Parent == null || !bodyPart.Parent.IsMultipart)
    // The other option is to simply check that we have no siblings.
    if (bodyPart.PreviousSibling == null && bodyPart.NextSibling == null)
    {
        return bodyPart;
    }
    // If we are here then we must have a parent and siblings. Get
    // the parent, find the lowest fidelity..
    IEnumerator<MimeNode> enumer = bodyPart.Parent.GetEnumerator();
    MimePart currentPart;
    while (enumer.MoveNext())
    {
        currentPart = (MimePart) enumer.Current;
        if (currentPart.ContentType.Equals(s_TextPlainContentType,
                                  StringComparison.OrdinalIgnoreCase))
        {
            return currentPart;
        }
    }
    return bodyPart;
}

15. The complete Visual Studio project can be found attached to this blog post. You can build the project by simply selecting build from the “Build” pull down menu using the release version.

16. Once the agent build is completed you can copy the StripIncomingLinkAgent from the \bin\release folder on the exchange server in the following folder:

C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\agents\SmtpAgents\StripIncomingLinkAgent

17. Use the following cmdlet to install and enable the agent on the Front End Transport on CAS. Note: if your CAS and Mailbox servers are on separate machine then you need to launch a window powershell to prevent the powershell proxy:

Launch a Window Powershell window and execute the following commands:

C:\Program Files\Microsoft\Exchange Server\V15\bin> Add-PSSnapin Microsoft.Exchange.Management.PowerShell.SnapIn

Install-TransportAgent -Name "FrontEndStripIncomingLinkAgent" -TransportAgentFactory "StripIncomingLinkAgent.StripLinkReceiveAgentFactory" -AssemblyPath "C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\agents\SmtpAgents\StripIncomingLinkAgent\StripIncomingLinkAgent.dll" -TransportService FrontEnd

Enable-TransportAgent -Identity "FrontEndStripIncomingLinkAgent" -TransportService FrontEnd

18. User the following cmdlet to install and enable the agent on the back end Transport on Mailbox:

Install-TransportAgent -Name "StripIncomingLinkAgent" -TransportAgentFactory "StripIncomingLinkAgent.StripLinkReceiveAgentFactory" -AssemblyPath "C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\agents\SmtpAgents\StripIncomingLinkAgent\StripIncomingLinkAgent.dll” -TransportService Hub

Enable-TransportAgent -Identity "FrontEndStripIncomingLinkAgent" -TransportService Hub

19. After installing the agent on back end, send an email from the pickup folder with a few link such as (www.msn.com, etc.)

20. The agent will remove all the links from the email body. Please remember that this is a sample only.

David Santamaria, Nasir Ali and Nasser Salemizadeh

StripIncomingLinkAgent.zip

[#2]

The article is great at explaining the changes in the Service and where those services live as well as describing the process for writing an agent.

What is missing is what Agents should be written where

an example of what agents should be written

•Front End Transport service:  

•Transport service:  

•Mailbox Transport service:

[#3]

I usually appreciate these blog posts but for those of us in corporate environments the Exchange 2013 posts are a little of the game right now. Those of us running Exchange are in almost all cases running earlier versions due to the fact that Exchange 2013 was released with out the ability to install in an existing exchange org. It will not be able to co-exist until SP3 for Exchange 2010 is released, not sure of 2007 and from what has been announced 2003 will not coexist at all although stopping 2003 co-existense does make sense.

[#4]

Agreed- most of us won't really use most of the Exchange 2013 posts for a while, but it's nice that the articles will already be there when we do need them.

[#5]

The problem is, when these articles comes out now, they will be closed at some point. And when you have a question you have no change to give feedback from the articles. So E2013 seems to be bad example :(

But I have also a question, when you write the script and implement it, how do you manage it? How you can troubleshooting it? Let say, you believe your script is the one which slow down your environment, how you can measure it?

[#6]

Nice and Clear Explanation - thank you so much !

[#7]

Why can't an SmtpReceiveAgent use the OnEndOfData event on the Front End Transport Service. I have a Transport Agent running on Exchange 2010 that needs OnEndOfData...

[#8]

@Robert - EndOfData Event is not supported in FrontEnd since proxying happens after the EndOfHeader event and it is not possible for FrontEnd to determine when is the end of data.

[#9]

Great article.

[#10]

Exchange 2013 CU1 Breaks my Transport Agent

I have a transport agent that works fine with Exchange 2013 RTM, but no longer installs with Exchange 2013 CU1!

install-transportagent fails with:

Error: The TransportAgentFactory type "xxxxx.yyyyy" doesn't exist. The TransportAgentFactory type must be the Microsoft .NET class type of the transport agent factory.

So I rebuilt the agent with the dlls shipped in CU1, and it installed and worked fine!

I am referring to these dlls of course:

Microsoft.Exchange.Data.Common.dll

Microsoft.Exchange.Data.Transport.dll

This is not good enough because I don’t want to redistribute a different agent build for each CU that MS releases.

Here are the details of my agent:

* VS 2012 C#

* .NET Framework 4

Project References to the MS DLLs are configured with Specific Version = False

[behrooz amiri]

Hi Exchange Team. We all noticed that recently Microsoft has removed all the Transport Agent Samples from the Docs.Microsoft.Com as well as the Download Center. even though Microsoft says (on the webpage!) that the contents are moved to either Archive or new Code Sample Experience section, but trust me, It's in neither places. I referred to the greater sample that you shared here during this year to develop 3 transport agents for our Env. and I wanted to thank you for the clear explanation and information that you guys provided. very much help full!

also, I was curious to know if, by any chance, anyone has this sample here (especially, "Build a body conversion transport agent")

https://docs.microsoft.com/en-us/exchange/client-developer/transport-agents/transport-agent-code-sam...

after Microsoft has removed access to these resources and samples, I have raised an issue to the Documentation repository in GitHub, however, they simply closed the issue and just referred me back to the new Code Sample section (obviously, as I mentioned, there's no sample there about Transport Agent development).

The Code Samples are still very handy and make the process of development much easier for guys like me, that are mostly Sys/Exchange admins with a little programming background.

[SonusProject]

Arriving at this thread with the goal of stripping hyperlinks embedded in inbound email via Exchange.  We have a hosted Exchange at Rackspace.  The implementation appears to be expecting the Exchange to be locally managed.

Has anyone implemented this on a hosted Exchange or is anyone familiar with another method for stripping a hyperlink before the email reaches the recipient?

Regards,

Lance

[Victor Ivanidze]

Hi Lance,

could you ask Rackspace if they allow you to install a transport agent?

If their answer is yes, I see no problem - you will create a transport agent that implements what you want.