Named Properties, Round 2: What lies Ahead

Platinum Contributor

Jun 12, 2009

In my last blog post on named properties, I detailed the history of named properties, how they are arranged, mapped, and how Exchange (since Exchange 2000) has created named property mappings for x-headers on inbound messages. Rollup 8 for Service Pack 1 contains the changes we made to restrict named property mapping further, and with Service Pack 2 on the horizon there are more changes in the waters. This post details what is changed in Service Pack 2, how that affects (or does not) you, and includes a request for feedback.

As mentioned before, Exchange 2007 creates named properties for x-headers during delivery. The RTM version of Exchange 2007 would NDR messages if the named property quota was reached, even if the property were only for an x-header. Service Pack 1 changed this behavior to only NDR the message if a non X-header named property could not be mapped (in other words, a property likely to contain critical data). Rollup 8 for Service pack 1 removes the ability of non-authenticated messages to consume named property IDs. Service Pack 2 goes even further.

Service Pack 2 is in line with Exchange2010 behavior - that is, no x-headers are ever promoted to individual properties if a client has not already requested (and mapped) them. Not even authenticated submissions can create new named property mappings. The x-headers are still stored in PR_TRANSPORT_HEADERS and still accessible to MAPI clients but they are not individual properties. This creates a problem for IMAP clients in particular:

Exchange2007 does not contain an STM file - it doesn't save the original mime content. Messages are converted to MAPI. IMAP has the ability to search based on header values, but if these are not promoted, the result is that the message is not contained in the results. If a client has previously mapped t he header in question (a MAPI client) then messages delivered after this will have the x-header promoted as an individual property.

So, the questions I ask are:

Do you own or run an application which requires a custom x-header to be promoted to a property but does NOT create the mapping?
Do you own or run an application which uses an IMAP header search for a custom x-header.

Things that aren't on the table:

Ability to remove named properties.
Ability to pre-register named properties.

We're aware that both of these would prove useful. Also, this is not a voting contest - we've seen how a vocal minority can skew results before. Just a heads up and if you have real world reasons for arguing for a particular request, let us know.

- Jason Nelson

Updated Jul 01, 2019

Version 2.0

Platinum Contributor

Joined April 19, 2019

View Profile

Exchange Team Blog

You Had Me at EHLO.

20 Comments

Deleted
Jul 06, 2009
Thanks for the response.

This is probably worse than you've considered for any app that is implemented according to the Exchange 2007 Foreign Connector guidelines and is meant to allow for IMAP clients. The important thing to keep in mind is that this will break in ways that are very hard to trace and explain to customers, and generate lots of support calls.

Is there any way you would consider leaving this the way it is in RU8, where a FC that creates a tnef encoded message is able to squeeze in new x-headers? Or possibly Exchange could map x-headers to ids for messages that are sent through the Replay directory only?

Since there is no longer a MAPI gateway architecture, it makes sense to allow a little more flexibility here for those of us porting older gateways to the new 2007 FC architecture.

You mentioned in one post that x-headers/id mappings are changing from per mailbox store to per mailbox. Does this mean that in Ex 2010 we'll need to find a way to log onto each mailbox that may receive a message from a foreign connector and create the mapping wheras logging into one mailbox per store is good enough for 2007?
Deleted
Jul 03, 2009
Radu - If (as you indicate) the client is actually asking for those x-headers by name then in fact they will be promoted to individual named properties. If the submission is authenticated they will also be promoted to individual x-headers. On E2k7 if one user on an MDB has mapped an x-header to a named property all other users on that mdb will see it mapped to an individual property.

If no users have mapped an x-header to a named property id and the submission is not authenticated and the application has not requested this id, it will NOT be promoted to an individual property. It is available in PR_TRANSPORT_HEADERS. You can get around this by having your application send one user on the MDB an authenticated message with the x-header. OR you could have one user on the mdb actually retrieve the property.

In Exchange2010 that will not work - mappings are per user and each user would need the property mapped to see it promoted as an individual property.

James - TNEF should be properly restricted as well. I believe RU8 does not apply the constriction to TNEF in all cases, something we've fixed. PS_PUBLIC_STRINGS is not limited...yet. It's on our radar (and has been for a few months) as something to keep an eye on and consider a similar constraint.
Deleted
Jul 02, 2009
Sorry for the typo, my first question at the bottom should be:

Will the "x-test" prop be available to an IMAP client as an x-header?

Also:

Will the x-test prop be available as a named prop to MAPI clients? Or in the PR_TRANSPORT_HEADERS prop?
Deleted
Jul 01, 2009
How does this work for tnef encoded messages? Consider the scenario:

User A has mapped a xheader ("x-test") via a mapi client.

User B (on a differenent store db) has not mapped the "x-test" header.

User A sends a tnef encoded message with the header encoded as a named prop in the tnef body of the message to user B :
Tag: 0x808E001E
Type: PT_STRING8
Named Prop Name: sz: "x-test"
Named Prop Guid: {00020386-0000-0000-C000-000000000046} = PS_INTERNET_HEADERS

Will the prop get mapped to user B's database and be available via MAPI? Would sending the property using PS_PUBLIC_STRINGS as the Named Prop Guid give different results?
Deleted
Jul 01, 2009
Hello,
I work for an antivirus company that relies on these x-headers to pass certain information to the user (like the spam score of an email, what rule was matched, what AV engine version was used to scan the mail and so on). Preliminary tests with update rollout 8 showed that our headers are no longer delivered in the user email. How can we get around this?
Thank you.
Deleted
Jun 18, 2009
Havent seen any app. that rely on the propagation of x-headers but why dont make an on/off configuration switch for the propagation
Deleted
Jun 15, 2009
We also have an anti-spam device, Ironport, that stamps the header with with a spam score. However we don't search for it later. Any issue with a particular email and we open the header to check the score.

I assume the header will still contain the info just not be be searchable?
Deleted
Jun 12, 2009
JSC - Yes, if you are at the limit you should increase the quota, I would recommend increasing it by 200 and then monitoring growth closely. Nothing recovers named properties. Nothing. So if you are at the quota, you can either stay there (and NDR messages when they contain NON internet properties) or increase the quota. Since anonymous submissions are now barred from consuming x-headers the growth rate will be slowed.

Jason Madden -
Yes - I consider an IMAP MUA an application. Whether or not your MUA will continue to work depends entirely on whether a MAPI client has requested those X-headers (which will cause them to continue to be promoted. Since this is database wide in Exchange 2007 that means that if any other user on the database has asked via a MAPI client for an x-header or if an authenticated submission contains an x-header it will be available.
Deleted
Jun 12, 2009
Do you count IMAP MUAs as an "application"?

I recieve between several hundred and several thousand emails a day from automated process using SMTP. My IMAP MUA (alpine) filters them into folders based on X-headers. Later, I view the contents of those folders and run additional searches based on those same x-headers to further trim what I want to deal with. I know that others at my organization have similar workflows, and it would be extraordinarily inconvenient (at least) should that suddenly stop working.
Deleted
Jun 12, 2009
We own a Barracuda Spam Firewall which adds x-header information to all mail from the internet. Some examples are:

X-Barracuda-Spam-Score
X-Barracuda-Start-Time
X-Barracuda-Connect

These are the only x-header information that is useful to us. We would like to be able to remove named properties that got added from internet mail before we started stripping them out with the HeaderFliter agent by Codeplex so we don't have to increase the quota on our databases for named properties.

Thanks for the update on this post, but it still doesn't give us the answer that if we are at the quota limit right now before applying Rollup 8 for SP1, after we install the update, what do we do with the quota limit? Do we still need to increase it?

In our case, we dont' have any applications that need to promote custom named properties with the exception of people installing stupid mailplugins every now and then.

Blog Post

Named Properties, Round 2: What lies Ahead