SOLVED

Authentication Failure for IMAP and POP3 using Client Credential flow for OAuth2.0 | Java

Brass Contributor

I am facing an authentication failure issue while trying to connect for both IMAP and POP3 protocols using the Client Credential Grant flow for OAuth2.0

Where, I have been following the steps suggested in "Authenticate an IMAP, POP or SMTP connection using OAuth"

 

I have been using this github project to fetch the Access Token using Client Credential Grant flow:

MSAL Client Credential Grant using Java

 

Java Code for IMAP:

 

 

public static void connectIMAP(String userEmail, String accessToken){
		String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
		Properties props= new Properties();
		
		props.put("mail.imap.ssl.enable", "true");
		props.put("mail.imap.port", "993");
		
		props.put("mail.imap.auth.mechanisms", "XOAUTH2");
		props.put("mail.imap.sasl.mechanisms", "XOAUTH2");
		
		props.put("mail.imap.auth.login.disable", "true");
		props.put("mail.imap.auth.plain.disable", "true");
		
		props.setProperty("mail.imap.socketFactory.class", SSL_FACTORY);
        props.setProperty("mail.imap.socketFactory.fallback", "false");
        props.setProperty("mail.imap.socketFactory.port", "993");
		props.setProperty("mail.imap.starttls.enable", "true");
		
		props.put("mail.debug", "true");
		props.put("mail.debug.auth", "true");

		Session session = Session.getInstance(props);
		session.setDebug(true);

	
		try {
			final Store store = session.getStore("imap");					
			store.connect("outlook.office365.com",userEmail, accessToken);	
			
			if(store.isConnected()){
				System.out.println("Connection Established using imap protocol successfully !");		
			}
		} catch (NoSuchProviderException e) {	// session.getStore()
			e.printStackTrace();
		} catch (MessagingException e) {		// store.connect()
			e.printStackTrace();
		}
	}

 

 

Java code for POP3:

 

 

public static void connectPOP(String email, String accessToken){
		Properties properties= new Properties();

        properties.put("mail.pop3.port", 995);
        properties.put("mail.pop3.forgettopheaders", "true");	

        properties.put("mail.pop3.auth.mechanisms", "XOAUTH2");
        properties.put("mail.pop3.auth.login.disable", "true");		// If true, prevents use of the USER and PASS commands. Default is false.
        properties.put("mail.pop3.auth.plain.disable", "true");		// If true, prevents use of the AUTH PLAIN command. Default is false.
        properties.put("mail.pop3.auth.xoauth2.disable","false");	// If true, prevents use of the AUTHENTICATE XOAUTH2 command. Hence set it to false
        properties.put("mail.pop3.auth.xoauth2.two.line.authentication.format", "true");  // If true, splits authentication command on two lines. Default is false.
       
        properties.put("mail.pop3.connectiontimeout", 15000);
        properties.put("mail.pop3.timeout", 15000);
        properties.put("mail.debug", "true");

        Session session = Session.getInstance(properties);
        session.setDebug(true);
        try{
	        Store store = session.getStore("pop3");
	        store.connect("outlook.office365.com", email, accessToken);
        
	        if(store.isConnected()){
	        	System.out.println("Connected with pop3 successfully !");
	        }
        }catch(Exception e){
        	e.printStackTrace();
        }
	}

 

 

 

Following are the credentials which I have used while performing the Client Credential Grant flow

  • userEmail:- Email of the user which is used to login to Azure portal (eg, email address removed for privacy reasons)
  •  authority=https://login.microsoftonline.com/<tenant - id - here>/
  •  client_id= <client(application) - id - here> 

ClientAndTenantID.jpg

Note: I have been using the Default Active Directory, and the default user(Admin) for my Azure account. Is it fine this way ? or does it require a new custom Azure AD and a separate tenant for performing client credential flow

 

Below Image contains list of permissions I have applied in my app:

APIpermissions.jpg

Error Logs:

Spoiler

*** IMAP ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: enable SASL
DEBUG IMAP: SASL mechanisms allowed: XOAUTH2
DEBUG IMAP: closeFoldersOnStoreFailure
DEBUG IMAP: trying to connect to host "outlook.office365.com", port 993, isSSL true
* OK The Microsoft Exchange IMAP4 service is ready. [UABOADMAU...<token_value>]
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTH: XOAUTH2
DEBUG IMAP: protocolConnect login, host=outlook.office365.com, user=email address removed for privacy reasons, password=<non-null>
DEBUG IMAP: SASL Mechanisms:
DEBUG IMAP: XOAUTH2
DEBUG IMAP:
DEBUG IMAP: SASL client XOAUTH2
DEBUG IMAP: SASL callback length: 2
DEBUG IMAP: SASL callback 0: javax.security.auth.callback.NameCallback@73f9ac
DEBUG IMAP: SASL callback 1: javax.security.auth.callback.PasswordCallback@1064425
A1 AUTHENTICATE XOAUTH2 dXNlc...<token-value-here>
A1 NO AUTHENTICATE failed.
javax.mail.AuthenticationFailedException: AUTHENTICATE failed.
at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:725)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectIMAP(ClientCredentialGrantAndConnect.java:166)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:45)


*** POP3 ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]
DEBUG POP3: mail.pop3.rsetbeforequit: false
DEBUG POP3: mail.pop3.disabletop: false
DEBUG POP3: mail.pop3.forgettopheaders: true
DEBUG POP3: mail.pop3.cachewriteto: false
DEBUG POP3: mail.pop3.filecache.enable: false
DEBUG POP3: mail.pop3.keepmessagecontent: false
DEBUG POP3: mail.pop3.starttls.enable: true
DEBUG POP3: mail.pop3.starttls.required: true
DEBUG POP3: mail.pop3.finalizecleanclose: false
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "outlook.office365.com", port 995, isSSL false
javax.mail.AuthenticationFailedException: Read timed out
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:213)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectPOP(ClientCredentialGrantAndConnect.java:209)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:50)

 

Following is the list of jars I have used as a part of this development:

listOfJars.jpg

 

My Java Code Link(ideone): ClientCredentialGrantAndConnect.java 

 

Please help and let me know if the program is not correct.

Or if any important step seems to be missing.

 

Thank you.

 

60 Replies
Hi, thank you for the link.
I created mail store through my code and also created it using Spring Integration Mail, one after another.
So when I removed one, I did not get error. Probably something unimportant.

I think I finally have working code. :D
Congrats on getting in working. A real mind bender Microsoft sent us on. If I may ask, where in the Azure GUI did you get the ORGANIZATION_ID from? Is this the same as the Directory Tenant ID in Azure Application GUI?

@jambo Yes, that is the Tenant ID.

Probably you can execute command without Organization ID if you only have one tenant, but it is easier with it, just to be sure.

Still can't get it to work. When executing the command "Get-ServicePrincipal -Organization <ORGANIZATION_ID> | fl" the serviceId outputted is the same value as OBJECT_ID in the New-ServicePrincipal command, which happens to be equals to Object ID field in the Azure application Overview. Is this correct? From your description, the serviceId value from Get-ServicePrincipal should be different to OBJECT_ID but in my case, they it is the same.

Not that you mentioned it, it looks like I saw wrong. ServiceId is the same as one that I used.
But I used ObjectId from Enterprise Application View, instead of App Registration view.

I used that object ID for creating new principal and for adding mail box permission.

I will retry everything again, to be sure and I will get back to you.
This took me back to the documentation to register the service principal in exchange. I was using the registered app's objectId in the New-Service-Principal and Add-MailBox-Permission cmdlets. I changed that to AAD Enterprise Applications's objectId in both commands. With that i'm able to authenticate and read mail from INBOX. The documentation is very confusing in terms of service principal identifier. The documentation says service principal identifier in the Add-MailBox-Permissions is different from the previous one. In my case both are same, which is the Enterprise Application's ObjectId. Thanks a lot for pointing this out. I have been stuck on this for days, now everything is working smoothly :)
Perfect. Working for me too! Thanks for the suggestion to use Enterprise Application View. I was using the object Id from the App Registration section.

@borist2 Thank you for pointing out this clue where we need to use OBJECT ID from the Enterprise application view in all the cmdlets i.e. New-ServicePrincipal, Get-ServicePrincipal and Add-MailboxPermission.
It is finally working fine (only for IMAP flow) for me after trying out the same set of steps on a new application, keeping in mind that I have to use OBJECT ID value from Enterprise Application view.

But still there is some issue while trying to connect with this application for POP3 flow.

 

As per my understanding, following is the list of parameters used while performing Service Principal related queries:

(Please correct me if I am wrong)

Parameters used (and where to find them):

  1. appId: Application (client) ID [ found in Application Overview screen, from both Enterprise and App reg]
  2. entObjId: Object ID(Enterprise app) [ found in Enterprise Application Overview screen only ]
  3. orgId: Directory (tenant) ID [ found in Azure AD overview screen ]

Commands:

  • New-ServicePrincipal -AppId appId -ServiceId entObjId -Organization orgId
  • Get-ServicePrincipal -Organization entObjId | fl
  • Add-MailboxPermission -Identity "<email_id_here>" -User entObjId -AccessRights FullAccess

 

Confusions:

  • In Add-MailboxPermission cmdlet, <SERVICE_PRINCIPAL_ID> creates confusion, because in order to apply permissions like "IMAP.AccessAsApp", the internet tells that "Service Principal ID" can be found at [ Azure AD -> Enterprise Application -> (chosen application) -> Permissions -> IMAP.AccessAsApp -> use the Service Principal ID from Flyout menu ]
  • Enterprise Object ID can be used in place of Organization ID in all 3 cmdlets

 

@jambo @Anjitha170 @DestryHines Thanks for your suggestions and findings on this issue that I have raised.

 

However I am still unable to establish a connection through the POP3 protocol, and I want to do it just like we did it for IMAP. Below is the error log for the issue I am facing while trying to connect with POP3. Any help and suggestions will be much appreciated.

 

Spoiler

*** pop3 ***

DEBUG: JavaMail version 1.5.5
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.5
DEBUG: getProvider() returning javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]
DEBUG POP3: mail.pop3.rsetbeforequit: false
DEBUG POP3: mail.pop3.disabletop: false
DEBUG POP3: mail.pop3.forgettopheaders: true
DEBUG POP3: mail.pop3.cachewriteto: false
DEBUG POP3: mail.pop3.filecache.enable: false
DEBUG POP3: mail.pop3.keepmessagecontent: false
DEBUG POP3: mail.pop3.starttls.enable: true
DEBUG POP3: mail.pop3.starttls.required: false
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "outlook.office365.com", port 995, isSSL false
+OK The Microsoft Exchange POP3 service is ready. [TQBBAFgAUABSADAAMQAwADEAQwBBAD...==]
CAPA
+OK
TOP
UIDL
SASL PLAIN XOAUTH2
USER
.
DEBUG POP3: authentication command trace suppressed
DEBUG POP3: authentication command failed
QUIT
<EOF>
javax.mail.AuthenticationFailedException: Protocol error. Connection is closed. 10
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:209)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrant.connectPOP(ClientCredentialGrant.java:242)
at test.ClientCredentialGrant.main(ClientCredentialGrant.java:53)

 

 

@manish1614 

 

It seems like your POP3 authentication command is incorrect?

The documentation says to use:

 

AUTH XOAUTH2
<base64 string in XOAUTH2 format>

 

For example:

[connection begins]
C: AUTH XOAUTH2
S: +
C: dXNlcj1zb21ldXNlckBleGFtcGxlLmNvbQFhdXRoPUJlYX
JlciB5YTI5LnZGOWRmdDRxbVRjMk52YjNSbGNrQmhkSFJoZG1semRHRXVZMjl0
Q2cBAQ==
S: +OK User successfully authenticated.
[connection continues...]



 

I read somewhere you to need add the following the session. Not sure if that will help.

props.put("mail.pop3.auth.xoauth2.two.line.authentication.format", "true");

 

@DestryHines Thanks for pointing this out. For me, this command is executed internally from the JavaMail library functions. After reviewing your comment I tried to split the command into 2 lines using the property, "mail.pop3.auth.xoauth2.two.line.authentication.format" as true

Reference was taken from list of pop3 properties

 

But I am still unable to establish a connection with POP3 protocol. And getting the same error message.

Note: Updated the POP3 code in this post

 

@jambo Thank you for suggesting me to apply this property. I have tried to implement it in my existing code but I haven't got any success in establishing a connection through POP3.

 

Apart from this property, I tried setting:-

  • mail.pop3.auth.xoauth2.disable as false
  • mail.pop3.auth.mechanisms as XOAUTH2
  • mail.pop3.starttls.enable as true

Please let me know if any other parameters are required, or an existing parameter needs to be removed.

I'm not using Java but I'm having a similar problem in PHP using a third-party library called PHPMailer with SMTP now. I think most implementations didn't properly support client_crendentials for the grant type because Microsoft didn't have it working properly. You may need to get an updated version or figure out in your Java code how it's handling the oAuth grant type and update the code until there's an update.
I have been using JavaMail jar 1.5.5 all this time, so I tried to update it to 1.6.2(latest).
I have checked the msal4j jar, which have version 1.12.0(released on May 06, 2022), seems fine.
Post updating the jars I ran the flow for POP3 connection again, but didn't got any success till now.

In some posts they say that we need to add the scope as "https://outlook.office.com/POP.AccessAsUser.All" explicitly in the code, but never tell how or where to specify it exactly.
As per my findings, the only valid scope value is "https://outlook.office365.com/.default"

@borist2 Is that Bad user authenticated not connected issue fixed?

I got the same thing after I finally fixed all the other issues and the next day it went away. Seems to be related to too many failed logins in this case.

I'm having a similar problem, where a long running process must have access to send emails using @outlook.com on user's behalf.
After spending a couple days I found this small paragraph in the docs:

 

Fabricio_Ferreira_0-1660777299729.png

 

Link: https://docs.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-i...

 

So, apparently the combination of client_credentials and SMTP is something that outlook.com doesn't really support.

 

I can't manage to get a token from my app registration with the correct scope. 

 

Anyone here got it working?

According to one of the Exchange engineers oAuth SMTP isn't supported yet. From what I've read it will never be enforced because devices like Printers that send SMTP cannot be updated to support oAuth SMTP. When it's finally supported, we can change the code to use it but don't really need to for most situations.
Hi Manish,

I am also having similar issue and spent significant amount of time in searching for proper solution. Can you provide the steps that you followed in setting up the app to resolve the issue. I really appreciate your help.
Yes. It was something that I messed up in my code. Thanks.

Hi @manish1614 

I had a similar problem with POP3 with the latest version of javax.mail (1.6.2)

However, I was looking at the documentation and the project move to jakarta mail. I replaced project dependency with jakarta.mail 1.6.7 and POP3 started to work.

Hopefully it can worked for you as well.