Authentication Failure for IMAP and POP3 using Client Credential flow for OAuth2.0 | Java

[manish1614]

I am facing an authentication failure issue while trying to connect for both IMAP and POP3 protocols using the Client Credential Grant flow for OAuth2.0

Where, I have been following the steps suggested in "Authenticate an IMAP, POP or SMTP connection using OAuth"

I have been using this github project to fetch the Access Token using Client Credential Grant flow:

MSAL Client Credential Grant using Java

Java Code for IMAP:

public static void connectIMAP(String userEmail, String accessToken){
		String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
		Properties props= new Properties();
		
		props.put("mail.imap.ssl.enable", "true");
		props.put("mail.imap.port", "993");
		
		props.put("mail.imap.auth.mechanisms", "XOAUTH2");
		props.put("mail.imap.sasl.mechanisms", "XOAUTH2");
		
		props.put("mail.imap.auth.login.disable", "true");
		props.put("mail.imap.auth.plain.disable", "true");
		
		props.setProperty("mail.imap.socketFactory.class", SSL_FACTORY);
        props.setProperty("mail.imap.socketFactory.fallback", "false");
        props.setProperty("mail.imap.socketFactory.port", "993");
		props.setProperty("mail.imap.starttls.enable", "true");
		
		props.put("mail.debug", "true");
		props.put("mail.debug.auth", "true");

		Session session = Session.getInstance(props);
		session.setDebug(true);

	
		try {
			final Store store = session.getStore("imap");					
			store.connect("outlook.office365.com",userEmail, accessToken);	
			
			if(store.isConnected()){
				System.out.println("Connection Established using imap protocol successfully !");		
			}
		} catch (NoSuchProviderException e) {	// session.getStore()
			e.printStackTrace();
		} catch (MessagingException e) {		// store.connect()
			e.printStackTrace();
		}
	}

Java code for POP3:

public static void connectPOP(String email, String accessToken){
		Properties properties= new Properties();

        properties.put("mail.pop3.port", 995);
        properties.put("mail.pop3.forgettopheaders", "true");	

        properties.put("mail.pop3.auth.mechanisms", "XOAUTH2");
        properties.put("mail.pop3.auth.login.disable", "true");		// If true, prevents use of the USER and PASS commands. Default is false.
        properties.put("mail.pop3.auth.plain.disable", "true");		// If true, prevents use of the AUTH PLAIN command. Default is false.
        properties.put("mail.pop3.auth.xoauth2.disable","false");	// If true, prevents use of the AUTHENTICATE XOAUTH2 command. Hence set it to false
        properties.put("mail.pop3.auth.xoauth2.two.line.authentication.format", "true");  // If true, splits authentication command on two lines. Default is false.
       
        properties.put("mail.pop3.connectiontimeout", 15000);
        properties.put("mail.pop3.timeout", 15000);
        properties.put("mail.debug", "true");

        Session session = Session.getInstance(properties);
        session.setDebug(true);
        try{
	        Store store = session.getStore("pop3");
	        store.connect("outlook.office365.com", email, accessToken);
        
	        if(store.isConnected()){
	        	System.out.println("Connected with pop3 successfully !");
	        }
        }catch(Exception e){
        	e.printStackTrace();
        }
	}

Following are the credentials which I have used while performing the Client Credential Grant flow

• userEmail:- Email of the user which is used to login to Azure portal (eg, email address removed for privacy reasons)
•  authority=https://login.microsoftonline.com/<tenant - id - here>/
•  client_id= <client(application) - id - here> 

• secret= <client - secret - key - here>
• scope= https://outlook.office365.com/.default

Note: I have been using the Default Active Directory, and the default user(Admin) for my Azure account. Is it fine this way ? or does it require a new custom Azure AD and a separate tenant for performing client credential flow

Below Image contains list of permissions I have applied in my app:

Error Logs:

Spoiler

*** IMAP ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: enable SASL
DEBUG IMAP: SASL mechanisms allowed: XOAUTH2
DEBUG IMAP: closeFoldersOnStoreFailure
DEBUG IMAP: trying to connect to host "outlook.office365.com", port 993, isSSL true
* OK The Microsoft Exchange IMAP4 service is ready. [UABOADMAU...<token_value>]
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTH: XOAUTH2
DEBUG IMAP: protocolConnect login, host=outlook.office365.com, user=email address removed for privacy reasons, password=<non-null>
DEBUG IMAP: SASL Mechanisms:
DEBUG IMAP: XOAUTH2
DEBUG IMAP:
DEBUG IMAP: SASL client XOAUTH2
DEBUG IMAP: SASL callback length: 2
DEBUG IMAP: SASL callback 0: javax.security.auth.callback.NameCallback@73f9ac
DEBUG IMAP: SASL callback 1: javax.security.auth.callback.PasswordCallback@1064425
A1 AUTHENTICATE XOAUTH2 dXNlc...<token-value-here>
A1 NO AUTHENTICATE failed.
javax.mail.AuthenticationFailedException: AUTHENTICATE failed.
at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:725)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectIMAP(ClientCredentialGrantAndConnect.java:166)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:45)

*** POP3 ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]
DEBUG POP3: mail.pop3.rsetbeforequit: false
DEBUG POP3: mail.pop3.disabletop: false
DEBUG POP3: mail.pop3.forgettopheaders: true
DEBUG POP3: mail.pop3.cachewriteto: false
DEBUG POP3: mail.pop3.filecache.enable: false
DEBUG POP3: mail.pop3.keepmessagecontent: false
DEBUG POP3: mail.pop3.starttls.enable: true
DEBUG POP3: mail.pop3.starttls.required: true
DEBUG POP3: mail.pop3.finalizecleanclose: false
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "outlook.office365.com", port 995, isSSL false
javax.mail.AuthenticationFailedException: Read timed out
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:213)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectPOP(ClientCredentialGrantAndConnect.java:209)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:50)

Following is the list of jars I have used as a part of this development:

My Java Code Link(ideone): ClientCredentialGrantAndConnect.java 

Please help and let me know if the program is not correct.

Or if any important step seems to be missing.

Thank you.

[#2]

@LinaMM2022 Thank you for suggesting this change, but for my case we are currently moving forward with only IMAP based connections because there were other modules in my product which highly depends on the JavaMail jar.
In case if we plan to implement POP3, we would definitely try to connect using the Jakarta Mail api.

Thanks alot.

[#3]

Hello, I have a question.
We have same code that you have posted, so right now we use application password to connect with outlook and fetch emails. will this stop from 1st oct, do we need to move to OAuth2.0.

[#4]

@shrey_soni Yes your understanding is correct, from 1st Oct 2022, you will be required to move to OAuth2 in case you are still using Basic Authentication technique.
Please refer to this post for more information about the announcement.
Deprecation of Basic authentication in Exchange Online | Microsoft Learn

[#5]

Thank you @manish1614 for answering my question.

[#6]

@manish1614, if u have any code snippet for java code on how to implement OAuth2.0 also if the client has Exchange online 1 subscription that includes outlook and Todo still have to use OAuth2.0?

[#7]

@shrey_soni I have attached the java code link above as "ClientCredentialGrantAndConnect.java" where you can easily find the java code which I used during implementation. Or else, you can import the project directly to your local by from MSAL Client Credential Grant using Java.
Apart from that I have followed the steps mentioned in "Authenticate an IMAP, POP or SMTP connection using OAuth"

Must check the answer in this thread which I have marked as best response before running the cmdlets

[#8]

Hi please help on pop3 oauth flow, getting protocol error connection failed ,any help will be appreciated.

[#9]

@manish1614 

This video explains the necessary steps to solve the problem, pay attention to the PowerShell commands

https://www.youtube.com/watch?v=bMYA-146dmM&t=356s

[#10]

I have not implemented OAuth2.0 still my services works fine. So do I still need to work on implementing OAuth2.0? I am bit confused please help.

[#11]

If you are still using plaintext IMAP/POP auth, yes you most certainly you do. Microsoft will randomly disable Basic authentication up until the 31st of December 2022. In the interim if Basic Authentication has been disabled on your tenant it can be re-enabled following the following steps. How to temporarily re-enable IMAP plaintext auth in Office 365. 

1. Login as azure administrator
2. Click link: https://aka.ms/PillarEXOBasicAuth
3. Click Run Tests
4. Select IMAP, click check box, click update

[#12]

Hi @LinaMM2022
I have a similar problem with POP3, and when I installed jakarta.mail 1.6.7 problem still exists. Did you do anything else?
I have this debug text:
DEBUG POP3: Attempt to authenticate using mechanisms: XOAUTH2
DEBUG POP3: Using mechanism XOAUTH2
DEBUG POP3: AUTH XOAUTH2 command trace suppressed
DEBUG POP3: mail.pop3s.auth.xoath2.two.line.authentication.format: false
DEBUG POP3: AUTH XOAUTH2 using one line authentication format
DEBUG POP3: AUTH XOAUTH2 failed

[#13]

Hi @kirill247742.

The most important property for me was (And according to your log you're not using it): 

properties.put("mail.pop3s.auth.xoauth2.two.line.authentication.format", "true");

Apart from that, properties were basically the same from IMAP:

"mail.pop3s.auth.mechanisms", "XOAUTH2"

"mail.pop3.ssl.enable", "true"

[#14]

Hi @LinaMM2022
Thanks for the advice, at least the error is already different)
Maybe you have encountered a similar error and can tell how to solve it

DEBUG POP3: AUTH XOAUTH2 using two line authentication format
DEBUG POP3: AUTH XOAUTH2 failed
Authentication failure: unknow user name or bad password

I am sure the data is correct. Perhaps I missed some access? Or something else..

[#15]

Hi @kirill247742.
Well, it should be working with those parameteres. Maybe you should confirme that your fonts are indeed using jakarta and you don't have maybe library conflict.
Apart from that, make sure you're using the token (not the password from account) and have all the permissions in Azure for POP3.

[#16]

Thanks, @LinaMM2022 for the help!
Well, I don't see any library conflict and the token is correct. As for permissions in Azure for POP3, I'm not sure.
Now I have 6 of them:
Microsoft Graph (6)
- email
- IMAP.AccessAsUser.All
- offline_access
- POP.AccessAsUser.All
- SMTP.Send
- User.Read
Did I miss something important?

[#17]

@kirill247742. I think you're mixing functionalities.
This is not Microsoft graph. Look at the documentation shared in this thread, because you need to create an application in order to make everything.
If you want to use Microsoft Graph is a complete different implementation.

[#18]

I just got IMAP to work thanks to this thread.

Be careful to pass the raw token to store.connect() though. The last mistake I did was to process the token as explained in https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-...

But the library already takes care of that.

[#19]

@manish1614 

I had exactly the same problem ; altought i followed the documentation of microsoft.

I have finaly solved it :

when i get request the oauth token, we should put "https://outlook.office365.com/.default" in the scope parameter

but, since it is an uri, it should be encoded (in c# i use UrlEncode)

after that, i was able to use the token in my sasl auth  message and then i was authenticated (and got the respons "+OK User successfully authenticated.")

[#20]

great to hear that your problem got solved.
Thanks for pointing the UrlEncode trick in C# while assigning the scope param

[#21]

@manish1614 first of all, thanks for your post - it helps me a lot (especially I'm not JAVA developer at all )

Secondly, I would like to return the favor. You looked for the answer about POP3 issue, that it still not authenticate.

If everything is set correctly, you just have to switch to POP3S.

       properties.put("mail.pop3s.port", "995");

        properties.put("mail.pop3s.ssl.enable", "false");
        properties.put("mail.pop3s.starttls.enable", "true");
        properties.put("mail.pop3s.starttls.required", "true");

        properties.put("mail.pop3s.connectiontimeout", 5000);
        properties.put("mail.pop3s.timeout", 5000);
        properties.put("mail.pop3s.partialfetch", false);
        properties.put("mail.pop3s.auth.mechanisms", "XOAUTH2");

        properties.put("mail.pop3s.forgettopheaders", "true");
        properties.put("mail.pop3s.sasl.enable", "true");
        properties.put("mail.pop3s.sasl.mechanisms", "XOAUTH2");
        properties.put("mail.pop3s.auth.login.disable", "true");
        properties.put("mail.pop3s.auth.plain.disable", "true");

DEBUG: JavaMail version 1.6.2
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.6.2
DEBUG: getProvider() returning javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle]
DEBUG POP3: mail.pop3s.rsetbeforequit: false
DEBUG POP3: mail.pop3s.disabletop: false
DEBUG POP3: mail.pop3s.forgettopheaders: true
DEBUG POP3: mail.pop3s.cachewriteto: false
DEBUG POP3: mail.pop3s.filecache.enable: false
DEBUG POP3: mail.pop3s.keepmessagecontent: false
DEBUG POP3: mail.pop3s.starttls.enable: true
DEBUG POP3: mail.pop3s.starttls.required: true
DEBUG POP3: mail.pop3s.finalizecleanclose: false
DEBUG POP3: mail.pop3s.apop.enable: false
DEBUG POP3: mail.pop3s.disablecapa: false
DEBUG POP3: connecting to host "outlook.office365.com", port 995, isSSL true
+OK The Microsoft Exchange POP3 service is ready.

Unfortunately it's failing then on something else, but at least we were authenticated... 

DEBUG POP3: STLS required but not supported
QUIT
+OK Microsoft Exchange Server POP3 server signing off.
javax.mail.AuthenticationFailedException: STLS required but not supported

But that's another reason to switch to IMAP, and stop using POP3 - pardon POP3S