Authentication Failure for IMAP and POP3 using Client Credential flow for OAuth2.0 | Java

[manish1614]

I am facing an authentication failure issue while trying to connect for both IMAP and POP3 protocols using the Client Credential Grant flow for OAuth2.0

Where, I have been following the steps suggested in "Authenticate an IMAP, POP or SMTP connection using OAuth"

I have been using this github project to fetch the Access Token using Client Credential Grant flow:

MSAL Client Credential Grant using Java

Java Code for IMAP:

public static void connectIMAP(String userEmail, String accessToken){
		String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
		Properties props= new Properties();
		
		props.put("mail.imap.ssl.enable", "true");
		props.put("mail.imap.port", "993");
		
		props.put("mail.imap.auth.mechanisms", "XOAUTH2");
		props.put("mail.imap.sasl.mechanisms", "XOAUTH2");
		
		props.put("mail.imap.auth.login.disable", "true");
		props.put("mail.imap.auth.plain.disable", "true");
		
		props.setProperty("mail.imap.socketFactory.class", SSL_FACTORY);
        props.setProperty("mail.imap.socketFactory.fallback", "false");
        props.setProperty("mail.imap.socketFactory.port", "993");
		props.setProperty("mail.imap.starttls.enable", "true");
		
		props.put("mail.debug", "true");
		props.put("mail.debug.auth", "true");

		Session session = Session.getInstance(props);
		session.setDebug(true);

	
		try {
			final Store store = session.getStore("imap");					
			store.connect("outlook.office365.com",userEmail, accessToken);	
			
			if(store.isConnected()){
				System.out.println("Connection Established using imap protocol successfully !");		
			}
		} catch (NoSuchProviderException e) {	// session.getStore()
			e.printStackTrace();
		} catch (MessagingException e) {		// store.connect()
			e.printStackTrace();
		}
	}

Java code for POP3:

public static void connectPOP(String email, String accessToken){
		Properties properties= new Properties();

        properties.put("mail.pop3.port", 995);
        properties.put("mail.pop3.forgettopheaders", "true");	

        properties.put("mail.pop3.auth.mechanisms", "XOAUTH2");
        properties.put("mail.pop3.auth.login.disable", "true");		// If true, prevents use of the USER and PASS commands. Default is false.
        properties.put("mail.pop3.auth.plain.disable", "true");		// If true, prevents use of the AUTH PLAIN command. Default is false.
        properties.put("mail.pop3.auth.xoauth2.disable","false");	// If true, prevents use of the AUTHENTICATE XOAUTH2 command. Hence set it to false
        properties.put("mail.pop3.auth.xoauth2.two.line.authentication.format", "true");  // If true, splits authentication command on two lines. Default is false.
       
        properties.put("mail.pop3.connectiontimeout", 15000);
        properties.put("mail.pop3.timeout", 15000);
        properties.put("mail.debug", "true");

        Session session = Session.getInstance(properties);
        session.setDebug(true);
        try{
	        Store store = session.getStore("pop3");
	        store.connect("outlook.office365.com", email, accessToken);
        
	        if(store.isConnected()){
	        	System.out.println("Connected with pop3 successfully !");
	        }
        }catch(Exception e){
        	e.printStackTrace();
        }
	}

Following are the credentials which I have used while performing the Client Credential Grant flow

• userEmail:- Email of the user which is used to login to Azure portal (eg, email address removed for privacy reasons)
•  authority=https://login.microsoftonline.com/<tenant - id - here>/
•  client_id= <client(application) - id - here> 

• secret= <client - secret - key - here>
• scope= https://outlook.office365.com/.default

Note: I have been using the Default Active Directory, and the default user(Admin) for my Azure account. Is it fine this way ? or does it require a new custom Azure AD and a separate tenant for performing client credential flow

Below Image contains list of permissions I have applied in my app:

Error Logs:

Spoiler

*** IMAP ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle]
DEBUG IMAP: mail.imap.fetchsize: 16384
DEBUG IMAP: mail.imap.ignorebodystructuresize: false
DEBUG IMAP: mail.imap.statuscachetimeout: 1000
DEBUG IMAP: mail.imap.appendbuffersize: -1
DEBUG IMAP: mail.imap.minidletime: 10
DEBUG IMAP: enable STARTTLS
DEBUG IMAP: enable SASL
DEBUG IMAP: SASL mechanisms allowed: XOAUTH2
DEBUG IMAP: closeFoldersOnStoreFailure
DEBUG IMAP: trying to connect to host "outlook.office365.com", port 993, isSSL true
* OK The Microsoft Exchange IMAP4 service is ready. [UABOADMAU...<token_value>]
A0 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
A0 OK CAPABILITY completed.
DEBUG IMAP: AUTH: PLAIN
DEBUG IMAP: AUTH: XOAUTH2
DEBUG IMAP: protocolConnect login, host=outlook.office365.com, user=email address removed for privacy reasons, password=<non-null>
DEBUG IMAP: SASL Mechanisms:
DEBUG IMAP: XOAUTH2
DEBUG IMAP:
DEBUG IMAP: SASL client XOAUTH2
DEBUG IMAP: SASL callback length: 2
DEBUG IMAP: SASL callback 0: javax.security.auth.callback.NameCallback@73f9ac
DEBUG IMAP: SASL callback 1: javax.security.auth.callback.PasswordCallback@1064425
A1 AUTHENTICATE XOAUTH2 dXNlc...<token-value-here>
A1 NO AUTHENTICATE failed.
javax.mail.AuthenticationFailedException: AUTHENTICATE failed.
at com.sun.mail.imap.IMAPStore.protocolConnect(IMAPStore.java:725)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectIMAP(ClientCredentialGrantAndConnect.java:166)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:45)

*** POP3 ***

DEBUG: JavaMail version 1.5.6
DEBUG: successfully loaded resource: /META-INF/javamail.default.providers
DEBUG: Tables of loaded providers
DEBUG: Providers Listed By Class Name: {com.sun.mail.smtp.SMTPSSLTransport=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], com.sun.mail.smtp.SMTPTransport=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle], com.sun.mail.imap.IMAPSSLStore=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], com.sun.mail.pop3.POP3SSLStore=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], com.sun.mail.imap.IMAPStore=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], com.sun.mail.pop3.POP3Store=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]}
DEBUG: Providers Listed By Protocol: {imaps=javax.mail.Provider[STORE,imaps,com.sun.mail.imap.IMAPSSLStore,Oracle], imap=javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Oracle], smtps=javax.mail.Provider[TRANSPORT,smtps,com.sun.mail.smtp.SMTPSSLTransport,Oracle], pop3=javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle], pop3s=javax.mail.Provider[STORE,pop3s,com.sun.mail.pop3.POP3SSLStore,Oracle], smtp=javax.mail.Provider[TRANSPORT,smtp,com.sun.mail.smtp.SMTPTransport,Oracle]}
DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map
DEBUG: setDebug: JavaMail version 1.5.6
DEBUG: getProvider() returning javax.mail.Provider[STORE,pop3,com.sun.mail.pop3.POP3Store,Oracle]
DEBUG POP3: mail.pop3.rsetbeforequit: false
DEBUG POP3: mail.pop3.disabletop: false
DEBUG POP3: mail.pop3.forgettopheaders: true
DEBUG POP3: mail.pop3.cachewriteto: false
DEBUG POP3: mail.pop3.filecache.enable: false
DEBUG POP3: mail.pop3.keepmessagecontent: false
DEBUG POP3: mail.pop3.starttls.enable: true
DEBUG POP3: mail.pop3.starttls.required: true
DEBUG POP3: mail.pop3.finalizecleanclose: false
DEBUG POP3: mail.pop3.apop.enable: false
DEBUG POP3: mail.pop3.disablecapa: false
DEBUG POP3: connecting to host "outlook.office365.com", port 995, isSSL false
javax.mail.AuthenticationFailedException: Read timed out
at com.sun.mail.pop3.POP3Store.protocolConnect(POP3Store.java:213)
at javax.mail.Service.connect(Service.java:366)
at javax.mail.Service.connect(Service.java:246)
at test.ClientCredentialGrantAndConnect.connectPOP(ClientCredentialGrantAndConnect.java:209)
at test.ClientCredentialGrantAndConnect.main(ClientCredentialGrantAndConnect.java:50)

Following is the list of jars I have used as a part of this development:

My Java Code Link(ideone): ClientCredentialGrantAndConnect.java 

Please help and let me know if the program is not correct.

Or if any important step seems to be missing.

Thank you.

[#2]

@manish1614I'm also facing the same issue with IMAP OAuth Authentication with Client Credentials flow. I have followed Microsoft's documentation as you mentioned above. I have given IMAP.AccessAsApp, POP.AccessAsApp permissions, but the access_token is not working with those permissions. Did you find any solution for this?

[#3]

I'm having the same issue as well. I've tried every combination of things the documentation provides. If anyone finds a solution, please post it here.

Not sure if it helps you guys but I noticed the first post shows the scope as "https://outlook.office.com/.default" but in this post from 7/12/22 it specifically says to use "https://outlook.office365.com/.default".

[#4]

@DestryHines Thanks for your response, it was a typo which I have corrected now from "https://outlook.office.com/.default" to "https://outlook.office365.com/.default".
I have cross-checked the documentation several times now, but I am still unable to resolve this problem.

[#5]

Thanks for you reply Anjitha. As of now I have not found any solution for this issue. I will update on the same thread as soon as I find out a working fix for this situation.

[#6]

We're in the same same boat here.. been working on this the entire day and haven't been able to get it to work. One thing I am confused about. How does Office 365 link the username (.e.g. email address removed for privacy reasons) to the OAUTH2 credential? Am I supposed to be able to login into any mailbox using the access token? I am getting the access token from a certificate attached to a created application in Azure.

[#7]

I was confused about that at first too. So I believe the answer is that in each of the mailboxes you want your app to access you must run the PowerShell commands that grant the App you've setup to the mailbox. You can see the app (if it's assigned correctly) in the user detail under "Applications". So, you request the access token as the client_id/secret for the app, then you can use that in the IMAP session, but for the "user" you use the email address you want to check. Whichever email addresses your app is assigned to it can read the email for. That's at least how I read it described.

[#8]

What once was a simple process, has now become infinitely more complicated! [cut this post because it contained potentially misleading info]

[#9]

I have the Microsoft Graph -> IMAP.AccessAsUser.All app permission but I *think* that only applies if you're accessing email from the Graph API (https://graph.microsoft.com) and not when using regular IMAP at https://outlook.office365.com. For that I added the IMAP.AccessAsApp permissions
the "Office 365 Exchange Online", which I think is the right resource.

[#10]

If I may ask, where did you add it? I don't see Office 365 Exchange Online API in the Azure app registration editor.

[#11]

In API Permissions, I click on "+Add a permission". Then in the popup on the right (at the top), select "APIs my organization uses". In there, type "office 365" in the search and you should see "Office 365 Exchange Online".

[#12]

Thanks, However, in my case it doesn't help as my app access Imap in the background, and therefore requires an application permission. I don't see the requisite permissions in the Azure GuI. To me, it looks like Microsoft has some unfinished work to do.

[#13]

My app is also (attempting) to access IMAP in the background (grant_type=client_credentials) and according to Microsoft this now works as of June 2022. You must use Application Permissions for Office 365 Exchange Online.

[#14]

Thanks for the hint. I added Exchange Online scope https://ps.outlook.com/IMAP.AccessAsApp as per your suggestion. I tried requesting https://outlook.office365.com/.default and then https://outlook.office.com/.default. Neither worked for me. Authentication failed.

[#15]

@jambo and @DestryHines As per my understanding, if you are unable to find "Office 365 Exchange Online" in <API-Permissions -> +Add a permission -> APIs my organization uses> then probably it is because you are not having an active subscription.
However I have tried applying the Service Principals (like POP.AccessAsApp and IMAP.AccessAsApp) as suggested by the step-by-step guide, using GUI as well as PowerShell(as given in "Register service principals in Exchange" section), but didn't got any success yet.

[#16]

All I wanted was to access a mailbox, did not know I was trying to build a nuclear submarine. Microsoft did a terrible job of replacing a trivial process with something incredibly complicated, with multiple steps, and next to none support

[#17]

Thank you Manish.

[#18]

If your app is a long-running process that runs in the background, my understanding is that you need to add https://ps.outlook.com/IMAP.AccessAsApp permission. I've done this, but still AUTHENTICATE failed. 

[#19 · Accepted solution]

Hi. I had the same problem, but I think I made some progress.

I read documentation few times, tried few times from the start with same error. I even have tried using client and object ids instead of email as username, in lack of better ideas.

So this is where I think I have made mistake previous times.

When you are at the part that you need to register service principal, you need to execute

New-ServicePrincipal -AppId <APPLICATION_ID> -ServiceId <OBJECT_ID> [-Organization <ORGANIZATION_ID>]

 here I put enterprise application object id as ServiceId argument. And that is ok.

But on 

Add-MailboxPermission -Identity "email address removed for privacy reasons" -User 
<SERVICE_PRINCIPAL_ID> -AccessRights FullAccess

I have put my registered application object id as User argument. I also tried setting object id of enterprise application, but it did not have success.

I also tried New-ServicePrincipal but with registered app object id as service id, but it gave me the same result.

When I executed 

Get-ServicePrincipal -Organization <ORGANIZATION_ID> | fl

I did not pay attention to ServiceId property, even with documentation specifying it and saying it will be different.

Now I cleared everything and started fresh.

I executed all the steps again, but on the step when I need to add mail permission, I list service principals, and then use `ServiceId` value from the output, as argument for user.

With that, I was able to authorise. 

Unfortunately, now I receive `C3 BAD User is authenticated but not connected.` when I try to list inbox. But it is step forward.

I am not sure if you made the same error as me, but maybe it will help you in some way.

I will post info, when I find fix for the new error if somebody comes across same issue.  

[#20]

That sounds very promising! I looked up the error you're getting and maybe the last part of this post helps?
https://www.limilabs.com/qa/196/office-365-bad-user-is-authenticated-but-not-connected