Java EE (Java Enterprise Edition) General Discussion

Turn on session debugging and post the protocol trace
(see the FAQ). What IMAP server are you using? Most
likely your server is broken.

Following is the protocol trace

DEBUG: setDebug: JavaMail version 1.4ea
DEBUG: getProvider() returning javax.mail.Provider[STORE,imap,com.sun.mail.imap.IMAPStore,Sun Microsystems, Inc]
DEBUG: mail.imap.fetchsize: 16384
* OK GroupWise IMAP4rev1 Server Ready
A0 CAPABILITY
* CAPABILITY IMAP4rev1 STARTTLS AUTH=XGWTRUSTEDAPP XGWEXTENSIONS
A0 OK CAPABILITY completed
IMAP DEBUG: AUTH: XGWTRUSTEDAPP
DEBUG: protocolConnect login, host=172.20.100.48, user=myimapemail, password=<non-null>
A1 LOGIN myimapemail Password123
A1 OK LOGIN completed
DEBUG: connection available -- size: 1
A2 SELECT INBOX
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1138647146]
* OK [UIDNEXT 122]
* FLAGS (\Answered \Flagged \Deleted \Draft \Seen)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Draft \Seen \*)]
A2 OK [READ-WRITE] SELECT completed
A3 LIST "" INBOX
* LIST (\Noinferiors \Unmarked) "/" "INBOX"
A3 OK LIST completed
Mailbox has: 1 messages
A4 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE FLAGS BODYSTRUCTURE)
* 1 FETCH (ENVELOPE ("Fri, 3 Nov 2006 15:18:01 -0500" "Test Email 1 for org id 00-00-01" (("myimapemail" NIL "myimapemail" "fastmail.fm")) (("myimapemail" NIL "myimapemail" "fastmail.fm")) (("myimapemail" NIL "myimapemail" "fastmail.fm")) ((NIL NIL "myimapemail" "fastmail.fm")) NIL NIL NIL "<454BA3F9.4010705@fastmail.fm>") INTERNALDATE "3-Nov-2006 15:19:37 -0500" RFC822.SIZE 546 FLAGS (\Seen) BODYSTRUCTURE ("text" "plain" ("charset" "iso-8859-1" "format" ""flowed"") NIL NIL "7bit" 17 2 NIL NIL NIL))
A4 OK FETCH completed
Exception in thread main
java.lang.NullPointerException
at javax.mail.internet.ParameterList.set(ParameterList.java:220)
at com.sun.mail.imap.protocol.BODYSTRUCTURE.parseParameters(BODYSTRUCTURE.java:278)
at com.sun.mail.imap.protocol.BODYSTRUCTURE.<init>(BODYSTRUCTURE.java:165)
at com.sun.mail.imap.protocol.FetchResponse.parse(FetchResponse.java:146)
at com.sun.mail.imap.protocol.FetchResponse.<init>(FetchResponse.java:56)
at com.sun.mail.imap.protocol.IMAPResponse.readResponse(IMAPResponse.java:124)
at com.sun.mail.imap.protocol.IMAPProtocol.readResponse(IMAPProtocol.java:230)
at com.sun.mail.iap.Protocol.command(Protocol.java:263)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1234)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1215)
at com.sun.mail.imap.IMAPMessage.fetch(IMAPMessage.java:1015)
at com.sun.mail.imap.IMAPFolder.fetch(IMAPFolder.java:902)
at TestProgram.main(TestProgram.java:53)

It's a bug in GroupWise, report it to Novell.

Notice the quoting of "flowed" in the FETCH response:

("charset" "iso-8859-1" "format" ""flowed"")

When I comment out the following lines of code and run my program, I don't get the NullPointerException.

.......
// fp.add(FetchProfile.Item.CONTENT_INFO);
....
....
//  System.out.println("Content type : " +msgs.getContentType());

......

 

Right, because then you're not fetching the BODYSTRUCTURE.
And if that's all you need to do in your program you can avoid the
bug. But if your program needs to access the content of your message,
you'll need to fetch the BODYSTRUCTURE information from the server,
and that's when you'll run into the problem.

Again, the bug is in the server. Report it.

Will do that... Thanks again....

I am facing some weird problem here. Why does the code work for some mailboxes on the same email server and not for others.
For e.g. When I tested using one mailbox I got all the emails and attachments as I wanted with no exception and when I tested it for a different mailbox it throws the same exception as above. Also I could not find "flowed" anywhere in the trace.

It probably depends on the exact form of the message in the mailbox.
It could even be an incorrectly formatted message that the server
is failing to process properly.

If thats the case then how should I handle an incorrectly formatted message? Also if this is what is happening then I should at least get all the messages and then I can check if its well formatted or not. Is this a correct statement?

Also why does the mailbox (throwing exception) work with email clients like Thundurbird and Outlook Express?

The server is supposed to do this for you, which again is
why you should report the bug to the server vendor and get
them to fix it.

You can catch all the exceptions that occur when processing
a message and fall back to a different strategy. How you handle
these cases is what distinguishes a casual program from a
bullet-proof production program. Strategies for dealing with
such exceptions is a longer discussion than I can have here,
but there's several things that will help.

For some errors, letting JavaMail parse the message instead
of depending on the server to parse the message might help.
Use the MimeMessage copy constructor to get a copy of the
message; the parsing of that copy will then be done by JavaMail.

The getRawInputStream method will allow you to access the
encoded content of a message part.

The writeTo method can be used to get a complete copy of the
message that you can process or parse yourself.

Because, in general, those applications make very simple use
of the IMAP protocol, treating it as little more than a variant of
the POP protocol. They download the entire message to the client
and do all the message parsing there. This is a very inefficient
approach that sacrifices most of the advantages of the IMAP
protocol. The advantage is that it isolates them from the many
bugs in low quality IMAP servers.

Many IMAP server vendors take advantage of this fact and only
test their products against these clients, and thus never discover
the many bugs in their products that only effect clients that make
full use of the IMAP protocol.

GroupWise is not one of the worst here, but it has had its share
of bugs.

I did some research on why the code was throwing exception and found out that the exception is thrown for email messages with an embedded image in it. I went through each and every message from my inbox and narrowed it to a specific message with an embedded image in it. Why does the getContentType() method throw an exception for this type of email message? It works for all other messages.

Are you talking about a new exception, or the same exception
that started this thread?

I thought I was pretty clear that the problem is that the server is
sending bogus data to JavaMail, and JavaMail fails to parse it
correctly and gets a NullPointerException. The source of the
problem is the bogus data sent by the server. Why does the
server send bogus data in this case? I don't know, you'll have
to ask the vendor.

Sorry if I confused you. This is what is happening.

Message[] msgs = folderMailbox.getMessages();
/*// Use a suitable FetchProfile
FetchProfile fp = new FetchProfile();  
fp.add(FetchProfile.Item.ENVELOPE);  
fp.add(FetchProfile.Item.CONTENT_INFO);
fp.add(FetchProfile.Item.FLAGS);
folderMailbox.fetch(msgs, fp);
*/ 
for (int i=1; i<msgs.length; i++){
      System.out.println("msgs : " +msgs.getReceivedDate());

System.out.println("Content type : " +msgs[i].getContentType());

}



In the above code i commented out the fetch profile part. It did not throw the exception as the fetch method was commented.

When I try reading the content type it throws the following exception.



Exception in thread main

java.lang.NullPointerException

	at javax.mail.internet.ParameterList.set(ParameterList.java:220)

	at com.sun.mail.imap.protocol.BODYSTRUCTURE.parseParameters(BODYSTRUCTURE.java:278)

	at com.sun.mail.imap.protocol.BODYSTRUCTURE.<init>(BODYSTRUCTURE.java:105)

	at com.sun.mail.imap.protocol.FetchResponse.parse(FetchResponse.java:146)

	at com.sun.mail.imap.protocol.FetchResponse.<init>(FetchResponse.java:56)

	at com.sun.mail.imap.protocol.IMAPResponse.readResponse(IMAPResponse.java:124)

	at com.sun.mail.imap.protocol.IMAPProtocol.readResponse(IMAPProtocol.java:230)

	at com.sun.mail.iap.Protocol.command(Protocol.java:263)

	at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1234)

	at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1226)

	at com.sun.mail.imap.protocol.IMAPProtocol.fetchBodyStructure(IMAPProtocol.java:954)

	at com.sun.mail.imap.IMAPMessage.loadBODYSTRUCTURE(IMAPMessage.java:1236)

	at com.sun.mail.imap.IMAPMessage.getContentType(IMAPMessage.java:414)

	at TestProgram.main(TestProgram.java:59)



And this occurs for the email messages with an embedded image in it. For all other messages the code works fine and I am able to get the content type.



Hope I am clear. 

Yes, you're clear, but apparently I'm not.

You keep asking "why does JavaMail do this weird thing
when the server sends garbage?"

I'm not sure what sort of answer you're looking for.

Do you understand that the bug is in the server?

I can make JavaMail throw a different exception if you don't
like NullPointerException (and I've done that in the newest version),
but I can't make it magically work correctly when the server sends
garbage.

Is that clear?

The issue with the server was finally resolved. Thank you very much for your help.

Glad to hear you got it fixed!

Do you have a reference to a bug report or patch that I can point
other people at if they have this problem?

Sorry, I do not. I reported the bug to my client who opened up the ticket with Novell and got it fixed.

I am getting the same Exception. Can you explain what part of the server message is in error?

DEBUG: connection available -- size: 1
A3 SELECT Inbox
* 1 EXISTS
* 0 RECENT
* OK [UNSEEN 0] 0 unseen messages
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Seen \Deleted \Answered \Flagged \Draft)] Full
A3 OK [READ-WRITE] SELECT completed
A4 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 1 FETCH (ENVELOPE ("Wed, 29 Aug 2007 12:34:14 -0400" "Voice message" ((NIL NIL "5136080528" NIL)) NIL NIL ((NIL NIL "8597606371" NIL)) NIL NIL NIL "<0046d6f3ac.MESSAGE-ID.4acc.3b73.12ec.4b95>") INTERNALDATE "Wed, 29 Aug 2007 12:34:14 -0400" RFC822.SIZE 46628)
A4 OK FETCH completed
A5 FETCH 1 (BODY.PEEK[HEADER])
* 1 FETCH (BODY[HEADER] {521}
Return-Path: 5136080528
MIME-Version: 1.0
From: 5136080528
To: 8597606371
Date: Wed, 29 Aug 2007 12:34:14 -0400
Subject: Voice message
Message-ID: <0046d6f3ac.MESSAGE-ID.4acc.3b73.12ec.4b95>
Importance:
X-Priority: 3
Sensitivity:
X-Mailer: CTI CallCourier
Content-Type: multipart/mixed; boundary="--_=0046d6f3ac.BOUNDARY.0697.61d1.687c.1c92";
Content-Transfer-Encoding: 7bit
Content-Description: CTI rfc822 library generated multi-part message.
Content-ID: <0046d6f3ac.MESSAGE-ID.6a84.322c.187c.31c5>

)
A5 OK FETCH completed
Aug 30, 2007 12:43:24 PM com.cbt.se.vvm.server.MailServiceImpl getFolder
INFO: calling getContentType()
A6 FETCH 1 (BODYSTRUCTURE)
* 1 FETCH (BODYSTRUCTURE (("audio" "wav" ("name" "message.wav" NIL NIL) NIL NIL "base64" 45828 NIL("attachment" ("filename" "message.wav" NIL NIL)) NIL)("boundary" "--_=0046d6f3ac.BOUNDARY.0697.61d1.687c.1c92" NIL NIL) NIL NIL))
A6 OK FETCH completed
java.lang.NullPointerException
at javax.mail.internet.ParameterList.set(ParameterList.java:220)
at com.sun.mail.imap.protocol.BODYSTRUCTURE.parseParameters(BODYSTRUCTURE.java:278)
at com.sun.mail.imap.protocol.BODYSTRUCTURE.<init>(BODYSTRUCTURE.java:165)
at com.sun.mail.imap.protocol.BODYSTRUCTURE.<init>(BODYSTRUCTURE.java:83)
at com.sun.mail.imap.protocol.FetchResponse.parse(FetchResponse.java:146)
at com.sun.mail.imap.protocol.FetchResponse.<init>(FetchResponse.java:55)
at com.sun.mail.imap.protocol.IMAPResponse.readResponse(IMAPResponse.java:124)
at com.sun.mail.imap.protocol.IMAPProtocol.readResponse(IMAPProtocol.java:230)
at com.sun.mail.iap.Protocol.command(Protocol.java:263)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1234)
at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:1226)
at com.sun.mail.imap.protocol.IMAPProtocol.fetchBodyStructure(IMAPProtocol.java:954)
at com.sun.mail.imap.IMAPMessage.loadBODYSTRUCTURE(IMAPMessage.java:1236)
at com.sun.mail.imap.IMAPMessage.getContentType(IMAPMessage.java:414)
at com.cbt.se.vvm.server.MailServiceImpl.getFolder(MailServiceImpl.java:97)
at Test.main(Test.java:18)

It's a server bug. What server are you using?

The bug is here:

"audio" "wav" ("name" "message.wav" NIL NIL)

The parenthesized list is a set of name/value pairs. NIL is not a valid name.

What do the headers in the original message look like? Can you save the
entire message to a file using msg.writeTo(new FileOutputStream("msg.txt"))
and then examine the headers?

It is a voice mail box.

utstarcom PCMS v 7.0.119
(http://www.utstar.com/Solutions/Switching/Personal_Communications/8250_Personal_Communication_System/)

Return-Path: 513xxxxxxx
MIME-Version: 1.0
From: 513xxxxxxx
To: 859xxxxxxx
Date: Wed, 29 Aug 2007 12:34:14 -0400
Subject: Voice message
Message-ID: <0046d709cb.MESSAGE-ID.4c8d.4669.705e.2d4a>
Importance: 
X-Priority: 3
Sensitivity: 
X-Mailer: CTI CallCourier
Content-Type: multipart/mixed; boundary="--_=0046d709cb.BOUNDARY.6b54.790f.4914.599c"
Content-Transfer-Encoding: 7bit
Content-Description: CTI rfc822 library generated multi-part message.
Content-ID: <0046d709cb.MESSAGE-ID.5717.16bb.287f.561c>

This is a multi-part message in MIME format.
----_=0046d709cb.BOUNDARY.6b54.790f.4914.599c
Content-Type: audio/wav; name="message.wav"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="message.wav"

[ uuencoded data follows...]

I suppose I could grab the output stream, find the uuencoded part and rip it. Are there any classes or libraries that could help me process the stream directly?

There's nothing obviously wrong with the message headers so I can't explain
why the server has this bug. Still, you should contact the vendor.

This workaround might help:
http://java.sun.com/products/javamail/FAQ.html#imapserverbug