5_19 builds, all 3 platforms are not displaying messages when the message subject in the thread pane is clicked on. Linux and Mac are crashing. Windows is just not displaying the message. (Peter or Esther may add detail about Mac & Win behavior.) On Linux, I crashed when selecting my randomly-selected first message three times. One time I was able to select two messages, they displayed fine, and then I crashed when I clicked on the third message. Stack trace follows...this was from when I crashed on the third message. Stack from crashing on the first message (randomly selected) was the same except it only got to in _init (). Program received signal SIGSEGV, Segmentation fault. 0x40550e63 in memcpy (dstpp=0x65004d, srcpp=0x818c14d, len=5) at ../sysdeps/gene ric/memcpy.c:57 ../sysdeps/generic/memcpy.c:57: No such file or directory. (gdb) where #0 0x40550e63 in memcpy (dstpp=0x65004d, srcpp=0x818c14d, len=5) at ../sysdeps/ generic/memcpy.c:57 #1 0x40cc5ac4 in MimeRichtextConvert () #2 0x40cc3058 in MakeAbsoluteURL () #3 0x40cc60a9 in MimeRichtextConvert () #4 0x40cb60ba in nsMimeConverter::Release () #5 0x401edd37 in NET_PlainTextConverter () #6 0x4016d540 in _init () #7 0x4016db6a in _init () #8 0x40206f3f in NET_ProcessNet () #9 0x4020c563 in NET_PollSockets () #10 0x40225561 in nsNetlibService::NetPollSocketsCallback () #11 0x40144cfa in TimerImpl::FireTimeout () #12 0x40145060 in nsTimerExpired () #13 0x80e6b6b in g_main_iteration () #14 0x80e60f0 in g_list_length () #15 0x80e656b in g_list_length () #16 0x80e6685 in g_main_iteration () #17 0x8084def in gtk_main () #18 0x4009281b in nsAppShell::Run () #19 0x40018d86 in nsAppShellService::Run () #20 0x80512d3 in main ()

Correction on Mac behavior. On today Mac May 19 Seamonkey (1999051909), you do not crash when you select the message. It behaves like windows where it displays at most the header of the message. It fails to displays the message body.

Actually, on Windows, you keep losing system resources.

Peter sees this problem on Mac on an html and a plain text message. Stacey sees this problem on only html messages on Linux. Plain text msgs sent from 4.6 are fine.

For a while, I thought this was Ok with plain text messages on Linux, but it appears that message selection just gets lucky sometimes. After a bit more playing around, it appears that I'm still crashing the vast majority of the time, with both html and plain text msgs.

M6

Ughhh, 6 hours of debugging later and we finally found it. We ended up just backing out changes from yesterday. Scott Putterman backed out some changes by ftang which eventually did the trick. Frank, your checkins into nsMetaCharsetObserver.cpp and nsHTMLDocument.cpp broke message display pretty badly last night. On windows we infinetly attempt to reload the message body into the message frame. We end up crashing on Linux and Mac due to this bug. I don't know if you are still in or not tonight, but we need this fixed for the QA builds tomorrow morning. If you aren't around, I'll back out the changes tonight for these two files and you can take a better look at this tomorrow. Thanks to sspitzer, rhp, ducarroz & putterman for spending their afternoon with me trying to track this nasty one down.

The file I attached isn't working right with apprunner, but if you browse to it with 4.x and save the document to TempMessage.eml then you can load that file into apprunner.

Frank, I'm going to take it that you aren't in tonight. I'm going to back out your changes so mailnews will be able to run in tomorrows builds.

the changes have been backed out.

frank, when you figure out the problem, please explain in the bug report. We were all really stumped today. something to note: make sure you verify that loading rfc822 message works from inside apprunner. rich showed us earlier that it worked fine in viewer, but in apprunner we saw the problems.

I've exchanged some messages with mscott on this. Just FYI, let me insert relevant parts of my comments into this bug: I've been using 5/19 Win32 build for last 30 minutes to look at some 144 messages and so far only a few messages get into an infinite loop and those are the messages I sent out using recent 5.0 Messenger. It seems to me that kostello, ducarroz and nhotta discussed a related issue yesterday in connection with Bug 6675. Recently (in the last few days), Mail started putting in a meta charset tag that it gets from Ender when composing a new message. It's only these messages which have the meta tags that seem to be failing with Frank's new Meta Charset checkin. This seems to me to be a collision of these 2 recent changes. Prior to the arrival of the meta charset tag into a newly composed mail, we were already paying attention to the charset parameter in Content-Type header and so something in the new code didn't agree with that, I guess. In 4.x, we don't put a meta charset tag into a Mail composer generated HTML messages. So Naoki asked whether or not it was necessary to insert a meta tag into the main body. I also question if it is really necessary (though IE4/5 does it.) So, it seems to me another option is not to insert the meta tag into new messages. In any case, as an experiment, I removed the meta tag lines from the messages which caused the hang problem for me above, and the problem stopped.

Let me add a few things we probably should be concerned about in fixing this bug. (Maybe these will be taken care of anyway but to be sure, let me verbalize them below.) As I understand it, we convert all msgs from a native charset ( based on the charset parameter in the Content-Type header)to UTF-8. They will be in HTML format and will bear an UTF-8 Meta tag and then we reply on Meta-charset handling code in the layout to automatically display it, i.e. no menu handling is required. What if the original message contained its own meta charset tag like "iso-8859-1"? The 2 charset tags don't match and will be in conflict unless we somehow ignore the document's charset tag. 1. Issue 1: If charset honoring is in effect, we probably need to let the charset parameter in Content-Type line predominate over a document charset tag and convert to UTF-8 accordingly and also ignore an embedded charset tag in display. 2. Issue 2: We are now able to turn off charset honoring via an insertion of a prefs50.js line. (There is no UI yet for this, and charset honoring is the default if this line does not exist.) In such a case, we face a choice: Should we let the user's encoding menu predominate or the document embedded charset tag? Probaly the encoding menu selection. Actually this might also be related to an issue of a general charset override we need to implement in mail display. I don't think we have a nailed-down spec for this yet, however. Issue 3: How about attachments? and maybe there are other issues to think about? Added nhotta to the cc line.

I guess I should have said that the layout uses "UCS-2" rather than "UTF-8 -- please substitute "UCS-2" for "UTF-8" above. In any case, I'm hoping that rhp and nhotta addressed these issues already.

I will look at it now

Naoki update the tree, and them back out the back out of my file in his local tree. But now we cannot recreate this problem on Window. Can someone have a build which can recreat the problem ?

Have the fixed, already verify on Mac and Window. sspitzer is verifing the UNIX now

mozilla/intl/chardet/src/nsMetaCharsetObserver.cpp at 1.6 mozilla/layout/html/document/src/nsHTMLDocument.cpp at 3.115 + franks patch no other differences in my tree. still crashes for me on Linux. Here's the stack trace: #0 0x4080fe18 in memcpy (dstpp=0x0, srcpp=0x80b9e47, len=45) at ../sysdeps/generic/memcpy.c:51 #1 0x400939f3 in nsCRT::memcpy (aDest=0x0, aSrc=0x80b9e47, aCount=45) at nsCRT.h:81 #2 0x411ab63b in mime_LineBuffer (net_buffer=0x80b9e47 "This is a multi-part message in MIME format.\n", '-' <repeats 14 times>, "90894A908EF7655CA0F23E98\nContent-Type: multipart/related; boundary=\"", '-' <repeats 12 times>, "10C1211A25B32D92B6ECF47E\"\n\n\n", '-' <repeats 14 times>, "10C1211A25B32D92B6E"..., net_buffer_size=7505, bufferP=0x82cede8, buffer_sizeP=0x82cedf0, buffer_fpP=0x82cedf8, convert_newlines_p=1, per_line_fn=0x411a2fcc <MimeMessage_parse_line(char *, int, MimeObject *)>, closure=0x82cedc0) at mimebuf.cpp:220 #3 0x411a6f55 in MimeObject_parse_buffer (buffer=0x80b9b98 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., size=8192, obj=0x82cedc0) at mimeobj.cpp:218 #4 0x411abfca in mime_display_stream_write (stream=0x82cee20, buf=0x80b9b98 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., size=8192) at mimemoz2.cpp:282 #5 0x41191e3f in MimePluginInstance::Write (this=0x83135d0, aBuf=0x80b9b98 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., aCount=8192, aWriteCount=0xbffff408) at plugin_inst.cpp:379 #6 0x40256d31 in plugin_stream_write (stream=0x82cee68, str=0x80b9b98 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., len=8192) at cvplugin.cpp:68 #7 0x401c1bae in net_read_file_chunk (cur_entry=0x829dd58) at mkfile.c:956 #8 0x401c2549 in net_ProcessFile (cur_entry=0x829dd58) at mkfile.c:1327 #9 0x40276f17 in NET_ProcessNet (ready_fd=0x0, fd_type=1) at mkgeturl.c:3355 #10 0x4027edf9 in NET_PollSockets () at mkselect.c:298 #11 0x402a124a in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libnetlib.so #12 0x4018bde9 in TimerImpl::FireTimeout (this=0x84bd918) at nsTimer.cpp:73 #13 0x4018c2d2 in nsTimerExpired (aCallData=0x84bd918) at nsTimer.cpp:189 #14 0x4066dc11 in ?? () from /usr/lib/libglib-1.2.so.0 #15 0x4066cdd2 in ?? () from /usr/lib/libglib-1.2.so.0 #16 0x4066d3bb in ?? () from /usr/lib/libglib-1.2.so.0 #17 0x4066d571 in ?? () from /usr/lib/libglib-1.2.so.0 #18 0x4059415b in ?? () from /usr/lib/libgtk-1.2.so.0 #19 0x400c1bdd in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libwidgetgtk.so #20 0x40020f9d in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libnsappshell.so #21 0x804bbb8 in main (argc=2, argv=0xbffffa24) at nsAppRunner.cpp:482

since it works on mac and windows, I'm going to double check everything. I'll report back in as soon as possible.

yes, I still think it is happening on linux. Repository revision: 3.115 mozilla/layout/html/document/src/nsHTMLDocument.cpp Repository revision: 1.6 mozilla/intl/chardet/src/nsMetaCharsetObserver.cpp plus franks patch. #0 0x411a57b5 in MimeMessage_close_headers (obj=0x82cebf8) at mimemsg.cpp:381 #1 0x411a536d in MimeMessage_parse_line (line=0x8314100 "\nontent-Type: multipart/related; boundary=\"", '-' <repeats 12 times>, "10C1211A25B32D92B6ECF47E\"\n", ', length=1, obj=0x82cebf8) at mimemsg.cpp:211 #2 0x411ad439 in convert_and_send_buffer (buf=0x8314100 "\nontent-Type: multipart/related; boundary=\"", '-' <repeats 12 times>, "10C1211A25B32D92B6ECF47E\"\n", ', length=1, convert_newlines_p=1, per_line_fn=0x411a4fcc <MimeMessage_parse_line(char *, int, MimeObject *)>, closure=0x82cebf8) at mimebuf.cpp:148 #3 0x411ad679 in mime_LineBuffer (net_buffer=0x80bb03c "\n\n", '-' <repeats 14 times>, "10C1211A25B32D92B6ECF47E\nContent-Type: text/html; charset=us-ascii\nContent-Transfer-Encoding: 7bit\n\n<HTML>\n<BODY BACKGROUND=\"cid:part1.33833388.2ECC9355@netscape.com\">\n\n<CENTER><TABLE "..., net_buffer_size=7340, bufferP=0x82cec20, buffer_sizeP=0x82cec28, buffer_fpP=0x82cec30, convert_newlines_p=1, per_line_fn=0x411a4fcc <MimeMessage_parse_line(char *, int, MimeObject *)>, closure=0x82cebf8) at mimebuf.cpp:235 #4 0x411a8f55 in MimeObject_parse_buffer (buffer=0x80bace8 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., size=8192, obj=0x82cebf8) at mimeobj.cpp:218 #5 0x411adfca in mime_display_stream_write (stream=0x82cec58, buf=0x80bace8 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., size=8192) at mimemoz2.cpp:282 #6 0x41193e3f in MimePluginInstance::Write (this=0x8492a10, aBuf=0x80bace8 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., aCount=8192, aWriteCount=0xbffff408) at plugin_inst.cpp:379 #7 0x40256d31 in plugin_stream_write (stream=0x82ceca0, str=0x80bace8 "From - Mon Jun 02 12:00:00 1997\nX-Mozilla-Status: 00011\nReturn-Path: <info@netscape.com>\nReceived: from info.mcom.com ([205.217.228.199]) by tintin.netscape.com\n (Netscape Messaging Server 3."..., len=8192) at cvplugin.cpp:68 #8 0x401c1bae in net_read_file_chunk (cur_entry=0x8313bc0) at mkfile.c:956 #9 0x401c2549 in net_ProcessFile (cur_entry=0x8313bc0) at mkfile.c:1327 #10 0x40276f17 in NET_ProcessNet (ready_fd=0x0, fd_type=1) at mkgeturl.c:3355 #11 0x4027edf9 in NET_PollSockets () at mkselect.c:298 #12 0x402a124a in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libnetlib.so #13 0x4018bde9 in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libgmbasegtk.so #14 0x4018c2d2 in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libgmbasegtk.so #15 0x4066dc11 in ?? () from /usr/lib/libglib-1.2.so.0 #16 0x4066cdd2 in ?? () from /usr/lib/libglib-1.2.so.0 #17 0x4066d3bb in ?? () from /usr/lib/libglib-1.2.so.0 #18 0x4066d571 in ?? () from /usr/lib/libglib-1.2.so.0 #19 0x4059415b in ?? () from /usr/lib/libgtk-1.2.so.0 #20 0x400c1bdd in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libwidgetgtk.so #21 0x40020f5d in ?? () from /builds/sspitzer/MOZILLA/05.18.1999/04.30/mozilla/dist/bin/libnsappshell.so #22 0x804bbb8 in main (argc=2, argv=0xbffffa24) at nsAppRunner.cpp:482 (gdb)

sspitzer, could you do me a favor. set a break point at nsHTMLDocument.cpp line mParser->SetDocumentCharset( charset, charsetSource); and see do they get called repeatly when you open one message. It should be break only once or twice (twice is reasonable) for viewing one message. If it happen multiple time, could you tell me what are the value of charset (the string it point to) and charsetSource (the integer value)

frank, I don't think we're going to be able to fix this over the phone tonight. it's very reproducable...it makes sense to debug it tomorrow.

Okay, I think I found the problem. But I don't think anyone is going to like it ands it is going to be a bear to fix. The problem currently only occurrs on Linux because netlib runs in the same thread as the app. The bug DOES occurr on windows & mac as well but we don't see it because there is a race ccondition (due to netlib running in a separate thread) that conviently lets it work. This explanation will be long, but hopefully, I'll be able to convey the problem. Here's the short simplified summary: we have a mime converter which just converted a message body into html. It writes the html into a stream listener which is the raptor document loader. At this point we are about 10 functions deep on the call stack inside if mime & the plugin converter. Since there is only one thread on linux, writing the data into the document loader immediately causes raptor to start processing data. This causes Frank's new code to get called which says we need to reload. So raptor interrupts the file we are tryoing to display. This then interrupts the current mime converter saying interrupt. The mime converter & emitter are then all released & destroyed. But wait! There are all these functions still on the call stack on objects which have just been released. Now raptor is done interrupting all the streams associated with the window being reloaded so we start to move back down the stack trace and start jumping into code that has been released because of the reload. Note: the same problem occurrs on windows except because netlib is in a separate thread, there is a gap btw the time mime writes the data & when layout actually lays it out. That race condition allows us to back up the stack so we don't run into the problem. Ugghhh, so this looks like an architecture problem between callingg reloads from a document loader & objects writing data into the document loader. I think this is going t be very hard for all of us to fix. One short term solution is to move enough of Frank's charset detection code into mime such that we'll never have to issue a reload when displaying the message. This would have the added benefit of not seeing the black window & then the reload of mail messages that you see now on windows. I think this problem has been so hard to track down because by the time you crash, your stack trace is garbage & pointing off to different functions that don't reflect reality.

three cheers for mscott for spending days tracking down this bug!

Well, I've tried to modify the plugin instance manger & the mime converter several times to try to fix this problem. I can get us out of trouble for the converter but the problem still exists for the core netlib calls which are still on the stack (where core in this case is mkhttp, mkfile, etc.) Basically, I rewrote the contract for abort with the plugin manager in network such that abort only closes....the plugin objects aren't necessarily released. However, the crash will now just be pushed down into the network protocol being run (http, file, etc.) because the I18N reload causes the doc loader to call interrupt. When a url is interrupted, the protocol instance is always destroyed. With the I18N change, it is making us re-entrant & that code just wasn't designed for it. In short, I don't believe I can fix it w/o serious effort invested in changing mkfile, mkhttp, etc. And this is high risk. Basically, I would need to be modifying the definition of how we interrupt urls such that it doesn't destroy the protocol connection for each old-netlib world protocol. This is not easy. The short: there is no "quick" or easy design solution to fixing the re-entrancy problems introduced by Frank's code to call reload from within the parser. I'll continue to look at the problem for the rest of today but I am not optimistic =(.

Seth mentioned one possibility to me which I thought I'd bring up here which could fix the re-entrancy problem for mail but not for other stream converters.We could basically "rig" it such that Frank's code never needs to trigger a reload when displaing messages. We would copy the logic he is using to determine if a reload is necessary into the mime converter. There, we could make sure that his conditions are always met (I assume the conditions are charset related) b4 writing to the parser. This would fix the problem for mail. But it would be a fragile fix as we'd have the same problem if anyone else started calling reload from w/in the parser. In addition, other pluggable stream converters will still have the re-entrancy problem. And we realy, aren't fixing the problem, we're just trying to prevent the case that causes re-entrancy from happening for mail.... I wish I was able to come up with a real solution this weekend but it doesn't look like I am.

mscott: could you kindly point out which object on the stack have been destroed, and by what call ? Also, in my funciton, I have to do two thing 1. StopDocumentLoad 2. ReloadDocument I wonder is that 1 or 2 destory the object on the stack, or the combination of these two. Also, is this cause by some code pattern below? nsIObject *obj = this->GetMeSomeObejct(); FunctionCouldReEnt(); FunctionWhichCrash(obj); If we do ref count correctly, we should do a obj->AddRef(); after GetMeSomeObject(); and a obj->Rlease() after FunctionWhichCrash(obj); (Assume does not do that for you.). I will try to put some sleep (several second) code in the nsMetaCharsetObserver , NotifyWebShell method on window/mac to force the reproduction on Windown. Hopefully this could help me debug it easier....

Frank, you might see function calls like the following: mkfile: net_read_next_chunk cnvtsplugin: plugin_write plugin_instance: Mime code to convert data to html. *old jwz c functions attached to data objects* then get called mimeEmitter: emits the html code to the doc loader doc loader: processes data, eventually falls into I18N code which triggers reload / stop mkfile: net_InterruptFile --> url gets interrupted. active entry for the file then goes away. as part of freeing the active entry state, we interrupt the plugin by calling: cnvtsplugin: plugin_abort which in turn calls into mime: plugin_inst.cpp MimePluginInstance::Abort. I can fix it such that the mime plugin code is safe for reentrancy & have done this on my linux build this weekend. However we will still crash because the mkfile protocol instance for the active entry is still on the stack but it is getting a re-entrant call to file interrupt which by design destroys the current protocol instance. I don't believe it is a simple matter of ref counting objects because the old networking code in mkfile knows nothing about ref counted objects. According to the networking model that old cruft is built on, when you are interupted, the protocol instance is torn down. This is why ref counting will not solve this problem without redesigning interrupt behavior for the old protocols: mkfile, mkhttp, etc . Ref counting & some other tricks I tried only push the problem down into core netlib but it still persists there.

ftang use nhotta's account We should try a hacky fix which temp put some hard wired code in nsParser.... nsresult nsParser::Parse(nsIURL* aURL,nsIStreamObserver* aListener,PRBool aVerifyEnabled, void* aKey) { NS_PRECONDITION(0!=aURL,kNullURL); nsresult result=kBadURL; mDTDVerification=aVerifyEnabled; if(aURL) { const char* spec; nsresult rv = aURL->GetSpec(&spec); if (rv != NS_OK) return rv; nsAutoString theName(spec); // Temp hack to prevent reload on mail message if(-1 != theName.Find("tempMessage.eml")) { nsAutoString utf8("UTF-8"); SetDocumentCharset(utf8, kCharsetFromPreviousLoading); }

will check into branch and tip tonight

The following file have been check in files in tip: mozilla/intl/chardet/src/nsMetaCharsetObserver.cpp 1.8 mozilla/layout/html/document/src/nsHTMLDocument.cpp 3.117 mozilla/htmlparser/src/nsParser.cpp 3.95 files in branch: mozilla/intl/chardet/src/nsMetaCharsetObserver.cpp 1.7.4.1 mozilla/layout/html/document/src/nsHTMLDocument.cpp 3.116.4.1 mozilla/htmlparser/src/nsParser.cpp 3.94.4.1 Move this to M7 and reassing this to nhotta so we will rememeber to move the code out of nsParser

Adding warren and rpotts. Is Linux truly not running netlib in a separate thread? Or is it just scheduling the netlib thread less favorably? /be

Yes netlib is truly not running in a separate thread on linux. Spence was working on getting this working b4 he left but didn't.

marking this fixed. so the hack will get some testing. open a new bug for additional changes.

Well, the original bug in this bug report (crash displaying messages) has been verified since 5/19 when mscott backed out of ftang's changes. So, far no problem displaying messages in English. I'm going to change the QA assigned to momoi to check for international messages. mscott, momoi, others: if there is a new bug against netlib, pls file it. Thanks.

There were 2 types of crashes for this bug. 1. On Linux: just about any messages crashed. 2. On Win and Mac, the crashing messages were those which contained a meta charset tag either in the body or in the attachments. This is what ftang's initial check-in fixed. I take it that Lisa's group's had done tesing on Linux on the 5/25/99 Linux M6 build. If you haven't seen crashes on this build then, the fix for Type 1 crash has been verified -- ftang's temporary fix. For type 2 bugs on Windows, I have been looking at the 5/25/99 M6 build since last night. No messages matching Type 2 description above has caused crashes or infinite loops. Based on these results, I'm making this temporary fix verified as working as intended. Someone please open a new bug as lchiang suggests -- that should be mscott, sspitzer or ftang.