The view source function in apprunner tries to reload the current page, to display in the source window. I say _try_ because the page I was trying to check was indeed a .cgi, and mozilla didn't repost the form data, so I got the source of an error message, instead of the source of the page I was using. Mozilla version: M5

Bill, do you see this happening?

Can you provide a URL for a page that exhibits the problem? I don't doubt that this is broken, but I don't know a page off the top of my head that exposes it. Thanks.

Move this to M7 while we wait for a test URL ...

Still need a URL to reproduce this ...

Sorry for the delay. Please forget my remarks about forms, they are not valid. But the problem stated in the summary still exists, here is a simple test: http://www.inf.bme.hu/~hyp-x/test.cgi The cgi is a simple counter. Reproduce: 1. Go to the above url with seamonkey 2. It displays: "The counter is: N" (where N will be some number) 3. View|Page source 4. The number in the source doesn't match the number shown in the browser window. (it is N+1). Expected result: The view|page source function should display the source of the document in the browser window.

Thanks, I can now clearly see the problem. I think this has to be fixed inside nsWebShell (or lower); it needs to request that the source be loaded from cache (if possible). I don't think this can (currently) be controlled on the calling side of LoadURL(). I'm cc:ing Nisheeth to get his input.

I'm ccing Rick Gessner who implemented the view-source functionality. Either we, the consumer of netlib, are not telling it to load from the cache, or netlib is exhibiting buggy behavior. Either way, this issue should probably get visited when Necko lands: there is no point in potentially repeating this bug fixing effort when Necko lands.

*** Bug 8998 has been marked as a duplicate of this bug. ***

Deferring to M9 per Nisheeth's comment and the fact that M9's when Necko lands.

This is still the case with necko. I might forgot to say: this is happening with back/forward too. Should I file a separate bug for that? I heard necko doesn't have cache yet, but it seems to refer to _disk_ cache, because it caches images nicely. So the functionality to fix this might already be there.

We'll have to tackle this after Necko landing has subsided (M10).

*** Bug 10305 has been marked as a duplicate of this bug. ***

Still waiting for cache. I'm making this dependent on bug #8305 which is close to a generic "no cache" bug as I could find.

*** Bug 14781 has been marked as a duplicate of this bug. ***

The following is from 14781. Basically, the important thing is that even for uncachable documents like results of POST forms, we should _still_ not be refetching the document. TO REPRODUCE: 1. Go to http://www.bath.ac.uk/%7Epy8ieh/cgi/formtester.pl 2. Click on the check box in the "Simple POST Form". 3. Submit the POST form. 4. Notice it says "Results of Last Submission" at the top of the page. 5. Choose View Source (in Viewer, File|ViewSource). 6. Look for the text "Results of Last Submission". ACTUAL RESULTS: The source shown in ViewSource is NOT that used to render the document. This can be clearly seen by the fact that the markup in ViewSource could not generate the document seen! The entire "Results" section at the top of the document is missing, because Mozilla went and refetched the page. EXPECTED RESULTS: Mozilla just uses its (in-memory) cached copy of the document for the ViewSource window. No network connection should be made for ViewSource. NOTES: IE does this fine. Also note that making sure that the document is refetched using exactly the same call IS NOT a suitable fix, since: a. If the document is the result of a POST submission, refetching it could trigger things like a bank transfer! b. In any case, dynamically created documents may be different on each fetch regardless of whether it is a GET or a POST -- think random quote pages. c. If the page takes a long time to load (e.g., it is created by a slow script) then you don't want to wait the time again! We must use a local copy of the page for View Source, even if the document is marked NOCACHE!

My take is that we need to load the URL from cache on "view source." Ultimately the web shell will need to support a flavor of its LoadURL that allows this to be specified. Once that's in place, we can wire whatever's required into the higher-level interfaces (nsBrowserInstance and navigator.js). I've added travis to the cc: list.

Bill is right: viewsource should be getting the content from the cache. My question is this: (assuming we have a cache) why wouldn't the networking layer look in the cache *before* proceeding to fetch the document from the net?

Rick: Some documents should 'never' be fetched from the cache, for instance the result of POSTs and documents marked 'nocache' via HTTP headers. Viewsource and print are cases where they should be fetched from the cache regardless.

Which seems as a good a reason as any to say the cache is not the place to get if from. Is the source dumped once it is parsed? Perhaps it should be kept.

Moving to M13. I don't think we HAVE to fix this for M12, do we?

*** Bug 19522 has been marked as a duplicate of this bug. ***

Not required for beta 1.

Composer requires that View Source work with the current DOM content. Looking at comments here, it is clear that refetching content from original URL won't work for Composer, and there is still discussion about whether to get contents from the cache. Either way, we need to have the editor save its contents to a cache or some other memory stream, and that should be parsed for view source. This might be the best way to implement view source for the browser as well, eliminating the cache dependency.

Charlie -- not quite. Viewsource in the browser wants the version from the file cache, and in a form that is unaffected by js.

*** Bug 22626 has been marked as a duplicate of this bug. ***

*** Bug 29390 has been marked as a duplicate of this bug. ***

It would be a nice feature if viewsource can switch between a version affected by js (like NS4.x does now) and another version unaffected by js (like IE). This could be very useful to debug pages generated by js. Any comments ? :)

Additional reproductible test case for this bug, on M14/Linux: visit <http://globaldsl.myway.com/> (which is a really ugly-looking page right now, btw), view source (works), try to visit <http://www.anadtech.com/>, which can't be found (read in a /. thread that it crashes M14, kind of odd that it doesn't exist), then right-click the document and hit view source; the view source window comes up again saying that http://www.anadtech.com/ could not be found. HTH.

*** Bug 30618 has been marked as a duplicate of this bug. ***

*** Bug 22935 has been marked as a duplicate of this bug. ***

This bug appears to still be in M-14/win32/talkback While try to view the script for site. site 1 Url http://cgi3.ebay.com/aw-cgi/eBayISAPI.dll?SignIn this URL also produces a form post bug.

*** Bug 31113 has been marked as a duplicate of this bug. ***

Move to M16 for now ...

spam: moving qa contact on some bugs from paulmac to sairuh@netscape.com

*** Bug 35441 has been marked as a duplicate of this bug. ***

Move to M21 target milestone.

Please see my RFE, bug 40867, for a discussion of cache for view source AND save as, and links to other relevant bugs.

*** Bug 45457 has been marked as a duplicate of this bug. ***

*** Bug 47916 has been marked as a duplicate of this bug. ***

This is a problem on mac and linux as well. It's a pretty major problem if data is reposted that the user doesn't desire to be. Ian's comments describe the ramifications of this problem very well. I'm upping the severity (somebody kick me if I'm out of line) and nominating for beta3. Also note that bug 17889, which (like this bug) also depends on bug 40867, is nsbeta3+. Very strange that 40867 is nsbeta3-, adding comments there.

*** Bug 39957 has been marked as a duplicate of this bug. ***

This isn't even just a view-source bug. This affects file saving too (which is bad, because many important transactions are followed by webpages which instruct the user to "save this page for your records"). That problem was noted in bug 39957 (marked duplicate). In 39957, neeti@netscape.com says: "I think we should be using SelectFileAndTransferLocation(..) instead of SelectFileAndTransferLocationSpec(..) in function savePage in navigator.js." davidm@netscape.com says: "This isn't really a cache bug. The save as peopple and print people should be setting the channel flag not to do any validation which will cause the cache to be used if the data is available ( it could be flushed if you have multiple windows open). I can't actually see where they are loadin the URLs but thats the theory;)" I haven't looked into it, but maybe those comments help? This should be a /very/ strong argument for nsbeta3+'ing bug 40867...

Sorry for the spam, but also nominating for dogfood, since you really don't want to be using this product for anything "important" without this being fixed... Also fixing the summary.

separating this bug (view source reloading page) from bug 39957 (save as reloading page), because they really seem like different bugs to me. if the fixes are similar, dependencies on bug 40867 (hold source for current page in memory) will tie this bug to bug 39957. (should the "major" and "dogfood" stuff stay here, go to bug 39957, or go to bug 40867?)

This bug isn't any different than bug 39957, but if you want them separate, have it your way :) Seriously, though, we already observe that a refetch happens on: - view source (this bug) - save (39957) - print (see matthew thomas's comments in 39957) - send page (also mpt's comments in 39957) and I'm sure there are others. This is all the same bug, and are all symptoms of bug 40867.

Ooh, forgot to mention that changing character set exhibits this behavior as well (bug 17889).

nav triage team: nsbeta3+ The higher priority is bug 39957, but this may affect that. Marking P2.

QA and eng use page source to investigate bugs. Putting on [dogfood+] radar. Please fix ASAP.

*** Bug 50333 has been marked as a duplicate of this bug. ***

bug 39957 now contains a patch which may be of some relevance...

We can't load pages for view-source until nsIWebNavigation::LoadURI supports load flags (so we can say VALIDATE_NEVER, or equivalent). This is being done for bug 50949 (for other, although similar, reasons). Once that blocker is fixed, it is trivial to pass the right flags on the loadURI call in viewsource.js. I'm cleaning up some of the depends-on/blocks cruft.

Is all this valid also for "View Frame Source"?

Yes; the same code handles both cases.

PDT thinks this is a P3

*** Bug 53090 has been marked as a duplicate of this bug. ***

Nav triage team: nsbeta3-

Bug 50949 is fixed today. I just checked. Per Law's statement on 9/5/2000, someone should be able to fix this in short order. Please do. If this doesn't work for RTM, a lot of developers are going to dismiss this browser. BTW, If I open a a blank window with window.open("","","") and then do a with statement and open that document for writing, write some content, then close the document for writing, I can view the text I wrote in that document, but a view source displays only "<html><body></body></html>". That is why this bug is so annoying. You should be able to see the source of the file you are viewing. For dynamic documents, that is the only file that exists. Please fix soon! Jake

I tried the following incantation in viewsource.js, but it didn't work: var loadFlags = Components.interfaces.nsIChannel.VALIDATE_NEVER; webNav.loadURI(window.arguments[0], loadFlags); One thing I noticed, though, is the following dump when trying to view source: Setting content window *** SetDocumentCharset(ISO-8859-1) Error loading URL about:blank: 804b0002 Document http://www.inf.bme.hu/~hyp-x/test.cgi loaded successfully The "error loading about:blank" tells me that we're doing something silly in viewsource.js when creating a new BrowserInstance... law: i'm betting this should be an easy hack for somebody (unlike me) who knows the code. is this quickly fixable? if so, should we try to re-trigger PDT evaluation to get this in?

You can't pass nsIChannel::VALIDATE_NEVER to nsIWebNavigation::LoadURI (at least not directly, I don't think). I'm trying to acertain what value to pass to accomplish VALIDATE_NEVER indirectly. See bug 50949.

Let's fix this in RTM.

nav triage team: [RTM NEED INFO] looking for a patch, P2

Lowering to P3.

Nav Triage Team agrees this can be minus for RTM. Sorry.

Gah...developers of dynamic content will spit on Netscape6...

Not fixing this for RTM??? ARRRGH!! This is a very significant bug. Releasing to manufacturing with this bug is a terrible mistake. If other bugs of this severity are also being skipped, Mozilla will get, and deserve, a terrible reputation. What a tragedy - this is not how open source is supposed to be. Boo! Hiss! Boo! Boooooo!

Excuse the double-attachment; Seamonkey glitch. This simple patch fixes part of the problem. It will get view-source pages loaded from cache whenever they're in the cache. Radha and I worked this out after some debate. It still needs module owner approval and super-review, but otherwise could probably be checked into the Mozilla trunk (even if Netscape can't get it into Netscape 6). Please note that there is still at least one major bug in view-source. You can't view source for pages generated via forms submitted with method=POST. There is no mechanism for passing the form post data along so that it gets used to load the page in the view-source window. I partially described a theoretical fix for that in email to Radha and Adam Lock. Basically, it involves passing the form post data from the nsBrowserInstance in the original browser window and then viewSource.js using that to formulate its request to nsIDocShell::LoadURI (it would have to switch from using nsIWebNavigation::LoadURI). Anyway, I just wanted to leave this information on the historical record before this bug is forgotten.

torrey: open source also is not about disparaging dedicated developers. if you want to boo and hiss, mozilla is open source -- fix the bug yourself. otherwise, you have /no/ right to pass high and mighty judgement, no matter how shameful it is, i agree, that a bug of this severity could slip through the cracks. bill: i'll try your patch out. perhaps rtm will accept it if you have r= and a= and they understand the consequences.

> Mozilla will get, and deserve, a terrible reputation. s/Mozilla/Netscape 6/ (I hope!)

bill: patch looks good -- simple fix, and works for me. r=dr. pdt: reviewed fix in hand, but super-review needed. i reset to the prior status of "need info" because this is negligible risk and very high payoff. i'll respect another rtm- (with my apologies for pushing hard), but i strongly urge reevaluation on the bases of low risk and high importance.

Rescue this because we have a fix.

One comment about Bill's comment on future work about the POST data issue (**nothing to do with the current patch**) - sending the post data again seems the wrong thing. What you really need to do is cache the source of the page, not resend the POST data.

The patch looks fine to me. Not a super-review, but r=adamlock@netscape.com

clearing rtm whiteboard comment so that this can get ++'d soon...

PDT, please approve.

We're having a problem locating the final super-reviewer. PDT, you can conditionally approve this if you want.

a=waterson if you file bugs on the *other* cases where view-source breaks (e.g., it can't load from cache because the web page told us not to cache).

OK, Bill will file the other bugs.

PDT marking [rtm++]

Fix checked in, trunk and branch.

What are the bugs for the remaining problems?

Ben, I can only find bug 55583: "view-source doesn't work for pages generated via forms with method=POST"

Reopening until all problems are covered by other bugs or fixed.

One other bug is for dynamically generated pages. That is, pages opened and then written to like this: mywin = window.open('','',windowprops); with (mywin.document) { open(); write(myheader + mytext + myfooter); close(); } You can see an example of this in this testcase: http://bugzilla.mozilla.org/showattachment.cgi?attach_id=16305 which you can find in bug 53683. Just click on the link which will open a window and write a bunch of repeated text to it then do a view source. All you will see is: <html><body></body></html> Whereas in IE, you can do a viewsource and view that dynamically generated content in the source which is, I think, how it should be. NOTE: Please ignore the fact that the link says "display all links on this page". I just forgot to change that text from the previous more complex testcase. This harkens back to a quote by David Baron: "What you really need to do is cache the source of the page" And that is exactly it. You don't need to go through hoops to do this. All you need to do is pull of a copy of the cached source of the page. Viewsource should be doing this in ALL cases whether a basic HTML page, a page viewed after a form post, or dynamically generated content. The source should be there. Does that make sense? Jake

bug 55583 is the new one I opened vis-a-vis form post data bug 40867 is the one that says we need a way to get at current page source without having to reload; that's the only way to fix the other problem I related to Chris Waterson: the fact that sometimes we'll go back to the server regardless of the VALIDATE_NEVER flag please open a new bug if you don't like the fact that view-source shows the JS script source rather than what it generates; that problem has nothing to do with this one (aside from being related to view-source) Reclosing

Everybody: Are all remaining problems covered now? > please open a new bug if you don't like the fact that view-source shows the JS > script source rather than what it generates For the record: I disagree that JS should be evaluated for View Source. I want to see the "on-the-wire" source.

>> please open a new bug if you don't like the fact that view-source shows the JS >> script source rather than what it generates >For the record: I disagree that JS should be evaluated for View Source. I want >to see the "on-the-wire" source. Maybe I wasn't clear enough on this one. The reason why I think this is a bug is that IE4+ (and likely IE3 even) is able to display result of the following in full view-source: mywin = window.open('','',windowprops); with (mywin.document) { open(); write(myheader + mytext + myfooter); close(); } There is a subtle difference here than what both Ben and Law commented on. Let's say that the variable "myheader", which I am currently writing to this document, contains (among other things...) something like: <script type='text/javascript'>document.write('something...')</script> I would NOT expect the result of this document.write() to show up in the source if you did a view source. I would expect the "on-the-wire" source. The difference is the fact that when I did the window.open() I set it to a variable which I then referenced to open the document for writing and then closed it. The document, in my opinion, was not closed until I intentionally closed it. Therefore, the new window was not done loading until everything was written to it in the "with" statement. So, in fact, I should be able to view the source of everything that was written to the document because it was done before the document was fully loaded whereas if it is done after the fact and I am just manipulating the dom of an already loaded document, then I would only expect to see the javascript that would do the writing, but not the writing itself in the source. Does that make more sense? Either way, I belive this is a valid concern becuase IE behaves exactly as I've described above and know that many developers count on this behavior. Jake

hoju, you can also do document.open(); document.write('overwrite some evil script and this script with the html source for the text') document.close(); not? What's the difference to your example? I suggest you file a new bug for this discussion (but it might get WONTFIX).

vrfy fixed using 2000.10.10.xx-n6 opt comm branch bits on linux, winnt and mac. needs trunk vrf'tion.

Current Mozilla still displays that upon each View Page Source in console: *** SetDocumentCharset(ISO-8859-1) Error loading URL about:blank: 804b0002 Document http://dev.scriptics.com/ loaded successfully I don't know wether that's good or not, but the counter in the test URL http://www.inf.bme.hu/~hyp-x/test.cgi doesn't get propagated on View Source. => verify on Win32

Verified Fixed on trunk builds linux 101708 RedHat 6.2 win32 101704 NT 4 mac 101704 Mac OS9 Setting bug to Verified and removing vtrunk keyword

I was reported that view source sometimes reloads in current nightlies even if there are no "anti cache" headers in the page. I'll try to get the testcase the person used and reopen the page if true. Anybody else still experiencing difficulties?

This bug has <b>not</b> been properly fixed. The current behavior is better, but it's still wrong, because it relies on the cache to find the answer. You can easily convince yourself that this bug still exists by opening TWO windows to the test.cgi URL on this bug. The second window will load the page, get a higher counter number (as it should), then overwrite the cache entry from the first page. If you then view a "View Page Source" from the older window, the displayed source has the counter number from the <b>new</b> window, since that's what's in the cache. The source does NOT show the same number as was displayed in the window. (Potentially, the page might not even be in the cache if it has been pushed out of a full cache by unrelated browsing, no?) <p> A proper fix for this bug depends on bug 40867.

Hmm. So much for using HTML tags for emphasis. :-)

Another example that demonstrates that this bug isn't really fixed is to load the test CGI in a single window, then clear the cache in the Preferences panel. A "View Page Source" command will then reload the page with a new counter number, since it's no longer in the cache.

Deven: you're right that this wasn't fixed for POST pages (which is what this bug was originally about), but that case is now covered by bug 55583.

This wasn't even fixed for the GET method, which was why I added the dependency on 40867. Both examples I gave (viewing the older copy of two windows, and viewing the source of a page loaded before clearing the cache) were GET method bugs, nothing to do with the POST method. Most of the effects of this bug have been masked by forcing it to look in the cache, but this bug isn't really fixed!

I am wary of the difficulty of "fixing" this bug. Essentially, we would have to (potentially) write out the source to every single page one visits. That's not a trivial thing to do. I wonder if we can't achieve an acceptable solution by offering the option to view/save the document content, converted to source form. Perhaps we determine whether the view/save will go back to the server, and if so, put up a dialog that asks the user if they want us to do that, or, to view/save the "page" they're viewing, instead? In any case, I think the options should be addressed by bug 40867. I'm adding a dependency on that one to this bug so we can get from there to here.

If mozilla is to be useful to those of us who develop programs that produce dynamic pages, it must be able to produce the source, verbatim, that was received by the browser from the server to any page that the browser is currently viewing. This set of pages is, in practice, much smaller than the set of all pages that the browser has visited: only those pages that can be accessed using the browser's BACK and FORWARD buttons really need to be saved for source-viewing. In actual use this is just a small chain of pages, in the worst case it is a set of chains, one for each browser window the user has open. I think the cache mechanism should allow verbatim source to be stored for pages in these BACK/FORWARD chains. The verbatim source could be expired once the page leaves the scope of the chain, such as when the user goes back, and clicks on another link.

Damn, law is right. It is potentially every page that might be pulled from the cache. I'm convinced now that the cache architecture is broken. It needs to store the source, and nothing but the source.

Pohl, I posted a long comment under bug 40867 about the cache architecture. The short summary of that comment is this: the LRU cache needed by Necko for normal caching purposes is inappropriate to use for history (back/forward) and functions such as "View Source" and "Save As", because the identical data may not actually BE in the LRU cache. I suggested either saving the data independently of the cache (possibly with the DOM) or making the cache manager independent of Necko, to serve both purposes properly. (See my full comment under bug 40867 for details.) Does this bug have to remain "VERIFIED FIXED"? Can't we reopen it, since it isn't completely fixed, and just lower the severity/priority somewhat?

I just found your comments a moment ago, and was happy to read that someone was giving so much thought to the problem. I also enjoyed Dan Rosen's 2000-08-24 11:37 comments in the same bug. He suggested that the DOM be lossless: that it be able to generate the verbatim source by preserving whitespace, capitalization, comments, and entity substitutions within.

This bug should remain marked as "fixed" because there's a new bug tracking the remaining problems (bug 55583).

Okay, I must point out (for the third time) that bug 55583 does not represent all the remaining problems from this bug. That bug is focused on dynamic content from a POST operation, and I have given two examples of a GET operation failing. (The examples I used were of dynamic content, but they could have been static content that happens to be changed during the lifetime of a browser window.) This problem isn't nearly as bad as it once was, which is why I suggested lowering the priority and/or severity. But is it really appropriate to call it "fixed" when it's only 90-95% fixed?

Perhaps another bug should be opened for the architectural issues?

I think bug 40867 covers the architectural issues well enough. If that bug is fixed, this one gets fixed "for free". I know people are reluctant to reopen old bugs, but this one may have been closed prematurely, no?

Anyone with editbugs capability who feels this bug has been wrongly closed is free to reopen it. I'd rather not reopen it myself, having just caught up on the commentary. /be

I broadened the scope of the successor bug 55583.

You'd have to broaden bug 55583 even more to fully cover the remaining aspects of this bug, since some scenarios (e.g. clearing the cache) can cause view-source to reload the page, which is what this bug was originally about. Since bug 55583 was originally about form POST data, it seems a little confusing to keep expanding it until it encompasses this bug, rather than simply reopening this one. Then again, maybe it doesn't matter if this bug (or that one) is open, since they'd both be fixed by fixing bug 40867, which is the important one...