Closed Bug 24944 Opened 25 years ago Closed 25 years ago

Cartman pages could not display non-ASCII

Categories

(Core :: Security, defect, P3)

x86
Windows NT
defect

Tracking

()

VERIFIED FIXED

People

(Reporter: rchen, Assigned: dougt)

Details

(Whiteboard: [PDT+])

Attachments

(3 files)

Cartman pages could not display Japanese. Apparently the character conversion doesn't work right. Please use the following two unicode characters to test on US machine: "\u00e0 \u00e1". They should display as "à á" if the conversion works correctly.
Mark it as beta1. Reassign QA to myself.
Keywords: beta1
QA Contact: junruh → rchen
rchen, please list how this can be reproduced. This is a page that you are going to, or are you localizing cartman?? Mark, is this a known problem with cartman?
Changed summary to say "non-ASCII" instead of "Japanese". In fact, the example provide uses "Latin1" characters. Ray, Please attach your modified .properties file that adds the "à á" and attach a screen shot to show the current results.
Summary: Cartman pages could not display Japanese → Cartman pages could not display non-ASCII
This bug relates to localizing the Cartman UI, NOT pages you visit and then use the Security Advisor to review. To repro, 1. Choose a prominent UI string to alter, let's call it 'foo'. The source you'll need to change is in the .properties files (psm_ui.properties or psm_text.properties, depending on what build you're using). 2. Add the two Unicode characters rchen suggests (\u00e0 \u00e1) before the UI string you're altering. This changes the string to 'à áfoo'. 3. Restart Communicator/Seamonkey and with it restart Cartman. 4. View the string you've changed. It should appear as 'à áfoo'. But it doesn't due to a problem with how the Unicode character is handled, converted, displayed or something.
Attached image psm screen shot (deleted) —
I have attached two files: screen shot and psm_text.properties with \u00e0 \u00e1 added at text_fullproductname. This bug is related to the localization of PSM.
assigning to cartman team.
Assignee: dougt → mwelch
Putting on the PDT+ radar for beta1.
Whiteboard: [PDT+]
Reassigning to javi, if he has a bugzilla account available.
Assignee: mwelch → javi
I'm extremely occupied by a project with the DoD right now. I'll come back and look at this later. What we really need is someone who knows how the libNLS libraries work to review PSM code and let us know what we're doing wrong. As far as the PSM team is concerned, our code is doing the right thing and the libNLS are somehow munging up the conversion from Unicode to UTF8.
Javi, thanks for response. If you need I18N engineer's help, please let bobj know. The fix need to be in beta1. Thanks.
The I18N engineers are fully booked for the next 2+ days on M14 bugs. I can offer help after that. Noone in my group has worked with libNLS. The cartman folks (e.g., mwelch) know more about libNLS than we do. But I can have one of my engineers to sit with javi and look at this in a debugger, if that's what he needs. I think the best solution would be to work with someone on hkomatsu's libNLS team...
The screen shot shows the UTF-8 representation of \u00e0 \u00e1 (C3 A0 C3 A1) displayed as iso-8859-1. Resource bundles correctly use UTF-8. So whatever has looked up the data in the resource bundles is interpreting it using the wrong charset.
Have verified that libnls does what it supposed to do and converts the charset correctly. Cartman UI still couldn't display correctly. Would need Javi to bring up the intermediate HTML to verify the data.
Dumb question, but is the charset set correctly in psm_text.properties?
How are we supposed to "correctly set the charset"? Is there a libNLS call we're supposed to make to set the charset for the session? If there is, then we're not doing that. This would be the question we'd need help for from the NLS group.
I finally found out what caused the problem. In psm_ui.properties, most of the pages doesn't setup the charset with nsm_charset. That's why even thought we changed nsm_charset and UI stings to Japanese they are not displayed correctly. Need to fix it for Beta1.
I grepped psm_ui.properties and found 106 occurences of "<head>", but only 3 occurences of nsm_charset. Assuming that all generated HTML pages include <head> sections, then you probably could remove the 3 occurences of nsm_charset and do a global replacement to change all <head> to <head>\r\n<meta http-equiv="Content-Type" content="text/html; charset={nsm_charset}>
Reassigning the bug to myself.
Assignee: javi → sjlee
Attached file psm_ui.properties (deleted) —
Reassigning it to dougt per conversation with him. Doug, I have attached the new psm_ui.properties file (the last one and that one only). There are tons of "\r\n"s but those are part of the file (nothing to worry about).
Assignee: sjlee → dougt
great. handed this to ssu. It should be in the next build. Sjlee, please make sure that this goes into the next version of cartman. marking as fixed.
Status: NEW → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
It's already checked into the PSM tree.
Bulk moving all Browser Security bugs to new Security: General component. The previous Security component for Browser will be deleted.
Component: Security → Security: General
Verified.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: