Closed Bug 4464 Opened 26 years ago Closed 26 years ago

PR_SetThreadPriority should cast enum PRThreadPriority to a signed int before doing comparisions

Categories

(NSPR :: NSPR, defect, P3)

All
FreeBSD
defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: wtc, Assigned: wtc)

Details

This bug is discovered by Jeremy Lea <reg@shale.csir.co.za> when running the 'priotest' test. In PR_SetThreadPriority, we compare the new priority with PR_PRIORITY_FIRST and PR_PRIORITY_LAST to make sure that it is in the legal range of the enum PRThreadPriority. Since all the legal values of the enum PRThreadPriority are 0 or positive numbers, the enum is implemented as an unsigned int under some compilers, such as gcc and egcs. So the value of -1 for the enum is treated as 0xffffffff and is greater than 0, which is absurd. We can force the compiler to do a signed comparison by casting the enum to PRIntn (a synonym for int) before doing the comparisons. PR_SetThreadPriority in the pthreads version (ptthread.c) already has the PRIntn casts. We just need to fix PR_SetThreadPriority in classic NSPR (pruthr.c).
Status: NEW → RESOLVED
Closed: 26 years ago
Resolution: --- → FIXED
To reproduce the bug, run the 'priotest' test with the -d option on FreeBSD 3.1, classic NSPR. You will get: % priotest -d PR_SetThreadPriority(-1) failed Adding the PRIntn casts fixes the problem. This fix is checked into the tip. /cvsroot/mozilla/nsprpub/pr/src/threads/combined/pruthr.c, revision 3.8. Also checked into the internal CVS repository. /m/src/ns/nspr20/pr/src/threads/combined/pruthr.c, revision 2.25.
Jeremy, can you verify my fix please? Then choose "Mark bug as VERIFIED" and commit. Thanks.
Status: RESOLVED → VERIFIED
NSPR now has its own Bugzilla product. Moving this bug to the NSPR product.
You need to log in before you can comment on or make changes to this bug.