Closed
Bug 15241
Opened 25 years ago
Closed 24 years ago
Method to reduce disk space requirements for build
Categories
(SeaMonkey :: Build Config, enhancement, P3)
Tracking
(Not tracked)
RESOLVED
WONTFIX
Future
People
(Reporter: jim_nance, Assigned: cls)
References
Details
Attachments
(5 files)
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
application/x-gzip
|
Details | |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
patch
|
Details | Diff | Splinter Review |
The make utility is able to keep archive libraries up to date
w/o requiring the .o files to be kept around. The attached
patch demonstrates this and cuts ~150M from the disk space
requirements for building mozilla. The patch is proof of
concept quality. There may be issues that need to be
solved/discovered before it goes into the main code.
Ok, after some testing, here are the main problems:
It bombs severely on parallel builds. Like the GNU make info page warns, it cal
ls ar for the archive concurrently which causes the archive to be missing some s
ymbols. When you remove a file from the build list, you must remove the ar
chive and rebuild it from scratch. If you build a file using some method other
than the implicit make rule, the .o will remain instead of being removed.
I upgraded to GNU make 3.78.1 (which has some -jN fixes) but it didn't help.
Which is a shame because I really like this patch. I'm going to do some more
digging to see how hard it would be to turn off parallel builds on a per
directory/target basis or (as the info page suggests) hack make to serialize all
commands that operate on the same archive.
OK, I think I have an idea on how to improve the way this works. Make
inserts the .o files into the .a file as soon as it generates the .o,
but it does not delete any .o files until all the necessary .o files
have been inserted into the archive. If we change the rule for adding
a single .o file into an archive so that it does nothing, and then add
all the .o files in at the same time it will be both faster and parallel
safe. I dont have a patch for mozilla yet, but I did create an example
Makefile that does this. It is attached.
I attached the file. Its a gzipped tar file, but had trouble telling
bugzilla this.
Here is a new patch for config/rules.mk. It should be faster and hold up
against parallel builds.
Found some problems with the improved method. It does not work well
when source files change. The generated .o files do not get inserted
into the archive. Investigating.
Reporter | ||
Comment 10•25 years ago
|
||
OK, I think this has all the features we wanted, and it seems to work too,
which is always a plus :-) We should now be able to:
1) Run parallel makes
2) Run as fast as the old method
3) Remove .o files from the archive which have been removed from the Makefile
Patch attached
Reporter | ||
Comment 11•25 years ago
|
||
Assignee | ||
Comment 12•25 years ago
|
||
Just one little tweak needed to fix the problem I found. If you removed a file
from the filelist but did not touch any other file, it would not be removed from
the library.
Change
$(LIBRARY): $(LIBRARY)($(OBJS) $(LOBJS)) $(LIBRARY:%.a=.%.timestamp)
to
$(LIBRARY): $(LIBRARY)($(OBJS) $(LOBJS)) $(LIBRARY:%.a=.%.timestamp) Makefile
Makefile.in
And I think we're set. A time comparison showed almost no performance hit.
cls@amadeus:cls> tail -2 test-with-03
2566.90user 307.36system 55:31.03elapsed 86%CPU (0avgtext+0avgdata
0maxresident)k
0inputs+0outputs (4270596major+2851241minor)pagefaults 611swaps
cls@amadeus:cls> tail -2 test-without-03
2582.87user 308.16system 54:56.31elapsed 87%CPU (0avgtext+0avgdata
0maxresident)k
0inputs+0outputs (4066718major+2814688minor)pagefaults 0swaps
cls@amadeus:cls>
Reporter | ||
Comment 13•25 years ago
|
||
I tested the patch on Solaris. It works there too, and cuts 200M from the
build :-)
Status: NEW → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
Assignee | ||
Comment 14•25 years ago
|
||
Dependencies do not work. After an afternoon of unsuccessful attempts of making
dependencies work w/o breaking parallel builds, I'm making the feature a build
option and reopening this bug.
Ramiro, do you have any further details on the problem that you mentioned on
irc?
<ramiro> c++: installation problem, cannot exec `as': Connection refused
<ramiro> make[4]: *** [nsHTMLSpanElement.o] Error 1
<ramiro> c++: installation problem, cannot exec `as': Connection refused
<ramiro> make[4]: *** [nsHTMLSelectElement.o] Error 1
<ramiro> c++: installation problem, cannot exec `as': Connection refused
<ramiro> make[4]: *** [nsHTMLStyleElement.o] Error 1
Comment 15•25 years ago
|
||
Clearing Fixed resolution due to reopen.
Updated•25 years ago
|
Target Milestone: M13 → M14
Comment 16•25 years ago
|
||
moving out to m14. better to do this at the start of milestones.
Comment 17•25 years ago
|
||
mass re-assign of all bugs where i was listed as the qa contact
QA Contact: cyeh → chofmann
Assignee | ||
Comment 18•24 years ago
|
||
I don't think this is going to work for us. Researching another solution.
Marking WONTFIX.
Status: REOPENED → RESOLVED
Closed: 25 years ago → 24 years ago
Resolution: --- → WONTFIX
Updated•20 years ago
|
Product: Browser → Seamonkey
You need to log in
before you can comment on or make changes to this bug.
Description
•