Can we run rebase.exe, which is a post-processing build utility which "rebases" a set of dlls so that they can load without having conflicting base addresses. Troy or Phil knows more about this. This could be a big win when loading dlls.

so does rebase.exe come with the standard msft tools? leaf: check in with phil about what's required.

Yes, it's part of the msdev tools. Phil thinks that you can just point it at a bunch of dlls and let it work its magic. I'd look at what 4.5 was doing as a starting point.

I should say that I'm not 100% sure we're not running it already on release builds. I don't think we are, but we could be.

So, i'm trying out rebase.exe today... it tells me i have to specify either the Initial base address to each dll, or give it a file which will tell it what base address to use. I'm not sure i know what values to give...

can someone (bienvenu) lend a hand as to what base address we should be using?

sorry, I have no clue (if I did, I would have volunteered to do it myself). What did 4.x use? I'll poke around a little bit, but someone with a clue (Phil? Troy?) should really provide the answer.

Here are the exciting lines from winfe\mkfiles32\mozilla.mak: REBASE=rebase.exe !if [for %i in (. %PATH%) do @if exist %i\$(REBASE) echo %i\$(REBASE) > rebase.yes] !endif !if exist(rebase.yes) !if [for %i in ($(OUTDIR)\*.dll) do @echo %i >> rebase.lst] !endif !if [for %i in ($(OUTDIR)\java\bin\*.dll) do @echo %i >> rebase.lst] !endif !if [for %i in ($(OUTDIR)\spellchk\*.dll) do @echo %i >> rebase.lst] !endif !endif rebase: !if exist(rebase.lst) $(REBASE) -b 60000000 -R . -G rebase.lst del rebase.lst !endif !if exist(rebase.yes) del rebase.yes !endif

okay, so i went and looked at mkfiles32/mozilla.mak and boned up on the usage of rebase. for it to be most effective, you would want to do this with a group of dll's that load together so that they don't have conflicting base addresses. it's not clear to me if we should do this with each dll or not, and what kind of side effects that will result if we do this on components etc. my suggestion is this: someone should go try this and see if you get a perfomance win. the way you can do this outside of a makefile is to do the following: create a file that has a list of dll's that you want to rebase together. (it's important that the list be complete, so that rebase can juggle all of the dll's together.) put it in a file named "rebase.lst" Then do this: rebase.exe -b 60000000 -R . -G rebase.lst If this results in a speed improvement, great, let's do it. But I'm leary of just applying tools to dll's because they worked under the mozilla classic codebase. for all we know right now, running rebase might slow down the code.

Further questions: in the self-registering component world, is rebase going to do what we think it will? I can rebase all the .dlls that are parallel to apprunner.exe (which are not of the self-registering variety, presumably). I still need someone that can do performance testing.

cyeh wrote: > for all we know right now, running rebase might slow down the code I can't think of a single reason why that would be. Why do you say that? leaf wrote: > in the self-registering component world, is rebase going to do what we > think it will? Yes, I think so. What does self-registration have to do with rebasing? The DLL still has to get loaded by the OS loader, and the loader still has to decide whether to leave the DLL at the address is was loaded at, or relocate it (with performance degradation) someplace else.

lets get some development testing on this early in m8. too late for the m7 train

I hear troy's back, so I thought I'd cc: him on this bug to say "hiya!" /be

so does it save time? and would setting the base address at 6MB's require a minimum footprint of 6MB for the application?

troy: we are waiting to hear whether rebasing is going to really help performance or not, and whether setting the base address at 6MB forces a minimum 6MB footprint.

Yes, rebasing is a big performance win, and no it does change the footprint. Basically, some sort of rebasing is going to happen. Either we do it when we build (or sometime before we ship), or it happens at runtime when the DLL is loaded. The OS loader will detect that the image base address conflicts with a DLL already loaded, and it will relocate it to a different address. That's an expensive operation that happens each and every time the browser is run.

The other thing that it's critical that we do is to "bind" the .exe and .dll files. Unline rebasing (which we can do during the build process), binding happens when the files are installed on the user's machine. Binding is basically a "fixup" of the external functions and doing it during install means that when the DLL is loaded less fixups are needed. It's probably obvious, but worth mentioning anyway, that if a DLL is rebased when loaded, then all DLLs that reference the rebased DLL must also be fixed up again. That means if we don't rebase our DLLs before we ship them, then we can't bind them when installing them That means we lose twice, and the startup time of our app will suffer. Sean Su knows about binding of the DLLs when doing an install, because that's something we do for Communicator

i've added rebase rules to the mozilla win32 build system, so that every build will get rebased at the end of the install phase. I've also added this to the rule that splits symbols in ns (since we should probably strip the symbols before doing the rebase).

Marking Verified.

When I launch the debug version of apprunner on Win32, I see tons of OS loader warning messages that DLLs have been reloated. So I don't think this is working right. Reopening.

Are you removing your component.reg? If you have an old one, you will get this kind of error. If you *are* removing it, note that here and i'll try and figure out what's going wrong... I think this might be related to the fact that the installer isn't binding our dlls right now.

I deleted dist\win32_d.obj\bin\component.reg, and deleted all of the DLLs and EXEs in bin, and then rebuilt from client.mak, and I still get the relocation warnings. Looking at the makefile, it appears to only rebase if MOZILLA_OFFICIAL is defined. Not sure what the purpose of that env var is, but I'll try it out. Seems like we should rebase all the time, though. Otherwise we're just wasting developers' time.

As expected, defining MOZILLA_OFFICIAL causes rebasing to happen properly.

Please accept my apology. I didn't realize this was something for developers as well as release builds. I'm not sure why errors are occuring when you don't rebase, since rebasing was added six weeks or so ago. Why didn't these errors occur then? I'll remove the MOZILLA_OFFICIAL wrapper tomorrow if there are no more pressing fires.

Moving from M8 to M10. leaf, if you will be doing this to the M9 branch build, then please set Target Milestone to M9.

> I didn't realize this was something for developers as > well as release builds. Yah, it'll make developer turnaround time a little faster, since relocating DLLs in memory takes time every time the app is launched. > I'm not sure why errors are occuring when you don't rebase, since rebasing was > added six weeks or so ago. Why didn't these errors occur then? I'm sure they did. I've been on sabbatical for six weeks, and am just getting back now.

yes, the errors have been there forever in debug builds, and they didn't go away when re-basing was added. I'm not sure that we've ever rebased debug dlls consistently, even in 4.x, since one could argue that doing the actual rebasing takes longer than resolving the conflicts at load time.

> I'm not sure that we've ever rebased debug dlls consistently, even in 4.x My recollection is that we always rebased, and that seems to be what the old mozilla.mak does, as I look at it now. > one could argue that doing the actual rebasing takes longer than resolving > the conflicts at load time Maybe if you always compile before running, but I figured people sometimes run without compiling. If that's true, rebasing seens like a win.

we're doing this even on debug now, right? if yes, then close the bug as fixed.

Nope. Still only when MOZILLA_OFFICIAL is defined.

Sorry, still waiting for a day of 1.) little enough tree fire fighting and 2.) enough open-green-tree time to check this build change in.

pushing milestone out to M14 (not a beta blocker) since we do this for the release builds. setting severity to normal. yes, it's a nice thing to do for debug builds, but trying to make bugs reflect priority.

Adding perf to keyword field.

The restart time is about 4 seconds with the debug build on my current NT system. I rebased the debug build and there wasn't a noticeable difference. I don't see a developer win. Troy mentioned that we should be binding the executable during install. Is there a separate bug for that?

Rebasing and binding are two separate issues. The rebasing issue concerns the base address of the DLL. We need to make sure that each of our DLLs has a different base address; otherwise, when a DLL is loaded the OS loader will rebase the DLL (move it to a different place in memory). This can get done as a post process step of the build process The binding step can't get done until install time. Both steps are important and we need to do them

But do we really need to do them on debug builds? (we already rebase for optimized builds, not sure if the installer does binding yet)

It doesn't hurt to rebase for debug builds, and it will mean the loader doesn't have to do it. What's the argument for not doing it for debug builds?

I just talked to jevering about this today, and he said that it ended up not being much of a win for him. (Since this is for -debug- builds only, it's kind of a "let's make life better for hackers" thing anyway, and has no effect on the end product. As a hacker, I'd prefer a quicker build, anyway.) I'm thinking we should close the bug & mark invalid. Troy: you seem to be the big champion for this on debug builds; does it make things noticibly better for you?

Whatever. My major concern was that we do it for optimized builds.

Ok, we've been doing it for release builds for... almost five months. Closing bug.

Marking Verified.