Closed Bug 37096 Opened 25 years ago Closed 25 years ago

Slow checkouts from cvs.mozilla.org

Categories

(mozilla.org Graveyard :: Server Operations, task, P3)

All
Mac System 8.5

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: sfraser_bugs, Assigned: rkotalampi)

References

Details

Checking out from cvs.mozilla.org is slow today, as reported by several people. The netscape firewall looks OK (2-3% lossage).
I/O is heavy, heavy, heavy... I changed some parameters in vxfs filesystem... do you see any difference yet?
Status: NEW → ASSIGNED
*** Bug 37185 has been marked as a duplicate of this bug. ***
I've been running vxfs inode extent defragmentation few times this afternoon. That might have slowed down checkouts little bit... but I'm on it. If these on-fly operations won't help I need to shut down whole cvs for an hour or so that I can reorg the disk layout and mounting options (all options doesn't take affect with remount for some reason).
I guess I might have found a reason to I/O slowness for /cvsdisk: the disk is _very_ fragmented. I will try to defragment it with fsadm during nights.
Extent Fragmentation Report Total Average Average Total Files File Blks # Extents Free Blks 44224 16 1 7601814 blocks used for indirects: 8 % Free blocks in extents smaller than 64 blks: 36.12 % Free blocks in extents smaller than 8 blks: 17.89 % blks allocated to extents 64 blks or larger: 63.68 Free Extents By Size 1: 7644 2: 5103 4: 8713 8: 2695 16: 942 32: 533 64: 354 128: 179 256: 93 512: 44 1024: 12 2048: 5 4096: 4 8192: 3 16384: 2 32768: 5
Ok... defragmenting didn't do very much good. I got <8k fragmentation finally to less than 10% but performance still sucked. Last night I broke existing disk volumes and created RAID0+1 with hotspares: d80: Mirror Submirror 0: d81 State: Okay Submirror 1: d82 State: Okay Pass: 1 Read option: roundrobin (default) Write option: parallel (default) Size: 41796000 blocks d81: Submirror of d80 State: Okay Hot spare pool: hsp203 Size: 41796000 blocks Stripe 0: (interlace: 32 blocks) Device Start Block Dbase State Hot Spare c0t2d0s3 0 No Okay c2t2d0s3 0 No Okay c3t2d0s3 0 No Okay c4t2d0s3 0 No Okay c5t2d0s3 0 No Okay d82: Submirror of d80 State: Okay Hot spare pool: hsp203 Size: 41796000 blocks Stripe 0: (interlace: 32 blocks) Device Start Block Dbase State Hot Spare c0t3d0s3 0 No Okay c2t3d0s3 0 No Okay c3t3d0s3 0 No Okay c4t3d0s3 0 No Okay c5t3d0s3 0 No Okay This filesystem have vxfs on it and is mounted as /export3 (not /cvsdisk as it used to be so we can possibly put something else on it too). /cvsdisk is a symlink to /export3/cvsdisk to maintain compatibility with all older paths. So far it has been acting very well and performance have been pretty good. But tinderboxes are still down and bringing them up will be the final test of how performance will be.
All tests looks very good... disk responses have been fast even if tinderboxes are pounding it heavily. So I consider the case closed... reopen if there are more problems.
Status: ASSIGNED → RESOLVED
Closed: 25 years ago
Resolution: --- → FIXED
Product: mozilla.org → mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.