memory usage in mod_perl

[Max Kanat-Alexander]

On Wed, 17 Jan 2007 08:22:41 -0500 (EST) "Jake" <jake at bugzilla.org>
wrote:
> According to that link, Perl won't return it to the kernel, but it
> will reuse it. So unless there's a ton of memory all being used at
> the exact same time on a thread, it doesn't seem like even a long
> buglist should cause memory usage to climb to 350MB.

	Well, it's per-process, not per-thread, if that makes a
difference in your understanding. We don't use a threaded MPM, and in
fact Bugzilla won't work under a threaded MPM. So the threads aren't
sharing anything, really. They share some memory for the compiled
modules and compiled CGIs, but that's it.

	That 350MB is every single memory allocation made during the
page call, and all of the HTML and template code needed to display that
thing.

	I just checked on landfill, and loading every bug that's in
landfill (4726 bugs) on a single buglist page causes Apache to allocate
40MB of memory on top of whatever it already has for that process.

	Given all of the template code and modules, that doesn't seem
like an unreasonable amount to me. The Template Toolkit is not
memory-efficient.

	Although I would be interested to know the breakdown of memory
usage of such a buglist. But without an understanding of perl's
internals (which I mostly lack), it's hard to write a script that will
explain that to me, when the memory is not allocated to any particular
variable.

> And that would require the unfortunate luck of those 20
> people all hitting the same mod_perl thread.

	No, they'd all have to hit separate ones. Otherwise, as you
pointed out, perl would re-use the memory.

> But as soon as their
> buglist is loaded, Perl should start reusing that RAM.

	Only that httpd process will re-use the RAM.

> Do attachments get cached in memory? Could those be what's causing
> memory usage to spike?

	Nothing gets cached in memory. But any display of anything
allocates memory under mod_perl, and that memory won't be returned to
the kernel.

	-Max