How do I determine appropriate swapspace settings? And other partitioning questions.

[Morningdew]

Hello!

I was just curious to know... What are some good rules of thumb for
configuring swap space under Linux? For that matter, are there any
comprehensive guides or articles on the subject? When I used to admin
NT boxes I used to set their "virtual memory" setting to 1.5 times
physical memory, giving 50% over physical as swap. That was just by
convention, or, "rule of thumb". No great scientific methodology, it is
just what had worked for me and my coworkers for so long. Granted,
Linux is different.

On my box I have installed Ubuntu and moved up to the 2.6.10 kernel, if
that matters. I am presently running on two IDE hard drives, 30Mb
Quantum Fireball and 60Mb Seagate ST360020A. As it stands, I have 1Gb
of physical RAM and have two swap partitions, one on each drive and both
494.16Mb. Initially I had set up a 32-bit install on one drive, and a
64-bit (AMD64) on the other. I decided to go with my old NT convention
for grins, and give 50% over physical for swap. But since I have both
swap partitions available, I have both the 32 and 64 bit installs using
both swaps, for a nearly 1:1 phys:swap ratio.

Reason I am asking is because I will be very soon installing a 250Gb
Western Digital SATA drive. Long story on the old one, but short
version is it shot craps before I could ever use it. But now since I
will be getting all this breathing room I will of course be
reconfiguring my partitioning scheme. So given the opportunity I would
like to put some method to the madness.

One consideration, besides size, is location. This question goes beyond
swap space, too. How does Linux like to have it's partitions metered
across multiple drives? Should the root and swap be on different
physical drives or does it matter? If it is better to separate, which
would benefit most from "the faster drive"? Does it actually help to
split the swap amongst partitions on differing drives, similar to (yes,
remotely similar to) how striping speeds up RAID performance?

While I am asking, I would also be interested in suggestions with regard
to partitions and placements. I have been running 5Gb roots that include
my /var and /home. Most of my "media", including music, video, photos,
and the like I have been keeping on a separate partition that I mount
under /mnt/share. This way I have access to it from whichever root I
boot to, as well as making it publicly readable to my family's logins.
Not much needs to be kept private, and, well, I manage that when need be.

I think it would be unwise for me to make the /home directories be the
same between the two installs (32 and 64-bit). But I am intruiged to
know how far such a notion could be taken. Being able to have
Thunderbird and my GPG keys available no matter which I boot to would be
rather convenient. I could spend more time in 64-bit land. Right now I
must use the 32-bit install for that.

Lastly, since I don't want to tap you all toooo much all at once, I am
curious about this whole "chroot" thing. Since I have both 32 and
64-bit installs, is there a way to make my 32-bit root BE the chroot
under 64-bit? That would totally rock! I have zero experience with this
and am only sort-of understanding how it all works. I would like to,
for instance, just run 64-bit firefox and have it use 32-bit
libflashplayer.so. That, of course, being one of the very few things
keeping me on the 32-bit side 90% of the time when I would much rather
go 64-bit as much as possible. But I have heard that you can't have a
64-bit app call a 32-bit library. Okay. So then 32-bit FF. But then
that also means all the dependencies for FF and for FlashPlayer, right?
Well, then, at what point DO the 64 and 32-bit parts commingle? Would
32-bit FF run on the 64-bit X session? That, I could see. Or is it
some "xnest" type set-up? If it dose run on the display, would it talk
to 64-bit Gnome? What kind of crazy nightmare am I getting myself into
with this chroot thingy? Am I better off sitting in a corner, beating my
head against the wall? Maybe I should stop wasting my time on this and
use it more (!) effectively, writing pleas to Macromedia to get off
their corporate duffs and spend the, what, hour or two it would (should:
can't know with closed-source) take to do the damned port.

Well that's all the time my meter had, and then some. Thanks in advance
for the time to read me and for any help and advice. And apologies for
the cross-posting. You know, I have not seen any "forum guides" of any
sort come down the pike on any of these newsgroups for a few months. So
which Linux newsgroup is good for what sorts of linuxy things I just
don't know. I'd "RTFM" on these newsgroups if I knew where to find it.

Peace!
Morningdew

[Richard Eggleston]

On Tue, 08 Feb 2005 14:30:22 -0600, Morningdew wrote:


(Apologies for top posting,)
Hi
I was told that twice the RAM is a good rule of thumb, upto a max of
1024MB
I guess that much more is a waste of resources

L & P

Richard

Post by Morningdew
Hello!
I was just curious to know... What are some good rules of thumb for
configuring swap space under Linux? For that matter, are there any
comprehensive guides or articles on the subject? When I used to admin
NT boxes I used to set their "virtual memory" setting to 1.5 times
physical memory, giving 50% over physical as swap. That was just by
convention, or, "rule of thumb". No great scientific methodology, it is
just what had worked for me and my coworkers for so long. Granted,
Linux is different.
On my box I have installed Ubuntu and moved up to the 2.6.10 kernel, if
that matters. I am presently running on two IDE hard drives, 30Mb
Quantum Fireball and 60Mb Seagate ST360020A. As it stands, I have 1Gb
of physical RAM and have two swap partitions, one on each drive and both
494.16Mb. Initially I had set up a 32-bit install on one drive, and a
64-bit (AMD64) on the other. I decided to go with my old NT convention
for grins, and give 50% over physical for swap. But since I have both
swap partitions available, I have both the 32 and 64 bit installs using
both swaps, for a nearly 1:1 phys:swap ratio.
Reason I am asking is because I will be very soon installing a 250Gb
Western Digital SATA drive. Long story on the old one, but short
version is it shot craps before I could ever use it. But now since I
will be getting all this breathing room I will of course be
reconfiguring my partitioning scheme. So given the opportunity I would
like to put some method to the madness.
One consideration, besides size, is location. This question goes beyond
swap space, too. How does Linux like to have it's partitions metered
across multiple drives? Should the root and swap be on different
physical drives or does it matter? If it is better to separate, which
would benefit most from "the faster drive"? Does it actually help to
split the swap amongst partitions on differing drives, similar to (yes,
remotely similar to) how striping speeds up RAID performance?
While I am asking, I would also be interested in suggestions with regard
to partitions and placements. I have been running 5Gb roots that include
my /var and /home. Most of my "media", including music, video, photos,
and the like I have been keeping on a separate partition that I mount
under /mnt/share. This way I have access to it from whichever root I
boot to, as well as making it publicly readable to my family's logins.
Not much needs to be kept private, and, well, I manage that when need be.
I think it would be unwise for me to make the /home directories be the
same between the two installs (32 and 64-bit). But I am intruiged to
know how far such a notion could be taken. Being able to have
Thunderbird and my GPG keys available no matter which I boot to would be
rather convenient. I could spend more time in 64-bit land. Right now I
must use the 32-bit install for that.
Lastly, since I don't want to tap you all toooo much all at once, I am
curious about this whole "chroot" thing. Since I have both 32 and
64-bit installs, is there a way to make my 32-bit root BE the chroot
under 64-bit? That would totally rock! I have zero experience with this
and am only sort-of understanding how it all works. I would like to,
for instance, just run 64-bit firefox and have it use 32-bit
libflashplayer.so. That, of course, being one of the very few things
keeping me on the 32-bit side 90% of the time when I would much rather
go 64-bit as much as possible. But I have heard that you can't have a
64-bit app call a 32-bit library. Okay. So then 32-bit FF. But then
that also means all the dependencies for FF and for FlashPlayer, right?
Well, then, at what point DO the 64 and 32-bit parts commingle? Would
32-bit FF run on the 64-bit X session? That, I could see. Or is it
some "xnest" type set-up? If it dose run on the display, would it talk
to 64-bit Gnome? What kind of crazy nightmare am I getting myself into
with this chroot thingy? Am I better off sitting in a corner, beating my
head against the wall? Maybe I should stop wasting my time on this and
use it more (!) effectively, writing pleas to Macromedia to get off
can't know with closed-source) take to do the damned port.
Well that's all the time my meter had, and then some. Thanks in advance
for the time to read me and for any help and advice. And apologies for
the cross-posting. You know, I have not seen any "forum guides" of any
sort come down the pike on any of these newsgroups for a few months. So
which Linux newsgroup is good for what sorts of linuxy things I just
don't know. I'd "RTFM" on these newsgroups if I knew where to find it.
Peace!
Morningdew

[Dan C]

Post by Richard Eggleston
(Apologies for top posting,)
Hi
I was told that twice the RAM is a good rule of thumb, upto a max of
1024MB
I guess that much more is a waste of resources

I almost don't believe this... You KNOW that top-posting sucks, you
apologize in advance for it, and yet YOU STILL DO IT??? WHAT THE FUCK IS
WRONG WITH YOU?

You also don't know how to snip out the irrelavent portions of the text
you quoted.

I didn't think it was possible, but you are now hereby declared THE ALL
TIME BIGGEST FUCKING MORON EVER SEEN ON USENET.

Congratulations, dipshit.

[Paul Sherwin]

Post by Morningdew
I was just curious to know... What are some good rules of thumb for
configuring swap space under Linux? For that matter, are there any
comprehensive guides or articles on the subject? When I used to admin
NT boxes I used to set their "virtual memory" setting to 1.5 times
physical memory, giving 50% over physical as swap. That was just by
convention, or, "rule of thumb". No great scientific methodology, it is
just what had worked for me and my coworkers for so long. Granted,
Linux is different.

There is *no* direct relationship between physical memory and swapfile
size. These rules of thumb originated in the 70s when people had to
commission systems before knowing what the application mix would be.
The idea was that a system with a lot of memory would be used for
memory intensive applications, so should have a lot of swapspace as
well. Some systems also used swapspace to hold a memory dump in the
event of a system crash, so the swap had to be at least as large as
physical memory.

Modern Linux systems with lots of physical memory will almost never
swap, unless you're doing something very unusual. If you find the
system is swapping a lot, you should add more physical memory rather
than fiddling around with swapspace, My main Linux server is currently
using 778k of swapspace, and this is quite typical. Despite this,
people keep configuring gigabytes of swapspace.

You can always add a swap file later, which will keep you going until
you upgrade memory.

HTH, Paul
--
Paul Sherwin Consulting http://paulsherwin.co.uk

[Morningdew]

Post by Paul Sherwin
There is *no* direct relationship between physical memory and swapfile
size. These rules of thumb originated in the 70s when people had to
commission systems before knowing what the application mix would be.

Okay. I would think that the books and distros installers would mention
that it is largely not need after a certain point.

Post by Paul Sherwin
The idea was that a system with a lot of memory would be used for
memory intensive applications, so should have a lot of swapspace as
well. Some systems also used swapspace to hold a memory dump in the
event of a system crash, so the swap had to be at least as large as
physical memory.

Well I am not going to be debugging a core dump like that anyway =)

Post by Paul Sherwin
Modern Linux systems with lots of physical memory will almost never
swap, unless you're doing something very unusual. If you find the
system is swapping a lot, you should add more physical memory rather
than fiddling around with swapspace, My main Linux server is currently
using 778k of swapspace, and this is quite typical. Despite this,
people keep configuring gigabytes of swapspace.

Yes, here I am with just shy of a gig. But I should not that besides
the new hard disk coming soon, I *have* been having problems with
memory, even at a Gig. Yeah, I know it seem ridiculous. Even with what
I would figure to be a "light" load running under Gnome I am sitting at
89% memory use, of which 53% is cache. Now 53% of the 89% or 53 _of_
the 89% is ambiguous in gnome-system-monitor. But, still... What is
that Cache? HDD cache? Why does that accumulate rather than flush? If
I have a power outage with that much cache sitting in RAM won't I be
likely to foul up a partition? I guess that'll put the journals to the
test. But dang, I am just sitting here typing and it won't go down.

See, at least three times now I have had my system schitzotically freeze
up on me when that memory ticker hit 100%. It starts to "stutter", then
gets to where I get a half-second of life between 30-second to several
minute seizures. Even at a console, outside of X. Firefox with lots of
tabs was part to blame once. Totem running a downloaded SWF another
time. Come to find out that on my 32-bit install, I didn't have either
swap partition in the /etc/fstab file. I fixed that but have not maxed
out to reproduce the problem yet. I'm sure I'll get to test with the
swap space soon, though.

Post by Paul Sherwin
You can always add a swap file later, which will keep you going until
you upgrade memory.

Too broke to joke, so more memory is not happening real soon. Why does
my Linux barf and sputter when physical memory gets full? You would
think it would have some more graceful contingencies. Perhaps my disk
cache (assuming that's what the "cache" portion is) is not configured
properly and needs adjusting. Ugh. Thanks for the reply, though.

Post by Paul Sherwin
HTH, Paul
--
Paul Sherwin Consulting http://paulsherwin.co.uk

Oh, and I, like, re-posted this message a few times because it was not
showing up for me. I tried cutting down some of the cross-posts, and
finally cut back the subject line and got it to post. Heh... of course
it seems you got at least one of them on your nntp server that mine
didn't want to show, 'cuz you replied and I don't see the original. Way
to go Charter Communications. Your NNTP service is not only slow, it
also bites.

Peace,
Morningdew

[J.O. Aho]

Post by Morningdew
Yes, here I am with just shy of a gig. But I should not that besides
the new hard disk coming soon, I *have* been having problems with
memory, even at a Gig. Yeah, I know it seem ridiculous. Even with what
I would figure to be a "light" load running under Gnome I am sitting at
89% memory use, of which 53% is cache. Now 53% of the 89% or 53 _of_
the 89% is ambiguous in gnome-system-monitor. But, still... What is
that Cache? HDD cache? Why does that accumulate rather than flush? If
I have a power outage with that much cache sitting in RAM won't I be
likely to foul up a partition? I guess that'll put the journals to the
test. But dang, I am just sitting here typing and it won't go down.

Cache is used to store data that can be used many times, so instead of loading
everything from harddrive you may have a copy in ram, which makes it faster to
load next time you start. Eg, you start mozilla, parts of it will be cached,
you turn off mozilla, next time you start (assuming the cache isn't
overwritten), it will start a lot faster as parts of it is already in ram.

Sooner or later you will come to a point where the whole memory is used
(mostly by cache), at that point there will be part of the cache overwritten
with more important/relevant data.

As long as you don't use XFS as your filesystem, you don't have to worry about
the filesystem to being stored in RAM, if you use XFS, then you will have
parts of the filesystem in RAM and you will get corruptions if you cut the
power to the computer, XFS are mainly used where you need high speed and where
you have the original data stored on another filesystem (common in video editing).

Post by Morningdew
See, at least three times now I have had my system schitzotically freeze
up on me when that memory ticker hit 100%. It starts to "stutter", then
gets to where I get a half-second of life between 30-second to several
minute seizures. Even at a console, outside of X. Firefox with lots of
tabs was part to blame once. Totem running a downloaded SWF another
time. Come to find out that on my 32-bit install, I didn't have either
swap partition in the /etc/fstab file. I fixed that but have not maxed
out to reproduce the problem yet. I'm sure I'll get to test with the
swap space soon, though.

If you run out of RAM and don't have SWAP, then you run into trouble, I think
you won't notice things as much next time you max out.

Post by Morningdew
Oh, and I, like, re-posted this message a few times because it was not
showing up for me. I tried cutting down some of the cross-posts, and
finally cut back the subject line and got it to post. Heh... of course
it seems you got at least one of them on your nntp server that mine
didn't want to show, 'cuz you replied and I don't see the original. Way
to go Charter Communications. Your NNTP service is not only slow, it
also bites.

Depending on your ISP/NGP, it can take a while before you see a post, even
your own, to appear on the newsgroups... Some have built in "spam" detectors
and will filter away posts and in those cases you won't see the "offending"
post at all (nothing is perfect, so valide mail can be filtered away too).


//Aho

[Paul Sherwin]

Post by Morningdew
See, at least three times now I have had my system schitzotically freeze
up on me when that memory ticker hit 100%. It starts to "stutter", then
gets to where I get a half-second of life between 30-second to several
minute seizures. Even at a console, outside of X. Firefox with lots of
tabs was part to blame once. Totem running a downloaded SWF another
time. Come to find out that on my 32-bit install, I didn't have either
swap partition in the /etc/fstab file. I fixed that but have not maxed
out to reproduce the problem yet. I'm sure I'll get to test with the
swap space soon, though.

This shouldn't happen when you start to swap, and it suggests you
didn't have your swap partition enabled properly. You *will* get a
performance hit though.

Post by Morningdew
Too broke to joke, so more memory is not happening real soon. Why does
my Linux barf and sputter when physical memory gets full? You would
think it would have some more graceful contingencies. Perhaps my disk
cache (assuming that's what the "cache" portion is) is not configured
properly and needs adjusting. Ugh. Thanks for the reply, though.

'cache' is filesystem cache. Linux will use free memory to cache the
filesystem. A large 'cache' figure indicates you have lots of free
memory, not that you're running out. It will get smaller as processes
grab more memory. What matters is how much space is being used on your
swap partition - if this is less than a couple of megs, your memory
configuration is fine.

Best regards, Paul
--
Paul Sherwin Consulting http://paulsherwin.co.uk

[Sir Jackery]

Post by Paul Sherwin

Post by Morningdew
I was just curious to know... What are some good rules of thumb for
configuring swap space under Linux? For that matter, are there any
comprehensive guides or articles on the subject? When I used to admin
NT boxes I used to set their "virtual memory" setting to 1.5 times
physical memory, giving 50% over physical as swap. That was just by
convention, or, "rule of thumb". No great scientific methodology, it is
just what had worked for me and my coworkers for so long. Granted,
Linux is different.

There is *no* direct relationship between physical memory and swapfile
size. These rules of thumb originated in the 70s when people had to
commission systems before knowing what the application mix would be.
The idea was that a system with a lot of memory would be used for
memory intensive applications, so should have a lot of swapspace as
well. Some systems also used swapspace to hold a memory dump in the
event of a system crash, so the swap had to be at least as large as
physical memory.
Modern Linux systems with lots of physical memory will almost never
swap, unless you're doing something very unusual. If you find the
system is swapping a lot, you should add more physical memory rather
than fiddling around with swapspace, My main Linux server is currently
using 778k of swapspace, and this is quite typical. Despite this,
people keep configuring gigabytes of swapspace.
You can always add a swap file later, which will keep you going until
you upgrade memory.

I agree, except you left out that X and its applications use huge amounts
of memory. All of my servers each have a 128m swap partition and run with
about 256-512m RAM. They NEVER swap, but there is no point in not having a
health size swap partition because it is very easy to reclaim the space.
With the cost of hdd space these days, why not allocate a 512m swap
partition? It could come in handy in the future. If your computer starts
to use it, do what morningdew says and add more memory. If your system is
steadily using your swap it will bottleneck systems performance
substantially.

-jackery

Post by Paul Sherwin
HTH, Paul
--
Paul Sherwin Consulting http://paulsherwin.co.uk

[Paul Sherwin]

Post by Sir Jackery
I agree, except you left out that X and its applications use huge amounts
of memory. All of my servers each have a 128m swap partition and run with
about 256-512m RAM. They NEVER swap, but there is no point in not having a
health size swap partition because it is very easy to reclaim the space.
With the cost of hdd space these days, why not allocate a 512m swap
partition? It could come in handy in the future. If your computer starts
to use it, do what morningdew says and add more memory. If your system is
steadily using your swap it will bottleneck systems performance
substantially.

Sure, it's a false economy to run with no swap at all, given the cost
of HD space. I think 256-512 RAM is pretty typical for a current Linux
desktop machine and they won't swap much even running big X desktops.
I usually configure 128m and keep an eye on things for the first few
hours. I've never had to add more swap yet :-)

You'll soon notice if your system is hitting the swap partition hard -
it'll run like a dog.

Best regards, Paul
--
Paul Sherwin Consulting http://paulsherwin.co.uk

[Grant Diffey]

Post by Morningdew
Hello!
I was just curious to know... What are some good rules of thumb for
configuring swap space under Linux? For that matter, are there any
comprehensive guides or articles on the subject? When I used to admin
NT boxes I used to set their "virtual memory" setting to 1.5 times
physical memory, giving 50% over physical as swap. That was just by
convention, or, "rule of thumb". No great scientific methodology, it is
just what had worked for me and my coworkers for so long. Granted,
Linux is different.

Yep. the fundamental difference is that NT uses a traditional VM design
(linux briefly had a traditional vm in the 2.4 series) based around keeping
what's currently in memory also in swap so that when you need more memory
you can instantly zap those clean pages and get more memory because they've
aready been written to disk in idle time. the upshot of this is that you
MUST have at least as much swap as you have physical memory.

Linux 2.4.0 -> 2.4.11 had a vm of this style. (Rick's vm) andrea's vm which
replaced rick's in the 2.4 stable series works like the linux vm has
historically worked that is to say pages are only kept in memory and disk
is used as an overflow only. 2.6 continues with andrea's vm.

Post by Morningdew
On my box I have installed Ubuntu and moved up to the 2.6.10 kernel, if
that matters. I am presently running on two IDE hard drives, 30Mb
Quantum Fireball and 60Mb Seagate ST360020A. As it stands, I have 1Gb
of physical RAM and have two swap partitions, one on each drive and both
494.16Mb. Initially I had set up a 32-bit install on one drive, and a
64-bit (AMD64) on the other. I decided to go with my old NT convention
for grins, and give 50% over physical for swap. But since I have both
swap partitions available, I have both the 32 and 64 bit installs using
both swaps, for a nearly 1:1 phys:swap ratio.

That's HEAPS for linux see above.

Post by Morningdew
Reason I am asking is because I will be very soon installing a 250Gb
Western Digital SATA drive. Long story on the old one, but short
version is it shot craps before I could ever use it. But now since I
will be getting all this breathing room I will of course be
reconfiguring my partitioning scheme. So given the opportunity I would
like to put some method to the madness.
One consideration, besides size, is location. This question goes beyond
swap space, too. How does Linux like to have it's partitions metered
across multiple drives? Should the root and swap be on different
physical drives or does it matter? If it is better to separate, which
would benefit most from "the faster drive"? Does it actually help to
split the swap amongst partitions on differing drives, similar to (yes,
remotely similar to) how striping speeds up RAID performance?

you could create a software raid of two small partitions and swap on that.
as the disk access is going to be way more expensive than the raiding.

Post by Morningdew
While I am asking, I would also be interested in suggestions with regard
to partitions and placements. I have been running 5Gb roots that include
my /var and /home. Most of my "media", including music, video, photos,
and the like I have been keeping on a separate partition that I mount
under /mnt/share. This way I have access to it from whichever root I
boot to, as well as making it publicly readable to my family's logins.
Not much needs to be kept private, and, well, I manage that when need be.

We have that in our house too. we have a 200GB XFS volume called /bigish
which will soon be expanded with another disk to 400GB yay for LVM

Post by Morningdew
I think it would be unwise for me to make the /home directories be the
same between the two installs (32 and 64-bit). But I am intruiged to
know how far such a notion could be taken. Being able to have
Thunderbird and my GPG keys available no matter which I boot to would be
rather convenient. I could spend more time in 64-bit land. Right now I
must use the 32-bit install for that.

if the distro is the same (ubuntu) there is no reason not to share /home
just make sure that /etc/passwd /etc/shadow /etc/groups and /etc/gshadow are
synced between the systems and you should be all good.

Post by Morningdew
Lastly, since I don't want to tap you all toooo much all at once, I am
curious about this whole "chroot" thing. Since I have both 32 and
64-bit installs, is there a way to make my 32-bit root BE the chroot
under 64-bit? That would totally rock! I have zero experience with this
and am only sort-of understanding how it all works. I would like to,
for instance, just run 64-bit firefox and have it use 32-bit
libflashplayer.so. That, of course, being one of the very few things
keeping me on the 32-bit side 90% of the time when I would much rather
go 64-bit as much as possible. But I have heard that you can't have a
64-bit app call a 32-bit library. Okay. So then 32-bit FF. But then
that also means all the dependencies for FF and for FlashPlayer, right?
Well, then, at what point DO the 64 and 32-bit parts commingle? Would
32-bit FF run on the 64-bit X session? That, I could see. Or is it
some "xnest" type set-up? If it dose run on the display, would it talk
to 64-bit Gnome? What kind of crazy nightmare am I getting myself into
with this chroot thingy? Am I better off sitting in a corner, beating my
head against the wall? Maybe I should stop wasting my time on this and
use it more (!) effectively, writing pleas to Macromedia to get off
can't know with closed-source) take to do the damned port.

A chroot won't do what you want. User Mode Linux might tho but it'd be
reasonably complicated.

regards Grant.