воскресенье, 25 июля 2010 г.

OOM Killer в действии: как это происходит в livecd на примере SlavankaOS v.1 rTorrent Edition

Когда операционная система целиком располагается в оперативной памяти, работая без swap и вообще без жесткого диска, есть все шансы столкнуться с нехваткой памяти и познакомится с работой OOM Killer'a. Если же такая система работает круглосуточно без выключения месяцами, то такая перспектива все более реальна. Необходимо понимать как работает этот механизм ядра, чтобы не вызвать его срабатывание. Уже месяц я тестирую сборку livecd c торрент-клентом и на момент прихода в мой подъезд "электрика дяди Вани" uptime составлял 23-е суток. Livecd и входящий в него rtorrent устроен так чтобы при работе максимально расходовать оперативную память и минимально использовать дисковый накопитель. Что происходит если памяти мало?

Rtorrent будет работать в той памяти какую ему выделили. Если ее будет не хватать, то он выдаст ошибку: "Cannot allocate memory", соответственно будет ждать когда загружаемые части будут докачены, записаны на жесткий диск и освободят память. Как следствие скорость загрузки будет низкой. Отсюда вытекает, что чем шире интернет-канал, соответственно тем больше закачек и тем больше нужно будет оперативки. За тем как rtorrent справляется со своими обязанностями внимательно наблюдет ядро linux. Оно специально оставляет небольшой процент 5-6 Mb памяти свободной и если rtorrent попытается занять ее, ядро убьет его. До такого у меня не доходило, но однажды мне потребовалось создать еще одного пользователя и запустить от него еще одну копию rottent чтобы раздельно управлять скоростью upload'a/download'a. Я попытался это сделать при запущенном первом rtorrent который занимал всю память. Ядро убивало новый rtorrent всякий раз когда я пытался запустить его. В итоге мне пришлось завершить работу первого клиента, чтобы он освободил память и затем уже запустить оба клиента. после этого вместе они уже работали без проблем. При каждом срабатывании OOM Killer оставляет отчет o том кого он убил и за что. Его можно посмотреть командой dmesg. В моем случае он выглядел так
rtorrent invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Pid: 1125, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c109e608>] T.381+0x48/0x120
 [<c109e733>] T.380+0x53/0x130
 [<c109e8f1>] __out_of_memory+0xe1/0x140
 [<c109e999>] out_of_memory+0x49/0xb0
 [<c10a11be>] __alloc_pages_nodemask+0x57e/0x590
 [<c10a2ff2>] __do_page_cache_readahead+0xc2/0x1d0
 [<c10a3126>] ra_submit+0x26/0x30
 [<c109d6aa>] filemap_fault+0x32a/0x340
 [<c10b2290>] __do_fault+0x40/0x420
 [<c10b2f56>] ? do_wp_page+0x2d6/0x7d0
 [<c10b361d>] handle_mm_fault+0x1cd/0x7a0
 [<c1350c5e>] ? ip_queue_xmit+0x14e/0x390
 [<c101f7e3>] do_page_fault+0xf3/0x2f0
 [<c101f6f0>] ? do_page_fault+0x0/0x2f0
 [<c13c4c5b>] error_code+0x6b/0x70
 [<c136007b>] ? tcp_ack+0x90b/0x1fb0
 [<c11ae05d>] ? copy_to_user+0x6d/0x130
 [<c13202db>] memcpy_toiovec+0x4b/0x70
 [<c1320c76>] skb_copy_datagram_iovec+0x56/0x230
 [<c1358f5a>] tcp_recvmsg+0x98a/0xb10
 [<c10268b8>] ? __wake_up_common+0x48/0x70
 [<c13192d8>] sock_common_recvmsg+0x48/0x60
 [<c13177bf>] sock_recvmsg+0xdf/0x110
 [<c10743bc>] ? handle_IRQ_event+0x4c/0x140
 [<c122c794>] ? put_device+0x14/0x20
 [<c1004e1d>] ? handle_irq+0x1d/0x30
 [<c13178fd>] sys_recvfrom+0xbd/0x150
 [<c13c2ddc>] ? schedule+0x20c/0x480
 [<c1315af4>] ? sock_poll+0x14/0x20
 [<c10f52f2>] ? ep_send_events_proc+0x62/0x110
 [<c10f58b6>] ? ep_scan_ready_list+0xf6/0x120
 [<c13179cb>] sys_recv+0x3b/0x40
 [<c1318e21>] sys_socketcall+0x1c1/0x2a0
 [<c10540a6>] ? do_gettimeofday+0x16/0x40
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  85
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:54 inactive_file:3597 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:1187 slab_reclaimable:1719 slab_unreclaimable:917
 mapped:1622 shmem:33 pagetables:250 bounce:0
DMA free:2052kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:8kB inactive_file:0kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:128kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:2696kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:14388kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3540kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 5*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2052kB
Normal: 602*4kB 36*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2696kB
20885 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5772 pages shared
124864 pages non-shared
Out of memory: kill process 2562 (bash) score 18444 or a child
Killed process 2566 (rtorrent) vsz:138616kB, anon-rss:3220kB, file-rss:3896kB
rtorrent: page allocation failure. order:0, mode:0x201da
Pid: 2566, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c10a109e>] __alloc_pages_nodemask+0x45e/0x590
 [<c10a2ff2>] __do_page_cache_readahead+0xc2/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  79
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:53 inactive_file:3597 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:1176 slab_reclaimable:1719 slab_unreclaimable:935
 mapped:1621 shmem:33 pagetables:250 bounce:0
DMA free:2008kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:4kB inactive_file:0kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:176kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:2696kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:14388kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3564kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 2*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2008kB
Normal: 602*4kB 36*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2696kB
20884 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5770 pages shared
124887 pages non-shared
rtorrent: page allocation failure. order:0, mode:0x4020
Pid: 2566, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c10a109e>] __alloc_pages_nodemask+0x45e/0x590
 [<c10c4de1>] __slab_alloc+0x351/0x440
 [<c104eecb>] ? hrtimer_forward+0x16b/0x1b0
 [<c10c5cfa>] __kmalloc_track_caller+0x10a/0x160
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d602>] __alloc_skb+0x52/0x140
 [<c131d8a3>] __netdev_alloc_skb+0x23/0x40
 [<e08a5961>] rtl8139_poll+0x191/0x330 [8139too]
 [<c1326586>] net_rx_action+0xa6/0x180
 [<c1037879>] __do_softirq+0x79/0x190
 [<c10743bc>] ? handle_IRQ_event+0x4c/0x140
 [<c10379c5>] do_softirq+0x35/0x40
 [<c1037ac5>] irq_exit+0x35/0x40
 [<c13c4e88>] do_IRQ+0x48/0xb0
 [<c1003470>] common_interrupt+0x30/0x40
 [<c10200e0>] ? clflush_cache_range+0x10/0x40
 [<c11ad5c8>] ? memset+0x18/0x30
 [<c10f1040>] do_mpage_readpage+0x2b0/0x6a0
 [<c10a3b32>] ? ____pagevec_lru_add+0x112/0x120
 [<c109c1ad>] ? add_to_page_cache_locked+0x4d/0xd0
 [<c10f1567>] mpage_readpages+0xc7/0x100
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<e091a4d0>] ? ext3_readpages+0x0/0x20 [ext3]
 [<e091a4ee>] ext3_readpages+0x1e/0x20 [ext3]
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<c10a304c>] __do_page_cache_readahead+0x11c/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  54
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:53 inactive_file:4339 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:679 slab_reclaimable:1719 slab_unreclaimable:945
 mapped:1621 shmem:33 pagetables:250 bounce:0
DMA free:2004kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:4kB inactive_file:1412kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:180kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:712kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:15944kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3600kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kB
Normal: 178*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 712kB
21631 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5776 pages shared
125400 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=0x20)
  cache: kmalloc-2048, object size: 2048, buffer size: 2048, default order: 2, min order: 0
  node 0: slabs: 24, objs: 96, free: 0
eth0: Memory squeeze, dropping packet.
rtorrent: page allocation failure. order:0, mode:0x4020
Pid: 2566, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c10a109e>] __alloc_pages_nodemask+0x45e/0x590
 [<c10c4de1>] __slab_alloc+0x351/0x440
 [<c10c5cfa>] __kmalloc_track_caller+0x10a/0x160
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d602>] __alloc_skb+0x52/0x140
 [<c131d8a3>] __netdev_alloc_skb+0x23/0x40
 [<e08a5961>] rtl8139_poll+0x191/0x330 [8139too]
 [<c1326586>] net_rx_action+0xa6/0x180
 [<c1037879>] __do_softirq+0x79/0x190
 [<c10743bc>] ? handle_IRQ_event+0x4c/0x140
 [<c10379c5>] do_softirq+0x35/0x40
 [<c1037ac5>] irq_exit+0x35/0x40
 [<c13c4e88>] do_IRQ+0x48/0xb0
 [<c1003470>] common_interrupt+0x30/0x40
 [<c10200e0>] ? clflush_cache_range+0x10/0x40
 [<c11ad5c8>] ? memset+0x18/0x30
 [<c10f1040>] do_mpage_readpage+0x2b0/0x6a0
 [<c10a3b32>] ? ____pagevec_lru_add+0x112/0x120
 [<c109c1ad>] ? add_to_page_cache_locked+0x4d/0xd0
 [<c10f1567>] mpage_readpages+0xc7/0x100
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<e091a4d0>] ? ext3_readpages+0x0/0x20 [ext3]
 [<e091a4ee>] ext3_readpages+0x1e/0x20 [ext3]
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<c10a304c>] __do_page_cache_readahead+0x11c/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  54
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:53 inactive_file:4339 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:679 slab_reclaimable:1719 slab_unreclaimable:945
 mapped:1621 shmem:33 pagetables:250 bounce:0
DMA free:2004kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:4kB inactive_file:1412kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:180kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:712kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:15944kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3600kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kB
Normal: 178*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 712kB
21631 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5776 pages shared
125400 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=0x20)
  cache: kmalloc-2048, object size: 2048, buffer size: 2048, default order: 2, min order: 0
  node 0: slabs: 24, objs: 96, free: 0
eth0: Memory squeeze, dropping packet.
rtorrent: page allocation failure. order:0, mode:0x4020
Pid: 2566, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c10a109e>] __alloc_pages_nodemask+0x45e/0x590
 [<c10c4de1>] __slab_alloc+0x351/0x440
 [<c10c5cfa>] __kmalloc_track_caller+0x10a/0x160
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d602>] __alloc_skb+0x52/0x140
 [<c131d8a3>] __netdev_alloc_skb+0x23/0x40
 [<e08a5961>] rtl8139_poll+0x191/0x330 [8139too]
 [<c1326586>] net_rx_action+0xa6/0x180
 [<c1037879>] __do_softirq+0x79/0x190
 [<c10743bc>] ? handle_IRQ_event+0x4c/0x140
 [<c10379c5>] do_softirq+0x35/0x40
 [<c1037ac5>] irq_exit+0x35/0x40
 [<c13c4e88>] do_IRQ+0x48/0xb0
 [<c1003470>] common_interrupt+0x30/0x40
 [<c10200e0>] ? clflush_cache_range+0x10/0x40
 [<c11ad5c8>] ? memset+0x18/0x30
 [<c10f1040>] do_mpage_readpage+0x2b0/0x6a0
 [<c10a3b32>] ? ____pagevec_lru_add+0x112/0x120
 [<c109c1ad>] ? add_to_page_cache_locked+0x4d/0xd0
 [<c10f1567>] mpage_readpages+0xc7/0x100
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<e091a4d0>] ? ext3_readpages+0x0/0x20 [ext3]
 [<e091a4ee>] ext3_readpages+0x1e/0x20 [ext3]
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<c10a304c>] __do_page_cache_readahead+0x11c/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  54
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:53 inactive_file:4339 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:679 slab_reclaimable:1719 slab_unreclaimable:945
 mapped:1621 shmem:33 pagetables:250 bounce:0
DMA free:2004kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:4kB inactive_file:1412kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:180kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:712kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:15944kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3600kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kB
Normal: 178*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 712kB
21631 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5776 pages shared
125400 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=0x20)
  cache: kmalloc-2048, object size: 2048, buffer size: 2048, default order: 2, min order: 0
  node 0: slabs: 24, objs: 96, free: 0
eth0: Memory squeeze, dropping packet.
rtorrent: page allocation failure. order:0, mode:0x4020
Pid: 2566, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c10a109e>] __alloc_pages_nodemask+0x45e/0x590
 [<c10c4de1>] __slab_alloc+0x351/0x440
 [<c10c5cfa>] __kmalloc_track_caller+0x10a/0x160
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d8a3>] ? __netdev_alloc_skb+0x23/0x40
 [<c131d602>] __alloc_skb+0x52/0x140
 [<c131d8a3>] __netdev_alloc_skb+0x23/0x40
 [<e08a5961>] rtl8139_poll+0x191/0x330 [8139too]
 [<c1326586>] net_rx_action+0xa6/0x180
 [<c1037879>] __do_softirq+0x79/0x190
 [<c10743bc>] ? handle_IRQ_event+0x4c/0x140
 [<c10379c5>] do_softirq+0x35/0x40
 [<c1037ac5>] irq_exit+0x35/0x40
 [<c13c4e88>] do_IRQ+0x48/0xb0
 [<c1003470>] common_interrupt+0x30/0x40
 [<c10200e0>] ? clflush_cache_range+0x10/0x40
 [<c11ad5c8>] ? memset+0x18/0x30
 [<c10f1040>] do_mpage_readpage+0x2b0/0x6a0
 [<c10a3b32>] ? ____pagevec_lru_add+0x112/0x120
 [<c109c1ad>] ? add_to_page_cache_locked+0x4d/0xd0
 [<c10f1567>] mpage_readpages+0xc7/0x100
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<e091a4d0>] ? ext3_readpages+0x0/0x20 [ext3]
 [<e091a4ee>] ext3_readpages+0x1e/0x20 [ext3]
 [<e091c1b0>] ? ext3_get_block+0x0/0x110 [ext3]
 [<c10a304c>] __do_page_cache_readahead+0x11c/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  54
active_anon:51216 inactive_anon:51321 isolated_anon:0
 active_file:53 inactive_file:4339 isolated_file:0
 unevictable:17201 dirty:6 writeback:0 unstable:0
 free:679 slab_reclaimable:1719 slab_unreclaimable:945
 mapped:1621 shmem:33 pagetables:250 bounce:0
DMA free:2004kB min:84kB low:104kB high:124kB active_anon:5868kB inactive_anon:6224kB active_file:4kB inactive_file:1412kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:104kB slab_unreclaimable:180kB kernel_stack:24kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:2 all_unreclaimable? no
lowmem_reserve[]: 0 492 492 492
Normal free:712kB min:2792kB low:3488kB high:4188kB active_anon:198996kB inactive_anon:199060kB active_file:208kB inactive_file:15944kB unevictable:68796kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:24kB writeback:0kB mapped:6484kB shmem:132kB slab_reclaimable:6772kB slab_unreclaimable:3600kB kernel_stack:440kB pagetables:960kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:564 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 2*16kB 1*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2004kB
Normal: 178*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 712kB
21631 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5776 pages shared
125400 pages non-shared
SLUB: Unable to allocate memory on node -1 (gfp=0x20)
  cache: kmalloc-2048, object size: 2048, buffer size: 2048, default order: 2, min order: 0
  node 0: slabs: 24, objs: 96, free: 0
eth0: Memory squeeze, dropping packet.
eth0: Memory squeeze, dropping packet.
eth0: Memory squeeze, dropping packet.
eth0: Memory squeeze, dropping packet.
eth0: Memory squeeze, dropping packet.
eth0: Memory squeeze, dropping packet.
oom_kill_process: 46 callbacks suppressed
rtorrent invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Pid: 2649, comm: rtorrent Not tainted 2.6.33.4 #1
Call Trace:
 [<c13c2a36>] ? printk+0x1d/0x1f
 [<c109e608>] T.381+0x48/0x120
 [<c109e733>] T.380+0x53/0x130
 [<c109e8f1>] __out_of_memory+0xe1/0x140
 [<c109e999>] out_of_memory+0x49/0xb0
 [<c10a11be>] __alloc_pages_nodemask+0x57e/0x590
 [<c10a2ff2>] __do_page_cache_readahead+0xc2/0x1d0
 [<c10a31a8>] force_page_cache_readahead+0x78/0xa0
 [<c10b03a3>] sys_madvise+0x283/0x510
 [<c10b7a51>] ? sys_mmap_pgoff+0x91/0x100
 [<c13c47bc>] syscall_call+0x7/0xb
 [<c13c0000>] ? pciserial_init_one+0x37/0x13c
Mem-Info:
DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  81
active_anon:50862 inactive_anon:51351 isolated_anon:0
 active_file:28 inactive_file:3918 isolated_file:0
 unevictable:17210 dirty:0 writeback:0 unstable:0
 free:1196 slab_reclaimable:1687 slab_unreclaimable:932
 mapped:1596 shmem:32 pagetables:214 bounce:0
DMA free:2052kB min:84kB low:104kB high:124kB active_anon:5904kB inactive_anon:6352kB active_file:0kB inactive_file:0kB unevictable:12kB isolated(anon):0kB isolated(file):0kB present:15872kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:16kB slab_unreclaimable:180kB kernel_stack:40kB pagetables:64kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 492 492 492
Normal free:2732kB min:2792kB low:3488kB high:4188kB active_anon:197544kB inactive_anon:199052kB active_file:112kB inactive_file:15672kB unevictable:68828kB isolated(anon):0kB isolated(file):0kB present:503872kB mlocked:0kB dirty:0kB writeback:0kB mapped:6384kB shmem:128kB slab_reclaimable:6732kB slab_unreclaimable:3548kB kernel_stack:416kB pagetables:792kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:174 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 2*8kB 3*16kB 2*32kB 0*64kB 1*128kB 1*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 2052kB
Normal: 593*4kB 45*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2732kB
21188 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
131056 pages RAM
0 pages HighMem
2703 pages reserved
5275 pages shared
125058 pages non-shared
Out of memory: kill process 2562 (bash) score 18381 or a child
Killed process 2649 (rtorrent) vsz:137736kB, anon-rss:2360kB, file-rss:4632kB
обратите внимание, что процессы убивались не сразу а в начале работы, когда пыталитсь получить память под части. Подробнее об алгоритме работы OOM Killer'a можно прочитать здесь:
http://catap.ru/blog/2009/05/03/about-memory-oom-killer
В общем случае правило работы OOM Killer'a следующее: при нехватке памяти все новые процессы убиваются, но нужно учесть, что часто память забита кешем различных данных, который по идее можно высвободить, поэтому все не столь очевидно.

Комментариев нет:

Отправить комментарий