server crash

BlogsNow linux this weblog

It all went to well.

suddenly the server was not reacting. Everything worked till the shell needed to do /anything/ with the disk.
Had to reset it. Sigh.

The syslog said:

May 29 19:14:25 andreaswacker kernel: kswapd0: page allocation failure. order:5, mode:0x50
May 29 19:14:25 andreaswacker kernel: [<0214b29a>] __alloc_pages+0x28b/0x298
May 29 19:14:25 andreaswacker kernel: [<0214b2bf>] __get_free_pages+0x18/0x24
May 29 19:14:25 andreaswacker kernel: [<0214e9c1>] kmem_getpages+0x15/0x94
May 29 19:14:25 andreaswacker kernel: [<0214f74c>] cache_grow+0x155/0x29a
May 29 19:14:25 andreaswacker kernel: [<0214fa9e>] cache_alloc_refill+0x20d/0x23d
May 29 19:14:25 andreaswacker kernel: [<0215004f>] __kmalloc+0x6b/0x7d
May 29 19:14:25 andreaswacker kernel: [<82964f54>] kmem_alloc+0x50/0x96 [xfs]
May 29 19:14:25 andreaswacker kernel: [<829479a1>] xfs_inode_item_format+0xe0/0x239 [xfs]
May 29 19:14:25 andreaswacker kernel: [<8295aaf3>] xfs_trans_fill_vecs+0x3a/0x86 [xfs]
May 29 19:14:25 andreaswacker kernel: [<8295a8a4>] xfs_trans_commit+0x18d/0x300 [xfs]
May 29 19:14:25 andreaswacker kernel: [<829494de>] xfs_iomap_write_allocate+0x248/0x436 [xfs]
May 29 19:14:25 andreaswacker kernel: [<82949520>] xfs_iomap_write_allocate+0x28a/0x436 [xfs]
May 29 19:14:25 andreaswacker kernel: [<0224e474>] generic_make_request+0x190/0x1a0
May 29 19:14:25 andreaswacker kernel: [<829485a3>] xfs_iomap+0x23b/0x3ed [xfs]
May 29 19:14:25 andreaswacker kernel: [<829486ba>] xfs_iomap+0x352/0x3ed [xfs]
May 29 19:14:25 andreaswacker kernel: [<8296c99f>] xfs_bmap+0x1a/0x1e [xfs]
May 29 19:14:25 andreaswacker kernel: [<82965219>] xfs_map_blocks+0x29/0x11e [xfs]
May 29 19:14:25 andreaswacker kernel: [<82965dd9>] xfs_page_state_convert+0x273/0x4e8 [xfs]
May 29 19:14:25 andreaswacker kernel: [<829664cb>] linvfs_writepage+0x91/0xc6 [xfs]
May 29 19:14:25 andreaswacker kernel: [<021522db>] pageout+0x83/0xc0
May 29 19:14:25 andreaswacker kernel: [<02152522>] shrink_list+0x20a/0x547
May 29 19:14:25 andreaswacker kernel: [<02151540>] __pagevec_release+0x15/0x1d
May 29 19:14:25 andreaswacker kernel: [<02152a92>] shrink_cache+0x233/0x4d5
May 29 19:14:25 andreaswacker kernel: [<0215357f>] shrink_zone+0x8f/0x9a
May 29 19:14:25 andreaswacker kernel: [<021538a5>] balance_pgdat+0x176/0x249
May 29 19:14:25 andreaswacker kernel: [<02153a3e>] kswapd+0xc6/0xc8
May 29 19:14:25 andreaswacker kernel: [<02120fcf>] autoremove_wake_function+0x0/0x2d
May 29 19:14:25 andreaswacker kernel: [<02120fcf>] autoremove_wake_function+0x0/0x2d
May 29 19:14:25 andreaswacker kernel: [<02153978>] kswapd+0x0/0xc8
May 29 19:14:25 andreaswacker kernel: [<021041d9>] kernel_thread_helper+0x5/0xb
May 29 19:14:25 andreaswacker kernel: deadlock in kmem_alloc (mode:0x50)
May 29 19:14:25 andreaswacker kernel: possible deadlock in kmem_alloc (mode:0x50)
May 29 19:14:25 andreaswacker last message repeated 85 times

Don’t hope that this is common for Fedora Core3 on a AMD machine with a big array. I start to load the machine with tasks now.
Let’s see if it happens again.
Sure enough Mysql was not happy since I run it with delay-key-write.

WordPress said:


WordPress database error: [Incorrect key file for table 'wp_comments'; try to repair it]

So I did a


mysqlcheck -pXXXXXX --auto-repair wordpress

which seems to have done the trick.