今天早上我的服务器坏了一个小时,我在aws ec2上使用ubuntu 16.04lts。

一些信息:这是一个t2.medium实例,我最近从ubuntu 14.04升级到16.04,更新了很多包,这是一个生产实例,我使用的是php版本5.6.40-10+ubuntu16.04.1+deb.sury.org+1,对于一些通过快速cgi php模块使用最新php的站点,我的设备内存/xvda1还有49%的存储空间。

从系统日志:

Oct 11 05:09:10 ip-10-0-x-xxx systemd[1]: Starting Clean php session files...
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[1]: Stopping User Manager for UID 110...
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Reached target Shutdown.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Starting Exit the Session...
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Stopped target Default.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Stopped target Basic System.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Stopped target Timers.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Stopped target Sockets.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Stopped target Paths.
    Oct 11 05:09:10 ip-10-0-x-xxx CRON[28971]: (root) CMD (  [ -

x/usr/lib/php5/maxlifetime ] && [ -x /usr/lib/php5/sessionclean ] && [ -d /var/lib/php5 ] && /usr/lib/php5/sessionclean /var/lib/php5 $(/usr/lib/php5/maxlifetime))
Oct 11 05:09:10 ip-10-0-x-xxx CRON[28972]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[19573]: Received SIGRTMIN+24 from PID 28967 (kill).
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[1]: Stopped User Manager for UID 110.
    Oct 11 05:09:10 ip-10-0-x-xxx systemd[1]: Removed slice User Slice of solr.
    Oct 11 05:10:32 ip-10-0-x-xxx kernel: [488908.756151] apache2 invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0
    Oct 11 05:10:32 ip-10-0-x-xxx kernel: [488908.756155] apache2 cpuset=/ mems_allowed=0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756160] CPU: 1 PID: 29346 Comm: apache2 Not tainted 4.4.0-157-generic #185-Ubuntu
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756162] Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756163]  0000000000000286 3f04de0b6dc470df ffff88005a2739e8 ffffffff8140c9a1
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756166]  ffff88005a273ba0 ffff8800e84e4600 ffff88005a273a58 ffffffff81219bde
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756169]  ffff8800eaf61c00 ffff88005a273ba0 ffff88005a273ab0 ffff8800e86a5400
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756171] Call Trace:
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756180]  [<ffffffff8140c9a1>] dump_stack+0x63/0x82
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756184]  [<ffffffff81219bde>] dump_header+0x5a/0x1c3
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756189]  [<ffffffff8119db2b>] oom_kill_process+0x20b/0x3d0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756191]  [<ffffffff8119df38>] out_of_memory+0x1f8/0x460
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756196]  [<ffffffff811a3f73>] __alloc_pages_slowpath.constprop.88+0x943/0xaf0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756199]  [<ffffffff811a43a8>] __alloc_pages_nodemask+0x288/0x2a0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756203]  [<ffffffff811efc6c>] alloc_pages_current+0x8c/0x110
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756205]  [<ffffffff81199c5b>] __page_cache_alloc+0xab/0xc0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756208]  [<ffffffff8119c580>] filemap_fault+0x160/0x440
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756211]  [<ffffffff812b3016>] ext4_filemap_fault+0x36/0x50
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756215]  [<ffffffff811c9d37>] __do_fault+0x77/0x110
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756218]  [<ffffffff811cdcdc>] handle_mm_fault+0x125c/0x1b80
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756222]  [<ffffffff81860041>] ? __schedule+0x301/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756224]  [<ffffffff8186004d>] ? __schedule+0x30d/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756225]  [<ffffffff81860041>] ? __schedule+0x301/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756227]  [<ffffffff8186004d>] ? __schedule+0x30d/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756229]  [<ffffffff81860041>] ? __schedule+0x301/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756231]  [<ffffffff81860041>] ? __schedule+0x301/0x810
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756235]  [<ffffffff8106edd4>] __do_page_fault+0x1a4/0x410
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756237]  [<ffffffff8106f062>] do_page_fault+0x22/0x30
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756241]  [<ffffffff81867ae8>] page_fault+0x28/0x30
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756242] Mem-Info:
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247] active_anon:919500 inactive_anon:18067 isolated_anon:0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247]  active_file:620 inactive_file:764 isolated_file:0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247]  unevictable:0 dirty:0 writeback:0 unstable:0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247]  slab_reclaimable:8517 slab_unreclaimable:10839
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247]  mapped:25419 shmem:36238 pagetables:21622 bounce:0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756247]  free:21367 free_pcp:140 free_cma:0
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756251] Node 0 DMA free:15820kB min:268kB low:332kB high:400kB active_anon:0kB inactive_anon:0kB active_file:8kB inactive_file:28kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15900kB mlocked:0kB dirty:0kB writeback:0kB mapped:8kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756256] lowmem_reserve[]: 0 3704 3893 3893 3893
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756260] Node 0 DMA32 free:64756kB min:64040kB low:80048kB high:96060kB active_anon:3513132kB inactive_anon:69560kB active_file:2472kB inactive_file:3028kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3915776kB managed:3834888kB mlocked:0kB dirty:0kB writeback:0kB mapped:95428kB shmem:137276kB slab_reclaimable:32652kB slab_unreclaimable:38928kB kernel_stack:4192kB pagetables:83352kB unstable:0kB bounce:0kB free_pcp:560kB local_pcp:400kB free_cma:0kB writeback_tmp:0kB pages_scanned:37356 all_unreclaimable? yes
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756265] lowmem_reserve[]: 0 0 189 189 189
    Oct 11 05:10:33 ip-10-0-x-xxx kernel: [488908.756268] Node 0 Normal free:4892kB min:3268kB low:4084kB high:4900kB active_anon:164868kB inactive_anon:2708kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:262144kB managed:193808kB mlocked:0kB dirty:0kB writeback:0kB mapped:6240kB shmem:7676kB slab_reclaimable:1416kB slab_unreclaimable:4428kB kernel_stack:720kB pagetables:3136kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes

然后我得到一个巨大的列表:

    Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174008] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174009] 37420 total pagecache pages
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174011] 0 pages in swap cache
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174012] Swap cache stats: add 0, delete 0, find 0/0
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174013] Free swap  = 0kB
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174014] Total swap = 0kB
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174015] 1048477 pages RAM
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174016] 0 pages HighMem/MovableOnly
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174017] 37328 pages reserved
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174018] 0 pages cma reserved
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174019] 0 pages hwpoisoned
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174020] [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174025] [  275]     0   275    10189      689      21       3        0             0 systemd-journal
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174028] [  303]     0   303    11224      309      23       3        0         -1000 systemd-udevd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174030] [  532]     0   532     7240      179      18       3        0             0 systemd-logind
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174032] [  534]     0   534     7470       52      20       3        0             0 cgmanager
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174034] [  536]     0   536     6512       49      18       3        0             0 atd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174037] [  544]     0   544     6933       71      18       3        0             0 cron
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174039] [  550]   102   550    10762      176      25       3        0          -900 dbus-daemon
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174041] [  600]     0   600     1099       16       7       3        0             0 acpid
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174043] [  621]     0   621    68655      244      36       3        0             0 accounts-daemon
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174045] [  626]   101   626    65157      313      29       3        0             0 rsyslogd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174047] [  670]     0   670     4869       65      15       3        0             0 irqbalance
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174050] [  689]     0   689    13193     2548      29       3        0             0 munin-node
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174052] [  691]     0   691     4031      216      11       3        0             0 dhclient
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174054] [  698]     0   698    69861      342      39       3        0             0 polkitd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174056] [  735]     0   735     9667      180      22       3        0             0 monit
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174058] [ 1228]     0  1228   133721     6781      85       5        0          -500 dockerd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174060] [ 1248]     0  1248   109978     1466     139       4        0             0 php-fpm7.2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174063] [ 1253]     0  1253    43332     2030      52       3        0             0 unattended-upgr
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174065] [ 1255]     0  1255    91709     3642     139       3        0             0 php-fpm7.0
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174067] [ 1321]   107  1321    28735      291      53       3        0             0 opendkim
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174069] [ 1324]     0  1324    16379      177      35       3        0         -1000 sshd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174072] [ 1327]   107  1327    88165      827      62       3        0             0 opendkim
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174074] [ 1352]     0  1352    43196      733      52       5        0             0 nginx
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174076] [ 1353]    33  1353    43196      733      54       5        0             0 nginx
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174078] [ 1354]    33  1354    43196      733      54       5        0             0 nginx
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174080] [ 1355]    33  1355    43196      733      54       5        0             0 nginx
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174087] [ 1362]     0  1362     3664       33      12       3        0             0 agetty
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174089] [ 1371]    33  1371   109978     1327     125       4        0             0 php-fpm7.2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174091] [ 1372]    33  1372   109978     1327     125       4        0             0 php-fpm7.2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174093] [ 1373]    33  1373   112297     9698     150       3        0             0 php-fpm7.0
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174095] [ 1374]    33  1374   112315    10265     151       3        0             0 php-fpm7.0
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174098] [ 1492]     0  1492    93681     2281      40       5        0          -500 docker-containe
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174100] [ 1506]     0  1506    16352      120      24       3        0             0 master
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174102] [ 1974]   111  1974    27509      166      26       3        0             0 ntpd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174105] [27112]    33 27112   111777     8536     150       3        0             0 php-fpm7.0
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174107] [30935]     0 30935   146764     2750      54       4        0             0 fail2ban-server
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174109] [ 6574]     0  6574   122194     3926     179       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174111] [27782]    33 27782    44694     2331      70       3        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174114] [27811]   108 27811    23037      213      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174116] [27812]   108 27812    23037      212      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174118] [27813]   108 27813    23037      211      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174120] [27814]   108 27814    23037      211      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174122] [27815]   108 27815    23037      211      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174124] [27816]   108 27816    23037      218      45       3        0             0 zabbix_agentd
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174127] [27291]   106 27291    16911      131      25       3        0             0 qmgr
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174129] [27413]   106 27413    20177      184      30       3        0             0 tlsmgr
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174131] [26417]   106 26417    16869      112      25       3        0             0 pickup
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174133] [27850]    33 27850   163843    26103     224       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174136] [27856]    33 27856   161347    23458     215       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174138] [27895]    33 27895   162572    24791     221       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174140] [28374]    33 28374   163657    33629     236       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174142] [28377]    33 28377   142967    31279     231       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174144] [28379]    33 28379   144883    32799     235       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174146] [28423]    33 28423   162756    24751     222       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174149] [28429]    33 28429   161587    31448     232       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174151] [28434]    33 28434   161233    23432     219       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174153] [28463]    33 28463   163658    33523     236       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174155] [28464]    33 28464   122401     2826     177       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174157] [28467]    33 28467   143221    31432     231       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174159] [28468]    33 28468   122473     2965     177       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174161] [28469]    33 28469   129756    26267     220       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174164] [28482]    33 28482   161283    22980     219       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174166] [28483]    33 28483   163618    25643     223       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174168] [28484]    33 28484   163367    25141     223       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174170] [28486]    33 28486   143209    31129     231       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174172] [28488]    33 28488   153425    24734     220       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174174] [28492]    33 28492   161351    23526     215       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174176] [28500]    33 28500   163574    25769     221       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174178] [28501]    33 28501   163577    26837     224       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174180] [28503]    33 28503   142863    23211     214       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174182] [28865]    33 28865   161160    23608     218       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174184] [28873]    33 28873   161345    23207     215       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174186] [28890]    33 28890   163621    25344     223       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174188] [28891]    33 28891   122250     2748     166       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174190] [28892]    33 28892   162593    24527     217       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174193] [28894]    33 28894   127346     7830     186       4        0             0 apache2
Oct 11 05:13:34 ip-10-0-x-xxx kernel: [489090.174195] [28899]    33 28899   163557    25511     223       4        0             0 apache2
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.936086] INFO: task apache2:28423 blocked for more than 120 seconds.
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.943206]       Not tainted 4.4.0-157-generic #185-Ubuntu
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.948135] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955958] apache2         D ffff8800e674fc58     0 28423   6574 0x00000000
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955963]  ffff8800e674fc58 0000000000000001 ffff880021adc600 ffff88001dac8000
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955966]  ffff8800e6750000 ffff880034195984 ffff88001dac8000 00000000ffffffff
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955969]  ffff880034195988 ffff8800e674fc70 ffffffff81860585 ffff880034195980
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955971] Call Trace:
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955981]  [<ffffffff81860585>] schedule+0x35/0x80
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955989]  [<ffffffff818608de>] schedule_preempt_disabled+0xe/0x10
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955992]  [<ffffffff81862777>] __mutex_lock_slowpath+0xb7/0x130
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955994]  [<ffffffff8186280f>] mutex_lock+0x1f/0x30
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.955998]  [<ffffffff8122bbf2>] path_openat+0x512/0x1360
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956001]  [<ffffffff8122e611>] do_filp_open+0x91/0x100
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956004]  [<ffffffff8123c2e7>] ? __alloc_fd+0xc7/0x190
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956007]  [<ffffffff8121c8c8>] do_sys_open+0x138/0x2a0
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956009]  [<ffffffff8121ca4e>] SyS_open+0x1e/0x20
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956012]  [<ffffffff81864f1b>] entry_SYSCALL_64_fastpath+0x22/0xcb
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.956026] INFO: task apache2:29254 blocked for more than 120 seconds.
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.962952]       Not tainted 4.4.0-157-generic #185-Ubuntu
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.977383] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.992125] apache2         D ffff880013db3bf8     0 29254   6574 0x00000000
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.992131]  ffff880013db3bf8 0000000000000000 ffffffff81e13500 ffff8800e8701c00
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.992135]  ffff880013db4000 ffff880034195984 ffff8800e8701c00 00000000ffffffff
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.992137]  ffff880034195988 ffff880013db3c10 ffffffff81860585 ffff880034195980
Oct 11 05:14:05 ip-10-0-x-xxx kernel: [489120.992140] Call Trace:

可能的原因: cron在systemd启动干净php会话文件的同时运行session clean? 只是没有正确地清理php会话? 我的公羊太低了?

    free -m #see memory info
              total        used        free      shared  buff/cache   available
Mem:           3949        1710        1397         141         842        1793
Swap:             0           0           0

另外,由于我更新了操作系统,我的站点不再使用php5,但这是迁移过来的更新之前的cron作业,也许这是一个问题? 谢谢!

最佳回复

您可能需要重新检查您的php版本。我在我的ubuntu 16.04中安装了php7.0作为捆绑版本。iirc,php5已被弃用/留用

另外,这表明内存不足:

Apache2调用的OOM杀手:gfp_mask=0x24201ca,order=0, 得分=0

可能添加更多的ram、磁盘交换或至少一个交换文件,看看情况是否有所改善。

总交换量=0KB