问题描述

线上的服务过段时间就会出现 oom 的错误，但是日志不太够，所以记录下整个问题的排查过程，用于后续进行借鉴。

常用命令

top

查询CPU占用情况以及进程PID

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+ COMMAND
14235 work      20   0 1550m  77m 3988 S  8.0  0.0  21226:25 instance_inspec
47449 root      39  19 1934m 207m  28m S  4.0  0.1   2957:30 webdir-agent
22546 work      20   0 1392m 232m 7256 S  2.7  0.1  33983:04 node /home/work
3752 root      39  19  802m  71m 9408 S  1.7  0.0   1458:30 baas_agent
26618 root      20   0 3779m  66m 6004 S  1.3  0.0 105:56.94 radar-agent-cli
5871 work      20   0 1283m 364m 7132 S  1.0  0.2  35418:49 node /home/work
30848 root      20   0  864m  25m  12m S  1.0  0.0 240:32.29 radar-agent-ser
2272 root      20   0 2269m 206m 7996 S  0.7  0.1 107:50.20 naming-agent
13171 root      20   0 52128  41m 2052 S  0.7  0.0   1452:10 uwsgi
13277 root      20   0  701m 6108 4360 S  0.7  0.0 429:44.24 monitor_gianod.
3588 root      20   0  700m 5924 4336 S  0.3  0.0 242:46.84 monitor_baas_ag
7764 root      20   0  957m  38m 9292 S  0.3  0.0   4607:52 agent.hosteye
7810 root      39  19 5862m  79m  12m S  0.3  0.0 757:45.74 argus-agent
8303 root      39  19 4982m  30m 1012 S  0.3  0.0  41:15.63 noah-client

jmap -heap pid

显示Java堆详细信息，打印一个堆的摘要信息，包括使用的GC算法、堆配置信息和各内存区域内存使用信息

Attaching to process ID 10, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.191-b12

using thread-local object allocation.
Parallel GC with 4 thread(s)

Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 6442450944 (6144.0MB)
   NewSize                  = 2147483648 (2048.0MB)
   MaxNewSize               = 2147483648 (2048.0MB)
   OldSize                  = 4294967296 (4096.0MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 1989672960 (1897.5MB)
   used     = 637523032 (607.9893417358398MB)
   free     = 1352149928 (1289.5106582641602MB)
   32.041599037461914% used
From Space:
   capacity = 79167488 (75.5MB)
   used     = 21287720 (20.301551818847656MB)
   free     = 57879768 (55.198448181152344MB)
   26.88947260774524% used
To Space:
   capacity = 78643200 (75.0MB)
   used     = 0 (0.0MB)
   free     = 78643200 (75.0MB)
   0.0% used
PS Old Generation
   capacity = 4294967296 (4096.0MB)
   used     = 53744560 (51.25480651855469MB)
   free     = 4241222736 (4044.7451934814453MB)
   1.2513380497694016% used

35830 interned Strings occupying 3985440 bytes.

jmap -histo:live pid

显示堆中对象的统计信息，其中包括每个Java类、对象数量、内存大小(单位：字节)、完全限定的类名。打印的虚拟机内部的类名称将会带有一个’*’前缀。如果指定了live子选项，则只计算活动的对象。

num     #instances         #bytes  class name
----------------------------------------------
  1:        313902       20147600  [C
  2:        324481        7787544  java.lang.String
  3:          8832        3892128  [B
  4:         36272        3191936  java.lang.reflect.Method
  5:          9066        2024104  [I
  6:         62343        1994976  java.util.HashMap$Node
  7:         62213        1990816  java.util.concurrent.ConcurrentHashMap$Node
  8:         14484        1610080  java.lang.Class
  9:         24598        1497704  [Ljava.lang.Object;
 10:          4082        1077648  com.auto.hr.ppi.base.ppitask.domain.DO.TEmployeeDO
 11:          9091        1069376  [Ljava.util.HashMap$Node;
 12:         43927        1054248  java.util.Date
 13:         34158         754920  [Ljava.lang.Class;
 14:         15543         621720  java.util.LinkedHashMap$Entry
 15:           490         515568  [Ljava.util.concurrent.ConcurrentHashMap$Node;
 16:          6889         496008  java.lang.reflect.Field
 17:          8731         488936  java.util.LinkedHashMap
 18:         30178         482848  java.lang.Object
 19:          6300         403200  com.mysql.jdbc.ConnectionPropertiesImpl$BooleanConnectionProperty
 20:          4756         380480  java.lang.reflect.Constructor
 21:          7618         365664  java.util.HashMap

jmap -dump:format=b,file=heapdump.bin pid

生成堆转储快照dump文件

sh-4.2# jmap -dump:format=b,file=heapdump.bin 10
Dumping heap to /a-one/bin/heapdump.bin ...
Heap dump file created

sz heapdump.bin

下载导出的堆快照dump文件，导入到MAT工具中

image_mat

转载自：https://www.cnblogs.com/liuzhiyue/p/14931162.html

参考链接

https://www.cnblogs.com/operationhome/p/10537018.html https://www.cnblogs.com/AloneSword/p/3821569.html