【JVM调优】调优命令/线上调优分析

 2023-01-15
原文作者:疯狂小周 原文地址:https://juejin.cn/post/6897139235053109255

常用 JVM 调优命令

JPS --- 查看程序对应的PID

    [root@ams_cms01 tes]# jps
    25841 Starter
    26691 jar
    13332 jar
    20134 BrokerStartup
    6361 jar
    18668 NamesrvStartup

JMAP ---查找线程内存使用情况

JMAP打印线程内类实例数和内存量

    // 可以查看线程中类的实例数和内存量
    jmap -histo 13332
     num     #instances         #bytes  class name
    ----------------------------------------------
       1:         15760       47434048  [B
       2:        161343       21937496  [C
       3:         48290        7713600  [Ljava.lang.Object;
       4:        140621        3374904  java.lang.String
       5:         29745        2617560  java.lang.reflect.Method
       6:         11216        1876680  [I
       7:          2418        1586208  io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue
       8:         14127        1571504  java.lang.Class
       9:         47662        1525184  java.util.concurrent.ConcurrentHashMap$Node
      10:         27057        1103008  [Ljava.lang.String;
      11:         31809        1017888  java.lang.ref.WeakReference

JMAP打印堆栈信息

    jmap -heap 13332
    Attaching to process ID 13332, please wait...
    Debugger attached successfully.
    Server compiler detected.
    JVM version is 25.172-b11
    
    using thread-local object allocation.
    Parallel GC with 13 thread(s)
    
    Heap Configuration:
       MinHeapFreeRatio         = 0
       MaxHeapFreeRatio         = 100
       MaxHeapSize              = 268435456 (256.0MB)
       NewSize                  = 44564480 (42.5MB)
       MaxNewSize               = 89128960 (85.0MB)
       OldSize                  = 89653248 (85.5MB)
       NewRatio                 = 2
       SurvivorRatio            = 8
       MetaspaceSize            = 21807104 (20.796875MB)
       CompressedClassSpaceSize = 1073741824 (1024.0MB)
       MaxMetaspaceSize         = 17592186044415 MB
       G1HeapRegionSize         = 0 (0.0MB)
    
    Heap Usage:
    PS Young Generation
    Eden Space:
       capacity = 41418752 (39.5MB)
       used     = 15461184 (14.74493408203125MB)
       free     = 25957568 (24.75506591796875MB)
       37.32894704311709% used
    From Space:
       capacity = 1572864 (1.5MB)
       used     = 700480 (0.66802978515625MB)
       free     = 872384 (0.83197021484375MB)
       44.535319010416664% used
    To Space:
       capacity = 1572864 (1.5MB)
       used     = 0 (0.0MB)
       free     = 1572864 (1.5MB)
       0.0% used
    PS Old Generation
       capacity = 133169152 (127.0MB)
       used     = 109122936 (104.06774139404297MB)
       free     = 24046216 (22.93225860595703MB)
       81.94310345987635% used

Jstack --- 查看线程情况

    jstack 13332
    "DubboServerHandler-172.10.0.7:28999-thread-79" #539 daemon prio=5 os_prio=0 tid=0x00007eff2c05e000 nid=0x1164 waiting on condition [0x00007efe3b9f8000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000f25f76b8> (a java.util.concurrent.SynchronousQueue$TransferStack)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
            at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
            at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)
    
    "DubboServerHandler-172.10.0.7:28999-thread-78" #534 daemon prio=5 os_prio=0 tid=0x00007eff2c05d000 nid=0x939 waiting on condition [0x00007efe3bbfa000]
       java.lang.Thread.State: WAITING (parking)
            at sun.misc.Unsafe.park(Native Method)
            - parking to wait for  <0x00000000f25f76b8> (a java.util.concurrent.SynchronousQueue$TransferStack)
            at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
            at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)
            at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
            at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)
            at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            at java.lang.Thread.run(Thread.java:748)

JSTAT --- 调优重点:查看 GC 情况

    // 监控pid = 13332 的线程,每过 1 秒打印 1 次,打印 10 次
    jstat -gc 13332 1000 10
     S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT     GCT   
    1536.0 1536.0 448.0   0.0   40448.0   5105.7   130048.0   106565.4  79664.0 75844.2 9776.0 9048.1   4082   33.338  248    31.693   65.032
    1536.0 1536.0 448.0   0.0   40448.0   5109.8   130048.0   106565.4  79664.0 75844.2 9776.0 9048.1   4082   33.338  248    31.693   65.032
    1536.0 1536.0 448.0   0.0   40448.0   5225.4   130048.0   106565.4  79664.0 75844.2 9776.0 9048.1   4082   33.338  248    31.693   65.032
    1536.0 1536.0 448.0   0.0   40448.0   5225.4   130048.0   106565.4  79664.0 75844.2 9776.0 9048.1   4082   33.338  248    31.693   65.032

字段意义

    S0C:第一个幸存区的大小,单位KB
    S1C:第二个幸存区的大小
    S0U:第一个幸存区的使用大小
    S1U:第二个幸存区的使用大小
    EC:伊甸园区的大小
    EU:伊甸园区的使用大小
    OC:老年代大小
    OU:老年代使用大小
    MC:方法区大小(元空间)
    MU:方法区使用大小
    CCSC:压缩类空间大小
    CCSU:压缩类空间使用大小
    YGC:年轻代垃圾回收次数
    YGCT:年轻代垃圾回收消耗时间,单位s
    FGC:老年代垃圾回收次数
    FGCT:老年代垃圾回收消耗时间,单位s
    GCT:垃圾回收消耗总时间,单位s

调用命令的使用场景

JMAP

应用出现OOM等内存问题时,可以通过JMAP分析

Jstack

应用出现死锁等线程问题时,可以使用

线上调优分析

上个月有个新项目上线了,中间好像也没什么告警报错。
今天上生产环境查看运行情况时发现,该应用已经进行了近250次 Full GC 了。平均3个小时发生一次!emmm.... 好在还没人投诉。

202301011658394831.png

jmap -heap 查看程序内存:

    Heap Configuration:
       MinHeapFreeRatio         = 0
       MaxHeapFreeRatio         = 100
       MaxHeapSize              = 268435456 (256.0MB)
       NewSize                  = 44564480 (42.5MB)
       MaxNewSize               = 89128960 (85.0MB)
       OldSize                  = 89653248 (85.5MB)
       NewRatio                 = 2
       SurvivorRatio            = 8
       MetaspaceSize            = 21807104 (20.796875MB)
       CompressedClassSpaceSize = 1073741824 (1024.0MB)
       MaxMetaspaceSize         = 17592186044415 MB
       G1HeapRegionSize         = 0 (0.0MB)

202301011658403802.png
小小的脑袋,大大的疑惑,为啥只有256M的堆内存?!

通过 JSTAT -GC 命令发现在每日高峰期,平均每分钟产生 20M 的对象,会在2-3分钟占满Eden区,并触发Minor GC,会有 8M 上下的存活对象被移到Survivor区(2 * 15M)。
下次 Minor GC时,会进行 对象动态年龄判断 ,发现上次一Minor GC 产生的对象已经超过S1区的 50% ,此时会将上次Minor GC存活的对象直接移入老年代(100M)。
所以高峰期,老年代会在30分钟内被占满,并进行一次 Full GC

所以通过以上分析,造成应用程序频繁GC的原因有两个:

  1. 堆内存太小。
  2. 对象动态年龄判断,导致对象过早的被放进老年代。

解决方案:

  1. 扩大堆内存大小:-Xms 1024M -Xmx 1024M
  2. 由于应用程序中的对象基本上都是 ”朝生夕死“,所以老年代没有太大的必要,所以修改新生代和老年代的比例/或者扩大大新生代的大小:-XX:NewRatio=1 / -Xmn600M