UDN-企业互联网技术人气社区

板块导航

浏览  : 1428
回复  : 0

[教程技巧] 使用Docker容器时不要忘记进行GC参数审查

[复制链接]
cat77的头像 楼主
  15年上半年在做统一服务优化时发现使用容器时垃圾回收偏长,而我们的服务是一个响应时间优先的应用,需要对GC进行一些调优,在对容器内Java应用进行GC时发现:

  (8核)如下是8核机器的GC,CMS,young gc基本在100ms左右
  1. [GC (Allocation Failure) 1540815K->708720K(1992320K), 0.1204106 secs]
  2. [GC (Allocation Failure) 1547632K->718037K(1992320K), 0.1039874 secs]
  3. [GC (Allocation Failure) 1568245K->734990K(1992320K), 0.0979778 secs]
  4. [GC (Allocation Failure) 1573902K->739338K(1992320K), 0.1270793 secs]
  5. [GC (Allocation Failure) 1578250K->743388K(1992320K), 0.1074495 secs]
  6. [GC (Allocation Failure) 1582300K->749564K(1992320K), 0.1199881 secs]
  7. [GC (Allocation Failure) 1588476K->752993K(1992320K), 0.1134528 secs]
  8. [GC (Allocation Failure) 1591905K->761514K(1992320K), 0.1208703 secs]
  9. [GC (Allocation Failure) 1600426K->762095K(1992320K), 0.1214570 secs]
  10. [GC (Allocation Failure) 1601007K->764275K(1992320K), 0.1035069 secs]
  11. [GC (CMS Initial Mark) 775325K(1992320K), 0.0122687 secs]
  12. [GC (CMS Final Remark) 1174093K(1992320K), 0.2219463 secs]
  13. [GC (Allocation Failure) 1075577K->241414K(1992320K), 0.0959227 secs]
  14. [GC (Allocation Failure) 1080326K->246942K(1992320K), 0.1073993 secs]
复制代码

  (8核)经发现与GC线程有关,通过指定并发GC线程数 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=4,得到
  1. [GC (Allocation Failure) 1631622K->798323K(1992320K), 0.0408915 secs]
  2. [GC (Allocation Failure) 1637235K->797560K(1992320K), 0.0423643 secs]
  3. [GC (Allocation Failure) 1636472K->803864K(1992320K), 0.0422181 secs]
  4. [GC (Allocation Failure) 1642776K->813965K(1992320K), 0.0429882 secs]
  5. [GC (Allocation Failure) 1686836K->850230K(1992320K), 0.0521587 secs]
  6. [GC (Allocation Failure) 1689142K->858658K(1992320K), 0.0496138 secs]
  7. [GC (Allocation Failure) 1697570K->858148K(1992320K), 0.0439780 secs]
  8. [GC (Allocation Failure) 1697059K->865706K(1992320K), 0.0517013 secs]
  9. [GC (CMS Initial Mark) 865828K(1992320K), 0.0184015 secs]
  10. [GC (CMS Final Remark) 1198017K(1992320K), 0.1831652 secs]
  11. [GC (Allocation Failure) 1132594K->301826K(1992320K), 0.0444428 secs]
  12. [GC (Allocation Failure) 1140738K->305794K(1992320K), 0.0428419 secs]
  13. [GC (Allocation Failure) 1144679K->312688K(1992320K), 0.0545760 secs]
  14. [GC (Allocation Failure) 1151600K->320109K(1992320K), 0.0410423 secs]
  15. [GC (Allocation Failure) 1159021K->320191K(1992320K), 0.0411587 secs]
  16. [GC (Allocation Failure) 1159103K->330729K(1992320K), 0.0387938 secs]
  17. [GC (Allocation Failure) 1169641K->331256K(1992320K), 0.0393934 secs]
  18. [GC (Allocation Failure) 1170163K->343068K(1992320K), 0.0419092 secs]
复制代码

  (16核)而下边是-XX:ParallelGCThreads=16 -XX:ConcGCThreads=8
  1. [GC (Allocation Failure) 1551548K->713703K(1992320K), 0.0400264 secs]
  2. [GC (Allocation Failure) 1552615K->716536K(1992320K), 0.0463409 secs]
  3. [GC (Allocation Failure) 1555448K->723921K(1992320K), 0.0313725 secs]
  4. [GC (Allocation Failure) 1562833K->728720K(1992320K), 0.0333375 secs]
  5. [GC (Allocation Failure) 1567629K->734094K(1992320K), 0.0352418 secs]
  6. [GC (Allocation Failure) 1573006K->739880K(1992320K), 0.0368524 secs]
  7. [GC (Allocation Failure) 1578792K->741607K(1992320K), 0.0418331 secs]
  8. [GC (Allocation Failure) 1580519K->749161K(1992320K), 0.0379511 secs]
  9. [GC (Allocation Failure) 1588073K->755358K(1992320K), 0.0347147 secs]
  10. [GC (Allocation Failure) 1594270K->761399K(1992320K), 0.0379215 secs]
  11. [GC (Allocation Failure) 1600311K->765948K(1992320K), 0.0348537 secs]
  12. [GC (Allocation Failure) 1604858K->774675K(1992320K), 0.0371332 secs]
  13. [GC (CMS Initial Mark) 774724K(1992320K), 0.0120999 secs]
  14. [GC (CMS Final Remark) 1067302K(1992320K), 0.1657415 secs]
  15. [GC (Allocation Failure) 1130442K->286546K(1992320K), 0.0299338 secs]
  16. [GC (Allocation Failure) 1125458K->298132K(1992320K), 0.0328611 secs]
  17. [GC (Allocation Failure) 1137044K->304370K(1992320K), 0.0357785 secs]
复制代码

  (8核)而如果换成UseParallelGC,young gc不错,而full gc偏慢,线上如果没有配置CMS的话,默认是用这个
  1. [GC (Allocation Failure) 1993894K->1000058K(2072576K), 0.0396914 secs]
  2. [GC (Allocation Failure) 1999482K->1007021K(2072576K), 0.0403703 secs]
  3. [GC (Allocation Failure) 2006445K->1011597K(2068480K), 0.0432666 secs]
  4. [GC (Allocation Failure) 2006925K->1016275K(2070528K), 0.0450037 secs]
  5. [GC (Allocation Failure) 2011603K->1020929K(2070528K), 0.0490808 secs]
  6. [GC (Allocation Failure) 2016257K->1025389K(2070528K), 0.0395652 secs]
  7. [GC (Allocation Failure) 2020717K->1029915K(2070528K), 0.0415800 secs]
  8. [GC (Allocation Failure) 2025243K->1035623K(2070528K), 0.0534249 secs]
  9. [GC (Allocation Failure) 2030951K->1041520K(2072576K), 0.0473277 secs]
  10. [GC (Allocation Failure) 2040944K->1046783K(2072576K), 0.0447377 secs]
  11. [GC (Allocation Failure) 2046207K->1052829K(2068480K), 0.0466185 secs]
  12. [GC (Allocation Failure) 2048157K->1057749K(2070528K), 0.0435571 secs]
  13. [GC (Allocation Failure) 2053077K->1063253K(2070528K), 0.0434343 secs]
  14. [GC (Allocation Failure) 2058581K->1067759K(2070528K), 0.0402552 secs]
  15. [Full GC (Ergonomics) 1067759K->235608K(2070528K), 0.4808497 secs]
  16. [GC (Allocation Failure) 1230936K->247560K(2070528K), 0.0318850 secs]
  17. [GC (Allocation Failure) 1242888K->254827K(2070528K), 0.0340009 secs]
  18. [GC (Allocation Failure) 1250155K->259247K(2066432K), 0.0337921 secs]
  19. [GC (Allocation Failure) 1250479K->264965K(2068480K), 0.0585058 secs]
复制代码

  虽然GC不如物理机,但是可以满足我们的需要了。

  注:

  1、因为容器不是物理隔离的,比如使用Runtime.getRuntime().availableProcessors() ,会拿到物理CPU个数,而不是容器申请时的个数,

  2、CMS在算GC线程时默认是根据物理CPU算的:(http://mail.openjdk.java.net/pipermail/hotspot-gc-dev/2013-November/008952.html
年轻代的并行线程数(ParallelGCThreads):
  1.     return (ncpus <= 8) ? ncpus : MAX2(8, ncpus / 2)  
复制代码

  CMS并行线程数(ConcGCThreads):
  1. if (AdjustGCThreadsToCores) {  
  2.   FLAG_SET_DEFAULT(ConcGCThreads, ParallelGCThreads / 2);  
  3. } else {  
  4.   FLAG_SET_DEFAULT(ConcGCThreads, (3 + ParallelGCThreads) / 4);  
  5. }  
复制代码

  因为我们升级到了JDK8/Tomcat8, 遇到以下问题:

  1、Tomcat8使用Nio2时有时候会遇到如下警告,暂时没有升级上去
  1. org.apache.tomcat.util.net.AbstractEndpoint countDownConnection
  2. WARNING: Incorrect connection count, multiple socket.close called on the same socket.
复制代码

  2、使用G1垃圾回收时,在我们的小堆上没有多大优势,暂时没有使用。

  我们线上某服务的JVM参数
  1. export JAVA_OPTS="-Djava.library.path=/usr/local/lib -server -XX:ReservedCodeCacheSize=64m -XX:TLABWasteTargetPercent=10 -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:+CMSClassUnloadingEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UnlockDiagnosticVMOptions -XX:ParallelGCThreads=8 -XX:ConcGCThreads=4 -Xss256k -server -Xms2g -Xmx2g -XX:MaxDirectMemorySize=256m -XX:MaxTenuringThreshold=3 -XX:NewRatio=1 -XX:SurvivorRatio=8 -XX:+UnlockDiagnosticVMOptions -XX:ParGCCardsPerStrideChunk=32768 -XX:+AlwaysPreTouch
复制代码

原文作者:张开涛 来源:开涛的博客

相关帖子

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关于我们
联系我们
  • 电话:010-86393388
  • 邮件:udn@yonyou.com
  • 地址:北京市海淀区北清路68号
移动客户端下载
关注我们
  • 微信公众号:yonyouudn
  • 扫描右侧二维码关注我们
  • 专注企业互联网的技术社区
版权所有:用友网络科技股份有限公司82041 京ICP备05007539号-11 京公网网备安1101080209224 Powered by Discuz!
快速回复 返回列表 返回顶部