为什么JVM实际可用内存比-Xmx指定的少
本文基于Oracle JDK1.8测试。
大家可能都知道,在生产环境中JVM两个最常见的内存参数配置-Xms
与-Xmx
一般要设置为一样大小,防止应用运行期间内存动态调整,影响性能。原本以为这样就可以保证我们的应用从一开始就拥有了-Xmx
指定大小的可用内存了,但是,最近将应用接入Prometheus监控之后,无意间发现监控面板上JVM最大内存与我配置的不一样!我给应用分配的是4GiB内存(-Xms4096m -Xmx4096m
),但是现在监控面板上却显示最大可用内存为3.75GiB,而且这个值还不固定,随着时间推移会发生变化,内心第一反应,是不是Prometheus哪里设置的不对,或者Prometheus采集指标有问题?赶紧翻源代码查看。
一 JVM最大可用内存
在Prometheus Java客户端中,内存信息的采集位于io.prometheus.client.hotspot.MemoryPoolsExports
这个类中,其值是取自java.lang.management.MemoryMXBean#getHeapMemoryUsage()
方法,该方法返回的是一个java.lang.management.MemoryUsage
实例,根据该类的Javadoc描述可知,该类的实例表示内存(堆或非堆)使用的一个快照,进入该类可以发现有四个值:init
,used
,committed
,max
。含义分别如下:init
: 表示JVM启动时的初始值大小,单位为字节。该值随着时间的推移会发生变化(增加或者减少);used
: 表示当前已使用内存大小,单位为字节。committed
: 已提交内存大小。该值是JVM保证可以使用的最大内存大小,单位为字节。已提交内存大小也会随着时间的推移发生变化(增加或者减少);甚至可能会小于init
的值;但保证会大于等于used
;max
: 可用于内存管理的最大内存。该值同样也会随着时间的推移发生变化(增加或者减少)。需要注意的是,该值并不一定是JVM可以使用的内存,当尝试分配的内存大于committed
时会失败,即使仍然满足used
<= max
。
注: Javadoc相关文档-A MemoryUsage object contains four values:
Below is a picture showing an example of a memory pool:
init represents the initial amount of memory (in bytes) that the Java virtual machine requests from the operating system for memory management during startup. The Java virtual machine may request additional memory from the operating system and may also release memory to the system over time. The value of init may be undefined. used represents the amount of memory currently used (in bytes). committed represents the amount of memory (in bytes) that is guaranteed to be available for use by the Java virtual machine. The amount of committed memory may change over time (increase or decrease). The Java virtual machine may release memory to the system and committed could be less than init. committed will always be greater than or equal to used. max represents the maximum amount of memory (in bytes) that can be used for memory management. Its value may be undefined. The maximum amount of memory may change over time if defined. The amount of used and committed memory will always be less than or equal to max if max is defined. A memory allocation may fail if it attempts to increase the used memory such that used > committed even if used <= max would still be true (for example, when the system is low on virtual memory). +----------------------------------------------+ +//////////////// | + +//////////////// | + +----------------------------------------------+ |--------| init |---------------| used |---------------------------| committed |----------------------------------------------| max
到此,我们已经知道,JVM最大可用内存确实是会变化的,总结下来,JVM可用内存主要受committed
影响,且满足:used <= committed <= max
。
二 JVM实际可用内存比-Xmx指定的少
JVM可用内存会发生变化这点已经证实,但是可用内存比我们的配置少是怎么回事?
2.1 缺失的内存去哪儿了
重现问题,使用以下测试代码:
public class HeapSizeTest {
private static final long KIB = 1024;
public static void main(String[] args) {
List<String> commandLineFlags = ManagementFactory.getRuntimeMXBean().getInputArguments();
System.out.println(String.format("CommandLineFlags: %s", commandLineFlags));
MemoryMXBean memoryBean = ManagementFactory.getMemoryMXBean();
printMemory(memoryBean);
}
private static void printMemory(MemoryMXBean memoryBean) {
MemoryUsage heapUsage = memoryBean.getHeapMemoryUsage();
System.out.println(String.format(" init: %s B = %s KiB", heapUsage.getInit(), heapUsage.getInit() / KIB));
System.out.println(String.format(" used: %s B = %s KiB", heapUsage.getUsed(), heapUsage.getUsed() / KIB));
System.out.println(String.format("commited: %s B = %s KiB", heapUsage.getCommitted(), heapUsage.getCommitted() / KIB));
System.out.println(String.format(" max: %s B = %s KiB", heapUsage.getMax(), heapUsage.getMax() / KIB));
System.out.println();
// 作为对比,使用另外一种API:
// 根据结果可以得出对应关系:
// heapUsage.getCommitted() = Runtime.getRuntime().totalMemory()
// heapUsage.getMax() = Runtime.getRuntime().maxMemory()
// heapUsage.getUsed() = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()
System.out.println(String.format(" Runtime.free: %s B = %s KiB", Runtime.getRuntime().freeMemory(), Runtime.getRuntime().freeMemory() / KIB));
System.out.println(String.format(" Runtime.total: %s B = %s KiB", Runtime.getRuntime().totalMemory(), Runtime.getRuntime().totalMemory() / KIB));
System.out.println(String.format(" Runtime.max: %s B = %s KiB", Runtime.getRuntime().maxMemory(), Runtime.getRuntime().maxMemory() / KIB));
System.out.println("===============================");
}
}
运行以下命令:
java \
-Xms1024m \
-Xmx1024m \
-XX:MetaspaceSize=8m \
-XX:NewRatio=1 \
-XX:SurvivorRatio=8 \
-XX:+UseParallelGC \
-XX:+PrintGCDetails \
HeapSizeTest
输出结果如下:
CommandLineFlags: [-Xms1024m, -Xmx1024m, -XX:MetaspaceSize=8m, -XX:NewRatio=1, -XX:SurvivorRatio=8, -XX:+UseParallelGC, -XX:+PrintGCDetails]
init: 1073741824 B = 1048576 KiB
used: 17196680 B = 16793 KiB
commited: 1020264448 B = 996352 KiB
max: 1020264448 B = 996352 KiB
Runtime.free: 1003067768 B = 979558 KiB
Runtime.total: 1020264448 B = 996352 KiB
Runtime.max: 1020264448 B = 996352 KiB
===============================
Heap
PSYoungGen total 472064K, used 25190K [0x00000000e0000000, 0x0000000100000000, 0x0000000100000000)
eden space 419840K, 6% used [0x00000000e0000000,0x00000000e1899b30,0x00000000f9a00000)
from space 52224K, 0% used [0x00000000fcd00000,0x00000000fcd00000,0x0000000100000000)
to space 52224K, 0% used [0x00000000f9a00000,0x00000000f9a00000,0x00000000fcd00000)
ParOldGen total 524288K, used 0K [0x00000000c0000000, 0x00000000e0000000, 0x00000000e0000000)
object space 524288K, 0% used [0x00000000c0000000,0x00000000c0000000,0x00000000e0000000)
Metaspace used 2753K, capacity 4486K, committed 4864K, reserved 1056768K
class space used 299K, capacity 386K, committed 512K, reserved 1048576K
从结果中可以看出:eden + from(S0) + to(S1) + ParOldGen =
419840K + 52224K + 52224K + 524288K = 1048576K = 1024M
这似乎没问题,但注意仔细看上面输出的第12行:
PSYoungGen total 472064K, used 25190K [0x00000000e0000000, 0x0000000100000000, 0x0000000100000000)
其中显示年轻代大小为:PSYoungGen = 472064K
,即461M
,通常我们认为Young = eden + from(S0) + to(S1)
,即:Young = 419840K + 52224K + 52224K = 524288K = 512M
,差异出现了:我们发现GC日志中所谓的年轻代大小实际上是按照PSYoungGen total = eden + from(或者 to)
来计算的,即:Young = 419840K + 52224K = 472064K = 461M
如此来计算HeapSize
的话,可以得出:HeapSize = Young + old = 472064K + 524288K = 996352K = 973M
该值也恰好与heapUsage.getMax()
和Runtime.getRuntime().maxMemory()
获取的值相等:
max: 1020264448 B = 996352 KiB
Runtime.max: 1020264448 B = 996352 KiB
至此我们发现,原来JVM最大内存中是少了一个S区的大小。另外,在这篇帖子Why does my JVM have access to less memory than -Xmx specifies?中,也得到了进一步证实,其中,在对应的OpenJDK8的collectedHeap.hpp
源代码中有一个方法max_capacity()
描述如下:
// Support for java.lang.Runtime.maxMemory(): return the maximum amount of
// memory that the vm could make available for storing 'normal' java objects.
// This is based on the reserved address space, but should not include space
// that the vm uses internally for bookkeeping or temporary storage
// (e.g., in the case of the young gen, one of the survivor
// spaces).
virtual size_t max_capacity() const = 0;
我们示例中是使用的并行垃圾收集器:-XX:+UseParallelGC
,这种垃圾收集器是采用分代思想,大致如下:
当指定-Xms
参数的值等于-Xmx
参数的值时,commited=max
,意味着在虚拟机初始化时,将保留堆的整个空间。如果-Xms
参数的值小于-Xmx
参数的值,则并非所有保留的空间都会立即提交给虚拟机。不过要注意的是,前面已经提到过,commited
和max
的值并不是固定的,会随着时间发生变化。
2.2 不同垃圾收集器的行为表现:
前面我们看了ParallelGC
的行为,那么其他垃圾收集器表现如何呢,继续测试如下:
GC算法 | commited | max |
---|---|---|
-XX:+UseSerialGC | 996160 KiB | 996160 KiB |
-XX:+UseParallelGC | 996352 KiB | 996352 KiB |
-XX:+UseConcMarkSweepGC | 996160 KiB | 996160 KiB |
-XX:+UseG1GC | 1048576 KiB | 1048576 KiB |
可以发现,只有G1垃圾收集器表现比较特殊,max
输出与-Xmx
一致,这应该是与G1算法特殊的分代思想有关,简单来说,在G1算法中,堆被划分为一组大小相等的堆区域,每个堆区域都有一个连续的虚拟内存范围,与其他常见分代有着明显区别。
最后,附上每种测试结果输出:
-XX:+UseSerialGC:
CommandLineFlags: [-Xms1024m, -Xmx1024m, -XX:MetaspaceSize=8m, -XX:NewRatio=1, -XX:SurvivorRatio=8, -XX:+UseSerialGC, -XX:+PrintGCDetails]
init: 1073741824 B = 1048576 KiB
used: 17180952 B = 16778 KiB
commited: 1020067840 B = 996160 KiB
max: 1020067840 B = 996160 KiB
Runtime.free: 1002886888 B = 979381 KiB
Runtime.total: 1020067840 B = 996160 KiB
Runtime.max: 1020067840 B = 996160 KiB
===============================
Heap
def new generation total 471872K, used 25167K [0x00000000c0000000, 0x00000000e0000000, 0x00000000e0000000)
eden space 419456K, 6% used [0x00000000c0000000, 0x00000000c1893f08, 0x00000000d99a0000)
from space 52416K, 0% used [0x00000000d99a0000, 0x00000000d99a0000, 0x00000000dccd0000)
to space 52416K, 0% used [0x00000000dccd0000, 0x00000000dccd0000, 0x00000000e0000000)
tenured generation total 524288K, used 0K [0x00000000e0000000, 0x0000000100000000, 0x0000000100000000)
the space 524288K, 0% used [0x00000000e0000000, 0x00000000e0000000, 0x00000000e0000200, 0x0000000100000000)
Metaspace used 2753K, capacity 4486K, committed 4864K, reserved 1056768K
class space used 299K, capacity 386K, committed 512K, reserved 1048576K
-XX:+UseParallelGC:
CommandLineFlags: [-Xms1024m, -Xmx1024m, -XX:MetaspaceSize=8m, -XX:NewRatio=1, -XX:SurvivorRatio=8, -XX:+UseParallelGC, -XX:+PrintGCDetails]
init: 1073741824 B = 1048576 KiB
used: 17196680 B = 16793 KiB
commited: 1020264448 B = 996352 KiB
max: 1020264448 B = 996352 KiB
Runtime.free: 1003067768 B = 979558 KiB
Runtime.total: 1020264448 B = 996352 KiB
Runtime.max: 1020264448 B = 996352 KiB
===============================
Heap
PSYoungGen total 472064K, used 25190K [0x00000000e0000000, 0x0000000100000000, 0x0000000100000000)
eden space 419840K, 6% used [0x00000000e0000000,0x00000000e1899b30,0x00000000f9a00000)
from space 52224K, 0% used [0x00000000fcd00000,0x00000000fcd00000,0x0000000100000000)
to space 52224K, 0% used [0x00000000f9a00000,0x00000000f9a00000,0x00000000fcd00000)
ParOldGen total 524288K, used 0K [0x00000000c0000000, 0x00000000e0000000, 0x00000000e0000000)
object space 524288K, 0% used [0x00000000c0000000,0x00000000c0000000,0x00000000e0000000)
Metaspace used 2753K, capacity 4486K, committed 4864K, reserved 1056768K
class space used 299K, capacity 386K, committed 512K, reserved 1048576K
-XX:+UseConcMarkSweepGC:
CommandLineFlags: [-Xms1024m, -Xmx1024m, -XX:MetaspaceSize=8m, -XX:NewRatio=1, -XX:SurvivorRatio=8, -XX:+UseConcMarkSweepGC, -XX:+PrintGCDetails]
init: 1073741824 B = 1048576 KiB
used: 17180952 B = 16778 KiB
commited: 1020067840 B = 996160 KiB
max: 1020067840 B = 996160 KiB
Runtime.free: 1002886888 B = 979381 KiB
Runtime.total: 1020067840 B = 996160 KiB
Runtime.max: 1020067840 B = 996160 KiB
===============================
Heap
par new generation total 471872K, used 25167K [0x00000000c0000000, 0x00000000e0000000, 0x00000000e0000000)
eden space 419456K, 6% used [0x00000000c0000000, 0x00000000c1893f08, 0x00000000d99a0000)
from space 52416K, 0% used [0x00000000d99a0000, 0x00000000d99a0000, 0x00000000dccd0000)
to space 52416K, 0% used [0x00000000dccd0000, 0x00000000dccd0000, 0x00000000e0000000)
concurrent mark-sweep generation total 524288K, used 0K [0x00000000e0000000, 0x0000000100000000, 0x0000000100000000)
Metaspace used 2753K, capacity 4486K, committed 4864K, reserved 1056768K
class space used 299K, capacity 386K, committed 512K, reserved 1048576K
-XX:+UseG1GC:
CommandLineFlags: [-Xms1024m, -Xmx1024m, -XX:MetaspaceSize=8m, -XX:NewRatio=1, -XX:SurvivorRatio=8, -XX:+UseG1GC, -XX:+PrintGCDetails]
init: 1073741824 B = 1048576 KiB
used: 0 B = 0 KiB
commited: 1073741824 B = 1048576 KiB
max: 1073741824 B = 1048576 KiB
Runtime.free: 1072693264 B = 1047552 KiB
Runtime.total: 1073741824 B = 1048576 KiB
Runtime.max: 1073741824 B = 1048576 KiB
===============================
Heap
garbage-first heap total 1048576K, used 1024K [0x00000000c0000000, 0x00000000c0102000, 0x0000000100000000)
region size 1024K, 2 young (2048K), 0 survivors (0K)
Metaspace used 2755K, capacity 4486K, committed 4864K, reserved 1056768K
class space used 300K, capacity 386K, committed 512K, reserved 1048576K
(完)
版权声明:知识共享署名-非商用-非衍生 (CC BY-NC-ND 3.0) 转载请注明出处