2023-09-13
原文作者:https://blog.csdn.net/wangwei19871103/category_9681495_2.html 原文地址: https://blog.csdn.net/wangwei19871103/article/details/104264130

UnpooledDirectByteBuf

202309132200405451.png
这个一看就知道是无池化直接缓冲区啦,跟上面的那些区别就是他的缓冲区是ByteBuffer的直接缓冲区,内部的原理我在这篇文章有讲过了:

        protected ByteBuffer allocateDirect(int initialCapacity) {
            return ByteBuffer.allocateDirect(initialCapacity);
        }

另外一个就是释放的问题:

        protected void freeDirect(ByteBuffer buffer) {
            PlatformDependent.freeDirectBuffer(buffer);
        }
        public static void freeDirectBuffer(ByteBuffer buffer) {
            CLEANER.freeDirectBuffer(buffer);
        }

这里有个CLEANER,好像就是清除器的意思,最后是调用他的freeDirectBuffer清除内存。

CLEANER

他是一个接口类型,定义了直接缓冲区的释放方法。

202309132200412952.png
其实这个还跟平台有关,我们可以PlatformDependent静态代码块的一这段源码:

     if (!isAndroid()) {
                // only direct to method if we are not running on android.
                // See https://github.com/netty/netty/issues/2604
                if (javaVersion() >= 9) {
                    CLEANER = CleanerJava9.isSupported() ? new CleanerJava9() : NOOP;
                } else {
                    CLEANER = CleanerJava6.isSupported() ? new CleanerJava6() : NOOP;
                }
            } else {
                CLEANER = NOOP;
            }

根据不同的java版本,会有不同的实现,分为CleanerJava9CleanerJava6两种,也就是9以及以上的版本用9,其他的用6

CleanerJava6

先说说这个,其实这个就是在这篇文章有讲过的Cleaner的释放方法。我们看看CleanerJava6的静态代码块,他会先申请一个容量为1的直接缓冲区,然后用反射获取他的cleaner属性,再获取cleaner对象以及clean方法,调用一次clean方法,释放掉刚才的内存,然后返回,当然如果支持unsafe的话还可以获得cleaner属性的偏移地址fieldOffset

     static {
            long fieldOffset;
            Method clean;
            Field cleanerField;
            Throwable error = null;
            final ByteBuffer direct = ByteBuffer.allocateDirect(1);
            try {
                Object mayBeCleanerField = AccessController.doPrivileged(new PrivilegedAction<Object>() {
                    @Override
                    public Object run() {
                        try {
                            Field cleanerField =  direct.getClass().getDeclaredField("cleaner");//反射获取cleaner属性
                            if (!PlatformDependent.hasUnsafe()) {
                                // We need to make it accessible if we do not use Unsafe as we will access it via
                                // reflection.
                                cleanerField.setAccessible(true);
                            }
                            return cleanerField;
                        } catch (Throwable cause) {
                            return cause;
                        }
                    }
                });
                if (mayBeCleanerField instanceof Throwable) {
                    throw (Throwable) mayBeCleanerField;
                }
    
                cleanerField = (Field) mayBeCleanerField;
    
                final Object cleaner;
    
                // If we have sun.misc.Unsafe we will use it as its faster then using reflection,
                // otherwise let us try reflection as last resort.
                if (PlatformDependent.hasUnsafe()) {
                    fieldOffset = PlatformDependent0.objectFieldOffset(cleanerField);
                    cleaner = PlatformDependent0.getObject(direct, fieldOffset);
                } else {
                    fieldOffset = -1;
                    cleaner = cleanerField.get(direct);
                }
                clean = cleaner.getClass().getDeclaredMethod("clean");
                clean.invoke(cleaner);//调用一次clean方法,释放内存
            } catch (Throwable t) {
                // We don't have ByteBuffer.cleaner().
                fieldOffset = -1;
                clean = null;
                error = t;
                cleanerField = null;
            }
    
            if (error == null) {
                logger.debug("java.nio.ByteBuffer.cleaner(): available");
            } else {
                logger.debug("java.nio.ByteBuffer.cleaner(): unavailable", error);
            }
            CLEANER_FIELD = cleanerField;//clear对象
            CLEANER_FIELD_OFFSET = fieldOffset;
            CLEAN_METHOD = clean;//获取clear的clean方法
        }

freeDirectBuffer

而他清除方法就是freeDirectBuffer

        @Override
        public void freeDirectBuffer(ByteBuffer buffer) {
            if (!buffer.isDirect()) {
                return;
            }
            if (System.getSecurityManager() == null) {
                try {
                    freeDirectBuffer0(buffer);
                } catch (Throwable cause) {
                    PlatformDependent0.throwException(cause);
                }
            } else {
                freeDirectBufferPrivileged(buffer);
            }
        }

最终是调用:

     private static void freeDirectBuffer0(ByteBuffer buffer) throws Exception {
            final Object cleaner;
            // If CLEANER_FIELD_OFFSET == -1 we need to use reflection to access the cleaner, otherwise we can use
            // sun.misc.Unsafe.
            if (CLEANER_FIELD_OFFSET == -1) {
                cleaner = CLEANER_FIELD.get(buffer);
            } else {
                cleaner = PlatformDependent0.getObject(buffer, CLEANER_FIELD_OFFSET);
            }
            if (cleaner != null) {
                CLEAN_METHOD.invoke(cleaner);//调用clean方法
            }
        }

可见最后就是调用cleanerclean方法啦。

CleanerJava9

他的静态代码块就直接是用反射取出unsafe的方法invokeCleaner,顺便调用一次,把申请的内存先释放了。

        static {
            final Method method;
            final Throwable error;
            if (PlatformDependent0.hasUnsafe()) {
                final ByteBuffer buffer = ByteBuffer.allocateDirect(1);
                Object maybeInvokeMethod = AccessController.doPrivileged(new PrivilegedAction<Object>() {
                    @Override
                    public Object run() {
                        try {
                            // See https://bugs.openjdk.java.net/browse/JDK-8171377
                            Method m = PlatformDependent0.UNSAFE.getClass().getDeclaredMethod(
                                    "invokeCleaner", ByteBuffer.class);
                            m.invoke(PlatformDependent0.UNSAFE, buffer);
                            return m;
                        } catch (NoSuchMethodException e) {
                            return e;
                        } catch (InvocationTargetException e) {
                            return e;
                        } catch (IllegalAccessException e) {
                            return e;
                        }
                    }
                });
    
                if (maybeInvokeMethod instanceof Throwable) {
                    method = null;
                    error = (Throwable) maybeInvokeMethod;
                } else {
                    method = (Method) maybeInvokeMethod;
                    error = null;
                }
            } else {
                method = null;
                error = new UnsupportedOperationException("sun.misc.Unsafe unavailable");
            }
            if (error == null) {
                logger.debug("java.nio.ByteBuffer.cleaner(): available");
            } else {
                logger.debug("java.nio.ByteBuffer.cleaner(): unavailable", error);
            }
            INVOKE_CLEANER = method;
        }

freeDirectBuffer

他的其实就是直接调用invokeCleaner这个方法。

        @Override
        public void freeDirectBuffer(ByteBuffer buffer) {
            // Try to minimize overhead when there is no SecurityManager present.
            // See https://bugs.openjdk.java.net/browse/JDK-8191053.
            if (System.getSecurityManager() == null) {
                try {
                    INVOKE_CLEANER.invoke(PlatformDependent0.UNSAFE, buffer);//反射调用
                } catch (Throwable cause) {
                    PlatformDependent0.throwException(cause);
                }
            } else {
                freeDirectBufferPrivileged(buffer);
            }
        }
Unsafe的invokeCleaner

我们来看看这个方法做了什么,其实也是获得缓冲区的clearn调用clean方法:

        public void invokeCleaner(java.nio.ByteBuffer directBuffer) {
            if (!directBuffer.isDirect())
                throw new IllegalArgumentException("buffer is non-direct");
    
            DirectBuffer db = (DirectBuffer)directBuffer;
            if (db.attachment() != null)
                throw new IllegalArgumentException("duplicate or slice");
    
            Cleaner cleaner = db.cleaner();
            if (cleaner != null) {
                cleaner.clean();
            }
        }

上面两个最终实现是一样的,但是CleanerJava9使用的反射方法比CleanerJava6要少,最后调用的方法也是直接调用的,不是用反射,所以性能要好一点。

设置和获取

我们可以看到这里操作都是直接操作buffer的,内部就是一堆unsafe的方法,直接操作对外内存。

        @Override
        protected void _setInt(int index, int value) {
            buffer.putInt(index, value);
        }
    
        @Override
        protected void _setIntLE(int index, int value) {
            buffer.putInt(index, ByteBufUtil.swapInt(value));
        }
        @Override
        protected int _getInt(int index) {
            return buffer.getInt(index);
        }
    
        @Override
        protected int _getIntLE(int index) {
            return ByteBufUtil.swapInt(buffer.getInt(index));
        }

ByteBufUtil

里面除了字节序转换方法之外还有好多字节缓冲区相关的方法,我们现在只关心字节序的转换,我们来看他内部怎么做的:

202309132200418203.png
基本上是调用了很多类型的reverseBytes方法,也就是把字节翻转,我们来看个Integer的:

        @HotSpotIntrinsicCandidate
        public static int reverseBytes(int i) {
            return (i << 24)            |
                   ((i & 0xff00) << 8)  |
                   ((i >>> 8) & 0xff00) |
                   (i >>> 24);
        }

其实就是做了字节位移取掩码做高低位互换,调用一次可以换一次,即是大端和小端互相切换。

UnpooledUnsafeDirectByteBuf

这个其实就是UnpooledDirectByteBuf的子类,用了unsafe,跟上面的差不多,我就不多说了。

202309132200426274.png

UnpooledUnsafeNoCleanerDirectByteBuf

202309132200434675.png
这个名字一看就是没有清除器的,就是前面讲的Cleaner,那他是怎么释放内存的呢,我们来看看这个类。
主要是这两个方法,其实没什么特备,只是申请和释放内存交给PlatformDependent了,这两个方法是对应的。

    //申请没有清除器的直接缓冲区
        @Override
        protected ByteBuffer allocateDirect(int initialCapacity) {
            return PlatformDependent.allocateDirectNoCleaner(initialCapacity);
        }
    //释放没有清除器的直接缓冲区
        @Override
        protected void freeDirect(ByteBuffer buffer) {
            PlatformDependent.freeDirectNoCleaner(buffer);
        }

PlatformDependent的allocateDirectNoCleaner

        public static ByteBuffer allocateDirectNoCleaner(int capacity) {
            assert USE_DIRECT_BUFFER_NO_CLEANER;
    
            incrementMemoryCounter(capacity);
            try {
                return PlatformDependent0.allocateDirectNoCleaner(capacity);
            } catch (Throwable e) {
                decrementMemoryCounter(capacity);
                throwException(e);
                return null;
            }
        }

PlatformDependent0的allocateDirectNoCleaner

可以看到调用了UNSAFE.allocateMemory,内部最终是调用了本地方法allocateMemory0申请内存,返回一个基地址,注意这里没有调用DirectByteBuffer(int cap),这个里面有很多操作的,这个前面讲过。

        static ByteBuffer allocateDirectNoCleaner(int capacity) {
            // Calling malloc with capacity of 0 may return a null ptr or a memory address that can be used.
            // Just use 1 to make it safe to use in all cases:
            // See: http://pubs.opengroup.org/onlinepubs/009695399/functions/malloc.html
            return newDirectBuffer(UNSAFE.allocateMemory(Math.max(1, capacity)), capacity);
        }

PlatformDependent0的newDirectBuffer

        static ByteBuffer newDirectBuffer(long address, int capacity) {
            ObjectUtil.checkPositiveOrZero(capacity, "capacity");
    
            try {
                return (ByteBuffer) DIRECT_BUFFER_CONSTRUCTOR.newInstance(address, capacity);
            } catch (Throwable cause) {
                // Not expected to ever throw!
                if (cause instanceof Error) {
                    throw (Error) cause;
                }
                throw new Error(cause);
            }
        }

然后通过newDirectBuffer用反射封装成DirectByteBuffer,这里的反射调用了DirectByteBuffer(long addr, int cap),可以到下面的静态代码块里获取的。

202309132200439946.png
而这个构造函数是没有清除器的,也没有一堆其他操作要做:

202309132200448017.png
前面文章讲过DirectByteBuffer(int cap)可是有一堆事情要做呢:

202309132200454788.png

PlatformDependent的freeDirectNoCleaner

        public static void freeDirectNoCleaner(ByteBuffer buffer) {
            assert USE_DIRECT_BUFFER_NO_CLEANER;
    
            int capacity = buffer.capacity();
            PlatformDependent0.freeMemory(PlatformDependent0.directBufferAddress(buffer));//获取地址,释放
            decrementMemoryCounter(capacity);
        }

PlatformDependent0的directBufferAddress

这里最后还是调用了unsafe的方法,其实就是本地方法freeMemory0

       static void freeMemory(long address) {
            UNSAFE.freeMemory(address);
        }

可见UnpooledUnsafeNoCleanerDirectByteBuf要比他的父类都要简洁多了,直接申请和释放内存,不需要清除器帮助,性能能提高不少。不过如果你忘记是释放内存的话,那就很尴尬了,不会有清除器为你释放内存了。

UnpooledUnsafeNoCleanerDirectByteBuf注意的点

注意JDK9以下的默认可以使用这个,以上的就得自己设置io.netty.tryReflectionSetAccessibletrue才可以。因为源码里有限制:

202309132200464239.png

2023091322004680910.png
这个会导致DirectByteBuffer(long addr, int cap)无法访问,抛出异常,自然就没办法用UnpooledUnsafeNoCleanerDirectByteBuf了:

2023091322004738011.png

2023091322004813312.png

2023091322004950213.png
我的是JDK11,所以直接缓冲区默认不是NoCleaner的:

2023091322005024314.png
所以我打算设置下-Dio.netty.tryReflectionSetAccessible=true

2023091322005122615.png
调试发现好了:

2023091322005226216.png
不过也给我警告了,其实关系不大,说设置可访问object.setAccessible(true)非法了,因为DirectByteBuffer(long addr, int cap)是包内私有的,外面不能访问,会报错,所以要设置可访问啦:

2023091322005319417.png
不过这回创建的缓冲区类型对了:

2023091322005408518.png

总结

这次我们讲了堆外缓冲区,讲了是否用unsafe方法操作的,堆外缓冲区的话是否有清除器的,还有一个要注意的就是JDK9以前的默认是无清除器的,因为涉及到反射不安全,不过没清除器的性能好点。JDK9以及以上的只能设置-Dio.netty.tryReflectionSetAccessible=true参数啦。

好了,今天就到这里了,希望对学习理解有帮助,大神看见勿喷,仅为自己的学习理解,能力有限,请多包涵。

阅读全文