JVM之类的热替换原理解读

一:引言

15年末,正好是Android热修复大火的时候,各种热修复技术从出不穷。

但是其实在服务端,热替换是个老概念了,一点也不新鲜,相比于客户端的各种hook&trick,今天我要说的其实就是JVM原生支持的类热替换的一种方法,其实就是利用Instrumention 和 ClassFileTransformer 来达到真正类的热替换,这个技术已经被广泛应用在Server热部署等领域,前段时间我在研究如何动态插入代码的过程中,刚好用上了这个技术,就顺便研究了下底层原理。

二:还是先讲讲怎么用吧

一上来就说原理还是不怎么合适的,先给大家讲下这个技术怎么用吧。但是这篇文章重点不是讲怎么用,所以我只讲个大概流程。

第一步:写个Agent类,获取Instrumentation对象

public class MyAgent {
  private static Instrumentation mInstrumentation;

  public static void agentmain(String agentArgs, Instrumentation inst) {
    mInstrumentation = inst;
  }

  // 拿到Instrumentation对象后就可以利用ClassModifierTransformer来进行类的热替换了
  public static void modifyClass(Class clazz){
    ClassFileTransformer transformer = new ClassModifierTransformer();
    mInstrumentation.addTransformer(transformer, true);
    mInstrumentation.retransformClasses(new Class[]{clazz});
    mInstrumentation.removeTransformer(transformer);
  }
}

第二步:写个ClassFileTransformer,利用Javassist等工具进行字节码修改

public class ClassModifierTransformer implements ClassFileTransformer {

  @Override
  public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
    // 在这里利用Javassist等工具修改类的字节码,返回修改后类的字节数组
    return null;
  }
}

目前已经有很多文章讲具体使用方法了,大家可以Google下,我这里先介绍两篇:

☆基于Java Instrument的Agent实现www.jianshu.com图标谈谈Java Intrumentation和相关应用www.fanyilun.me

三:原理探究

热替换的核心就在于Instrumentation的两个方法:

 void addTransformer(ClassFileTransformer transformer, boolean canRetransform);

 void retransformClasses(Class<?>... classes) throws UnmodifiableClassException;

addTransformer()用来注册类的修改器;

retransformClasses()会让类重新加载,从而使得注册的类修改器能够重新修改类的字节码。

下面让我们细细讲讲这两个函数:

3.1: addTransformer()

addTransformer的实现在InstrumentationImpl中:


    //sun.instrument.InstrumentationImpl

    public synchronized void addTransformer(ClassFileTransformer transformer, boolean canRetransform) {
        ......
        mRetransfomableTransformerManager.addTransformer(transformer);
        ......
    }

上面代码省略了一些,可见我们的ClassFileTransformer又被添加到了TransformerManager中,让我们跟进去看看:

    //sun.instrument.TransformerManager

    public synchronized void addTransformer( ClassFileTransformer    transformer) {
        TransformerInfo[] oldList = mTransformerList;
        TransformerInfo[] newList = new TransformerInfo[oldList.length + 1];
        System.arraycopy(   oldList,
                            0,
                            newList,
                            0,
                            oldList.length);
        newList[oldList.length] = new TransformerInfo(transformer);
        mTransformerList = newList;
    }

ClassFileTransformer对象这次被放入了TransformerManager的一个数组中。

OK,注册完毕,很简单对不对?下面我们再来看下稍微复杂点的retransformClasses()吧。

3.2: retransformClasses()

这个方法的实现是个Native方法。

    private native void retransformClasses0(long var1, Class<?>[] var3);

很多同学看到Native方法就头疼,不要急,Native方法也是人写的,不过是一段文本而已。我们来看下他的具体实现吧:

// src/java.instrument/share/native/libinstrument/InstrumentationImplNativeMethods.c

JNIEXPORT void JNICALL
Java_sun_instrument_InstrumentationImpl_retransformClasses0
  (JNIEnv * jnienv, jobject implThis, jlong agent, jobjectArray classes) {
    retransformClasses(jnienv, (JPLISAgent*)(intptr_t)agent, classes);
}

retransformClasses()最后会调用到 jvmtiEnv.cpp中的RetransformClasses

// src/hotspot/share/prims/jvmtiEnv.cpp

jvmtiError
JvmtiEnv::RetransformClasses(jint class_count, const jclass* classes) {

  int index;
  JavaThread* current_thread = JavaThread::current();
  ResourceMark rm(current_thread);

  jvmtiClassDefinition* class_definitions =
                            NEW_RESOURCE_ARRAY(jvmtiClassDefinition, class_count);

  for (index = 0; index < class_count; index++) {
    HandleMark hm(current_thread);

    jclass jcls = classes[index];
    oop k_mirror = JNIHandles::resolve_external_guard(jcls);
    ......
    Klass* klass = java_lang_Class::as_Klass(k_mirror);

    jint status = klass->jvmti_class_status();
    if (status & (JVMTI_CLASS_STATUS_ERROR)) {
      return JVMTI_ERROR_INVALID_CLASS;
    }

    InstanceKlass* ik = InstanceKlass::cast(klass);
    if (ik->get_cached_class_file_bytes() == NULL) {
      JvmtiClassFileReconstituter reconstituter(ik);
      if (reconstituter.get_error() != JVMTI_ERROR_NONE) {
        return reconstituter.get_error();
      }

      class_definitions[index].class_byte_count = (jint)reconstituter.class_file_size();
      class_definitions[index].class_bytes      = (unsigned char*)
                                                       reconstituter.class_file_bytes();
    } else {
      // it is cached, get it from the cache
      class_definitions[index].class_byte_count = ik->get_cached_class_file_len();
      class_definitions[index].class_bytes      = ik->get_cached_class_file_bytes();
    }
    class_definitions[index].klass              = jcls;
  }
  VM_RedefineClasses op(class_count, class_definitions, jvmti_class_load_kind_retransform);
  VMThread::execute(&op);
  return (op.check_error());
} 

上面这段主要干了两件事:

(1) 根据java层的Class对象,找到JVM层的类实例InstanceKlass,并获取类的字节码,存放在class_definitions数组中。因为可以一次替换多个类,所以这里加了一个循环体,遍历每个要修改的类。

(2) 调用VMThread::execute(&op),进入下一步。

VMThread::execute(&op) 中会调用到 VM_RedefineClasses::doit_prologue(),最终调用到VM_RedefineClasses::load_new_class_versions():

jvmtiError VM_RedefineClasses::load_new_class_versions(TRAPS) {

    ......
    InstanceKlass* the_class = get_ik(_class_defs[i].klass);
    Symbol*  the_class_sym = the_class->name();

    ClassFileStream st((u1*)_class_defs[i].class_bytes,
                       _class_defs[i].class_byte_count,
                       "__VM_RedefineClasses__",
                       ClassFileStream::verify);

    Handle the_class_loader(THREAD, the_class->class_loader());
    Handle protection_domain(THREAD, the_class->protection_domain());
    
    state->set_class_being_redefined(the_class, _class_load_kind);

    InstanceKlass* scratch_class = SystemDictionary::parse_stream(
                                                      the_class_sym,
                                                      the_class_loader,
                                                      protection_domain,
                                                      &st,
    ......
    
}

上面这个方法调用了parse_stream(),从文件流中解析类,最终触发类的重新加载:

InstanceKlass* SystemDictionary::load_shared_class(InstanceKlass* ik,
                                                   Handle class_loader,
                                                   Handle protection_domain, TRAPS) {
    ......

    InstanceKlass* new_ik = KlassFactory::check_shared_class_file_load_hook(
        ik, class_name, class_loader, protection_domain, CHECK_NULL);

    if (new_ik != NULL) {
      return new_ik;
    }

   ......
   return ik;
}

这里又调用了KlassFactory::check_shared_class_file_load_hook(),看名字就知道是个hook方法,它会调用post_class_file_load_hook(),利用JvmtiClassFileLoadHookPoster来通知类修改器进行类的修改。

消息的处理者为:eventHandlerClassFileLoadHook():

void JNICALL
eventHandlerClassFileLoadHook(  jvmtiEnv *              jvmtienv,
                                JNIEnv *                jnienv,
                                jclass                  class_being_redefined,
                                jobject                 loader,
                                const char*             name,
                                jobject                 protectionDomain,
                                jint                    class_data_len,
                                const unsigned char*    class_data,
                                jint*                   new_class_data_len,
                                unsigned char**         new_class_data) {
    JPLISEnvironment * environment  = NULL;

    environment = getJPLISEnvironment(jvmtienv);

    /* if something is internally inconsistent (no agent), just silently return without touching the buffer */
    if ( environment != NULL ) {
        jthrowable outstandingException = preserveThrowable(jnienv);
        transformClassFile( environment->mAgent,
                            jnienv,
                            loader,
                            name,
                            class_being_redefined,
                            protectionDomain,
                            class_data_len,
                            class_data,
                            new_class_data_len,
                            new_class_data,
                            environment->mIsRetransformer);
        restoreThrowable(jnienv, outstandingException);
    }
}

eventHandlerClassFileLoadHook()在收到消息后,会调用transformClassFile():

void
transformClassFile(             JPLISAgent *            agent,
                                JNIEnv *                jnienv,
                                jobject                 loaderObject,
                                const char*             name,
                                jclass                  classBeingRedefined,
                                jobject                 protectionDomain,
                                jint                    class_data_len,
                                const unsigned char*    class_data,
                                jint*                   new_class_data_len,
                                unsigned char**         new_class_data,
                                jboolean                is_retransformer) {
            ......
            transformedBufferObject = (*jnienv)->CallObjectMethod(
                                                jnienv,
                                                agent->mInstrumentationImpl,
                                                agent->mTransform,
                                                moduleObject,
                                                loaderObject,
                                                classNameStringObject,
                                                classBeingRedefined,
                                                protectionDomain,
                                                classFileBufferObject,
                                                is_retransformer);
         ......
}

这里会利用JNI调用 java 层InstrumentationImpl的transform(),你看,我们又绕到Java层了:

private byte[] transform(  Module              module,
                ClassLoader         loader,
                String              classname,
                Class<?>            classBeingRedefined,
                ProtectionDomain    protectionDomain,
                byte[]              classfileBuffer,
                boolean             isRetransformer) {
        TransformerManager mgr = isRetransformer?
                                        mRetransfomableTransformerManager :
                                        mTransformerManager;
        // module is null when not a class load or when loading a class in an
        // unnamed module and this is the first type to be loaded in the package.
        if (module == null) {
            if (classBeingRedefined != null) {
                module = classBeingRedefined.getModule();
            } else {
                module = (loader == null) ? jdk.internal.loader.BootLoader.getUnnamedModule()
                                          : loader.getUnnamedModule();
            }
        }
        if (mgr == null) {
            return null; // no manager, no transform
        } else {
            return mgr.transform(   module,
                                    loader,
                                    classname,
                                    classBeingRedefined,
                                    protectionDomain,
                                    classfileBuffer);
        }
    }

上面主要就是调用TransformerManager的transform():

 public byte[] transform(  Module              module,
                ClassLoader         loader,
                String              classname,
                Class<?>            classBeingRedefined,
                ProtectionDomain    protectionDomain,
                byte[]              classfileBuffer) {
        boolean someoneTouchedTheBytecode = false;

        TransformerInfo[]  transformerList = getSnapshotTransformerList();

        byte[]  bufferToUse = classfileBuffer;

        // order matters, gotta run 'em in the order they were added
        for ( int x = 0; x < transformerList.length; x++ ) {
            TransformerInfo         transformerInfo = transformerList[x];
            ClassFileTransformer    transformer = transformerInfo.transformer();
            byte[]                  transformedBytes = null;

            try {
                transformedBytes = transformer.transform(   module,
                                                            loader,
                                                            classname,
                                                            classBeingRedefined,
                                                            protectionDomain,
                                                            bufferToUse);
            }
            catch (Throwable t) {
                // don't let any one transformer mess it up for the others.
                // This is where we need to put some logging. What should go here? FIXME
            }

            if ( transformedBytes != null ) {
                someoneTouchedTheBytecode = true;
                bufferToUse = transformedBytes;
            }
        }

        // if someone modified it, return the modified buffer.
        // otherwise return null to mean "no transforms occurred"
        byte [] result;
        if ( someoneTouchedTheBytecode ) {
            result = bufferToUse;
        }
        else {
            result = null;
        }

        return result;
    }

看到这儿,大家还记得我们开始的时候,会将我们自定义的ClassFileTransformer对象注册到TransformerManager中吗?这里终于派上用场了,TransformerManager的transform()方法会遍历它的注册数组,调用每个ClassFileTransformer对象的transform()方法,并将我们修改后的类字节码返回,返回后的字节码最终又回到了上面JVM层的transformClassFile()中,并最终交还给给class_file_load_hook 消息的发送方。

让我们回到消息的发送方:check_shared_class_file_load_hook()中去看看:

InstanceKlass* KlassFactory::check_shared_class_file_load_hook(
                                          InstanceKlass* ik,
                                          Symbol* class_name,
                                          Handle class_loader,
                                          Handle protection_domain, TRAPS) {
#if INCLUDE_CDS && INCLUDE_JVMTI
  assert(ik != NULL, "sanity");
  assert(ik->is_shared(), "expecting a shared class");

  if (JvmtiExport::should_post_class_file_load_hook()) {
    assert(THREAD->is_Java_thread(), "must be JavaThread");

    // Post the CFLH
    JvmtiCachedClassFileData* cached_class_file = NULL;
    JvmtiCachedClassFileData* archived_class_data = ik->get_archived_class_data();
    assert(archived_class_data != NULL, "shared class has no archived class data");
    unsigned char* ptr =
        VM_RedefineClasses::get_cached_class_file_bytes(archived_class_data);
    unsigned char* end_ptr =
        ptr + VM_RedefineClasses::get_cached_class_file_len(archived_class_data);
    unsigned char* old_ptr = ptr;
    JvmtiExport::post_class_file_load_hook(class_name,
                                           class_loader,
                                           protection_domain,
                                           &ptr,
                                           &end_ptr,
                                           &cached_class_file);
    // 这里判断类是否被修改了
    if (old_ptr != ptr) {
      ......
      // 根据返回的类字节码指针及指针范围,构造ClassFileStream
      ClassFileStream* stream = new ClassFileStream(ptr,
                                                    end_ptr - ptr,
                                                    pathname,
                                                    ClassFileStream::verify);
      // 构建新返回的类字节码解析器
      ClassFileParser parser(stream,
                             class_name,
                             loader_data,
                             protection_domain,
                             NULL,
                             NULL,
                             ClassFileParser::BROADCAST, // publicity level
                             CHECK_NULL);
      // 解析新返回的类字节码,构造出新的类实例InstanceKlass
      InstanceKlass* new_ik = parser.create_instance_klass(true /* changed_by_loadhook */,
                                                           CHECK_NULL);
      if (cached_class_file != NULL) {
        new_ik->set_cached_class_file(cached_class_file);
      }

      if (class_loader.is_null()) {
        ResourceMark rm;
        ClassLoader::add_package(class_name->as_C_string(), path_index, THREAD);
      }

      return new_ik;
    }
  }
#endif

  return NULL;
}

check_shared_class_file_load_hook()中会解析新返回的类字节码,构造出新的类new_ik,最后,基于修改后的字节码构造出来的新类new_ik 会被返回给上一层,完成类的链接等任务,最终实现了类的热替换。

四:总结

所谓类的热替换,本质上,就是触发了类的重新加载,并在类加载过程中,通过Hook方法,修改类的字节码,并基于修改后的字节码,重新初始化类而已。


欢迎关注我的知乎专栏:

半栈工程师zhuanlan.zhihu.com图标

听说喜欢点关注的同学都长得帅

编辑于 2019-05-28

文章被以下专栏收录