(ActiveMQ-client-netty-threads)] org.apache.catalina.loader.Weba.................: 此 Web 应用程序实例已停止

(ActiveMQ-client-netty-threads)] org.apache.catalina.loader.Weba.................: this web application instance has been stopped already

我有一个 spring-boot 应用程序连接到 ActiveMQ Artemis。当部署在 Tomcat 时,它工作正常,但是当部署同一应用程序的另一个版本并停止旧版本后,我不断在日志中收到此错误:

(ActiveMQ-client-netty-threads)] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading
Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.collection.IntObjectHashMap].
The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access. 
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.collection.IntObjectHashMap]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
    at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
    at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
    at org.apache.logging.log4j.util.LoaderUtil.loadClass(LoaderUtil.java:163)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.loadClass(ThrowableProxy.java:582)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.toExtendedStackTrace(ThrowableProxy.java:698)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:138)
    at org.apache.logging.log4j.core.impl.ThrowableProxy.<init>(ThrowableProxy.java:122)
    at org.apache.logging.log4j.core.impl.Log4jLogEvent.getThrownProxy(Log4jLogEvent.java:566)
    at org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:64)
    at org.springframework.boot.logging.log4j2.ExtendedWhitespaceThrowablePatternConverter.format(ExtendedWhitespaceThrowablePatternConverter.java:53)
    at org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:38)
    at org.apache.logging.log4j.core.layout.PatternLayout$PatternSerializer.toSerializable(PatternLayout.java:334)
    at org.apache.logging.log4j.core.layout.PatternLayout.toText(PatternLayout.java:233)
    at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:218)
    at org.apache.logging.log4j.core.layout.PatternLayout.encode(PatternLayout.java:58)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:177)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:170)
    at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:161)
    at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:156)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:129)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:120)
    at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:84)
    at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:448)
    at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:433)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
    at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:403)
    at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:63)
    at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:146)
    at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2163)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2118)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2101)
    at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1995)
    at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1967)
    at org.apache.logging.slf4j.Log4jLogger.warn(Log4jLogger.java:259)
    at io.netty.util.internal.logging.Slf4JLogger.warn(Slf4JLogger.java:151)
    at io.netty.channel.epoll.EpollEventLoop.handleLoopException(EpollEventLoop.java:334)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328)
    at io.netty.util.concurrent.SingleThreadEventExecutor.run(SingleThreadEventExecutor.java:886)
    at java.lang.Thread.run(Thread.java:745)

2020-04-02 11:37:33.587  WARN 10941 --- [-netty-threads)] i.n.c.e.EpollEventLoop: Unexpected exception in the selector loop.
java.lang.NoClassDefFoundError: io/netty/util/collection/IntObjectHashMap
    at io.netty.util.collection.IntObjectHashMap.values(IntObjectHashMap.java:221) ~[netty-common-4.1.22.Final.jar:4.1.22.Final]
    at io.netty.channel.epoll.EpollEventLoop.closeAll(EpollEventLoop.java:355) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:322) [netty-transport-native-epoll-4.1.22.Final-linux-x86_64.jar:4.1.22.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.run(SingleThreadEventExecutor.java:886) [netty-common-4.1.22.Final.jar:4.1.22.Final]
    at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]

重启Tomcat后问题解决了,但是我需要在不重启的情况下解决这个问题Tomcat,因为我不想重启生产Tomcat。

我在 spring-boot app 中的 artemis 配置:

@Bean("connectionFactory")
public ConnectionFactory connectionFactory(JmsProperties appProperties) {
   ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory(appProperties.getArtemis().getBrokerUrl());
   cf.setUser(appProperties.getArtemis().getUser());
   cf.setPassword(appProperties.getArtemis().getPassword());
   return cf;
}

我在这里创建了 ActiveMQ 连接工厂,因为 spring-boot 只允许 application.yml 中的用户名和密码用于 artemis,但我需要在代理 url 中设置 consumerWindowSize像这样:tcp://localhost:61616?consumerWindowSize=0

如果 url 具有 tcp 架构,连接工厂 ActiveMQConnection 使用 netty 连接到 ActiveMQ Artemis。当应用程序被 tomcat 停止时,在 netty 组上调用 shutdownGracefully 以停止所有 netty 线程。

但是,shutdownGracefully 方法是异步的,因此 tomcat 在所有 netty 线程停止之前完成停止过程。如果 netty 线程试图在相关应用程序停止后加载 class,则 tomcat class 加载程序会抛出 IllegalStateException 导致日志中出现错误。

带有暂停的 CachingConnectionFactory 应该可以缓解这个问题:

@Bean(destroyMethod = "destroy")
public ConnectionFactory connectionFactory(JmsProperties appProperties) {
    ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory(appProperties.getArtemis().getBrokerUrl());
    cf.setUser(appProperties.getArtemis().getUser());
    cf.setPassword(appProperties.getArtemis().getPassword());

    return new CachingConnectionFactory(cf) {
        @Override
        public void destroy() {
            super.destroy();

            try {
                Thread.sleep(30000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }

            System.out.println("CachingConnectionFactory is destroyed!");
        }
    };
}