RejectedExecutionException on shutdown...

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

RejectedExecutionException on shutdown...

Peter Smith-2
I am building (a subset of) SM from the 2.0.2 source

My Main class is modified slightly so it can be notified externally. So
instead of just killing the standalone SM (which would otherwise wait
forever) with a ^C I am notifying the Main lock properly so graceful
shutdown ought to occur.

9 times out of 10 it works properly and shutdown is clean as expected;
but occasionally it fails with the exception below.

Because I haven't modified any of the code in the stacktrace I guess
this exception might also be happening in the"official" SM.

Do you have any idea what might be the cause of this exception?

Thanks,
Peter.

========


FINE: Invoking destroy() on bean with name 'jbi'
7/12/2005 14:12:53
org.springframework.beans.factory.support.AbstractBeanFactory destroyBean
SEVERE: Destroy method on bean with name 'jbi' threw an exception
edu.emory.mathcs.backport.java.util.concurrent.RejectedExecutionException
    at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1499)
    at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:391)
    at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:847)
    at
org.servicemix.jbi.management.BaseStandardMBean.sendNotification(BaseStandardMBean.java:367)
    at
org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:421)
    at
org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:436)
    at
org.servicemix.jbi.management.BaseStandardMBean.updateAttribute(BaseStandardMBean.java:211)
    at
org.servicemix.jbi.management.BaseStandardMBean.propertyChange(BaseStandardMBean.java:228)
    at
org.servicemix.jbi.management.BaseLifeCycle.firePropertyChanged(BaseLifeCycle.java:148)
    at
org.servicemix.jbi.management.BaseLifeCycle.setCurrentState(BaseLifeCycle.java:96)
    at
org.servicemix.jbi.management.BaseLifeCycle.shutDown(BaseLifeCycle.java:76)
    at
org.servicemix.jbi.nmr.flow.seda.SedaQueue.shutDown(SedaQueue.java:179)
    at org.servicemix.jbi.nmr.flow.seda.SedaFlow.shutDown(SedaFlow.java:119)
    at org.servicemix.jbi.nmr.Broker.shutDown(Broker.java:206)
    at
org.servicemix.jbi.container.JBIContainer.shutDown(JBIContainer.java:454)
    at
org.servicemix.jbi.container.SpringJBIContainer.destroy(SpringJBIContainer.java:169)
    at
org.springframework.beans.factory.support.AbstractBeanFactory$1.destroy(AbstractBeanFactory.java:916)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1007)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
    at
org.springframework.beans.factory.support.AbstractBeanFactory.destroySingletons(AbstractBeanFactory.java:557)
    at
org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:528)
    at
org.springframework.context.support.AbstractApplicationContext.destroy(AbstractApplicationContext.java:544)
    at org.fast.FJMain.servicemix(FJMain.java:113)

[end]


Reply | Threaded
Open this post in threaded view
|

Re: RejectedExecutionException on shutdown...

Guillaume Nodet-3
I think i fixed that.
This is due to the thread pool of the
org.servicemix.jbi.management.BaseStandardMBean
class which is sthudown before the object has been completely shut down.
So when the object becomes completely shut down, it changes its state,
and thus
try to send a jmx notification.

Guillaume

Peter Smith wrote:

> I am building (a subset of) SM from the 2.0.2 source
>
> My Main class is modified slightly so it can be notified externally.
> So instead of just killing the standalone SM (which would otherwise
> wait forever) with a ^C I am notifying the Main lock properly so
> graceful shutdown ought to occur.
>
> 9 times out of 10 it works properly and shutdown is clean as expected;
> but occasionally it fails with the exception below.
>
> Because I haven't modified any of the code in the stacktrace I guess
> this exception might also be happening in the"official" SM.
>
> Do you have any idea what might be the cause of this exception?
>
> Thanks,
> Peter.
>
> ========
>
>
> FINE: Invoking destroy() on bean with name 'jbi'
> 7/12/2005 14:12:53
> org.springframework.beans.factory.support.AbstractBeanFactory destroyBean
> SEVERE: Destroy method on bean with name 'jbi' threw an exception
> edu.emory.mathcs.backport.java.util.concurrent.RejectedExecutionException
>    at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1499)
>
>    at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:391)
>
>    at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:847)
>
>    at
> org.servicemix.jbi.management.BaseStandardMBean.sendNotification(BaseStandardMBean.java:367)
>
>    at
> org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:421)
>
>    at
> org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:436)
>
>    at
> org.servicemix.jbi.management.BaseStandardMBean.updateAttribute(BaseStandardMBean.java:211)
>
>    at
> org.servicemix.jbi.management.BaseStandardMBean.propertyChange(BaseStandardMBean.java:228)
>
>    at
> org.servicemix.jbi.management.BaseLifeCycle.firePropertyChanged(BaseLifeCycle.java:148)
>
>    at
> org.servicemix.jbi.management.BaseLifeCycle.setCurrentState(BaseLifeCycle.java:96)
>
>    at
> org.servicemix.jbi.management.BaseLifeCycle.shutDown(BaseLifeCycle.java:76)
>
>    at
> org.servicemix.jbi.nmr.flow.seda.SedaQueue.shutDown(SedaQueue.java:179)
>    at
> org.servicemix.jbi.nmr.flow.seda.SedaFlow.shutDown(SedaFlow.java:119)
>    at org.servicemix.jbi.nmr.Broker.shutDown(Broker.java:206)
>    at
> org.servicemix.jbi.container.JBIContainer.shutDown(JBIContainer.java:454)
>    at
> org.servicemix.jbi.container.SpringJBIContainer.destroy(SpringJBIContainer.java:169)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.destroy(AbstractBeanFactory.java:916)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1007)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>
>    at
> org.springframework.beans.factory.support.AbstractBeanFactory.destroySingletons(AbstractBeanFactory.java:557)
>
>    at
> org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:528)
>
>    at
> org.springframework.context.support.AbstractApplicationContext.destroy(AbstractApplicationContext.java:544)
>
>    at org.fast.FJMain.servicemix(FJMain.java:113)
>
> [end]
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: RejectedExecutionException on shutdown...

Peter Smith-2
Hi Guillaume

I would like to apply your fix in my environment. Can you please let me know what source was changed....

(Or let me know how I can figure it out myself)

- I couldn't find any JIRA about this
- I looked online at source for BaseStandardMBean  but source directory structure seems recently changed and ALL change history is gone - aaarrgh :-( 
- Is there any way to migrate that old history to the new directory structure?

Thanks,
Peter



Guillaume Nodet wrote:
I think i fixed that.
This is due to the thread pool of the org.servicemix.jbi.management.BaseStandardMBean
class which is sthudown before the object has been completely shut down.
So when the object becomes completely shut down, it changes its state, and thus
try to send a jmx notification.

Guillaume

Peter Smith wrote:

I am building (a subset of) SM from the 2.0.2 source

My Main class is modified slightly so it can be notified externally. So instead of just killing the standalone SM (which would otherwise wait forever) with a ^C I am notifying the Main lock properly so graceful shutdown ought to occur.

9 times out of 10 it works properly and shutdown is clean as expected; but occasionally it fails with the exception below.

Because I haven't modified any of the code in the stacktrace I guess this exception might also be happening in the"official" SM.

Do you have any idea what might be the cause of this exception?

Thanks,
Peter.

========


FINE: Invoking destroy() on bean with name 'jbi'
7/12/2005 14:12:53 org.springframework.beans.factory.support.AbstractBeanFactory destroyBean
SEVERE: Destroy method on bean with name 'jbi' threw an exception
edu.emory.mathcs.backport.java.util.concurrent.RejectedExecutionException
   at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1499)
   at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:391)
   at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:847)
   at org.servicemix.jbi.management.BaseStandardMBean.sendNotification(BaseStandardMBean.java:367)
   at org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:421)
   at org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:436)
   at org.servicemix.jbi.management.BaseStandardMBean.updateAttribute(BaseStandardMBean.java:211)
   at org.servicemix.jbi.management.BaseStandardMBean.propertyChange(BaseStandardMBean.java:228)
   at org.servicemix.jbi.management.BaseLifeCycle.firePropertyChanged(BaseLifeCycle.java:148)
   at org.servicemix.jbi.management.BaseLifeCycle.setCurrentState(BaseLifeCycle.java:96)
   at org.servicemix.jbi.management.BaseLifeCycle.shutDown(BaseLifeCycle.java:76)
   at org.servicemix.jbi.nmr.flow.seda.SedaQueue.shutDown(SedaQueue.java:179)
   at org.servicemix.jbi.nmr.flow.seda.SedaFlow.shutDown(SedaFlow.java:119)
   at org.servicemix.jbi.nmr.Broker.shutDown(Broker.java:206)
   at org.servicemix.jbi.container.JBIContainer.shutDown(JBIContainer.java:454)
   at org.servicemix.jbi.container.SpringJBIContainer.destroy(SpringJBIContainer.java:169)
   at org.springframework.beans.factory.support.AbstractBeanFactory$1.destroy(AbstractBeanFactory.java:916)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1007)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
   at org.springframework.beans.factory.support.AbstractBeanFactory.destroySingletons(AbstractBeanFactory.java:557)
   at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:528)
   at org.springframework.context.support.AbstractApplicationContext.destroy(AbstractApplicationContext.java:544)
   at org.fast.FJMain.servicemix(FJMain.java:113)

[end]







--
Peter Smith
Fujitsu Australia Software Technology
14 Rodborough Road, Frenchs Forest NSW 2086
T: +61 2 9452 9130, F: +61 2 9975 2899
[hidden email]
http://www.fastware.com
Reply | Threaded
Open this post in threaded view
|

Re: RejectedExecutionException on shutdown...

Guillaume Nodet-3
In BaseStandardMBean  , use the following

    public void sendNotification(final Notification notification) throws
MBeanException, RuntimeOperationsException {
        if (notification != null) {
            if (!executorService.isShutdown()) {
                executorService.execute(new Runnable() {
                    public void run() {
                        broadcasterSupport.sendNotification(notification);
                    }
                });
            }
        }
    }

Btw, the history has not been lost, but i think viewcvs is not able to
show it
across moves.  Maybe with a real svn client (i have it in Eclipse).

Cheers,
Guillaume

Peter Smith wrote:

> Hi Guillaume
>
> I would like to apply your fix in my environment. Can you please let
> me know what source was changed....
>
> (Or let me know how I can figure it out myself)
>
> - I couldn't find any JIRA about this
> - I looked online at source for BaseStandardMBean  but source
> directory structure seems recently changed and ALL change history is
> gone - aaarrgh :-(
> - Is there any way to migrate that old history to the new directory
> structure?
>
> Thanks,
> Peter
>
>
>
> Guillaume Nodet wrote:
>
>> I think i fixed that.
>> This is due to the thread pool of the
>> org.servicemix.jbi.management.BaseStandardMBean
>> class which is sthudown before the object has been completely shut down.
>> So when the object becomes completely shut down, it changes its
>> state, and thus
>> try to send a jmx notification.
>>
>> Guillaume
>>
>> Peter Smith wrote:
>>
>>> I am building (a subset of) SM from the 2.0.2 source
>>>
>>> My Main class is modified slightly so it can be notified externally.
>>> So instead of just killing the standalone SM (which would otherwise
>>> wait forever) with a ^C I am notifying the Main lock properly so
>>> graceful shutdown ought to occur.
>>>
>>> 9 times out of 10 it works properly and shutdown is clean as
>>> expected; but occasionally it fails with the exception below.
>>>
>>> Because I haven't modified any of the code in the stacktrace I guess
>>> this exception might also be happening in the"official" SM.
>>>
>>> Do you have any idea what might be the cause of this exception?
>>>
>>> Thanks,
>>> Peter.
>>>
>>> ========
>>>
>>>
>>> FINE: Invoking destroy() on bean with name 'jbi'
>>> 7/12/2005 14:12:53
>>> org.springframework.beans.factory.support.AbstractBeanFactory
>>> destroyBean
>>> SEVERE: Destroy method on bean with name 'jbi' threw an exception
>>> edu.emory.mathcs.backport.java.util.concurrent.RejectedExecutionException
>>>
>>>    at
>>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1499)
>>>
>>>    at
>>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:391)
>>>
>>>    at
>>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:847)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseStandardMBean.sendNotification(BaseStandardMBean.java:367)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:421)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseStandardMBean.sendAttributeChangeNotification(BaseStandardMBean.java:436)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseStandardMBean.updateAttribute(BaseStandardMBean.java:211)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseStandardMBean.propertyChange(BaseStandardMBean.java:228)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseLifeCycle.firePropertyChanged(BaseLifeCycle.java:148)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseLifeCycle.setCurrentState(BaseLifeCycle.java:96)
>>>
>>>    at
>>> org.servicemix.jbi.management.BaseLifeCycle.shutDown(BaseLifeCycle.java:76)
>>>
>>>    at
>>> org.servicemix.jbi.nmr.flow.seda.SedaQueue.shutDown(SedaQueue.java:179)
>>>    at
>>> org.servicemix.jbi.nmr.flow.seda.SedaFlow.shutDown(SedaFlow.java:119)
>>>    at org.servicemix.jbi.nmr.Broker.shutDown(Broker.java:206)
>>>    at
>>> org.servicemix.jbi.container.JBIContainer.shutDown(JBIContainer.java:454)
>>>
>>>    at
>>> org.servicemix.jbi.container.SpringJBIContainer.destroy(SpringJBIContainer.java:169)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory$1.destroy(AbstractBeanFactory.java:916)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1007)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyBean(AbstractBeanFactory.java:1001)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroyDisposableBean(AbstractBeanFactory.java:979)
>>>
>>>    at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.destroySingletons(AbstractBeanFactory.java:557)
>>>
>>>    at
>>> org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:528)
>>>
>>>    at
>>> org.springframework.context.support.AbstractApplicationContext.destroy(AbstractApplicationContext.java:544)
>>>
>>>    at org.fast.FJMain.servicemix(FJMain.java:113)
>>>
>>> [end]
>>>
>>>
>>>
>>>
>>
>>
>
> --
> /Peter Smith/
> Fujitsu Australia Software Technology
> 14 Rodborough Road, Frenchs Forest NSW 2086
> T: +61 2 9452 9130, F: +61 2 9975 2899
> [hidden email] <mailto:[hidden email]>
> http://www.fastware.com