Camel route to MongoDB

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Camel route to MongoDB

Tyler Durvik
I have a system composed of 2 OSGi bundles that process incoming data and
insert the data into MongoDB for future processing.  Data is received via a
network stream then parsed and sent to an internal ActiveMQ queue for
another bundle to pull the data off of the queue and push to MongoDB.

I've noticed that ServiceMix can quickly become unstable when MongoDB is
not running (memory fills up).  I'm not sure if this is a Camel, ActiveMQ
or MongoDB question, but is there any functionality within ServiceMix or
any of the other related components to prevent ServiceMix from becoming
unstable when the system doesn't know what to do with the data when MongoDB
is down.

Does anyone have advice or experience with this type of situation.

Thanks,
Tyler
Reply | Threaded
Open this post in threaded view
|

Re: Camel route to MongoDB

shomeier
Hi Tyler,

I am not sure if this is the cause of the problem but afaik Blueprint uses proxies.
So when an optional component is not available Blueprint will block until the service comes available or timeout. So if you are using Blueprint maybe that is the cause. Maybe you can adjust the Blueprint behaviour in such a case or decreasing the timeout (or even switch to DS/SCR).

But as I am not very familiar with Blueprint spec I am not sure if the proxying can lead to such a behaviour you described.

Cheers
Sascha
Reply | Threaded
Open this post in threaded view
|

Re: Camel route to MongoDB

Willem.Jiang
In reply to this post by Tyler Durvik
Can you use Java heap analyser to check the unstable service mix states?

When you use ActiveMQ to connect two bundles, the message should not be stored into the memory.

--  
Willem Jiang

Red Hat, Inc.
Web: http://www.redhat.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang  
Weibo: 姜宁willem



On December 31, 2014 at 10:48:26 PM, Tyler Durvik ([hidden email]) wrote:

> I have a system composed of 2 OSGi bundles that process incoming data and
> insert the data into MongoDB for future processing. Data is received via a
> network stream then parsed and sent to an internal ActiveMQ queue for
> another bundle to pull the data off of the queue and push to MongoDB.
>  
> I've noticed that ServiceMix can quickly become unstable when MongoDB is
> not running (memory fills up). I'm not sure if this is a Camel, ActiveMQ
> or MongoDB question, but is there any functionality within ServiceMix or
> any of the other related components to prevent ServiceMix from becoming
> unstable when the system doesn't know what to do with the data when MongoDB
> is down.
>  
> Does anyone have advice or experience with this type of situation.
>  
> Thanks,
> Tyler
>  

Reply | Threaded
Open this post in threaded view
|

Re: Camel route to MongoDB

Raúl Kripalani
In reply to this post by Tyler Durvik
Investigate Producer Flow Control in AMQ.

Regards,

*Raúl Kripalani*
Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Wed, Dec 31, 2014 at 2:47 PM, Tyler Durvik <[hidden email]> wrote:

> I have a system composed of 2 OSGi bundles that process incoming data and
> insert the data into MongoDB for future processing.  Data is received via a
> network stream then parsed and sent to an internal ActiveMQ queue for
> another bundle to pull the data off of the queue and push to MongoDB.
>
> I've noticed that ServiceMix can quickly become unstable when MongoDB is
> not running (memory fills up).  I'm not sure if this is a Camel, ActiveMQ
> or MongoDB question, but is there any functionality within ServiceMix or
> any of the other related components to prevent ServiceMix from becoming
> unstable when the system doesn't know what to do with the data when MongoDB
> is down.
>
> Does anyone have advice or experience with this type of situation.
>
> Thanks,
> Tyler
>
Reply | Threaded
Open this post in threaded view
|

回覆: Camel route to MongoDB

Paul Hsu
Dear Sir/ Madam,

 

I am no longer on the job market.

You can remove me from your Email List and Database.

I will contact you in the future.

 

Paul Hsu

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

-----原始�]件-----
寄件者: Raul Kripalani [mailto:[hidden email]]
寄件日期: Wednesday, January 07, 2015 10:27 AM
收件者: [hidden email]
主旨: Re: Camel route to MongoDB

Investigate Producer Flow Control in AMQ.

Regards,

*Raúl Kripalani*
Apache Camel PMC Member & Committer | Enterprise Architect, Open Source
Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk

On Wed, Dec 31, 2014 at 2:47 PM, Tyler Durvik <[hidden email]> wrote:

> I have a system composed of 2 OSGi bundles that process incoming data and
> insert the data into MongoDB for future processing.  Data is received via
a
> network stream then parsed and sent to an internal ActiveMQ queue for
> another bundle to pull the data off of the queue and push to MongoDB.
>
> I've noticed that ServiceMix can quickly become unstable when MongoDB is
> not running (memory fills up).  I'm not sure if this is a Camel, ActiveMQ
> or MongoDB question, but is there any functionality within ServiceMix or
> any of the other related components to prevent ServiceMix from becoming
> unstable when the system doesn't know what to do with the data when
MongoDB
> is down.
>
> Does anyone have advice or experience with this type of situation.
>
> Thanks,
> Tyler
>