ServiceMix › ServiceMix - User

Production deployment strategy...

Classic

List

Threaded

23 messages Options

James Carman

Production deployment strategy...

Fellow ServiceMixers,

How do people typically go about doing deployments to a running,
production environment? We would like to be able to deploy only the
pieces that need updating, leaving everything else running (web
services, camel routes, etc.). Do people do this in practice? If so,
how do you go about it? How do you set up your deployments? We are
currently using one big feature file that serves as a "manifest" for
what's supposed to be there, but this monolithic approach doesn't seem
to be very conducive to targeted deployments.

Thanks,

James

Jean-Baptiste Onofré

Re: Production deployment strategy...

Hi James,

it depends about what you use:
- JBI ?
- Camel ?
- OSGi ?

If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
with others running.

It requires a high level of granularity in the features. So it mainly
depends about your features structure.

Regards
JB

On 06/12/2012 02:35 PM, James Carman wrote:

> Fellow ServiceMixers,
>
> How do people typically go about doing deployments to a running,
> production environment? We would like to be able to deploy only the
> pieces that need updating, leaving everything else running (web
> services, camel routes, etc.). Do people do this in practice? If so,
> how do you go about it? How do you set up your deployments? We are
> currently using one big feature file that serves as a "manifest" for
> what's supposed to be there, but this monolithic approach doesn't seem
> to be very conducive to targeted deployments.
>
> Thanks,
>
> James

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com

James Carman

Re: Production deployment strategy...

I'm using:

OSGi (blueprint-configured)
Camel
CXF
OpenJPA

I am using the features-maven-plugin provided by karaf to generate my
features.xml file. I guess I can do this at any level of granularity
by specifying dependencies in the pom.xml file.

Do you typically set these things up as "boot features" or do you just
use the karaf console to install the features needed at runtime. If
you use the console, how do you script your deployments? Do you just
put a bunch of karaf commands in a file and say "run this"? If you
use boot features, you can shutdown SMX, change the version of your
feature repository, blast the data directory, and restart (which is
what we're doing right now).

What do folks typically do with bundle-specific configuration files
that they need in the etc directory? There are certain properties
that we just don't know until we get to the environment (JDBC urls,
usernames, passwords, etc), so we obviously can't default them. Do
the "middleware guys" just manage those?

On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré <[hidden email]> wrote:

> Hi James,
>
> it depends about what you use:
> - JBI ?
> - Camel ?
> - OSGi ?
>
> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
> with others running.
>
> It requires a high level of granularity in the features. So it mainly
> depends about your features structure.
>
> Regards
> JB
>
>
> On 06/12/2012 02:35 PM, James Carman wrote:
>>
>> Fellow ServiceMixers,
>>
>> How do people typically go about doing deployments to a running,
>> production environment? We would like to be able to deploy only the
>> pieces that need updating, leaving everything else running (web
>> services, camel routes, etc.). Do people do this in practice? If so,
>> how do you go about it? How do you set up your deployments? We are
>> currently using one big feature file that serves as a "manifest" for
>> what's supposed to be there, but this monolithic approach doesn't seem
>> to be very conducive to targeted deployments.
>>
>> Thanks,
>>
>> James
>
>
> --
> Jean-Baptiste Onofré
> [hidden email]
> http://blog.nanthrax.net
> Talend - http://www.talend.com

Christian Schneider

Re: Production deployment strategy...

I typically just use a normal feature file to describe my application to
be deployed.

At runtime I log into the console, add the feature file and install the
needed features. There is no need to
add the features to the boot features as karaf will remember the running
bundles anyway.

An update is a little complicated at the moment as you have to uninstall
the features, uninstall the feature
file and then do the above. We are already working on better ways to do
this though.

Basically the idea is to have an update command where you just give the
new version and karaf does the update.

You can install default configs and config files with your feature and
then configure them using the files or the config admin service (jmx,
webconsole, ..).

For DB and JMS it is a good practice to use services for the DataSource
or ConnectionFactory. See
http://www.liquid-reality.de/x/LYBk . This way you can really let the
admins do these configs. Your app only knows about a named DataSource.

Christian

Am 12.06.2012 14:56, schrieb James Carman:

--

Christian Schneider
http://www.liquid-reality.de

Open Source Architect
Talend Application Integration Division http://www.talend.com

Jean-Baptiste Onofré

Re: Production deployment strategy...

I extend the purpose about:
- do you need a central deployment tool ?
- do you need a "extend" the CI chain with a continuous deployment

If we talk about one "deployment tool" with "one SMX instance", just a
feature is enough, just have to define a good features shape.

If we talk about "multiple SMX instances" with one repo, Cellar or ACE
could be helpful.

If we talk about "one central deployment tool", with "multiple
environments" (a environment can contain several SMX instances, plus
others software), Apache Kalumet could help.

Cellar, ACE or Kalumet allows you (in different ways) to define a
configuration specific to an instance, an application or an environment.

Regards
JB

On 06/12/2012 04:47 PM, Christian Schneider wrote:

> I typically just use a normal feature file to describe my application to
> be deployed.
>
> At runtime I log into the console, add the feature file and install the
> needed features. There is no need to
> add the features to the boot features as karaf will remember the running
> bundles anyway.
>
> An update is a little complicated at the moment as you have to uninstall
> the features, uninstall the feature
> file and then do the above. We are already working on better ways to do
> this though.
>
> Basically the idea is to have an update command where you just give the
> new version and karaf does the update.
>
> You can install default configs and config files with your feature and
> then configure them using the files or the config admin service (jmx,
> webconsole, ..).
>
> For DB and JMS it is a good practice to use services for the DataSource
> or ConnectionFactory. See
> http://www.liquid-reality.de/x/LYBk . This way you can really let the
> admins do these configs. Your app only knows about a named DataSource.
>
> Christian
>
> Am 12.06.2012 14:56, schrieb James Carman:
>> I'm using:
>>
>> OSGi (blueprint-configured)
>> Camel
>> CXF
>> OpenJPA
>>
>> I am using the features-maven-plugin provided by karaf to generate my
>> features.xml file. I guess I can do this at any level of granularity
>> by specifying dependencies in the pom.xml file.
>>
>> Do you typically set these things up as "boot features" or do you just
>> use the karaf console to install the features needed at runtime. If
>> you use the console, how do you script your deployments? Do you just
>> put a bunch of karaf commands in a file and say "run this"? If you
>> use boot features, you can shutdown SMX, change the version of your
>> feature repository, blast the data directory, and restart (which is
>> what we're doing right now).
>>
>> What do folks typically do with bundle-specific configuration files
>> that they need in the etc directory? There are certain properties
>> that we just don't know until we get to the environment (JDBC urls,
>> usernames, passwords, etc), so we obviously can't default them. Do
>> the "middleware guys" just manage those?
>>
>>
>

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com

James Carman

Re: Production deployment strategy...

In reply to this post by Christian Schneider

On Tue, Jun 12, 2012 at 10:47 AM, Christian Schneider
<[hidden email]> wrote:
> I typically just use a normal feature file to describe my application to be
> deployed.
>

That's what we're doing now.

> At runtime I log into the console, add the feature file and install the
> needed features. There is no need to
> add the features to the boot features as karaf will remember the running
> bundles anyway.
>

The reason we went with boot features is we were having troubles
getting stuff to start up correctly with karaf commands. I have a
feeling that blueprint will solve this without the need to use boot
features, so I'm anxious to try it out!

> An update is a little complicated at the moment as you have to uninstall the
> features, uninstall the feature
> file and then do the above. We are already working on better ways to do this
> though.
>
> Basically the idea is to have an update command where you just give the new
> version and karaf does the update.
>

That's what I'm thinking, but I just wanted to understand what all of
you experts were doing. When doing this, does it end up bringing
everything down anyway? Or is that all part of strategically crafting
your bundles so that they can go up/down independently of one another?

> You can install default configs and config files with your feature and then
> configure them using the files or the config admin service (jmx, webconsole,
> ..).
>

How do you install default configs when using the features plugin? We
really like using the plugin because it automatically takes care of
the correct versions for you. I could go back to hand-writing it,
but what a pain!

> For DB and JMS it is a good practice to use services for the DataSource or
> ConnectionFactory. See
> http://www.liquid-reality.de/x/LYBk . This way you can really let the admins
> do these configs. Your app only knows about a named DataSource.
>

We're somewhat doing this. I've set up a blueprint file that
basically defines all of the connections we need. In it, it has
properties that the admins can set to override the default settings
(like pool sizes, etc.). We deploy this blueprint file as a "feature"
in our features repository and our admins just have to make sure they
set all the correct properties.

Jean-Baptiste Onofré

Re: Production deployment strategy...

Yes, currently, using features, we do feature:uninstall && feature:install.

It means that the feature are uninstall (at least on one instance, using
multiple instances configuration, we can be only in a degraded mode).

Regards
JB

On 06/12/2012 05:23 PM, James Carman wrote:

> On Tue, Jun 12, 2012 at 10:47 AM, Christian Schneider
> <[hidden email]> wrote:
>> I typically just use a normal feature file to describe my application to be
>> deployed.
>>
>
> That's what we're doing now.
>
>> At runtime I log into the console, add the feature file and install the
>> needed features. There is no need to
>> add the features to the boot features as karaf will remember the running
>> bundles anyway.
>>
>
> The reason we went with boot features is we were having troubles
> getting stuff to start up correctly with karaf commands. I have a
> feeling that blueprint will solve this without the need to use boot
> features, so I'm anxious to try it out!
>
>> An update is a little complicated at the moment as you have to uninstall the
>> features, uninstall the feature
>> file and then do the above. We are already working on better ways to do this
>> though.
>>
>> Basically the idea is to have an update command where you just give the new
>> version and karaf does the update.
>>
>
> That's what I'm thinking, but I just wanted to understand what all of
> you experts were doing. When doing this, does it end up bringing
> everything down anyway? Or is that all part of strategically crafting
> your bundles so that they can go up/down independently of one another?
>
>> You can install default configs and config files with your feature and then
>> configure them using the files or the config admin service (jmx, webconsole,
>> ..).
>>
>
> How do you install default configs when using the features plugin? We
> really like using the plugin because it automatically takes care of
> the correct versions for you. I could go back to hand-writing it,
> but what a pain!
>
>
>> For DB and JMS it is a good practice to use services for the DataSource or
>> ConnectionFactory. See
>> http://www.liquid-reality.de/x/LYBk . This way you can really let the admins
>> do these configs. Your app only knows about a named DataSource.
>>
>
> We're somewhat doing this. I've set up a blueprint file that
> basically defines all of the connections we need. In it, it has
> properties that the admins can set to override the default settings
> (like pool sizes, etc.). We deploy this blueprint file as a "feature"
> in our features repository and our admins just have to make sure they
> set all the correct properties.

--
Jean-Baptiste Onofré
[hidden email]
http://blog.nanthrax.net
Talend - http://www.talend.com

Gert Vanthienen

Re: Production deployment strategy...

Administrator

In reply to this post by James Carman

L.S.,

I recently came across an environment where they had a very nice
solution for your configuration problem. They only had a single
configuration file in their config directory, which basically
contained the environment's name (development, test, staging,
production). They also implemented their own property placeholder
that, depending on the environment set, was loading another set of
properties from the bundle. Every bundle would just come with the
configuration for all environments and depending on where you dropped
the bundle, it would just behave differently, connect to another
broker or database or send/receive to other servers. Perhaps we
should look into adding support for a setup like this in ServiceMix
directly?

Next to everything that has been said already and all the technologies
suggested for managing your containers, you may also want to take a
look at http://fuse.fusesource.org/fabric/ as a tool for handling
this.

Regards,

Gert Vanthienen
------------------------
FuseSource
Web: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

On Tue, Jun 12, 2012 at 2:56 PM, James Carman
<[hidden email]> wrote:

> I'm using:
>
> OSGi (blueprint-configured)
> Camel
> CXF
> OpenJPA
>
> I am using the features-maven-plugin provided by karaf to generate my
> features.xml file. I guess I can do this at any level of granularity
> by specifying dependencies in the pom.xml file.
>
> Do you typically set these things up as "boot features" or do you just
> use the karaf console to install the features needed at runtime. If
> you use the console, how do you script your deployments? Do you just
> put a bunch of karaf commands in a file and say "run this"? If you
> use boot features, you can shutdown SMX, change the version of your
> feature repository, blast the data directory, and restart (which is
> what we're doing right now).
>
> What do folks typically do with bundle-specific configuration files
> that they need in the etc directory? There are certain properties
> that we just don't know until we get to the environment (JDBC urls,
> usernames, passwords, etc), so we obviously can't default them. Do
> the "middleware guys" just manage those?
>
>
> On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
>> Hi James,
>>
>> it depends about what you use:
>> - JBI ?
>> - Camel ?
>> - OSGi ?
>>
>> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
>> with others running.
>>
>> It requires a high level of granularity in the features. So it mainly
>> depends about your features structure.
>>
>> Regards
>> JB
>>
>>
>> On 06/12/2012 02:35 PM, James Carman wrote:
>>>
>>> Fellow ServiceMixers,
>>>
>>> How do people typically go about doing deployments to a running,
>>> production environment? We would like to be able to deploy only the
>>> pieces that need updating, leaving everything else running (web
>>> services, camel routes, etc.). Do people do this in practice? If so,
>>> how do you go about it? How do you set up your deployments? We are
>>> currently using one big feature file that serves as a "manifest" for
>>> what's supposed to be there, but this monolithic approach doesn't seem
>>> to be very conducive to targeted deployments.
>>>
>>> Thanks,
>>>
>>> James
>>
>>
>> --
>> Jean-Baptiste Onofré
>> [hidden email]
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com

Regards,

Gert Vanthienen
------------------------
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

James Carman

Re: Production deployment strategy...

So, you would check your configurations (including database
usernames/passwords) into source control?

On Tue, Jun 12, 2012 at 3:57 PM, Gert Vanthienen
<[hidden email]> wrote:

> L.S.,
>
>
> I recently came across an environment where they had a very nice
> solution for your configuration problem. They only had a single
> configuration file in their config directory, which basically
> contained the environment's name (development, test, staging,
> production). They also implemented their own property placeholder
> that, depending on the environment set, was loading another set of
> properties from the bundle. Every bundle would just come with the
> configuration for all environments and depending on where you dropped
> the bundle, it would just behave differently, connect to another
> broker or database or send/receive to other servers. Perhaps we
> should look into adding support for a setup like this in ServiceMix
> directly?
>
> Next to everything that has been said already and all the technologies
> suggested for managing your containers, you may also want to take a
> look at http://fuse.fusesource.org/fabric/ as a tool for handling
> this.
>
>
> Regards,
>
> Gert Vanthienen
> ------------------------
> FuseSource
> Web: http://fusesource.com
> Blog: http://gertvanthienen.blogspot.com/
>
>
> On Tue, Jun 12, 2012 at 2:56 PM, James Carman
> <[hidden email]> wrote:
>> I'm using:
>>
>> OSGi (blueprint-configured)
>> Camel
>> CXF
>> OpenJPA
>>
>> I am using the features-maven-plugin provided by karaf to generate my
>> features.xml file. I guess I can do this at any level of granularity
>> by specifying dependencies in the pom.xml file.
>>
>> Do you typically set these things up as "boot features" or do you just
>> use the karaf console to install the features needed at runtime. If
>> you use the console, how do you script your deployments? Do you just
>> put a bunch of karaf commands in a file and say "run this"? If you
>> use boot features, you can shutdown SMX, change the version of your
>> feature repository, blast the data directory, and restart (which is
>> what we're doing right now).
>>
>> What do folks typically do with bundle-specific configuration files
>> that they need in the etc directory? There are certain properties
>> that we just don't know until we get to the environment (JDBC urls,
>> usernames, passwords, etc), so we obviously can't default them. Do
>> the "middleware guys" just manage those?
>>
>>
>> On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré <[hidden email]> wrote:
>>> Hi James,
>>>
>>> it depends about what you use:
>>> - JBI ?
>>> - Camel ?
>>> - OSGi ?
>>>
>>> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
>>> with others running.
>>>
>>> It requires a high level of granularity in the features. So it mainly
>>> depends about your features structure.
>>>
>>> Regards
>>> JB
>>>
>>>
>>> On 06/12/2012 02:35 PM, James Carman wrote:
>>>>
>>>> Fellow ServiceMixers,
>>>>
>>>> How do people typically go about doing deployments to a running,
>>>> production environment? We would like to be able to deploy only the
>>>> pieces that need updating, leaving everything else running (web
>>>> services, camel routes, etc.). Do people do this in practice? If so,
>>>> how do you go about it? How do you set up your deployments? We are
>>>> currently using one big feature file that serves as a "manifest" for
>>>> what's supposed to be there, but this monolithic approach doesn't seem
>>>> to be very conducive to targeted deployments.
>>>>
>>>> Thanks,
>>>>
>>>> James
>>>
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> [hidden email]
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com

Christian Schneider

Re: Production deployment strategy...

In reply to this post by Gert Vanthienen

I have also seen this and think it is a very bad practice. You either
end up having sensitive information like
username/passwords in the bundle or have to add another mechanism for
these. In any case I think the bundle should not know about the
environments. The config should be kept in each environment.

Of course my prefered strategy has the problem that you have to edit the
config at least at first deployment. I think this is not ideal but
better than having all in the bundle.

Christian

Am 12.06.2012 21:57, schrieb Gert Vanthienen:

> L.S.,
>
>
> I recently came across an environment where they had a very nice
> solution for your configuration problem. They only had a single
> configuration file in their config directory, which basically
> contained the environment's name (development, test, staging,
> production). They also implemented their own property placeholder
> that, depending on the environment set, was loading another set of
> properties from the bundle. Every bundle would just come with the
> configuration for all environments and depending on where you dropped
> the bundle, it would just behave differently, connect to another
> broker or database or send/receive to other servers. Perhaps we
> should look into adding support for a setup like this in ServiceMix
> directly?
>
> Next to everything that has been said already and all the technologies
> suggested for managing your containers, you may also want to take a
> look at http://fuse.fusesource.org/fabric/ as a tool for handling
> this.
>
>
> Regards,
>
> Gert Vanthienen
> ------------------------
> FuseSource
> Web: http://fusesource.com
> Blog: http://gertvanthienen.blogspot.com/
>
>
> On Tue, Jun 12, 2012 at 2:56 PM, James Carman
> <[hidden email]> wrote:
>> I'm using:
>>
>> OSGi (blueprint-configured)
>> Camel
>> CXF
>> OpenJPA
>>
>> I am using the features-maven-plugin provided by karaf to generate my
>> features.xml file. I guess I can do this at any level of granularity
>> by specifying dependencies in the pom.xml file.
>>
>> Do you typically set these things up as "boot features" or do you just
>> use the karaf console to install the features needed at runtime. If
>> you use the console, how do you script your deployments? Do you just
>> put a bunch of karaf commands in a file and say "run this"? If you
>> use boot features, you can shutdown SMX, change the version of your
>> feature repository, blast the data directory, and restart (which is
>> what we're doing right now).
>>
>> What do folks typically do with bundle-specific configuration files
>> that they need in the etc directory? There are certain properties
>> that we just don't know until we get to the environment (JDBC urls,
>> usernames, passwords, etc), so we obviously can't default them. Do
>> the "middleware guys" just manage those?
>>
>>
>> On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré<[hidden email]> wrote:
>>> Hi James,
>>>
>>> it depends about what you use:
>>> - JBI ?
>>> - Camel ?
>>> - OSGi ?
>>>
>>> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
>>> with others running.
>>>
>>> It requires a high level of granularity in the features. So it mainly
>>> depends about your features structure.
>>>
>>> Regards
>>> JB
>>>
>>>
>>> On 06/12/2012 02:35 PM, James Carman wrote:
>>>> Fellow ServiceMixers,
>>>>
>>>> How do people typically go about doing deployments to a running,
>>>> production environment? We would like to be able to deploy only the
>>>> pieces that need updating, leaving everything else running (web
>>>> services, camel routes, etc.). Do people do this in practice? If so,
>>>> how do you go about it? How do you set up your deployments? We are
>>>> currently using one big feature file that serves as a "manifest" for
>>>> what's supposed to be there, but this monolithic approach doesn't seem
>>>> to be very conducive to targeted deployments.
>>>>
>>>> Thanks,
>>>>
>>>> James
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> [hidden email]
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com

--

Christian Schneider
http://www.liquid-reality.de

Open Source Architect
Talend Application Integration Division http://www.talend.com

Christian Schneider

Re: Production deployment strategy...

In reply to this post by James Carman

Answers inline

Christian

Am 12.06.2012 17:23, schrieb James Carman:

> On Tue, Jun 12, 2012 at 10:47 AM, Christian Schneider
> <[hidden email]> wrote:
>> I typically just use a normal feature file to describe my application to be
>> deployed.
>>
> That's what we're doing now.
>
>> At runtime I log into the console, add the feature file and install the
>> needed features. There is no need to
>> add the features to the boot features as karaf will remember the running
>> bundles anyway.
>>
> The reason we went with boot features is we were having troubles
> getting stuff to start up correctly with karaf commands. I have a
> feeling that blueprint will solve this without the need to use boot
> features, so I'm anxious to try it out!

You probably use start levels to work around this. OSGi services and
blueprint should help you
with this. You can make sure bundles only start when required services
or configs are present.

>
>> An update is a little complicated at the moment as you have to uninstall the
>> features, uninstall the feature
>> file and then do the above. We are already working on better ways to do this
>> though.
>>
>> Basically the idea is to have an update command where you just give the new
>> version and karaf does the update.
>>
> That's what I'm thinking, but I just wanted to understand what all of
> you experts were doing. When doing this, does it end up bringing
> everything down anyway? Or is that all part of strategically crafting
> your bundles so that they can go up/down independently of one another?

You will have to solve this in your bundles. The framework just stops
your bundle. If the bundle has a bug in the stop mechanism there can be
even threads and code left over after uninstall.
>
>> You can install default configs and config files with your feature and then
>> configure them using the files or the config admin service (jmx, webconsole,
>> ..).
>>
> How do you install default configs when using the features plugin? We
> really like using the plugin because it automatically takes care of
> the correct versions for you. I could go back to hand-writing it,
> but what a pain!
You can use kind of a template feature file and let the plugin fill in
the transitive deps.

>
>
>> For DB and JMS it is a good practice to use services for the DataSource or
>> ConnectionFactory. See
>> http://www.liquid-reality.de/x/LYBk . This way you can really let the admins
>> do these configs. Your app only knows about a named DataSource.
>>
> We're somewhat doing this. I've set up a blueprint file that
> basically defines all of the connections we need. In it, it has
> properties that the admins can set to override the default settings
> (like pool sizes, etc.). We deploy this blueprint file as a "feature"
> in our features repository and our admins just have to make sure they
> set all the correct properties.

That is probably a quite clean way. I often go the simpler way of
deploying the blueprint file to the deploy folder. This is easier as the
admin can simply fill in the params in the blueprint file.

--

Christian Schneider
http://www.liquid-reality.de

Open Source Architect
Talend Application Integration Division http://www.talend.com

Gert Vanthienen

Re: Production deployment strategy...

Administrator

In reply to this post by Christian Schneider

L.S.,

Yeah, the credentials being in the bundles would be the biggest
drawback of this approach. One thing you could do is use encrypted
passwords, at least that would avoid having them in there in a
human-readable format. If that's still an issue, there probably is no
other way than having your administrators set this up directly on the
machine (I agree a Blueprint XML file in the deploy folder is probably
the easiest way for them), but even there that only works if your
developers don't have access to the production machines, because
otherwise they can still cat the file from within the Karaf console.

Even if you still have to set up the database and probaby
broker/content repository/... connections in this way, your test
environments probably also has a lot of other web services to connect
to or other Camel URLs that differ between environments so you would
still gain that.

Perhaps we can come up with a solution that only requires you to add
the dev information to your bundle - if you expose the configuration
as an OSGi service too, you could use the service ranking to override
it in a production/test system. In a development environment, only
the low-ranked development setup would be available so your developers
can just "start their engines" and get to work. In the production
environment, your administrators could set up the Blueprint file
upfront with the production information and your bundles would pick
that configuration instead. Not sure it would work for your
environment though.

Regards,

Gert Vanthienen
------------------------
FuseSource
Web: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

On Tue, Jun 12, 2012 at 10:53 PM, Christian Schneider
<[hidden email]> wrote:

> I have also seen this and think it is a very bad practice. You either end up
> having sensitive information like
> username/passwords in the bundle or have to add another mechanism for these.
> In any case I think the bundle should not know about the environments. The
> config should be kept in each environment.
>
> Of course my prefered strategy has the problem that you have to edit the
> config at least at first deployment. I think this is not ideal but better
> than having all in the bundle.
>
> Christian
>
>
> Am 12.06.2012 21:57, schrieb Gert Vanthienen:
>
>> L.S.,
>>
>>
>> I recently came across an environment where they had a very nice
>> solution for your configuration problem. They only had a single
>> configuration file in their config directory, which basically
>> contained the environment's name (development, test, staging,
>> production). They also implemented their own property placeholder
>> that, depending on the environment set, was loading another set of
>> properties from the bundle. Every bundle would just come with the
>> configuration for all environments and depending on where you dropped
>> the bundle, it would just behave differently, connect to another
>> broker or database or send/receive to other servers. Perhaps we
>> should look into adding support for a setup like this in ServiceMix
>> directly?
>>
>> Next to everything that has been said already and all the technologies
>> suggested for managing your containers, you may also want to take a
>> look at http://fuse.fusesource.org/fabric/ as a tool for handling
>> this.
>>
>>
>> Regards,
>>
>> Gert Vanthienen
>> ------------------------
>> FuseSource
>> Web: http://fusesource.com
>> Blog: http://gertvanthienen.blogspot.com/
>>
>>
>> On Tue, Jun 12, 2012 at 2:56 PM, James Carman
>> <[hidden email]> wrote:
>>>
>>> I'm using:
>>>
>>> OSGi (blueprint-configured)
>>> Camel
>>> CXF
>>> OpenJPA
>>>
>>> I am using the features-maven-plugin provided by karaf to generate my
>>> features.xml file. I guess I can do this at any level of granularity
>>> by specifying dependencies in the pom.xml file.
>>>
>>> Do you typically set these things up as "boot features" or do you just
>>> use the karaf console to install the features needed at runtime. If
>>> you use the console, how do you script your deployments? Do you just
>>> put a bunch of karaf commands in a file and say "run this"? If you
>>> use boot features, you can shutdown SMX, change the version of your
>>> feature repository, blast the data directory, and restart (which is
>>> what we're doing right now).
>>>
>>> What do folks typically do with bundle-specific configuration files
>>> that they need in the etc directory? There are certain properties
>>> that we just don't know until we get to the environment (JDBC urls,
>>> usernames, passwords, etc), so we obviously can't default them. Do
>>> the "middleware guys" just manage those?
>>>
>>>
>>> On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré<[hidden email]>
>>> wrote:
>>>>
>>>> Hi James,
>>>>
>>>> it depends about what you use:
>>>> - JBI ?
>>>> - Camel ?
>>>> - OSGi ?
>>>>
>>>> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
>>>> with others running.
>>>>
>>>> It requires a high level of granularity in the features. So it mainly
>>>> depends about your features structure.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>>
>>>> On 06/12/2012 02:35 PM, James Carman wrote:
>>>>>
>>>>> Fellow ServiceMixers,
>>>>>
>>>>> How do people typically go about doing deployments to a running,
>>>>> production environment? We would like to be able to deploy only the
>>>>> pieces that need updating, leaving everything else running (web
>>>>> services, camel routes, etc.). Do people do this in practice? If so,
>>>>> how do you go about it? How do you set up your deployments? We are
>>>>> currently using one big feature file that serves as a "manifest" for
>>>>> what's supposed to be there, but this monolithic approach doesn't seem
>>>>> to be very conducive to targeted deployments.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> James
>>>>
>>>>
>>>> --
>>>> Jean-Baptiste Onofré
>>>> [hidden email]
>>>> http://blog.nanthrax.net
>>>> Talend - http://www.talend.com
>
>
>
> --
>
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> Talend Application Integration Division http://www.talend.com
>

Regards,

Gert Vanthienen
------------------------
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

Guillaume Nodet

Re: Production deployment strategy...

Administrator

In reply to this post by Christian Schneider

Maybe I misunderstood, but I don't see where the bundle know about the
environment.
It's just that the configuration knows where to find the correct
configuration bits, depending on the environment (dev, test, prod for
example). That could be a system wide property and the bundles should
not have any knowledge of that (which of course would not be the
smartest idea to add if/then/else blocks hardcoded in the bundle).

On Tue, Jun 12, 2012 at 10:53 PM, Christian Schneider
<[hidden email]> wrote:

--
------------------------
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
FuseSource, Integration everywhere
http://fusesource.com

Gert Vanthienen

Re: Production deployment strategy...

Administrator

L.S.,

Ah yeah, I assumed they wanted to package everything up for easy
deployment, so I was thinking about properties files in a bundle, but
they can very wel be externalized too obviously. It shouldn't really
matter that much, the main idea was to add a way to configure the
container with an environment property and then provide some hooks or
services to leverage this information for configuring stuff based on
the environment.

Regards,

Gert Vanthienen
------------------------
FuseSource
Web: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

On Wed, Jun 13, 2012 at 8:56 AM, Guillaume Nodet <[hidden email]> wrote:

> Maybe I misunderstood, but I don't see where the bundle know about the
> environment.
> It's just that the configuration knows where to find the correct
> configuration bits, depending on the environment (dev, test, prod for
> example). That could be a system wide property and the bundles should
> not have any knowledge of that (which of course would not be the
> smartest idea to add if/then/else blocks hardcoded in the bundle).
>
> On Tue, Jun 12, 2012 at 10:53 PM, Christian Schneider
> <[hidden email]> wrote:
>> I have also seen this and think it is a very bad practice. You either end up
>> having sensitive information like
>> username/passwords in the bundle or have to add another mechanism for these.
>> In any case I think the bundle should not know about the environments. The
>> config should be kept in each environment.
>>
>> Of course my prefered strategy has the problem that you have to edit the
>> config at least at first deployment. I think this is not ideal but better
>> than having all in the bundle.
>>
>> Christian
>>
>>
>> Am 12.06.2012 21:57, schrieb Gert Vanthienen:
>>
>>> L.S.,
>>>
>>>
>>> I recently came across an environment where they had a very nice
>>> solution for your configuration problem. They only had a single
>>> configuration file in their config directory, which basically
>>> contained the environment's name (development, test, staging,
>>> production). They also implemented their own property placeholder
>>> that, depending on the environment set, was loading another set of
>>> properties from the bundle. Every bundle would just come with the
>>> configuration for all environments and depending on where you dropped
>>> the bundle, it would just behave differently, connect to another
>>> broker or database or send/receive to other servers. Perhaps we
>>> should look into adding support for a setup like this in ServiceMix
>>> directly?
>>>
>>> Next to everything that has been said already and all the technologies
>>> suggested for managing your containers, you may also want to take a
>>> look at http://fuse.fusesource.org/fabric/ as a tool for handling
>>> this.
>>>
>>>
>>> Regards,
>>>
>>> Gert Vanthienen
>>> ------------------------
>>> FuseSource
>>> Web: http://fusesource.com
>>> Blog: http://gertvanthienen.blogspot.com/
>>>
>>>
>>> On Tue, Jun 12, 2012 at 2:56 PM, James Carman
>>> <[hidden email]> wrote:
>>>>
>>>> I'm using:
>>>>
>>>> OSGi (blueprint-configured)
>>>> Camel
>>>> CXF
>>>> OpenJPA
>>>>
>>>> I am using the features-maven-plugin provided by karaf to generate my
>>>> features.xml file. I guess I can do this at any level of granularity
>>>> by specifying dependencies in the pom.xml file.
>>>>
>>>> Do you typically set these things up as "boot features" or do you just
>>>> use the karaf console to install the features needed at runtime. If
>>>> you use the console, how do you script your deployments? Do you just
>>>> put a bunch of karaf commands in a file and say "run this"? If you
>>>> use boot features, you can shutdown SMX, change the version of your
>>>> feature repository, blast the data directory, and restart (which is
>>>> what we're doing right now).
>>>>
>>>> What do folks typically do with bundle-specific configuration files
>>>> that they need in the etc directory? There are certain properties
>>>> that we just don't know until we get to the environment (JDBC urls,
>>>> usernames, passwords, etc), so we obviously can't default them. Do
>>>> the "middleware guys" just manage those?
>>>>
>>>>
>>>> On Tue, Jun 12, 2012 at 8:49 AM, Jean-Baptiste Onofré<[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hi James,
>>>>>
>>>>> it depends about what you use:
>>>>> - JBI ?
>>>>> - Camel ?
>>>>> - OSGi ?
>>>>>
>>>>> If you use Karaf/OSGi (with features/bundles/kar), you can update pieces
>>>>> with others running.
>>>>>
>>>>> It requires a high level of granularity in the features. So it mainly
>>>>> depends about your features structure.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>>
>>>>> On 06/12/2012 02:35 PM, James Carman wrote:
>>>>>>
>>>>>> Fellow ServiceMixers,
>>>>>>
>>>>>> How do people typically go about doing deployments to a running,
>>>>>> production environment? We would like to be able to deploy only the
>>>>>> pieces that need updating, leaving everything else running (web
>>>>>> services, camel routes, etc.). Do people do this in practice? If so,
>>>>>> how do you go about it? How do you set up your deployments? We are
>>>>>> currently using one big feature file that serves as a "manifest" for
>>>>>> what's supposed to be there, but this monolithic approach doesn't seem
>>>>>> to be very conducive to targeted deployments.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> James
>>>>>
>>>>>
>>>>> --
>>>>> Jean-Baptiste Onofré
>>>>> [hidden email]
>>>>> http://blog.nanthrax.net
>>>>> Talend - http://www.talend.com
>>
>>
>>
>> --
>>
>> Christian Schneider
>> http://www.liquid-reality.de
>>
>> Open Source Architect
>> Talend Application Integration Division http://www.talend.com
>>
>
>
>
> --
> ------------------------
> Guillaume Nodet
> ------------------------
> Blog: http://gnodet.blogspot.com/
> ------------------------
> FuseSource, Integration everywhere
> http://fusesource.com

Regards,

Gert Vanthienen
------------------------
Open Source SOA: http://fusesource.com
Blog: http://gertvanthienen.blogspot.com/

James Carman

Re: Production deployment strategy...

In reply to this post by Christian Schneider

On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
<[hidden email]> wrote:
>
>
> You probably use start levels to work around this. OSGi services and
> blueprint should help you
> with this. You can make sure bundles only start when required services or
> configs are present.
>

We actually don't mess with start levels. Blueprint will put the
bundle into the "Grace Period" until it finds everything it needs.
However, I am intrigued by that last part there where you said I can
tell BP to only start when required configs are present. How do I
stall the BP startup when configs aren't present?

Achim Nierbeck

Re: Production deployment strategy...

Hi James,

you need to use managed Services via the Configuration Admin service.
Take a look at the blueprint documentation to find some more details
on how to create that.
Or take a look at some of those tutorials christian did. (I'm sure he
has a link in his signature :) )

regards, Achim

2012/6/13 James Carman <[hidden email]>:

> On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
> <[hidden email]> wrote:
>>
>>
>> You probably use start levels to work around this. OSGi services and
>> blueprint should help you
>> with this. You can make sure bundles only start when required services or
>> configs are present.
>>
>
> We actually don't mess with start levels. Blueprint will put the
> bundle into the "Grace Period" until it finds everything it needs.
> However, I am intrigued by that last part there where you said I can
> tell BP to only start when required configs are present. How do I
> stall the BP startup when configs aren't present?

--

Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/>
Committer & Project Lead
OPS4J Pax for Vaadin
<http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter & Project
Lead
blog <http://notizblog.nierbeck.de/>

James Carman

Re: Production deployment strategy...

I looked at Christian's tutorial here:

http://www.liquid-reality.de/display/liquid/2011/09/23/Karaf+Tutorial+Part+2+-+Using+the+Configuration+Admin+Service

However, I don't see anything that specifically says that the
Blueprint container will not start until the configuration is
provided. Also, does this mean that the configuration file is there
or that each property I try to reference is there?

On Wed, Jun 13, 2012 at 11:00 AM, Achim Nierbeck
<[hidden email]> wrote:

> Hi James,
>
> you need to use managed Services via the Configuration Admin service.
> Take a look at the blueprint documentation to find some more details
> on how to create that.
> Or take a look at some of those tutorials christian did. (I'm sure he
> has a link in his signature :) )
>
> regards, Achim
>
> 2012/6/13 James Carman <[hidden email]>:
>> On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
>> <[hidden email]> wrote:
>>>
>>>
>>> You probably use start levels to work around this. OSGi services and
>>> blueprint should help you
>>> with this. You can make sure bundles only start when required services or
>>> configs are present.
>>>
>>
>> We actually don't mess with start levels. Blueprint will put the
>> bundle into the "Grace Period" until it finds everything it needs.
>> However, I am intrigued by that last part there where you said I can
>> tell BP to only start when required configs are present. How do I
>> stall the BP startup when configs aren't present?
>
>
>
> --
>
> Apache Karaf <http://karaf.apache.org/> Committer & PMC
> OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/>
> Committer & Project Lead
> OPS4J Pax for Vaadin
> <http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter & Project
> Lead
> blog <http://notizblog.nierbeck.de/>

Christian Schneider

Re: Production deployment strategy...

I am not sure about stalling the start but
update-strategy="reload"
makes the blueprint context restart when the config is changed. So it
will always reflect the newest config. In most cases this is good enough.

The config admin service always handles complete configs so it is more
about the file than the property.

Christian

Am 13.06.2012 17:07, schrieb James Carman:

> I looked at Christian's tutorial here:
>
> http://www.liquid-reality.de/display/liquid/2011/09/23/Karaf+Tutorial+Part+2+-+Using+the+Configuration+Admin+Service
>
> However, I don't see anything that specifically says that the
> Blueprint container will not start until the configuration is
> provided. Also, does this mean that the configuration file is there
> or that each property I try to reference is there?
>
> On Wed, Jun 13, 2012 at 11:00 AM, Achim Nierbeck
> <[hidden email]> wrote:
>> Hi James,
>>
>> you need to use managed Services via the Configuration Admin service.
>> Take a look at the blueprint documentation to find some more details
>> on how to create that.
>> Or take a look at some of those tutorials christian did. (I'm sure he
>> has a link in his signature :) )
>>
>> regards, Achim
>>
>> 2012/6/13 James Carman<[hidden email]>:
>>> On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
>>> <[hidden email]> wrote:
>>>>
>>>> You probably use start levels to work around this. OSGi services and
>>>> blueprint should help you
>>>> with this. You can make sure bundles only start when required services or
>>>> configs are present.
>>>>
>>> We actually don't mess with start levels. Blueprint will put the
>>> bundle into the "Grace Period" until it finds everything it needs.
>>> However, I am intrigued by that last part there where you said I can
>>> tell BP to only start when required configs are present. How do I
>>> stall the BP startup when configs aren't present?
>>
>>
>> --
>>
>> Apache Karaf<http://karaf.apache.org/> Committer& PMC
>> OPS4J Pax Web<http://wiki.ops4j.org/display/paxweb/Pax+Web/>
>> Committer& Project Lead
>> OPS4J Pax for Vaadin
>> <http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter& Project
>> Lead
>> blog<http://notizblog.nierbeck.de/>

--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
Talend Application Integration Division http://www.talend.com

James Carman

Re: Production deployment strategy...

It would be great if I could get blueprint to spit out a message when
I'm missing a property and stall the startup of the container like it
does when you don't have a required dependency!

On Wed, Jun 13, 2012 at 11:35 AM, Christian Schneider
<[hidden email]> wrote:

> I am not sure about stalling the start but
> update-strategy="reload"
> makes the blueprint context restart when the config is changed. So it will
> always reflect the newest config. In most cases this is good enough.
>
> The config admin service always handles complete configs so it is more about
> the file than the property.
>
> Christian
>
> Am 13.06.2012 17:07, schrieb James Carman:
>>
>> I looked at Christian's tutorial here:
>>
>>
>> http://www.liquid-reality.de/display/liquid/2011/09/23/Karaf+Tutorial+Part+2+-+Using+the+Configuration+Admin+Service
>>
>> However, I don't see anything that specifically says that the
>> Blueprint container will not start until the configuration is
>> provided. Also, does this mean that the configuration file is there
>> or that each property I try to reference is there?
>>
>> On Wed, Jun 13, 2012 at 11:00 AM, Achim Nierbeck
>> <[hidden email]> wrote:
>>>
>>> Hi James,
>>>
>>> you need to use managed Services via the Configuration Admin service.
>>> Take a look at the blueprint documentation to find some more details
>>> on how to create that.
>>> Or take a look at some of those tutorials christian did. (I'm sure he
>>> has a link in his signature :) )
>>>
>>> regards, Achim
>>>
>>> 2012/6/13 James Carman<[hidden email]>:
>>>>
>>>> On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
>>>> <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> You probably use start levels to work around this. OSGi services and
>>>>> blueprint should help you
>>>>> with this. You can make sure bundles only start when required services
>>>>> or
>>>>> configs are present.
>>>>>
>>>> We actually don't mess with start levels. Blueprint will put the
>>>> bundle into the "Grace Period" until it finds everything it needs.
>>>> However, I am intrigued by that last part there where you said I can
>>>> tell BP to only start when required configs are present. How do I
>>>> stall the BP startup when configs aren't present?
>>>
>>>
>>>
>>> --
>>>
>>> Apache Karaf<http://karaf.apache.org/> Committer& PMC
>>>
>>> OPS4J Pax Web<http://wiki.ops4j.org/display/paxweb/Pax+Web/>
>>> Committer& Project Lead
>>> OPS4J Pax for Vaadin
>>> <http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter& Project
>>> Lead
>>> blog<http://notizblog.nierbeck.de/>
>
>
>
> --
> Christian Schneider
> http://www.liquid-reality.de
>
> Open Source Architect
> Talend Application Integration Division http://www.talend.com
>

Christian Schneider

Re: Production deployment strategy...

Good idea can you add this as a jira issue in the aries project?

I know that you can define defaults perhaps that also helps in your case.

Christian

Am 13.06.2012 18:02, schrieb James Carman:

> It would be great if I could get blueprint to spit out a message when
> I'm missing a property and stall the startup of the container like it
> does when you don't have a required dependency!
>
> On Wed, Jun 13, 2012 at 11:35 AM, Christian Schneider
> <[hidden email]> wrote:
>> I am not sure about stalling the start but
>> update-strategy="reload"
>> makes the blueprint context restart when the config is changed. So it will
>> always reflect the newest config. In most cases this is good enough.
>>
>> The config admin service always handles complete configs so it is more about
>> the file than the property.
>>
>> Christian
>>
>> Am 13.06.2012 17:07, schrieb James Carman:
>>> I looked at Christian's tutorial here:
>>>
>>>
>>> http://www.liquid-reality.de/display/liquid/2011/09/23/Karaf+Tutorial+Part+2+-+Using+the+Configuration+Admin+Service
>>>
>>> However, I don't see anything that specifically says that the
>>> Blueprint container will not start until the configuration is
>>> provided. Also, does this mean that the configuration file is there
>>> or that each property I try to reference is there?
>>>
>>> On Wed, Jun 13, 2012 at 11:00 AM, Achim Nierbeck
>>> <[hidden email]> wrote:
>>>> Hi James,
>>>>
>>>> you need to use managed Services via the Configuration Admin service.
>>>> Take a look at the blueprint documentation to find some more details
>>>> on how to create that.
>>>> Or take a look at some of those tutorials christian did. (I'm sure he
>>>> has a link in his signature :) )
>>>>
>>>> regards, Achim
>>>>
>>>> 2012/6/13 James Carman<[hidden email]>:
>>>>> On Tue, Jun 12, 2012 at 4:59 PM, Christian Schneider
>>>>> <[hidden email]> wrote:
>>>>>>
>>>>>> You probably use start levels to work around this. OSGi services and
>>>>>> blueprint should help you
>>>>>> with this. You can make sure bundles only start when required services
>>>>>> or
>>>>>> configs are present.
>>>>>>
>>>>> We actually don't mess with start levels. Blueprint will put the
>>>>> bundle into the "Grace Period" until it finds everything it needs.
>>>>> However, I am intrigued by that last part there where you said I can
>>>>> tell BP to only start when required configs are present. How do I
>>>>> stall the BP startup when configs aren't present?
>>>>
>>>>
>>>> --
>>>>
>>>> Apache Karaf<http://karaf.apache.org/> Committer& PMC
>>>>
>>>> OPS4J Pax Web<http://wiki.ops4j.org/display/paxweb/Pax+Web/>
>>>> Committer& Project Lead
>>>> OPS4J Pax for Vaadin
>>>> <http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter& Project
>>>> Lead
>>>> blog<http://notizblog.nierbeck.de/>
>>
>>
>> --
>> Christian Schneider
>> http://www.liquid-reality.de
>>
>> Open Source Architect
>> Talend Application Integration Division http://www.talend.com
>>

--
Christian Schneider
http://www.liquid-reality.de

Open Source Architect
Talend Application Integration Division http://www.talend.com