How to create a ConfigSource for Quarkus that knows about existing properties

We are building a machinery for ephemeral environments (“Clowder”) and one of the details is that the environment will provide different resource names, than the ones we expect in application.properties.

Not the name we requested (Picture from Pixabay)

For example, we may have the following entry in application.properties

mp.messaging.outgoing.egress.topic=ingress

We thus request a topic name of ingress, but the environment may provide us with a dynamic name of e.g. ingress-123. This means we need to dynamically override what is set up in applications.properties.

About MicroProfile Config

Quarkus is internally using the SmallRye implementation of Eclipse MicroProfile Config (MP-Config). MP-Config allows to add so called Configuration Sources, that allow to supply config objects in a programmable way from e.g. a file in a strange format or perhaps even from a central repository. The config engine ensures via the ordinal value of the source a deterministic ordering in case more than one source has a certain key.

The left example shows the config sources from MicroProfile Config (yellow boxes) and Quarkus (blue). Sources with a higher ordinal have precedence over those with a lower one.
A request for the key “a” is directly answered from the system properties config source, thus overriding the value set in application.properties.

The logical conclusion is to add a config source with an ordinal > 250 that supplies the values from the ephemeral environment, which is pictured in green and uses an ordinal of 270. Requests for “c” would then be satisfied by this source and not from application.properties.

The tricky part

Only adding a new source, that reads and parses the contents of the special file is not enough in our case. The special file does not have the config keys we need, but only pairs of requested value and replacement value:

"kafka": {
  "topics": [
    {
      "name": "platform-tmp-12345",
      "requestedName": "platform.notifications.ingress"
    },
    {
      "name": "platform-tmp-666",
      "requestedName": "platform.notifications.alerts"
    }
  ]
},

What we have to do is to find the config value in application.properties for a given key and then look up this requested name in the special file and return the new name

Unfortunately ConfigSources in MicroProfile Config do not know about each other. This means that the ClowderCS cannot easily just call getValue() and get the value from application.properties.

SmallRye Config to the rescue

Quarkus is using the SmallRye implementation as I wrote above. And this implementation allows to construct new config sources via a factory, which has the knowledge of all the previously defined sources. This allows me to read those values in the factory and pass them to my new ConfigSource:

@Override
public Iterable<ConfigSource> 
      getConfigSources(ConfigSourceContext ctx) {

    Map<String, ConfigValue> exProp = new HashMap<>();
    Iterator<String> stringIterator = ctx.iterateNames();
    while (stringIterator.hasNext()) {
        String key = stringIterator.next();
        ConfigValue value = ctx.getValue(key);
        exProp.put(key,value);
    }

    return Collections.singletonList(
          new ClowderConfigSource(..., exProp));

The context ctx allows to iterate over all the existing keys, and to retrieve the values, which are then put in the map and passed on. SmallRye Config ensures that the non-factory defined sources are available before this initialisation takes place (full docs).

Within the ConfigSource itself, the above mentioned translation then happens. I am not going to discuss this here. The full source code is available at https://github.com/RedHatInsights/clowder-quarkus-config-source

One thing to mention though is that the availability of all config properties in our source allows us to e.g define the file to be read via an entry in application.properties, which can then be overridden again via environment or system property

%dev.clowder.file=/tmp/my-config.json
clowder.file=/prod/config.json
%test.clowder.file=src/test/resources/config.json

( Entries that start with % are overrides for the given run-profile of Quarkus).

Transparent to the application

Now the last part is how to register the ConfigSource and ConfigSourceFactory. Luckily this happens via the Java ServiceLoader interface. A file called io.smallrye.config.ConfigSourceFactory is put into META-INF/services/ which provides the fully qualified name of the source.

We are going to use the source in multiple applications and have thus packaged it in its own maven repo and jar-file. The only change developers of a Quarkus application have to do is to include that jar in their application, which is done by simply adding one dependency.

<dependency>
  <groupId>com.redhat.cloud.common</groupId>
  <artifactId>clowder-quarkus-config-source</artifactId>
  <version>0.1.1</version>
</dependency>

Otherwise, it is fully transparent to them. No single line of code change required.

Shoutout

A big shoutout goes to Roberto Cortez who helped me a lot during development and was there to answer all my strange questions.