2012-08-06

Optional Dependency Strategies for Java Libraries

In software, contrary to common belief, lines of code are a liability not an asset. As you gradually accumulate code, little by little the pain sets in. The complexity increases. It gets harder to understand. And eventually, quality suffers.

Dependencies only exacerbate this problem. With each dependency you add, you pull in more code, more APIs to learn and more potential bugs.

Transitive dependencies take this problem to 11. You now have the possible issue of conflicting dependencies! And before you know it ... you have landed in JAR Hell! So much for laughing at our Windows friends stuck in DLL Hell...

And yet, dependencies are a necessary evil. If you want to avoid reinventing the wheel or if you want to integrate with the outside world, there very often is no way to avoid them. In the spirit of doing the simplest thing that could possibly work, strive to reduce the number of dependencies to the absolute minimum required. Every single dependency you add should be weighed carefully: does the functional benefit outweigh the complexity cost?

Special case: Libraries and Frameworks

A library or framework in broad general use (like Flyway) usually supports a wide range of scenarios for a very diverse set of users. In Flyway's case this currently means dependencies on:

• 10+ Jdbc Drivers
• Spring Jdbc
• OSGi (Equinox)
• JBoss VFS v2
• JBoss VFS v3
• Ant
• Maven

Nobody in their right mind would be keen on pulling all these dependencies in just to use the subset they need. And in fact, it wouldn't even be possible as some of them are conflicting! Yes, JBoss VFS v2 and v3, it's you I'm looking at!

How do we solve this? How do we support all users without forcing all these dependencies onto each and everyone of them?

The answer lies in optional dependencies.

There are 5 basic strategies for dealing with optional dependencies:

• Create a separate module which depends on the base module and the optional dependency
• Create a module per optional dependency which the base module depends on.
• Mark the dependency as non-transitive and activate the functionality at runtime if present
• Use reflection and activate the functionality at runtime if present
• Use a Service Provider Interface and call the correct implementation at runtime

Each of these strategies has its pros and cons. Let's look at them in turn.

Separate Module

Typical use case:

Well-isolated dependencies that do not need to be combined together. In Flyway's case, this applies to the Ant and Maven dependencies. You will not need the Ant dependency when executing the Maven plugin and vice versa.

Advantage for the End-User:

Smallest amount of code in KB as no code to support other optional dependencies is pulled in.

All dependencies are transitive.

Disadvantage for the End-User:

Must know which specific module from the library should be imported.
Can only make use of one of the optional dependencies at a time.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

Disadvantage for the Library Developer:

Two modules to maintain and release instead of one.

Module per Dependency

Typical use case:

Dependencies that require a large amount of custom code to deal with. This becomes especially relevant in environments where application size has a direct impact on user experience. (Installation size vs download times on mobile, ...)

Advantage for the End-User:

Smallest amount of code in KB as no code to support other optional dependencies is pulled in.

Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

No transitive dependency support. Must manually reference the correct library module for the optional dependency.
Largest amount of modules to manage overall.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

Disadvantage for the Library Developer:

Two modules to maintain and release instead of one.
All access to the code of the optional dependency support module must be carefully guarded to avoid NoClassDefFoundError.

Non-transitive Dependency

Typical use case:

Dependencies that can be combined together. In Flyway's case, this applies to JBoss VFS, OSGi and Spring Jdbc. It is very well possible for a JBoss or an OSGi user to have Spring on board. Being able to use both at the same time is critical.

Advantage for the End-User:

A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

Must pull in some unused code to support the other missing optional dependencies.

No transitive dependency support. Must manually reference optional dependency in own project.

Advantage for the Library Developer:

Compile-time checking and IDE completion when working with the optional dependency.

A single module to maintain and release.

Disadvantage for the Library Developer:

All access to the code relying on the optional dependency must be carefully guarded to avoid NoClassDefFoundError.

Reflection

Typical use case:

Support multiple, mutually incompatible versions of the same dependency. In Flyway's case, this applies to JBoss VFS v2 and v3. Even though they have different package structures, they share the same artifact id. Maven will therefore always pull in the higher version. One of them can be declared and used as regular optional dependency, while the other must then be used via reflection (and not appear in the POM).

Advantage for the End-User:

A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.

Disadvantage for the End-User:

Must pull in some unused code to support the other missing optional dependencies.
No transitive dependency support. Must manually reference optional dependency in own project.

Advantage for the Library Developer:

Possibility to support multiple, mutually incompatible versions of the same dependency.

A single module to maintain and release.

Disadvantage for the Library Developer:

The code relying on the optional dependency must use reflection, sacrificing readability and compiler safety.

All access to the code relying on the optional dependency must be carefully guarded to avoid NoClassDefFoundError.

Service Provider Interface

Typical use case:
Support various implementations of the same interface. In Flyway's case, this applies to the Jdbc drivers. Flyway supports many of them, and yet they are all accessed through the same API (Jdbc).

Advantage for the End-User:

A single library module to depend upon.
Smallest amount of code in KB as only code to support the common SPI is pulled in.

Disadvantage for the End-User:

No transitive dependency support. Must manually reference optional dependency in own project.
May have to configure which SPI implementation to use.

Advantage for the Library Developer:

Common SPI to program against, with compile-time checking and IDE completion.

A single module to maintain and release.

Disadvantage for the Library Developer:

Must match the SPI with its implementation at runtime. Failure to properly do so will result in Exceptions at runtime.
Should test the library with the different SPI implementations to ensure they behave as expected.

Checking if a dependency is present at runtime

A number of these strategies depend on being able to check whether a certain dependency is available at runtime and guard against using its related features if it isn't.

This sounds complicated, but it turns out to be relatively easy on the Java platform:

public static boolean isPresent(String className) {
    try {
        Class.forName(className);
        return true;
    } catch (Throwable ex) {
        // Class or one of its dependencies is not present...
        return false;
    }
}

...

if (isPresent("com.optionaldependency.DependencyClass")) {
    // This block will never execute when the dependency is not present
    // There is therefore no more risk of code throwing NoClassDefFoundException.
    executeCodeLinkingToDependency();
}

Conclusion

The 5 different strategies each deal with different scenarios and different forces that must be balanced. Even in a relatively small library like Flyway, it wasn't possible to simply rely on a single one. Know them well and know when to use them. But if there has to be a single most important guideline to remember, let it be this one:

Always favor the convenience of the end-user over your own when designing your library or framework.

Thanks for reading this far! Find out more and stay in touch by following me on Twitter

About Axel Fontaine

I'm an entrepreneur, public speaker and software development expert based in Munich.

I'm the creator of Sprinters. Sprinters lets you run your GitHub Actions jobs 10x cheaper on your own AWS account with secure, ephemeral, high-performance, low-cost runners within the privacy of your own VPC.

I also created CloudCaptain, previously known as Boxfuse. CloudCaptain is a cloud deployment platform enabling small and medium size companies to focus on development, while it takes care of infrastructure and operations.

Back in 2010, I bootstrapped Flyway, and grew it into the world's most popular database migration tool. Starting late 2017, I expanded the project beyond its open-source roots into a highly profitable business, acquiring many of the world's largest companies and public institutions as customers. After two years of exponential growth, I sold the company to Redgate in 2019.

In the past I also spoke regularly at many large international conferences including JavaOne, Devoxx, Jfokus, JavaZone, JAX and more about a wide range of topics including modular monoliths, immutable infrastructure and continuous delivery. As part of this I received the JavaOne RockStar speaker award. As a recognition for my contributions to overall Java industry, Oracle awarded me the Java Champion title.

You can find me on 𝕏 as @axelfontaine and email me at axel@axelfontaine.com