2012-08-06
Optional Dependency Strategies for Java Libraries
In software, contrary to common belief, lines of code are a
liability not an asset. As you gradually accumulate code, little by little the pain sets in. The complexity
increases. It gets harder to understand. And eventually, quality suffers.
Dependencies only exacerbate this problem. With each dependency you add, you pull in more code, more APIs to learn and more potential bugs.
Transitive dependencies take this problem to 11. You now have the possible issue of conflicting dependencies! And before you know it ... you have landed in JAR Hell! So much for laughing at our Windows friends stuck in DLL Hell...
And yet, dependencies are a necessary evil. If you want to avoid reinventing the wheel or if you want to integrate with the outside world, there very often is no way to avoid them. In the spirit of doing the simplest thing that could possibly work, strive to reduce the number of dependencies to the absolute minimum required. Every single dependency you add should be weighed carefully: does the functional benefit outweigh the complexity cost?
A library or framework in broad general use (like Flyway) usually supports a wide range of scenarios for a very diverse set of users. In Flyway's case this currently means dependencies on:
Nobody in their right mind would be keen on pulling all these dependencies in just to use the subset they need. And in fact, it wouldn't even be possible as some of them are conflicting! Yes, JBoss VFS v2 and v3, it's you I'm looking at!
Each of these strategies has its pros and cons. Let's look at them in turn.
Advantage for the End-User:
Advantage for the End-User:
Advantage for the End-User:
Advantage for the End-User:
Advantage for the End-User:
A number of these strategies depend on being able to check whether a certain dependency is available at runtime and guard against using its related features if it isn't.
This sounds complicated, but it turns out to be relatively easy on the Java platform:
The 5 different strategies each deal with different scenarios and different forces that must be balanced. Even in a relatively small library like Flyway, it wasn't possible to simply rely on a single one. Know them well and know when to use them. But if there has to be a single most important guideline to remember, let it be this one:
Dependencies only exacerbate this problem. With each dependency you add, you pull in more code, more APIs to learn and more potential bugs.
Transitive dependencies take this problem to 11. You now have the possible issue of conflicting dependencies! And before you know it ... you have landed in JAR Hell! So much for laughing at our Windows friends stuck in DLL Hell...
And yet, dependencies are a necessary evil. If you want to avoid reinventing the wheel or if you want to integrate with the outside world, there very often is no way to avoid them. In the spirit of doing the simplest thing that could possibly work, strive to reduce the number of dependencies to the absolute minimum required. Every single dependency you add should be weighed carefully: does the functional benefit outweigh the complexity cost?
Special case: Libraries and Frameworks
A library or framework in broad general use (like Flyway) usually supports a wide range of scenarios for a very diverse set of users. In Flyway's case this currently means dependencies on:
- 10+ Jdbc Drivers
- Spring Jdbc
- OSGi (Equinox)
- JBoss VFS v2
- JBoss VFS v3
- Ant
- Maven
Nobody in their right mind would be keen on pulling all these dependencies in just to use the subset they need. And in fact, it wouldn't even be possible as some of them are conflicting! Yes, JBoss VFS v2 and v3, it's you I'm looking at!
How do we solve this? How do we support all users without forcing all these dependencies onto each and everyone
of them?
The answer lies in optional dependencies.
There are 5 basic strategies for dealing with optional dependencies:
- Create a separate module which depends on the base module and the optional dependency
- Create a module per optional dependency which the base module depends on.
- Mark the dependency as non-transitive and activate the functionality at runtime if present
- Use reflection and activate the functionality at runtime if present
- Use a Service Provider Interface and call the correct implementation at runtime
Each of these strategies has its pros and cons. Let's look at them in turn.
Separate Module
Typical use case:
Well-isolated dependencies that do not need to be combined together. In Flyway's case, this applies to the
Ant and Maven dependencies. You will not need the Ant dependency when executing the Maven plugin and vice
versa.
Smallest amount of code in KB as no code to support other optional dependencies is pulled in.
All dependencies are transitive.
Disadvantage for the End-User:
Must know which specific module from the library should be imported.
Can only make use of one of the optional dependencies at a time.
Can only make use of one of the optional dependencies at a time.
Advantage for the Library Developer:
Compile-time checking and IDE completion when working with the optional dependency.
Disadvantage for the Library Developer:
Two modules to maintain and release instead of one.
Module per Dependency
Typical use case:
Dependencies that require a large amount of custom code to deal with. This becomes especially relevant
in environments where application size has a direct impact on user experience. (Installation size vs
download times on mobile, ...)
Smallest amount of code in KB as no code to support other optional dependencies is pulled in.
Can depend on multiple optional dependencies at the same time.
Disadvantage for the End-User:
No transitive dependency support. Must manually reference the correct library module for the optional
dependency.
Largest amount of modules to manage overall.
Largest amount of modules to manage overall.
Advantage for the Library Developer:
Compile-time checking and IDE completion when working with the optional dependency.
Disadvantage for the Library Developer:
Two modules to maintain and release instead of one.
All access to the code of the optional dependency support module must be carefully guarded to avoid NoClassDefFoundError.
All access to the code of the optional dependency support module must be carefully guarded to avoid NoClassDefFoundError.
Non-transitive Dependency
Typical use case:
Dependencies that can be combined together. In Flyway's case, this applies to JBoss VFS, OSGi and Spring
Jdbc. It is very well possible for a JBoss or an OSGi user to have Spring on board. Being able to use both
at the same time is critical.
Advantage for the End-User:
A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.
Can depend on multiple optional dependencies at the same time.
Disadvantage for the End-User:
Must pull in some unused code to support the other missing optional dependencies.
No transitive dependency support. Must manually reference optional dependency in own project.
Advantage for the Library Developer:
Compile-time checking and IDE completion when working with the optional dependency.
A single module to maintain and release.
Disadvantage for the Library Developer:
All access to the code relying on the optional dependency must be carefully guarded to avoid
NoClassDefFoundError.
Reflection
Typical use case:
Support multiple, mutually incompatible versions of the same dependency. In Flyway's case, this applies to
JBoss VFS v2 and v3. Even though they have different package structures, they share the same artifact id.
Maven will therefore always pull in the higher version. One of them can be declared and used as regular
optional dependency, while the other must then be used via reflection (and not appear in the POM).
A single library module to depend upon.
Can depend on multiple optional dependencies at the same time.
Can depend on multiple optional dependencies at the same time.
Disadvantage for the End-User:
Must pull in some unused code to support the other missing optional dependencies.
No transitive dependency support. Must manually reference optional dependency in own project.
Advantage for the Library Developer:
No transitive dependency support. Must manually reference optional dependency in own project.
Advantage for the Library Developer:
Possibility to support multiple, mutually incompatible versions of the same dependency.
A single module to maintain and release.
Disadvantage for the Library Developer:
The code relying on the optional dependency must use reflection, sacrificing readability and compiler safety.
All access to the code relying on the optional dependency must be carefully guarded to avoid
NoClassDefFoundError.
Service Provider Interface
Typical use case:
Support various implementations of the same interface. In Flyway's case, this applies to the Jdbc drivers. Flyway supports many of them, and yet they are all accessed through the same API (Jdbc).
Support various implementations of the same interface. In Flyway's case, this applies to the Jdbc drivers. Flyway supports many of them, and yet they are all accessed through the same API (Jdbc).
A single library module to depend upon.
Smallest amount of code in KB as only code to support the common SPI is pulled in.
Smallest amount of code in KB as only code to support the common SPI is pulled in.
Disadvantage for the End-User:
No transitive dependency support. Must manually reference optional dependency in own project.
May have to configure which SPI implementation to use.
May have to configure which SPI implementation to use.
Advantage for the Library Developer:
Common SPI to program against, with compile-time checking and IDE completion.
A single module to maintain and release.
Disadvantage for the Library Developer:
Must match the SPI with its implementation at runtime. Failure to properly do so will result in Exceptions
at runtime.
Should test the library with the different SPI implementations to ensure they behave as expected.
Should test the library with the different SPI implementations to ensure they behave as expected.
Checking if a dependency is present at runtime
A number of these strategies depend on being able to check whether a certain dependency is available at runtime and guard against using its related features if it isn't.
This sounds complicated, but it turns out to be relatively easy on the Java platform:
public static boolean isPresent(String className) {
try {
Class.forName(className);
return true;
} catch (Throwable ex) {
// Class or one of its dependencies is not present...
return false;
}
}
...
if (isPresent("com.optionaldependency.DependencyClass")) {
// This block will never execute when the dependency is not present
// There is therefore no more risk of code throwing NoClassDefFoundException.
executeCodeLinkingToDependency();
}
Conclusion
The 5 different strategies each deal with different scenarios and different forces that must be balanced. Even in a relatively small library like Flyway, it wasn't possible to simply rely on a single one. Know them well and know when to use them. But if there has to be a single most important guideline to remember, let it be this one:
Always favor the convenience of the end-user over your own when designing your library or framework.
Thanks for reading this far! Find out more and stay in touch by following
me on Twitter
About Axel Fontaine
Axel Fontaine is the founder and CEO of Boxfuse the easiest way to deploy JVM and Node.js applications to AWS.
Axel is also the creator and project lead of Flyway, the open-source tool that makes database migration easy.
He is a Continuous Delivery and Immutable Infrastructure expert, a Java Champion, a JavaOne Rockstar and a regular speaker at many large international conferences including JavaOne, Devoxx, Jfokus, JavaZone, QCon, JAX, ...
You can follow him on Twitter at @axelfontaine
Architecting for Continuous Delivery and Zero Downtime
Two day intensive on-site training with Axel Fontaine
Upcoming dates
Iasi, Romania (May 10-11, 2017)
Oslo, Norway (Oct 16-17, 2017)