My Corner of the Web
3. Avoiding architecture.
The final step in our escalation of bad laziness is the avoidance of architectural concerns in the mad rush to ship new features. Time and again I have heard the same tired old refrain from team leaders that say, “Sorry, that architectural concern (scalability, reliability, security, composability, etc.) is not part of our feature[1].” Oh, really? Well, then just go ahead and focus on your sprint goals and keep your head in the sand. Before we examine specific examples, let us briefly define architecture to “set the stage”.
Software Architecture – the design decisions that affect system qualities that we call the “-ilities”: Specifically, Reliability, Usability, Scalability, and Extensibility. Reliability enables the system to operate robustly in both common and uncommon conditions (i.e. peak load, extended periods of heavy use, through intermittent failures). Usability enables the system to perform its functions securely and intuitively. Scalability enables the system to handle increasing workloads without significant degradation in performance. Extensibility enables new functions to be easily added to the system. There are other “ilities” (like flexibility) but the ones we discussed are the most important. Each of these characteristics flow down from the system level to the component level because each component must do its part in these areas in order to support the system’s overall goals.
A common phrase that fits this situation is that a “system is only as strong as its weakest link”. The same holds true in architecture. So, like the avoidance of smelly code and technical debt, the avoidance of architectural weakness is usually intentional. What is different than the other types of avoidance is the excuse for the avoidance and the ramifications of such avoidance. The excuse for avoiding architectural weakness is often that the issue is “a system problem that is out of scope of my work”. In other words, the excuse is that the issue is “too big to handle now.” Unfortunately, the ramification for this avoidance is a massive hole in your architectural qualities that injures not just one part of the system but the entire system! Of course, the argument that a problem is “too big to handle immediately” is very subjective. There are often architectural features that are based on the steady accrual of thousands of techniques that mesh together to create a “bullet-proof” system. Let’s look at an example of such a technique that affects system reliability as demonstrated in Listing 7.
Listing 7 RetryConnection
package us.daconta.lazy.programmers.examples;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
/**
* Adds retry logic to reading from a URL.
* Simulates reading from a web service, database or any other server.
* @author mdaconta
*/
public class RetryConnection {
public static final int RETRY_WAIT=1000;
public static final int RETRY_ATTEMPTS=5;
public static final int BUFFER_SIZE = 4096;
public String getContent(URL url)
{
StringBuffer buffer = new StringBuffer();
int attempts = 0;
boolean done = false;
while (!done && attempts++ < RETRY_ATTEMPTS)
{
try
{
InputStream inputStream = url.openStream();
int bytesRead = 0;
byte [] byteArray = new byte[BUFFER_SIZE];
while ((bytesRead = inputStream.readNBytes(byteArray,
0, BUFFER_SIZE)) > 0)
{
buffer.append(new String(byteArray));
}
done = true;
} catch(IOException ioe)
{
System.out.println("ERROR: "
+ ioe.getMessage() + ". Retrying ...");
}
}
return buffer.toString();
}
public static void main(String args[])
{
try
{
if (args.length < 1) {
System.out.println("Usage: RetryConnection <url>");
System.exit(1);
}
URL url = new URL(args[0]);
RetryConnection accessor = new RetryConnection();
System.out.println("Getting content from:" + args[0]);
String content = accessor.getContent(url);
System.out.println("Content:" + content);
} catch (Throwable t)
{
System.out.println("ERROR - Reason: " + t.getLocalizedMessage());
t.printStackTrace();
}
}
}
In listing 7, I have implemented a simple, but often overlooked or ignored technique of adding retry logic to communicating with another system component that could either intermittently fail or take longer than expected. This can be coupled with a reasonable timeout period (or even an exponential backoff) so that your code does not wait forever. That is also the reason behind having an “attempt threshold” where you only try so many times (usually at least 3 or more times) before giving up. While the example communicates with a URL, the same retry logic would be useful for communicating with a web service or micro-service. Additionally, this code can be improved by reading the constant values (like RETRY_ATTEMPTS) from a configuration file.
Addressing architectural issues is key to a robust system because they typically affect the system as a whole and not just one team’s feature. While listing 7 is a minor example, I have witnessed numerous larger examples of architectural avoidance. Three such examples are hogging resources, shifting the workload (and blame) to other parts of the system, and not-invented-here syndrome. If you are working on a component that is IO bound, there is always a temptation to create a local memory cache to avoid frequent re-fetching. While this is a good strategy for your component, it can be a bad strategy for the overall program or system.
I have seen examples where the same data was cached multiple times due to short-sighted developers not knowing another component had already cached the same data. Rogue caches, rogue executors, rogue connection pools, rogue properties, rogue files and even rogue database tables often follow the same pattern: short-term convenience that causes long-term maintenance problems. While it should be obvious, shared resources require big-picture thinking, not myopic thinking.
The second example is when a component shifts a burden to another system component (like the database) or another service. I have seen this many times where poorly constructed queries or “one-over-the-world” queries are sent to the database and then the results are filtered locally (in memory) by the component. While it may “work” in the short-run or when the database tables are small (like often occurs on test systems), it is a ticking time bomb when the system goes into production. While sometimes these blatant examples of poor practice cause a good laugh, they also are variants of robbing Peter to pay Paul. Pay specific attention to any queries that retrieve “all records”[2] of one or more tables. You may say, “No one would be that stupid”… and you would be wrong.
Our final example of architectural avoidance, “not invented here” involves the double-edged sword of distrust. There are times when distrusting other’s code is warranted but that should be the exception and not the rule. Too many times, I have seen developers re-invent the wheel instead of reusing existing code in the system that does the same or similar function. Now, sometimes this is a failure of training because a developer may not be aware that a good alternative already exists. Unfortunately, lazy programmers often have an itchy trigger finger when they see that implementing it “quickly” in “their way” is faster then learning how someone else did it. This is especially true if data needs to be transformed or the API may not be precisely what is required. Such, simple “off by a little” scenarios should not be excuses to re-implement the code “in your ‘better’ way” but instead be opportunities to improve the system-wide utility for the benefit of the entire program.
To recap, we have covered three categories of bad-lazy techniques: taking shortcuts, sweeping dirt under the rug, and avoiding architecture. So, how do you avoid all of the “bad lazy” techniques we covered in this chapter? The first step is to be cognizant that every programmer, including yourself, will be faced with the temptation to take the expedient path. So, knowing what not to do is just as important as knowing what to do. Furthermore, being aware of these temptations will help you police your peers and assist your team leader in creating quality code. This does take courage to carry out. It takes courage to push back against schedule pressure. It takes courage to tactfully confront another developer about their coding habits. And finally, it takes courage to refactor code to support architectural goals. So, do the right thing and not the expedient thing.