Jun 03 2013


Three Book Ideas

I've been gestating three different books in my head for a while. You would expect that having three books in your head is great for productivity, but it seems that the opposite is true. It's hard to decide which one to work on, so I get vapor-locked by indecision and work on none.

With a deep sigh, it's time for me to admit that none of these is going to emerge like Athena, fully-formed, from my head. To my chagrin, it looks like I actually need to sit down and do the work of writing one of them. But which one?

In the spirit of garnering feedback early and often, I'd like your input. Leave a comment, please. Which, if any, of these three books would you most want to read?

Blueprint for a Web Company

A cross between Chasing the Rabbit and The Art of Scalability. The best companies today know how to bring together a handful of ideas across several dimensions. Each of these ideas gives an individual a bigger lever to impact the company. Instead of focusing on risk avoidance, these companies make many survivable bets.

This book would show how to combine:

  • Product-oriented organizational structure
  • Data-driven decision making
  • Decoupled software architecture
  • Cloud computing and cloud-native application architecture
  • Lean software development
  • DevOps

Doing any of these things will deliver benefits. Do them all, and it's like a turbo boost for your company.

This would be a full-length book.

The Hickey/Halloway Paradigm

Clojure and Datomic express a distinct set of ideas about software design. Rich Hickey and Stu Halloway have articulated these in various presentations. As we have worked with Clojure, Datomic, and ClojureScript, and as we have built Pedestal, many of us at Relevance have adopted this paradigm for programming.

This book would articulate the principles of this style: an explicit approach to time and context, the separation of identity from state, what makes a good abstraction, and how to structure real-world systems from functional, explicit, evident components.

This would be a medium-length ebook.

The Truth About Data

E.W. Dijkstra famously said "It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration." I would make a similar case about relational database structures. We get such early training in normalization, entities, and relations that we are tempted to think of these as the "natural" form of data. Absolutely not.

Normalized databases as we practice them are not essential to the relational model, and the relational model itself is an arbitrary set of constraints applied to a much richer fundamental view of data.

In "Data and Reality," William Kent described many ways in which our models are arbitrary. "Arbitrary" not in the sense of capricious, but in the sense of representing choices from a set of potential models with equal validity.

There is another way to understand data. We can apply some very general abstractions to apprehend a data set as a kind of Platonic ideal, from which we project our actual data sets. In this view, entire databases look like points in a many-dimensional space and data models are just projections of this space into reified memory, storage, and index structures. Queries are further projections. Transactions move the database from one point in this space to another.

This would be a medium-length ebook, and may be a bit challenging.

Need Your Voice

Please leave a comment below. Which of these, if any, sounds interesting to you?

May 31 2013


Where to Find Relevancers: June Edition

Want to meet a Relevancer in person? Here's where you can find us during the month of June:

Oslo, Norway 6/10-6/14 @ 6pm
Norwegian Developers Conference
Speaking: Stuart Halloway: Day of Clojure; Day of Datomic; Simulation Testing

Oslo, Norway 6/12 @ 7pm
Oslo Socially Functional Programmers Meetup Group
Speaking: Stuart Halloway: TBD

Washington, D.C. 6/14-6/15
Ruby Nation
Speaking: Russ Olsen: Intuition and Programming

Durham, NC 6/18
West End Ruby Hack Night @ Relevance HQ
Attending: Yoko Harada

Santa Clara, CA 6/18-6/20
Velocity Conference
Attending: Michael Nygard

St. Petersburg, FL 6/20-6/21 @ 6:30pm
Front-End Conf
Speaking: Michael Parenteau: The myth of the half-brained designer... and their magic vacuum

Waltham, MA 6/28
Engineers 4 Engineers
Speaking: Russ Olsen: Fighting Robots and Flying Cement

Austin, TX 6/28-6/30
Lone Star Software Symposium
Speaking: Stuart Halloway: Sessions include: Generative Testing, Get Logical with Datalog, Pure Fun, Simulation Testing with Simulant, edn and Fressian: Flexible Languages for Data

May 29 2013


Stuart Sierra - Episode 032

cover art

This past February, I found myself driving across rural Virginia with Stuart Sierra enroute to Relevance World HQ. We had a few hours to kill, and I had recently had the opportunity to work extensively with Stuart on a Clojure project for a client. Stuart has a particular - and, in my experience, productive - way of arranging things, so I thought it would be fun and informative to record an episode exploring it.

Our conversation covered some of the same ground as Stuart's Clojure/West presentation "Clojure in the Large", so if you want to hear more, keep an eye out for that video, to be released in the coming months.

I hope you enjoy this episode!

Read More »

May 21 2013


Entropy and Evolution of a Codebase

Let's do a thought experiment about change over time. We will start with a "clean" codebase, organized into a bunch of modules. These can be classes, subsystems, namespaces... it doesn't matter for our purposes.

As time goes on, we have to make changes in the code. Suppose the changes are distributed randomly across the codebase. Some modules will get changed a lot more often than others. Empirically, the distribution of changes almost always follows a power law. That is, a few modules get changed very frequently while a decreasing number have an increasing interval between changes.

(In fact, concave shapes resembling a power law are very common in software. We see them in object lifespans, coupling, and the distribution of complexity across modules.)

We started with a clean codebase, but what does that mean? To me, it means that the code is simple, cohesive, and loosely coupled. In short, it is easy to change, and most changes are local.

With every change a developer makes, the module might lose cohesion, gain coupling, or become complex. These are entropy-increasing changes. On the other hand, the developer could split an incohesive module, decouple a coupled one, or factor out complexity. These are entropy-reducing changes.

Even with a very disciplined team, some proportion of changes will be entropy-increasing. We would all like to avoid them, but they happen anyway. I think of these as introducing stress into the codebase. The stress is trying to collapse the codebase into a Ball of Mud. We must eventually relieve that stress through refactoring. At every change, there is a chance that the module transitions from clean to dirty, or from dirty to clean.

If the odds of a clean-to-dirty transition and a dirty-to-clean transition were equal, then we should expect the codebase to reach a kind of thermodynamic equilibrium where dirtiness is conserved, but shuttled around. Sadly, the transition odds are not equal. The definition of clean means that it is easy to change. Dirty code is hard to change. We all intuitively understand that it is much harder to clean up a module than to dirty it in the first place. Therefore, the likelihood of clean-to-dirty transitions is higher than that of the reverse. From this alone, we could expect the fraction of dirty modules to increase over time. It is entropic decay.

There's a secondary effect at play that exacerbates this problem. Because it is easier to change a clean module than a dirty one, we will see some kinds of changes "sliding sideways" from a dirty module to a cleaner one. This is just another way of saying that kludges cluster. A developer who needs to change a dirty module in a difficult way is very likely to make a nearby change instead, especially under time or deadline pressure. That nearby change inevitably dirties the module it lands in, often by adding coupling.

If you imagine the modules in a codebase like cells in a Game of Life automaton, you could see cells fading from healthy blue to sickly red, then transmitting their disease elsewhere. Very occasionally, you'll see a cell or cluster of cells brightening to health as a developer restructures that area. Mostly, the codebase will decay until it must be discarded or rewritten.

Can this entropic decay be prevented?

Yes, but it is not easy. We know of several ways to avoid it.

  1. Surgical anti-entropy. Have people consistently tackle the gnarliest, smelliest parts of the system and bring them back to health.
  2. Ongoing stewardship. Have people nurture their part of the codebase indefinitely. They should have the time, skill, and desire to tend to their garden.
  3. Methodical attacks on suspected entropy strongholds. Use version control records to find the least-frequently-changed modules that are coupled to areas with high change rates. Odds are, the infrequently changed module is really dirty. The frequent changes in nearby modules indicate "sliding away" from the problem. The long time between changes on the module in the middle is a sign of sickness, not stability.
  4. Community spirit. Make sure the whole team knows that every single person is responsible for cleaning modules, fixing broken windows, reducing coupling, and reversing entropy. (This is the XP approach of Collective Code Ownership.)
  5. Disposable code. Discard the entire codebase periodically and recreate it. Do this often enough and you will create smaller and smaller codebases, each isolated from the others. Sam Umbach points out that many effective startups are built on this idea: if the startup fails, you throw the code away; if it succeeds, you throw it away and rewrite.

Some applications live for decades. These require strong anti-entropy measures. Others live for days or weeks. Dirty code is more expensive to maintain, but with such a short lifespan, who cares if these get dirty? Whatever your approach, think about the expected lifespan of the codebase. Be deliberate about your approach to entropy.

May 14 2013


Chas Emerick, Mostly Lazy - Podcast Episode 031

cover art

Chas Emerick is a well-known and respected member of the Clojure community. His contributions to the Clojure world include the Friend authentication library, the annual State of Clojure Survey, and, of course, his book, to name just a few. On top of that, he's a very insightful and interesting person. So I was thrilled to get the chance to sit down with him and record what I think turned out to be a fascinating conversation. We talked about his book, his business, his secret new project, "100% time", and the weight of the word "should".

As a bonus, although he and I disagree about whose idea this was (I still say it was his), we continued the conversation on Mostly Lazy, his podcast. That episode is available here.

Download the episode.

Read More »

May 07 2013


Jason Wolfe of Prismatic - Podcast Episode 030

cover art

One of the few unfortunate aspects to working at Relevance is that we sometimes know about people using Clojure in cool ways that we can't talk about. So when Ben Moss suggested that we talk to someone at Prismatic, I jumped at the chance. They have definitely put Clojure into production in a big and awesome way, and I was really glad to get the chance to talk to Jason Wolfe about their experience building Prismatic, including some of the cool stuff they've open sourced out of it. A big thanks to Jason, and to you, our listeners.

Download the episode.

Read More »

May 01 2013


Where to Find Relevancers: May Edition

Want to meet a Relevancer in person? Here's where you can find us during the month of May:

Little Rock, AR 5/4
Made by Few
Attending: Kevin Altman

Norfolk, VA 5/6 @ 6pm
757.rb Ruby Meetup Group
Speaking: Russ Olsen: Looking Inside Your Ruby Implementation

Sydney, Australia 5/9
YOW! Night
Speaking: Stuart Sierra: Clojure: Lisp for the Real World

Cambridge, MA 5/9 @ 6:30pm
Clojure Meetup
Attending: Gabriel Horner

Sydney, Australia 5/10-5/11
Intro to Clojure Training
Trainer: Stuart Sierra

Brisbane, Australia 5/14-5/15
Intro to Clojure Training
Trainer: Stuart Sierra

Portland, OR 5/15
Refresh PDX
Speaker: Michael Parenteau: The Myth of the Half-Brained Designer… And Their Magic Vacuum

Brisbane, Australia 5/16-5/17
YOW! Lambda Jam
Speaking: Stuart Sierra: Keynote, Data, Visibility & Abstraction; Datomic Jam

Melbourne, Australia 5/20-5/21
Intro to Clojure Training
Trainer: Stuart Sierra

Melbourne, Australia 5/21
YOW! Night
Speaking: Stuart Sierra: Clojure: Lisp for the Real World

Apr 23 2013


Lake Denman - Podcast Episode 029

cover art

Michael Parenteau suggested to me that we ought to have Lake Denman on the show. "He's done some cool stuff, man." I remember hearing about a crazy and awesome Arduino project that Lake had worked on, so I thought, "Yeah, sure, we should totally have Lake on." And I'm very glad we did: Lake has indeed worked on some amazingly creative projects, and he has even more ambitious and interesting ones in mind. I hope you enjoy listening to him talk about them on this episode as much as I enjoyed hearing them.

Read More »

Apr 22 2013


The Shoemakers

Over an email thread a little while back, one of my colleagues suggested that programming should be a required subject in high school. To survive in today's world, everyone should have a basic understanding of programming because technology is so essential to everyone's lives. His friend retorted that, "Since shoes are also essential, high school students should take a mandatory year of cobbling."

Read More »

Apr 09 2013


Ben Vandgrift - Podcast Episode 028

cover art

Pretty much every conversation I've ever had with Ben Vandgrift has involved him saying something profound or insightful. I think this episode of the podcast was no exception, as we talked about Ben's experience as a relative newcomer to Clojure, and he had some constructive (and slightly painful) feedback for the Clojure community. We also got to talk about (inc Relevance), the quarterly gathering of Relevance employees, which Ben spearheads. I hope you enjoy the show!

Read More »

Popular Tags