I've been a professional software developer for a quarter of a century now. And before that, I was writing code in my room instead of going to college. And before that I wrote simple applications in Turbo Pascal and C when I was in high school. So you might say I've got some coding experience. Of course, in those beginner days, there was no internet. So it wasn't easy to learn from other people. Nowadays, the possibilities have much expanded. But still, from time to time, I struggle with certain problems. Ever since I've started writing functional code (in my case, with Clojure) I have learned even more about coding. Since about a year and a half, I've been working on MonkeyCI, a full-blown SaaS, completely written in Clojure.

And still I'm learning. I have written several Clojure-based applications in my previous company, but I've always had trouble with how state is managed. This is certainly a common problem in functional programming. Actually, it's a common problem in programming in general, but functional applications take extra care to handle state well. This means, we want to have to do as little as possible with state. Ideally, functional code is completely pure, with no side effects whatsoever. Which is of course unachievable, since applications without side effects are useless. But still we try to isolate the "state-part" of the process as much as we can. Whole libraries have been written on how to manage state and side effects, but they remain difficult to handle.

In this blog entry, I'm lumping state and side effects together, but they are of course completely different things. State is data that is kept around, and that may be modified by one part of the code, to be read by another part. Side effects are things that have impact outside of your application. A very simple example of a side effect is writing to screen. Or reading input from the user, or sending queries to databases. You don't have full control over those things, and they likely have impact on the state of your application. For example, updating a record in the database is a side effect. But the data in the database is also state. But you can also have in-memory state, which is kept inside your process, and is destroyed when your application exits.

State and side effects are both problematic: they can be modified by other parts of your application, or even by other applications. And that makes reasoning about it difficult, and is very often a case of bugs or instability in your applications. Functional programming tries to eliminate uncertainty as much as possible, by making all code (ideally) "pure", which means it does not have state and does not do side effects. But of course, a real life, useful application requires state and side effects. It needs input from the user, should return output back to the user, etc.

In OOP, we use interfaces and different implementations to take care of this. Coupled with an IOC framework, this works fairly well. But how do you do it in functional code?

The most straightforward way to do this is just make your functions "impure": do the messy stuff right in your code. But this makes it difficult to test, and hard to reproduce problems. And I have to admit that I, too, have been doing that in many cases. Knowing that it was bad practice, but I just didn't know how to do it any other way.

Until several months ago. I had known about the interceptor pattern for a long time. And I have even used it, from time to time. Most design patterns are only needed for object-oriented programming (OOP) and tend to just not be needed when doing functional coding. But this is one pattern that is useful for functional programming. It already is widely used in HTTP servers, where requests are received, and a reply should be returned. You use interceptors to do all sorts of changes on the input request, like security checks, content negotation, etc. But it can be used for so much more. You can write your own interceptors to fetch information from the database, or save it back after you've set up your response.

MonkeyCI is an application that also has a HTTP API, but it's mostly event-driven. A build is triggered by an incoming webhook, or when the user manually starts it from the GUI. But after that, it's all events. New containers are started, jobs run, and they all send events to inform other interested parties of their progress and results. This is where I initially got stuck. I couldn't figure out an elegant way to post events in my code. I kept falling back to messy post-event calls right in the middle of my functions. Eventually, I was forced to take a step back from the event-driven architecture, even though I knew it was a bad idea. But I just didn't know how to solve it. And many of you may know, but coding is also a creative process. And you can't force inspiration, you just have to wait for it to enlighten you. And inspiration did come, eventually. After coding for several months (with ugly side effects) I finally had the idea to incorporate the interceptor pattern in my event handlers.

Why not push the side effects like starting processes and sending out events to interceptors? That's what I did, and it resulted in a major overhaul of the MonkeyCI code, but the result is worth it. I also extracted the generic part of doing event-driven code with interceptors in a separate library, called Mailman. Check it out, it may be useful to you too! In a future post, I'll dive deeper into how to use it and show you some examples.