Even large companies are still grasping at straws when it comes to good code. Meanwhile there are articles I wrote years ago which explain clearly from first principles why the correct philosophy is "Generic core, specific shell."
I actually remember early in my career working for a small engineering/manufacturing prototyping firm which did its own software, there was a senior developer there who didn't speak very good English but he kept insisting that the "Business layer" should be on top. How right he was. I couldn't imagine how much wisdom and experience was packed in such simple, malformed sentences. Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
I've seen it many times. And then every task takes longer than the last one, which is what pushes teams to start rewrites.
"There's never enough time to do it right, but always time to do it again."
These are great and succinct, yours and your teammate’s.
I still find myself debating this internally, but one objective metric is how smoothly my longer PTOs go:
The only times I haven’t received a single emergency call were when I left teammates a a large and extremely specific set of shell scripts and/or executables that do exactly one thing. No configs, no args/opts (or ridiculously minimal), each named something like run-config-a-for-client-x-with-dataset-3.ps1 that took care of everything for one task I knew they’d need. Just double click this file when you get the new dataset, or clone/rename it and tweak line #8 if you need to run it for a new client, that kind of thing.
Looking inside the scripts/programs looks like the opposite of all of the DRY or any similar principles I’ve been taught (save for KISS and others similarly simplistic)
But the result speaks for itself. The further I go down that excessively basic path, the more people can get work done without me online, and I get to enjoy PTO. Anytime i make a slick flexible utility with pretty code and docs, I get the “any chance you could hop on?” text. Put the slick stuff in the core libraries and keep the executables dumb
Yes I feel that when to apply certain techniques is frequently under-discussed. But I can't blame people for err-ing on the side of 'do everything properly' - as this makes life more pleasant in teams.
Although I think if you squint, the principle still applies to your example. The further you get from the 'core' of your platform/application/business/what-have-you, the less abstract you need to be.
> The more specific, the more brittle. The more general, the more stable. Concerns evolve/decay at different speeds, so do not couple across shearing layers. Notice how grammar/phonology (structure) changes slowly while vocabulary (functions, services) changes faster.
...
> Coupling across layers invites trouble (e.g. encoding business logic with “intuitive” names reflecting transient understanding). When requirements shift (features, regulations), library maintainers introduce breaking changes or new processor architectures appear, our stable foundations, complected with faster-moving parts, still crack!
Bertrand Meyer suggested another way to consider this that ends up in a similar place.
For concerns of code complexity and verification, code that asks a question and code that acts on the answers should be separated. Asking can be done as pure code, and if done as such, only ever needs unit tests. The doing is the imperative part, and it requires much slower tests that are much more expensive to evolve with your changing requirements and system design.
The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable, and they are easy to accidentally expose to third party code through the addition of subsequent features, even if initially they are unreachable. Sun biffed this way a couple of times with Java.
But for non crosscutting concerns this advice can also be a step toward FC/IS, both in structuring the code and acclimating devs to the paradigm. Because you can start extracting pure code sections in place.
Command-Query Separation is the term for that. However, I find this statement odd:
> having functions that do things without verifying preconditions are exploitable
Why would you do this? The separation between commands and queries does not mean that executing a command must succeed. It can still fail. Put queries inside the commands (but do not return the query results, that's the job of the query itself) and branch based on the results. After executing a command which may fail, you can follow it with a query to see if it succeeded and, if not, why not.
In asynchronous environments, you may not be able to repeat the same query with the same result (unless you control a cache of results, which has its own issues). If some condition is determined by the command’s implementation that subsequent code is interested in (a condition that isn’t preventing the command from succeeding), it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
> it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
You don't need the command to return anything (though it can be more efficient or convenient). It can set state indicating, "Hey, I was called but by the time I tried to do the thing the world and had changed and I couldn't. Try using a lock next time."
if (query(?)) {
command(x)
result := status(x) // ShouldHaveUsedALockError
}
The caller can still obtain a result following the command, though it does mean the caller now has to explicitly retrieve a status rather than getting it in the return value.
Where is that state stored, in an environment where the same command could be executed with the same parameters but resulting in a different status, possibly in parallel? How do you connect the particular command execution with the particular resulting status? And if you manage to do so, what is actually won over the command just returning the status?
I’d argue that the separation makes things worse here, because it creates additional hidden state.
Also, as I stated, this is not about error handling.
CQRS should really only guide you to designing separate query and command interfaces. If your processing is asynchronous then you have no choice but to have state about processing-in-flight, and your commands should return an acknowledgement of successful receipt of valid commands with a unique identifier for querying progress or results. If your processing is synchronous make your life easier by just returning the result. Purity of CQRS void-only commands is presentation fodder, not practicality.
(One might argue that all RPC is asynchronous; all such arguments eventually lead to message buses, at-least-once delivery, and the reply-queue pattern, but maybe that's also just presentation fodder.)
The example in the wiki page is far more rudimentary than the ones I encountered when I was shown this concept. Trivial, in fact.
CQS will rely on composition to do any If A Then B work, rather than entangling the two. Nothing forces composition except information hiding. So if you get your interface wrong someone can skip over a query that is meant to short circuit the command. The constraint system in Eiffel I don’t think is up to providing that sort of protection on its own (and the examples I was given very much assumed not). Elixir’s might end up better, but not by a transformative degree. And it remains to be seen how legible that code will be seen as by posterity.
That's still not really answering my question for you, which was less clear than intended. To restate it:
> The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable
My understanding of your comment was that "this advice" is CQS. So you're saying that CQS commands do not verify preconditions and that this is a weakness in CQS, in particular.
Where did you get the idea that CQS commands don't verify preconditions? I've never seen anything in any discussion of it, including my (admittedly 20 years ago) study of Eiffel.
You may have a command sub-routine that is used by multiple higher-level commands, or even called multiple times within by a higher-level command. If the validation lives in the subroutine, that validation will be called multiple times, even when it only needs to be called once.
So you are forced to choose either efficiency or the security of colocating validation, which makes it impossible to call the sub-routine with unvalidated input.
hinkley poses this as a fault in CQS, but CQS does not require your commands to always succeed. Command-Query Separation means your queries return values, but produce no effects, and your commands produce effects, but return no values. Nothing in that requires you to have a command which always succeeds or commands which don't make use of queries (queries cannot make use of commands, though). So a better question than what I originally posed:
My "Why would you do this?" is better expanded to: Why would you use CQS in a way that makes your system less secure (or safe or whatever) when CQS doesn't actually require that?
Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say.
I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
What makes it hard to reason about is that your code is one-dimensional, you have functions like `getExpiredUsers` and `generateExpiryEmails` which could be expressed as composition of more general functions. Here is how I would have written it in JavaScript:
const emails = db.getUsers()
.filter(user => user.isExpired(Date.now())) // Some property every user has
.map(generateExpiryEmail); // Maps a single user to a message
email.bulkSend(emails);
The idea is that you have small but general functions, methods and properties and then use higher-order functions and methods to compose them on the fly. This makes the code two-dimensional. The outer dimension (`filter` and `map`) tells the reader what is done (take all users, pick out only some, then turn each one into something else) while the outer dimension tells you how it is done. Note that there is no function `getExpiredUsers` that receives all users, instead there is a simple and more general `isExpired` method which is combined with `filter` to get the same result.
In a functional language with pipes it could be written in an arguably even more elegant design:
I guess I just never encounter code like this in the big enterprise code bases I have had to weed through.
Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Do you just do the const emails =
three different times?
In my coding world it looks a lot like doing a
SELECT * ON users WHERE isExpired < Date.now
but in some cases you just grab it all, loop through it all, and do little switches to do different things based on different isExpired.
If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Well, in that case you wouldn't want to pipe them all through generateExpiryEmail.
But perhaps you can write a more generic function like generateExpiryEmailOrWhatever that understands the user object and contains the logic for what type of email to draft. It might need to output some flag if, for a particular user, there is no need to send an email. Then you could add a filter before the final (send) step.
myCoolSubroutine = do
now <- getCurrentTime
users <- getUsers
forM users (sendEmail now)
sendEmail now user =
if user.expiry <= now
then sendExpiryEmail user
else sendNonExpiryEmail user
> Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
>
> Do you just do the const emails =
>
> three different times?
If it's just two or three cases I might actually just copy-paste the entire thing. But let's assume we have twenty or so cases. I'll use Python notation because that's what I'm most familiar with. When I write `Callable[[T, U], V]` that means `(T, U) -> V`.
Let's first process one user at a time. We can define an enumeration for all our possible categories of user. Let's call this enumeration `UserCategory`. Then we can define a "categorization function" type which maps a user to its category:
type UserCategorization = Callable[[User], UserCategory]
I can then map each user to a tuple of category and user:
categorized_users = map(categorize, db.get_users()) # type Iterable[tuple[UserCategory, User]]
Now I need a mapping from user category to processing function. I'll assume we call the processing function for side effects only and that it has no return value (`None` in Python):
type ProcessingSpec = Mapping[UserCategory, Callable[[User], None]
This mapping uses the user category to look up a function to apply to a user. We can now put it all together: map each user to a pair of the user's category and the user, then for each pair use the mapping to look up the processing function:
def process_users(how: ProcessingSpec, categorize: UserCategorization) -> None:
categorized_users = map(categorize, db.get_users())
for category, user in categorized_users:
process = how[category]
process(user)
OK, that's processing one user a time, but what if we want to process users in batches? Meaning I want to get all expired users first, and then send a message to all of them at once instead of one at a time. We can actually reuse most of our code because how how generic it is. The main difference is that instead of using `map` we want to use some sort of `group_by` function. There is `itertools.groupby` in the Python standard library, but it's not exactly what we need, so let's write our own:
def group_by[T, U](what: Iterable[T], key: Callable[[T], U]) -> Mapping[U, list[T]]:
result = defaultdict(list)
# When we try to look up a key that does not exist defaultdict will create a new
# entry with an empty list under that key
for x in what:
result[key(x)].append(x)
return x
Now we can categorize our users into batches based on their category:
batches = group_by(db.get_users(), categorize)
To process these batches we need a mapping from batch to a function which process an iterable of users instead of just a single user.
type BatchProcessingSpec = Mapping[UserCategory, Callable[[Iterable[User]], None]
Now we can put it all together:
def process_batched_users(how: BatchProcessingSpec, categorize: UserCategorization) -> None:
batches = group_by(db.get_users(), categorize)
for category, users in batches:
process = how[category]
process(users)
There are quite a lot of small building block functions, and if all I was doing was sending emails to users it would not make sense to write these small function that add indirection. However, in a large application these small functions become generic building blocks that I can use in higher-order functions to define more concrete routines. The `group_by` function can be used for many other purposes with any type. The categorization function was used for both one-at-a-time and batch processing.
I have been itching to write a functional programming book for Python. I don't mean a "here is how to do FP in Python" book, you don't need that, the documentation of the standard library is good enough. I mean a "learn how to think FP in general, and we are going to use Python because you probably already know it". Python is not a functional language, but it is good enough to teach the principles and there is value in doing things with "one hand tied behind your back". The biggest hurdle in the past to learning FP was that books normally teach FP in a functional language, so now the reader has to learn two completely new things.
I may have gotten nerd sniped here, but I believe all of these examples so far have some subtle errors. Using elixir syntax, I would think something like this covers most of the cases:
expiry_date = DateTime.now!("Etc/UTC")
query =
from u in User,
where:
u.expiry_date > ^expiry_date
and u.expiry_email_sent == false,
select: u
MyAppRepo.all(query)
|> Enum.map(u, &generate_expiry_emails(&1, expiry_date))
|> Email.bulkSend() # Returns {:ok, %User{}} or {:err, _reason}
|> Enum.filter(fn
{:ok, _} -> true
_ -> false
end)
|> Enum.map(fn {:ok, user} ->
User.changeset(user, %{expiry_email_sent: true})
|> Repo.update()
end)
Mainly a lot of these examples do the expiry filtering on the application side instead of the database side, and most would send expiry emails multiple times which may or may not be desired behavior, but definitely isn't the best behavior if you automatically rerun this job when it fails.
----
Edit: I actually see a few problems with this, too, since Email.bulkSend probably shouldn't know about which user each email is for. I always see a small impedance mismatch with this sort of pipeline, since if we sent the emails individually it would be easy to wrap it in a small function that passes the user through on failure.
If I were going to build a user contacting system like this I would probably want a separate table tracking emails sent, and I think that the email generation could be made pure, the function which actually sends email should probably update a record including a unique email_type id and a date last sent, providing an interface like: `send_email(user_query, email_id, email_template_function)`
The nice thing with the Elixir example is that you can easily `tap()` to inspect how the data looks at any point in the pipeline. You can also easily insert steps into the pipeline, or reuse pipeline steps. And due to the way modules are usually organized, it would more realistically read like this, if we were in a BulkEmails module:
The nice thing here is that we can easily log to the console, and also filter out nil expiry emails. In production code, `generate_expiry_email/1` would likely return a Result (a tuple of `{:ok, email}` or `{:error, reason}`), so we could complicate this a bit further and collect the errors to send to a logger, or to update some flag in the db.
It just becomes so easy to incrementally add functionality here.
---
Quick syntax reference for anyone reading:
- Pipelines apply the previous result as the first argument of the next function
- The `/1` after a function name indicates the arity, since Elixir supports multiple dispatch
That's pretty hardcore, like you want to restrict the runtime substitution of function calls with their result values? Even Haskell doesn't go that far.
Generally you'd distinguish which function call introduces the error with the function call stack, which would include the location of each function's call-site, so maybe the "low-effort" label is accurate. But I could see a benefit in immediately knowing which functions are "pure" and "impure" in terms of manipulating non-local state. I don't think it changes any runtime behavior whatsoever, really, unless your runtime schedules function calls on an async queue and relies on the order in code for some reason.
It has been so long since I worked on the code that had chaining functions and caused problems that I am not sure I can do justice to describing the problems.
I vaguely remember the problem was one function returned a very structured array dealing with regex matches. But there was something wrong with the regex where once in a blue moon, it returned something odd.
So, the chained functions did not error. It just did something weird.
Whenever weird problems would pop up, it was always passed to me. And when I looked at it, I said, well...
I am going to rewrite this chain into steps and debug each return. Then run through many different scenarios and that was how I figured out the regex was not quite correct.
You could write the logic in a more straight forward, but less composable way, so that all the logic resides in one pure function. This way you can also keep the code to only loop over the users once.
I would have written each statement on its own line:
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);
email.bulkSend(expiryEmails);
This is not only much easier to read, it's also easier to follow in a stack trace and it's easier to debug. IMO it's just flat out better unless you're code golfing.
I'd also combine the first two steps by creating a DB query that just gets expired users directly rather than fetching all users and filtering them in memory:
expiredUsers = db.getExpiredUsers(Date.now());
Now I'm probably mostly getting zero or a few users rather than thousands or millions.
This is actually closer to the way the first draft of this article was written. Unfortunately, some readability was lost to make it fit on a single page. 100% agree that a statement like this is harder to reason about and should be broken up into multiple statements or chained to be on multiple lines.
Yeah. I did not mention what I would do, but what you wrote is pretty much what I prefer. I guess nobody likes it these days because it is old procedural style.
There's nothing procedural about binding return values to variables, so long as you aren't mutating them. Every functional language lets you do that. That's `let ... in` in Haskell.
What if a FCF (functional core function) calls another FCF which calls another FCF? Or do we do we rule out such calls?
Object Orientation is only a skin-deep thing and it boils down to functions with call stack. The functions, in turn, boil down to a sequenced list of statements with IF and GOTO here and there. All that boils boils down to machine instructions.
So, at function level, it's all a tree of calls all the way down. Not just two layers of crust and core.
Functional core usually means pure functional functions, aka the return value is know if the arguments is known, no side effects required. All the side effects is then pushed up the imperative shell.
You’ll find usually that side effect in imperative actions is usually tied to the dependencies (database, storage, ui, network connections). It can be quite easy to isolate those dependencies then.
It’s ok to have several layers of core. But usually, it’s quite easy to have the actual dependency tree with interfaces and have the implementation as leaves for each node. But the actual benefits is very easy testing and validation. Also fast feedback due to only unit tests is needed for your business logic.
I saw Gary posted his blog link on twitter, and I really like his article. I really didn't expect it to surface up at this moment (2025), and it's referred from a google blog. :shrug:
Sometimes, sure - but sometimes, passing around a fat wrapper around a DB cursor is worse, and the code would be better off paginating and materializing each page of data in memory. As usual, it depends.
I invented this pattern when I was working on a small ecommerce system (written in Scheme, yay!) in the early 2000s. It just became much easier to do all the pricing calculations, which were subject to market conditions and customer choices, if I broke it up into steps and verified each step as a side-effect-free, data-in-data-out function.
Of course by "invented" I mean that far smarter people than me probably invented it far earlier, kinda like how I "invented" intrusive linked lists in my mid-teens to manage the set of sprites for a game. The idea came from my head as the most natural solution to the problem. But it did happen well before the programming blogosphere started making the pattern popular.
I don't really like the example (and it's from Google) because, beyond the general concept, it seems like the trigger for sending emails is calling bulkSend with Date.now() instead of the user actually triggering an email when it's really expired: user.subscriptionEndDate change to < Date.now().
This works right up to the point where you try to make the code to support opening transactions functional. :D
Some things are flat out imperative in nature. Open/close/acquire/release all come to mind. Yes, the RAI pattern is nice. But it seems to imply the opposite? Functional shell over an imperative core. Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
Edit: I certainly think having some sort of affordance in place to indicate if you are in different sections is nice.
> Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
That's not what functional core, imperative shell means though. It's a given that CPUs aren't functional. The advice is for people programming in languages that have expressions - ruby, in the case of the original talk. The functional paradigm mostly assumes automatic memory management.
Right, I was just using that as "at the extreme" and how it largely exists to allow you to put a functional feel on top of the imperative below it.
I'm sympathetic to the idea, as you can see it in most instruction manuals that people are likely to consume. The vast majority of which (all of them?) are imperative in nature. There is something about the "for the humans" layer being imperative. Step by step, if you will.
I don't know that it fully works, though. I do think you are well served being consistent in how you layer something. Where all code at a given layer should probably stick to the same styles. But knowing which should be the outer and which the inner? I'm not clear that we have to pick, here. Feel free to have more than two layers. :D
Right, but even in those, you typically have the more imperative operations as the lower levels, no? Especially when you have things where the life cycle of what you are starting is longer than the life cycle of the code that you use to do it?
Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
> but even in those, you typically have the more imperative operations as the lower levels
Yes, the monadic part is the functional core, and the runtime is the imperative shell.
> Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
What do you mean by Monad trick? That's precisely the kind of thing the IO monad exists for. If you need to fetch things on an API: IO. If you need to read/save things on a DB: IO. DB Transaction: IO.
I have not seen too many (any?) times where the monad trick is done in such a way that they don't combine everything in a single context wrapper and talk about the "abnormal" case where things don't complete during execution.
Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)
My idea here is that, in many domains, you will have operations that are somewhat definitionally in the imperative camp. OpenTransaction being the easy example.
Can you implement it using functional code? Yes. Just make sure you wind up with partial states. And often times you are best off explicitly not using the RAI pattern for some of these. (I have rarely seen examples where they deal with this. Creating and reconciling transactions often have to be separate pieces of code. And the reconcile code cannot, necessarily, fallback to create a transaction if they get a "not found" fault.)
Even large companies are still grasping at straws when it comes to good code. Meanwhile there are articles I wrote years ago which explain clearly from first principles why the correct philosophy is "Generic core, specific shell."
I actually remember early in my career working for a small engineering/manufacturing prototyping firm which did its own software, there was a senior developer there who didn't speak very good English but he kept insisting that the "Business layer" should be on top. How right he was. I couldn't imagine how much wisdom and experience was packed in such simple, malformed sentences. Nothing else matters really. Functional vs imperative is a very minor point IMO, mostly a distraction.
> Even large companies are still grasping at straws when it comes to good code
Probably many reasons for this, but what I've seen often is that once the code base has been degraded, it's a slippery slope downhill after that.
Adding functionality often requires more hacks. The alternative is to fix the mess, but that's not part of the task at hand.
I've seen it many times. And then every task takes longer than the last one, which is what pushes teams to start rewrites. "There's never enough time to do it right, but always time to do it again."
These are great and succinct, yours and your teammate’s.
I still find myself debating this internally, but one objective metric is how smoothly my longer PTOs go:
The only times I haven’t received a single emergency call were when I left teammates a a large and extremely specific set of shell scripts and/or executables that do exactly one thing. No configs, no args/opts (or ridiculously minimal), each named something like run-config-a-for-client-x-with-dataset-3.ps1 that took care of everything for one task I knew they’d need. Just double click this file when you get the new dataset, or clone/rename it and tweak line #8 if you need to run it for a new client, that kind of thing.
Looking inside the scripts/programs looks like the opposite of all of the DRY or any similar principles I’ve been taught (save for KISS and others similarly simplistic)
But the result speaks for itself. The further I go down that excessively basic path, the more people can get work done without me online, and I get to enjoy PTO. Anytime i make a slick flexible utility with pretty code and docs, I get the “any chance you could hop on?” text. Put the slick stuff in the core libraries and keep the executables dumb
Yes I feel that when to apply certain techniques is frequently under-discussed. But I can't blame people for err-ing on the side of 'do everything properly' - as this makes life more pleasant in teams. Although I think if you squint, the principle still applies to your example. The further you get from the 'core' of your platform/application/business/what-have-you, the less abstract you need to be.
I’d love to know more, do you have any links to your articles?
"Specific on the surface, generic underneath" (Medium paywalled): https://medium.com/tech-renaissance/generic-internals-specif...
> The more specific, the more brittle. The more general, the more stable. Concerns evolve/decay at different speeds, so do not couple across shearing layers. Notice how grammar/phonology (structure) changes slowly while vocabulary (functions, services) changes faster.
...
> Coupling across layers invites trouble (e.g. encoding business logic with “intuitive” names reflecting transient understanding). When requirements shift (features, regulations), library maintainers introduce breaking changes or new processor architectures appear, our stable foundations, complected with faster-moving parts, still crack!
https://alexalejandre.com/programming/coupling-language-and-...
Bertrand Meyer suggested another way to consider this that ends up in a similar place.
For concerns of code complexity and verification, code that asks a question and code that acts on the answers should be separated. Asking can be done as pure code, and if done as such, only ever needs unit tests. The doing is the imperative part, and it requires much slower tests that are much more expensive to evolve with your changing requirements and system design.
The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable, and they are easy to accidentally expose to third party code through the addition of subsequent features, even if initially they are unreachable. Sun biffed this way a couple of times with Java.
But for non crosscutting concerns this advice can also be a step toward FC/IS, both in structuring the code and acclimating devs to the paradigm. Because you can start extracting pure code sections in place.
Command-Query Separation is the term for that. However, I find this statement odd:
> having functions that do things without verifying preconditions are exploitable
Why would you do this? The separation between commands and queries does not mean that executing a command must succeed. It can still fail. Put queries inside the commands (but do not return the query results, that's the job of the query itself) and branch based on the results. After executing a command which may fail, you can follow it with a query to see if it succeeded and, if not, why not.
https://en.wikipedia.org/wiki/Command%E2%80%93query_separati...
In asynchronous environments, you may not be able to repeat the same query with the same result (unless you control a cache of results, which has its own issues). If some condition is determined by the command’s implementation that subsequent code is interested in (a condition that isn’t preventing the command from succeeding), it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
> it’s generally more robust for the command to return that information to the caller, who then can make use of it. But now the command is also a query.
You don't need the command to return anything (though it can be more efficient or convenient). It can set state indicating, "Hey, I was called but by the time I tried to do the thing the world and had changed and I couldn't. Try using a lock next time."
The caller can still obtain a result following the command, though it does mean the caller now has to explicitly retrieve a status rather than getting it in the return value.Where is that state stored, in an environment where the same command could be executed with the same parameters but resulting in a different status, possibly in parallel? How do you connect the particular command execution with the particular resulting status? And if you manage to do so, what is actually won over the command just returning the status?
I’d argue that the separation makes things worse here, because it creates additional hidden state.
Also, as I stated, this is not about error handling.
CQRS should really only guide you to designing separate query and command interfaces. If your processing is asynchronous then you have no choice but to have state about processing-in-flight, and your commands should return an acknowledgement of successful receipt of valid commands with a unique identifier for querying progress or results. If your processing is synchronous make your life easier by just returning the result. Purity of CQRS void-only commands is presentation fodder, not practicality.
(One might argue that all RPC is asynchronous; all such arguments eventually lead to message buses, at-least-once delivery, and the reply-queue pattern, but maybe that's also just presentation fodder.)
The example in the wiki page is far more rudimentary than the ones I encountered when I was shown this concept. Trivial, in fact.
CQS will rely on composition to do any If A Then B work, rather than entangling the two. Nothing forces composition except information hiding. So if you get your interface wrong someone can skip over a query that is meant to short circuit the command. The constraint system in Eiffel I don’t think is up to providing that sort of protection on its own (and the examples I was given very much assumed not). Elixir’s might end up better, but not by a transformative degree. And it remains to be seen how legible that code will be seen as by posterity.
That's still not really answering my question for you, which was less clear than intended. To restate it:
> The one place this advice falls down is security - having functions that do things without verifying preconditions are exploitable
My understanding of your comment was that "this advice" is CQS. So you're saying that CQS commands do not verify preconditions and that this is a weakness in CQS, in particular.
Where did you get the idea that CQS commands don't verify preconditions? I've never seen anything in any discussion of it, including my (admittedly 20 years ago) study of Eiffel.
> Why would you do this?
Performance and re-use are two possible reasons.
You may have a command sub-routine that is used by multiple higher-level commands, or even called multiple times within by a higher-level command. If the validation lives in the subroutine, that validation will be called multiple times, even when it only needs to be called once.
So you are forced to choose either efficiency or the security of colocating validation, which makes it impossible to call the sub-routine with unvalidated input.
Perhaps I was unclear, to add to my comment:
hinkley poses this as a fault in CQS, but CQS does not require your commands to always succeed. Command-Query Separation means your queries return values, but produce no effects, and your commands produce effects, but return no values. Nothing in that requires you to have a command which always succeeds or commands which don't make use of queries (queries cannot make use of commands, though). So a better question than what I originally posed:
My "Why would you do this?" is better expanded to: Why would you use CQS in a way that makes your system less secure (or safe or whatever) when CQS doesn't actually require that?
I never liked encountering code that chains functions calls together like this
email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now())));
Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say.
I have to wonder if programming should have kept pascals distinction between functions that only return one thing and procedures that go off and manipulate other things and do not give a return value.
https://docs.pascal65.org/en/latest/langref/funcproc/
> email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now())));
What makes it hard to reason about is that your code is one-dimensional, you have functions like `getExpiredUsers` and `generateExpiryEmails` which could be expressed as composition of more general functions. Here is how I would have written it in JavaScript:
The idea is that you have small but general functions, methods and properties and then use higher-order functions and methods to compose them on the fly. This makes the code two-dimensional. The outer dimension (`filter` and `map`) tells the reader what is done (take all users, pick out only some, then turn each one into something else) while the outer dimension tells you how it is done. Note that there is no function `getExpiredUsers` that receives all users, instead there is a simple and more general `isExpired` method which is combined with `filter` to get the same result.In a functional language with pipes it could be written in an arguably even more elegant design:
I also like Python's generator expressions which can express `map` and `filter` as a single expression:I guess I just never encounter code like this in the big enterprise code bases I have had to weed through.
Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data....
Do you just do the const emails =
three different times?
In my coding world it looks a lot like doing a SELECT * ON users WHERE isExpired < Date.now
but in some cases you just grab it all, loop through it all, and do little switches to do different things based on different isExpired.
But perhaps you can write a more generic function like generateExpiryEmailOrWhatever that understands the user object and contains the logic for what type of email to draft. It might need to output some flag if, for a particular user, there is no need to send an email. Then you could add a filter before the final (send) step.
since were just making up functions..
The whole pipeline thing is a red herring IMO.> Question. If you want to do one email for expired users and another for non expired users and another email for users that somehow have a date problem in their data.... > > Do you just do the const emails = > > three different times?
If it's just two or three cases I might actually just copy-paste the entire thing. But let's assume we have twenty or so cases. I'll use Python notation because that's what I'm most familiar with. When I write `Callable[[T, U], V]` that means `(T, U) -> V`.
Let's first process one user at a time. We can define an enumeration for all our possible categories of user. Let's call this enumeration `UserCategory`. Then we can define a "categorization function" type which maps a user to its category:
I can then map each user to a tuple of category and user: Now I need a mapping from user category to processing function. I'll assume we call the processing function for side effects only and that it has no return value (`None` in Python): This mapping uses the user category to look up a function to apply to a user. We can now put it all together: map each user to a pair of the user's category and the user, then for each pair use the mapping to look up the processing function: OK, that's processing one user a time, but what if we want to process users in batches? Meaning I want to get all expired users first, and then send a message to all of them at once instead of one at a time. We can actually reuse most of our code because how how generic it is. The main difference is that instead of using `map` we want to use some sort of `group_by` function. There is `itertools.groupby` in the Python standard library, but it's not exactly what we need, so let's write our own: Now we can categorize our users into batches based on their category: To process these batches we need a mapping from batch to a function which process an iterable of users instead of just a single user. Now we can put it all together: There are quite a lot of small building block functions, and if all I was doing was sending emails to users it would not make sense to write these small function that add indirection. However, in a large application these small functions become generic building blocks that I can use in higher-order functions to define more concrete routines. The `group_by` function can be used for many other purposes with any type. The categorization function was used for both one-at-a-time and batch processing.I have been itching to write a functional programming book for Python. I don't mean a "here is how to do FP in Python" book, you don't need that, the documentation of the standard library is good enough. I mean a "learn how to think FP in general, and we are going to use Python because you probably already know it". Python is not a functional language, but it is good enough to teach the principles and there is value in doing things with "one hand tied behind your back". The biggest hurdle in the past to learning FP was that books normally teach FP in a functional language, so now the reader has to learn two completely new things.
I may have gotten nerd sniped here, but I believe all of these examples so far have some subtle errors. Using elixir syntax, I would think something like this covers most of the cases:
Mainly a lot of these examples do the expiry filtering on the application side instead of the database side, and most would send expiry emails multiple times which may or may not be desired behavior, but definitely isn't the best behavior if you automatically rerun this job when it fails.----
Edit: I actually see a few problems with this, too, since Email.bulkSend probably shouldn't know about which user each email is for. I always see a small impedance mismatch with this sort of pipeline, since if we sent the emails individually it would be easy to wrap it in a small function that passes the user through on failure.
If I were going to build a user contacting system like this I would probably want a separate table tracking emails sent, and I think that the email generation could be made pure, the function which actually sends email should probably update a record including a unique email_type id and a date last sent, providing an interface like: `send_email(user_query, email_id, email_template_function)`
In Elixir this would be written as:
I think Elixir hits the nail on the head when it comes to finding the right balance between functional and imperative style code.The nice thing with the Elixir example is that you can easily `tap()` to inspect how the data looks at any point in the pipeline. You can also easily insert steps into the pipeline, or reuse pipeline steps. And due to the way modules are usually organized, it would more realistically read like this, if we were in a BulkEmails module:
The nice thing here is that we can easily log to the console, and also filter out nil expiry emails. In production code, `generate_expiry_email/1` would likely return a Result (a tuple of `{:ok, email}` or `{:error, reason}`), so we could complicate this a bit further and collect the errors to send to a logger, or to update some flag in the db.It just becomes so easy to incrementally add functionality here.
---
Quick syntax reference for anyone reading:
- Pipelines apply the previous result as the first argument of the next function
- The `/1` after a function name indicates the arity, since Elixir supports multiple dispatch
- `&fun/1` expands to `fn arg -> fun(arg) end`
- `&fun(&1, "something")` expands to `fn arg -> fun(arg, "something") end`
Not sure I like how the binding works for user in this example, but tbh, I don't really have any better idea.
Writing custom monad syntax is definitely quite a nice benefit of functional languages IMO.
That's pretty hardcore, like you want to restrict the runtime substitution of function calls with their result values? Even Haskell doesn't go that far.
Generally you'd distinguish which function call introduces the error with the function call stack, which would include the location of each function's call-site, so maybe the "low-effort" label is accurate. But I could see a benefit in immediately knowing which functions are "pure" and "impure" in terms of manipulating non-local state. I don't think it changes any runtime behavior whatsoever, really, unless your runtime schedules function calls on an async queue and relies on the order in code for some reason.
My verdict is, "IDK", but worth investigating!
It has been so long since I worked on the code that had chaining functions and caused problems that I am not sure I can do justice to describing the problems.
I vaguely remember the problem was one function returned a very structured array dealing with regex matches. But there was something wrong with the regex where once in a blue moon, it returned something odd.
So, the chained functions did not error. It just did something weird.
Whenever weird problems would pop up, it was always passed to me. And when I looked at it, I said, well...
I am going to rewrite this chain into steps and debug each return. Then run through many different scenarios and that was how I figured out the regex was not quite correct.
You could write the logic in a more straight forward, but less composable way, so that all the logic resides in one pure function. This way you can also keep the code to only loop over the users once.
email.sendBulk(generateExpiryEmails(db.getUsers(), Date.now()));
I would have written each statement on its own line:
var users = db.getUsers();
var expiredUsers = getExpiredUsers(users, Date.now());
var expiryEmails = generateExpiryEmails(expiredUsers);
email.bulkSend(expiryEmails);
This is not only much easier to read, it's also easier to follow in a stack trace and it's easier to debug. IMO it's just flat out better unless you're code golfing.
I'd also combine the first two steps by creating a DB query that just gets expired users directly rather than fetching all users and filtering them in memory:
expiredUsers = db.getExpiredUsers(Date.now());
Now I'm probably mostly getting zero or a few users rather than thousands or millions.
(author here)
This is actually closer to the way the first draft of this article was written. Unfortunately, some readability was lost to make it fit on a single page. 100% agree that a statement like this is harder to reason about and should be broken up into multiple statements or chained to be on multiple lines.
Yeah. I did not mention what I would do, but what you wrote is pretty much what I prefer. I guess nobody likes it these days because it is old procedural style.
There's nothing procedural about binding return values to variables, so long as you aren't mutating them. Every functional language lets you do that. That's `let ... in` in Haskell.
Functions can have complexity or side effects, but not both.
Reminds me of this clean architecture talk with Python explains this very well: https://www.youtube.com/watch?v=DJtef410XaM
I think it's just your way of looking at things.
What if a FCF (functional core function) calls another FCF which calls another FCF? Or do we do we rule out such calls?
Object Orientation is only a skin-deep thing and it boils down to functions with call stack. The functions, in turn, boil down to a sequenced list of statements with IF and GOTO here and there. All that boils boils down to machine instructions.
So, at function level, it's all a tree of calls all the way down. Not just two layers of crust and core.
Functional core usually means pure functional functions, aka the return value is know if the arguments is known, no side effects required. All the side effects is then pushed up the imperative shell.
You’ll find usually that side effect in imperative actions is usually tied to the dependencies (database, storage, ui, network connections). It can be quite easy to isolate those dependencies then.
It’s ok to have several layers of core. But usually, it’s quite easy to have the actual dependency tree with interfaces and have the implementation as leaves for each node. But the actual benefits is very easy testing and validation. Also fast feedback due to only unit tests is needed for your business logic.
this looks like a post from 2007 im shocked at the date
Yeah, it’s based on an old post: https://www.destroyallsoftware.com/screencasts/catalog/funct...
I saw Gary posted his blog link on twitter, and I really like his article. I really didn't expect it to surface up at this moment (2025), and it's referred from a google blog. :shrug:
Me too. Aren't we already doing this? This is the basic I have been taught first.
If your language supports generators, this works a lot better than making copies of the entire dataset too.
Sometimes, sure - but sometimes, passing around a fat wrapper around a DB cursor is worse, and the code would be better off paginating and materializing each page of data in memory. As usual, it depends.
You don't need your programming language to implement generators for you. You can implement them yourself.
Something like that was popular in perl world: functional core, oop external interface.
One of the core design principles at https://github.com/aperoc/toolkami
Spam
Not sure man, I specifically stated this in my README way before this post: https://github.com/aperoc/toolkami/blob/main/README.md#comma....
I mean it's not much, but the concept just resonates with me and I want to share it. Sad I can't share even simple opinion nowadays ...
I invented this pattern when I was working on a small ecommerce system (written in Scheme, yay!) in the early 2000s. It just became much easier to do all the pricing calculations, which were subject to market conditions and customer choices, if I broke it up into steps and verified each step as a side-effect-free, data-in-data-out function.
Of course by "invented" I mean that far smarter people than me probably invented it far earlier, kinda like how I "invented" intrusive linked lists in my mid-teens to manage the set of sprites for a game. The idea came from my head as the most natural solution to the problem. But it did happen well before the programming blogosphere started making the pattern popular.
I don't really like the example (and it's from Google) because, beyond the general concept, it seems like the trigger for sending emails is calling bulkSend with Date.now() instead of the user actually triggering an email when it's really expired: user.subscriptionEndDate change to < Date.now().
destory all software
that's nice, so should I get all the db users and then filter them in app?
This works right up to the point where you try to make the code to support opening transactions functional. :D
Some things are flat out imperative in nature. Open/close/acquire/release all come to mind. Yes, the RAI pattern is nice. But it seems to imply the opposite? Functional shell over an imperative core. Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
Edit: I certainly think having some sort of affordance in place to indicate if you are in different sections is nice.
> Indeed, the general idea of imperative assembly comes to mind as the ultimate "core" for most software.
That's not what functional core, imperative shell means though. It's a given that CPUs aren't functional. The advice is for people programming in languages that have expressions - ruby, in the case of the original talk. The functional paradigm mostly assumes automatic memory management.
Right, I was just using that as "at the extreme" and how it largely exists to allow you to put a functional feel on top of the imperative below it.
I'm sympathetic to the idea, as you can see it in most instruction manuals that people are likely to consume. The vast majority of which (all of them?) are imperative in nature. There is something about the "for the humans" layer being imperative. Step by step, if you will.
I don't know that it fully works, though. I do think you are well served being consistent in how you layer something. Where all code at a given layer should probably stick to the same styles. But knowing which should be the outer and which the inner? I'm not clear that we have to pick, here. Feel free to have more than two layers. :D
whispers in monads
It can be done "functionally" but doesn't necessarily have to be done in an FP paradigm to use this pattern.
There are other strategies to push resource handling to the edges of the program: pools, allocators, etc.
Right, but even in those, you typically have the more imperative operations as the lower levels, no? Especially when you have things where the life cycle of what you are starting is longer than the life cycle of the code that you use to do it?
Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
> but even in those, you typically have the more imperative operations as the lower levels
Yes, the monadic part is the functional core, and the runtime is the imperative shell.
> Consider your basic point of sale terminal. They get a payment token from your provider using the chip, but they don't resolve the transaction with your card/chip still inserted. I don't know any monad trick that would let that general flow appear in a static piece of the code?
What do you mean by Monad trick? That's precisely the kind of thing the IO monad exists for. If you need to fetch things on an API: IO. If you need to read/save things on a DB: IO. DB Transaction: IO.
I have not seen too many (any?) times where the monad trick is done in such a way that they don't combine everything in a single context wrapper and talk about the "abnormal" case where things don't complete during execution.
Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)
I'm unclear what you're suggesting here. Are you suggesting you couldn't write a POS in Haskell, say?
My idea here is that, in many domains, you will have operations that are somewhat definitionally in the imperative camp. OpenTransaction being the easy example.
Can you implement it using functional code? Yes. Just make sure you wind up with partial states. And often times you are best off explicitly not using the RAI pattern for some of these. (I have rarely seen examples where they deal with this. Creating and reconciling transactions often have to be separate pieces of code. And the reconcile code cannot, necessarily, fallback to create a transaction if they get a "not found" fault.)
The for-loop is just better.