Getting about (this post)

• This post is big! Skip whatever bores… Follow the nice ToC!

• It is an "I to I" explanation I wish I had long ago. I've referenced getting started material and "batteries included" Clojure web stacks toward the end, which may be the most practically useful section of this post.

  • Update (2024-09-04): I just published a complete project that follows up the concepts set up in this post. adityaathalye/usermanager-first-principles is a stripped-down variant of seancorfield/usermanager-example. The project README explains all.

• Errors and inaccuracies are all mine ¹. If you spot any, please write to complaints @ my domain. (And if you want to say nice things, write to compliments @ my domain :). Feel free to discuss on HackerNews (here) and r/Clojure (here).

• A basic grounding in the Clojure programming language is assumed ². Familiarity with web development will help.

• I will stick to discussing the Clojure web application stack in relation to classical Web Frameworks. Primarily ³.

• Yes, a Meditation Two is in draft hell. It is about Getting Pretty Deep In The Woods. If it sees the light of day, it will also be a giant post. Lambda help us.

Getting Parable-ic

Multitudes of sworn "Rails developer"s, "Laravel developer"s, "Django developer"s, "Next.js developer"s and suchlike throng the universe… Why?

Novices don't know they don't know.
The Framework saves them from themselves.
So they survive.

Intermediates know enough but would rather not have to.
But it is a living. A person must eat.
So that's that. RTFM.

Experts know enough to care deeply ⁴.
The Framework is muscle memory; all the hacks,
the tricks, the deep dark secrets.
It bends to will. Mostly.

Masters know enough to not roll their own.
And yet… sometimes they do.
A new cycle begins.

Once upon a time, there was one.
WebObjects.
Now they are numberless.

Getting the Big Picture

Picture the Clojure web stack this way… to a good first approximation:

• Our business logic (written in Clojure),
  • relies on a bunch of Clojure libraries (frequently, ring-clojure),
    • that know how to use an application server (Jetty).
• In the simplest deployment model, this picture fits within a single compute instance (e.g. a PC, Cloud VM, or "Serverless" container).

|                      |
|  The bulk of our     | Our Business Domain's
|  application logic,  | data representations
|  written in Clojure. | {} [] #{} '() 'x 42
|                      |
+- - - - - - - - - - - + -- RING SPEC -- -- -- -- -- --
|                      |    ^       |
|  The subset of Ring  |    |       |
|  libraries we use    |   { } REQUEST hash-maps
|  as-provided, and    |    |       |
|  as utilities to     |    |       |
|  make custom handlers|    |       |
|  and middleware in   |    |      { } RESPONSE hash-maps
|  Clojure.            |    |       |
|                      |    |       v
+----------------------+ -- RING SPEC -- -- -- -- -- --
|    CLOJURE MAPS      |  Clojure facing interfaces
|                      |  (functions, hash-maps)
+- ring.adapter.jetty -+- - - - - - - - - - - - - - - -
|                      |  Java facing interfaces
|    JAVA OBJECTS      |  (Servlet API, Jetty config API)
+----------------------+ ------------------------------
|       Jetty          |  (deserialize ^)
| (Application server, |  HttpServlet Objects
|  "embedded" mode.)   |  (serialize   v)
+----------------------+ ------------------------------
                          (Plaintext HTTP Responses v)
     NETWORK BOUNDARY facing the WWW side
                          (Plaintext HTTP Requests ^)
+----------------------+ ------------------------------
|  Public Web Server   |  (deserialize ^)
| (for SSL termination |  HTTP Objects
|  static assets etc.) |  (encrypt, serialize v)
+----------------------+ ------------------------------
                          (HTTPS Responses v)
   NETWORK BOUNDARY with the Public WWW
                          (HTTPS Requests ^)

But before getting too practical, an indulgent philosophical interlude.

Getting Philosophical

I think frameworks are a form of industrial automation (of choices, behaviour, workflows, detail and so forth). Perceived this way, they embody the tradeoffs of industrial automation ⁵.

Tim Ewald makes astute observations about this phenomenon, in his talk "Clojure: Programming with Hand Tools". As he remarks, pervasive use of automation has the insidious quality of changing how we view the world and how we perceive problems. To a mind invested in a framework, all web software will look irresistibly framework shaped. Squint just right, and the answer reveals itself. And they may very well be right. Until they are not.

The Clojure world does it the hard way; viz. the not-framework way.

What does a Killer App kill?

Building web applications is arguably the most well trodden path into the software industry. Naturally. Many of the most valuable companies on Earth are web applications. Well-heeled web forms dominate the world.

As the Web evolved, the Web Application Framework gained status as the "killer app" of any respectable programming language ecosystem. Framework makers work hard to serve the multi-faceted, ever-evolving demands on the Web Application. Their products contain time tested ideas; accumulated knowledge of many minds, battle scars from full contact Kumite with the Wild Wild Web. In polite society, we call these scars "design patterns".

Knowing a framework well can liberate a person ⁶ from the rabbit holes of composing software to solve for things like:

• App architecture (MVC) and code layout (project templates)
• HTTP request/response parsing and handling
• Routing and dispatch
• HTML templating and/or rendering
• API design and use (HTML / text / JSON etc.)
• Form handling
• Data serialisation / deserialisation
• Sessions
• Persistent connections (websockets, long polling)
• Database connector / driver (e.g. JDBC)
• Database ORM
• Sending emails
• Managing job queues
• Configuration (via. environment variables, files, remote sources)
• App runtime lifecycle (dependency injection, starting/stopping etc.)
• Security (encryption, data sanitisation etc.)
• Authentication and/or Authorization
• Application logging
• Monitoring (with metrics and/or probes to monitor the live runtime)
• Building and Packaging
• Deployment (new-age frameworks)
• Boilerplate and glue code required to make all these work together.
• Developer Experience (framework-aware tools and IDEs are life savers).
• More…

That said, as with all things, TANSTAAFL.

What's the catch?

Tradeoffs of using a framework stem from the degree of control ceded to it and its ecosystem (ideas, plugins, packages, tools etc.). One accepts a form of vendor lock-in, in lieu of anticipated benefits. Some tradeoffs are:

• Fixed core architecture.
  Any framework's architecture is fixed; unchangeable from the outside. e.g. If you don't like the router or ORM or template engine built into your chosen framework, can you just rip them out and put in other choices (for API or performance or security reasons)? No, you would have to migrate to an entire alternate framework, and bet that this one will fulfill all your current and (unknown) future requirements.

• Leaky abstractions.
  The design choices and mental models of the framework and/or plugin authors inevitably flow into the app. It comes to rely on how they encoded explicit and implicit behaviours, software design patterns, opinions about deployment and operations etc. The more one uses, the stronger it binds.

• Upkeep.
  You own the design and upkeep of the whole composite, especially your self curated and/or bespoke parts that patch, adapt, or work around those leaky abstractions. Unavoidable framework updates are par for the course (e.g. security patches and/or access to new functionality). Even with no custom parts, app makers must carefully update all off-the-shelf plugins and tools to remain API - compatible. And then also update their own application code to be compatible with any updated third-party API.

• Production expertise.
  Debugging production issues can rather quickly become about grokking the inner workings of the framework. Meaning, sooner or later one must become a student of that specific web framework.

• Choices are an expert matter.
  Though popular language ecosystems have a canonical web framework or two, all have a plethora of alternatives. Choosing between frameworks is an expert matter. Newcomers are directed to the most canonical one for good reason. Each alternative embodies a concrete set of tradeoffs, community support, lore and so forth, all opaque to the newbie, and difficult to parse even for an expert outsider. This is perhaps why teams get built around a framework and one or two framework experts.

In Clojureland you stack libraries and the odds yourself

The culture here strongly prefers libraries over frameworks. Here is a quick overview of what we have in our web ecosystem, and the implications thereof.

The Ring world

The Ring project, by James Reeves (a.k.a. weavejester), is the Clojure ecosystem's canonical collection of HTTP libraries. Its design choices have a far-reaching effect on the whole Clojure web ecosystem. So it's worth becoming familiar with Ring.

James also created hiccup (HTML rendering) and compojure (routing), which used with ring and Clojure's standard library are enough to create a functional traditional multi-page web application, backed by the file system. To use a database, all we need is a library like next-jdbc. And making a "modern-feeling" web UI has become easy with HTMX, which "just works" with hiccup.

IMO, most web apps can start this way (and can probably stay this way).

Framework-like web stack projects

Several framework-like web stacks also grace Clojureland, viz. Fulcro, Biffweb, Kit (successor to Luminus), Duct, Pedestal etc. However, unlike object oriented frameworks that are fully integrated monolithic systems, these are open-ended sets of libraries that represent the project developer's opinion of how to build web applications. Newer projects like sitefox and donut aim to be more "fully integrated" frameworks. Single Page Application enjoyers may find hoplon cool. And if you want truly novel systems, check out hyperfiddle/electric, and Rama by Red Planet Labs.

Dependency injection for those in the know

Another approach is to use something like a dependency injection framework to connect and orchestrate all our app's moving parts through some common system. Libraries like component, integrant, mount, donut-system serve this purpose. These are favoured by people who already have specific opinions about what set of libraries and pieces of infrastructure they need (and why).

That's not all folks

We haven't even begun to enumerate a constellation of other libraries needed for databases, caches, security, logging, monitoring, queues, jobs and so forth.

Even otherwise seasoned programmers, who are new to Clojure, can struggle to find their bearings amid this dizzying array of choices.

"There is no spoon architecture"

Alas, not only is there no obvious One True Framework, there is no obvious One True Framework Architecture either. This adds to every Clojure newcomer's struggle, even grizzled web veterans. As a thought experiment, I feel a Rails developer will find it easy to make sense of a Django or Laravel project, versus any of the apps built with tools we have in the Clojure ecosystem.

Popular web frameworks, going all the way back to WebObjects (1996) are object oriented GUI software; products of convergent evolution along common industry-wide OOP patterns. They are designed for use via Public APIs. Core parts are welded together. Thus, a competent Rails developer parachuting into a Django project can reasonably expect to follow their nose down familiar-feeling Class hierarchies and method chains, across familiar Model, View, Controller structures.

Why become a student of the web stack?

The Clojure world, though built with Java for the JVM, .Net for the .Net CLR, and Javascript for node and browser engines, departs wildly from those underlying Object Oriented foundations.

This fact deeply influences everything, including making web apps. So, although freshly-minted intrepid Clojurians will do well to pick the Ring stack, or one of the popular "starter kits", we must consciously become students of web framework architecture too.

For out here, the problem of making a web application is also the meta-problem of composing a bespoke web stack.

Getting First-Principled

Many wonderful resources teach Clojure/ClojureScript web development. However, I struggled to build a coherent picture, until I worked out a first-principles model, upon which to build my understanding. So here are the bare essentials, to motivate further learning, using material I reference later.

A web app is just a polymorphic dispatcher

Think… what does a web app reeeeeally do?

HTTP request ->
  /pattern-1/ method-1
  /pattern-2/ method-2
  /pattern-3/ method-3
              -> HTTP response

Shell scripting enjoyers will immediately think of AWK programs, and their design sense would not be wrong. But there is more to the story, of course. For a "pattern" is a set of one or more pieces of information culled from HTTP requests, most crucially the URI and the HTTP verb. ⁷

HTTP request ->
  GET    /uri-1/ getter-method
  PUT    /uri-1/ putter-method
  POST   /uri-1/ poster-method
  PATCH  /uri-1/ patcher-method
  DELETE /uri-1/ deleter-method
                 -> HTTP response

This pattern tempts us to construct an HttpObject, and is arguably why modern-day OOP style appears to be a natural fit. Yes, the tiniest piece looks like an Object. And yes, the whole web app as a system is very Object Oriented. However, IMHO, the monolithic design of frameworks is rooted in having to use the smallest datum as some concrete HttpObject, instead of generic data.

Clojurists favour generic data over concrete objects and composition over inheritance, because building with composable parts gives us almost unlimited control over the shape, size, and sophistication of our application. The initial learning curve pays off over time, as we get to keep simple apps dead simple, and to ensure not-so-simple apps are only as complex as they need to be.

We build our polymorphic systems using Functional Programming parts.

With this in mind, we construct the core intuition of the anatomy of Clojure web apps, which lives in the heart of ring-clojure…

Ring with Jetty is the classic combo

Refer back to the Big Picture.

The Ring project is a crowd favourite for production Clojure ⁸ web apps. It established the Ring specification along with the request / response handling model that many other Clojure web libraries support, or complement.

Jetty is a popular server of choice in the Clojure community. It is generally used as an "Application Server" in "embedded" mode, i.e. we put the server inside our web application, as a regular library dependency. We can alternately invert the model and run Jetty in "standalone" mode, as a "container" runtime, i.e. we put our application inside the Jetty server.

We will briefly peek at the Servlet API in a later section, as that is the common base for both modes of operation. But this post assumes we run our app in the community-preferred way. By and large, Clojurians prefer the embedded jetty way over the servlet container way ⁹.

Bare minimum ring.adapter.jetty web app

Now we make a bare-minimum web app where the handler function is a catch-all method. It will return a string containing the request information for any HTTP request made to any route. This seemingly pointless code is actually useful to check that your project is set up right. Use it as a starter template.

Bare minimum directory structure

$ tree . # root of our project directory
.
├── deps.edn  # project configuration
├── classes   # target for compiled code
└── src
    └── first_principles
        └── core.clj # our bare minimum app

Bare minimum library dependencies

Our deps.edn file contains this configuration; only Clojure and the Jetty adapter library from the Ring project. We use the Jetty adapter as-provided, to avoid rewriting a whole bunch of code to do Java interop and implement the Ring specification. We rely on these as standards, so we can assume they are available as a given.

{:paths ["src" "classes"]
 :deps
 {org.clojure/clojure {:mvn/version "1.11.3"}
  ring/ring-jetty-adapter {:mvn/version "1.12.1"}}}

Bare minimum code

The lone -main function in src/first_principles/core.clj is the entry point of our web application. It contains a catch-all "handler" function that takes any incoming request and "echoes" it back as a string in the response. We compile and run this little web app as a Java process.

(ns first-principles.core
  (:require [ring.adapter.jetty :as adapter])
  (:gen-class))

(defn -main
  [& args]
  (adapter/run-jetty ; [1.] Jetty adapter's public API

   (fn [request]     ; [2.] Handler function (required).
     {:status 200
      :headers {"Content-Type" "text/plain;charset=UTF-8"}
      :body (str "echo request: " request)})

   {:port 3000       ; [3.] Jetty server config. (optional)
    :join? false}))

In the code above:

1. The Jetty adapter's public API expects:
   • A Ring-compliant handler function.
   • An optional configuration map.
2. Our bare minimum handler function.
   • The handler expects Ring-compliant request hash-maps, which the Ring Jetty adapter crafts for us.
   • And it must return Ring-compliant response hash-maps, for consumption by the adapter.
3. Optional Jetty server configuration.
   • Again, just a Clojure hash-map, also specified by the Ring spec.
   • Our Jetty adapter translates and applies any configuration we pass, to Jetty via its Java configuration API.

Bare minimum live application

Here is how we can run it from the terminal using Clojure CLI.

Compile and run as a Java process.

$ clj # in the root directory of our project
Clojure 1.11.3
user=> (compile 'first-principles.core)
first-principles.core

# Ctrl-d to exit the REPL, then run the compiled code

$ java --class-path $(clj -Spath) first_principles.core

Or directly from the REPL session.

$ clj # in the root directory of our project
user=> (compile 'first-principles.core)
first-principles.core

user=> (first-principles.core/-main) ; start the server
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
#object[org.eclipse.jetty.server.Server 0x6331250e "Server@6331250e{STARTED}[11.0.20,sto=0]"]

user=> ; Ignore the SLF4J messages. Ctrl-d to exit, when done.

Bare minimum HTTP requests

Our bare minimum live app responds to any HTTP request. Observe the request maps echoed back, for what changes, and what doesn't.

• Try curl http://localhost:3000 , the bare minimum GET request.
• Try other URI paths, with and without query params.
• Try any of those combinations with other HTTP verbs e.g. curl -XPOST http://localhost:3000 (or -XDELETE or -XPUT or -XPATCH).

Here is a sample result of a GET request to some made-up path with some arbitrary query parameters.

$ curl -XGET \
       "http://localhost:3000/foo/bar/baz?search=wassup%20world"

echo request: {:ssl-client-cert nil,
:protocol "HTTP/1.1",
:remote-addr "127.0.0.1",
:headers {"accept" "*/*",
          "user-agent" "curl/7.81.0",
          "host" "localhost:3000"},
:server-port 3000,
:content-length nil,
:content-type nil,
:character-encoding nil,
:uri "/foo/bar/baz",
:server-name "localhost",
:query-string "search=wassup%20world",
:body #object[org.eclipse.jetty.server.HttpInput 0x2a91914a "HttpInput@714182986 cs=HttpChannelState@2eae00c0{s=HANDLING rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0} cp=org.eclipse.jetty.server.BlockingContentProducer@6bac9b71 eof=false"],
:scheme :http,
:request-method :get}

Though small, our "barebones" app is still doing a lot of stuff. To figure out what's going on, let's deconstruct it further.

Bare minimum Ring project derived from first principles

Hint: It's functions all the way down.

Continuing with reference to the Big Picture, I feel like a minimal web application stack must, at the very least, facilitate the following:

• Interface with the outside world, relative to our application.
• Interface with us, in the language / domain of said app.
• Provide creature comforts to automate the drudgery of interpreting HTTP requests and creating HTTP responses.
• Provide some mechanism to orchestrate and control handler execution. It turns out that the mechanism of handlers alone is not enough to cater to all our request/response needs. We use another mechanism called "middleware".

Interface with the outside world

ring-jetty-adapter is our interface (ref: Big Picture). It is a Clojure wrapper over Jetty's Java APIs.

For us "outside" is the land of Java objects, viz. Jetty's HTTP object model, Servlet interface, and server configuration interface. These bits of the library's "outside-facing" code illustrate how it "adapts" between Jetty and Clojure:

• A Ring request can have a lot of stuff in it. For example, here is the function that moves HTTP request information from the Jetty server's request object into the corresponding Clojure hash-map that conforms to the Ring specification. Compare this with the response of the echo handler we saw a few paragraphs earlier.

  (defn build-request-map
    "Create the request map from the HttpServletRequest object."
    [^HttpServletRequest request]
    {:server-port        (.getServerPort request)
     :server-name        (.getServerName request)
     :remote-addr        (.getRemoteAddr request)
     :uri                (.getRequestURI request)
     :query-string       (.getQueryString request)
     :scheme             (keyword (.getScheme request))
     :request-method     (keyword (.toLowerCase (.getMethod request) Locale/ENGLISH))
     :protocol           (.getProtocol request)
     :headers            (get-headers request)
     :content-type       (.getContentType request)
     :content-length     (get-content-length request)
     :character-encoding (.getCharacterEncoding request)
     :ssl-client-cert    (get-client-cert request)
     :body               (.getInputStream request)})

• A Ring response contains the HTTP status code, headers, and optional body. The adapter uses these functions to move information from a response hash-map, into the corresponding Jetty servlet response object.

  (defn update-servlet-response
    "Update the HttpServletResponse using a response map. Takes an optional
    AsyncContext."
    ([response response-map]
     (update-servlet-response response nil response-map))
    ([^HttpServletResponse response context response-map]
     (let [{:keys [status headers body]} response-map]
       (when (nil? response)
         (throw (NullPointerException. "HttpServletResponse is nil")))
       (when (nil? response-map)
         (throw (NullPointerException. "Response map is nil")))
       (when status
         (.setStatus response status))
       (set-headers response headers)
       (let [output-stream (make-output-stream response context)]
         (protocols/write-body-to-stream body response-map output-stream)))))

Thankfully, weavejester has done all the heavy lifting for us, so we only have to care about the public API of this adapter library, which is a single function, run-jetty. Take a gander at its API doc.

Interface with us

ring-jetty-adapter is, again, our interface (ref: Big Picture).

For Clojure programmers, generic Clojure data is our programming model, not custom objects. So the adapter's Clojure facing side lets us:

• Configure Jetty from our Clojure app, using plain Clojure hash-maps.
• Manipulate HTTP requests and responses from our Clojure app as plain Clojure hash-maps.
• Rely on a standard specification of requests and responses as hash-maps that mirror the HTTP standard.

The Clojure data version of Ring's request/response specification is human and machine readable (within our Clojure runtime).

• A Ring Request map has a lot of stuff in it, as seen in the build-request-map function featured above.

• The Ring response map is much simpler. A valid response is just the following hash-map.

  {:status 200 ; [1.]
   :headers {"Content-Type" "text/html;charset=UTF-8"} ; [2.]
   :body "<h1>optional</h1>"} ; [3.]

  1. :status is mandatory, and must be an Integer.
  2. :header is mandatory, and must be a map of type {String String}.
  3. :body is optional, and must be a ring.core.protocols/StreamableResponseBody.

See the Ring specification and compare it with the code in these two namespaces of the main ring project: ring.adapter.jetty and ring.util.jakarta.servlet.

Provide HTTP creature comforts

… to automate the drudgery of interpreting HTTP requests and creating HTTP responses. Illustrating this will require a bit of set up.

First, I'll copy down our barebones app code, and slightly refactor it so -main looks more like it would in a production Clojure app.

(ns first-principles.core
  (:require [ring.adapter.jetty :as adapter])
  (:gen-class))

(defn echo-handler [request]
  {:status 200
   :headers {"Content-Type" "text/plain;charset=UTF-8"}
   :body (str "echo request: " request)})

(defn -main
  [& args]
  (adapter/run-jetty ; [1.] Jetty adapter's public API
   echo-handler      ; [2.] Handler function (required).
   {:port 3000       ; [3.] Jetty server config. (optional)
    :join? false}))

Now, I'll modify echo-handler to a generic handler that:

• Redirects the root URI "/" to a new "/echo" URI.
• Echos responses only for the echo URI.
• Returns a "pong" for a "/health-check" URI.
• Returns 404 for all other routes.

(defn generic-handler
  [request]
  (case (:uri request)
    "/" ; Try: curl -L localhost:3000 # -L means follow redirects
    {:status 303 ; "See Other"
     :headers {"Location" "/echo"}
     :body ""}

    "/echo"
    {:status 200
     :headers {"Content-Type" "text/plain;charset=UTF-8"}
     :body (str "echo request: " request)}`

    "/health-check"
    {:status 200
     :headers {"Content-Type" "text/plain;charset=UTF-8"}
     :body "Pong"}

    ;; Default case is not found.
    {:status 404 ; "Not Found"
     :headers {}
     :body "Not Found."}))

(defn -main
    [& args]
    (adapter/run-jetty ; [1.] Jetty adapter's public API
     generic-handler   ; [2.] Handler function (required).
     {:port 3000       ; [3.] Jetty server config. (optional)
      :join? false}))

Now, it's totally fine to hand-write maps the way I just did, but well-chosen utility functions will help us reduce some repetition and do things like:

• Easily construct Ring spec-compliant requests/responses.
• Provide semantics of common HTTP actions, so we don't have to remember them.
• Provide sane defaults when constructing responses.
• Check constraints when it matters, etc.

So, finally, lets refactor handler and pull out utility functions. This is just the seed of intuition for a whole set of utilities designed around the ring spec. See the official API docs for ring.util.response functions, for example. Also check out its sibling ring.util.* namespaces. These utilities address i/o, requests, mime-type, parsing etc.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Ring-compliant HTTP utilities
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defn status
  "Set or override status of response."
  [response status-code]
  (assoc response :status status-code))

(defn header
  [response header-name header-value]
  (assoc-in response [:headers header-name] header-value))

(defn content-type
  [response content-type]
  (header response "Content-Type" content-type))

(defn response
  "Skeleton response with status 200 OK."
  [body]
  {:status 200
   :headers {}
   :body body})

(defn not-found
  [body]
  {:status 404
   :headers {}
   :body body})

(defn see-other
  [uri]
  {:status 303
   :headers {"Location" uri}
   :body ""})

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Handler(s)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(defn generic-handler
  [request]
  ;; It is trivial to look up information in hash-maps.
  (case (:uri request)
    "/"
    (see-other "/echo")

    "/echo"
    (-> (response (str "echo request: " request))
        (content-type "text/plain;charset=UTF-8"))`

    "/health-check"
    (-> (response "Pong")
        (content-type "text/plain;charset=UTF-8"))

    ;; Default case is not found.
    (not-found "Not Found.")))

In production code, we either directly use Ring-provided utility functions, and/or use utility libraries by other people, and/or design our own high-level utilities using any of those as building blocks.

Ring request/response utilities and our handlers are not enough, because we also need to…

Orchestrate and control handler execution using middleware

In OOP terms, Ring middleware are a functional mechanism to use handlers as visitors. They also can perform dependency injection. Since middleware accept handlers and return handlers, the Ring middleware pattern is akin to the factory pattern of OOP.

This section sets up the intuition for actual Ring middleware (see official ring.middleware.* API docs). These are also "just functions", like the one below. We will see how and why such middleware work, but first a bit on why we need this mechanism at all.

(defn wrap-request-barebones-middleware
  [handler]
  (fn [request]
    (handler request)))

Remember the notion that a web app is just a ploymorphic dispatcher?

Real-world Ring handlers are specialised functions, unlike our barebones handler, which is a monolithic function tasked with handling all requests and responses. The single responsibility principle directs us to (typically) dedicate one handler function per dispatch case, yielding us isolated, maintainable, testable, composable parts.

This means handlers ought not handle shared or cross-cutting concerns, and ought not be aware of each other either. This means they cannot cooperate to manage their own control flow. Shared dependencies and/or behaviours need to sit elsewhere and be passed into or interleaved with handlers. These interventions happen anywhere "in the middle" of request/response processing. Thus we call such logic "middleware", in the Ring world.

Some scenarios that handlers do not address, but middleware can:

• How to handle state generically across handlers? e.g.
  • Inject per-request state into request context, like a database connection or a resource path.
  • Maintain state across requests, e.g. browser sessions and cookies.
  • Handle errors thrown that no handler catches.
  • Selectively log data for some requests or responses.
• How to massage the structure and formatting of request/response hash-maps for every request, so that any handler can more easily access information?
  • Parse out query or form params and make them easy to access.
  • Automatically inject header information, such as content-type with mime type guessed based on the response body.
  • Automatically coerce information to expected types.
  • Automatically decode / encode body content.
• How to exercise control over handler execution itself?
  • Gate handler evaluation based on authentication and authorization.
  • Route specific requests to specific handlers. This lets us write independently testable handler functions and compose them later using the routing mechanism.
  • Short-circuit and return early if we already have a response.
  • Do content negotiation and fix headers, to make handlers return the type of response the client expects.

We can write "request" middleware to exercise control over the behaviour of the inbound path of our app, and "response" middleware to manage behaviour of the outbound path. The simplest middleware ("no-op" or identity function of middleware) looks like this:

(defn wrap-barebones-request-handler
  "Uses a request handler as a visitor."
  [handler] ; expects a request-handling function
  ;; returns a new function that expects a request map
  (fn [request]
    ;; This in turn calls the handler function, that it has
    ;; captured in its scope, on the request that it receives.
    (handler request)))

(defn wrap-barebonse-response-handler
  "Uses a response handler as a visitor."
  [handler] ; expects a response-handling function
  ;; returns a new function that expects a response map
  (fn [response]
    ;; This in turn calls the handler function, that it has
    ;; captured in its scope, on the response that it receives.
    (if (response? response)
      (handler response))))

The category of functions that accept functions as arguments and/or return functions as responses are "higher order functions" (HoFs) ¹⁰.

Order of invocation matters. Wrapping your head around a stack of HoFs can get a little freaky. It is fine to just believe and simply follow the usual pattern of Ring projects one sees in examples and tutorials. One will find said belief handy even if, with some labour, one convinces oneself that the model works by walking down the middleware stack of function calls and returns.

Here is a step-by-step expansion of our aforementioned "barebones" middleware stack. We use the "substitution principle" of function evaluation.

1. Pass our generic-handler function to our barebones middleware stack, and call the resulting handler on the request map we expect from the ring-jetty-adapter's run-jetty interface. We are evaluating a single invocation, and so we can lift our web app's absolute minimal core functionality out of the -main entry point.

   (defn -main
     [& args]
     (adapter/run-jetty ; [1.] Jetty adapter's public API
      generic-handler   ; [2.] Handler function (required).
      {:port 3000       ; [3.] Jetty server config. (optional)
       :join? false}))
   
   ;; Can be stripped down to just run-jetty
   ;; without the optional config. map...
   
   (adapter/run-jetty
    generic-handler)
   
   ;; This creates a live server that calls our
   ;; generic-handler on every inbound request.
   (generic-handler
    inbound-request-map-from-jetty-adapter)

2. Now, what if we wrap generic-handler in our barebones middleware stack? How and why would this even work?

   ((wrap-barebones-request-handler
     (wrap-barebones-response-handler
      generic-handler))
    inbound-request-map-from-jetty-adapter)

3. We know any middleware must return a Ring compliant handler function, because we designed it that way. The cumulative result of calling more than one middleware on a handler, also reduces down to a Ring handler.

   Abstractly, our barebones middleware would evaluate this way:

   ((middleware-2 (middleware-1 handler-fn)) request-map)
   ((middleware-2 handler-fn') request-map)
   (handler-fn'' request-map)
   ;; => response-map

4. So, concretely, we can take our "barebones middleware stack" from point 2. above, and use the insight from point 3. to evaluate the stack down to a response, as follows.

   1. Substitute the name wrap-barebones-response-handler with its corresponding function body.

      ((wrap-barebones-request-handler
        ((fn [handler]
           (fn [response]
             (handler response))) generic-handler))
       inbound-request-map-from-jetty-adapter)

   2. Pass generic-handler to the just-substituted function body of the response handler and replace the whole with the resulting handler function (a middleware accepts a handler and returns a handler… quite like the Factory pattern).

      ((wrap-barebones-request-handler
        (fn [response]
          (generic-handler response)))
       inbound-request-map-from-jetty-adapter)

   3. Simplify the resulting function call because any expression of the form (fn [x] (some-func x)) is equivalent to just some-func ¹¹:

      ((wrap-barebones-request-handler
        generic-handler)
       inbound-request-map-from-jetty-adapter)

   4. Repeat the same type of function body substitution done in step 4.1, except now for the barebones request handler.

      (((fn [handler]
          (fn [request]
            (handler request))) generic-handler)
       inbound-request-map-from-jetty-adapter)

   5. Repeat the simplification operation of steps 4.2 and 4.3.

      ((fn [request]
         (generic-handler request))
       inbound-request-map-from-jetty-adapter)

      Simplifies to…

      (generic-handler
       inbound-request-map-from-jetty-adapter)

      Which ought to return us a response, because that is how we have constructed generic-handler.

5. This chain makes sense because both our "barebones" middleware apply their incoming handler argument as-is.

   Even so, reading and understanding code with real-world middleware requires belief that it does :-)

   (defn -main
     [& args]
     (-> generic-handler
         wrap-keyword-params
         wrap-params
         wrap-multipart-params
         wrap-cookies
         wrap-session
         wrap-resource)
     {:port 3000
      :join? false})

There is one final core-to-web-apps functionality that the Ring project does not include, which is polymorphic dispatch mechanism, which is "Routing". Routers like compojure, reitit, bidi, pedestal exist, but we still need a first-principles understanding of Routing. So, we round things up with…

Bare minimum router - the polymorphic dispatcher appears

The Ring project is focused on being a very good HTTP abstraction and toolkit, based on the Ring specification. It does not offer a router, even though it is a critical piece of the web stack ¹². Many routing libraries exist, each with their own particular design choices, feature sets, and performance goals. It is just fine to use compojure in your first little web app or three. Swap it out for another library later, to explore other ways of routing.

Here is the bare minimum intuition for routing.

We pull apart our monolithic generic-handler into little handlers and wire them back together with our poor man's router using something more flexible than a plain old case expression.

First, a copy of the generic-handler for quick reference.

(defn generic-handler
  [request]
  (case (:uri request)
    "/"
    (see-other "/echo")

    "/echo"
    (-> (response (str "echo request: " request))
        (content-type "text/plain;charset=UTF-8"))

    "/health-check"
    (-> (response "Pong")
        (content-type "text/plain;charset=UTF-8"))

    ;; Default case
    (not-found "Not Found.")))

Since we want open-ended polymorphic dispatch, we can use Clojure multimethods and dispatch over a much richer pattern space. Here we combine HTTP verb and URI, for example. But our dispatch function is almost unlimited in the pattern space it can generate.

(defmulti generic-handler
  (fn [{:keys [request-method uri] :as _request}]
    [request-method uri]))

(defmethod generic-handler :default
  [_request]
  (not-found "Not Found."))

(defmethod generic-handler [:get "/health-check"]
  [_request]
  (-> (response "Pong")
      (content-type "text/plain;charset=UTF-8"))  )

(defn- echo
  "Helper function to handle Echo requests."
  [request]
  (-> (response (format "%s: %s\n"
                        (:request-method request)
                        (:uri request)))
      (content-type "text/plain;charset=UTF-8")))

(defmethod generic-handler [:get "/echo"]
  [request]
  (echo request))

(defmethod generic-handler [:post "/echo"]
  [request]
  (echo request))

(defmethod generic-handler [:put "/echo"]
  [request]
  (echo request))

(defmethod generic-handler [:delete "/echo"]
  [request]
  (echo request))

;; Let's not support PATCH for echo. It should 404.

(defn -main
  [& _args]
  (adapter/run-jetty generic-handler
                     {:port 3000
                      :join? false}))

Given a HTTP abstraction like Ring, and some robust non-icky way to do routing, we can potentially build the rest of the web app like they did in the last century… string-bash HTML and SQL for the UI and the DB. But we can do better, and use more of the web stack.

Getting Started and Tutored

what is the most common stack for building web apps in clojure at the moment? is it kit/pedestal or do most clojurians prefer to roll everything from scratch?

— asked by growthesque in the #beginners channel of the Clojurians Slack, 2024-07-11 (many suggestions in the thread)

I suggest don't "roll your own", at the outset. Make old-skool web apps with Ring + Jetty + Compojure + Hiccup + next-jdbc stack, which is fine for ordinary production use. Sprinkle in some HTMX for fancier web frontend. Rest assured that it is possible to swap out any of these later, if specific needs arise.

Speed run through small demos

It's a good idea to work through existing examples and demo apps, over a weekend or two. This should give you enough finger feel to choose one of the state-of-the-art stacks listed later in this section, or roll your own too.

• Review the docs and wiki of the main Ring project, and keep these handy. Ring is a fantastic reference. I notice weavejester has been adding example projects too, so you may like to follow that repo.

• Speed-code through Eric Normand's "Learn to build a Clojure Web App" tutorial. I feel like my post is a nice conceptual complement to his more practical post. In it you'll learn some useful real-world tricks and techniques that we use in day-to-day web development.

• Watch Nir Rubinstein live code a similar tiny demo web app at Wix Engineering Tech Talk. He walks us through his thinking, various little details of the Clojure web development workflow, as well as some comparisons with the more popular Object-oriented approaches.

Do more hands-on practice

I like to copy example apps (type them out from scratch in my own words) ¹³. Here are some good options.

• Sean Corfield's usermanager-example demo app, and its variants linked in the README.

• Learn the tricks of web development workflows used by Clojurians, by watching them code example apps using real-world workflows.

  • I quite like Andrey Fadeev's video tutorial series, Building a real-world Clojure application from SCRATCH.

  • I'm not a full-stack developer and don't generally need ClojureScript, but I've enjoyed the video series by Kelvin Mai: Full Stack Clojure Contact Book and by Daniel Amber: look up the "full stack" videos in his assorted collection of Clojure videos.

  • Parens of the Dead is a terrific screencast series of zombie themed games written with Clojure / ClojureScript. Watch two expert Clojure programmers teach newcomers how to build everything from scratch, with clear explanations, run-time foibles, and some friendly banter. As of this post, the series is still undead!

• Good paid material is available too (no affiliation with any).

  • Eric Normand's video courses offer firm grounding, especially with Ring libraries.
  • Courses by (and curated by) Jacek Shae cover a range of Clojure web stacks, especially full-stack Clojure/ClojureScript web dev.
  • Web development in Clojure book, which uses a "batteries included" stack called Luminus.

Review the current state of the art

• kit-clj, biffweb, and edge are "batteries included, production grade" web stacks that look pretty good to me, all with thorough documentation. These should get you from quickstart to production.

• Eric Normand opines on: What Web Framework Should I Use in Clojure?

• Another recent opinion piece; The Clojure Web Developer's Toolkit: A Framework Face-off, compares Luminus, Kit, and Pedestal.

• The Clojure-doc site features a page on Ecosystem: Web Development (thanks, Sean Corfield for reviving clojure-doc.org!).

• You may also like to check out hoplon and keep tabs on emerging full-stack-y projects: donut, sitefox, and electric-clojure.

Putting one's neck on the line

The afore-linked references use many web stack pieces not seen in this post. These pieces are specialised solutions (libraries) for problems like routing, content negotiation, security, safe templating, safe SQL etc.; each solving for its particular domain of devilish edge cases.

The whys and wherefores thereof are being meditated upon in the next post with the working title "Getting Pretty Deep In The Woods".

Or posts.

Egad.

Oh how your states go round and round,
Webmachine.
Spring, summer, rain, fall, winter, spring.

Footnotes