Finding Fine-Grained Reactive Programming

Something has been happening in the front-end world this past year that has left a lot of developers confused. Just when ES6 Classes had become ubiquitous, popular libraries are abandoning them at a rapid rate. Just when things were starting to feel like they were settling down this curveball seemed to come out of nowhere.

What I’m talking about are these functional Components with things called Hooks, and Computeds. I’m talking about this recent Vue RFC. Things like observable data and explicit dependency declarations. The slew of on_____ and use_____ that are sure to cover your screens for years to come.

The truth is, this isn’t out of nowhere. Although why this suddenly exploded now is a mystery to me as much as the next person. We are talking about a programming paradigm that was gaining steam on the front-end before React showed up on the scene but then lay mostly dormant for the past 5 years. However, I’m one of those poor souls that continued to use this swearing up and down it was the superior pattern to develop front-end UI both for developer experience and performance. So who better to introduce you to your new overlords?

Why?

You cry as you throw your laptop off the table. We get pretty invested in our technology choices on the front-end. We ride our chariots and wave our banners. “Virtual DOM for life” or “HOCs ‘R US”. But with anything, where there is progression, there often is little regression too. Old becomes new again.

To put it simply, the reason this suddenly became a topic of interest is that after years of trying to solve issues with Components by creating yet more Components, something clicked somewhere that there are other compose-able patterns that could be applied. That Classes and Mixins never cleanly solved the problem and sometimes Components are just too much — their weight too heavy for their granularity. I suspect that several problems the React team was having kept coming back to these issues. But once React came forward, the same tribe that condemned the pattern in the first place, it was like the veil was lifted. And everyone was like, “What have I been waiting for?”

The key here is this approach allows for Declarative Data. Not only your views are declarative now but your state and state derivations. Instead of splitting your code over what I like to call the 5 Stages of Grief (or the artist formerly known as Lifecycle Functions) and the fun chain of state conditionals they spread across your components, you flip it on its head grouping your code by each data atom’s journey. What is amazing about this, is now that your data is together, it is abstractable. You can create behaviors and apply them across any Component you wish.

Ironically React Hooks are not a true fine-grained reactivity system which is why there are “Hook Rules” and so many caveats around closures and references that are just unnecessary to the approach in general. But I will be using them along with examples in various other libraries to bring you up to speed on the fundamentals.

The Basics: Observables & Computeds

No matter the library: MobX, Vue, Ember, KnockoutJS, React, Solid, Svelte, etc.. All fine-grained reactive systems are based around 2 primitives. They have different names in different libraries but there are always 2: a̶ ̶m̶a̶s̶t̶e̶r̶ ̶a̶n̶d̶ ̶a̶n̶ ̶a̶p̶p̶r̶e̶n̶t̶i̶c̶e̶, p̶e̶a̶n̶u̶t̶ ̶b̶u̶t̶t̶e̶r̶ ̶a̶n̶d̶ ̶j̶e̶l̶l̶y̶…. observables and computeds!

Note: You may have heard the term Observable in Functional Reactive Programming (FRP) like RxJS, CycleJS, Bacon, which refer to streams. What I’m calling Fine Grained Reactive Programming is related to (although not necessarily the same as) Synchronous Reactive Programming (SRP). These Observables can be viewed as discrete signals settling to a resting state, much like a digital circuit.

Observables

Coding with Fine-Grained Reactive Programming is a lot like working with a Spreadsheet. There are cells that contain data and other cells whose value is calculated based on those data cells. Observables are those data cells. Mechanically they are a simple primitive that contains a getter and a setter. They need to detect both when their value is accessed and when their value is set. They might look like:

// KnockoutJS
const x1 = observable(5);
console.log(x1()); // get value
x1(8); //set value
// Vue RFC
const x2 = value(5);
console.log(x2.value);
x2.value = 8;
// Solid
const [x3, setX3] = createSignal(5);
console.log(x3());
setX3(8);
// MobX
const x4 = observable({data: 5});
console.log(x4.data);
x4.data = 8;
// React (no actual getter but for comparison purposes)
const [x5, setX5] = useState(5);
console.log(x5);
setX5(8);

They can use a single function with variable arguments, use separate functions, or even use object getter/setters or ES6 proxies. The key part is to understand that any change needs to be captured by the setter, and all accesses get tracked upon the execution of the getter. When you access a value is really important for this approach as I will explain in the next section.

Computeds

If observables are the data cells in your spreadsheet, computeds are the calculation cells. They know their dependencies and can re-execute whenever their values change. Generally, there are 2 types of computeds you will find in libraries: Pure Computeds that are used to derive values and Effectful Computeds that create side effects.

Note: Side effects are when you modify values outside the scope of the calling procedure. Since it is not self-contained in the function the result of these methods are not guaranteed to yield the same output with the same input. Hence they are referred to as impure.

Here are some examples of Pure Computeds:

// KnockoutJS
const c1 = pureComputed(() => x1() * 2);
console.log(c1()); // get value
// Vue RFC
const c2 = computed(() => x2.value * 2);
console.log(c2.value);
// Solid
const c3 = createMemo(() => x3() * 2);
console.log(c3());
// MobX
const c4 = computed(() => x4.data * 2);
console.log(c4.get());
// React (no actual getter but for comparison purposes)
const c5 = useMemo(() => x5 * 2, [x5]);
console.log(c5);

And here are some examples of Effectful (impure) Computeds:

// KnockoutJS
computed(() => console.log(x1() / 10));
// Vue RFC
watch(() => console.log(x2.value / 10));
// Solid
createEffect(() => console.log(x3() / 10));
// MobX
autorun(() => console.log(x4.data / 10));
// React
useEffect(() => console.log(x5 / 10), [x5]);

Notice how Pure Computeds return accessors themselves whereas Effectful Computeds just run the code in the function. Truthfully nothing stops you from running having side effects in your Pure Computeds. But it’s important to understand these 2 generally serve different purposes.

As you can see all the libraries basically follow the same pattern. Some libraries like MobX have a larger API surface to allow for more complicated reactions out of the box. Svelte, on the other hand, hides their computed behind their compiler using $: labels. But regardless all libraries use the same primitives. Have we finally stumbled upon the Grand Unifying Theory of Front-End?

How it Works

Well, some have claimed it was magic in the past and that we should be wary of those who carry wands and wear large draping cloaks, but I think the JavaScript community has moved beyond its stage of burning witches at the stake. We can talk about this openly and sanely without being fearful about what we don’t understand. Thanks to React, Vue, Svelte, and the progression of the machination of frontend over the past few years I’d like to think the climate has changed a bit. Ok, ready:

Automatic Dependency Detection

I said it. We’re still here.

It isn’t all that magical these days. We have things called Proxies and compilers that will take our JSX, our Component Code, our ES6, and make it into something completely different. So what is Automatic Dependency Detection?

Remember how all Observables have a getter or accessor when fetching their value? Every Computed upon execution registers itself on the global scope and when an Observable is accessed it adds itself to the list of dependencies. An overly simple implementation might look like this:

let currentContext;
function observable(value) {
  const subscribers = [];
  return function() {
    // setter
    if (arguments.length) { /* update value & notify subs */ }
    // getter    
    else {
      if (currentContext) subscribers.push(currentContext);
      return value;
    }
  }
}
function computed(fn) {
  let value;
  function execute() {
    /* do some initialization/cleanup of previous run */
    const outerContext = currentContext;
    currentContext = execute;
    value = fn();
    currentContext = outerContext;
  }
  
  //initial run
  execute();
  return /* getter of value */
}

Couple things to notice here upfront. First of all, computations execute once to start with. This is necessary to establish the dependencies so that whenever the observables update the computeds execute again. When the computed executes the passed-in function it will be calling getters on the Observable creating subscriptions. When the Observable updates it will notify its subscribers and the Computed will execute again.

Secondly, the context wraps each execution so you can nest computations within computations. This gives the ability for the reactive graph to be hierarchical. Each context will get rerun based on its own dependencies. When a parent reruns all children are recreated. However, when a child or sibling re-evaluates that context will not evaluate.

The real power here though is upon re-evaluation all dependencies are cleaned up and rebuilt on each execution. This means that dependencies are dynamic. If a conditional in a computation returns early, dependencies from the other branch will not be registered. Only if the condition changes will the computation be re-evaluated. This allows for dynamic dependencies and reduces the need for unnecessary re-evaluation.

Basic Example

Consider a situation where depending on a mode in the UI you display a user’s name differently. Either you show their username or you show their full name. I will use Solid’s syntax since it clearly shows reads versus writes (and is the most similar to React Hooks):

const [showFullName, setShowFullName] = createSignal(true);
const [getUserName, setUserName] = createSignal('JSmith');
const [getFullName, setFullName] = createSignal('John Smith');
const getDisplayName = createMemo(() =>
  showFullName() ? getFullName() : getUserName()
);
createEffect(() => console.log(getDisplayName()));
// console: John Smith

This is not a particularly expensive operation one might want to make a computation over but it will help us understand how updates are propagated. Predictably upon resolution of this code John Smith will be outputted to the console. Now, let's toggle the display mode:

setShowFullName(false);
// console: JSmith

What happens in order is :

1. The ShowFullName observable’s value is updated to true.
2. It notifies its subscribing computations causing re-evaluation. In this case, DisplayName computation cleans all dependencies. Then it re-executes tracking new dependencies as it comes across them.
3. showFullName and getUserName are accessed during execution and the computation subscribes to each of them.
4. The new value is resolved and set on DisplayName which notifies its subscribing computation (the createEffect) cleaning all dependencies.
5. Effect function executes accessing getDisplayName, subscribing to the effect again, and writing JSmith to the console.

Straightforward enough. But what happens now if we update the FullName to add a middle initial:

setFullName('John R. Smith');

FullName observable value updates. It notifies its subscribers but there are none since its subscription was cleared on the last execution of DisplayName (which now shows UserName). No other code is executed. The reactive graph is smart enough to know that no further updates need to happen at this time.

If you were to toggle ShowFullName back to true you would see the new updated name or if the name was updated while it was set to true you would have seen the change immediately, but as it stands this change is recorded but not propagated. This is the power of fine-grained change detection as it only does the work (including branch path evaluation) when the dependent values are actually changed.

What about Performance?

Now is probably a good time to deal with those nagging concerns you might be having. Like anything, there are costs and tradeoffs. On the positive, updates are incredibly fast. Much faster than a naive diffing mechanism. Every node has memoized values to shortcut evaluation. For those familiar with React and the Virtual DOM it is like having your componentShouldUpdate always already written for you. This means out of the box you should not be doing unnecessary updates.

Now some of you might remember when React first came on the scene and they were showing the downsides of such reactive systems. Most of the problems in earlier versions of these libraries are that while being fine-grained can lend to incredible performance it can also create more overhead if each thing needed to re-evaluate separately, perhaps multiple times. With React you know when updates are scheduled it will occur as a single execution that will represent the stable state at that point in time. Pretty much all popular reactive systems have solved that problem at this point. From deferred execution on the next micro-task(KnockoutJS) to creating transactions(MobX), to even using SRP clock cycles (S.js, Solid), we are long past those days. Svelte even found a way to do this by using the compiler to order the dependent statements in proper execution order.

It is worth noting that there is additional overhead in setting up the reactive graph which can affect initial rendering. Recent years have seen a variety of techniques to address this like pre-compilation, but it is something to keep in mind.

Fine-Grained Rendering & Lifecycles

Fine-grained rendering is a topic that I think often gets overlooked in introductory tutorials. There is often an assumption that you would use these techniques alongside traditional DOM rendering techniques you already use in your current frameworks. This is definitely a way of doing things, reserving fine-grained reactivity for store technology. But this approach also unlocks completely different ways to schedule and manage rendering. That is where things really get interesting.

Generally speaking, there are 2 different approaches taken to rendering with fine-grained libraries. Either they feed into an existing component system where essentially the render function is wrapped in a computed, or they tie directly into the DOM binding system. The next sections will explain how these work in more detail.

Components

This is the more common approach these days. The fine-grained reactions serve as a way of triggering the existing update cycle of the component system. This means that the underlying system is still one based on top-down diffing and patching of the DOM tree on every update. This is often accomplished by using a Virtual DOM. The benefit of fine-grained here is that change management is automated and optimized with no need for shouldComponentUpdate.

MobX with React, Vue, modern Ember, and even Svelte basically use a variation of this. With the exceptions of store technology(MobX, Vuex, etc..), these systems are locally optimized and tend to have very shallow graphs. Svelte can even hide the observables and computed from the developer behind the compiler since they do not need to worry about their scope outside the life of the component. These systems can be made very lightweight since their disposal logic can be tied to the life of the Component. However, their Component boundaries tend to be much heavier since they tend to resolve values and rewrap into observables. You are not often passing observables down as props but binding their values, and then rewrapping them in new locally scoped observables in the child component.

These systems do not need many lifecycle methods. Generally, outside of the initial setup, they need onMounted and onDestroy hooks. While they can before/after update hooks more generic scheduling mechanisms with computeds are often employed instead. This use of primitives not explicitly tied to a single component's lifecycle increases composability. With computeds, scheduled timeouts, and microtasks, you can model any traditional update cycle.

This approach still takes a little getting used to because it directs you to think about changes on a finer scale. Even if the whole render/template method re-runs and diffs, everything gets cached in these computeds so it is a simple equality check to decide updates. You can view it as if it were only updating what gets changed since memoization saves you from having to worry about should and would updates.

Component-less

This is an older approach although it still exists in a couple of fringe libraries like Knockout, Solid, and Surplus. This is a more purist view of fine-grained. Each expression or combination of expressions in the view gets their own computeds. While they may have a Component-like composition the Component is essentially just a function that runs once, generates its graph and then is no longer a consideration. The benefit here is that these libraries don’t have costs related to arbitrary boundaries since observability transcends Components. They also are fine-grained enough that they do not need diffing or patching routines as found in Virtual DOM libraries. This leads to significant performance benefits.

Note: The results above are a recent snapshot of the top 12 implementations of the JS Framework Benchmark a benchmark that pits more than 100 frontend libraries against each other in a suite of tests. After you get past the first 5 entries which are just vanilla reference implementations that do direct DOM manipulation. Every library except domc and ivi use this Component-less Fine Grained approach.

Instead of lifecycle functions, they take the above a step further. They generally will have the means to schedule disposal generically with an onCleanup method. On the surface, this looks a lot like onDestroy but it isn’t tied to a Component’s lifecycle but rather the current computed’s context. Also to handle onMounted or any after update scenarios generally you’d use generic JavaScript scheduling mechanisms like setTimeout. This also takes a bit of getting used to but it can be abstracted behind hooks to give the illusion of the same API as the Component one above. This approach is more generic as it can be applied to any computation context not just ones that have DOM rendering side effects.

Comparison By Example

Let’s look at a simple timer on an interval with each approach. First classic lifecycles with React Classes:

import React from 'react';
export default class Counter extends React.Component {
  state = {
    count: 0,
    delay: 1000,
  };

  componentDidMount() {
    this.interval = setInterval(this.tick, this.state.delay);
  }
  componentDidUpdate(prevProps, prevState) {
    if (prevState.delay !== this.state.delay) {
      clearInterval(this.interval);
      this.interval = setInterval(this.tick, this.state.delay);
    }
  }
  componentWillUnmount() {
    clearInterval(this.interval);
  }
  tick = () => {
    this.setState({
      count: this.state.count + 1
    });
  }
  handleDelayChange = (e) => {
    this.setState({ delay: Number(e.target.value) });
  }

  render() {
    return (
      <div>
        <h1>{this.state.count}</h1>
        <input
          value={this.state.delay}
          onInput={this.handleDelayChange}
        />
      </div>
    );
  }
  }

Next using Vue’s new RFC to do it with Fine-Grained Components:

<template>
  <div>
    <h1>{{count}}</h1>
    <input v-model="delay">
  </div>
</template>

<script>
import { value, watch } from 'vue'

export default {
  setup(props) {
    const count = value(0);
    const delay = value(1000);
    
    watch(() => delay.value, (delay, prevDelay, onCleanup) => {
      const interval = setInterval(() =>
        count.value++
      , delay);
      onCleanup(() => clearInterval(interval));
    });

    return { count, delay }
  }
}
</script>

Finally, an example using Solid with pure Fine-Grained binding:

import {createState, createEffect, onCleanup} from 'solid-js';
export default function Counter(props) {
  const [state, setState] = createState({
    count: 0, delay: 1000
  });
  createEffect(() => {
    const interval = setInterval(() =>
      setState('count', c => c + 1)
    , state.delay);
    onCleanup(() => clearInterval(interval));
  });
  return (<div>
    <h1>{state.count}</h1>
    <input
      value={state.delay}
      onInput={({ target }) => setState('delay', target.value)}
    />
  </div>);
}

As you can see other than framework quirks the fine-grained API’s generally are the same even though the render engines work completely differently. There are more differences as you dig deeper. It’s pretty clear how each computation or observable data atom can manage itself in this approach with abstractable declarations rather than spread across a marathon of lifecycle methods or split config options.

Conclusion

Hopefully you have a better understanding now of how Fine-Grained Reactive Programming has been changing the landscape of Frontend Development in 2019. We’ve covered the fundamentals but there is still a lot to learn. Check out the links at the end of the article for some of the libraries mentioned here.

More importantly, perhaps you now have a glimpse of where things might be heading. I think announcements like the recent Vue RFC for a Function API caught a lot of people off guard. It’s almost a no brainer if you understand Vue’s internals and what it solves for the library. But a community built off the inertia of not being React is going to be slow to accept a React-like API, even though Vue has a better claim to that fine-grained heritage.

In all honesty, this might be just the latest in a long series of trends. Or perhaps we just have finally caught up to what Steve Sanderson, creator of KnockoutJS, knew back in 2010. Maybe this looks horrible to you, in the same way as when I first saw React lifecycle functions I had a terrifying flashback to the dark ages of the web and ASP.NET webforms. But one thing is for certain: it is happening right now and never have JavaScript Libraries looked so similar.

To learn more, check out the following:

• Knockout
• MobX
• Solid
• Svelte
• Vue