Inside ag-Grid: techniques to make the fastest JavaScript datagrid in the world

I remember in the beginning of my programming career I read a great book “Code Complete”by Steve McConnell. There I found a very insightful idea:

“Managing complexity is the most important technical topic in software development. In my view, it’s so important that Software’s Primary Technical Imperative has to be managing complexity.

We can’t agree more with this statement at ag-Grid. In just 4 years we built the best JavaScript datagrid in the world. We managed to do that because we first implemented a technical foundation that helped us tackle the complexity of building such a complex widget. We call this foundation “ag-stack”. This is what I want to explore in this series of articles.

The main idea is to review patterns and techniques that we use under the hood to build the fastest datagrid while also providing ample customization opportunities. Since ag-Grid is open source, you can follow along by exploring source code on github when I link to it.

Complexity of datagrids

Datagrids are incredibly complex widgets which might not be immediately obvious. The problem stems from the amount of features that a datagrid should support and the correct interoperability between them all. ag-Grid started with the simple set of features like filtering, sorting and selection. But then we quickly realized that users needed editing, pinning, dragging, grouping and a ton of other functionality that we’ve added over the years.

Today, ag-Grid has more than 100 different features including a set of features like pivoting required for sophisticated analysis. Having all this functionality is a must have for grids designed to be part of dynamic analytical systems. As we add new features, we need to ensure that all existing features continue working properly and the interoperability between them, e.g. sorting with filtering, isn’t broken. With each new feature the complexity grows.

Another challenge is the customization of the grid. Customers want to have a way to add their own logic and UI into many parts of the datagrid. ag-Grid is highly customizable and support most use cases of enterprise users. We even allow customization using components from your favorite frontend framework (Angular, React and so on). But to make it possible, we needed to implement a lot of bridges to connect grid internals and customization layers. It’s quite a big chunk of infrastructure code that needs to be checked every time we make changes to the architecture.

The complexity could be one of the reasons why our founder Niall, who was working in the banks in London before starting ag-Grid, couldn't find a JavaScript datagrid powerful enough to be used in financial applications. Moreover, even a simple grid that could handle large amounts of data with support for grouping and pinning was difficult to come by back then. Hence ag-Grid was born.

Managing complexity in ag-Grid

TypeScript is an essential part of our tool set. The team of ag-Grid mostly consists of developers with more than 10 years of experience using OOP languages like C++ and Java. So it’s no wonder that “Old School” Object Oriented Design is part of the philosophy behind the grid’s internal architecture. We’re not strangers to functional programming either. We use a lot of low-level functional programming inside classes, however we still prefer OOP design for the higher level architecture like defining modules and setting up their interaction.

TypeScript significantly helps with OOP programming. We’ve been relying on this language almost since it’s inception. We like it because it documents interfaces. Since we’re a multi developer team, TypeScript enables easier collaboration. We also believe it prevents more bugs than pure JS. Overall, we feel like it frees up brain energy.

And now to ag-stack. This concept describes a few pillars of grid’s internal architecture, particularly IoC Container and Component Framework, alongside some optimizations techniques that we use. In the following articles in the series I’ll cover in detail the container and the framework that we built internally.

It’s worth pointing out that we implemented all parts of ag-stack ourselves from scratch. That’s our philosophy, if you need it, build it. That’s why ag-Grid has zero dependencies which is a huge advantage for consumers of our datagrid.

In this article I want to explore a number of interesting techniques that we use to improve performance of our JavaScript datagrid.

Row / Column virtualization

Virtualization is a clever technique that allows to load and display large amounts of data without performance degradation. You can think of it as an alternative way of paging in which a datagrid gradually renders DOM nodes while a user is scrolling vertically or horizontally. Here, for example, you can see a demo of ag-Grid that can be configured with a dataset of up to 100 000 records. It’s a lot, but play with the grid and notice how responsive it remains while scrolling. No jittering. This is virtualization in effect.

Every frontend developer knows that DOM operations are expensive. So virtualization is achieved by only rendering rows (row virtualization) or columns (column virtualization) that are visible in content viewport. This is usually a small portion of the entire dataset. And it means that you can feed as much data as you want into the grid. The amount of data is only limited by the browser’s VM heap. If you’re using ag-Grid even that is not a problem. You can overcome this limitation by using server-side row model. To improve performance even more, we cache rendered nodes and reuse them when they are needed for next time.

Animations using CSS transforms

At ag-Grid we think a lot about user experience while interacting with our datagrid. To enhance it we’ve implemented a lot of animations to provide a pleasant visual feedback. One of the most frequently occurring animations in the datagrid are row animations. For example, ag-Grid animates rows after they are sorted or filtered or when a row group is opened (as rows are moved down to expand the group):

This particular use case for animations required special attention from the team and involved a lot of testing and experimentation. In the end, we ended up using CSS transforms to run animations. This approach makes use of GPU and layering to render parts of the page involved in animations and make them as smooth as possible.

In CSS we’re simply using transform property instead of the left and top properties to animate the position of rows. You can see it in CSS stylesheet defined for rows:

This approach is particularly helpful with either very large datasets or when using ag-Grid on lower powered devices such as tablets. One caveat though about using too much CSS transforms is that it’s possible to run out of layers with CSS translate. To avoid that problem, at ag-Grid we apply transforms to entire rows, not cells.

Dirty checking during change detection

The last interesting bit is change detection. Datagrid needs to constantly update values in the DOM. The updates are triggered by changes in a dataset or through direct user input into cells. When this change happens, we need to update state of the component and render the updated value in the DOM. This process is pretty straightforward.

During change detection the grid calls refreshCell method for each cell. This is where DOM updates happen synchronously. There’s no asynchronous scheduling mechanism like, for example, in React fiber. While running change detection, there’s no point in updating the DOM if the value of the cell hasn’t changed. So before we do the update, we first run dirty checking — each cell stores current value and compares it to the new value it gets. Only if changes are detected or update is forced the DOM is updated. As you can imagine, this approach significantly reduces time required to process changes.

So, that’s it. I hope that this information has been useful to you. In the following articles in the series I’ll cover in detail the IoC container and the framework that we built internally. You’ll learn about the need for Dependency Injection in web applications and the benefits having our component framework brings. I plan to have the articles ready next week so stay tuned!