Surely the best thing about Macs is that they are UNIX. Yes macOS tends to be user friendly, and to hide almost all of the complexity behind glossy settings panel, but to a power user that knows how to use a terminal (or how to google) the customisation possibilities are surprisingly wide.

All of this to say that macOS is my daily driver, and as a developer I spend much of my day inside a terminal. In the last few years I created what I consider the most ergonomic terminal experience I ever had, much thanks to modern tools and commands which replace or encapsulate good 'ole programs, like cat , curl or find. My coworkers, after months of me bugging them about this stuff, found this a perfect subject on which to write a curated list! So, here they come.

Modern replacements

This tools are meant to replace existing common commands, sometimes as a drop in that requires just an alias in your dotfile.

lsd

lsd is a replacement for ls which introduces fancy colors and icons, while also being very configurable. It can be installed through almost any package manager, including Homebrew.

To set it as the "new" ls command just drop alias ls="lsd" in your terminal dotfile.

bat

bat is another really pretty tool , which is a drop-in for cat (a cat clone with wings). Bat supports syntax coloring, line numbers and git status, and can be configured via environment variables.

I've set alias cat="bat -P" in my config.fish and this is the result:

To remove line numbers just set alias ccat="bat -Pp" or something similar. Bat is available in all package managers.

zoxide and fzf

One of the most tedious things to do while working in a terminal is getting around directories and jumping back and forth between them. Zoxide comes to our rescue, and paired with fzf, the command-line fuzzy finder, it could well remove almost entirely cd from your command history.

I recommend you go and take a look at all the usages and installation steps on Github.

z could easily replace the cd command, by being smarter and almost magical in its way to determine to which directory you want to navigate to (but you will need fzf for this).

mcfly

mcfly is a nice little utility that replaces the ctrl+r behaviour of your shell with a search engine powered by a small neural network. I don't often use the ctrl+r , but when I do mcfly saves much much time and effort. Installation and configuration instructions can be found at the Github repo, I will just drop here a screenshot of its workings.

The Silver Searcher

The Silver Searcher, aka ag , is a powerful (and fast!) code searching tool that saves me much time for example when I need to edit something inside some file in the current directory. It respects by default your ignore files, and has some handy options to filter the files. Install it with the package the_silver_searcher and try it out.

All the .ts files that contain "ArticleCardComponent", executed in 30ms. Needless to say this is much much faster than any find and grep combo.

ripgrep is a very similar tool, but I like ag more.

curlie

Curlie is a really nice drop-in replacement for curl. Curl is all powerful and almighty, but curlie takes it a step further mantaining parameter compatibility with the former. It supports syntax highlighting for headers and content, and prettifies json output when running interactively. alias curl=curlie is really a no brainer.

jq

jq is an essential tool to slice, filter, map and transform json data in command pipes. To get a sense on what it can do take a look here. It is a life saver while working on apis via curl.

fx

fx is less but for json, and smart. It is super useful to explore long json files or outputs, and requires zero learning time to get proficient with it. It accepts files as arguments or piped input, like less, and has mouse support.

TUI tools

These are tools which use a terminal user interface to make things faster and I definitely use them very often.

lazygit

If I ever hosted "CLI Program of The Year" award then lazygit would win. I'm not a git cli expert, I've always used GUIs because I'm lazy and I like having a graphical representation of the status of my repos. I started with SourceTree years ago, then moved to GitKraken when SourceTree started to require a full minute to commit a handful of files, then moved to Tower when I felt the need for a native app.

If you're a mere mortal like me and you're tired of hearing how powerful git is when in your daily life it's a powerful pain in your ass, lazygit might be for you. (lazygit author jesseduffield)

Then I gave a try at lazygit, and now Tower is only for complicated squashes, rebases and advanced stuff. Lazygit is a wonderful TUI git client, made for speed and convenience: to stage all file just press a, to commit them press c, type your message, and then hit enter. To push P , to pull p, space to checkout a branch. If you ever feel lost x brings up the menu, with all available commands shown together with their shortcut. To give you an idea of the convenience and the ease of use, I tried multiple times to use vim or emacs as an everyday "light" editor, but were never able to get really used to all commands, shortcuts and whatnot. With lazygit everything felt almost as second nature for me. It's strongly suggested to at least give it a try.

nnn

n³ (recursive acronym of Nnn's Not Noice!) is a nice and fast TUI file manager. It has a ton of features and the author has a strong opinion on how it should be used. It so rich of features that I felt fairly overwhelmed, but this didn't stop me from using its most easy and straightforward features, like simple navigation and quick editing. Give it a try, it's way faster and handy than the usual ls + cd flow.

broot

broot covers similar use cases to nnn , but it is different, and may better appease your personal taste, and is more a "tree replacement" Like n³ it support searching, paning, and file previews. For instance, it has a much better support for the "cd on exit" action.

micro

Micro is a simple and intuitive text editor, that relies on common conventions (eg. ctrl+s to save, ctrl+q to quit) and useful comforts (scroll and select with the mouse). It supports theming and syntax highlighting, and has some nice plugins, although the ecosystem is quite small. It is perfect to do edits on the fly, it is the first thing I install on any new system (it ships in a single static binary).

Working with curved elements in HTML is not simple, developers usually find themselves having to use elements like <canvas> or <svg> to accomplish their goals.

For example, an horizontal progress bar is very simple to make in HTML + CSS + JS, however, to make it round, the complexity is much greater.

In this article I'm going to explain how to use only <div/> elements and some clever CSS to build a cool circular progress bar.

How Does It Work?

To build a circular progress bar we'll use two important CSS properties:

• Clip: this property allows us to identify a visibility rectangle for an element whose position is absolute (es. clip: rect(1px, 10em, 3rem, 2ch);). Any children that is not positioned inside the visibility rectangle will not be rendered.
• Transform: this property is very powerful, but we will only use it to rotate some elements.

The technique I will show you breaks the circle into two halves, the one on the right (from 0% to 50%) and the one on the left (from 50% to 100%). To better understand how we can achive this goal, we'll initially analize To understand more easily how it works, initially we'll analyze only the right half, and, for the left half, we'll just need to apply the same reasoning but mirrored.

So, for the first half of the circle there are two main elements:

• a square container div, that contains...
• ...the actual full circle div.

The container height and width are the same as the circle's ones, like this:

With the clip property applied to the circle we can hide his right half, obtaining only a left half circle.

With the clip property applied to the container we can hide anything that is in the left half and only show what is in the right half. Since there is nothing to show in the right half we now have an empty square, that secretly hides half a circle.

Now, if we slightly rotate the hidden half circle with the tranform property, we will see it come out on the right side, like this:

So, if we want to show some progress, we just need to rotare the half colored circle so that it starts to show in the non-clipped right side.

This process only covers the progress of the right half (from 0% to 50%), but if we apply the same reasoning for the other half and we put some logic behind the rotation, we can achieve the illusion of a circular progress bar.

Let's build it with HTML, CSS and JavaScript

Let's put everything we said in practice with a simple and plain HTML, CSS and JavaScript application.

Feel free to follow along step by step, or, if you want to skip to the final solution, you can find all the code that will be used in this section here.

Step 1: Basic Structure

The first step we'll take is to get all the elements we need in place:

Let's start with a simple index.html that contains two main elements:

• a slider to control the percentage of progress and
• a group of <div/> elements that represent the ciruclar progress bar as explained in the above section.

index.html

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Circular progress bar</title>
    <link rel="stylesheet" type="text/css" href="./index.css" />
  </head>
  <body>
    <div class="container">
      <div class="circular">
        <div class="circle">
          <div class="bar right">
            <div class="progress"></div>
          </div>
          <div class="bar left">
            <div class="progress"></div>
          </div>
        </div>
      </div>
    </div>
    <div class="container">
      <span>0</span>
      <input id="input-range" type="range" />
      <span>100</span>
    </div>
    <script src="./index.js"></script>
  </body>
</html>

Let's better understand what the purpose of each of these <div/> elements is:

<div class="bar right">
  <div class="progress" />
</div>
<div class="bar left">
  <div class="progress" />
</div>

• The two <div class="progress" /> are the actual half-circle elements that correctly rotated will represent the right-half of the circle (from 0% to 50%) and the left-half of the circle (from 50% to 100%).
• <div class="bar right"> and <div class="bar left"> are the two containers we were talking about in the previous section. Their purpose is to use the clip css property to hide and show part of their children elements.

Now let's apply some basic css to get everything in place:

index.css

/* this css rule is used only to help us with visual debug */
* {
  border: 1px solid red;
}

body {
  display: flex;
  flex-direction: column;
  justify-content: center;
  align-items: center;
}

.container {
  width: 80%;

  display: flex;
  justify-content: center;
  align-items: center;

  padding: 20px;
}

.container input {
  width: 100%;
  margin: 0px 30px;
}

.circular {
  position: relative;
  margin: 0;
  padding: 0;

  /*
  * Diameter of the circle: 500px, 
  * we'll fix the fact that this value is hardcoded later 
  */
  height: 500px;
  width: 500px;
}

.circular .bar {
  position: absolute;
  height: 100%;
  width: 100%;
  border-radius: 50%;
}

.circle .bar .progress {
  position: absolute;
  height: 100%;
  width: 100%;
  border-radius: 50%;
}

All this css is pretty straight forward, the only things to note are:

• * { border: 1px solid red; } is used only for visual debug purpose
• in the .circular { /*...*/ } rule we hardcode the height and the width to match the diamater of the circle of the progress bar we are going to create, but this is only temporary, we'll remove it later.

All that is left is some basic JavaScript:

index.js

/**
 * Waits for the document to be ready before running the init function
 */
function docReady(fn) {
  if (document.readyState === "complete"
      || document.readyState === "interactive") {
    setTimeout(fn, 1);
  } else {
    document.addEventListener("DOMContentLoaded", fn);
  }
}
docReady(init);

function init() {
  // set up will happen here...

  makeProgressBarInteractive();
}

function makeProgressBarInteractive() {
  let inputRef = document.getElementsByTagName("input")[0];
  inputRef.addEventListener("input", updateCiruclarProgressBar);
}

function updateCiruclarProgressBar(event) {
  console.log(event.target.value);

  // fix the rotation of the circles to match the slider value...
}

Step 2: Clip

The next step is to define a diameter in pixels and a color for our circles:

const DIAMETER = 500;
const COLOR = "#ff0000";

Now the init function will get the references of all the <div/> elements and set up the background-color and the clip properties:

let inputRef = null;
let rightBar = null;
let leftBar = null;
let rightProgress = null;
let leftProgress = null;

function init() {
  rightBar = document.getElementsByClassName("bar")[0];
  leftBar = document.getElementsByClassName("bar")[1];
  rightProgress = document.getElementsByClassName("progress")[0];
  leftProgress = document.getElementsByClassName("progress")[1];

  // clip the right part of the circle
  rightBar.style.clip = `rect(0px, ${DIAMETER}px, ${DIAMETER}px, ${DIAMETER / 2}px)`;
  // clip the left part of the circle
  rightProgress.style.clip = `rect(0px, ${DIAMETER / 2}px, ${DIAMETER}px, 0px)`;

  // clip the left part of the cicle
  leftBar.style.clip = `rect(0px, ${DIAMETER / 2}px, ${DIAMETER}px, 0px)`;
  // clip the right part of the circle
  leftProgress.style.clip = `rect(0px, ${DIAMETER}px, ${DIAMETER}px, ${DIAMETER / 2}px)`;

  // set background color
  rightProgress.style.backgroundColor = `${COLOR}`;
  leftProgress.style.backgroundColor = `${COLOR}`;

  makeProgressBarInteractive();
}

Step 3: Rotate

Now the fun part! Let's rotate the rightProgress and leftProgress elements accordingly to the percentage value of the slider.

function makeProgressBarInteractive() {
  inputRef = document.getElementsByTagName("input")[0];

  inputRef.addEventListener("input", updateCiruclarProgressBar);
}

function updateCiruclarProgressBar(event) {
  // percentage value is get from the slider
  const percentage = event.target.value / 100;

  // the right side of the circle handles the progress from 0% up to 50%
  // if the progress is over 50% then the rotation will max at 180 degrees
  rightProgress.style.transform = `rotate(${percentage < 0.5 ? percentage * 2 * 180 : 180}deg)`;

  // the left side of the circle handles the progress from 50% up to 100%
  // if the progress is under 50% then the rotation will be of 0 degrees.
  leftProgress.style.transform = `rotate(${percentage > 0.5 ? percentage * 2 * 180 + 180 : 0}deg)`;
}

An important thing to note is that the leftProgress element must start to rotate only when the percentage of progress reaches 50%, on the other side, the rightProgress element must stop his rotation when the percentage of progress reaches 50%.

Step 4: Polishing the details

To have a beautiful circle all that is left is to get rid of the * { border: 1px solid red; } property:

We also need to handle with JavaScript the question of the height and width that was initially hardcoded in the css:

.circular {
  /*...*/
  /* remove -> height: 500px; */
  /* remove -> width: 500px; */
}

and replace it with:

let circularRef = null;

function init() {
  circularRef = document.getElementsByClassName("circular")[0];
  circularRef.style.height = DIAMETER + "px";
  circularRef.style.width = DIAMETER + "px";

  //...
}

At the end our JavaScript file will look like this:

const DIAMETER = 500;
const COLOR = "#ff0000";

let inputRef = null;
let rightBar = null;
let leftBar = null;
let rightProgress = null;
let leftProgress = null;
let circularRef = null;

function makeProgressBarInteractive() {
  inputRef = document.getElementsByTagName("input")[0];

  inputRef.addEventListener("input", updateCiruclarProgressBar);
  inputRef.value = 0;
}

function init() {
  circularRef = document.getElementsByClassName("circular")[0];
  circularRef.style.height = DIAMETER + "px";
  circularRef.style.width = DIAMETER + "px";

  rightBar = document.getElementsByClassName("bar")[0];
  leftBar = document.getElementsByClassName("bar")[1];
  rightProgress = document.getElementsByClassName("progress")[0];
  leftProgress = document.getElementsByClassName("progress")[1];

  rightBar.style.clip = `rect(0px, ${DIAMETER}px, ${DIAMETER}px, ${DIAMETER / 2}px)`;
  rightProgress.style.clip = `rect(0px, ${DIAMETER / 2}px, ${DIAMETER}px, 0px)`;

  leftBar.style.clip = `rect(0px, ${DIAMETER / 2}px, ${DIAMETER}px, 0px)`;
  leftProgress.style.clip = `rect(0px, ${DIAMETER}px, ${DIAMETER}px, ${DIAMETER / 2}px)`;

  rightProgress.style.backgroundColor = `${COLOR}`;
  leftProgress.style.backgroundColor = `${COLOR}`;

  makeProgressBarInteractive();
}

function updateCiruclarProgressBar(event) {
  const percentage = event.target.value / 100;
  rightProgress.style.transform = `rotate(${percentage < 0.5 ? percentage * 2 * 180 : 180}deg)`;
  leftProgress.style.transform = `rotate(${percentage > 0.5 ? percentage * 2 * 180 + 180 : 0}deg)`;
}

function docReady(fn) {
  if (document.readyState === "complete" || document.readyState === "interactive") {
    setTimeout(fn, 1);
  } else {
    document.addEventListener("DOMContentLoaded", fn);
  }
}
docReady(init);

Step 5: Content inside the circle

Now that we have a full circle that fills up based on a percentage, the next step would be to put some content inside the circle. I have decided to simply show a slightly smaller circle colored as the background (white) with the percentage written in the center.

index.html

<!-- ... -->
<div class="container">
  <div class="circular">
    <!-- ... -->
  </div>
  <div class="content">
    <span>0%</span>
  </div>
</div>
<!-- ... -->

index.css

/* ... */

.content {
  position: absolute;
  z-index: 1000;

  display: flex;
  justify-content: center;
  align-items: center;

  border-radius: 50%;
}

index.js

const BORDER_WIDTH = 20;
const BACKGROUND_COLOR = "#ffffff";

let content = null;

// ...

function init() {
  // ...

  content = document.getElementsByClassName("content")[0];
  content.style.height = DIAMETER - BORDER_WIDTH + "px";
  content.style.width = DIAMETER - BORDER_WIDTH + "px";
  content.style.backgroundColor = BACKGROUND_COLOR;

  // ...
}

function updateCiruclarProgressBar(event) {
  // ...

  content.firstElementChild.innerHTML = `${event.target.value}%`;
}

The final result looks like this:

4. Let's make it in React

Now that we have a good understangind of the mechanics behind this circular progress bar, let's wrap everything we have said so far in a React component.

We want the react component to be as customizable as possible throught props like:

• percentage of progress
• diameter
• color
• border width
• content background color
• content as a children element

Note: this component is typescript friendly.

circularProgressBar.tsx

import { MutableRefObject, useEffect, useRef } from "react";
import "./circularProgressBar.css";

interface circularProgressBarPropsInterface {
  color: string;
  diameter: number;
  percentage: number;
  borderWidth?: number;
  contentBackgroundColor?: string;
  className?: string;
  contentClassName?: string;
  children?: JSX.Element;
}

export default function CircularProgressBar({
  color,
  diameter,
  percentage,
  borderWidth,
  contentBackgroundColor,
  className,
  contentClassName,
  children,
}: circularProgressBarPropsInterface) {
  const rightBar = useRef<HTMLDivElement>(null) as MutableRefObject<HTMLDivElement>;
  const rightProgress = useRef<HTMLDivElement>(null) as MutableRefObject<HTMLDivElement>;
  const leftBar = useRef<HTMLDivElement>(null) as MutableRefObject<HTMLDivElement>;
  const leftProgress = useRef<HTMLDivElement>(null) as MutableRefObject<HTMLDivElement>;
  const content = useRef<HTMLDivElement>(null) as MutableRefObject<HTMLDivElement>;

  // setup dimensions, colors, and clip properties
  useEffect(() => {
    if (borderWidth) {
      content.current.style.height = diameter - borderWidth + "px";
      content.current.style.width = diameter - borderWidth + "px";
    }

    if (contentBackgroundColor) {
      content.current.style.backgroundColor = contentBackgroundColor;
    }

    rightBar.current.style.clip = `rect(0px, ${diameter}px, ${diameter}px, ${diameter / 2}px)`;
    rightProgress.current.style.clip = `rect(0px, ${diameter / 2}px, ${diameter}px, 0px)`;

    leftBar.current.style.clip = `rect(0px, ${diameter / 2}px, ${diameter}px, 0px)`;
    leftProgress.current.style.clip = `rect(0px, ${diameter}px, ${diameter}px, ${diameter / 2}px)`;

    rightProgress.current.style.backgroundColor = `${color}`;
    leftProgress.current.style.backgroundColor = `${color}`;
  }, [color, diameter, borderWidth, contentBackgroundColor]);

  // handles the rotation
  useEffect(() => {
    if (!percentage) {
      rightProgress.current.style.transform = `rotate(0deg)`;
      leftProgress.current.style.transform = `rotate(0deg)`;
      return;
    }

    let floatPercentage = percentage / 100;
    rightProgress.current.style.transform = `rotate(${floatPercentage < 0.5 ? floatPercentage * 2 * 180 : 180}deg)`;
    leftProgress.current.style.transform = `rotate(${floatPercentage > 0.5 ? floatPercentage * 2 * 180 + 180 : 0}deg)`;
  }, [percentage]);

  return (
    <div className={`container ${className ?? ""}`}>
      <div className='circular' style={{ height: diameter, width: diameter }}>
        <div className='circle'>
          <div className='bar right' ref={rightBar}>
            <div className='progress' ref={rightProgress} />
          </div>
          <div className='bar left' ref={leftBar}>
            <div className='progress' ref={leftProgress} />
          </div>
        </div>
      </div>
      <div className={`content ${contentClassName ?? ""}`} ref={content}>
        {children}
      </div>
    </div>
  );
}

circularProgressBar.css

.container {
  display: flex;
  justify-content: center;
  align-items: center;
  padding: 20px;
}

.circular {
  position: relative;
  margin: 0;
  padding: 0;
}

.circular .bar {
  position: absolute;
  height: 100%;
  width: 100%;
  border-radius: 50%;
}

.circle .bar .progress {
  position: absolute;
  height: 100%;
  width: 100%;
  border-radius: 50%;
}

.content {
  position: absolute;
  z-index: 1000;

  display: flex;
  justify-content: center;
  align-items: center;

  border-radius: 50%;
}

That's it. Now we have a fully customized and reusable circular progress bar that we can easily throw in any of our projects.

npm package

If you want to use this circular progress bar in your React application we have made an npm package for you.

First we need to install the package with npm:

npm install @monade/react-circular-progress-bar

or with yarn:

yarn add @monade/react-circular-progress-bar

Then we just need to import and use it like this:

import { CircularProgressBar } from "@monade/react-circular-progress-bar";

<CircularProgressBar
  diameter="{...}"
  color="{...}"
  percentage="{...}"
  borderWidth="{...}"
  contentBackgroundColor="{...}"
  className="{...}"
  contentClassName="{...}"
>
  <span>{...}</span>
</CircularProgressBar>

An use case: Hippopod's Player

My journey through the problem of circular progress bars in web development started when I was working on Hippopod, an open source project that takes an RSS feed and transform it in a static podcast websites.

During the development I found myself having to develop an audio player, whose play / pause button component had to take care of showing the progress of the track being played like this:

If you are interested in what Hippopod is, check it out at hippopod.xyz

The road to using this framework effectively can be quite a bumpy one though, or at least I know it was for me! As stakeholders started to use my first app and requirements came in to add features, my code started to get increasingly messy, and the UI got pretty sluggish, pretty fast.

Luckily, the web abounds with resources and the Angular team worked overtime to give us the tools we need to make things better. In this article, I will present you with some concepts that are sure to help you build faster apps AND improve their architecture.

Please note that all the info I mention here assumes that we are working with Angular 14.

Lazy loading Feature Modules

Here is a beginner tip, yet one that is worth remarking: using your Angular modules properly can decrease the initial loading time of your app dramatically thanks to lazy loading. If you come from a different background, you might realize soon enough that you can mostly ignore NgModules when building your application and create a plain component tree; this would not be a smart choice, though. As modules are one of the most defining and useful features of this framework, not taking the time to understand them would kill the purpose of using Angular for your project.

Let's say we are building an app that includes a couple of sections, one for managing users and one for browsing the history of orders. I know this is very simplistic, but please bear with me. The Angular way of doing this would consist of creating two Feature Modules, UsersModule and OrdersModule. A Feature Module (or Domain Module, according to the official Angular docs) is an NgModule that is organized around a feature, business domain, or user experience. By using these we can split concerns throughout our application, thus improving its architecture, and this is a very neat achievement per se.

Now, let's configure some routes in the AppRoutingModule and see how it is going to affect performance. A very old way of doing this would be the following

import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';
import { OrdersModule } from './features/orders/orders.module';
import { UsersModule } from './features/users/users.module';
import { HomeComponent } from './home/home.component';

const routes: Routes = [
  { path: '', component: HomeComponent },
  {
    path: 'users',
    loadChildren: () => UsersModule
  },
  {
    path: 'orders',
    loadChildren: () => OrdersModule
  }
];

@NgModule({
  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule]
})
export class AppRoutingModule { }

We will not settle for this solution even if it works, and here is why. As we import OrdersModule and UsersModule, Webpack is going to bundle them with AppModule, which means that when the user browses OrdersModule, it is not going to be loaded until UsersModule is also available and vice versa. Worse still, when the user browses the home page they are going to download two modules they don't need yet before they get access to the one they need, which is AppModule.

We can avoid this by using a different syntax:

import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';
import { HomeComponent } from './home/home.component';

const routes: Routes = [
  { path: '', component: HomeComponent },
  {
    path: 'users',
    loadChildren: () => import('./features/users/users.module').then(m => m.UsersModule)
  },
  {
    path: 'orders',
    loadChildren: () => import('./features/orders/orders.module').then(m => m.OrdersModule)
  }
];

@NgModule({
  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule]
})
export class AppRoutingModule { }

This way UsersModule and OrdersModule each get bundled in their own file and get lazily loaded when the user browses the proper route. Imagine the impact this could have on a real-world, large-scale application startup. On the other hand, as the user browses the application they will experiment some lag due to the relevant chunks getting downloaded. Well, we can do even better!

@NgModule({
  imports: [RouterModule.forRoot(routes, { preloadingStrategy: PreloadAllModules })],
  exports: [RouterModule]
})
export class AppRoutingModule { }

By adding { preloadingStrategy: PreloadAllModules }, we change the loading behavior to this:

1. The user browses a route
2. The browser downloads the relevant chunk along with the main one (the one which contains AppModule)
3. The UI is loaded and the user can start using the web app
4. In the background, the other chunks get downloaded
5. When the user browses a different route the relevant chunk is already loaded, making for a smooth navigation

Nice! We now know everything we need to speed up the initial rendering while separating concerns in our application at a high level. I highly recommend spending some time looking up other commonly used NgModule patterns you can use to split concerns even further, such as the usage of Shared Modules.

This does not improve rendering performance inside our Feature Modules at all though, can we do something about that?

Enter ChangeDetectionStrategy.OnPush

In most cases, the key as to why Angular apps slow down as they get bigger resides in how many times each component is rendered, that is in which cases the framework detects changes that are relevant to each component (thus re-rendering it). In fact, by default, an Angular component gets re-rendered in a variety of cases most of which are not actually useful. A simple yet instructive way to observe this would be to pick a component, preferably one that is nested in a complex hierarchy, write an onRender method that just performs a console.log, and add {{ onRender() }} to the template to have an idea of how many times it is rendered. You will have a fair amount of Why is it getting re-rendered again? moments, even while interacting with different components! We can change this by using the OnPush change detection strategy for our component:

@Component({
  selector: 'app-users-list',
  changeDetection: ChangeDetectionStrategy.OnPush,
  template: `
    <ng-container *ngIf="users">
      <app-user-card *ngFor="let user of users" [user]="user"></app-user-card>
    </ng-container>
  `
})
export class UsersListComponent implements OnInit {
  users?: UserApi[];

  constructor(private usersService: UsersService) { }

  ngOnInit(): void {
    this.usersService.getUsers().subscribe(users => this.users = users);
  }
}

This adjustment will dramatically decrease the number of renders in complex apps, but while experimenting with it you will notice that it does not display everything you expect; this would be the case if, as in the snippet above, it fetches data through a service and then it is supposed to show it. Why is that? OnPush changes the behavior for the component to make it re-render only when one of its Inputs changes. Therefore, you should only use OnPush in components that behave like pure functions, that is if they only depend on their Inputs and not on any other external factor (e.g.: an XHR call, an attribute inside a service).

@Component({
  selector: 'app-users-list',
  changeDetection: ChangeDetectionStrategy.OnPush,
  template: `
    <app-user-card *ngFor="let user of users" [user]="user"></app-user-card>
  `
})
export class UsersListComponent {
  @Input() users!: UserApi[];
}

@Component({
  selector: 'app-users',
  template: `
    <app-users-list *ngIf="(users$ | async) as users" [users]="users"></app-users-list>
  `
})
export class UsersComponent {
  users$ = this.usersService.getUsers();

  constructor(private usersService: UsersService) { }
}

Container and Presentational Components

The two snippets above introduce us to a very useful pattern for component-based applications, that is Container and Presentational Components a.k.a. Smart and Dumb Components. This pattern is especially useful when using a state manager, but it remains a godsend even when that's not the case.

Container components, such as UsersComponent, are allowed to contain side effects along with state and logic that belongs to the business domain. If you are using NgRx or another state manager of your choice, then the responsibility of a Container component is to make data of interest from the store available through selectors and to map Output events from Presentational components into action dispatches. We cannot use OnPush here! We usually keep Container components in the containers/ subfolder of our feature modules.

Presentational components, such as UsersListComponent, have a state which only comprises their Inputs and, if needed, some UI state (e.g.: is this dropdown open?); they pass on user interactions to their parent components through Outputs. Injecting services is not recommended unless they have nothing to do with business logic. This is strictly when you should use ChangeDetectionStrategy.OnPush, as there is no downside at all. Sometimes you will still need to trigger change detection for very particular reasons; in those cases, you will want to trigger it explicitly with a ChangeDetectorRef instance. We usually keep Presentational components in the components/ subfolder of our feature modules. As they are stateless from a business logic perspective, Presentational components are easily reusable and much easier to unit test than your classic, obscure, spaghetti component.

Methods and Getters in the template

If you have been using methods and getters in your bindings, you might want to reconsider that as it does slow down your app as opposed to other frameworks. As Angular can only render a component template in its entirety, all bindings are evaluated at each render, which prompts all related methods and getters to be called even if they have not changed. This can be improved, of course.

The most obvious way to work around with issue is to replace these getters and methods with additional variables which only get reassigned when they actually change. This can happen in a variety of cases, such as at the end of an HTTP request, after another variable changes, or, most common for Presentational Components, as an Input changes via the onChanges lifecycle hook. This sounds pretty tedious, is there another way?

Sure, we can use pipes.

@Pipe({
  name: 'userDescription'
})
export class UserDescriptionPipe implements PipeTransform {
  transform(user: UserApi): string {
    return `${user.firstName} ${user.lastName}`;
  }
}

This pretty much looks like using a method in the component class, but it's going to perform better because pipes are pure by default, which means that they are only going to be re-evaluated when their inputs change. Plus, it is reusable!

There is a caveat, though. Consider this example:

@Pipe({
  name: 'userDescription'
})
export class UserDescriptionPipe implements PipeTransform {
  transform(user: UserApi): string {
    return `${user.firstName} ${user.lastName} - ${user.address.zipCode}`;
  }
}

The issue is that pure pipes do not get re-rendered if a non-primitive attribute inside one of its parameters changes (in this case, that would be user.address). To account for this, we can either:

1. set pure: false in the Pipe decorator to make it react to impure changes. This would make it less efficient
2. treat the parameters as immutable objects, that is reassign the object instead of mutating it, e.g. with this.user = { ...user, address: newAddress } as opposed to this.user.address = newAddress. By doing this we can keep the pipe pure; besides, we probably want to do this anyway because of how Input change detection works.

Conclusions

Some of the techniques are pretty quick to implement, while others bring you to review your app architecture. That, along with how each one affects a different aspect of Angular app slowdowns should hopefully make it helpful to most developers who need a performance boost for their app. If you liked this article, especially the OnPush part, you might want to give NgRx a chance next. While it does not improve performance directly, it fits the architecture I described perfectly and it will make your app structure more predictable.

About Us

This article is part of the Mònade Developers Blog, written by Andrea Massioli.

setState

This example shows the most common situation where a stateful widget has to share its state with some stateless sub widgets and this is accomplished by passing props. If we want to update the state from a sub-widget we have to share also a method to update it, and we can pass it in props too. It's a common pattern that you can find in almost all declarative UI frontend frameworks.

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  // state
  int count = 0;

  // method to update state
  void increment() {
    setState(() {
      count++;
    });
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Center(
        child: Parent(
          count: count,
          increment: increment,
        ),
      ),
    );
  }
}

class Parent extends StatelessWidget {
  final int count;
  final void Function() increment;

  const Parent({Key? key, required this.count, required this.increment})
      : super(key: key);

  @override
  Widget build(BuildContext context) {
    return Child(
      count: count,
      increment: increment,
    );
  }
}

class Child extends StatelessWidget {
  final int count;
  final void Function() increment;

  const Child({Key? key, required this.count, required this.increment})
      : super(key: key);

  @override
  Widget build(BuildContext context) {
    return ElevatedButton(
      onPressed: increment,
      child: Text("$count"),
    );
  }
}

For an app state that you need to modify from many different places, you’d have to pass around a lot of callbacks, and this gets old and messy pretty quickly

We can also pass Widgets as props With widget composition, we can solve part of this problem, with this trick the child component can access the state without getting it from the parent

class MyApp extends StatefulWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  // state
  int count = 0;

  // method to update state
  void increment() {
    setState(() {
      count++;
    });
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Center(
        child: Parent(
          child: Child(
            count: count,
            increment: increment,
          ),
        ),
      ),
    );
  }
}

class Parent extends StatelessWidget {
  final Widget child;

  const Parent({Key? key, required this.child}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return child;
  }
}

class Child extends StatelessWidget {
  final int count;
  final void Function() increment;

  const Child({Key? key, required this.count, required this.increment})
      : super(key: key);

  @override
  Widget build(BuildContext context) {
    return ElevatedButton(
      onPressed: increment,
      child: Text("$count"),
    );
  }
}

Drilling props could also have some performance issues, because every time you update a state, all the chain of widgets from the stateful widget will be rebuilt

And what if we need to share some global states, maybe persistent through route changes, this can be hard to handle only by drilling props

So setState is useful for widget-specific, ephemeral states. For global and persistent states we need something more

Provider package

https://pub.dev/packages/provider

Flutter indeed has mechanisms for widgets to provide data and services to their descendants with some low-level widgets (InheritedWidget, InheritedNotifier, InheritedModel, and more)

But instead of using those low-level widgets, we can use this package called Provider. We just need to understand these 3 concepts:

• ChangeNotifier
• ChangeNotifierProvider
• Consumer / Provider.of

ChangeNotifier

ChangeNotifier is a simple mixin included in the Flutter SDK which provides change notifications to its listeners

To notify the listeners you have to call the notifyListeners() method from ChangeNotifier every time you update the state

class Counter with ChangeNotifier {
  // state
  int count = 0;

  // increment counter
  void increment() {
    count++;
    notifyListeners();
  }
}

ChangeNotifier doesn't require widgets to work so you can test it standalone

test('increment counter', () {
    final model = Counter();
    final count = model.count;
    model.addListener(() {
      expect(model.count, greaterThan(count));
    });
    model.increment();
  });

ChangeNotifierProvider

ChangeNotifierProvider is the widget that provides an instance of a ChangeNotifier to its descendants. It comes from the Provider package.

Usually, you don’t want to place ChangeNotifierProvider higher than necessary, but just above the widgets that need to access it

ChangeNotifierProvider(
  create: (context) => Counter(),
  child: const MyApp(),
),

If you want to provide more than one ChangeNotifier, you can use MultiProvider

MultiProvider(
  providers: [
    ChangeNotifierProvider(
      create: (context) => Counter(),
    ),
    ...
  ],
  child: const MyApp(),
),

Consumer

To listen to state changes we can use the Consumer widget, specifying which model you want to access in the template

Consumer<Counter>(
  builder: (context, model, child) {
    return Column(
      children: [
        ElevatedButton(
          onPressed: model.increment,
          child: Text("${model.count}"),
        ),
        // ExpensiveWidget without rebuilding every time.
        child,
      ],
    );
  },
  child: const ExpensiveWidget(),
)

• builder is the function that is called for rebuilding the returned widgets, whenever the ChangeNotifier notifies events (when you call notifyListeners() in your model)
  • child parameter, is there for optimization. If you have a large widget subtree under your Consumer that doesn’t change when the model changes, you can construct it once and get it through the builder.

It is best practice to put your Consumer widgets as deep in the tree as possible. You don’t want to rebuild large portions of the UI just because some detail somewhere changed.

Provider.of

Another way to access the state and attach widgets as listeners to ChangeNotifier is by calling Provider.of(context) inside the build function of a widget

...
Widget build(BuildContext context) {
  // access the state and register listener to ChangeNotifier similar to using Consumer as a wrapper of this widget
  final model = Provider.of<Counter>(context)
  ...
}

Sometimes you don't want to register a listener that rebuilds the current widget on every change, but you just want to access the current state, for example in callbacks

void myFunction() {
  final model = Provider.of<Counter>(context, listen: false);
  ...
}

Combine providers

ProxyProvider is a provider that combines multiple values from other providers into a new object and sends the result to Provider This new object will then be updated whenever one of the providers we depend on gets updated.

MultiProvider(
  providers: [
    ChangeNotifierProvider(create: (_) => Counter()),
    ChangeNotifierProxyProvider<Counter, SubCounter>(
      create: (context) => SubCounter(0),
      update: (context, counter, subcounter) => SubCounter(counter.count),
    ),
  ],
  child: Foo(),
);

class SubCounter with ChangeNotifier {
  SubCounter(this.start);

  final int start;
  int count = 0;

  int get total => start + count;

  void increment() {
    count++;
    notifyListeners()
  }
}

Final considerations

With Provider and ChangeNotifier you can accomplish all your needs in terms of state management, but having your models and their business logic in the same class is kind of messy in complex projects

BLoC pattern

This design pattern helps to separate presentation from business logic. Following the BLoC pattern facilitates testability and reusability

Data will be flowing from the BLOC to the UI or even from UI to BLOC in form of streams

To visualize the notion of a stream you can consider a pipe with two ends, only one allowing to insert something into it. When you insert something into the pipe it flows inside and gets out from the other end The concept is the same as rxjs Observables if you are familiar with

Implementing the pattern

https://pub.dev/packages/flutter_bloc

This package uses Provider under the hood to share the streams of your changing states inside the application

Cubit and Bloc are the classes in this package to implement these streams

Both Blocs and Cubits will ignore duplicate states so it's a good practice to create comparable states, overriding ==operator and hashValue or using equatable package

It's a good practice to emit immutable objects on streams

Bloc

Implementing the Bloc class creates 2 streams, one to emit the state from the Bloc to the UI and one to emit events from the UI to the Bloc

In the constructor, we can define how the Bloc should react to the events emitted from the UI and we can set the initial state

With the on\ method, we can listen to events coming from the UI, and in the callback, we can emit a new state using the emitter that you will receive from the parameters or you can emit errors with addError method

Inside a Bloc, we can access the current state via the state getter

From the UI we can emit new events with add method exposed by the Bloc class

We can retrieve our Bloc from the UI using Provider

Events are handled by default concurrently, but we can trace them very well with the Bloc class and we can observe the Bloc and do stuff by overriding methods like:

• onEvent
• onChange (you will receive the old state and the new one)
• onTransition (similar to onChange but it contains also the event that triggered the state change)
• onError methods

You can also provide a custom EventTransformer to change the way incoming events are processed by the Bloc. Check out package bloc_concurrency for an opinionated set of event transformers

class CounterBloc extends Bloc<CounterEvent, CounterState> {
  CounterBloc() : super(CounterState(0)){
    on<CounterIncrementEvent>(_increment, transformer: sequential());
  }

  void _increment(CounterEvent event, Emitter<CounterState> emit) async {
    emit(CounterState(state.value + 1));
    // addError(Exception('increment error!'), StackTrace.current);
  }

  @override
  void onEvent(CounterEvent event) {
    super.onEvent(event);
    print(event);
  }

  @override
  void onChange(Change<int> change) {
    super.onChange(change);
    print(change);
  }

  @override
  void onTransition(Transition<CounterEvent, CounterState> transition) {
    super.onTransition(transition);
    print(transition);
  }

  @override
  void onError(Object error, StackTrace stackTrace) {
    print('$error, $stackTrace');
    super.onError(error, stackTrace);
  }
}

@immutable
abstract class CounterEvent {}

class CounterIncrementEvent extends CounterEvent {}

@immutable
class CounterState {
  final int value;

  const CounterState(this.value);

  @override
  int get hashCode => value.hashCode;

  @override
  bool operator ==(Object other) {
    if (other is CounterState) {
      return other.value == value;
    }
    return false;
  }
}

bloc_test is a usefull package for testing Blocs https://pub.dev/packages/bloc_test

blocTest<CounterBloc, CounterState>(
  'increment counter',
  build: () => CounterBloc(),
  act: (bloc) => bloc
    ..add(CounterIncrementEvent())
    ..add(CounterIncrementEvent()),
  expect: () => <CounterState>[
    CounterState(1),
    CounterState(2),
  ],
);

Cubit

This is a subset of the Bloc class, it reduces complexity and simplifies boilerplate for managing simpler states It eliminates the event stream and uses functions instead, to update the state

We can define a new Cubit specifying the object type to be transmitted into the stream (in this case CounterState) and emitting new values with emit method or emitting errors with addError

In the constructor, it requires an initial state

Inside a Cubit, we can access the current state via the state getter

We can observe the Cubit and do stuff like in Bloc but we can override only the onChange and onError methods

Compared with Bloc we lose some traceability to reduce complexity

class CounterCubit extends Cubit<CounterState> {
  CounterCubit() : super(const CounterState(0));

  void increment() {
    emit(CounterState(state.value + 1));
    // addError(Exception('increment error!'), StackTrace.current);
  }

  @override
  void onChange(Change<CounterState> change) {
    super.onChange(change);
    print(change);
  }

  @override
  void onError(Object error, StackTrace stackTrace) {
    print('$error, $stackTrace');
    super.onError(error, stackTrace);
  }
}

@immutable
class CounterState {
  final int value;

  const CounterState(this.value);

  @override
  int get hashCode => value.hashCode;

  @override
  bool operator ==(Object other) {
    if (other is CounterState) {
      return other.value == value;
    }
    return false;
  }
}

You can test it in the same way as Bloc with the bloc_test package

blocTest<CounterCubit, CounterState>(
  'increment counter',
  build: () => CounterCubit(),
  act: (bloc) => bloc
    ..increment()
    ..increment(),
  expect: () => <CounterState>[
    CounterState(1),
    CounterState(2),
  ],
);

If you don't know if you should implement your state as Cubit or Bloc, just start with Cubit and the eventually refactor to Bloc on needs

Combine Blocs

You should not do this, sibling dependencies between two entities in the same architectural layer should be avoided because are hard to maintain, no Bloc should know about any other Bloc

But if you have to combine Blocs and get notified of state changes, we can subscribe to other Bloc dependencies in the Bloc constructor These dependencies have to be provided in the constructor's arguments

class FilteredCounterBloc extends Bloc<FilteredCounterEvent, FilteredCounterState> {
  final counterBloc CounterBloc;
  StreamSubscription counterSubscription;

  FilteredCounterBloc({@required this.counterBloc}) {
    counterSubscription = counterBloc.listen((state) {
      if (state is CounterState) {
        add(this.counterUpdated);
      }
    });
  }

  void counterUpdate(CounterState state) {
    // do something
  }
  // ...

Widgets

BlocProvider

Same as with Provider we have to provide our Blocs to be accessible in our application You don’t want to place BlocProvider higher than necessary, but just above the widgets that need to access it

BlocProvider(
  create: (context) => CounterCubit(), // or CounterBloc()
  child: MyApp(),
)

If you want to provide more than one Bloc, you can use MultiBlocProvider

MultiBlocProvider(
  providers: [
    BlocProvider(
      create: (context) => CounterCubit(),
    ),
    BlocProvider(
      create: (context) => CounterBloc(),
    ),
    ...
  ],
  child: const MyApp(),
),

RepositoryProvider is a Flutter widget that provides a repository to its children via RepositoryProvider.of(context). It is used as dependency injection (DI) widget so that a single instance of a repository can be provided to multiple widgets within a subtree. BlocProvider should be used to provide Blocs whereas RepositoryProvider should only be used for repositories. If you want to provide more than one Repository, you can use MultiRepositoryProvider

BlocBuilder, BlocListener, and other widgets to react to state changes

To listen to state changes we can use the BlocBuilder widget specifying which Bloc you want to access and the emitted data type in the template

BlocBuilder<CounterCubit, CounterState>(
  builder: (context, state) {
    return ElevatedButton(
      onPressed: context.read<CounterCubit>().increment,
      child: Text("${state.value}"),
    );
  },
)

BlocBuilder<CounterBloc, CounterState>(
  builder: (context, state) {
    return ElevatedButton(
      onPressed: () => context.read<CounterBloc>().add(CounterIncrementEvent()),
      child: Text("${state.value}"),
    );
  },
)

Use BlocListener (or MultiBlocListener) if the UI child isn't directly affected by state changing and you just need to do something in response to state changes like navigation or showing modals etc...

 BlocListener<CounterCubit, CounterState>(
  listener: (context, state) {
    if(state.value == 10) {
      Navigator.of(context).pushNamed("/someroute");
    }
  },
  child: ChildWidget(),
)

You can use BlocConsumer if you need both builder and listener

BlocConsumer<CounterCubit, CounterState>(
  listener: (context, state) {
    if(state.value == 10) {
      Navigator.of(context).pushNamed("/someroute");
    }
  },
  builder: (context, state) {
    return Text("${state.value}");
  },
)

You can use BlocSelector if you need to react only to a part of state changing

BlocSelector<CounterCubit, CounterState, bool>(
  selector: (state) {
    return state.value > 0;
  },
  builder: (context, state) {
    return Text("is greater than zero? ${state}");
  },
)

BlocProvider.of and context shortcuts

As anticipated, we can access our Blocs and Cubits from the UI using Provider

final cubit = BlocProvider.of<CounterCubit>(context);

Or more simply using context

// do not use it in the build method to access the state, use it to access the Bloc in callbacks
final cubit = context.read<CounterCubit>();

If we want the widget to be notified of state change

// use it in the build method to rebuild the widget on state changes, not in callbacks only to access the state
final cubit = context.watch<CounterCubit>();

In addition, context.select can be used to retrieve part of a state and react to changes only when the selected part changes.

// use it in build method to rebuild the widget on selected state changes, not in callbacks only to access the state
final isCounterGreaterThenZero = context.select((CounterBloc b) => b.state.value > 0);

Observe all blocs

BlocObserver can be used to observe all Blocs

class MyBlocObserver extends BlocObserver {
  @override
  void onCreate(BlocBase bloc) {
    super.onCreate(bloc);
    print('onCreate -- ${bloc.runtimeType}');
  }

  @override
  void onEvent(Bloc bloc, Object? event) {
    super.onEvent(bloc, event);
    print('onEvent -- ${bloc.runtimeType}, $event');
  }

  @override
  void onChange(BlocBase bloc, Change change) {
    super.onChange(bloc, change);
    print('onChange -- ${bloc.runtimeType}, $change');
  }

  @override
  void onTransition(Bloc bloc, Transition transition) {
    super.onTransition(bloc, transition);
    print('onTransition -- ${bloc.runtimeType}, $transition');
  }

  @override
  void onError(BlocBase bloc, Object error, StackTrace stackTrace) {
    print('onError -- ${bloc.runtimeType}, $error');
    super.onError(bloc, error, stackTrace);
  }

  @override
  void onClose(BlocBase bloc) {
    super.onClose(bloc);
    print('onClose -- ${bloc.runtimeType}');
  }
}

void main() {
  BlocOverrides.runZoned(
    () {
      // Bloc instances will use MyBlocObserver instead of the default BlocObserver.
    },
    blocObserver: MyBlocObserver(),
  );
}

Insights

Flutter dev - state management options

Bloc library documentation

Bloc implementation examples

About Us

This article is part of the Mònade Developers Blog, written by Mattia Cecchini.

While read queries can heavily impact performance, writes too can do some damage. In this article, part 2 on our Rails Performance Series, we will examine ways to speed up write queries.

Checkout the part 1 about DB reads here.

Transactions

Transactions allow us to execute a series of operations atomically, as they were a single operation. If any operation inside the transaction fails, the whole blocks is canceled, and no operation is persisted (rollback).

A nice thing to know is that a database is much faster at committing a transaction with n writes than n writes each. So generally:

1000.times do
  User.create!(email: 'hey@example.com')
end

is much slower than

User.transaction do
  1000.times do
    User.create!(email: 'hey@example.com')
  end
end

Validations

Rails validations are very useful, although they introduce an overhead on record save. Uniqueness constraints are particularly costly, as they imply a SELECT query before saving to check if the value respects the constraint.
In some cases disabling validations can significantly speed up operations, especially when we are dealing with massive operations like imports and such.
To skip validations we can call the save method as:

record.save(validate: false)

DO NOT do this unless you are sure that performance is a real problem.

Massive insertions / updates

From Rails 6 onwards some handy methods were added to do mass operations, using a single INSERT INTO or UPDATE operation.

Those methods are insert_all, insert_all!, upsert_all and upsert_all!.

The main advantage is that insertion time is orders of magnitude faster.

Warning: with those methods model validations and callbacks will not be invoked. (before_create, after_save, etc.)

If you're using Rails < 6, you can achieve the same functionality using this gem:
https://github.com/jamis/bulk_insert

Massive deletions

The same thing is valid for deletions.

If you need to prune mass data, consider that this:

User.each(&:destroy)

Is much slower than this:

User.transaction do
  User.each(&:destroy)
end

And it's much much slower than this:

User.delete_all

But, as noted before, delete_all skips all validations and before_destroy hooks, so be aware of it.

Minimize number of queries

It's pretty obvious, but a way to reduce writing performance is, as always, to reduce the number of queries.

So, try to make changes to a model all in the same place or collect changes in the model and invoke a query just once.

For instance:

• Don't make call an #update in the after_create hook. Use before_create and change the column, instead.

Don't

class User
  after_create do
    update(is_valid_user: true) if email.ends_with?('@gmail.com')
  end
end

Do

class User
  before_create do
    self.is_valid_user = true if email.ends_with?('@gmail.com')
  end
end

• Don't call #touch if you just made a save to the model. It's been already touch-ed by the save
• Don't call save multiple times on the same model during the same request.

Don't

user.update!(email: '___', password: '___')
...
...
user.update!(is_valid: true) if user.email.ends_with?('@gmail.com')
user.update!(metadata: metadata) unless metadata.nil?

Do

user.email = '___'
user.password: '___'
...
user.is_valid = true if user.email.ends_with?('@gmail.com')
user.metadata = metadata unless metadata.nil?
user.save! # Just one query
######
# Or, as an alternative:
######
user.assign_attributes(email: '___', password: '___')
...
user.is_valid = true if user.email.ends_with?('@gmail.com')
user.metadata = metadata unless metadata.nil?
user.save! # Just one query

Indexes, triggers and performance

Indexes are great for performance, in general. They speed up read queries a LOT, making a trade-off between speed and occupied memory.

However, you should consider that they also add an extra overhead during writes. In general, this is trivial.

In some scenario, however, if you have too many (unused) indexes, you may see some slowdowns during inserts.

To find it out, just run the query in the console and check the execution time:

User Create (4.3ms)  INSERT INTO "users" ("email", "created_at", "updated_at", "confirmation_token", "confirmation_sent_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id"

In general, it should be a few milliseconds. If it's much higher (like 300ms), it could be caused by too many indexes. Try to prune redundant ones.

A few hints:

• Indexes on column with a low cardinality are usually less effective and useless (example: booleans, enumerators, values often NULL)
• Compound indexes always cover single indexes on the first column. So INDEX ON (first_name, last_name) also covers searches on the first_name column on ly (but not on last_name).

The same issue can appear with triggers: don't over use them, because they can slow down a LOT your write performance.

About Us

This article is part of the Mònade Developers Blog, written by Piero Dotti.

Ruby on Rails is rarely seen as a framework oriented towards speed and performance. And that's true: Rails was born as a tool to speed up development and to make developers' life easier.

This must be contextualized, though.

Those who say this usually are talking about scaling problems. That is to say that when requests and concurrency grows Rails is easily outperformed by other more performant frameworks or languages: Rust, Elixir, and Java are good examples.

The point is that when we talk about this kind of problems we are talking about very heavy loads, thousands or tens of thousands requests per minutes.

On smaller loads Rails has no particular problems, and generally performance is more a developer water cooler argument of discussion.

We agree on the fact that a response time of 5ms vs 50ms is nicer to see, but to the final user this is rarely detectable, as we can expect 100-300ms of global network overhead.

So if you have performance problems on an application with a modest load the problems is your code, not Rails.

Don't get discouraged, this means two things:

• there is a lot of measures to take to increase performance;
• you DO NOT have to rewrite your app in Rust🚀.

Note

Rails in dev mode is significantly slower than production mode. Test in production mode before concluding that your app is slow.

Query performance

The largest part of Rails performance problems are not actual Rails problems, but rather problems in the way we use ActiveRecord and the number of queries ran for each request.

To check wether this is our case it's enough to run the slow network request and check the server logs for something like this:

Completed 200 OK in 72ms (Views: 43.6ms | ActiveRecord: 24.6ms | Allocations: 27391)

This is the recap of the request time, divided by each component. As you can see this request is fairly fast and ActiveRecord has a moderate impact on the overall request time.

On the contrary this log:

Completed 200 OK in 2800ms (Views: 80.6ms | ActiveRecord: 2719.2ms | Allocations: xxxx)

means that there is a problem: the queries take to long!.

First we have to identify the kind of queries that are slow. Read queries (SELECT) or write queries (INSERT, UPDATE, DELETE).

Read queries

As a rule of thumb: the fewer queries, the better. This is because:

• Running a query has a fixed network and database processing overhead, not depending on the query complexity. Running 10 light queries costs more than running one big query.
• Parsing a query is not free in Rails, as we have to instantiate models based on the query result
• Databases are generally faster than Rails: it's better to make the database do the heavy search&filtering work.

The n+1 queries problem

Rails allows to handle models and relations in a very simple way, but this can lead to unoptimized queries. A typical problem is the "n+1 queries problem".

This is the use case: we want to iterate on a models collection and, for each record, access one of its relationships to extract some attribute from this relationship.

Normally every time we access a relationship on a model Rails runs a SELECT to load the relationship data. This is also valid for the fetched relationships: trying to access a relationship of the previous one triggers another select, and so on.

Naturally, the more the record, the more the queries. An example:

  users = User.all
  commenters = users.flat_map do |user|
    # calling user.posts, triggers SELECT * FROM posts WHERE user_id = ?
    user.posts.flat_map do |post| 
      # post.comments triggers "SELECT * FROM comments WHERE post_id = ?"
      # .author also triggers "SELECT * FROM users WHERE user_id = ? LIMIT 1"
      post.comments.map(&:author)
    end
  end

In this examples, 100 users with 100 posts each, each post with 10 comments, means ... 100+100 * 10 * 2 = 2100 queries. ARGH

How can we solve this?

The idea is to preload with one (or few) queries the data before the for loop, to avoid running all the queries and to have all the data ready from the beginning.

Rails offers three main ways to preload, each composable with the existing query:

• The eager_load method alters the main query, adding a LEFT OUTER JOIN for each relationship to preload. It's the most efficient way counting the number of queries because only one very large query is executed, but can create conflicts with the other SQL constructs used. For example it can mangle with select, distinct and group methods and generate invalid sql.
• The preload method loads the requested relationships in a separate query, with a single query for each relationship. It's a little less efficient than eager loading, but it's safer.
• The includes method chooses based on context wether to use a LEFT OUTER JOIN or a separate query, checking for conflicts.

The previous example optimized looks like this:

 users = User.preload(posts: { comments: :author })
 commenters = users.flat_map do |user|
    # no query here
    user.posts.flat_map do |post| 
      # neither here
      post.comments.map(&:author)
    end
  end

Using preload we will run a query for the users, a query for the posts, one for authors and one for comments. A total of 4 queries, which even if large and time-consuming (for example 50ms each, which is a lot) will still be reasonably fast (200ms in total).

We are talking 50x just with a little optimization.

Warning: often the n+1 problem is hidden, because Rails iterates implicitly on the data. As an example, every time we serialize a model that has a relation, that will trigger the problem.

Pro tip: there is a little lovely gem which can, in some specific cases, solve the problem automagically: goldiloader. It's always a good idea to install it.

Manual preloading

Sometimes the Rails automatic preload does not work, but it can serve as an inspiration on how to optimize our query!

Manual preloading is most useful when we are looping over data stored in the database.

The idea is: instead of using find_by! or where methods inside a loop, we preload the data before the loop and keep a key-value map to access rapidly preloaded data.

For example: we want to update all the Posts inside our app. Posts had a text field containing the author name, but we made the author a relationship on the post, and want to migrate from a text field to a reference field.

We would usually do:

  Post.find_each do |p|
    p.user = User.find_by!(name: p.author_name) # a query for each post!
    p.save!
  end

Bad news: we have 100.000 posts, and Rails is still running after an hour. This is obviously slow.

Solution:

  # preloading all users in a name -> user map
  user_map = User.where(name: Post.select('DISTINCT author_name')).index_by(&:name)
  Post.find_each do |p|
    # users are already there, this is very fast
    p.user = user_map[p.author_name] || (raise ActiveRecord::RecordNotFound)
    p.save!
  end

This saves us all find_by! calls, 50% of the total queries is now gone!

Unwanted Queries

Some rails’ models methods trigger queries, often unwanted, unexpected or unnecessary. It’s important to know when and where this could happen!

count

Calling #count on a has_many relationship always triggers a query. Even if data has been loaded previously. For example:

user = User.first
user.posts.each { |p| ... } # this fires "SELECT * FROM posts WHERE user_id = ?"
user.posts.size # posts are cached, no query
user.posts.count # this triggers SELECT COUNT(*) FROM posts WHERE user_id = ?

tldr; never use #count, use #size.

any?, exists? and present?

These methods could potentially trigger an unwanted query if the relationship has not been already loaded. For example:

user = User.first
if user.posts.any? # SELECT 1 FROM posts WHERE user_id = ? LIMIT 1
  ...
end
user.posts.each { ...stuff... } # SELECT * FROM posts WHERE user_id = ?

Forcing the relationship loading with #load is enough to avoid the redundant SELECT.

user = User.first
if user.posts.load.any? # SELECT * FROM posts WHERE user_id = ?
  ...
end
user.posts.each { ...stuff... } # cached: no query

Further readings on this can be found here.

find_each

Have you ever been tempted to iterate on a Rails models using #each? Yeah, that’s cool, but if your database is somewhat large you will see your console explode and your dev machine’s ram disappear in a sneeze.

This is because #each loads the whole table in memory with a SELECT * FROM table, and then iterates one value at a time. Before starting to iterate it has to fit the whole table, Rails overhead included, inside the memory. So what’s the correct way to iterate on a large table?

This is where #find_each comes in. This method works exactly as #each, chunking the iteration in 1000-elements batches. It will load and iterate on 1000 elements, empty the ram, and load the next 1000.

The batch size can be controlled by setting the, surprise surprise, batch_size parameter.

Database indexes

Indexes deserve a long and through discussion, which we will tackle in another article.

Indexes help if you already optimized the queries Rails-side, but the single query still takes too long to complete. Here are some quick tips.

Put indexes on all field of tables with 1000+ records on which you expect to launch a where or find_by. For example usernames, ids, emails, vat numbers, etc.

Consider that indexes work only in these cases:

• if the query filters the dataset using equality operators;
• if comparison operators on numbers are used, with some limitations, more here
• if the query is a partial string search, only if the match in on the start of the string, eg: WHERE column LIKE 'something%', not WHERE column LIKE '%something%'. - To check wether a query uses an index you can ask your database to EXPLAIN it:

Post.where(user_id: 5).explain
# => EXPLAIN (SELECT * from posts [...]) ;

If you use PostgreSQL you are done, for almost every use case.

If you use MySQL on all tables with 10k+ records create a compound index on all the fields/columns you will search together often.

For example, if this query is run often:

User.where(first_name: 'john', last_name: 'doe')

It is a good idea to create a compound index on first_name and last_name:

  add_index :users, [:first_name, :last_name]

Note: it is nonsense to add an index on a field with a low cardinality, it could do more harm than good (for example an enum fields with a few elements or a boolean).

Query caching

We previously alluded on query caching in a previous paragraph, let’s dig deeper!

If a request is made twice rails does not repeat the query, but serves a cached version of the results. This can be leveraged in many ways: for example, it makes no sense to make two queries, one after the other, when the second is a filtered version of the first one. It is much better to call again the first query and then filter programmatically. Concretely, this is bad:

users = User.where(role: :admin) # first query
users.do_some_stuff
...
users = User.where(role: :admin).where(id: [10, 50, 100]) # this is not cached! you are wasting a query
users.do_some_stuff_again

This is good:

users = User.where(role: :admin) # first query
users.do_some_stuff
...
users = User.where(role: :admin).filter { |e| e.id.in?([10, 50, 100]) } # again the first query, which is cached
users.do_some_stuff_again

Too large columns

Some models may contain particularly heavy columns, like large JSON or TEXT fields. If these fields are used only in some views it may help to do one of the following:

1. disable the column loading

In rails a model can be configured to ignore a column, for example:

class User < ApplicationRecord
  self.ignored_columns = ['fat_json_column']
end

To load it you will need to select it manually with select or pluck:

class User < ApplicationRecord
  self.ignored_columns = ['fat_json_column']
  def fat_json_column # this is almost like a computed attr
    # but it loads it from the database
    User.where(id: id).pick(:fat_json_column)
  end
end

2. move the field

The idea is simple: move the large column in another table, with a 1-to-1 relationship with the main table, and then load the relationship when needed:

class User < ApplicationRecord
  has_one :user_data
end
# fat_json_column is here
class UserData < ApplicationRecord
  belongs_to :user
end

What’s next?

We saw how to optimize read queries. The next obvious step is to tackle read queries. See you in the next article of the series!

About Us

This article is part of the Mònade Developers Blog, written by Piero Dotti, edited and translated by Fabrizio Mele.

In this scenario we came in contact with Inversify, a nifty library we are using to fade out mixins as "place where api calls are made". Some time ago we realised that the same library could be also used to inject singletons inside vue components.

The case

While scaffolding a new view, a fairly complex one, in a company project, a very complex one, we had to get smart about the business logic of the view: long story short, the view has a big data table showing different records populated from a backend API, and various filters scattered in split panes, modals and whatnot. It seemed a clear case of moving the logic inside the store:

• the data table watches a getter on the store that returns Model[];
• the various filters update the FilterObject stored inside the store;
• the store updates its internal state calling the API with some filter parameters every time the FilterObject is updated.

This way the getter returns always a filtered Model and everything is totally decoupled, to allow designers to tinker with the UI without making us dev really unhappy moving around events, wrappers and whatnot.

The store

Having some previous knowledge about Inversify container and injections it came to us that it would be much easier to implement and use as a store a singleton service.

The idea is simple: the store is a Typescript class which extends Vue (so it is technically a Vue component, more on this later). The store is bound to the IOC container in singleton scope. The store is injected inside every component which updates the FilterObject or displays the Model[]. It looks like this:

// store.ts

import { inject, injectable } from "inversify-props";
import { IAnotherService } from "@/services/AnotherService";
import { SYMBOLS } from "@/app.container";

export interface IStore extends Vue {
  get items(): Model[];

  set items(items: Model[]);

  get filters(): FilterObject;

  set filters(filters: FilterObject);

}

@injectable() // Inversify here
export default Store extends Vue implements IStore {
  _items: Model[] = [];
  _filters: FilterObject = {};

  private anotherService!: IAnotherService;

  constructor( // Constructor injection with Inversify
    @inject(SYMBOLS.AnotherService) anotherService: IAnotherService
  ) {
    this.anotherService = anotherService;
  }

  set items(items: Model[]) {
    this._items = items;
    this.$emit('items-updated', this._items); // Vue!
  }

  get items(): Model[] {
    return this._items;
  }

  get filters() {
    return this._filters;
  }

  set filters(filters: FilterObject) {
    this._filters = filters;
    this.$emit('filters-updated', this._filters); // More Vue!

    this.updateItems();
  }

  private async updateItems() {
    this.items = await this.anotherService.findSomeModels(this.filters);
  }

  // ...
}

Glossing over missing type declarations its working are quite simple: we leverage Typescript getters and setters to react to attribute updates: every time a the filter is updated we advertise the filter update event and then call the API (hopefully with a debouncer). The API call will update the items attribute and advertise the fact that the items may have changed.

Extending Vue (i.e. declaring the store as a Vue component) allows us to leverage Vue events to advertise stuff! Event bus baby! (line 31)

Now we just need to:

1. singleton this class with more Inversify
2. subscribe to relevant events from the view components.

The container

Now we need to bind our shiny new Store to the container. Here we encountered some roadblocks, but it turns out it was a bad written guide!

In the main.ts where we declare and mount the Vue instance we must initialise the dependency container. Not that reflect-metadata is imported before everything else. Weird errors could be thrown otherwise.

// main.ts

import "reflect-metadata";
// ...
import buildDependencyContainer from "@/app.container";

// ...

buildDependencyContainer();

// ...

new Vue({}).$mount("#app");

not before building the container:

// app.container.ts

import { container } from "inversify-props";

export const SYMBOLS = {
  AnotherService: Symbol.for("IAnotherService"),
  Store: Symbol.for("IStore"),
};

export default function buildDependencyContainer(): void {
  container
    .bind<IAnotherService>(SYMBOLS.AnotherService)
    .to(AnotherService)
    .inTransientScope();
  container
    .bind<IStore>(SYMBOLS.Store)
    .to(Store)
    .inSingletonScope(); // Singleton is the keyword here
}

This functions makes a bit clearer the @inject syntax found in the Store.ts. We will anyway see more on that just now. Note that the Store is declared in singleton scope: this means that it will be instantiated just once. Everyone asking for the Store object anywhere in the single page application will get the same instance, allowing us to use it as a store!

The components

Now that we have a IOC container and some injectable stuff fired up and running we can do fancy stuff without anything more than an @inject decorator. For example, a component updating the filter could be:

// SearchBox.vue
<template>
  <div>
    <input type="text" v-model="term" />
  <div>
</template>
<script lang="ts">
import { Vue, Component, Watch } from "vue-property-decorators";
import { inject } from "inversify-props";
import { SYMBOLS } from "@/app.container";
import { IStore } from "@/services/Store";
@Component({})
export default class SearchBox extends Vue {
  @inject(SYMBOLS.Store) readonly store!: IStore;

  term = "";

  mounted() {
    this.updateTerm();
    this.store.$on('filter-updated', this.updateTerm);
  }

  updateTerm() {
    if (this.store?.filter?.term != this.term) {
      this.term = this.store?.filter?.term ?? "";
    }
  }

  @Watch("term")
  onTermUpdated() {
    this.store.filters = Object.assign(
        {},
        this.store.filters,
        { term: this.term }
     );
  }
}
</script>

Vue on and off methods can be used to hook into the store events. This SearchBox components can be dropped anywhere on a view and it will always be synchronised with the store singleton. Note that the store is injected with a decorator, it is possible to manually get it from the container by calling container.get inside mounted. I find decorators much more elegant and concise. On the other side, with some imagination we can image this component dynamically refreshing its contents as the store updates its state:

// DataDisplay.vue
<template>
  <div>
    <span
      v-for="item in items"
    >{{ item }}<
    /span>
  <div>
</template>
<script lang="ts">
import { Vue, Component } from "vue-property-decorators";
import { inject } from "inversify-props";
import { SYMBOLS } from "@/app.container";
import { IStore } from "@/services/Store";
@Component({})
export default class DataDisplay extends Vue {
  @inject(SYMBOLS.Store) readonly store!: IStore;

  items: any[] = [];

  mounted() {
    this.updateTerm();
    this.store.$on('items-updated', this.updateItems);
  }

  updateTerm(items: any[]) { // Note that being a Vue event there's a payload
    this.items = items;
  }
}
</script>

Even more quick and easy. Just subscribe to the items-updated event and the component will (not so) magically be synchronised with the search box.

Conclusions

Obviously this is a very simple example, but it should satisfy the need to se a complete example of Inversify DI in actions with prop injection in class components.

The store could contain more logic and even actions (as simple methods), even if we prefer to keep inside it the logic strictly related to items searching and filtering. Adding stuff to it just because it’s easy and it works can cause “big store syndrome”.

There are obviously limitations: for one any persistence must be done by hand. We did not yet encounter this requirement, as this is mostly a runtime support for a view. Also, no mutations mean no history and rollbacks, so this is not the thing you are looking for if you need an action history.