Isolates

Seamlessly run moor on a background isolate and unblock the main thread

Preparations

To use the isolate api, first enable the appropriate build option by creating a file called build.yaml in your project root, next to your pubspec.yaml. It should have the following content:

targets:
  $default:
    builders:
      moor_generator:
        options:
          generate_connect_constructor: true

Next, re-run the build. You can now add another constructor to the generated database class:

@UseMoor(...)
class TodoDb extends _$TodoDb {
  TodoDb() : super(VmDatabase.memory());

  // this is the new constructor
  TodoDb.connect(DatabaseConnection connection) : super.connect(connection);
}

Using moor in a background isolate

With the database class ready, let’s open it on a background isolate

import 'package:moor/isolates.dart';

// This needs to be a top-level method because it's run on a background isolate
DatabaseConnection _backgroundConnection() {
    // construct the database. You can also wrap the VmDatabase in a "LazyDatabase" if you need to run
    // work before the database opens.
    final database = VmDatabase.memory();
    return DatabaseConnection.fromExecutor(database);
}

void main() async {
    // create a moor executor in a new background isolate. If you want to start the isolate yourself, you
    // can also call MoorIsolate.inCurrent() from the background isolate
    MoorIsolate isolate = await MoorIsolate.spawn(_backgroundConnection);

    // we can now create a database connection that will use the isolate internally. This is NOT what's
    // returned from _backgroundConnection, moor uses an internal proxy class for isolate communication.
    DatabaseConnection connection = await isolate.connect();

    final db = TodoDb.connect(connection);

    // you can now use your database exactly like you regularly would, it transparently uses a 
    // background isolate internally
}

Common operation modes

The MoorIsolate object itself can be sent across isolates, so if you have more than one isolate from which you want to use moor, that’s no problem!

One executor isolate, one foreground isolate: This is the most common usage mode. You would call MoorIsolate.spawn from the main method in your Flutter or Dart app. Similar to the example above, you could then use moor from the main isolate by connecting with MoorIsolate.connect and passing that connection to a generated database class.

One executor isolate, multiple client isolates: The MoorIsolate can be sent across multiple isolates, each of which can use MoorIsolate.connect on their own. This is useful to implement a setup where you have three or more threads:

  • The moor executor isolate
  • A foreground isolate, probably for Flutter
  • Another background isolate, which could be used for networking.

You can the read data from the foreground isolate or start query streams, similar to the example above. The background isolate would also call MoorIsolate.connect and create its own instance of the generated database class. Writes to one database will be visible to the other isolate and also update query streams.

How does this work? Are there any limitations?

All moor features are supported on background isolates and work out of the box. This includes

  • Transactions
  • Auto-updating queries (even if the table was updated from another isolate)
  • Batched updates and inserts
  • Custom statements or those generated from an sql api

Please note that, will using a background isolate can reduce lag on the UI thread, the overall database is going to be much slower! There’s a large overhead involved in sending data between isolates, and that’s exactly what moor is doing internally. If you’re not running into dropped frames because of moor, using a background isolate is a overkill.

Internally, moor uses the following model to implement this api:

  • A server isolate: A single isolate that executes all queries and broadcasts tables updates. This is the isolate created by MoorIsolate.spawn. It supports any number of clients via an rpc-like connection model. Connections are established via SendPorts and ReceivePorts. Internally, the MoorIsolate class only contains a reference to a SendPort that can be used to establish a connection to the background isolate. This lets users share the MoorIsolate object across many isolates and connect multiple times. The actual server logic that listens on the port is in a private _MoorServer class.
  • Client isolates: Any number of clients in any number of isolates can connect to a MoorIsolate. The client acts as a moor backend, which means that all queries are built on the client isolate. The raw sql string and parameters are then sent to the server isolate, which will enqueue the operation and execute it eventually. Implementing the isolate commands at a low level allows users to re-use all their code used without the isolate api.
Last modified November 5, 2019: Documentation for the isolate executor (44cb7c05)