Skip to content
libspiffy

Run in a background isolate

On a phone, libspiffy shares the UI thread with your app unless you move it. The first header sync from a CDN can run for minutes, and Isar writes, signing and merkle checks all take time. We recommend running the whole libspiffy actor system in a dedicated isolate. This page describes how to set that up.

The shape we recommend

Section titled “The shape we recommend”
Main isolate (UI) Wallet isolate
Widgets, state management LibSpiffyActorSystem and its coordinator
Plugins that need platform channels: path_provider, the platform keystore Isar, opened here and only here
A handle with ask() and on<E>(), the same shape as the coordinator Header sync, P2P, ARC: all network I/O
A secrets server: answers the wallet isolate’s key reads and writes A SecureStorage that asks the main isolate

Two flows cross the boundary over one pair of ports:

  • Requests and events, main → wallet → main. The UI sends libspiffy requests, which run through coordinator.ask() inside the wallet isolate. The replies, failures and events come back.
  • Secrets, wallet → main → wallet. libspiffy reads and writes keys through its SecureStorage. The wallet isolate forwards those calls to the main isolate, the only place that can reach the platform keystore.

Practices

Section titled “Practices”
  1. Mirror the coordinator. Give the UI a handle with the same ask() and on<E>() as WalletCoordinator. Code then reads the same whether libspiffy runs in-process (in tests, or on a server) or behind the isolate.
  2. Send libspiffy’s own requests. Isolates started with Isolate.spawn share an isolate group, so SendPort.send copies libspiffy’s request, reply and event objects as they are. We checked commands, queries and events. You don’t need a second set of message classes. Each request carries its requestId across, so the handle matches replies the same way the coordinator does.
  3. Resolve paths and secrets on the main isolate. Plugins that use platform channels belong on the main isolate. Pass the data directory in when you start, and serve keys over the secrets flow.
  4. Answer “ready” on ready, not on initialize(). LibSpiffyActorSystem.ready completes once the actors take requests. initialize() returns only after the CDN header sync and the P2P start. ready never completes if initialize() fails first, so wait for whichever comes first.
  5. Report progress as events. initialize(onHeaderSyncProgress:, onHeaderSyncResult:) tells you how the header sync is going. Forward it so the UI can show a progress bar while the wallet already works.
  6. Shut down before you kill. Stop libspiffy and close Isar inside the isolate, wait for that to finish, and only then kill the isolate. Killing it while Isar is writing can crash the whole process. Bound the wait so a stuck wallet can’t block quitting, and make the bound longer than libspiffy’s own 60-second wait for ARC work in flight.
  7. Surface isolate errors. Spawn with errorsAreFatal: false and onError/onExit ports. A crash then fails pending requests and shows up in the UI, instead of leaving it waiting forever.

The sketch

Section titled “The sketch”

One file, in four parts: the configuration, the messages, the main-isolate handle, and the wallet isolate itself.

Configuration and the secrets interface

Section titled “Configuration and the secrets interface”
lib/wallet_isolate.dart
// A sketch of running libspiffy in its own isolate. Not part of libspiffy:
// adapt it to your app.
import 'dart:async';
import 'dart:isolate';


import 'package:isar_community/isar.dart';
import 'package:libspiffy/coordinator.dart';
import 'package:libspiffy/libspiffy.dart';


/// What the wallet isolate needs to start. Resolve paths on the main
/// isolate: plugins such as path_provider need platform channels.
class WalletIsolateConfig {
  final String dataDirectory;
  final String networkType;
  final String arcBaseUrl;
  final List<String> peerAddresses;


  const WalletIsolateConfig({
    required this.dataDirectory,
    required this.networkType,
    required this.arcBaseUrl,
    this.peerAddresses = const [],
  });
}


/// The app's secret store, on the main isolate: the platform keystore.
abstract interface class HostSecrets {
  Future<String?> read(String key);
  Future<void> write(String key, String value);
  Future<void> delete(String key);
  Future<Map<String, String>> readAll();
  Future<void> deleteAll();
}

HostSecrets is the one thing the app plugs in. In a Flutter app it wraps your keystore plugin.

The messages

Section titled “The messages”
lib/wallet_isolate.dart (continued)
// What crosses the boundary. Isolates started with Isolate.spawn share an
// isolate group, so these objects, and libspiffy's requests and events
// inside them, are copied as they are.
final class _Hello {
  final SendPort inbox;
  _Hello(this.inbox);
}


final class _Ready {}


final class _StartFailed {
  final String error;
  _StartFailed(this.error);
}


final class _Ask {
  final CoordinatorRequest request;
  final Duration? timeout;
  _Ask(this.request, this.timeout);
}


final class _Replied {
  final CoordinatorReply reply;
  _Replied(this.reply);
}


final class _Failed {
  final String requestId;
  final String message;
  final bool timedOut;
  _Failed(this.requestId, this.message, {this.timedOut = false});
}


final class _Event {
  final CoordinatorEvent event;
  _Event(this.event);
}


final class _Secret {
  final int id;
  final String op; // read, write, delete, readAll, deleteAll
  final String? key;
  final String? value;
  _Secret(this.id, this.op, [this.key, this.value]);
}


final class _SecretAnswer {
  final int id;
  final Object? value;
  final String? error;
  _SecretAnswer(this.id, this.value, [this.error]);
}


final class _Stop {}


final class _Stopped {}

The wallet isolate says _Hello first, so the main isolate can answer secret requests while libspiffy is still starting. It says _Ready once libspiffy takes requests.

The main-isolate handle

Section titled “The main-isolate handle”
lib/wallet_isolate.dart (continued)
/// The main isolate's handle on the wallet isolate. Its [ask] and [on]
/// mirror libspiffy's WalletCoordinator.
class WalletIsolate {
  final Isolate _isolate;
  final ReceivePort _fromWallet;
  final ReceivePort _errors;
  final SendPort _toWallet;
  final HostSecrets _secrets;
  final _pending = <String, Completer<CoordinatorReply>>{};
  final _events = StreamController<CoordinatorEvent>.broadcast();
  Completer<void>? _stopped;


  WalletIsolate._(this._isolate, this._fromWallet, this._errors,
      this._toWallet, this._secrets);


  /// Starts the wallet isolate and completes once libspiffy takes requests.
  /// Header sync goes on in the background.
  static Future<WalletIsolate> spawn(
      WalletIsolateConfig config, HostSecrets secrets) async {
    final fromWallet = ReceivePort();
    final errors = ReceivePort();
    final isolate = await Isolate.spawn(
      _walletMain,
      (fromWallet.sendPort, config),
      debugName: 'wallet',
      errorsAreFatal: false,
      onError: errors.sendPort,
      onExit: errors.sendPort,
    );
    final ready = Completer<WalletIsolate>();
    WalletIsolate? wallet;
    fromWallet.listen((message) {
      switch (message) {
        case _Hello(:final inbox):
          wallet = WalletIsolate._(isolate, fromWallet, errors, inbox, secrets);
        case _Ready():
          ready.complete(wallet!);
        case _StartFailed(:final error):
          if (!ready.isCompleted) ready.completeError(StateError(error));
        default:
          wallet?._receive(message);
      }
    });
    errors.listen((error) {
      // An uncaught error in the wallet isolate (a list), or its exit (null).
      if (!ready.isCompleted) {
        ready.completeError(StateError('wallet isolate failed: $error'));
      }
      wallet?._failAll('wallet isolate stopped: $error');
    });
    return ready.future;
  }


  /// Sends [request] to libspiffy and completes with its reply. Throws
  /// [CoordinatorFailure] when it failed, and [TimeoutException] when it
  /// timed out (it may still finish).
  Future<R> ask<R extends CoordinatorReply>(CoordinatorRequest<R> request,
      {Duration? timeout}) async {
    final reply = Completer<CoordinatorReply>();
    _pending[request.requestId] = reply;
    _toWallet.send(_Ask(request, timeout));
    return await reply.future as R;
  }


  /// Events of type [E], of wallet [walletId] when given.
  Stream<E> on<E extends CoordinatorEvent>({String? walletId}) => _events.stream
      .where((e) => e is E && (walletId == null || e.walletId == walletId))
      .cast<E>();


  /// Stops libspiffy and closes Isar inside the isolate, then ends it.
  /// Killing the isolate while Isar is writing can crash the process, so
  /// this waits, but no longer than [limit]: libspiffy itself waits up to
  /// 60 s for ARC work in flight.
  Future<void> shutdown({Duration limit = const Duration(seconds: 90)}) async {
    final stopped = _stopped = Completer<void>();
    _toWallet.send(_Stop());
    await stopped.future.timeout(limit, onTimeout: () {});
    _isolate.kill(priority: Isolate.immediate);
    _failAll('wallet isolate shut down');
    _fromWallet.close();
    _errors.close();
    await _events.close();
  }


  void _receive(Object? message) {
    switch (message) {
      case _Replied(:final reply):
        _pending.remove(reply.requestId)?.complete(reply);
      case _Failed(:final requestId, :final message, :final timedOut):
        _pending.remove(requestId)?.completeError(timedOut
            ? TimeoutException(message)
            : CoordinatorFailure(requestId, message));
      case _Event(:final event):
        _events.add(event);
      case _Secret():
        unawaited(_serveSecret(message));
      case _Stopped():
        _stopped?.complete();
    }
  }


  Future<void> _serveSecret(_Secret s) async {
    try {
      Object? value;
      switch (s.op) {
        case 'read':
          value = await _secrets.read(s.key!);
        case 'write':
          await _secrets.write(s.key!, s.value!);
        case 'delete':
          await _secrets.delete(s.key!);
        case 'readAll':
          value = await _secrets.readAll();
        case 'deleteAll':
          await _secrets.deleteAll();
      }
      _toWallet.send(_SecretAnswer(s.id, value));
    } catch (e) {
      _toWallet.send(_SecretAnswer(s.id, null, '$e'));
    }
  }


  void _failAll(String why) {
    for (final entry in _pending.entries) {
      entry.value.completeError(CoordinatorFailure(entry.key, why));
    }
    _pending.clear();
  }
}

Inside the wallet isolate

Section titled “Inside the wallet isolate”
lib/wallet_isolate.dart (continued)
Future<void> _walletMain((SendPort, WalletIsolateConfig) args) async {
  final (toMain, config) = args;
  final inbox = ReceivePort();
  final secrets = _HostSecureStorage(toMain);
  LibSpiffyActorSystem? libspiffy;
  Isar? isar;


  // Listen first: initialize() may already need secrets.
  toMain.send(_Hello(inbox.sendPort));
  inbox.listen((message) async {
    switch (message) {
      case _Ask(:final request, :final timeout):
        try {
          final reply = await libspiffy!.coordinator.ask(request, timeout: timeout);
          toMain.send(_Replied(reply));
        } on CoordinatorFailure catch (f) {
          toMain.send(_Failed(f.requestId, f.message));
        } on TimeoutException catch (t) {
          toMain.send(_Failed(request.requestId, '$t', timedOut: true));
        }
      case _SecretAnswer():
        secrets.answer(message);
      case _Stop():
        await libspiffy?.shutdown();
        await isar?.close();
        toMain.send(_Stopped());
        inbox.close();
    }
  });


  try {
    // In a Flutter app isar_community_flutter_libs provides the native
    // library; a plain Dart program calls Isar.initializeIsarCore first.
    isar = await Isar.open(LibSpiffySchemas.allSchemas,
        directory: config.dataDirectory, name: 'wallet_${config.networkType}');
    final system = libspiffy = LibSpiffyActorSystem();
    final init = system.initialize(
      isar: isar,
      secureStorage: secrets,
      networkType: config.networkType,
      peerAddresses: config.peerAddresses,
      arcConfig: ArcServiceConfig(baseUrl: config.arcBaseUrl),
    );
    // ready completes when commands can be taken, before header sync ends;
    // it never completes if initialize() fails first.
    await Future.any([system.ready, init]);
    system.coordinatorEvents!.listen((event) {
      try {
        toMain.send(_Event(event));
      } catch (_) {
        // An event holding something that cannot be copied: skip it.
      }
    });
    // A failure after ready (P2P could not start) leaves the wallet usable.
    unawaited(init.catchError((Object e) {}));
    toMain.send(_Ready());
  } catch (e) {
    toMain.send(_StartFailed('$e'));
  }
}


/// libspiffy's SecureStorage, answered by the main isolate's [HostSecrets].
class _HostSecureStorage extends SecureStorage {
  final SendPort _toMain;
  final _waiting = <int, Completer<Object?>>{};
  var _next = 0;


  _HostSecureStorage(this._toMain);


  Future<Object?> _ask(String op, [String? key, String? value]) {
    final id = _next++;
    final answer = _waiting[id] = Completer<Object?>();
    _toMain.send(_Secret(id, op, key, value));
    return answer.future;
  }


  void answer(_SecretAnswer a) {
    final waiting = _waiting.remove(a.id);
    if (a.error != null) {
      waiting?.completeError(SecureStorageException(a.error!));
    } else {
      waiting?.complete(a.value);
    }
  }


  @override
  Future<String?> getString(String key) async => await _ask('read', key) as String?;


  @override
  Future<void> setString(String key, String value) => _ask('write', key, value);


  @override
  Future<bool> containsKey(String key) async => await getString(key) != null;


  @override
  Future<void> delete(String key) => _ask('delete', key);


  @override
  Future<void> deleteAll() => _ask('deleteAll');


  @override
  Future<Map<String, String>> getAll() async =>
      (await _ask('readAll') as Map).cast<String, String>();
}

_HostSecureStorage implements the six members libspiffy’s SecureStorage leaves abstract (getString, setString, containsKey, delete, deleteAll, getAll). The key helpers, such as getMnemonic and setXPub, are built on those six. Have getAll and deleteAll ask the main isolate too, rather than answering from a cache: libspiffy uses them to enumerate identities and account metadata.

Using it

Section titled “Using it”
final wallet = await WalletIsolate.spawn(
  WalletIsolateConfig(
    dataDirectory: (await getApplicationSupportDirectory()).path,
    networkType: 'main',
    arcBaseUrl: 'https://arc.gorillapool.io/v1',
  ),
  KeystoreSecrets(), // your HostSecrets over the platform keystore
);


final invoice = await wallet.ask(CreateInvoiceCommand(
  walletId: 'bob',
  amount: BigInt.from(100000),
));


wallet.on<BalanceUpdatedEvent>(walletId: 'bob').listen(render);


// When the app closes:
await wallet.shutdown();

The calls read exactly as they would against libspiffy.coordinator directly. Unit tests and server code can run libspiffy in-process and share the rest of your app code.

Things to adapt

Section titled “Things to adapt”
  • Key preloading. Each SecureStorage call costs a round trip between isolates. If signing feels slow, have the wallet isolate read each wallet’s keys once after start and keep them in memory. Write through to the main isolate on every change.
  • Header sync progress. Pass onHeaderSyncProgress and onHeaderSyncResult to initialize() and forward them as your own messages. See Header sync via CDN.
  • ARC configuration. Pass whatever your app needs (ArcServiceConfig presets, an API key, minimumFeeRate) through WalletIsolateConfig. See Network, ARC & Arcade.
  • Other isolates. The same handle-and-messages shape works for any other long-running service your app moves off the UI thread.