SpiderNode In Positron

Last Friday, Brendan Dahl landed SpiderNode integration into Positron. Now, when you run an Electron app in Positron, the app’s main script runs in a JavaScript context that includes the Node module loader and the core Node modules.

The hello-world-server test app demonstrates an Electron BrowserWindow connecting to a Node HTTP server started by the main script. It’s similar to the hello-world test app (a.k.a. the Electron Quick Start app), with this added code to create the HTTP server:

// Load the http module to create an HTTP server.
var http = require('http');

// Configure our HTTP server to respond with Hello World to all requests.
var server = http.createServer(function (request, response) {
  response.writeHead(200, {"Content-Type": "text/plain"});
  response.end("Hello World from node " + process.versions.node + "\n");

The main script then loads a page from the server in an Electron BrowserWindow:

const electron = require('electron');
const app = electron.app;  // Module to control application life.
const BrowserWindow = electron.BrowserWindow;  // Module to create native browser window.
var mainWindow = null;
// This method will be called when Electron has finished
// initialization and is ready to create browser windows.
app.on('ready', function() {
    // Create the browser window.
    mainWindow = new BrowserWindow({width: 800, height: 600});

    // and load the index.html of the app.

Which results in:

Hello World Server Demo

The simplicity of that screenshot belies the complexity of the implementation! It requires SpiderNode, which depends on SpiderShim (based on ChakraShim from node-chakracore). And it must expose Node APIs to the existing JavaScript context created by the Positron app runner while also synchronizing the SpiderMonkey and libuv event loops.

It’s also the first example of using SpiderNode in a real-world (albeit experimental) project, which is an achievement for that effort and a credit to its principal contributors, especially Ehsan Akhgari, Trevor Saunders, and Brendan himself.

Try it out for yourself:

git clone https://github.com/mozilla/positron
cd positron
git submodule update --init
MOZCONFIG=positron/config/mozconfig ./mach build
./mach run positron/test/hello-world-server/

Or, for a more interesting example, run the basic browser app:

git clone https://github.com/hokein/electron-sample-apps
./mach run electron-sample-apps/webview/browser/

(Note: Positron now works on Mac and Linux but not Windows, as SpiderNode doesn’t yet build there.)


The Once And Future GeckoView

GeckoView is an Android library for embedding Gecko into an Android app. Mark Finkle introduced it via GeckoView: Embedding Gecko in your Android Application back in 2013, and a variety of Fennec hackers have contributed to it, including Nick Alexander, who described a Maven repository for GeckoView in 2014. It’s also been reused, at least experimentally, by Joe Bowser to implement MozillaView – GeckoView Proof of Concept.

But GeckoView development hasn’t been a priority, and parts of it have bitrotted. It has also remained intertwined with Fennec, which makes it more complicated to reuse for another Android app. And the core WebView class in Android (along with the cross-platform implementation in Crosswalk), already address a variety of web rendering use cases for Android app developers, which complicates its value proposition.

Nevertheless, it may have an advantage for the subset of native Android apps that want to provide a consistent experience across the fragmented Android install base or take advantage of the features Gecko provides, like WebRTC, WebVR, and WebAssembly. More research (and perhaps some experimentation) will be needed to determine to what extent that’s true. But if “there’s gold in them thar hills,” then I want to mine it.

So Nick recently determined what it would take to completely separate GeckoView from Fennec, and he filed a bunch of bugs on the work. I then filed meta-bug 1291362 — standalone Gradle-based GeckoView libary to track those bugs along with the rest of the work required to build and distribute a standalone Gradle-based GeckoView library reusable by other Android apps. Nick, Jim Chen, and Randall Barker have already made some progress on that project.

It’s still early days, and I’m still pursuing the project’s prioritization (say that ten times fast). So I can’t yet predict when we’ll complete that work. But I’m excited to see the work underway, and I look forward to reporting on its progress!


Graphene On Gecko Without B2G

Recently, I’ve been experimenting with a reimplementation of Graphene on Gecko that doesn’t require B2G. Graphene is a desktop runtime for HTML apps, and Browser.html uses it to run on Servo. There’s a Gecko implementation of Graphene that is based on B2G, but it isn’t maintained and appears to have bitrotted. It also entrains all of B2G, although Browser.html doesn’t use many B2G features.

My reimplementation is minimal. All it does is load a URL in a native window and expose a few basic APIs (including <iframe mozbrowser>). But that’s enough to run Browser.html. And since Browser.html is a web app, it can be loaded from a web server (with this change that exposes <iframe mozbrowser> to https: URLs).

I forked browserhtml/browserhtml to mykmelez/browserhtml, built it locally, and then pushed the build to my GitHub Pages website at mykmelez.github.io/browserhtml. Then I forked mozilla/gecko-dev to mykmelez/graphene-gecko and reimplemented Graphene in it. I put the implementation into embedding/graphene/, but I could have put it anywhere, including in a separate repo, per A Mozilla App Outside Central.

To try it out, clone, build, and run it:

git clone https://github.com/mykmelez/graphene-gecko
cd graphene-gecko
./mach build && ./mach run https://mykmelez.github.io/browserhtml/

You’ll get a native window with Browser.html running in Graphene:

Graphene on Gecko


A Mozilla App Outside Central

After forking gecko-dev again for an experiment, I wondered if there was a better way to create a Mozilla app outside the mozilla-central repository. After all, that repository, mirrored as mozilla/gecko-dev on GitHub, is huge. And forking the same repo twice to the same GitHub account (or organization) requires cumbersome workarounds (and workarounds for the workarounds in some cases).

Plus, Mozilla apps don’t necessarily need to modify Gecko. But even when they do, they could share a single fork, with branches for each app, if there was a way to create a Mozilla app that imported gecko-dev as a dependency without having to live inside a fork of gecko-dev itself.

So I created an example app that does just that. It happens to use a shallow Git submodule to import gecko-dev, but it could use a Git subtree or any other dependency management tool.

The build process touches only one file in the gecko-dev subdirectory: it creates a symlink to the parent (app) directory to work around the limitation that Mozilla app directories must be subdirectories of gecko-dev.

To try it for yourself, clone the mykmelez/mozilla-app repo and build it:

git clone --recursive --shallow-submodules https://github.com/mykmelez/mozilla-app
cd mozilla-app/
./mach build && ./mach run

You should see a window like this:

Mozilla app window

And your clone will contain the app’s files, the gecko-dev submodule directory, and an obj directory containing the built app:

> ls
LICENSE  app.mozbuild  components   gecko-dev  modules    moz.configure  obj
app      branding      confvars.sh  mach       moz.build  mozconfig


The app builds Gecko from source, so you still need Mozilla build prerequisites. On some platforms, you can install these via ./mach bootstrap.

I’ve only tested it on Mac, but it probably works on Linux, and it probably doesn’t work on Windows (because of the symlink).

Although the Mozilla application framework is mature and robust, it isn’t explicitly generalized for use beyond Firefox anymore, and it could change or break at any time.


Continuous Integration of Positron with TaskCluster

I recently enabled continuous integration of Positron using TaskCluster’s support for GitHub repositories. I’ve used Travis CI to integrate other GitHub projects, but Positron is too large for Travis to build without timing out, since it builds all of Gecko. So I needed an alternative.

Caveat: TaskCluster only supports GitHub repositories in the mozilla organization or another org that is explicitly configured for it, as described by Setting up Taskcluster with a Github Organization. So it won’t work with your personal fork.

Also, while the Positron repository is based on gecko-dev, I didn’t use GitHub’s “fork” feature to create it, since GitHub won’t let you fork a repo to another repo in the same organization. Instead, I created a new repository and then copied gecko-dev’s commits to it, as described in Copy/fork a git repo on github into same organization.

(I also had to push the copied commits to GitHub in batches, as described in Github remote push pack size exceeded. And I had to push the first batch via special syntax, per Pushing a large github repo fails with “unable to push to unqualified destination: master”.)

To enable CI, I first created this Docker image, based on Ubuntu 14.04 (Trusty), that includes the Mozilla build toolchain and dependencies installed by Gecko’s bootstrap.py script. The image also includes the taskcluster-vcs utility, which caches gecko-dev and clones from the cache so that TaskCluster pulls from GitHub are lighter-weight.

Here’s the Dockerfile for that image:

FROM ubuntu:trusty

RUN apt-get update -y && \
    apt-get install -y \
      curl \
      git \
      nodejs \
      nodejs-legacy \
      npm \
      python \

RUN npm install -g taskcluster-vcs

RUN wget -O bootstrap.py https://raw.githubusercontent.com/mozilla/positron/master/python/mozboot/bin/bootstrap.py && \
    python bootstrap.py --no-interactive --application-choice=browser

Then I replaced the upstream .taskcluster.yml file with one that clones and builds Positron when a developer submits a pull request or lands a change. .taskcluster.yml is like .travis.yml, but for TaskCluster. Here’s that file:

version: 0
    name: "Positron"
    description: "continuous integration for Positron"
    owner: "{{ event.head.user.email }}"
    source: "{{ event.head.repo.url }}"
    - provisionerId: "{{ taskcluster.docker.provisionerId }}"
      workerType: "{{ taskcluster.docker.workerType }}"
              env: true
                  - pull_request.opened
                  - pull_request.synchronize
                  - pull_request.reopened
                  - push
          maxRunTime: 7200 # seconds (i.e. two hours)
          image: "mykmelez/docker-build-positron:latest"
              - "/bin/bash"
              - "--login"
              - "-c"
              - "tc-vcs checkout repo https://github.com/mozilla/gecko-dev $GITHUB_HEAD_REPO_URL $GITHUB_HEAD_BRANCH &&
                 cd repo/ && SHELL=/bin/bash ./mach build"
          name: "checkoutandbuild"
          description: "check out and build the repository"
          owner: "{{ event.head.user.email }}"
          source: "{{ event.head.repo.url }}"

Finally, I added TaskClusterRobot to the list of collaborators in the repo’s Settings > Collaborators & teams list, giving it Write permissions, so it can annotate pull requests with icons representing the state of the job, just like Travis does:

Screen Shot of GitHub PR with CI annotations

Et voilà, continuous integration for Positron!

(Once Positron starts running Node and Electron tests, I’ll add tasks to run those tests and report their results.)


a basic browser app in Positron

Over in Positron, we’ve implemented enough of Electron’s <webview> element to run the basic browser app in this collection of Electron sample apps. To try it out:

git clone https://github.com/hokein/electron-sample-apps
git clone https://github.com/mozilla/positron
cd positron
./mach build
./mach run ../electron-sample-apps/webview/browser/

Project Positron

Along with several colleagues, I recently started working on Project Positron, an effort to build an Electron-compatible runtime on top of the Mozilla technology stack (Gecko and SpiderMonkey). Mozilla has long supported building applications on its stack, but the process is complex and cumbersome. Electron development, by comparison, is a dream. We aim to bring the same ease-of-use to Mozilla.

Positron development is proceeding along two tracks. In the SpiderNode repository, we’re working our way up from Node, shimming the V8 API so we can run Node on top of SpiderMonkey. Ehsan Akhgari details that effort in his Project SpiderNode post.

In the Positron repository, we’re working our way down from Electron, importing Electron (and Node) core modules, stubbing or implementing their native bindings, and making the minimal necessary changes (like basing the <webview> element on <iframe mozbrowser>) so we can run Electron apps on top of Gecko. Eventually we aim to join these tracks, even though we aren’t yet sure exactly where the last spike will be located.

It’s early days. As Ehsan noted, SpiderNode doesn’t yet link the Node executable successfully, since we haven’t shimmed all the V8 APIs it accesses. Meanwhile, Positron supports only a tiny subset of the Electron and Node APIs.

Nevertheless, we reached a milestone today: the tip of the Positron trunk now runs the Electron Quick Start app described in the Electron tutorial. That means it can open a BrowserWindow, hook up a DevTools window to it (with Firefox DevTools integration contributed by jryans), and handle basic application lifecycle events. We’ve imported that app into the Positron repository as our “hello world” example.

Clone and build Positron to see it in action!