MultiModal Big Image Data Sharing and Exploration

MoBIE specification

The MoBIE specification describes a valid configuration for the MoBIE viewer. The specification defines four different concepts:

The specification is defined via jsonschema and the schema files are located here. It is versioned, following the semantic versioning convention. The current version is 0.2.0.

Using jsonschema:

The jsonschema files can be used in the following ways:


The project and its associated datasets are stored in a directory structure with the project corresponding to the root directory. This directory must contain the file project.json, which must contain a valid project schema See an example project structure, slightly adapted from the zebrafish-lm project:

├── project.json
├── actin
├── cisgolgi
├── lysosomes
├── membrane
├── nuclei
└── trans_golgi

Project Metadata

The project metadata, stored in project.json, has the following structure:

For the zebrafish-lm project the project.json looks like this:

  "datasets": [
  "defaultDataset": "membrane",
  "description": "A quantitative atlas of the cellular architecture for the zebrafish posterior lateral line primoridum.",
  "references": [""],
  "specVersion": "0.2.0"

Local & Remote Projects

Projects can be either stored locally or hosted on a remote object store. In addition, the image data, i.e. the image data in one of the supported data formats and metadata, corresponding to the json files for this project and tables can be accessed from different storage locations for one project.

In more detail, MoBIE currently supports the following storage options:

This enables different combinations of hosting a project, see also this figure:

  1. only on filesystem: project is only available locally; this is the best mode for development.
  2. only on s3 object store: project is self-contained in object store and can be shared with collaborators privately (using a privat bucket) or shared publicly (using a public bucket).
  3. on github and s3 object store: the metadata is stored on github, which also serves as entrypoint for the viewer. The image sources are stored on the s3 bucket. This set-up has the advantage that metadata is under version control.



A dataset consists of a root directory, which must contain the file dataset.json with a valid dataset schema. It should contain the subdirectories images, containing the image data, tables, containing the table data and may contain the directory misc, containing additional data associated with this dataset. Note that the location of image and table data is determined in dataset.json and it is thus possible to choose a different directory structure than images/ and tables/ to store them, but the layout using images/ and tables/ is recommended for consistency with other MoBIE projects and assumed throughout this document.

The images directory contains the metadata files describing the image data, see data specification for details. It may contain additional subdirectories to organise these files. By convention the files for different data formats are often separated into folders named accordingly, e.g. images/bdv-n5 and images/bdv-n5-s3.

The tables directory contains all tabular data assoicated with segmentations or grid views (see data specification and view specification for details) All tables associated with a segmentation or view, must be located in the same subdirectory. This subdirectory must contain a default table, which should be called default.tsv, and may contain additional tables. See the table data specification for details on how tables are stored.

The misc directory may contain the subdirectory views with additional views stored in json files according to the views spec. It may also contain the file leveling.json, which specifies the “natural” orientation of the dataset and other subdirectories and other files that are associated with this dataset.

See an example dataset directory structure and dataset.json (left incomplete for brevity) for one of the zebrafish-lm project’s dataset below.

├── sources.json
├── images
│   ├── bdv-n5
│   └── bdv-n5-s3
├── misc
│   └── views
│   └── leveling.json
└── tables
    ├── segmentation_sample1
    └── segmentation_sample2

Dataset Metadata

The dataset metadata, stored in dataset.json, has the following structure:

For the zebrafish-lm dataset the dataset.json looks like this (source and view metadata omitted for brevity):

    "description": "Zebrafish primordium with actin staining.",
    "is2d": false,
    "sources": {
        "membrane_sample1": {"..."},
        "membrane_sample2": {"..."},
        "segmentation_sample1": {"..."}
        "segmentation_sample2": {"..."}
    "views": {
        "default": {"..."}


A source consists of an image (volume) and associated metadata like tables. Two different types of sources are supported:

Image Data

The data is stored in a chunked format for multi-dimensional data. Currently MoBIE supports the following data formats:

For the data formats using a BigDataViewer xml, each xml must only contain a single setup id and the value of the field name must be the same as the name in the source metadata.

Table Data

MoBIE supports tables associated with segmentations, where each row corresponds to some properties of an object in the segmentation. The tables should be stored as tab separated values .tsv files; they may also be stored as comma separated values .csv.

The default segmentation table (which should be stored in default.tsv in the corresponding table, see also dataset specification) must contain the columns label_id, anchor_x, anchor_y, anchor_z, bb_min_x, bb_min_y, bb_min_z and bb_min_x, bb_min_y, bb_min_z. The anchor columns specify a reference point for the object corresponding to this row. It will be centered when the corresponding obejct is selected. The bb_min and bb_max columns specify the start and stop of the bounding box for the object. Both anchor and bounding box coordinates must be given in phyisical units. It may contain additional columns.

Additional segmentation tables must contain the column label_id and may contain arbitraty additional columns. The label_ids in the additional tables may be incomplete, but they must not contain ids not present in the default table. For example, given label_ids [1, 2, 3, 4, 5] in the default table the label_ids [1, 2, 3, 4, 5] or [1, 3, 4] are valid but [1, 3, 4, 7] is invalid. The label id 0 is reserved for the background and must not be listed in any of the tables.

See an example segmentation default table for 8 objects with additional column n_pixels giving the number of pixels of the object.

label_id    anchor_x    anchor_y    anchor_z    bb_min_x    bb_min_y    bb_min_z    bb_max_x    bb_max_y    bb_max_z    n_pixels
1.0 49.78   41.25   8.7 45.23   37.20386467793656   3.36    56.053  44.24   17.73   127867.0
2.0 42.79   40.06   9.0 38.29   36.31097192566608   3.81    48.513  43.55   15.26   112529.0
3.0 52.79   44.33   9.9 48.41   40.477804769594975  3.81    56.946  47.02   17.96   111520.0
4.0 57.90   42.95   7.8 53.77   39.18807079409317   3.59    62.204  45.63   16.61   102318.0
5.0 47.19   45.17   9.8 43.45   42.26359027413593   4.49    51.490  47.91   17.06   95659.0
6.0 40.31   41.75   9.9 37.00   35.31886886758777   4.71    46.430  44.84   15.94   98376.0
7.0 64.68   43.91   8.4 60.12   39.98175324055582   4.71    70.439  46.72   15.71   104657.0
8.0 35.57   43.39   10. 30.55   40.676225381210635  4.93    40.676  46.03   16.16   104961.0

The color scheme used to display the segmentation can also be loaded from a table, see colorByColumn in the source metadata. In order to set an explicit color map, the field color may be set to argbColumn. In this case, the values in the column must follow the format alpha-red-green-blue, e.g. 255-0-0-255.

Source Metadata

The metadata for the sources of a dataset is specified in the field sources of dataset.json (see also dataset metadata). sources contains a mapping of source names to source metadata. The metadata entries have the following structure (see below for an example json file):

  "image": {
    "imageData": {
      "bdv.ome.zarr.s3": {
        "relativePath": "dolor"
      "bdv.n5.s3": {
        "relativePath": "laboris anim id amet minim"
      "bdv.n5": {
        "relativePath": "minim qui esse laborum"
      "bdv.hdf5": {
        "relativePath": "amet"
      "bdv.ome.zarr": {
        "relativePath": "incididunt in est"


A view stores all metadata necessary to fully reproduce a MoBIE viewer state.

Grid Views

Grid views can be used to arrange sources in a grid automatically. They must have at least one associated table. Tables for grid views should be stored as tab separated values, but may also be comma separated. They must contain the column grid_id, which is used for navigation in the viewer, and must contain at least one more column. The grid_id column indexes the 2d grid position, assuming the same order of sources as in the sources list given in the view metadata.

See an example grid view table for 4 grid positions that also gives the presence of different organelles for each position.

grid_id mitochondria    vesicles    golgi   er
0   1   0   1   0
1   1   0   1   1
2   0   0   0   1
3   0   1   0   1

View Metadata

The metadata for the views of a dataset is specified in the field views of dataset.json (see also dataset metadata). views contains a mapping of view names to view metadata.

Additional views can be stored as json files with the field views mapping view names to metadata in the folder misc/views

The metadata entries have the following structure (see below for an example json file):

  "isExclusive": false,
  "uiSelectionGroup": "~8}cn|^BSKD",
  "sourceDisplays": [
      "segmentationDisplay": {
        "opacity": 0.3046827220289676,
        "lut": "blueWhiteRed",
        "name": "S-{fbv3P",
        "sources": [
        "selectedSegmentIds": [
        "tables": [
        "resolution3dView": [
        "scatterPlotAxes": [
      "segmentationDisplay": {
        "opacity": 0.42985613388634425,
        "lut": "glasbeyZeroTransparent",
        "name": "9",
        "sources": [
        "selectedSegmentIds": [
        "colorByColumn": "%kK.fa^",
        "scatterPlotAxes": [
      "segmentationDisplay": {
        "opacity": 0.2241566426144641,
        "lut": "glasbeyZeroTransparent",
        "name": "i!I>lkrxPH",
        "sources": [
        "colorByColumn": ".lF1&v<",
        "showScatterPlot": true,
        "valueLimits": [
        "resolution3dView": [
        "scatterPlotAxes": [
        "showSelectedSegmentsIn3d": true,
        "tables": [
        "selectedSegmentIds": [
      "segmentationDisplay": {
        "opacity": 0.9195656267427257,
        "lut": "blueWhiteRed",
        "name": "z#Gqa*H6",
        "sources": [
        "valueLimits": [