Regions of interest

In image processing, two of the most often needed yet complex operations are segmentation and registration. Regions of interest (ROIs) are an effective way of expressing and visualizing the results of a segmentation. But ROIs are a deceptively complex topic, difficult to implement well in a software package.

Discussion

• G. Landini: Imglib or ImageJ2 and ROIs – ROIs should be drawn from the center of each pixel
• S. Preibisch: Imglib or ImageJ2 and ROIs - Where is a pixel? – Two kinds of ROIs: discrete and continuous
• D. White: Imglib or ImageJ2 and ROIs - A pixel is not a little square – Alvy Ray's classic article
• J. Tinevez: ImageJ class hierarchy suggestions – a proposed interface-driven design for ROIs

There is clearly is a division between ROIs as a data structure, and ROIs for display. For clarity, we propose the following terminology:

1. But there could be value in allowing regions with non-integer or non-binary boundaries. For example, a region with alpha channel would allow for a "fuzzy" boundary representing the confidence level of inclusion within the region, that can be later adjusted with a threshold.

2. For example, CellProfiler represents segmentation results using an 8-bit mask, with 0 as background, 1 as the first region, 2 as the second region, etc.

Open Microscopy Environment

The OME team has put a lot of thought into ROIs. They have an evolving proposal for handling 5D ROIs, which are collections of 2D regions over multiple planes. They are planning to expand it to N dimensions, with collections of 3D regions over multiple volumes (see ticket #112 progress for technical details). You can also check out the currently released 2010-04 schema for ROIs, which is already partially deprecated.

The OME team has also done a lot of research on how to store ROIs efficiently, since automated segmentation can produce hundreds of thousands or millions of distinct regions, which ideally must be queriable efficiently.

Class diagram of org.openmicrosopy.shoola.util.roi added as file attachment (GBH)

Shortcomings

Johannes Schindelin summarizes the shortcomings of ROIs in the current version of ImageJ:

• only 2D (extends identically to n dimensions)—in Fiji, we only have several 3D extensions to ImageJ's ROIs, but they are in effect stacks of 2D ROIs, while it would be pretty interesting to have a proper surface model for 3D ROIs, at least as an option
• no distinction between region of interest and lines/points of interest
• tries to fit in a geometrical model, where the geometry of pixels is a highly ambiguous thing (just think about it: what does a ROI containing exactly one pixel look like?)
• is way too tied in with the rest of the GUI (some ROIs even know which ImagePlus they belong to!)
• no common interface, rather a base class that describes rectangular (about the most unnatural shape of regions of interest in biological data) regions of interest
• no notion of disconnected regions of interest (it is possible to have compound regions of interest, but some algorithms consider them as separate particles while others treat the union as one region of interest)
• inherent limitation to integer coordinates
• no clear-cut distinction between the necessary interfaces, such as visualization, geometrical measures, determining whether a pixel is considered inside/outside
• (added by Gabriel Landini) inconsistent behavior regarding boundary pixels of ROIs, illustrated by the Draw command (try maximizing a rectangular ROI and applying it with the Draw command; only two borders are drawn because the other two are outside the image)

Relevant Java classes and packages

• java.awt.Area
• java.awt.Shape
• ini.trakem2.display.AreaWrapper
• javax.media.jai.ROI
• javax.media.jai.ROIShape
• JHotDraw
• Leonardo

Use of the JAI classes would not imply use of JAI generally. ROIShape is independent  of other JAI classes; ROI is dependent on an iterator.