Cornerstone 2.0

[sedghi]

---

Memory rework docs here #1419

---

Here are the breaking changes from the cornerstone 1.x to 2.x.

Most of the changes are related to the new Segmentation model, but there are also changes in the DICOM Image Loader, Viewport APIs, Cache, and Events. Let's dive into the details.

Building And Bundling

Typescript Version

We have upgraded the typescript version from 4.6 to 5.5 in the 2.0 version of the cornerstone3D.
This upgrade most likely don't require any changes in your codebase, but it is recommended to update the typescript version in your project to 5.5
to avoid any issues in the future.

Why?

The upgrade to TypeScript 5.4 allows us to leverage the latest features and improvements offered by the TypeScript standard. You can read more about it here: https://devblogs.microsoft.com/typescript/announcing-typescript-5-5/

ECMAScript Target

In Cornerstone3D version 1.x, we targeted ES5. With the release of version 2.0, we have updated our target to ES2022.

Why?

It will result in a smaller bundle size and improved performance. There is a good chance that your setup already supports ES2022:

https://compat-table.github.io/compat-table/es2016plus/

Remove of CJS

Starting with Cornerstone3D 2.x, we will no longer ship the CommonJS (CJS) build of the library. You most likely won't need to make any changes to your codebase. If you are aliasing the cjs library in your bundler, you can remove it completely.

Why?

Both Node.js and modern browsers now support ECMAScript Modules (ESM) by default. However, in the rare case where you need a non-ESM version, you can use the Universal Module Definition (UMD) build of the library.

---

DICOM Image Loader

Decoders Update

@cornerstonejs/dicomImageLoader previously utilized the old API for web workers, which is now deprecated. It has transitioned to the new web worker API via the comlink package. This change enables more seamless interaction with web workers and facilitates compiling and bundling the web workers to match the ESM version of the library.

Why?

To consolidate the web worker API using a new ES module format, which will enable new bundlers like vite to work seamlessly with the library.

Removing support for non-worker decoders

We have removed support for non-web worker decoders in the 2.0 version of the cornerstone3D. This change is to ensure that the library is more performant and to reduce the bundle size.

Why?

We see no compelling reason to use non-worker decoders anymore. Web worker decoders offer superior performance and better compatibility with modern bundlers.

DICOM Image Loader ESM default

We have changed the default export of the DICOM Image Loader to ESM in the 2.0 version of the cornerstone3D and correctly
publish types

This mean you don't need to have an alias for the dicom image loader anymore

Probably in your webpack or other bundler you had this

 alias: {
  '@cornerstonejs/dicom-image-loader':
    '@cornerstonejs/dicom-image-loader/dist/dynamic-import/cornerstoneDICOMImageLoader.min.js',
},

Now you can remove this alias and use the default import

Why?

ESM is the future of JavaScript, and we want to ensure that the library is compatible with modern bundlers and tools.

InitCornerstoneDICOMImageLoader

We have cleaned up how you initialize the DICOM Image Loader in the 2.0 version of the cornerstone3D:

cornerstoneDICOMImageLoader.external.cornerstone = cornerstone;
cornerstoneDICOMImageLoader.external.dicomParser = dicomParser;
cornerstoneDICOMImageLoader.configure({
  useWebWorkers: true,
  decodeConfig: {
    convertFloatPixelDataToInt: false,
    use16BitDataType: preferSizeOverAccuracy || useNorm16Texture,
  },
});

let maxWebWorkers = 1;

if (navigator.hardwareConcurrency) {
  maxWebWorkers = Math.min(navigator.hardwareConcurrency, 7);
}

var config = {
  maxWebWorkers,
  startWebWorkersOnDemand: false,
  taskConfiguration: {
    decodeTask: {
      initializeCodecsOnStartup: false,
      strict: false,
    },
  },
};

cornerstoneDICOMImageLoader.webWorkerManager.initialize(config);

let maxWebWorkers = 1;

if (navigator.hardwareConcurrency) {
  maxWebWorkers = Math.min(navigator.hardwareConcurrency, 7);
}

cornerstoneDICOMImageLoader.configure({
  cornerstone,
  dicomParser,
  useWebWorkers: true,
  maxWebWorkers,
  decodeConfig: {
    convertFloatPixelDataToInt: false,
    use16BitDataType: preferSizeOverAccuracy || useNorm16Texture,
  },
});

Why?

Due to circular dependencies in the previous version, we modified the initialization process for the DICOM image loader. This change enhances the library's robustness and simplifies maintenance.

---

Viewport APIs

Reset Camera

Previously, we had a resetCamera method that took positional arguments. Now it takes an object argument.

viewport.resetCamera(false, true, false);

viewport.resetCamera({
  resetZoom: true,
  resetPan: false,
  resetToCenter: false,
});

Why?

This change enhances our future development process by ensuring we won't need to modify the method signature later. It also improves readability for users calling the method.

Rotation

The rotation property has been removed from getProperties

viewport.getProperties().rotation;
viewport.setProperties({ rotation: 10 });

const { rotation } = viewport.getViewPresentation();
viewport.setViewPresentation({ rotation: 10 });

Why?

rotation is not a property of the viewport but rather a view prop. You can now access it through getViewPresentation.

getReferenceId

is now getViewReferenceId

getReferenceId-- > getViewReferenceId;

Why?

It is more accurate to use getViewReferenceId to reflect the actual function of the method since it returns view-specific information.

Actor property referenceId

is now renamed to referencedId

export type ActorEntry = {
  uid: string,
  actor: Actor | VolumeActor | ImageActor | ICanvasActor,
  /** the id of the reference volume from which this actor is derived or created*/
  referenceId?: string,
  slabThickness?: number,
  clippingFilter?: any,
};

export type ActorEntry = {
  uid: string,
  actor: Actor | VolumeActor | ImageActor | ICanvasActor,
  /** the id of the referenced object (e.g., volume) from which this actor is derived or created*/
  referencedId?: string,
  slabThickness?: number,
  clippingFilter?: any,
};

Why?

The term referencedId is more accurate and reflects the actual function of the property. It aligns with our library's naming conventions, such as referencedImageId and referencedVolumeId. Since an Actor can be derived from either a volume or an image, using referencedId instead of referenceId is more precise.

---

Cache

VolumeCache

By default when you create an image volume in the VolumeCache we allocate the memory for each image in the ImageCache as well.

You don't need to make any changes to your codebase

Why?

Since it's free, we can allocate memory for the images in the ImageCache and assign a view for their pixelData on a portion of the volume. This approach offers several benefits:

1. Converting between stack and volume viewports becomes faster.
2. When dealing with stack and volume labelmaps, updates in a volume viewport take effect instantly in the stack viewport and vice versa.

ImageVolume

convertToCornerstoneImage is now deprecated in favor of getCornerstoneImage

volume.convertToCornerstoneImage(imageId, imageIdIndex) --> volume.getCornerstoneImage(imageId, imageIdIndex)

Why?

1. The naming was incorrect. It was not actually a cornerstone image, but a cornerstone image load object, which is different. 2. It was a duplicate.

---

Events and Event Details

VOLUME_SCROLL_OUT_OF_BOUNDS

is now VOLUME_VIEWPORT_SCROLL_OUT_OF_BOUNDS

Why?

This change was made to maintain consistency with the naming of other events in the library.

CameraModifiedEventDetail

Does not publish the rotation anymore, and it has moved to ICamera which is published in the event

type CameraModifiedEventDetail = {
  previousCamera: ICamera,
  camera: ICamera,
  element: HTMLDivElement,
  viewportId: string,
  renderingEngineId: string,
};

access the rotation from the camera object which previously was in the event detail root

STACK_VIEWPORT_NEW_STACK publisher

Is not the element not the eventTarget

eventTarget.addEventListener(Events.STACK_VIEWPORT_NEW_STACK, newStackHandler);

// should be now

element.addEventListener(Events.STACK_VIEWPORT_NEW_STACK, newStackHandler);

Why?

We made this change to maintain consistency, as all other events like VOLUME_NEW image were occurring on the element. This modification makes more sense because when the viewport has a new stack, it should trigger an event on the viewport element itself.

---

Renaming and Nomenclature

Units

In the annotation cachedStats you need to use the new units

unit-- > lengthUnits;
areaUnit-- > areaUnits;
modalityUnit-- > pixelValueUnits;

Also the function getModalityUnit is now getPixelValueUnits if you were using it.

getModalityUnit-- > getPixelValueUnits;

As a side effect getCalibratedLengthUnitsAndScale now returns {areaUnits, lengthUnits, scale} instead of {units, areaUnits, scale}

Why?

There was too much inconsistency in the units used throughout the library. We had `unit`, `areaUnits`, `modalityUnit`, and various others. Now, we have consolidated these units. You need to update your codebase to reflect the new unit system if you are hydrating annotations for Cornerstone3D.

In addition modalityUnit is now pixelValueUnits to reflect the correct term, since for a single modality there can be multiple pixel values (e.g, PT SUV, PT RAW, PT PROC)

Other

cloneDeep

The structuredClone function has replaced the previous method. You don't need to make any changes to your codebase that uses Cornerstone3D.

Why?

Why to depend on a third-party library when we can use the native browser API?

Always Prescale

By default, Cornerstone3D always prescales images with the modality LUT. You probably don't need to make any changes to your codebase.

Why?

Previously, the decision to prescale was made by the viewport, and all viewports were doing it. However, we observed prescaling bugs in some custom image loaders that users had implemented. These issues have now been resolved by always prescaling.

getDataInTime

The imageCoordinate is renamed to worldCoordinate in the 2.0 version of the cornerstone3D. As it
is the correct term and was misleading in the previous version.

const options = {
    imageCoordinate
  };

function getDataInTime(
  dynamicVolume,
  options
):

const options = {
    worldCoordinate
  };

function getDataInTime(
  dynamicVolume,
  options
):

Why?

This is the way

triggerAnnotationRenderForViewportIds

Now only requires viewportIds and doesn't need renderingEngine anymore

triggerAnnotationRenderForViewportIds(renderingEngine, viewportIds) ---> triggerAnnotationRenderForViewportIds(viewportIds)

Why?

Since there is one rendering engine per viewport, there is no need to pass the rendering engine as an argument.

---

New Segmentation Model

SegmentationDisplayTool

There's no need to add the SegmentationDisplayTool to the toolGroup anymore.

Before

toolGroup2.addTool(SegmentationDisplayTool.toolName);

toolGroup1.setToolEnabled(SegmentationDisplayTool.toolName);

Now

// nothing

Why?

We have eliminated the unnecessary connection between the toolGroup and segmentation display. The segmentation display now automatically appears in the viewport when you add a segmentation representation to it.

---

Viewport-based Representations

In the 2.0 version of Cornerstone3D, we have transitioned from tool group-based segmentation representation rendering to viewport-based ones.

Why? (important enough to not be collapsed)

1. We discovered that tying rendering to a tool group is not an effective approach. In Cornerstone3D 1.x, segmentation rendering was linked to tool groups, which typically consist of multiple viewports. This created complications when users wanted to add segmentations to some viewports but not others within the same tool group. It often necessitated creating an extra tool group for a specific viewport to customize or prevent rendering.

2. We realized this decision was flawed. While it's appropriate for tools to be bound to tool groups, viewport-specific functionalities like segmentation rendering should be the responsibility of individual viewports. Son the second version of our library, we transitioned from tool group-based segmentation representations to viewport-based ones. Now, instead of adding or removing representations to a tool group, users can add them directly to viewports. This change provides much finer control over what each viewport renders. The new approach has proven to be highly effective, and we recognize its significant potential for further improvements.

3. In addition there were numerous methods using the term segment when they actually referred to segmentIndex. Many places used segmentIndex and segment interchangeably. Now, a segment is consistently referred to as a segment, and a segmentIndex is consistently referred to as a segmentIndex.

State

// Add , remove, get
addSegmentationRepresentations(toolGroupId, representationsArray, config?) --> addSegmentationRepresentations(viewportId, representationsArray, config?)
addSegmentationRepresentation(toolGroupId, representation) --> addSegmentationRepresentation(viewportId, representation)
removeSegmentationsFromToolGroup(toolGroupId, representationUIDs) --> removeSegmentationRepresentations(viewportId, representationUIDs)
getSegmentationRepresentations(toolGroupId) --> getSegmentationRepresentations(viewportId)

//
getSegmentationRepresentationByUID(toolGroupId, representationUID) --> getSegmentationRepresentation(representationUID)
findSegmentationRepresentationByUID(repUID) --> getSegmentationRepresentation(representationUID)

As a result of moving from toolGroup to viewports, our segmentation state hierarchy has changed as well.

export type SegmentationState = {
  colorLUT: Types.ColorLUT[];
  segmentations: Segmentation[];
  globalConfig: SegmentationRepresentationConfig;
  toolGroups: {
    [key: string]: {
      segmentationRepresentations: ToolGroupSpecificRepresentations;
      config: SegmentationRepresentationConfig;
    };
  };
};

export type SegmentationState = {
  colorLUT: Types.ColorLUT[];
  segmentations: Segmentation[];
  globalConfig: SegmentationRepresentationConfig;
  representations: {
    [key: string]: SegmentationRepresentation;
  };
  /** viewports association with segmentation representations */
  viewports: {
    [viewportId: string]: {
      [segRepresentationUID: string]: {
        visible: boolean;
        active: boolean;
        segmentsHidden: Set<number>;
      };
    };
  };
};

```

As you see there is a new viewports object that holds the association between the viewports and the segmentation representations

Config

Previously, we had three types of configurations: global, tool group-specific, and segment-specific. Let's examine how each has changed:

Global Config

Remains the same, only change is

renderInactiveSegmentations --> renderInactiveRepresentations

Tool Group Specific and Segment Specific Config

Previously we had

• segmentationRepresentationSpecificConfig which was the config for the representation
• segmentSpecificConfig which was the config for the segments in that representation

Now we have moved to a single config at the root of the representation state level

export type ToolGroupSpecificRepresentationState = {
  /**
   * Segmentation Representation UID
   */
  segmentationRepresentationUID: string;
  /**
   * The segmentationId that this representation is derived from
   */
  segmentationId: string;
  /**
   * The representation type
   */
  type: Enums.SegmentationRepresentations;
  /**
   * Whether the segmentation is the active (manipulatable) segmentation or not
   * which means it is inactive
   */
  active: boolean;
  /**
   * Hidden segment indices in the segmentation
   */
  segmentsHidden: Set<number>;
  /**
   * The index of the colorLUT from the state that this segmentationData is
   * using to render
   */
  colorLUTIndex: number;
  /**
   * Poly Seg generated
   */
  polySeg?: {
    enabled: boolean;
    options?: any;
  };
  // rendering config
  config: LabelmapRenderingConfig;
  // appearance config
  segmentationRepresentationSpecificConfig?: RepresentationConfig;
  segmentSpecificConfig?: SegmentSpecificRepresentationConfig;
};

export type BaseSegmentationRepresentation = {
  /**
   * Segmentation Representation UID
   */
  segmentationRepresentationUID: string;
  /**
   * The segmentationId that this representation is derived from
   */
  segmentationId: string;
  /**
   * The representation type
   */
  type: Enums.SegmentationRepresentations;
  /**
   * The index of the colorLUT from the state that this segmentationData is
   * using to render
   */
  colorLUTIndex: number;
  /**
   * Poly Seg generated
   */
  polySeg?: {
    enabled: boolean;
    options?: any;
  };
  /** rendering config for display of this representation */
  rendering: LabelmapRenderingConfig;
  /** appearance config for display of this representation */
  config: {
    /** default configuration for the representation - applied to all segments*/
    allSegments?: RepresentationConfig;
    /**
     * segment specific configuration for the representation, might be different
     * for each segment. Use cases: to highligh a specific segment with a brighter
     * color
     */
    perSegment?: SegmentRepresentationConfig;
  };
};

Note the segmentationRepresentationSpecificConfig and segmentSpecificConfig have been moved to the config object

and the config has been renamed to rendering to reflect the actual purpose of the object.

Methods

getSegmentationRepresentationSpecificConfig(toolGroupId, segmentationRepresentationUID) --> getSegmentationRepresentationConfig(segmentationRepresentationUID)
setSegmentationRepresentationSpecificConfig(toolGroupId, segmentationRepresentationUID, config) --> setSegmentationRepresentationConfig(segmentationRepresentationUID, config)

and

getSegmentSpecificConfig(toolGroupId, segmentationRepresentationUID, segmentIndex) --> getSegmentIndexConfig(segmentationRepresentationUID, segmentIndex)
setSegmentSpecificConfig(toolGroupId, segmentationRepresentationUID, segmentIndex, config) --> setSegmentIndexConfig(segmentationRepresentationUID, segmentIndex, config)

and we have removed the ToolGroupSpecificConfig both getters and setters

getToolGroupSpecificConfig --> Removed
setToolGroupSpecificConfig --> Removed

---

Active

getActiveSegmentationRepresentation(toolGroupId) -> getActiveSegmentationRepresentation(viewportId)
setActiveSegmentationRepresentation(toolGroupId, representationUID) --> setActiveSegmentationRepresentation(viewportId, representationUID)
getActiveSegmentation(toolGroupId) --> getActiveSegmentation(viewportId)

Other renaming

getSegmentationIdRepresentations(segmentationId) --> getSegmentationRepresentationsForSegmentation(segmentationId)

getToolGroupIdsWithSegmentation(segmentationId) --> getViewportIdsWithSegmentation(segmentationId)

Visibility

setSegmentationVisibility(toolGroupId, representationUID, visibility) --> setSegmentationRepresentationVisibility(viewportId, representationUID, visibility)
getSegmentationVisibility(toolGroupId, representationUId) --> getSegmentationRepresentationVisibility(viewportId, representationUID)
setSegmentsVisibility(toolGroupId, representationUID, segmentIndices, visibility) --> setSegmentIndicesVisibility(viewportId, representationUID, segmentIndices, visibility)

// segments
getSegmentVisibility(toolGroupId, representationUID, segmentIndex) -> getSegmentIndexVisibility(viewportId, representationUID, segmentIndex)
setSegmentVisibility(toolGroupId, representationUID, segmentIndex, visibility) -> setSegmentIndexVisibility(viewportId, representationUID, segmentIndex, visibility)

// Hidden
getSegmentsHidden(toolGroupId, representationUID) --> getHiddenSegmentIndices(viewportId, representationUID)

Why?

Since the visibility should be set on the representation, and segmentation is not the owner of the visibility, a segmentation can have
two representations with different visibility on each viewport

Locking

getLockedSegments -> getLockedSegmentIndices

Color

getColorForSegmentIndex --> getSegmentIndexColor
setColorForSegmentIndex --> setSegmentIndexColor

Why?

Consistency is key, we already had setSegmentVisibility and getSegmentVisibility and many more

Stack Labelmaps

To create a Stack Labelmap, you no longer need to manually create a reference between labelmap imageIds and viewport imageIds. We now handle this process automatically for you.

segmentation.addSegmentations([
  {
    segmentationId,
    representation: {
      type: csToolsEnums.SegmentationRepresentations.Labelmap,
      data: {
        imageIdReferenceMap:
          cornerstoneTools.utilities.segmentation.createImageIdReferenceMap(
            imageIds,
            segmentationImageIds
          ),
      },
    },
  },
]);

segmentation.addSegmentations([
  {
    segmentationId,
    representation: {
      type: csToolsEnums.SegmentationRepresentations.Labelmap,
      data: {
        imageIds: segmentationImageIds,
      },
    },
  },
]);

Why?

This is a long Why ...

The previous model required users to provide an imageIdReferenceMap, which linked labelmap imageIds to viewport imageIds. This approach presented several challenges when implementing advanced segmentation use cases:

1. Manual creation of the map was error-prone, particularly regarding the order of imageIds.

2. Once a segmentation was associated with specific viewport imageIds, rendering it elsewhere became problematic. For example:

   • Rendering a CT image stack segmentation on a single key image.
   • Rendering a CT image stack segmentation on a stack that includes both CT and other images.
   • Rendering a DX dual energy segmentation from energy 1 on energy 2.
   • Rendering a CT labelmap from a stack viewport on a PT labelmap in the same space.

These scenarios highlight the limitations of the previous model.

We've now transitioned to a system where users only need to provide imageIds. During rendering, we match the viewport's current imageId against the labelmap imageIds and render the segmentation if there's a match. This matching process occurs in the SegmentationStateManager, with the criterion being that the segmentation must be in the same plane as the referenced viewport.

This new approach enables numerous additional use cases and offers greater flexibility in segmentation rendering.

---

Events

triggerSegmentationRepresentationModified

triggerSegmentationRepresentationModified now only requires the representationUID

triggerSegmentationRepresentationModified(toolGroupId, representationUID) --> triggerSegmentationRepresentationModified(representationUID)

and it will not publish toolGroupId anymore

triggerSegmentationRepresentationRemoved

triggerSegmentationRepresentationRemoved now only requires the representationUID

triggerSegmentationRepresentationRemoved(toolGroupId, representationUID) --> triggerSegmentationRepresentationRemoved(representationUID)

and it will not publish toolGroupId anymore

triggerSegmentationRender

Before, the function required a toolGroupId, but now it requires an optional viewportId. If you don't provide it, it will render segmentations of all viewports.

triggerSegmentationRender(toolGroupId) --> triggerSegmentationRender(viewportId)

Additionally, there's a new method called triggerSegmentationRenderBySegmentationId which accepts a segmentationId and will render only that specific segmentation:

triggerSegmentationRenderBySegmentationId(segmentationId);

Other renaming

getSegmentAtWorldPoint --> getSegmentIndexAtWorldPoint
getSegmentAtLabelmapBorder --> getSegmentIndexAtLabelmapBorder
getToolGroupIdFromSegmentationRepresentationUID --> removed since it's not needed anymore

Why?

Since it returns an index and not a segment

[netlify]

❌ Deploy Preview for cornerstone-3d-docs failed. Why did it fail? →

Name	Link
🔨 Latest commit	f07f340
🔍 Latest deploy log	https://app.netlify.com/sites/cornerstone-3d-docs/deploys/66e35a9959dfe300089d4122

[wayfarer3130]

Could you leave this as a generic viewport and call setDataIds instead? I think that would be more consistent across viewport types.

[sedghi]

I want the migration to be as minimal as possible. I understand it will be consistent, but perhaps this could be implemented in Cornerstone3D 3.0?

[wayfarer3130]

For the image volume, the access to that data is pretty low level, assuming that the internal organization of the pixel data is exactly a typed array and that there may or may not be slices associated with it. That causes us memory issues as we try to allocate larger buffers, and also forces us to share the image slice data in order to not need to copy the data all the time. If we could start having a better interface on top of the ImageVolume and other image data objects to hide some of the internal details, it would eventually allow us to use a single representation consistently. The VoxelManager is designed to be an abstraction on top of different types of views of both single and multiple images. Given that we don't get new versions of CS3D very often, starting to use the VoxelManager more consistently and perhaps deprecating the use of the full data object would be worthwhile.