Abstract

Segmenting complex and hybrid layers, including skin, subcutaneous fat, skeletal muscle, and bone in arm musculoskeletal ultrasound (MSKUS), is extremely crucial for diagnosing and monitoring Breast-Cancer-Related Lymphedema (BCRL). Nevertheless, previous research has primarily focused on individual muscle or bone segmentation in MSKUS, overlooking the horizontal and curvilinear hybrid morphologies that characterize these structures. To address this limitation, we propose a novel hybrid structure-oriented Transformer with structural prior (HSformer+) to promote structural understanding, a framework capable of simultaneously segmenting multilayer hybrid structures. Specifically, HSformer+ first combines hierarchical-consistency relative position encoding to impose structural morphology bias within local windows. Then we propose a structure-biased constraint for attention calculation, innovatively enhancing feature representation. Ultimately, a structure-oriented objective function is introduced to constrain the spatial location and tissue distribution. Our experiments on a public arm MSKUS dataset demonstrate that HSformer+ achieves state-of-the-art segmentation performance. Further experiments on a private arm MSKUS dataset prove that our method has a strong generalization to clinical small-scale MSKUS datasets.

Examples of arm MSKUS with a hybrid and hierarchical structure, including horizontal layers paralleling to the skin surface and curvilinear layers.

Overall Framework

In light of the aforementioned issues and with reference to above figure, we propose a novel hybrid structure-oriented Transformer with structural prior (HSformer+) for arm MSKUS segmentation. This model simultaneously captures effective feature representations for both the correlations and differences of interlayer and intralayer structures from horizontal and vertical perspectives. To address the challenge of curvilinear structures due to the inherent characteristics of the arm MSKUS and the effect of BCRL, hierarchical-consistency relative position encoding (HCPE) imposes structure bias onto the elements within the local window. Meanwhile, to enhance layer differentiation and emphasize critical features, a structure-biased constraint (SBC) is developed to calculate attention weights. Additionally, we design a novel structure-oriented loss based on spatial location and tissue distribution in arm MSKUS images.

Illustration of the proposed HSformer+ framework.

Comparison Experiments on the Public Arm MSKUS Datasets

Considering the significant characteristics of arm MSKUS segmentation, we design three types of comparative experiments to verify the effectiveness of our model from different perspectives:

• We compare HSformer+ with nine representative image segmentation models: in addition to UNet, Attention UNet, UNet++, DeepLabV3, TransUNet, there also AAU-Net, BUSSeg EMCADNet and MADGNet.
• We select four models recently developed for MSKUS images segmentation, such as GSCNN, AMCNet, EnsemSeg and SpineSeg.
• Furthermore, comparisons are made between HSformer+ and four existing algorithms designed for segmentation of layer-structure medical images, involving TCCT, CoherentSeg, RetiFluidNet and Y-Net.

Table I - Comparisons with nine representative image segmentation models. The green area indicates five classic segmentation models, the blue area represents two recent US segmentation architectures, and the orange area represents the latest SOTA methods.

Table II Comparisons with eight MSKUS or layer-structure segmentation models. The blue area displays four representative MSKUS segmentation models, the green area demonstrates four layer-structure segmentation architectures.

Segmentation results predicted by seventeen models to verify the effectiveness of HSformer+ from different perspectives. Yellow, pink, green, blue and red respectively represent the segmentation of gel, skin, fat, muscle and bone layers.