English | 简体中文
- Introduction
- News
- EISeg-Med3D
- Performance
- Quick Start
- Full Tutorial
- Train On Your Dataset
- Structure
- TODO
- License
- Acknowledgement
Medical image segmentation is a pixel-wise/voxel-wise classification of images generated by medical imaging, so that different organs or tissues can be distinguished. It is widely used in medical diagnosis and treatment planning.
Medical image segmentation can be divided into 2D medical image segmentation and 3D medical image segmentation. 2D medical image segmentation is supported in PaddleSeg. For details, please see Fundus Data Segmentation Instructions, while 3D image segmentation is handled by MedicalSeg.
MedicalSeg is an easy-to-use 3D medical image segmentation solution supporting the whole segmentation process including data preprocessing, model training, and model deployment.
The MedicalSeg panorama is as follows, and its main features include:
- Contains the APIs of the whole process medical image segmentation process from data labeling, training, to deployment.
- Including the 3D medical image annotation platform EISeg-Med3D to achieve efficient, accurate and easy-to-use labeling.
- Support six cutting-edge models nnUNet, nnFormer, SwinUNet, TransUNet, UNETR, VNet and corresponding high-precision pre-training models.
.gif)
VNet segmentation result on COVID-19 CT scans (mDice on evalset is 97.04%) & MRISpineSeg (16 class mDice on evalset is 89.14%)
MedicalSeg Whole View
- [2022-9] Release MedicalSegV2. It provides 3 cutting-edge models to support whole process deployment applications, including nnformer, TransUnet and nnUnet, allowing you to experience a stronger and more accurate segmentation effect; a new 3D medical image intelligent annotation platform EISeg-Med3D to quickly and easily achieve accurate 3D medical image annotation.
- [2022-4] Release MedicalSegV1. It provides the whole process from data preprocessing in 3D medical image segmentation to training and deployment, including native support for five datasets, and high-precision preprocessing on vertebrae and lungs Train the model.
MedicalSeg just released! If you find any problem using it or want to share any future develop suggestions, please open a github issue or join us by scanning the following QR code.
In order to solve the problem of low efficiency of 3D medical manual annotation, and to truly empower medical care with AI starting from data annotation, we built EISeg-Med3D, a user-friendly, efficient and intelligent 3D medical image annotation platform, which realizes intelligent and efficient 3D medical data annotation by integrating 3D interactive segmentation models in the annotation process. The main features are as follows:
-
Efficient：Each category only needs a few clicks to generate 3d segmentation results, ten times efficient compares to time-consuming and laborious manual annotation.
-
Accurate：The mIOU can reach 0.85 with only 3 clicks. with the segmentation editor equipped with machine learning algorithm and manual annotation, 100% accuracy is right on your hand.
-
Convenient：Install our plugin within three steps; labeling results and progress are automatically saved; the transparency of labeling results can be adjusted to improve labeling accuracy; user-friendly interface interaction makes labeling worry-free and hassle-free。
The detailed doc of using EISeg-Med3D is here
We have added cutting-edge models including nnUNet, nnFormer, SwinUNet, and TransUNet, and all of them surpassed the original paper in terms of segmentation accuracy to varying degrees. Among them, the accuracy of the reproduced TransUNet exceeded the original paper by 3.6%. 81.8% mDice segmentation accuracy.
Below we show our existing models, pre-trained model parameters and accuracy in the form of a table, welcome to download the log to view the results or load the pre-trained model to improve the training effect on the relevant data set :).
| Model | Segment | Dataset | mDice | README | Links |
|---|---|---|---|---|---|
| nnFormer | Heart | ACDC | 91.8% | README | model | log| vdl |
| Vnet | Lung | COVID-19 CT scans | 97.0% | README | model | log | vdl |
| nnUNet | Lung Tumor | MSD-Lung | 67.9% | README | model | log |
| Vnet | Spine | MRISpineSeg | 74.7% | README | model | log | vdl |
| UNETR | Brain Tumor | MSD-brain | 71.8% | README | model | log | vdl |
| SwinUNet | Multi-orgrans | Synapse | 82.1% | README | model | log | vdl |
| TransUNet | Multi-orgrans | Synapse | 81.1% | README | model | log | vdl |
We add GPU acceleration in data preprocess using CuPy. Compared with preprocess data on CPU, acceleration enable us to use about 40% less time in data prepeocessing. The following shows the time we spend in process COVID-19 CT scans.
| Device | Time(s) |
|---|---|
| CPU | 50.7 |
| GPU | 31.4( ↓ 38%) |
This part introduce a easy to use the demo on COVID-19 CT scans dataset. This demo is available on our Aistudio project as well. Detailed steps on training and add your own dataset can refer to this tutorial.
-
Download our repository.
git clone https://github.com/PaddlePaddle/PaddleSeg.git cd contrib/MedicalSeg/ -
Install requirements:
pip install -r requirements.txt -
(Optional) Install CuPY if you want to accelerate the preprocess process. CuPY installation guide
-
Get and preprocess the data. Remember to replace prepare_lung_coronavirus.py with different python script that you need here:
- change the GPU setting here to True if you installed CuPY and want to use GPU to accelerate.
python tools/prepare_lung_coronavirus.py -
Run the train and validation example. (Refer to the tutorial for details.)
sh run-vnet.sh
This part shows the details on how to use our repository from setting configurations to deploy.
Change configuration about loss, optimizer, dataset, and so on here. Our configurations is organized as follows:
├── _base_ # base config, set your data path here and make sure you have enough space under this path.
│ └── global_configs.yml
├── lung_coronavirus # each dataset has one config directory.
│ ├── lung_coronavirus.yml # all the config besides model is here, you can change configs about loss, optimizer, dataset, and so on.
│ ├── README.md
│ └── vnet_lung_coronavirus_128_128_128_15k.yml # model related config is here
└── schedulers # the two stage scheduler, we have not use this part yet
└── two_stage_coarseseg_fineseg.ymlWe use the data preparation script to download, preprocess, convert, and split the data automatically. If you want to prepare the data as we did, you can run the data prepare file like the following:
python tools/prepare_lung_coronavirus.py # take the CONVID-19 CT scans as example.
After changing your config, you are ready to train your model. A basic training and validation example is run-vnet.sh. Let's see some of the training and validation configurations in this file.
# set your GPU ID here
export CUDA_VISIBLE_DEVICES=0
# set the config file name and save directory here
yml=vnet_lung_coronavirus_128_128_128_15k
save_dir=saved_model/${yml}
mkdir save_dir
# Train the model: see the train.py for detailed explanation on script args
python3 train.py --config configs/lung_coronavirus/${yml}.yml \
--save_dir $save_dir \
--save_interval 500 --log_iters 100 \
--num_workers 6 --do_eval --use_vdl \
--keep_checkpoint_max 5 --seed 0 >> $save_dir/train.log
# Validate the model: see the val.py for detailed explanation on script args
python3 val.py --config configs/lung_coronavirus/${yml}.yml \
--save_dir $save_dir/best_model --model_path $save_dir/best_model/model.pdparams
With a trained model, we support deploying it with paddle inference to boost the inference speed. The instruction to do so is as follows, and you can see a detailed python deployment tutorial here and a cpp deployment tutorial here.
cd MedicalSeg/
# Export the model with trained parameter
python export.py --config configs/lung_coronavirus/vnet_lung_coronavirus_128_128_128_15k.yml --model_path /path/to/your/trained/model
# Infer it with Paddle Inference Python API
python deploy/python/infer.py \
--config /path/to/model/deploy.yaml \
--image_path /path/to/image/path/or/dir/
--benchmark True # Use it after installed AutoLog, to record the speed, see ../deploy/python/README.md for detail to install AutoLog.
If you see the "finish" output, you have sucessfully upgrade your model's infer speed.
If you want to train on your dataset, simply add a dataset file, a data preprocess file, a configuration directory, a training script and you are good to go. Details on how to add can refer to the links above.
As we mentioned, every dataset has its own configuration directory. If you want to add a new dataset, you can replicate the lung_coronavirus directory and change relevant names and configs.
├── _base_
│ └── global_configs.yml
├── lung_coronavirus
│ ├── lung_coronavirus.yml
│ ├── README.md
│ └── vnet_lung_coronavirus_128_128_128_15k.yml
Your data needs to be convert into numpy array and split into trainset and valset as our format. You can refer to the prepare script:
├── lung_coronavirus_phase0 # the preprocessed file
│ ├── images
│ │ ├── imagexx.npy
│ │ ├── ...
│ ├── labels
│ │ ├── labelxx.npy
│ │ ├── ...
│ ├── train_list.txt # put all train data names here, each line contains: /path/to/img_name_xxx.npy /path/to/label_names_xxx.npy
│ └── val_list.txt # put all val data names here, each line contains: img_name_xxx.npy label_names_xxx.npyOur dataset file inherits MedicalDataset base class, where data split is based on the train_list.txt and val_list.txt you generated from previous step. For more details, please refer to the dataset script.
The run script is used to automate a series of process. To add your config file, just replicate the run-vnet.sh and change it based on your thought. Here is the content of what they mean:
# set your GPU ID here
export CUDA_VISIBLE_DEVICES=0
# set the config file name and save directory here
yml=lung_coronavirus/vnet_lung_coronavirus_128_128_128_15k # relative path to your yml from config dir
config_name = vnet_lung_coronavirus_128_128_128_15k # name of the config yml
save_dir_all=saved_model # overall save dir
save_dir=saved_model/${config_name} # savedir of this exp
This part shows you the whole picture of our repository, which is easy to expand with different model and datasets. Our file tree is as follows:
├── configs # All configuration stays here. If you use our model, you only need to change this and run-vnet.sh.
├── data # Data stays here.
├── deploy # deploy related doc and script.
├── medicalseg
│ ├── core # the core training, val and test file.
│ ├── datasets
│ ├── models
│ ├── transforms # the online data transforms
│ └── utils # all kinds of utility files
├── export.py
├── run-vnet.sh # the script to reproduce our project, including training, validate, infer and deploy
├── tools # Data preprocess including fetch data, process it and split into training and validation set
├── train.py
├── val.py
└── visualize.ipynb # You can try to visualize the result use this file.
We have several thoughts in mind about what should our repo focus on. Your contribution will be very much welcomed.
- [✔️] Add PP-nnunet with acceleration in preprocess, automatic configuration for all dataset and better performance compared to nnunet.
- [✔️] Add EISeg-Med3D, a AI-assisted annotation platform.
- Add pretrain model on various dataset.
License of MedicalSeg is Apache 2.0 license.
- Many thanks to Lin Han, Lang Du, onecatcn for their contribution in our repository
- Many thanks to itkwidgets for their powerful visualization toolkit that we used to present our visualizations.
- Many thans to Idea icons created by Vectors Market - Flaticon for their beautiful icons.