The dataset contains 30 ssTEM (serial section Transmission Electron Microscopy) images taken from the Drosophila larva ventral nerve cord (VNC). The images represent a set of consecutive slices within one 3D volume. Corresponding segmentation ground truths are also provided in this dataset.
Source: https://developer.nvidia.com/blog/accelerating-medical-image-segmentation-tensor-cores-tensorflow-2/
Downloading
Download data from
https://archive.org/details/academictorrents_42714f859770f1a9d8b27985f9f16ea17e8ba2f6
It wasn't possible to download from the EM segmentation challenge dataset
or from
https://academictorrents.com/details/42714f859770f1a9d8b27985f9f16ea17e8ba2f6
Dataset details
Training and test data are comprised of three 512x512x30 TIF volumes (test-volume.tif, train-volume.tif and train-labels.tif). Files test-volume.tif and train-volume.tif contain grayscale 2D slices to be segmented. Additionally, training masks are provided in train-labels.tif as a 512x512x30 TIF volume.
The term "fold" could be used to refer to a specific partition or grouping of the training data. Each fold would contain a subset of the training data.
Additional details:
train-volume.tif (7.5 MB) |Original training image, 8-bit grayscale, 512x512x30 pixels|
train-labels.tif (7.5 MB)| Training image labels (0 - membranes, 255 - non-membranes), 8-bit grayscale, 512x512x30 pixels
test-volume.tif (7.5 MB) |Test image, 8-bit grayscale, 512x512x30 pixels The training and test datasets are two stacks of 30 sections from a serial section Transmission Electron Microscopy (ssTEM) data set of the Drosophila first instar larva ventral nerve cord (VNC). The microcube measures 2 x 2 x 1.5 microns approx., with a resolution of 4x4x50 nm/pixel.
Preprocessing the data
The UNet script main.py operates on data from the ISBI Challenge, the dataset originally employed in the UNet paper.
Preprocessing the data with
python main.py --data_dir /data
the sample output looks as follows
...
DLL 2023-06-12 06:03:29.213088 - (700, 1000) train_ce_loss : 0.284275084733963 train_dice_loss : 0.1452435553073883 train_total_loss : 0.4295186400413513
DLL 2023-06-12 06:21:17.149753 - (800, 1000) train_ce_loss : 0.20779138803482056 train_dice_loss : 0.11122724413871765 train_total_loss : 0.3190186321735382
DLL 2023-06-12 06:39:02.915866 - (900, 1000) train_ce_loss : 0.19838161766529083 train_dice_loss : 0.0974794328212738 train_total_loss : 0.29586106538772583
DLL 2023-06-12 06:57:04.944396 - (1000, 1000) train_ce_loss : 0.19091825187206268 train_dice_loss : 0.09579619765281677 train_total_loss : 0.28671443462371826
No fold specified for evaluation. Please use --fold [int] to select a fold.
TroubleShooting
[1].For the error
"File "/workspace/unet/model/tf_trt.py", line 51, in TFTRTModel
@tf.function(jit_compile=False)
TypeError: function() got an unexpected keyword argument 'jit_compile' "
You can try setting the following
@tf.function(experimental_compile=True)[2]. For the error, "tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
I had done
ln -s libcudart.so.11.0 libcudart.so.10.1
and set LIB PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.2/targets/x86_64-linux/lib
Additional Reading
U-Net: Convolutional Networks for Biomedical Image Segmentation, Olaf Ronneberger, Philipp Fischer, and Thomas Brox, 2015, arXiv:1505.04597v1 [cs.CV]
Horovod Trouble Shooting - https://horovod.readthedocs.io/en/stable/troubleshooting_include.html