We present a new data set for 3d wheat seed reconstruction, propose a challenge “Wheat Seed 3d Reconstruction Challenge”, and provide baseline methods [1].

The dataset consists of 2964 seeds, split into 2520 seeds for training/validation and 444 for testing. Ground truth data for the test set is not provided, however, test results can be evaluated in the “Wheat Seed 3d Reconstruction Challenge” on https://helmholtz-data-challenges.de/.

Per seed there are: (0) 36 images captured with a 10-degree increment in the rotation in front of the camera, ensuring a complete 360-degree view (1) Point cloud, reconstructed from these images with volume carving (2) 36 projection matrices, projecting point cloud (2) back to corresponding images (1) (3) Top view taken from another camera

We provide preprocessed versions of (1) and (2): (5) and (6) respectively.

Detailed content: (0) raw_3d_station_images/ -- .tif -- 1800x1000, 8 bit grayscale (1) raw_point_clouds/ -- .ply -- N points, N<70000, x-y-z coordinates, ascii format (2) projection_matrices/ -- .txt -- 36x4x4 floats (3) 2d_station_images/ -- .png -- 700x700, 24 bit, RGB color (4) preprocessed_3d_station_images/ -- .tif -- stabelized and cropped to 373x200, 8 bit, grayscale (5) preprocessed_gt_point_clouds/ --.ply – 2000 points, x-y-z coordinates, ascii format (6) general_files/ -- additional files, files for competition submission

(1) consists of point clouds, one per seed. (5) consists of point clouds in 36 corresponding poses per seed. Raw 3d station images (0) were preprocessed, stabilized and resampled, which results in (4). Raw point clouds (1) were preprocessed (resampled) such that each of them contains 2000 points, lying in the fixed set of directions /general_files/directions.csv. Preprocessed point clouds (5) are convertible to triangular mesh using indices of vertex triplets /general_files/triangles.txt. A sphere sampled with these vertices and triangles is in /general_files/fibo_msh.ply.

The test set does not include point clouds, and has 3 views per seed: 0, 120 and 240 degrees.

Zero-padded integers XXXX are seed indices (here depicted 0000 and 0003). Zero-padded integers YYY in rotation_YYY are rotation angles in degrees: 0, 10, .. 350 degrees.

Indices of train and test sets are provided in general_files/indices_train.txt and general_files/indices_test.txt files. Volumes of raw point clouds of the train set are located in general_files/train_gt_volumes.csv.

Point cloud files .ply (1) and (5) were created with open3d python library, the header of the file is:

format ascii 1.0 element vertex ????? property float x property float y property float z end_header

These .ply are visualizable with, e.g., MeshLab (https://www.meshlab.net/) in windows.

There are 16 files in this data record: seed_dataset.zip, numpy_arrays.zip, raw_3d_station_images_train_part_X.zip with X in [0, 1, .., 12], raw_3d_station_images_test.zip

The file seed_dataset.zip contains the data, where each file is presented separately. The file numpy_arrays.zip contains numerical arrays, aggregating these separate files into multidimensional arrays readable with the python library numpy as np.load('file_name.npy'). raw_3d_station_images_*.zip contain images from turntable setup (0).

Structure of the data after extraction of the numpy_arrays.zip into corresponding directories:

numpy_arrays ├── general_files │ ├── directions.csv │ ├── fibo_msh.ply │ ├── indices_test.txt │ ├── indices_train.txt │ ├── train_gt_volumes.csv │ └── triangles.txt ├── 2d_station_images_test.npy ├── 2d_station_images_train.npy ├── gt_points_train.npy ├── preprocessed_3d_station_images_test.npy ├── preprocessed_3d_station_images_train.npy ├── projection_matrices_test.npy └── projection_matrices_train.npy

2d_station_images_test.npy (444, 3, 700, 700) uint8 (seed_idx_test, RGB_channel, height, width) 2d_station_images_train.npy (2520, 3, 700, 700) uint8 (seed_idx_train, RGB_channel, height, width) gt_points_train.npy (2520, 36, 3, 2000) float32 (seed_idx_train, rotation_idx, x-y-z, point_idx) preprocessed_3d_station_images_test.npy (444, 3, 200, 373) uint8 (seed_idx_test, ~rotation_idx~, height, width) preprocessed_3d_station_images_train.npy (2520, 36, 200, 373) uint8 (seed_idx_train, rotation_idx, height, width) projection_matrices_test.npy (444, 36, 4, 4) float32 (seed_idx_test, rotation_idx, row, column) projection_matrices_train.npy (2520, 36, 4, 4) float32 (seed_idx_train, rotation_idx, row, column)

rotataion_idx [0, 1, .., 35] corresponds to [0, 10, .., 350] degrees. ~rotation_idx~~ [0, 1, 2] corresponds to [0, 120, 240] degrees. seed_idx_test and seed_idx_train are in following order: general_files/indices_test.txt and general_files/indices_train.txt respectively.

Structure of the data after extraction of the seed_dataset.zip and raw_3d_station_images_*.zip into corresponding directories:

seed_dataset ├── raw_3d_station_images │ ├── train │ │ └── 0000 │ │ ├── rotation_000.tif │ │ ├── rotation_010.tif │ │ ├── ... │ │ └── rotation_350.tif │ └── test │ └── 0003 │ ├── rotation_000.tif │ ├── rotation_120.tif │ └── rotation_240.tif ├── raw_point_clouds │ └── train │ └── 0000_Surface.ply ├── projection_matrices │ ├── train │ │ └── 0000_ProjectionMatrices.txt │ └── test │ └── 0003_ProjectionMatrices.txt ├── 2d_station_images │ ├── train │ │ └── 0000_2D.png │ └── test │ └── 0003_2D.png ├── preprocessed_3d_station_images │ ├── train │ │ └── 0000 │ │ ├── rotation_000.tif │ │ ├── rotation_010.tif │ │ ├── ... │ │ └── rotation_350.tif │ └── test │ └── 0003 │ ├── rotation_000.tif │ ├── rotation_120.tif │ └── rotation_240.tif ├── preprocessed_gt_point_clouds │ └── train │ ├── 0000 │ ├── rotation_000.ply │ ├── rotation_010.ply │ ├── ... │ └── rotation_350.ply └── general_files ├── directions.csv ├── fibo_msh.ply ├── indices_test.txt ├── indices_train.txt ├── train_gt_volumes.csv └── triangles.txt

It is mandatory to cite our paper when using this dataset: [1] V.Cherepashkin, E.Yildiz, A.Fischbach, L.Kobbelt, and H. Scharr. Deep learning based 3d reconstruction for phenotyping of wheat seeds: a dataset, challenge, and baseline method. CVPPA at ICCV 2023, Paris, 2 October 2023