We present a new data set for 3d wheat seed reconstruction, propose a challenge “Wheat Seed 3d Reconstruction Challenge”, and provide baseline methods [1].

The dataset consists of 2964 seeds, split into 2520 seeds for training/validation and 444 for testing. Ground truth data for the test set is not provided, however, test results can be evaluated in the “Wheat Seed 3d Reconstruction Challenge” on https://helmholtz-data-challenges.de/.

Per seed there are: (1) Point cloud, reconstructed from 36 images from turntable setup (2) 36 projection matrices, projecting point cloud (2) back to corresponding images from turntable setup (3) Top view taken from another camera

We provide raw (1) and preprocessed (5) versions of point clouds, and preprocessed versions of images from turntable setup (4). Raw images from turntable setup are available in the other data record on https://b2share.fz-juelich.de/ "Deep learning based 3d reconstruction for phenotyping of wheat seeds: dataset (with raw images)" (http://doi.org/10.34730/3541bf71388946b3a3a906fef7aed491).

======================= Detailed content: (1) raw_point_clouds/ -- .ply -- N points, N<70000, x-y-z coordinates, ascii format (2) projection_matrices/ -- .txt -- 36x4x4 floats (3) 2d_station_images/ -- .png -- 700x700, 24 bit, RGB color (4) preprocessed_3d_station_images/ -- .tif -- stabilized and cropped to 373x200, 8 bit, grayscale (5) preprocessed_gt_point_clouds/ --.ply – 2000 points, x-y-z coordinates, ascii format (6) general_files/ -- additional files, files for competition submission

(1) consists of point clouds, one per seed. (5) consists of point clouds in 36 corresponding poses per seed. Raw 3d station images from turntable setup were preprocessed, stabilized and resampled, which results in (4). Raw point clouds (1) were preprocessed (resampled) such that each of them contains 2000 points, lying in the fixed set of directions /general_files/directions.csv. Preprocessed point clouds (5) are convertible to triangular mesh using indices of vertex triplets /general_files/triangles.txt. A sphere sampled with these vertices and triangles is in /general_files/fibo_msh.ply.

The test set does not include point clouds, and has 3 views per seed: 0, 120 and 240 degrees.

Zero-padded integers XXXX are seed indices (here depicted 0000 and 0003). Zero-padded integers YYY in rotation_YYY are rotation angles in degrees: 0, 10, .. 350 degrees.

Indices of train and test sets are provided in general_files/indices_train.txt and general_files/indices_test.txt files. Volumes of raw point clouds of the train set are located in general_files/train_gt_volumes.csv.

Point cloud files .ply (1) and (5) were created with open3d python library, the header of the file is:

format ascii 1.0 element vertex ????? property float x property float y property float z end_header

These .ply are visualizable with, e.g., MeshLab (https://www.meshlab.net/) in windows.

There are two files in this data record: seed_dataset.zip and numpy_arrays.zip. The file seed_dataset.zip contains the data, where each file is presented separately. The file numpy_arrays.zip contains numerical arrays, aggregating these separate files into multidimensional arrays readable with the python library numpy as np.load('file_name.npy').

Structure of the data after extraction of the numpy_arrays.zip into corresponding directories:

numpy_arrays ├── general_files │ ├── directions.csv │ ├── fibo_msh.ply │ ├── indices_test.txt │ ├── indices_train.txt │ ├── train_gt_volumes.csv │ └── triangles.txt ├── 2d_station_images_test.npy ├── 2d_station_images_train.npy ├── gt_points_train.npy ├── preprocessed_3d_station_images_test.npy ├── preprocessed_3d_station_images_train.npy ├── projection_matrices_test.npy └── projection_matrices_train.npy

2d_station_images_test.npy (444, 3, 700, 700) uint8 (seed_idx_test, RGB_channel, height, width) 2d_station_images_train.npy (2520, 3, 700, 700) uint8 (seed_idx_train, RGB_channel, height, width) gt_points_train.npy (2520, 36, 3, 2000) float32 (seed_idx_train, rotation_idx, x-y-z, point_idx) preprocessed_3d_station_images_test.npy (444, 3, 200, 373) uint8 (seed_idx_test, ~rotation_idx~, height, width) preprocessed_3d_station_images_train.npy (2520, 36, 200, 373) uint8 (seed_idx_train, rotation_idx, height, width) projection_matrices_test.npy (444, 36, 4, 4) float32 (seed_idx_test, rotation_idx, row, column) projection_matrices_train.npy (2520, 36, 4, 4) float32 (seed_idx_train, rotation_idx, row, column)

rotataion_idx [0, 1, .., 35] corresponds to [0, 10, .., 350] degrees. ~rotation_idx~~ [0, 1, 2] corresponds to [0, 120, 240] degrees. seed_idx_test and seed_idx_train are in following order: general_files/indices_test.txt and general_files/indices_train.txt respectively.

Structure of the data after extraction of the seed_dataset.zip into corresponding directories:

seed_dataset ├── raw_point_clouds │ └── train │ └── 0000_Surface.ply ├── projection_matrices │ ├── train │ │ └── 0000_ProjectionMatrices.txt │ └── test │ └── 0003_ProjectionMatrices.txt ├── 2d_station_images │ ├── train │ │ └── 0000_2D.png │ └── test │ └── 0003_2D.png ├── preprocessed_3d_station_images │ ├── train │ │ └── 0000 │ │ ├── rotation_000.tif │ │ ├── rotation_010.tif │ │ ├── ... │ │ └── rotation_350.tif │ └── test │ └── 0003 │ ├── rotation_000.tif │ ├── rotation_120.tif │ └── rotation_240.tif ├── preprocessed_gt_point_clouds │ └── train │ ├── 0000 │ ├── rotation_000.ply │ ├── rotation_010.ply │ ├── ... │ └── rotation_350.ply └── general_files ├── directions.csv ├── fibo_msh.ply ├── indices_test.txt ├── indices_train.txt ├── train_gt_volumes.csv └── triangles.txt

It is mandatory to cite our paper when using this dataset: [1] V.Cherepashkin, E.Yildiz, A.Fischbach, L.Kobbelt, and H. Scharr. Deep learning based 3d reconstruction for phenotyping of wheat seeds: a dataset, challenge, and baseline method. CVPPA at ICCV 2023, Paris, 2 October 2023