Report

Below are the details in the report.

1.Project Detail: This table records the basic information of the current training project.

For example:

Function Description

• Generated Time: The time when the report was generated, corresponding to the project file name.
• Pattern: The mode, including training and inference. Currently, only the training report is printed.
• Model: The trained model, including PCSS, WS-PCSS, SPV-PCSS.
• Optimizer: The optimizer used during training.
• Criterion: The loss function used during training.

2.Dataset

For example:

Function Description

• Grid Size: The voxel grid size is used for sampling the training point cloud.
• Ignored Labels: The label indices to be ignored during training.
• Number of Classes: The number of classes involved in training.
• Train Data Directory: The directory path for the training data files.
• Validation Data Directory: The directory path for the validation data files.

3.Train

For example:

Function Description

• Whether to Use GPU: Whether to use GPU for training.
• GPU Ids: The IDs of the GPUs to be used.
• Batch Size: The batch size used during training.
• Max Train Epoch: The maximum number of training epochs, indicating the number of times the training data is looped during training.
• The Number of Steps per Epoch: The number of steps per epoch in training.
• Whether to Use Pre-trained: Whether to use a pre-trained model.
• Pre-trained Model Path: The file path of the pre-trained model.
• Label Ratio: The ratio of labelled points to all points in the training data. (This information is only displayed when training with the WS-PCSS model).

4.Feature Whether to Use Delta

For example:

Function Description

• Whether to Use Delta z: Whether to use the delta z feature in training.
• Whether to Use Intensity: Whether to use the intensity feature in training.
• Whether to Use RGB: Whether to use the RGB color feature in training.
• Whether to Use Normal: Whether to use the normal vector feature in training.

5.Result

For example:

Function Description

• Completed Training Epochs: The number of epochs completed in the training process for which the report is printed.
• Final Loss: The final value of the loss function.
• The Path of Trained Model: The file path where the trained model is saved.
• The Max Validation Intersection over Union (mIoU): The maximum average intersection over union (IoU) on the validation set.
• Mean Precision: The average precision at the epoch with the maximum IoU.
• Mean Recall: The average recall at the epoch with the maximum IoU.
• F1: The average F1 score at the epoch with the maximum IoU.

6.Training Loss: This graph represents the trend of loss value (loss) during training as the number of steps increases (each epoch contains a certain number of steps). The x-axis represents the step number, and the y-axis represents the loss value. Normally, the loss value decreases as the number of steps increases and eventually stabilizes. If the loss does not exhibit this characteristic, it indicates that there may be issues with the training. Please check the data and consider retraining.

For example:

7.Training Accuracy: This graph represents the trend of prediction accuracy of the sampled point clouds during training as the number of steps increases. The x-axis represents the step number, and the y-axis represents the prediction accuracy. Normally, the accuracy increases as the number of steps increases and eventually stabilizes.

For example:

8.The Mean Intersection over Union (mIoU) of the Validation: This graph represents the trend of mIoU (Mean Intersection over Union) during the evaluation process as the number of epochs increases. The x-axis represents the epoch number, and the y-axis represents the mIoU value. Normally, the mIoU increases gradually and starts to stabilize at a certain epoch. The arrow in the graph points to the epoch with the highest mIoU value and indicates the corresponding epoch and model name in the model folder. If the mIoU does not stabilize, it means that the model still has the potential for further training. In that case, you can continue training by loading the pre-trained model until the mIoU stabilizes.

As shown in the figure, the maximum mIoU is achieved at Epoch 78.

9.The Max Intersection over Union (IoU) and Epoch of each Class: This table indicates the epoch at which the maximum IoU (Intersection over Union) occurs for each class during the validation process, along with other corresponding information. The first row of the table represents the class name, epoch, and mIoU for each corresponding class. The second row in the table indicates that the class "Never Classified" has a maximum IoU of 25.62 at epoch 41 and the average IoU for participating classes is 65.20 followed by the IoUs of other categories. If you are particularly interested in a specific class during the classification task, you can choose the model file corresponding to the epoch with the highest IoU of that class for inference.

As shown in the figure,

10.The Precision and Epoch of each Class: This table indicates the epoch at which the maximum precision occurs for each class during the validation process, along with the precision value for each class. As shown in the figure:

11.The Recall and Epoch of each Class: This table indicates the epoch at which the maximum recall occurs for each class during the validation process, along with the recall value for each class. As shown in the figure:

12.The F1 and Epoch of each Class: This table indicates the epoch at which the maximum F1 score occurs for each class during the validation process, along with the F1 score value for each class. As shown in the figure:

13.The Mean Intersection over Union (mIoU) for each Class: This graph represents the IoU for each class when the maximum mIoU is achieved. The x-axis represents the class, and the y-axis represents the mIoU value. The closer the value is to 100, the more accurate it is. As shown in the figure:

14.Percentage of Training Data for each Class: This graph represents the percentage of each class in the total number of training points. The x-axis represents the class, and the y-axis represents the percentage. Ideally, when all classes are evenly distributed, the training results are the best. Therefore, it is advisable to ensure class balance to improve classification accuracy. As shown in the figure: