Classify by Custom Deep Learning

​ This function uses a deep learning segmentation model to classify point cloud data. This function uses supervised classification, in the same batch of data, need to manually edit a small number of data categories, after training the model to batch process a large number of data. Two processes are supported: select and generate training models, processing data to be classified; Use the existing models to process the data to be classified.

Requirement

Usage

Click Classification >Custom Deep Learning Classification

​ Usage Procedure: Data preparation -->Training point cloud classification model -->Use training model for classification

​ Please note that when you enablethe back-end for the first time.

The following is the task management page, which can display the status/management of training point cloud classification model tasks and use the training model to classify tasks.

​ Tool bar from left to right Create a task,delete a task, start a task, pause a task, export a task, task information,import task,model management,setting,refresh

​ The task management page, it can display state/manage training point cloud classification model tasks and use training model classification tasks. Note the connection to the back end should be set at the first startup.

Data Preparation

The data required for training (labeled data) should be prepared before training, and you can use the profile editing function provided by lidar360 to edit the category data. ​ The training data should be real scene data, and data in las, lidata and laz formats are allowed. The amount of training data is adjusted according to different scenarios and algorithm processing methods. In principle, the more annotated data involved in the training, the better. We recommend that at least 100m*100m data be prepared for use to obtain a good use experience.Please note that data annotation must be consistent with the principle, and the setting of categories should not be changed according to the scene. Mislabeled data can have a negative impact.

Training point cloud classification model

​ Click Classification > Classify by Deep Learning-> Training point cloud classification model

After filling in the data path as required, the user clicksdefault使用默认参数即可开始使用

• Task Name: The task name will be used as the trained model name.
• Training Data: If the data marked by categories is placed in the same folder, all data in the supported formats of the folder will be read.
• Validation Data: Put the data marked by categories in the same folder. The data in this folder will be used as indicators such as the accuracy of sample calculation.
• Preprocess Transform:
  • Split Function: Data preprocessing, you can select data preprocessing function here, usually include statistical filtering, voxel filtering, etc.
• Split Transform:
  • Split Function: Data segmentation, in order to avoid the use of too much memory, users can segment the data to the appropriate size, the size is related to data distribution, algorithm and so on. We will continue to describe how to choose the appropriate size later.
• Select Class: Category mapping, where one or more source classes correspond to target classes, can save a lot of work when adjusting algorithm classification categories and correcting category definitions.
• Model:
  • Model List: The model temporarily provides three classical models: PVCNN, KPConv and MinkUNet (not the original implementation, with a small number of changes). For these three algorithms, we have provided corresponding recommended parameters settings. You can use the Default button in the lower left corner to automatically fill in subsequent parameters.
• Training:
  • epochs: Iteration period. Each complete loop that processing all data is an epochs.
  • batch size: For each batch of data, which means the model processing several data. If memory allows, larger data has more advantages in model accuracy and training speed. For GPU with dedicated RAM of 8GB, please use the default batch size 2. If you find that there’s still lots of GPU memory available during training, you can safely increase the batch size to process more blocks at once.
  • lr: Learning rate cooperate with the optimizer control the speed and direction of the model learning (generally, the lower the learning rate, the more stable and easier to fall into the saddle point).
  • Optimizer: Optimizer, as above.
  • lr scheduler: Learning rate controller, the learning rate varies with the period to obtain better accuracy in some cases.
  • loss: The default loss function is cross entropy CrossEntropyLoss. FocalLoss generally has a better effect when the categories are not balanced, and may have worse effect when the label is wrong. So using CrossEntropyLoss is better.

The preceding bold parameters are mandatory. Other parameters are optional.

Use training model for classification

​Click Classification >Classification by Deep Learning >Use training model for classification

• Model: Select the Model to use. The name of the Model will be named after the training task name. The model will be automatically added into the Model management after training.

Note: The model will be generated when the training task is completely stopped (the training task is completed, the training task is paused, and the training task exits when the memory exceeds).

Evaluate point cloud training results

Three indicators MIoU, Acc and MAcc are provided, which are mainly calculated based on confusion matrix

The True value is positive, the number that the model considers to be positive (True Positive=TP). The True value is positive, the number that the model considers to be negative (False Negative=FN) The true value is negative, the number that the model considers to be positive (False Positive=FP) The true value is negative, the number that the model considers to be negative (True Negative=TN)

Such a table of four indicators becomes the confusion matrix

IoU (Intersection over Union)

The intersection of goals and predictions/the union of goals and predictions

MIoU(Mean Intersection over Union) MIoU

Acc(Accuracy)

In general, we only need to focus on the miou metric, which is the average intersection ratio

Set back-end

​ Click the following buttons in turn. Note that the third button will not be clicked when the back-end is not connected.

Model management

The model management interface manages available models and imports external models (lidar360 client output models only).

Import project

External models can be imported to obtain parameters/weights.

​- Model File: External model file path (only supported by Lidar360 client output) ​- Model Weight: indicates the weight of the model. If it is checked, training continues. If it is not checked, only model training parameters are obtained

Deep learning model comparison

The following data are for the training stage. Use the voxel size of 0.5 for voxel subsampling, with a single data point of 20w and a batch of 2 statistics. The time complexity and space complexity are mainly affected by the degree of data density. It is difficult to control the point density after multiple subsampling of the model, and can only provide a general value for reference.

Hardware Configuration

​ cpu:i7-10700k(8t16c)

​ gpu:RTX3060

KPConv generally has the highest level of precision and detail, and is recommended for small scenarios that require fine classification. The MinkUNet has a larger field of view with a decent speed and is recommended for large scenes. PVCNN has great detail performance and fastest speed, and is generally recommended.

Feature selection

rgb and intensity features can be added according to the specific data The rgb and intensity features can make the model fit earlier, with little difference in final accuracy (depending on the actual data), and negligible difference in memory consumption and computing speed.

Debugging guide

• Insufficient memory: Reduce the batch_size parameter and split transform to the split size.
• Category imbalance: Recommend that loss change to FocalLoss.
• Loss jitter back and forth: At the beginning of training, you can wait for more rounds. If in 10-20 epoch it is not stable, can try to make lr / 10 smaller. It is recommended to you to use CosineAnnealingWarmRestarts for lr_scheduler. If necessary, the large lr training can be used in the early stage, and the use of import model training function in the later stage can change to the small lr training.

​

Note

1. The validation set uses data enhancement from the training set by default

2. If users didn’t specify the input feature, we will input the basic feature as required by the model