[Enhancement] Update user api and docs with tools/xxx (#2603)

Author: juncaipeng

configs/pp_liteseg/README.md

@@ -54,7 +54,7 @@ export model=pp_liteseg_stdc1_cityscapes_1024x512_scale0.5_160k     # test resol
 # export model=pp_liteseg_stdc2_cityscapes_1024x512_scale1.0_160k
 # export model=pp_liteseg_stdc1_camvid_960x720_10k
 # export model=pp_liteseg_stdc2_camvid_960x720_10k
-python -m paddle.distributed.launch train.py \
+python -m paddle.distributed.launch tools/train.py \
     --config configs/pp_liteseg/${model}.yml \
     --save_dir output/${model} \
     --save_interval 1000 \
@@ -77,7 +77,7 @@ Refer to [doc](../../docs/evaluation/evaluate/evaluate.md) for the detailed usag
 export CUDA_VISIBLE_DEVICES=0
 export model=pp_liteseg_stdc1_cityscapes_1024x512_scale0.5_160k
 # export other model
-python val.py \
+python tools/val.py \
     --config configs/pp_liteseg/${model}.yml \
     --model_path output/${model}/best_model/model.pdparams \
     --num_workers 3

configs/pssl/README.md

@@ -36,7 +36,6 @@ Make sure that the datasets have structures as follows:
 
 ```
 PaddleSeg
-│   train.py
 │   ...  
 │
 └───data
@@ -72,7 +71,7 @@ Having installed PaddlePaddle and PaddleSeg and prepared datasets (ImageNet and
 export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
 
 path_save="work_dirs_stdc2_pssl"
-python -m paddle.distributed.launch --log_dir $path_save train.py \
+python -m paddle.distributed.launch --log_dir $path_save tools/train.py \
        --config configs/pssl/stdc2_seg_pssl.yml \
        --log_iters 200 \
        --num_workers 12 \
@@ -87,7 +86,7 @@ python -m paddle.distributed.launch --log_dir $path_save train.py \
 export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
 
 path_save="work_dirs_pp_liteseg_stdc2_pssl"
-python -m paddle.distributed.launch --log_dir $path_save train.py \
+python -m paddle.distributed.launch --log_dir $path_save tools/train.py \
        --config configs/pssl/pp_liteseg_stdc2_pssl.yml \
        --log_iters 100 \
        --num_workers 12 \

configs/smrt/README.md

@@ -119,7 +119,7 @@ export CUDA_VISIBLE_DEVICES=0 # Linux下设置1张可用的卡
 
 cd PaddleSeg
 
-python train.py \
+python tools/train.py \
        --config configs/smrt/pp_liteseg_stdc2.yml \
        --do_eval \
        --use_vdl \
@@ -142,7 +142,7 @@ python train.py \
 ```
 export CUDA_VISIBLE_DEVICES=0,1,2,3 # 设置4张可用的卡
 
-python -m paddle.distributed.launch train.py \
+python -m paddle.distributed.launch tools/train.py \
        --config configs/smrt/pp_liteseg_stdc2.yml \
        --do_eval \
        --use_vdl \
@@ -155,7 +155,7 @@ python -m paddle.distributed.launch train.py \
 训练得到精度符合预期的模型后，可以导出预测模型，进行部署。详细的模型导出方法请参考[文档](../../docs/model_export_cn.md)。
 
 ```
-python export.py \
+python tools/export.py \
        --config configs/smrt/pp_liteseg_stdc2.yml \
        --model_path output/pp_liteseg_stdc2/best_model/model.pdparams \
        --save_dir output/pp_liteseg_stdc2/infer_models

contrib/CityscapesSOTA/README.md

@@ -47,7 +47,7 @@ Firstly please download 3 files from [Cityscapes dataset](https://www.cityscapes
 Run the following commands to do the label conversion:
 ```shell
 pip install cityscapesscripts
-python ../../tools/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
+python ../../tools/data/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
 ```
 Where 'cityscapes_path' should be adjusted according to the actual dataset path. 'num_workers' determines the number of processes started and the size can be adjusted according to the actual situation.
 

contrib/PP-HumanSeg/README.md

@@ -354,7 +354,7 @@ Run the following command to start finetuning. You should change the details, su
 ```bash
 export CUDA_VISIBLE_DEVICES=0 # Set GPU on Linux
 # set CUDA_VISIBLE_DEVICES=0  # Set GPU on Windows
-python ../../train.py \
+python ../../tools/train.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --save_dir output/human_pp_humansegv2_lite \
   --save_interval 100 --do_eval --use_vdl
@@ -365,7 +365,7 @@ python ../../train.py \
 Load model and trained weights and start model evaluation. The full usage of model evaluation in [url](../../docs/evaluation/evaluate/evaluate.md).
 
 ```bash
-python ../../val.py \
+python ../../tools/val.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams
 ```
@@ -375,7 +375,7 @@ python ../../val.py \
 Load model and trained weights and start model prediction. The result are saved in `./data/images_result/added_prediction` and `./data/images_result/pseudo_color_prediction`
 
 ```bash
-python ../../predict.py \
+python ../../tools/predict.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams \
   --image_path data/images/human.jpg \
@@ -387,7 +387,7 @@ python ../../predict.py \
 Load model and trained weights and export inference model. The full usage of model exporting in [url](../../docs/model_export.md).
 
 ```shell
-python ../../export.py \
+python ../../tools/export.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams \
   --save_dir output/human_pp_humansegv2_lite \

contrib/PP-HumanSeg/README_cn.md

@@ -349,7 +349,7 @@ configs
 ```bash
 export CUDA_VISIBLE_DEVICES=0 # Linux下设置1张可用的卡
 # set CUDA_VISIBLE_DEVICES=0  # Windows下设置1张可用的卡
-python ../../train.py \
+python ../../tools/train.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --save_dir output/human_pp_humansegv2_lite \
   --save_interval 100 --do_eval --use_vdl
@@ -360,7 +360,7 @@ python ../../train.py \
 执行如下命令，加载模型和训练好的权重，进行模型评估，输出验证集上的评估精度。模型评估的详细文档，请参考[链接](../../docs/evaluation/evaluate/evaluate_cn.md)。
 
 ```bash
-python ../../val.py \
+python ../../tools/val.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams
 ```
@@ -370,7 +370,7 @@ python ../../val.py \
 执行如下命令，加载模型和训练好的权重，对单张图像进行预测，预测结果保存在`./data/images_result`目录下的`added_prediction`和`pseudo_color_prediction`文件夹中。
 
 ```bash
-python ../../predict.py \
+python ../../tools/predict.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams \
   --image_path data/images/human.jpg \
@@ -382,7 +382,7 @@ python ../../predict.py \
 执行如下命令，加载模型和训练好的权重，导出预测模型。模型导出的详细文档，请参考[链接](../../docs/model_export_cn.md)。
 
 ```shell
-python ../../export.py \
+python ../../tools/export.py \
   --config configs/human_pp_humansegv2_lite.yml \
   --model_path pretrained_models/human_pp_humansegv2_lite_192x192_pretrained/model.pdparams \
   --save_dir output/human_pp_humansegv2_lite \

contrib/PaddleLabel/doc/CN/training/PdLabel_PdSeg.md

@@ -125,7 +125,7 @@ export CUDA_VISIBLE_DEVICES=0
 # --config 参数表示指定使用哪个配置文件
 # --do_eval 参数表示一遍训练一遍验证
 # --save_interval 参数表示每经过100个iters，进行一个模型的保存
-python PaddleSeg/train.py \
+python PaddleSeg/tools/train.py \
        --config PaddleSeg/configs/mynet.yml \
        --do_eval \
        --use_vdl \

docs/apis/datasets.md

@@ -74,7 +74,7 @@ dataset = Dataset(transforms = transforms,
 > CLASS paddleseg.datasets.PascalVOC(transforms, dataset_root=None, mode='train', edge=False)
 
     PascalVOC2012 dataset `http://host.robots.ox.ac.uk/pascal/VOC/`.
-    If you want to augment the dataset, please run the voc_augment.py in tools.
+    If you want to augment the dataset, please run the voc_augment.py in tools/data.
 
 > > Args
 > > > - **transforms** (list): Transforms for image.

docs/apis/datasets/datasets.md

@@ -78,7 +78,7 @@ class paddleseg.datasets.Cityscapes(transforms, dataset_root, mode='train', edge
 class paddleseg.datasets.PascalVOC(transforms, dataset_root=None, mode='train', edge=False)
 ```
     PascalVOC2012 dataset `http://host.robots.ox.ac.uk/pascal/VOC/`.
-    If you want to augment the dataset, please run the voc_augment.py in tools.
+    If you want to augment the dataset, please run the voc_augment.py in tools/data.
 
 ### Args
 * **transforms** (list): Transforms for image.

docs/apis/datasets/datasets_cn.md

@@ -77,7 +77,7 @@ class paddleseg.datasets.Cityscapes(transforms, dataset_root, mode='train', edge
 class paddleseg.datasets.PascalVOC(transforms, dataset_root=None, mode='train', edge=False)
 ```
     PascalVOC2012 数据集 `http://host.robots.ox.ac.uk/pascal/VOC/`。
-    如果你想对数据集做数据增强, 请运行 tools 中的 voc_augment.py。
+    如果你想对数据集做数据增强, 请运行 tools/data 中的 voc_augment.py。
 
 ### 参数
 * **transforms** (list): 对图像的变换方法。

docs/data/custom/data_prepare.md

@@ -50,7 +50,7 @@ If your dataset is not organized as the aforementioned structure, we suggest tha
 
 The commands used are as follows, which supports enabling specific functions through different Flags.
 ```
-python tools/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
+python tools/data/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
 ```
 Parameters:
 - dataset_root: Dataset root directory
@@ -74,18 +74,18 @@ After running, `train.txt`, `val.txt`, `test.txt` and `labels.txt` will be gener
 
 #### Example
 ```
-python tools/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
+python tools/data/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
 ```
 
 ### 1.2 Generate txt files
 If you only have a divided dataset, you can generate a file list by executing the following script:
 ```
 # Generate a file list, the separator is a space, and the data format of the picture and the label set is png
-python tools/create_dataset_list.py <your/dataset/dir> --separator " " --format png png
+python tools/data/create_dataset_list.py <your/dataset/dir> --separator " " --format png png
 ```
 ```
 # Generate a list of files. The folders for pictures and tag sets are named img and gt, and the folders for training and validation sets are named training and validation. No test set list is generated.
-python tools/create_dataset_list.py <your/dataset/dir> \
+python tools/data/create_dataset_list.py <your/dataset/dir> \
         --folder img gt --second_folder training validation
 ```
 **Note:** A custom dataset directory must be specified, and FLAG can be set as needed. There is no need to specify `--type`.

docs/data/custom/data_prepare_cn.md

@@ -50,7 +50,7 @@
 
 使用命令如下，支持通过不同的Flags来开启特定功能。
 ```
-python tools/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
+python tools/data/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
 ```
 参数说明：
 - dataset_root: 数据集根目录
@@ -73,7 +73,7 @@ FLAGS说明：
 
 使用示例：
 ```
-python tools/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
+python tools/data/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
 ```
 
 
@@ -82,11 +82,11 @@ python tools/split_dataset_list.py <dataset_root> images annotations --split 0.6
 如果你只有划分好的数据集，可以通过执行以下脚本生成文件列表：
 ```
 # 生成文件列表，其分隔符为空格，图片和标签集的数据格式都为png
-python tools/create_dataset_list.py <your/dataset/dir> --separator " " --format png png
+python tools/data/create_dataset_list.py <your/dataset/dir> --separator " " --format png png
 ```
 ```
 # 生成文件列表，其图片和标签集的文件夹名为img和gt，训练和验证集的文件夹名为training和validation，不生成测试集列表
-python tools/create_dataset_list.py <your/dataset/dir> \
+python tools/data/create_dataset_list.py <your/dataset/dir> \
         --folder img gt --second_folder training validation
 ```
 

docs/data/marker/marker.md

@@ -28,7 +28,7 @@ At the same time, PaddleSeg is also compatible with gray-scale icon annotations.
 If users need to convert to pseudo-color annotation maps, they can use our conversion tool. Applies to the following two common situations:
 * If you want to convert all grayscale annotation images in a specified directory to pseudo-color annotation images, execute the following command to specify the directory where the grayscale annotations are located.
 ```buildoutcfg
-python tools/gray2pseudo_color.py <dir_or_file> <output_dir>
+python tools/data/gray2pseudo_color.py <dir_or_file> <output_dir>
 ```
 
 |Parameter|Effection|
@@ -38,7 +38,7 @@ python tools/gray2pseudo_color.py <dir_or_file> <output_dir>
 
 * If you only want to convert part of the gray scale annotated image in the specified dataset to pseudo-color annotated image, execute the following command, you need an existing file list, and read the specified image according to the list.
 ```buildoutcfg
-python tools/gray2pseudo_color.py <dir_or_file> <output_dir> --dataset_dir <dataset directory> --file_separator <file list separator>
+python tools/data/gray2pseudo_color.py <dir_or_file> <output_dir> --dataset_dir <dataset directory> --file_separator <file list separator>
 ```
 |Parameter|Effection|
 |-|-|
@@ -84,7 +84,7 @@ For all data that is not divided into training set, validation set, and test set
 The following commands support enabling specific functions through different Flags.
 
 ```
-python tools/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
+python tools/data/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
 ```
 
 Parameters:
@@ -103,7 +103,7 @@ FLAGS:
 
 The example of usage:
 ```
-python tools/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
+python tools/data/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
 ```
 
 After running, `train.txt`, `val.txt`, `test.txt` and `labels.txt` will be generated in the root directory of the dataset.

docs/data/marker/marker_cn.md

@@ -30,7 +30,7 @@ PaddleSeg既支持灰度标注图，也支持伪彩色标注图。
 * 如果希望将指定目录下的所有灰度标注图转换为伪彩色标注图，则执行以下命令。
 
 ```buildoutcfg
-python tools/gray2pseudo_color.py <dir_or_file> <output_dir>
+python tools/data/gray2pseudo_color.py <dir_or_file> <output_dir>
 ```
 
 |参数|用途|
@@ -41,7 +41,7 @@ python tools/gray2pseudo_color.py <dir_or_file> <output_dir>
 * 如果仅希望将指定数据集中的部分灰度标注图转换为伪彩色标注图，则执行以下命令。
 
 ```buildoutcfg
-python tools/gray2pseudo_color.py <dir_or_file> <output_dir> --dataset_dir <dataset directory> --file_separator <file list separator>
+python tools/data/gray2pseudo_color.py <dir_or_file> <output_dir> --dataset_dir <dataset directory> --file_separator <file list separator>
 ```
 |参数|用途|
 |-|-|
@@ -85,7 +85,7 @@ custom_dataset
 PaddleSeg提供了切分数据并生成文件列表的脚本。
 
 ```
-python tools/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
+python tools/data/split_dataset_list.py <dataset_root> <images_dir_name> <labels_dir_name> ${FLAGS}
 ```
 
 参数说明：
@@ -104,7 +104,7 @@ FLAGS说明：
 
 使用示例：
 ```
-python tools/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
+python tools/data/split_dataset_list.py <dataset_root> images annotations --split 0.6 0.2 0.2 --format jpg png
 ```
 
 运行后将在数据集根目录下生成`train.txt`、`val.txt`和`test.txt`，如下。

docs/data/pre_data.md

@@ -62,7 +62,7 @@ We recommend that you store dataset in `PaddleSeg/data` for full compatibility w
 Run the following command to convert labels:
 ```shell
 pip install cityscapesscripts
-python tools/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
+python tools/data/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
 ```
 where `cityscapes_path` should be adjusted according to the actual dataset path. `num_workers` determines the number of processes to be started. The value can be adjusted as required.
 
@@ -74,7 +74,7 @@ Generally, we will use [SBD(Semantic Boundaries Dataset)](http://home.bharathh.i
 Run the following commands to download the SBD dataset and use it to expand:
 ```shell
 cd PaddleSeg
-python tools/voc_augment.py --voc_path data/VOCdevkit --num_workers 8
+python tools/data/voc_augment.py --voc_path data/VOCdevkit --num_workers 8
 ```
 where `voc_path`should be adjusted according to the actual dataset path.
 
@@ -106,7 +106,7 @@ We recommend that you store dataset in `PaddleSeg/data` for full compatibility w
 Run the following command to convert labels:
 
 ```shell
-python tools/convert_cocostuff.py --annotation_path /PATH/TO/ANNOTATIONS --save_path /PATH/TO/CONVERT_ANNOTATIONS
+python tools/data/convert_cocostuff.py --annotation_path /PATH/TO/ANNOTATIONS --save_path /PATH/TO/CONVERT_ANNOTATIONS
 ```
 where `annotation_path` should be filled according to the `cocostuff/annotations` actual path. `save_path` determines the location of the converted label.
 
@@ -139,7 +139,7 @@ We recommend that you store dataset in `PaddleSeg/data` for full compatibility w
 Run the following command to convert labels:
 
 ```shell
-python tools/convert_voc2010.py --voc_path /PATH/TO/VOC ----annotation_path /PATH/TO/JSON
+python tools/data/convert_voc2010.py --voc_path /PATH/TO/VOC ----annotation_path /PATH/TO/JSON
 ```
 where `voc_path` should be filled according to the voc2010 actual path. `annotation_path` is the trainval_merged.json saved path.
 

docs/data/pre_data_cn.md

@@ -63,7 +63,7 @@ Cityscapes是关于城市街道场景的语义理解图片数据集。它主要
 
 ```shell
 pip install cityscapesscripts
-python tools/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
+python tools/data/convert_cityscapes.py --cityscapes_path data/cityscapes --num_workers 8
 ```
 
 ### ADE20K数据集
@@ -84,7 +84,7 @@ Pascal VOC 2012数据集以对象分割为主，包含20个类别和背景类，
 
 ```shell
 cd PaddleSeg
-python tools/voc_augment.py --voc_path data/VOCdevkit --num_workers 8
+python tools/data/voc_augment.py --voc_path data/VOCdevkit --num_workers 8
 ```
 
 ### Coco Stuff数据集
@@ -111,7 +111,7 @@ Coco Stuff是基于Coco数据集的像素级别语义分割数据集。它主要
 运行下列命令进行标签转换，其中`annotation_path`应根据下载cocostuff/annotations文件夹的实际路径填写，`save_path`决定转换后标签的存放位置。
 
 ```shell
-python tools/convert_cocostuff.py --annotation_path /PATH/TO/ANNOTATIONS --save_path /PATH/TO/CONVERT_ANNOTATIONS
+python tools/data/convert_cocostuff.py --annotation_path /PATH/TO/ANNOTATIONS --save_path /PATH/TO/CONVERT_ANNOTATIONS
 ```
 
 
@@ -142,7 +142,7 @@ Pascal Context是基于PASCAL VOC 2010数据集额外标注的像素级别的语
 运行下列命令进行标签转换：
 
 ```shell
-python tools/convert_voc2010.py --voc_path /PATH/TO/VOC ----annotation_path /PATH/TO/JSON
+python tools/data/convert_voc2010.py --voc_path /PATH/TO/VOC ----annotation_path /PATH/TO/JSON
 ```
 其中`voc_path`应根据下载VOC2010文件夹的实际路径填写。 `annotation_path`决定下载trainval_merged.json的存放位置。