You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+3-4
Original file line number
Diff line number
Diff line change
@@ -44,7 +44,6 @@ TencentPretrain has the following features:
44
44
* argparse
45
45
* packaging
46
46
* regex
47
-
* For the mixed precision training you will need apex from NVIDIA
48
47
* For the pre-trained model conversion (related with TensorFlow) you will need TensorFlow
49
48
* For the tokenization with sentencepiece model you will need [SentencePiece](https://github.com/google/sentencepiece)
50
49
* For developing a stacking model you will need LightGBM and [BayesianOptimization](https://github.com/fmfn/BayesianOptimization)
@@ -135,7 +134,7 @@ The above content provides basic ways of using TencentPretrain to pre-process, p
135
134
<br/>
136
135
137
136
## Pre-training data
138
-
This section provides links to a range of :arrow_right:[__pre-training data__](https://github.com/Tencent/TencentPretrain/wiki/Pretraining-data):arrow_left: .
137
+
This section provides links to a range of :arrow_right:[__pre-training data__](https://github.com/Tencent/TencentPretrain/wiki/Pretraining-data):arrow_left: . TencentPretrain can load these pre-training data directly.
139
138
140
139
<br/>
141
140
@@ -145,7 +144,7 @@ This section provides links to a range of :arrow_right: [__downstream datasets__
145
144
<br/>
146
145
147
146
## Modelzoo
148
-
With the help of TencentPretrain, we pre-trained models of different properties (e.g. models based on different modalities, encoders, and targets). Detailed introduction of pre-trained models and their download links can be found in :arrow_right:[__modelzoo__](https://github.com/Tencent/TencentPretrain/wiki/Modelzoo):arrow_left: . All pre-trained models can be loaded by TencentPretrain directly. More pre-trained models will be released in the future.
147
+
With the help of TencentPretrain, we pre-trained models of different properties (e.g. models based on different modalities, encoders, and targets). Detailed introduction of pre-trained models and their download links can be found in :arrow_right:[__modelzoo__](https://github.com/Tencent/TencentPretrain/wiki/Modelzoo):arrow_left: . All pre-trained models can be loaded by TencentPretrain directly.
149
148
150
149
<br/>
151
150
@@ -183,7 +182,7 @@ TencentPretrain/
183
182
184
183
```
185
184
186
-
The code is well-organized. Users can use and extend upon it with little efforts.
185
+
The code is organized based on components (e.g. embeddings, encoders). Users can use and extend upon it with little efforts.
187
186
188
187
Comprehensive examples of using TencentPretrain can be found in :arrow_right:[__instructions__](https://github.com/Tencent/TencentPretrain/wiki/Instructions):arrow_left: , which help users quickly implement pre-training models such as BERT, GPT-2, ELMo, T5, CLIP and fine-tune pre-trained models on a range of downstream tasks.
0 commit comments