Skip to content

Commit a9210be

Browse files
committed
fix \r in pre-processing
1 parent 2d38c0d commit a9210be

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

python_model/load_data.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ def csv_df(self, csv_fields):
6767
def pre_process(self, df):
6868
print("Note: pre-process changes the dataframe inplace.")
6969
# remove new line char
70-
df['text'].replace(regex=True,inplace=True,to_replace=r'\\n',value=r'')
70+
df['text'].replace(regex=True,inplace=True,to_replace='(\\n|\\r|\\r\\n)',value='')
7171
# remove https links
7272
df['text'].replace(regex=True,inplace=True,to_replace=r'(http|https):\/\/[^(\s|\b)]+',value=r'')
7373
# remove user name

0 commit comments

Comments
 (0)