-
Notifications
You must be signed in to change notification settings - Fork 407
Open
Description
I was face this issue,
Traceback (most recent call last):
File "D:\Devolvedai_work\Devolved_AI_Data_Preparation\perser_experiments\test.py", line 9, in <module>
megaparse.save(r"D:/Devolvedai_work/Devolved_AI_Data_Preparation/perser_experiments/1.md")
File "D:\Devolvedai_work\Devolved_AI_Data_Preparation\venv\Lib\site-packages\megaparse\megaparse.py", line 133, in save
f.write(self.last_parsed_document)
File "C:\Users\hp\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u0338' in position 12572: character maps to <undefined>
It solved by creating a save function manually. Here i just added the encoding parameters
def save(file_path:str) -> None:
os.makedirs(os.path.dirname(file_path), exist_ok=True)
with open(file_path, "w+", encoding="utf-8") as f: # Specify UTF-8 encoding
f.write(megaparse.last_parsed_document)
chloedia
Metadata
Metadata
Assignees
Labels
No labels