[FIXED] How to Handle Special Characters When Exporting to CSV

Issue

My Python code pulls data from a website and exports the result to a CSV. The code breaks when a foreign letter is detected and not all data get downloaded.

This is the error I am getting:

UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f6f5' in position 91: character maps to "undefined"

An example would be when a row contains russian alphabet or special characters such as the number sign # or a *.

This is my code that exports data to CSV. Please let me know how I can fix this error:

# save it to csv
  with open('links.csv', 'w', newline='') as csv_out:
       csv_writer = csv.writer(csv_out)
# Create the header rows
  csv_writer.writerow(['url', 'title','price','location'])
  for row in data:
      csv_writer.writerow([str(row['link']),str(row['title']),str(row['price']),
      str(row['location'])])
    

Solution

Are you running on Windows? If so, try specifying the file encoding when opening the file, e.g.

open('links.csv', 'w', newline='', encoding='utf-8')

Answered By - Mark Reed