[FIXED] How to save the output as csv file in python

Issue

I use this Python snippet to obtain the rabbitmq queue size output, which is already in csv format, however I would need to know how to save this data to a file.

import subprocess

proc = subprocess.Popen("/usr/sbin/rabbitmqctl list_queues --formatter csv", shell=True, stdout=subprocess.PIPE)
stdout_value = proc.communicate()[0]
print stdout_value

Output i get from this snippet

"name","messages"
"search_publish_queue","1"
.
.

I tried to write the output to a csv file using with open, it writes however it add , next to every character.

import subprocess
import csv

with open('test.csv', 'w', ) as f:
    writer = csv.writer(f)
    proc = subprocess.Popen("/usr/sbin/rabbitmqctl list_queues --formatter csv", shell=True, stdout=subprocess.PIPE)
    stdout_value = proc.communicate()[0]
    writer.writerow(stdout_value)

It writes to file as below

",n,a,m,e,",",m,e,s,s,a,g,e,s,"
",s,e,a,r,c,h,_,p,u,b,l,i,s,h,_,q,u,e,u,e,",,,",1,"

Whats the write way to do this? if not using

Solution

The issue you're facing is that writer.writerow expects an iterable of fields to write as a single row. In your case, stdout_value is actually a byte string, so when you pass it to writer.writerow, the function tries to iterate over each byte, which leads to the incorrect output you're seeing.

There are a few ways to solve this problem. One approach is to decode stdout_value to a regular string and write it directly to the file. Since your data is already in CSV format, you don't need to use the csv module for this particular case:

import subprocess

with open('test.csv', 'wb') as f:
    proc = subprocess.Popen("/usr/sbin/rabbitmqctl list_queues --formatter csv", shell=True, stdout=f)
    proc.communicate()

In this example, I used 'wb' as the file mode to write bytes directly to the file. I also passed f directly as the stdout parameter to Popen, which makes it write the output directly to the file.

Alternatively, if you still want to read the data into a variable first, you can do the following:

import subprocess

proc = subprocess.Popen("/usr/sbin/rabbitmqctl list_queues --formatter csv", shell=True, stdout=subprocess.PIPE)
stdout_value = proc.communicate()[0]

with open('test.csv', 'wb') as f:
    f.write(stdout_value)

In this case, I've used the 'wb' mode to write the bytes as they are. Note that the stdout_value is a bytes object, so we can write it directly to a file opened in binary mode ('wb'). I hope this helps!

Answered By - Christof Garbers