[FIXED] redis / rq worker breaking on python3

Issue

So I am getting error while executing code in rq worker

 File "/usr/local/bin/rq", line 11, in <module>
   sys.exit(main())
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
   return self.main(*args, **kwargs)
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
   rv = self.invoke(ctx)
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
   return _process_result(sub_ctx.command.invoke(sub_ctx))
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
   return ctx.invoke(self.callback, **ctx.params)
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
   return callback(*args, **kwargs)
 File "/usr/local/lib/python3.7/site-packages/rq/cli/cli.py", line 75, in wrapper
   return ctx.invoke(func, cli_config, *args[1:], **kwargs)
 File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
   return callback(*args, **kwargs)
 File "/usr/local/lib/python3.7/site-packages/rq/cli/cli.py", line 236, in worker
   worker.work(burst=burst, logging_level=logging_level)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 493, in work
   self.execute_job(job, queue)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 662, in execute_job
   self.fork_work_horse(job, queue)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 599, in fork_work_horse
   self.main_work_horse(job, queue)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 677, in main_work_horse
   success = self.perform_job(job, queue)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 781, in perform_job
   self.prepare_job_execution(job)
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 706, in prepare_job_execution
   registry.add(job, timeout, pipeline=pipeline)
 File "/usr/local/lib/python3.7/site-packages/rq/registry.py", line 47, in add
   return pipeline.zadd(self.key, score, job.id)
 File "/usr/local/lib/python3.7/site-packages/redis/client.py", line 2263, in zadd
   for pair in iteritems(mapping):
 File "/usr/local/lib/python3.7/site-packages/redis/_compat.py", line 123, in iteritems
return iter(x.items())

AttributeError: 'int' object has no attribute 'items'
14:04:29 Moving job to 'failed' queue (work-horse terminated unexpectedly; waitpid returned 256)

I am trying to run this code in decoupled containers on openshift. Same images work locally. I guess only difference is I was running rq worker on system instead on container when trying locally and my local have both python 2 and 3 while openshift have python3 only.

Can anyone resolve why it started behaving like this. I suspect its due to python version. but i dont know how to run redis/ rq worker with python 2.

Solution

That is a known issue, resulting from redis-py's new version 3 - see https://github.com/rq/rq/issues/1014

Answered By - Itamar Haber