Reply to: Subprocess Hanging: PIPE is your enemy

I experienced a similar issue that I need to run zcat, and then head, tail or wc -l on the output, all called from within python 2.65. Apparently the output is much larger than the 65K (it is a few hundreds MB to a few GB). It has puzzled me for a while why zcat always hang until I read this wonderful post. For my case, I would like to avoid creating temp file as well since it could be very big, and I don't necessarily want to write out the entire file. I ended up wrapping the two commands with unix pipe ("|") connected into one string, than pass that into subprocess.Popen, with shell=True (which I try to avoid usually). This has been working fine for my case.

formatting is with Markdown syntax. Comments are not displayed until they are approved by a moderator. Moderators will not approve unless the comment contributes value to the discussion.

remember info?