Skip to content

Commit

Permalink
Fix call logging by unbuffering stdout/err in call threadpool (#724)
Browse files Browse the repository at this point in the history
Because we started running method calls in a threadpool, the output was buffered and only flushed/printed at the end (https://stackoverflow.com/questions/18234469/python-multithreaded-print-statements-delayed-until-all-threads-complete-executi). This changes the StreamTee utility we use to print stdout/logging to flush on each write, per https://mail.python.org/pipermail/tutor/2003-November/026645.html
  • Loading branch information
dongreenberg committed Apr 12, 2024
1 parent 6c51e62 commit 6b7cfea
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions runhouse/resources/provenance.py
Original file line number Diff line number Diff line change
Expand Up @@ -527,6 +527,15 @@ def write(self, message):
for stream in self.outstreams:
if message:
stream.write(message)
# We flush here to ensure that the logs are written to the file immediately
# see https://github.com/run-house/runhouse/pull/724
stream.flush()

def writelines(self, lines):
self.instream.writelines(lines)
for stream in self.outstreams:
stream.writelines(lines)
stream.flush()

def flush(self):
self.instream.flush()
Expand Down

0 comments on commit 6b7cfea

Please sign in to comment.