Output not streamed to stdout

[jeevcat]

It seems all output is captured and then returned to stdout only once the underlying Unreal process is complete. This means it's hard to determine the progress of the command, especially for long-running test suites (using ue4 test).

[sleeptightAnsiC]

This is indeed a huge problem.

Seems like UnrealManagerBase::automationTest uses Utility::capture under the hood so it can grab the output of the test and check if it contains a common sub-string patterns indicating that test has failed:

ue4cli/ue4cli/UnrealManagerBase.py

Lines 578 to 602 in 39d6f6a

	# Attempt to run the automation tests
	Utility.printStderr('Running automation tests...')
	logOutput = self.runAutomationCommands(projectFile, command, extraArgs, capture=True, enableRHI=enableRHI)
	# Propagate the log output
	print(logOutput.stdout)
	print(logOutput.stderr)
	# Detect abnormal exit conditions (those not triggered by `automation quit`)
	# In Unreal Engine 4.27.0, the exit method changed from RequestExit to RequestExitWithStatus
	if 'PlatformMisc::RequestExit(' not in logOutput.stdout and 'PlatformMisc::RequestExitWithStatus(' not in logOutput.stdout:
	sys.exit(1)
	# Since UE4 doesn't consistently produce accurate exit codes across all platforms, we need to rely on
	# text-based heuristics to detect failed automation tests or errors related to not finding any tests to run
	errorStrings = [
	'Incorrect automation command syntax!',
	'Automation Test Failed',
	'Result={Failed}',
	'Found 0 Automation Tests, based on',
	'is not a valid flag to filter on!'
	]
	for errorStr in errorStrings:
	if errorStr in logOutput.stdout:
	sys.exit(1)

Most of other ue4cli commands use Utility::run so this isn't a problem for them.

To be honest, the way of detecting if test has failed is really fragile and probably should be changed.
If that's not possible, the output of subproccess must be streamed.

Hmnn...