Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #2872 - Driver memory leaks #2885

Merged
merged 15 commits into from
Oct 6, 2023

Conversation

tig
Copy link
Collaborator

@tig tig commented Oct 4, 2023

Fixes #2872 - Driver memory leaks

Pull Request checklist:

  • I've named my PR in the form of "Fixes #issue. Terse description."
  • My code follows the style guidelines of Terminal.Gui - if you use Visual Studio, hit CTRL-K-D to automatically reformat your files before committing.
  • My code follows the Terminal.Gui library design guidelines
  • I ran dotnet test before commit
  • I have made corresponding changes to the API documentation (using /// style comments)
  • My changes generate no new warnings
  • I have checked my code and corrected any poor grammar or misspellings
  • I conducted basic QA to assure all features are working

@tig tig added the v2 For discussions, issues, etc... relavant for v2 label Oct 4, 2023
@tig tig marked this pull request as draft October 4, 2023 14:59
@tig
Copy link
Collaborator Author

tig commented Oct 4, 2023

@BDisp, I'm trying to figure out the wait param on CheckTimers:

		/// <summary>
		/// Checks if there are any outstanding timers that need to be processed. Called from the driver's <see cref="IMainLoopDriver"/>.
		/// </summary>
		/// <param name="wait">If <see langword="false"/> <paramref name="waitTimeout"/> will be set to 0.</param>
		/// <param name="waitTimeout">Returns the number of milliseconds remaining in the current timer.</param>
		/// <returns></returns>
		public bool CheckTimers (bool wait, out int waitTimeout)
		{

		if (wait == false) {
			throw new InvalidOperationException ("CheckTimers should not be called with wait == false");
    		}
			long now = DateTime.UtcNow.Ticks;

			if (_timeouts.Count > 0) {
				waitTimeout = (int)((_timeouts.Keys [0] - now) / TimeSpan.TicksPerMillisecond);
				if (waitTimeout < 0)
					// This avoids 'poll' waiting infinitely if 'waitTimeout < 0' until some action is detected
					// This can occur after IMainLoopDriver.Wakeup is executed where the pollTimeout is less than 0
					// and no event occurred in elapsed time when the 'poll' is start running again.
					waitTimeout = 0; 
					return true;
			} else {
				waitTimeout = -1;
			}

			if (!wait) {
				waitTimeout = 0;
			}

			int ic;
			lock (_idleHandlers) {
				ic = _idleHandlers.Count;
			}

			return ic > 0;
		}

I have looked over all the code in the project, including unit tests, and I can't find any cases where it will ever bet set to false.

I bubbles up to:

Application.RunMainLoopIteration:

/// <summary>
/// Run one iteration of the <see cref="MainLoop"/>.
/// </summary>
/// <param name="state">The state returned by <see cref="Begin(Toplevel)"/>.</param>
/// <param name="wait">If <see langword="true"/> will execute the <see cref="MainLoop"/> waiting for events. If <see langword="false"/>
/// the method will return after a single iteration.</param>
/// <param name="firstIteration">Set to <see langword="true"/> if this is the first run loop iteration. Upon return,
/// it will be set to <see langword="false"/> if at least one iteration happened.</param>
public static void RunMainLoopIteration (ref RunState state, bool wait, ref bool firstIteration)
{

image

I added the throw you see above and ran the unit tests. It never fires.

So, I believe this whole wait parameter starting at Application.RunMainLoopIteration, then at IMainLoopDriver.EventsPending, then CheckTimers is un-used and un-needed.

Am I missing something?

@tig tig marked this pull request as ready for review October 4, 2023 22:51
@BDisp
Copy link
Collaborator

BDisp commented Oct 4, 2023

You are right. I also never seen it as false, but it's an option. If false the timeout will be 0 and the loop will continue infinitely.

@tig
Copy link
Collaborator Author

tig commented Oct 4, 2023

@BDisp, also nobody ever checks for a waitTimeout == -1.

In fact, both Windows and NetDriver have a bug w.r.t this because _keyReady.Wait will be called with a timeout of -1!

	bool IMainLoopDriver.EventsPending ()
	{
		_waitForProbe.Set ();

		if (_mainLoop.CheckTimersAndIdleHandlers (out var waitTimeout)) {
			return true;
		}

		try {
			if (!_tokenSource.IsCancellationRequested) {
				// BUGBUG: CheckTimers can return false with waitTimeOut set to -1 if there 
				// are no idle handlers or timers. The code below doesn't account for this.
				_keyReady.Wait (waitTimeout, _tokenSource.Token);
			}
		} catch (OperationCanceledException) {
			return true;
		} finally {
			_keyReady.Reset ();
		}

		if (!_tokenSource.IsCancellationRequested) {
			return _inputResult.Count > 0 || _mainLoop.CheckTimersAndIdleHandlers (out _);
		}

		_tokenSource.Dispose ();
		_tokenSource = new CancellationTokenSource ();
		return true;
	}

I think I can change CheckTimers to always set waitTimeout = 0 if there are no active timers or idle handlers. Right?

@tig
Copy link
Collaborator Author

tig commented Oct 4, 2023

Here's my proposed new function on MainLoop:

		/// <summary>
		/// Called from <see cref="IMainLoopDriver.EventsPending"/> to check if there are any outstanding timers or idle handlers.
		/// </summary>
		/// <param name="waitTimeout">Returns the number of milliseconds remaining in the current timer (if any).</param>
		/// <returns><see langword="true"/> if there is a timer or idle handler active.</returns>
		public bool CheckTimersAndIdleHandlers (out int waitTimeout)
		{
			var now = DateTime.UtcNow.Ticks;

			waitTimeout = 0;

			lock (_timeouts) {
				if (_timeouts.Count > 0) {
					waitTimeout = (int)((_timeouts.Keys [0] - now) / TimeSpan.TicksPerMillisecond);
					if (waitTimeout < 0) {
						// This avoids 'poll' waiting infinitely if 'waitTimeout < 0' until some action is detected
						// This can occur after IMainLoopDriver.Wakeup is executed where the pollTimeout is less than 0
						// and no event occurred in elapsed time when the 'poll' is start running again.
						waitTimeout = 0;
					}
					return true;
				}
			}

			// There are no timers set, check if there are any idle handlers

			lock (_idleHandlers) {
				return _idleHandlers.Count > 0;
			}
		}

@BDisp
Copy link
Collaborator

BDisp commented Oct 5, 2023

I think I can change CheckTimers to always set waitTimeout = 0 if there are no active timers or idle handlers. Right?

I don't recommend that. Return -1 is normally used to wait until some keystroke occurs. If it's necessary to interrupt, then the keyReady.Set will interrupt the wait for WindowsDriver and NetDriver. The CursesDriver write to a file descriptor to interrupt the poll call.
I agree that can return 0 if a timeout has elapsed a negative value, but is there is no timeouts should return -1 if wait is true and 0 if wait is false.

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

Please explain how it makes sense to pass -1 to _keyReady.Wait?

@BDisp
Copy link
Collaborator

BDisp commented Oct 5, 2023

Please explain how it makes sense to pass -1 to _keyReady.Wait?

That's the way how ManualResetEvent works when the call in the Wait handles internally for any event. The poll function also accept -1 to wait for an event. Otherwise the mainloop will always running without any event.

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

ManualResetEvent

I see now:

image

Very confusing and a great example of where just a few comments in the code would be super helpful.

New version:

/// <summary>
/// Called from <see cref="IMainLoopDriver.EventsPending"/> to check if there are any outstanding timers or idle handlers.
/// </summary>
/// <param name="waitTimeout">Returns the number of milliseconds remaining in the current timer (if any). Will be -1 if
/// there are no active timers.</param>
/// <returns><see langword="true"/> if there is a timer or idle handler active.</returns>
public bool CheckTimersAndIdleHandlers (out int waitTimeout)
{
	var now = DateTime.UtcNow.Ticks;

	waitTimeout = 0;

	lock (_timeouts) {
		if (_timeouts.Count > 0) {
			waitTimeout = (int)((_timeouts.Keys [0] - now) / TimeSpan.TicksPerMillisecond);
			if (waitTimeout < 0) {
				// This avoids 'poll' waiting infinitely if 'waitTimeout < 0' until some action is detected
				// This can occur after IMainLoopDriver.Wakeup is executed where the pollTimeout is less than 0
				// and no event occurred in elapsed time when the 'poll' is start running again.
				waitTimeout = 0;
			}
			return true;
		}
		// ManualResetEventSlim.Wait, which is called by IMainLoopDriver.EventsPending, will wait indefinitely if
		// the timeout is -1.
		waitTimeout = -1;
	}

	// There are no timers set, check if there are any idle handlers

	lock (_idleHandlers) {
		return _idleHandlers.Count > 0;
	}
}

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

As I was fixing mem leak this am in windows driver I found the tearing on resize no longer happens if we use the input event!

I think we can remove the CheckWinChanges task, at least for recent windows / wt versions!

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

As I was fixing mem leak this am in windows driver I found the tearing on resize no longer happens if we use the input event!

I think we can remove the CheckWinChanges task, at least for recent windows / wt versions!

Meh. I couldn't get it to work; the sizes given in the event and during Init are still wrong and it's not until the first time the ChangeWin thread runs that the right size is returned.

I did document it all better in #define HACK_CHECK_WINCHANGED.

This will be moot once WindowsDriver is refactored via #2610.

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

Ok, WindowsDriver is now fixed in this PR.

now on to NetDriver:

image

@tig
Copy link
Collaborator Author

tig commented Oct 5, 2023

FWIW, Performance Profiler doesn't let you specify cmd line args to the Startup Project.

I've enabled a ConfigurationManager setting to force it:

image

@tig
Copy link
Collaborator Author

tig commented Oct 6, 2023

@BDisp,

I fixed the most egregious memory leak in Netdriver here.

However, the interplay between the threading Tasks in NetMainLoop and NetEvents is horrifically complex and I'm not sure I got it right. I'd really appreciate a thorough code review by you.

In addition, there still is a very weird memory leak (or allocation of objects that doesn't make sense):

  1. Set "Application.UseSystemConsole": true in ./UICatalog/bin/Debug/net7.0/.tui/UICatalog.config
{
    "Application.UseSystemConsole" : true
} 
  1. Start Performance Profiler with memory profiling on
  2. Take a snapshot right away
  3. Double click on "Generic"
  4. Hit ESC to close Generic
  5. Take a snapshot

You'll see something like this:
image

Click on the +1612 and see this (sort by "Count Diff"):
image

For some reason ~1000 "Task+DelayPromiseWithCancellation" objects are being created. The number is different each run, but not in an deterministic way. I've also noticed that it has something do with mouse vs. keyboard input. But I've not been able to find a pattern. Nor have I figured out how to fix it (obviously).

Maybe you can see what's going on?

@tig tig merged commit 03e29f3 into gui-cs:v2_develop Oct 6, 2023
1 check passed
@tig tig deleted the v2_fixes_2872_driver_mem_leaks branch April 3, 2024 01:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v2 For discussions, issues, etc... relavant for v2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants