Import W3C media-source/dedicated worker WPT

https://bugs.webkit.org/show_bug.cgi?id=269564
rdar://123076441

Reviewed by Eric Carlson.

Import WPT media-source/dedicated-worker.

Somes changes were required to make all tests pass.
1- When the MediaSource in the worker is being deleted, clean up its SourceBufferLists
and notify the HTMLMediaElement about it. The steps added aren't defined in the MSE spec.
Issue was raised as w3c/media-source#348
2- TrackBase's creation of the nullLogger wasn't thread-safe and could have been
concurrently accessed from both the main and worker thread.
3- Using a MediaSourceHandle already detached or previously attached to a media
element must fail
4- The [[has ever been attached]] slot is to be set to true prior changing the
delaying the load flag to false

media-source/dedicated-worker/mediasource-worker-play.html times out but so does chrome. Will investigate in
a follow-up post.

Upstream commit: web-platform-tests/wpt@32864fa

* LayoutTests/TestExpectations:
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-message-util.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element.js: Added.
(onmessage):
(catch):
(sourceBuffer.onerror):
(postMessage):
(bufferInto):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-duration-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-duration.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-duration.js: Added.
(processPhase):
(checkAckVerificationData.messageValuesEqual):
(checkAckVerificationData):
(bufferMediaAndSendDurationVerificationRequest):
(assert):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-get-objecturl.js: Added.
(onmessage):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle-transfer-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle-transfer-to-main.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle-transfer.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle-transfer.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-handle.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-must-fail-if-unsupported.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-objecturl-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-objecturl.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-objecturl.js: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play-terminate-worker-expected.txt: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play-terminate-worker.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play-terminate-worker.js: Added.
(catch):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play.html: Added.
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-play.js: Added.
(onmessage):
(catch):
(sourceBuffer.onerror):
(sourceBuffer.onupdateend):
(postMessage):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-util.js: Added.
(MediaSourceWorkerUtil):
(MediaSourceWorkerUtil.loadBinaryAsync):
* LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/w3c-import.log: Added.
* LayoutTests/platform/mac-wk2/TestExpectations:
* LayoutTests/platform/mac/TestExpectations:
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/Modules/mediasource/MediaSource.cpp:
(WebCore::MediaSource::~MediaSource):
(WebCore::MediaSource::setPrivateAndOpen):
(WebCore::MediaSource::attachToElement):
* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::selectMediaResource):
(WebCore::HTMLMediaElement::mediaSourceWasDetached):
* Source/WebCore/html/HTMLMediaElement.h:
* Source/WebCore/html/track/TrackBase.cpp:
(WebCore::nullLogger):
(WebCore::TrackBase::TrackBase):

Canonical link: https://commits.webkit.org/274959@main

LayoutTests/TestExpectations

@@ -1311,6 +1311,10 @@ http/wpt/offscreen-canvas/transferToImageBitmap-webgl.html [ ImageOnlyFailure Fa
 imported/w3c/web-platform-tests/html/canvas/offscreen/manual/the-offscreen-canvas/offscreencanvas.resize.html [ Failure Pass ]
 imported/w3c/web-platform-tests/html/canvas/offscreen/the-offscreen-canvas/size.large.html [ Skip ]
 
+# MSE in worker is only supported on platform with the GPU process active and where MSE is enabled by default
+# (this excludes iPhone (no MediaSource) and Linux/Windows (no GPUP))
+imported/w3c/web-platform-tests/media-source/dedicated-worker [ Skip ]
+
 # Tests a setting behind the ENABLE_FULLSCREEN_API flag, which is WK2-only
 media/video-supports-fullscreen.html [ Skip ]
 

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-message-util.js

@@ -0,0 +1,16 @@
+// This script provides an object with common message subjects to assist main
+// and worker thread communication.
+
+const messageSubject = {
+  ERROR: "error",  // info field may contain more detail
+  OBJECT_URL: "object url", // info field contains object URL
+  HANDLE: "handle", // info field contains the MediaSourceHandle
+  STARTED_BUFFERING: "started buffering",
+  FINISHED_BUFFERING: "finished buffering",
+  VERIFY_DURATION: "verify duration", // info field contains expected duration
+  AWAIT_DURATION: "await duration", // wait for element duration to match the expected duration in the info field
+  VERIFY_HAVE_NOTHING: "verify have nothing readyState",
+  VERIFY_AT_LEAST_HAVE_METADATA: "verify readyState is at least HAVE_METADATA",
+  ACK_VERIFIED: "verified", // info field contains the message values that requested the verification
+  WORKER_DONE: "worker done", // this lets worker signal main to successfully end the test
+};

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element-expected.txt

@@ -0,0 +1,18 @@
+
+
+PASS Test element detachment from worker MediaSource after at least 0 main thread setTimeouts, starting counting after setting srcObject
+PASS Test element detachment from worker MediaSource after at least 1 main thread setTimeouts, starting counting after setting srcObject
+PASS Test element detachment from worker MediaSource after at least 2 main thread setTimeouts, starting counting after setting srcObject
+PASS Test element detachment from worker MediaSource after at least 3 main thread setTimeouts, starting counting after setting srcObject
+PASS Test element detachment from worker MediaSource after at least 4 main thread setTimeouts, starting counting after setting srcObject
+PASS Test element detachment from worker MediaSource after at least 0 main thread setTimeouts, starting counting after receiving Started Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 1 main thread setTimeouts, starting counting after receiving Started Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 2 main thread setTimeouts, starting counting after receiving Started Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 3 main thread setTimeouts, starting counting after receiving Started Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 4 main thread setTimeouts, starting counting after receiving Started Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 0 main thread setTimeouts, starting counting after receiving Finished Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 1 main thread setTimeouts, starting counting after receiving Finished Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 2 main thread setTimeouts, starting counting after receiving Finished Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 3 main thread setTimeouts, starting counting after receiving Finished Buffering message from worker
+PASS Test element detachment from worker MediaSource after at least 4 main thread setTimeouts, starting counting after receiving Finished Buffering message from worker
+

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element.html

@@ -0,0 +1,75 @@
+<!DOCTYPE html>
+<html>
+<title>MediaSource-in-Worker buffering test case with media element detachment at various places</title>
+<script src="/resources/testharness.js"></script>
+<script src="/resources/testharnessreport.js"></script>
+<script src="mediasource-message-util.js"></script>
+<body>
+<script>
+
+const AFTER_SETTING_SRCOBJECT = "after setting srcObject";
+const AFTER_STARTED_BUFFERING = "after receiving Started Buffering message from worker";
+const AFTER_FINISHED_BUFFERING = "after receiving Finished Buffering message from worker";
+
+[ AFTER_SETTING_SRCOBJECT, AFTER_STARTED_BUFFERING, AFTER_FINISHED_BUFFERING ].forEach(when => {
+  for (let timeouts = 0; timeouts < 5; ++timeouts) {
+    async_test(test => { startWorkerAndDetachElement(test, when, timeouts); },
+        "Test element detachment from worker MediaSource after at least " + timeouts +
+          " main thread setTimeouts, starting counting " + when);
+  }
+});
+
+function detachElementAfterMultipleSetTimeouts(test, element, timeouts_remaining) {
+  if (timeouts_remaining <= 0) {
+    // While not the best way to detach, this triggers interoperable logic that
+    // includes detachment.
+    element.srcObject = null;
+    test.step_timeout(() => { test.done(); }, 10);
+  } else {
+    test.step_timeout(() => {
+      detachElementAfterMultipleSetTimeouts(test, element, --timeouts_remaining);
+    }, 0);
+  }
+}
+
+function startWorkerAndDetachElement(test, when_to_start_timeouts, timeouts_to_await) {
+  // Fail fast if MSE-in-Workers is not supported.
+  assert_true(MediaSource.hasOwnProperty("canConstructInDedicatedWorker"), "MediaSource hasOwnProperty 'canConstructInDedicatedWorker'");
+  assert_true(MediaSource.canConstructInDedicatedWorker, "MediaSource.canConstructInDedicatedWorker");
+
+  const worker = new Worker("mediasource-worker-detach-element.js");
+  worker.onerror = test.unreached_func("worker error");
+
+  const video = document.createElement("video");
+  document.body.appendChild(video);
+
+  worker.onmessage = test.step_func(e => {
+    let subject = e.data.subject;
+    assert_true(subject != undefined, "message must have a subject field");
+    switch (subject) {
+      case messageSubject.ERROR:
+        assert_unreached("Worker error: " + e.data.info);
+        break;
+      case messageSubject.HANDLE:
+        const handle = e.data.info;
+        video.srcObject = handle;
+        if (when_to_start_timeouts == AFTER_SETTING_SRCOBJECT) {
+          detachElementAfterMultipleSetTimeouts(test, video, timeouts_to_await);
+        }
+        break;
+      case messageSubject.STARTED_BUFFERING:
+        if (when_to_start_timeouts == AFTER_STARTED_BUFFERING)
+          detachElementAfterMultipleSetTimeouts(test, video, timeouts_to_await);
+        break;
+      case messageSubject.FINISHED_BUFFERING:
+        if (when_to_start_timeouts == AFTER_FINISHED_BUFFERING)
+          detachElementAfterMultipleSetTimeouts(test, video, timeouts_to_await);
+        break;
+      default:
+        assert_unreached("Unrecognized message subject: " + subject);
+    }
+  });
+}
+</script>
+</body>
+</html>

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-detach-element.js

@@ -0,0 +1,79 @@
+// This is similar to mediasource-worker-play.js, except that the buffering is
+// longer and done in tiny chunks to enable a better chance of the main thread
+// detaching the element while interesting buffering work is still occurring. To
+// assist the main thread understanding when the buffering has started already
+// or has completed already, we also perform extra messaging.
+importScripts("mediasource-worker-util.js");
+
+onmessage = function(evt) {
+  postMessage({ subject: messageSubject.ERROR, info: "No message expected by Worker" });
+};
+
+let util = new MediaSourceWorkerUtil();
+
+let sentStartedBufferingMessage = false;
+
+util.mediaSource.addEventListener("sourceopen", () => {
+  let sourceBuffer;
+  try {
+    sourceBuffer = util.mediaSource.addSourceBuffer(util.mediaMetadata.type);
+  }  catch(e) {
+    // Detachment may have already begun, so allow exception here.
+    // TODO(https://crbug.com/878133): Consider a distinct readyState for the case
+    // where exception occurs due to "Worker MediaSource attachment is closing".
+    // That would assist API users and narrow the exception handling here.
+    return;
+  }
+
+  sourceBuffer.onerror = (err) => {
+    postMessage({ subject: messageSubject.ERROR, info: err });
+  };
+  util.mediaLoadPromise.then(mediaData => bufferInto(sourceBuffer, mediaData, 100, 0),
+                             err => { postMessage({ subject: messageSubject.ERROR, info: err }) } );
+}, { once : true });
+
+let handle = util.mediaSource.handle;
+
+postMessage({ subject: messageSubject.HANDLE, info: handle }, { transfer: [handle] } );
+
+// Append increasingly large pieces at a time, starting/continuing at |position|.
+// This allows buffering the test media without timeout, but also with enough
+// operations to gain coverage on detachment concurrency with append.
+function bufferInto(sourceBuffer, mediaData, appendSize, position) {
+  if (position >= mediaData.byteLength) {
+    postMessage({ subject: messageSubject.FINISHED_BUFFERING });
+    try {
+      util.mediaSource.endOfStream();
+    }  catch(e) {
+      // Detachment may have already begun, so allow exception here.
+      // TODO(https://crbug.com/878133): Consider a distinct readyState for the case
+      // where exception occurs due to "Worker MediaSource attachment is closing".
+      // That would assist API users and narrow the exception handling here.
+      // FALL-THROUGH - return.
+    }
+    return;
+  }
+
+  var nextPosition = position + appendSize;
+  const pieceToAppend = mediaData.slice(position, nextPosition);
+  position = nextPosition;
+  appendSize += 100;
+
+  sourceBuffer.addEventListener("updateend", () => {
+    if (!sentStartedBufferingMessage) {
+      postMessage({ subject: messageSubject.STARTED_BUFFERING});
+      sentStartedBufferingMessage = true;
+    }
+    bufferInto(sourceBuffer, mediaData, appendSize, position);
+  }, { once : true });
+
+  try {
+    sourceBuffer.appendBuffer(pieceToAppend);
+  }  catch(e) {
+    // Detachment may have already begun, so allow exception here.
+    // TODO(https://crbug.com/878133): Consider a distinct readyState for the case
+    // where exception occurs due to "Worker MediaSource attachment is closing".
+    // That would assist API users and narrow the exception handling here.
+    // FALL-THROUGH - return.
+  }
+}

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-duration-expected.txt

@@ -0,0 +1,4 @@
+
+
+PASS Test worker MediaSource duration updates before and after HAVE_METADATA
+

LayoutTests/imported/w3c/web-platform-tests/media-source/dedicated-worker/mediasource-worker-duration.html

@@ -0,0 +1,86 @@
+<!DOCTYPE html>
+<html>
+<title>Test MediaSource-in-Worker duration updates before and after HAVE_METADATA</title>
+<script src="/resources/testharness.js"></script>
+<script src="/resources/testharnessreport.js"></script>
+<script src="mediasource-message-util.js"></script>
+<body>
+<script>
+
+function awaitDuration(t, video, worker, requestingMessage, expectedDuration) {
+  let durationAwaiter = t.step_func(() => {
+    if ((!Number.isNaN(expectedDuration) && video.duration === expectedDuration) ||
+        (Number.isNaN(expectedDuration) && Number.isNaN(video.duration))) {
+      worker.postMessage({ subject: messageSubject.ACK_VERIFIED, info: requestingMessage });
+      return;
+    }
+
+    // Otherwise, wait for one or more 'durationchange' events to see if video
+    // eventually has the expectedDuration.
+    video.addEventListener('durationchange', durationAwaiter, { once: true });
+  });
+
+  durationAwaiter();
+}
+
+async_test(t => {
+  // Fail fast if MSE-in-Workers is not supported.
+  assert_true(MediaSource.hasOwnProperty("canConstructInDedicatedWorker"), "MediaSource hasOwnProperty 'canConstructInDedicatedWorker'");
+  assert_true(MediaSource.canConstructInDedicatedWorker, "MediaSource.canConstructInDedicatedWorker");
+
+  const video = document.createElement("video");
+  document.body.appendChild(video);
+  video.onerror = t.unreached_func("video element error");
+  video.onended = t.unreached_func("video element ended");
+  assert_equals(video.duration, NaN, "initial video duration before attachment should be NaN");
+  assert_equals(video.readyState, HTMLMediaElement.HAVE_NOTHING, "initial video readyState before attachment should be HAVE_NOTHING");
+
+  let worker = new Worker("mediasource-worker-duration.js");
+  worker.onerror = t.step_func(e => {
+    assert_unreached("worker error: [" + e.filename + ":" + e.lineno + ":" + e.colno + ":" + e.error + ":" + e.message + "]");
+  });
+  worker.onmessage = t.step_func(e => {
+    let subject = e.data.subject;
+    assert_true(subject !== undefined, "message must have a subject field");
+    switch (subject) {
+      case messageSubject.ERROR:
+        assert_unreached("Worker error: " + e.data.info);
+        break;
+      case messageSubject.HANDLE:
+        const handle = e.data.info;
+        assert_equals(video.duration, NaN, "initial video duration before attachment should still be NaN");
+        assert_equals(video.readyState, HTMLMediaElement.HAVE_NOTHING,
+                      "initial video readyState before attachment should still be HAVE_NOTHING");
+        video.srcObject = handle;
+        break;
+      case messageSubject.VERIFY_DURATION:
+        assert_equals(video.duration, e.data.info, "duration should match expectation");
+        worker.postMessage({ subject: messageSubject.ACK_VERIFIED, info: e.data });
+        break;
+      case messageSubject.AWAIT_DURATION:
+        awaitDuration(t, video, worker, e.data, e.data.info);
+        break;
+      case messageSubject.VERIFY_HAVE_NOTHING:
+        assert_equals(video.readyState, HTMLMediaElement.HAVE_NOTHING, "readyState should match expectation");
+        worker.postMessage({ subject: messageSubject.ACK_VERIFIED, info: e.data });
+        break;
+      case messageSubject.VERIFY_AT_LEAST_HAVE_METADATA:
+        assert_greater_than_equal(video.readyState, HTMLMediaElement.HAVE_METADATA, "readyState should match expectation");
+        worker.postMessage({ subject: messageSubject.ACK_VERIFIED, info: e.data });
+        break;
+      case messageSubject.WORKER_DONE:
+        // This test is a worker-driven set of verifications, and it will send
+        // this message when it is complete. See comment in the worker script
+        // that describes the phases of this test case.
+        assert_not_equals(video.srcObject, null, "test should at least have set srcObject.");
+        t.done();
+        break;
+      default:
+        assert_unreached("Unexpected message subject: " + subject);
+    }
+  });
+}, "Test worker MediaSource duration updates before and after HAVE_METADATA");
+
+</script>
+</body>
+</html>