From ff19edf0def13fa53d9f7842d6018c8867be8512 Mon Sep 17 00:00:00 2001 From: Matt Toohey Date: Mon, 2 Nov 2026 12:24:43 +1100 Subject: [PATCH] test(mobile): gate push snapshot tests on a settled host, name the observed QoS Follow-up to f0bf2c0c8 from its review (no blockers; one suggestion and one warning, both addressed here). Suggestion: the QoS assertion in `testRestrictionCleanupFailurePreservesSnapshotAndRestoreReleasesAuthority` compared two raw `qos_class_t` arrays, which fails as `[__C.qos_class_t(rawValue: 17), ...]`. It now asserts each recorded call individually with a message naming the observed and expected raw classes. `deletionCalls == 2` already covers the count. Warning: on a saturated 3-vCPU runner `.userInitiated` still sits below the Flutter UI and raster threads of the full-app TEST_HOST, so the QoS change may narrow the window rather than close it. Accepted. The complement added here is test-side only and touches no production code or timeout: `BuzzPushSnapshotEnrichmentTests` gains a class `setUp` that runs a once-per-process `BuzzTestHostSettleBarrier`. The barrier dispatches the same hop the bridge's replies take (a user-initiated serial queue, then `DispatchQueue.main`), times it, and proceeds once three consecutive hops complete in under 0.5 s, capped at 60 s. It is a readiness gate on the measured symptom, not a sleep: on an idle host it costs about 10-60 ms (measured 0.04-0.38 s here under a load average of about 25), and in the CI window it would wait only while hops are actually slow. The thresholds come from the failing runs: the trivial hop measured 1.3 s and then over 10 s during the ~40 s after host launch; RunnerTests began about 9 s after launch and this class about 5 s later, so the cap covers the whole window with margin. 0.5 s leaves 20x headroom under the 10 s waits. If the host never settles, the tests run anyway and fail with the real symptom, and the barrier prints one line ("gave up after N s (M slow hops)") so the next reader can tell starvation from a missed callback. Residual risk: a burst that recurs mid-test or outlasts the cap is not covered; the remaining levers are then the test host itself or the wait length. Considered and declined: deriving `callbackTimeout` from a measured host-load factor. That still waits inside each test and only scales the same timeout, so it is a timeout increase in disguise and gives no signal about why the host was slow. Verification (Xcode 27.2, iOS 26.2 simulator runtime, 16-core host; the earlier commit's "iOS 27.2 simulator" meant the 27.2 SDK): - Barrier behaviour, with temporary variants not committed: forcing the first four hops to sleep 0.6 s gave "settled after 2.43 s (4 slow hops)"; the same with a 2 s cap gave "gave up after 2.00 s (3 slow hops)" and the class still ran and passed. - Class in isolation with the final code: 10/10 green, barrier settled in 0.04-0.38 s with 0 slow hops. - Full RunnerTests bundle with CI's `-parallel-testing-enabled NO`: 2/2 green (101 tests, about 22 s), barrier settled in about 0.06 s. - No swiftformat or swiftlint configuration in the repo; `git diff --check` is clean. Co-Authored-By: Claude Fable 5.1 Signed-off-by: Matt Toohey --- .../BuzzCommunicationNotificationTests.swift | 58 ++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/mobile/ios/RunnerTests/BuzzCommunicationNotificationTests.swift b/mobile/ios/RunnerTests/BuzzCommunicationNotificationTests.swift index 8ade99d81f..e3e1317d33 100644 --- a/mobile/ios/RunnerTests/BuzzCommunicationNotificationTests.swift +++ b/mobile/ios/RunnerTests/BuzzCommunicationNotificationTests.swift @@ -242,12 +242,63 @@ final class BuzzCommunicationNotificationTests: XCTestCase { } } +/// RunnerTests load the full Flutter app as their host. On CI's 3-vCPU runners +/// its launch saturated the machine for about 40 s (both 2026-11-01 failures): +/// a trivial queue-to-main hop took 1.3 s, then over 10 s, while a synchronous +/// main-thread test slowed from 0.013 s to 2.65 s, and the bridge's replies +/// missed their 10 s waits. This barrier measures that same hop, at the QoS +/// the bridge uses, and lets the class begin only once three consecutive hops +/// are fast again. On an idle host it costs a few milliseconds. The cap keeps a +/// host that never settles from hanging the run; its tests then fail with the +/// real symptom, and the printed line says the barrier gave up. +enum BuzzTestHostSettleBarrier { + /// A hop this slow leaves too little margin under the class's 10 s waits. + static let slowHop: TimeInterval = 0.5 + static let consecutiveFastHops = 3 + static let cap: TimeInterval = 60 + + /// Waits once per process; later calls return immediately. + static func waitUntilSettled() { + _ = settled + } + + private static let settled: Void = { + let queue = DispatchQueue(label: "xyz.block.buzz.test-host-settle", qos: .userInitiated) + let started = Date() + let deadline = started.addingTimeInterval(cap) + var fastHops = 0 + var slowHops = 0 + while fastHops < consecutiveFastHops, Date() < deadline { + let hopStarted = Date() + let hop = XCTestExpectation(description: "queue to main hop") + queue.async { DispatchQueue.main.async { hop.fulfill() } } + _ = XCTWaiter().wait(for: [hop], timeout: max(deadline.timeIntervalSinceNow, 0)) + if Date().timeIntervalSince(hopStarted) < slowHop { + fastHops += 1 + } else { + fastHops = 0 + slowHops += 1 + } + } + let outcome = fastHops < consecutiveFastHops ? "gave up" : "settled" + print( + "BuzzTestHostSettleBarrier \(outcome) after \(Date().timeIntervalSince(started)) s" + + " (\(slowHops) slow hops)" + ) + }() +} + /// Callbacks only record what they receive and assertions run after the wait, /// so a callback that lands after its test returned cannot fail a later test. /// The timeout is a cap; a passing test returns as soon as its callback runs. final class BuzzPushSnapshotEnrichmentTests: XCTestCase { private let callbackTimeout: TimeInterval = 10 + override class func setUp() { + super.setUp() + BuzzTestHostSettleBarrier.waitUntilSettled() + } + func testAgeSignalRequestDoesNotDependOnNotificationStorage() { let delegate = MissingAppGroupDelegate() var completed = false @@ -373,7 +424,12 @@ final class BuzzPushSnapshotEnrichmentTests: XCTestCase { XCTAssertFalse(BuzzAgeRestrictionSession.isRestricted(containerURL: directory)) XCTAssertEqual(deletionCalls, 2) // Flutter awaits these replies; a lower QoS left them starved on busy hosts. - XCTAssertEqual(deletionQoS, [QOS_CLASS_USER_INITIATED, QOS_CLASS_USER_INITIATED]) + for qos in deletionQoS { + XCTAssertEqual( + qos, QOS_CLASS_USER_INITIATED, + "deletion ran at QoS class \(qos.rawValue), expected user-initiated (\(QOS_CLASS_USER_INITIATED.rawValue))" + ) + } XCTAssertEqual(try Data(contentsOf: snapshotURL), original) }