mirror of
https://github.com/block/buzz.git
synced 2026-11-06 09:21:45 +00:00
test(mobile): gate push snapshot tests on a settled host, name the observed QoS
Follow-up to f0bf2c0c8 from its review (no blockers; one suggestion and
one warning, both addressed here).
Suggestion: the QoS assertion in
`testRestrictionCleanupFailurePreservesSnapshotAndRestoreReleasesAuthority`
compared two raw `qos_class_t` arrays, which fails as
`[__C.qos_class_t(rawValue: 17), ...]`. It now asserts each recorded call
individually with a message naming the observed and expected raw classes.
`deletionCalls == 2` already covers the count.
Warning: on a saturated 3-vCPU runner `.userInitiated` still sits below the
Flutter UI and raster threads of the full-app TEST_HOST, so the QoS change
may narrow the window rather than close it. Accepted. The complement added
here is test-side only and touches no production code or timeout:
`BuzzPushSnapshotEnrichmentTests` gains a class `setUp` that runs a
once-per-process `BuzzTestHostSettleBarrier`. The barrier dispatches the
same hop the bridge's replies take (a user-initiated serial queue, then
`DispatchQueue.main`), times it, and proceeds once three consecutive hops
complete in under 0.5 s, capped at 60 s. It is a readiness gate on the
measured symptom, not a sleep: on an idle host it costs about 10-60 ms
(measured 0.04-0.38 s here under a load average of about 25), and in the
CI window it would wait only while hops are actually slow. The thresholds
come from the failing runs: the trivial hop measured 1.3 s and then over
10 s during the ~40 s after host launch; RunnerTests began about 9 s after
launch and this class about 5 s later, so the cap covers the whole window
with margin. 0.5 s leaves 20x headroom under the 10 s waits. If the host
never settles, the tests run anyway and fail with the real symptom, and the
barrier prints one line ("gave up after N s (M slow hops)") so the next
reader can tell starvation from a missed callback. Residual risk: a burst
that recurs mid-test or outlasts the cap is not covered; the remaining
levers are then the test host itself or the wait length.
Considered and declined: deriving `callbackTimeout` from a measured
host-load factor. That still waits inside each test and only scales the
same timeout, so it is a timeout increase in disguise and gives no signal
about why the host was slow.
Verification (Xcode 27.2, iOS 26.2 simulator runtime, 16-core host; the
earlier commit's "iOS 27.2 simulator" meant the 27.2 SDK):
- Barrier behaviour, with temporary variants not committed: forcing the
first four hops to sleep 0.6 s gave "settled after 2.43 s (4 slow hops)";
the same with a 2 s cap gave "gave up after 2.00 s (3 slow hops)" and the
class still ran and passed.
- Class in isolation with the final code: 10/10 green, barrier settled in
0.04-0.38 s with 0 slow hops.
- Full RunnerTests bundle with CI's `-parallel-testing-enabled NO`: 2/2
green (101 tests, about 22 s), barrier settled in about 0.06 s.
- No swiftformat or swiftlint configuration in the repo; `git diff --check`
is clean.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
This commit is contained in:
@@ -242,12 +242,63 @@ final class BuzzCommunicationNotificationTests: XCTestCase {
|
||||
}
|
||||
}
|
||||
|
||||
/// RunnerTests load the full Flutter app as their host. On CI's 3-vCPU runners
|
||||
/// its launch saturated the machine for about 40 s (both 2026-11-01 failures):
|
||||
/// a trivial queue-to-main hop took 1.3 s, then over 10 s, while a synchronous
|
||||
/// main-thread test slowed from 0.013 s to 2.65 s, and the bridge's replies
|
||||
/// missed their 10 s waits. This barrier measures that same hop, at the QoS
|
||||
/// the bridge uses, and lets the class begin only once three consecutive hops
|
||||
/// are fast again. On an idle host it costs a few milliseconds. The cap keeps a
|
||||
/// host that never settles from hanging the run; its tests then fail with the
|
||||
/// real symptom, and the printed line says the barrier gave up.
|
||||
enum BuzzTestHostSettleBarrier {
|
||||
/// A hop this slow leaves too little margin under the class's 10 s waits.
|
||||
static let slowHop: TimeInterval = 0.5
|
||||
static let consecutiveFastHops = 3
|
||||
static let cap: TimeInterval = 60
|
||||
|
||||
/// Waits once per process; later calls return immediately.
|
||||
static func waitUntilSettled() {
|
||||
_ = settled
|
||||
}
|
||||
|
||||
private static let settled: Void = {
|
||||
let queue = DispatchQueue(label: "xyz.block.buzz.test-host-settle", qos: .userInitiated)
|
||||
let started = Date()
|
||||
let deadline = started.addingTimeInterval(cap)
|
||||
var fastHops = 0
|
||||
var slowHops = 0
|
||||
while fastHops < consecutiveFastHops, Date() < deadline {
|
||||
let hopStarted = Date()
|
||||
let hop = XCTestExpectation(description: "queue to main hop")
|
||||
queue.async { DispatchQueue.main.async { hop.fulfill() } }
|
||||
_ = XCTWaiter().wait(for: [hop], timeout: max(deadline.timeIntervalSinceNow, 0))
|
||||
if Date().timeIntervalSince(hopStarted) < slowHop {
|
||||
fastHops += 1
|
||||
} else {
|
||||
fastHops = 0
|
||||
slowHops += 1
|
||||
}
|
||||
}
|
||||
let outcome = fastHops < consecutiveFastHops ? "gave up" : "settled"
|
||||
print(
|
||||
"BuzzTestHostSettleBarrier \(outcome) after \(Date().timeIntervalSince(started)) s"
|
||||
+ " (\(slowHops) slow hops)"
|
||||
)
|
||||
}()
|
||||
}
|
||||
|
||||
/// Callbacks only record what they receive and assertions run after the wait,
|
||||
/// so a callback that lands after its test returned cannot fail a later test.
|
||||
/// The timeout is a cap; a passing test returns as soon as its callback runs.
|
||||
final class BuzzPushSnapshotEnrichmentTests: XCTestCase {
|
||||
private let callbackTimeout: TimeInterval = 10
|
||||
|
||||
override class func setUp() {
|
||||
super.setUp()
|
||||
BuzzTestHostSettleBarrier.waitUntilSettled()
|
||||
}
|
||||
|
||||
func testAgeSignalRequestDoesNotDependOnNotificationStorage() {
|
||||
let delegate = MissingAppGroupDelegate()
|
||||
var completed = false
|
||||
@@ -373,7 +424,12 @@ final class BuzzPushSnapshotEnrichmentTests: XCTestCase {
|
||||
XCTAssertFalse(BuzzAgeRestrictionSession.isRestricted(containerURL: directory))
|
||||
XCTAssertEqual(deletionCalls, 2)
|
||||
// Flutter awaits these replies; a lower QoS left them starved on busy hosts.
|
||||
XCTAssertEqual(deletionQoS, [QOS_CLASS_USER_INITIATED, QOS_CLASS_USER_INITIATED])
|
||||
for qos in deletionQoS {
|
||||
XCTAssertEqual(
|
||||
qos, QOS_CLASS_USER_INITIATED,
|
||||
"deletion ran at QoS class \(qos.rawValue), expected user-initiated (\(QOS_CLASS_USER_INITIATED.rawValue))"
|
||||
)
|
||||
}
|
||||
XCTAssertEqual(try Data(contentsOf: snapshotURL), original)
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user