Files
buzz/mobile/ios/Runner/AppDelegate.swift
Tom Brow 52cb6818fa perf(mobile): keep remote images on disk and show small ones as thumbnails (#9158)
## Summary

The mobile app now saves remote images on disk. After a relaunch,
avatars, custom emoji, message attachments and agent artwork show
immediately. The app does not download them again.

Small images on a relay now load only the relay's thumbnail. The app
never downloads the full-size upload to show a small image.

## Why

Before this change, the app kept remote images only in memory. Each cold
launch downloaded every image again. On a slow connection, attachments
showed a "Loading image" placeholder for a long time.

Also, when a relay had no thumbnail for an image, the app downloaded the
full-size upload. That upload can be up to 20 MiB, for an avatar that is
a few dozen points wide.

## What changes

### Images on disk

- `MediaImageProvider` reads and writes a `MediaDiskCache`. Each account
in each community has its own cache, at `<root>/communities/<community
id>/media/<signer pubkey>`. The cache is built on
`flutter_cache_manager`, with a JSON index.
- On iOS, the root is the app group's `Library/Caches` (from a new
`buzz/app_group` method channel). App extensions can then read the same
cache. On other platforms, the root is the app cache directory.
- Each cache holds a maximum of 250 MB. When it is full, the least
recently used images go first. `MediaCacheIndex` measures each file when
it writes it, because `flutter_cache_manager` limits only the number of
files. Every disk read marks the file used, and every write schedules
eviction, including a refresh that replaces bytes the package already
holds in memory (by default the package does neither for those).
- The cache does not save an image that is larger than 5 MiB. The app
still shows that image from memory. Thus, one large image cannot push
out all the others.
- Relay media URLs contain a content hash, so their bytes never change.
The cache keeps them until it evicts them. For other URLs, the cache
shows the saved image, and then gets a new copy in the background after
one week. A maximum of two of these refreshes run at the same time. The
app decodes a refreshed image before it replaces the saved copy.
- When you remove a community, the app deletes its cache. When a
community gets a new signing key, the app deletes the media of the old
account. At launch, the app also deletes caches of communities and
accounts that are no longer present.
- Startup does not wait for the cache. The cache directory resolves in
the background. Image reads wait for it for a maximum of 3 seconds.

### Thumbnails for small images

- A small relay image loads the relay's WebP thumbnail. If the relay has
no WebP thumbnail, the image loads the JPEG thumbnail. If the relay has
neither, the widget shows its placeholder (initials for an avatar, an
empty tile for artwork).
- Native avatars drawn by UIKit (the navigation bar, the community icon
on the create button, and the New DM and Add Members pickers) use the
same choice, through a new `smallMediaImageProviders`. They try the same
thumbnails, fall back the same way, and show initials when neither
thumbnail exists. Before, they downloaded the full-size upload and only
decoded it small.
- Larger layouts and the full-screen viewer load the full-size upload,
as before.

### Agent artwork

- Preset agent artwork (the tray, flight, video placeholder, avatar hero
and the avatar editor's blurred backdrop) now loads through
`MediaImage`, so it uses the disk cache too.
- The create-agent and empty-bestie pages warm the same
`MediaImageProvider` that their widgets read, through a new
`mediaImageProviderFor(ref, url)`.
- No code in `mobile/lib` uses `Image.network` or `NetworkImage` now.

## Demo

A cold relaunch (the app is stopped, then started) into a synthetic "QA
demo" Solo chat that holds a few generated images. Shown at half speed.

Before: the attachment shows a "Loading image" placeholder while it
downloads.


https://github.com/user-attachments/assets/107f43a3-63d0-40c3-bd80-4ea9d7897a97

After: the attachment shows with the chat, from disk.


https://github.com/user-attachments/assets/d3985bb2-838c-4efa-93cb-44cef51f677d

## Tests

- `test/shared/relay/media_disk_cache_test.dart` follows one journey
through the real provider and a real cache in a temporary directory. The
first launch downloads and saves. An offline relaunch shows the
full-size image, its thumbnail and an external image from disk, and the
oversized image goes back to the network. Eight days later, only the
external image refreshes. Removing the community deletes its directory.
- A second test checks the order of eviction when the cache is full,
that a file read from disk is evicted after files that were not, and
that replacing a file's bytes after eviction last ran evicts again when
the cache goes over budget.
- `community_provider_test` checks that launch deletes orphan media
directories, that a new signing key deletes only the old account's
media, and that removing a community deletes its media.
- `avatar_image_test` checks that a missing WebP thumbnail falls back to
the JPEG thumbnail, and that an avatar with no thumbnail shows its
initials.
- `native_avatar_image_test` checks the same for native avatars: a WebP
404 paints the JPEG thumbnail, the full-size upload is never requested,
a relay with no thumbnail paints initials and is not asked again, and
any other failure stops at the initials.
- `flutter analyze` and the full mobile test suite pass.
- Automated app runs on an iPhone 17 Pro simulator (iOS 27): the channel
list, a DM, Home, the Agents grid and an agent details sheet show their
images. After a relaunch, the images came from disk and the app wrote no
new files. In the "QA demo" chat, a new image from the app showed
immediately.
- Automated run of the native-thumbnail fix on an iPhone 17 Pro
simulator (iOS 27): the native New DM picker shows photo avatars, and
the cache gained only relay `.thumb.webp` entries for them, no full-size
uploads. After a cold relaunch, the picker showed the same avatars
within about 1.5 seconds, and the cache directory gained no files. The
avatar editor backdrop was not exercised in the app. The test community
has no remote agents, and the backdrop tests cover only emoji and
picked-image selections.

## Before merge

- [x] Automated simulator run of this head: avatars and agent artwork
show their thumbnails, and a cold relaunch shows images from disk.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Tom Brow <tomb@squareup.com>
Co-authored-by: Tom Brow <tomb@squareup.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: studio <dc2fc481343672df31a689840c09299972e704dac592e0da6f18486646a7ee3e@localhost>
2026-11-09 07:21:21 -08:00

1111 lines
37 KiB
Swift

import AuthenticationServices
import AVFoundation
import BuzzPushKit
import DeclaredAgeRange
import Flutter
import UIKit
import UserNotifications
import os.log
@main
@objc class AppDelegate: FlutterAppDelegate, FlutterImplicitEngineDelegate {
private var nativeMessagePresentationCoordinator: NativeMessagePresentationCoordinator?
private var mediaUploadChannel: FlutterMethodChannel?
private var pushChannel: FlutterMethodChannel?
private let apnsRegistrationBuffer = APNsRegistrationBuffer()
private let pushNavigationBuffer = BuzzPushNavigationBuffer()
private var apnsDeviceToken: Data?
private lazy var endpointGrantStore = BuzzPushEndpointGrantKeychainStore(
accessGroup: Bundle.main.object(forInfoDictionaryKey: "BuzzKeychainAccessGroup") as? String
)
private var enrollmentTask: Task<Void, Never>?
var appGroupIdentifier: String? {
Bundle.main.object(forInfoDictionaryKey: "BuzzAppGroupIdentifier") as? String
}
private var pushKeychainAccessGroup: String? {
Bundle.main.object(forInfoDictionaryKey: "BuzzKeychainAccessGroup") as? String
}
private lazy var pushSnapshotBridge = BuzzPushSnapshotBridge(
appGroupIdentifier: appGroupIdentifier,
endpointGrantStore: endpointGrantStore,
keychainAccessGroup: pushKeychainAccessGroup
)
private var communityKeyboardChannel: FlutterMethodChannel?
private var communityKeyboardObserver: NSObjectProtocol?
private var hapticsChannel: FlutterMethodChannel?
private var appGroupChannel: FlutterMethodChannel?
private var qrScannerChannel: FlutterMethodChannel?
private var inlinePhotoPickerSupportChannel: FlutterMethodChannel?
private var ageSignalChannel: FlutterMethodChannel?
var requestPlatformAgeSignal: @MainActor (UIViewController) async throws -> [String: Any] =
AppDelegate.platformAgeSignal
private var ageSignalTask: Task<Void, Never>?
private var ageSignalRequestID: UUID?
private var ageSignalResult: FlutterResult?
private var concentricSheetSurfaceChannel: FlutterMethodChannel?
private var nativeAttachmentPopoverCoordinator: NativeAttachmentPopoverCoordinator?
private var nativeEmojiPickerCoordinator: NativeEmojiPickerCoordinator?
private var nativeCreateChannelSheetCoordinator: NativeCreateChannelSheetCoordinator?
private var nativeAddMembersSheetCoordinator: NativeAddMembersSheetCoordinator?
private var nativeConfirmationDialogCoordinator: NativeConfirmationDialogCoordinator?
private var nativeProfileSetupCoordinator: NativeProfileSetupCoordinator?
private var nativeCommunityEntryCoordinator: NativeCommunityEntryCoordinator?
private var nativeProfileTextEditorCoordinator: NativeProfileTextEditorCoordinator?
private var identityFilePlugin: IdentityFilePlugin?
private var hostedHpkeChannel: FlutterMethodChannel?
private var browserAuthBridge: BuzzBrowserAuthBridge?
private var huddleMediaPlugin: HuddleMediaPlugin?
override func application(
_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
) -> Bool {
// Age checking and notification restoration run asynchronously from
// Flutter. No age-related storage or platform request may delay launch.
UNUserNotificationCenter.current().delegate = self
return super.application(application, didFinishLaunchingWithOptions: launchOptions)
}
func didInitializeImplicitFlutterEngine(_ engineBridge: FlutterImplicitEngineBridge) {
GeneratedPluginRegistrant.register(with: engineBridge.pluginRegistry)
engineBridge.pluginRegistry.registrar(forPlugin: "BuzzToastLayer")?.register(
ToastLayerFactory(), withId: "buzz/toast_layer")
if let registrar = engineBridge.pluginRegistry.registrar(forPlugin: "BuzzIosMediaCarousel") {
registrar.register(IosMediaCarouselFactory(messenger: registrar.messenger()), withId: "buzz/ios_media_carousel")
}
let messenger = engineBridge.applicationRegistrar.messenger()
let keyboardChannel = FlutterMethodChannel(
name: "buzz/community_keyboard", binaryMessenger: messenger)
communityKeyboardChannel = keyboardChannel
if let observer = communityKeyboardObserver {
NotificationCenter.default.removeObserver(observer)
}
communityKeyboardObserver = NotificationCenter.default.addObserver(
forName: UIResponder.keyboardWillShowNotification, object: nil, queue: .main
) { _ in
// Deliver after UIKit finishes preparing the keyboard in this run-loop turn.
DispatchQueue.main.async {
keyboardChannel.invokeMethod("willShow", arguments: nil)
}
}
identityFilePlugin = IdentityFilePlugin(messenger: messenger)
browserAuthBridge = BuzzBrowserAuthBridge(messenger: messenger)
hostedHpkeChannel = FlutterMethodChannel(name: "buzz/hosted_hpke", binaryMessenger: messenger)
hostedHpkeChannel?.setMethodCallHandler { call, result in
if call.method == "isSupported" {
if #available(iOS 17.0, *) { result(true) } else { result(false) }
return
}
guard call.method == "seal" else { result(FlutterMethodNotImplemented); return }
guard #available(iOS 17.0, *) else {
result(FlutterError(code: "unsupported", message: "Backup requires iOS 17 or later", details: nil)); return
}
guard let args = call.arguments as? [String: Any],
let recipient = args["recipient"] as? FlutterStandardTypedData,
let plaintext = args["plaintext"] as? FlutterStandardTypedData,
let aad = args["aad"] as? FlutterStandardTypedData else {
result(FlutterError(code: "invalid_input", message: nil, details: nil)); return
}
do {
let sealed = try HostedHpkeBackup.seal(recipient: recipient.data, plaintext: plaintext.data, aad: aad.data)
result(["enc": FlutterStandardTypedData(bytes: sealed.enc), "ciphertext": FlutterStandardTypedData(bytes: sealed.ciphertext)])
} catch {
result(FlutterError(code: "backup_unavailable", message: nil, details: nil))
}
}
huddleMediaPlugin = HuddleMediaPlugin(messenger: messenger)
if let registrar = engineBridge.pluginRegistry.registrar(forPlugin: "BuzzIosNavigationBar") {
registrar.register(IosNavigationBarFactory(messenger: messenger, parent: registrar.viewController), withId: "buzz/ios_navigation_bar")
}
mediaUploadChannel = FlutterMethodChannel(
name: "buzz/media_upload",
binaryMessenger: messenger
)
mediaUploadChannel?.setMethodCallHandler { [weak self] call, result in
self?.handleMediaUploadMethodCall(call, result: result)
}
pushChannel = FlutterMethodChannel(
name: "buzz/push",
binaryMessenger: messenger
)
pushChannel?.setMethodCallHandler { [weak self] call, result in
self?.handlePushMethodCall(call, result: result)
}
apnsRegistrationBuffer.attach { [weak self] update in
self?.pushChannel?.invokeMethod(update.method, arguments: update.arguments)
}
appGroupChannel = FlutterMethodChannel(
name: "buzz/app_group",
binaryMessenger: messenger
)
appGroupChannel?.setMethodCallHandler { [weak self] call, result in
guard call.method == "cachesDirectory" else {
result(FlutterMethodNotImplemented)
return
}
// Caches the app's extensions can read too. The OS may purge it.
let caches = self?.appGroupIdentifier
.flatMap {
FileManager.default.containerURL(forSecurityApplicationGroupIdentifier: $0)
}?
.appendingPathComponent("Library/Caches", isDirectory: true)
result(caches?.path)
}
hapticsChannel = FlutterMethodChannel(
name: "buzz/haptics",
binaryMessenger: messenger
)
hapticsChannel?.setMethodCallHandler { call, result in
guard call.method == "success" || call.method == "error" else {
result(FlutterMethodNotImplemented)
return
}
let generator = UINotificationFeedbackGenerator()
generator.prepare()
generator.notificationOccurred(call.method == "error" ? .error : .success)
result(nil)
}
qrScannerChannel = FlutterMethodChannel(
name: "buzz/qr_scanner",
binaryMessenger: messenger
)
qrScannerChannel?.setMethodCallHandler { call, result in
Self.handleQrScannerMethodCall(call, result: result)
}
inlinePhotoPickerSupportChannel = FlutterMethodChannel(
name: "buzz/inline_photo_picker",
binaryMessenger: messenger
)
inlinePhotoPickerSupportChannel?.setMethodCallHandler { call, result in
guard call.method == "isSupported" else {
result(FlutterMethodNotImplemented)
return
}
if #available(iOS 17.0, *) {
result(true)
} else {
result(false)
}
}
ageSignalChannel = FlutterMethodChannel(
name: "buzz/age_signal",
binaryMessenger: messenger
)
let ageSignalRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzAgeSignal"
)
ageSignalChannel?.setMethodCallHandler { [weak self] call, result in
self?.handleAgeSignalMethodCall(
call,
viewController: ageSignalRegistrar?.viewController,
result: result
)
}
if let inlinePhotoPickerRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzInlinePhotoPicker"
) {
inlinePhotoPickerRegistrar.register(
InlinePhotoPickerFactory(
messenger: messenger,
parentViewController: inlinePhotoPickerRegistrar.viewController
),
withId: "buzz/inline_photo_picker"
)
}
if let concentricSheetRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzConcentricSheetSurface"
) {
concentricSheetRegistrar.register(
ConcentricSheetSurfaceFactory(messenger: messenger),
withId: "buzz/concentric_sheet_surface"
)
concentricSheetSurfaceChannel = FlutterMethodChannel(
name: "buzz/concentric_sheet_surface",
binaryMessenger: messenger
)
concentricSheetSurfaceChannel?.setMethodCallHandler { call, result in
guard call.method == "isSupported" else {
result(FlutterMethodNotImplemented)
return
}
if #available(iOS 26.0, *) {
result(true)
} else {
result(false)
}
}
}
if let jumpToLatestGlassRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzJumpToLatestGlassButton"
) {
jumpToLatestGlassRegistrar.register(
JumpToLatestGlassButtonFactory(messenger: messenger),
withId: "buzz/jump_to_latest_glass"
)
}
if let navigationGlassRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNavigationGlassButton"
) {
navigationGlassRegistrar.register(
NavigationGlassButtonFactory(messenger: messenger),
withId: "buzz/navigation_glass"
)
}
if let segmentedControlRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeSegmentedControl"
) {
segmentedControlRegistrar.register(
NativeSegmentedControlFactory(messenger: messenger),
withId: "buzz/native_segmented_control"
)
}
if let skinToneRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeSkinToneControl"
) {
skinToneRegistrar.register(
NativeSkinToneControlFactory(messenger: messenger),
withId: "buzz/native_skin_tone_control"
)
}
if let stickyDateGlassRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzStickyDateGlassHeader"
) {
stickyDateGlassRegistrar.register(
StickyDateGlassHeaderFactory(messenger: messenger),
withId: "buzz/sticky_date_glass"
)
}
if let themePaginationGlassRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzThemePaginationGlassControl"
) {
themePaginationGlassRegistrar.register(
ThemePaginationGlassControlFactory(messenger: messenger),
withId: "buzz/theme_pagination_glass"
)
}
nativeMessagePresentationCoordinator = NativeMessagePresentationCoordinator(
messenger: messenger,
parentViewController: engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeMessagePresentation"
)?.viewController
)
let nativeAttachmentRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeAttachmentPopover"
)
nativeAttachmentPopoverCoordinator = NativeAttachmentPopoverCoordinator(
messenger: messenger,
parentViewController: nativeAttachmentRegistrar?.viewController
)
let nativeEmojiPickerRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeEmojiPicker"
)
nativeEmojiPickerCoordinator = NativeEmojiPickerCoordinator(
messenger: messenger,
parentViewController: nativeEmojiPickerRegistrar?.viewController
)
engineBridge.pluginRegistry.registrar(forPlugin: "BuzzNativePresenceMenu")?.register(
NativePresenceMenuFactory(messenger: messenger), withId: "buzz/presence_menu"
)
nativeCreateChannelSheetCoordinator = NativeCreateChannelSheetCoordinator(
messenger: messenger,
parentViewController: engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeCreateChannelSheet"
)?.viewController
)
nativeAddMembersSheetCoordinator = NativeAddMembersSheetCoordinator(
messenger: messenger,
parentViewController: engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeAddMembersSheet"
)?.viewController
)
nativeConfirmationDialogCoordinator = NativeConfirmationDialogCoordinator(
messenger: messenger,
parentViewController: engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeConfirmationDialog"
)?.viewController
)
nativeProfileSetupCoordinator = NativeProfileSetupCoordinator(messenger: messenger) { [weak self] engine in
let channel = FlutterMethodChannel(name: "buzz/media_upload", binaryMessenger: engine.binaryMessenger)
channel.setMethodCallHandler { [weak self] call, result in self?.handleMediaUploadMethodCall(call, result: result) }
}
nativeCommunityEntryCoordinator = NativeCommunityEntryCoordinator(messenger: messenger)
let nativeProfileTextEditorRegistrar = engineBridge.pluginRegistry.registrar(
forPlugin: "BuzzNativeProfileTextEditor"
)
nativeProfileTextEditorCoordinator = NativeProfileTextEditorCoordinator(
messenger: messenger,
parentViewController: nativeProfileTextEditorRegistrar?.viewController
)
}
func handleAgeSignalMethodCall(
_ call: FlutterMethodCall,
viewController: UIViewController?,
result: @escaping FlutterResult
) {
// iOS can retire the request in process. The generation fence prevents
// a late result from the cancelled task from completing a fresh request.
if call.method == "cancelAgeSignalRequest" || call.method == "restartForAgeSignal" {
cancelAgeSignalRequest()
result(true)
return
}
guard call.method == "requestAgeSignal" else {
result(FlutterMethodNotImplemented)
return
}
guard #available(iOS 26.0, *) else {
result(Self.noAgeSignalResponse)
return
}
guard let viewController else {
result(
FlutterError(
code: "age_signal_unavailable",
message: "The age signal presenter is unavailable.",
details: nil
)
)
return
}
let requestID = UUID()
ageSignalRequestID = requestID
ageSignalResult = result
let request = requestPlatformAgeSignal
ageSignalTask = Task { @MainActor [weak self] in
do {
let payload = try await request(viewController)
self?.completeAgeSignalRequest(requestID, value: payload)
} catch {
self?.completeAgeSignalRequest(
requestID,
value:
FlutterError(
code: "age_signal_unavailable",
message: "The age signal request failed.",
details: String(describing: type(of: error))
)
)
}
}
}
@MainActor
private static func platformAgeSignal(_ viewController: UIViewController) async throws -> [String: Any] {
guard #available(iOS 26.0, *) else { return noAgeSignalResponse }
let response = try await AgeRangeService.shared.requestAgeRange(ageGates: 18, in: viewController)
switch response {
case .declinedSharing:
return noAgeSignalResponse
case .sharing(let range):
return BuzzAgeSignalPayload.sharing(
exclusiveUpperBound: range.upperBound, lowerBound: range.lowerBound)
@unknown default:
throw NSError(domain: "BuzzAgeSignal", code: 1,
userInfo: [NSLocalizedDescriptionKey: "Unsupported age signal response"])
}
}
private func completeAgeSignalRequest(_ requestID: UUID, value: Any?) {
guard ageSignalRequestID == requestID, let result = ageSignalResult else { return }
ageSignalRequestID = nil
ageSignalResult = nil
ageSignalTask = nil
result(value)
}
private func cancelAgeSignalRequest() {
let result = ageSignalResult
ageSignalRequestID = nil
ageSignalResult = nil
ageSignalTask?.cancel()
ageSignalTask = nil
result?(
FlutterError(
code: "age_signal_cancelled",
message: "The age signal request was cancelled.",
details: nil
)
)
}
private static let noAgeSignalResponse: [String: Any] = [
"status": "noSignal",
"ageUpper": NSNull(),
]
private static func handleQrScannerMethodCall(
_ call: FlutterMethodCall,
result: @escaping FlutterResult
) {
switch call.method {
case "usesDynamicIslandQrScannerPortal":
result(
UIDevice.current.userInterfaceIdiom == .phone
&& usesDynamicIslandQrScannerPortal(
safeAreaTopInset: activeWindowSafeAreaTopInset()
)
)
case "setDynamicIslandScannerStatusBarHidden":
guard let hidden = call.arguments as? Bool else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected a Bool status-bar visibility value.",
details: nil
)
)
return
}
let controller = UIApplication.shared.connectedScenes
.compactMap { $0 as? UIWindowScene }
.flatMap(\.windows)
.compactMap { $0.rootViewController as? BuzzFlutterViewController }
.first
controller?.setScannerStatusBarHidden(hidden)
result(nil)
case "performDynamicIslandQrScanSuccessHaptic":
let generator = UINotificationFeedbackGenerator()
generator.prepare()
generator.notificationOccurred(.success)
result(nil)
default:
result(FlutterMethodNotImplemented)
}
}
static func usesDynamicIslandQrScannerPortal(
safeAreaTopInset: CGFloat
) -> Bool {
safeAreaTopInset > 50
}
private static func activeWindowSafeAreaTopInset() -> CGFloat {
UIApplication.shared.connectedScenes
.compactMap { $0 as? UIWindowScene }
.filter { $0.activationState == .foregroundActive }
.flatMap(\.windows)
.first(where: \.isKeyWindow)?
.safeAreaInsets.top ?? 0
}
override func application(
_ application: UIApplication,
didRegisterForRemoteNotificationsWithDeviceToken deviceToken: Data
) {
super.application(application, didRegisterForRemoteNotificationsWithDeviceToken: deviceToken)
apnsDeviceToken = deviceToken
apnsRegistrationBuffer.recordToken(deviceToken)
}
override func application(
_ application: UIApplication,
didFailToRegisterForRemoteNotificationsWithError error: Error
) {
super.application(application, didFailToRegisterForRemoteNotificationsWithError: error)
apnsRegistrationBuffer.recordError(error.localizedDescription)
}
override func userNotificationCenter(
_ center: UNUserNotificationCenter,
didReceive response: UNNotificationResponse,
withCompletionHandler completionHandler: @escaping () -> Void
) {
BuzzPushNotificationResponseCoordinator.handle(
actionIdentifier: response.actionIdentifier,
userInfo: response.notification.request.content.userInfo,
onTarget: { target in
pushNavigationBuffer.record(target)
deliverPushNavigationTarget(target)
},
forwardToFlutter: { pluginCompletion in
self.forwardPushNotificationResponseToFlutter(
center,
response: response,
completion: pluginCompletion
)
},
completion: completionHandler
)
}
private func forwardPushNotificationResponseToFlutter(
_ center: UNUserNotificationCenter,
response: UNNotificationResponse,
completion: @escaping () -> Void
) {
super.userNotificationCenter(
center,
didReceive: response,
withCompletionHandler: completion
)
}
private func deliverPushNavigationTarget(_ target: BuzzPushNavigationTarget) {
pushChannel?.invokeMethod(
"notificationOpened",
arguments: target.flutterArguments
) { [weak self] result in
guard result as? String == "handled" else { return }
self?.pushNavigationBuffer.remove(ifMatching: target)
}
}
private func handlePushMethodCall(
_ call: FlutterMethodCall,
result: @escaping FlutterResult
) {
if pushSnapshotBridge.handle(call, result: result) {
return
}
switch call.method {
case "startRegistration":
startPushRegistration(result: result)
case "takePendingNotificationResponse":
result(pushNavigationBuffer.take()?.flutterArguments)
case "endpointGrants":
do {
guard let arguments = call.arguments as? [String: Any],
let gatewayText = arguments["gatewayUrl"] as? String,
let gatewayURL = URL(string: gatewayText)
else { throw BuzzDevPushEnrollmentError.invalidGatewayURL }
let driver = try BuzzDevPushEnrollmentDriver(
gatewayBaseURL: gatewayURL,
store: endpointGrantStore,
appAttestKeychainAccessGroup: pushKeychainAccessGroup
)
result(try driver.endpointGrants().map(\.flutterArguments))
} catch {
result(
FlutterError(
code: "endpoint_grant_read_failed",
message: "Unable to read persisted push endpoint grants.",
details: error.localizedDescription
)
)
}
case "enrollPush":
handleDevPushEnrollment(call, result: result)
default:
result(FlutterMethodNotImplemented)
}
}
private func startPushRegistration(result: @escaping FlutterResult) {
UNUserNotificationCenter.current().requestAuthorization(options: [.alert, .badge, .sound]) {
_, error in
if let error {
os_log(
"Buzz notification authorization request failed: %{public}@",
type: .error,
error.localizedDescription
)
}
}
// APNs token registration is independent from display authorization. A
// denied or failed prompt must not prevent gateway enrollment and leases.
UIApplication.shared.registerForRemoteNotifications()
result(nil)
}
private func handleDevPushEnrollment(
_ call: FlutterMethodCall,
result: @escaping FlutterResult
) {
guard enrollmentTask == nil else {
result(
FlutterError(
code: "enrollment_in_progress",
message: "Development push enrollment is already running.",
details: nil
)
)
return
}
guard let deviceToken = apnsDeviceToken else {
result(
FlutterError(
code: "missing_apns_token",
message: "APNs has not supplied a device token.",
details: nil
)
)
return
}
guard !deviceToken.isEmpty else {
result(
FlutterError(
code: "invalid_apns_token",
message: "APNs supplied an empty device token.",
details: nil
)
)
return
}
guard let arguments = call.arguments as? [String: Any],
let relayText = arguments["relayUrl"] as? String,
let relayURL = URL(string: relayText),
let gatewayText = arguments["gatewayUrl"] as? String,
let gatewayURL = URL(string: gatewayText)
else {
result(
FlutterError(
code: "invalid_arguments",
message: "Development push enrollment requires relayUrl and gatewayUrl.",
details: nil
)
)
return
}
do {
let driver = try BuzzDevPushEnrollmentDriver(
gatewayBaseURL: gatewayURL,
store: endpointGrantStore,
appAttestKeychainAccessGroup: Bundle.main.object(
forInfoDictionaryKey: "BuzzKeychainAccessGroup"
) as? String
)
enrollmentTask = Task { [weak self] in
defer { self?.enrollmentTask = nil }
do {
let record = try await driver.enroll(
deviceToken: deviceToken,
relayURL: relayURL
)
await MainActor.run { result(record.flutterArguments) }
} catch {
await MainActor.run {
result(
FlutterError(
code: "dev_enrollment_failed",
message: "Development push enrollment failed.",
details: error.localizedDescription
)
)
}
}
}
} catch {
result(
FlutterError(
code: "dev_enrollment_configuration_failed",
message: "Development push enrollment is not configured.",
details: error.localizedDescription
)
)
}
}
private func handleMediaUploadMethodCall(
_ call: FlutterMethodCall,
result: @escaping FlutterResult
) {
switch call.method {
case "sanitizeImageForUpload":
guard
let arguments = call.arguments as? [String: Any],
let typedData = arguments["bytes"] as? FlutterStandardTypedData,
let mimeType = arguments["mimeType"] as? String
else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected image bytes and mime type.",
details: nil
)
)
return
}
guard let image = UIImage(data: typedData.data) else {
result(
FlutterError(
code: "sanitize_failed",
message: "Unable to decode picked image.",
details: nil
)
)
return
}
do {
guard let sanitizedData = try MediaSanitizer.sanitizeImage(image, mimeType: mimeType) else {
result(
FlutterError(
code: "sanitize_failed",
message: "Unable to sanitize picked image.",
details: mimeType
)
)
return
}
result(FlutterStandardTypedData(bytes: sanitizedData))
} catch {
result(
FlutterError(
code: "sanitize_failed",
message: "Unable to sanitize picked image.",
details: mimeType
)
)
}
case "transcodeImageToJpeg":
guard let typedData = call.arguments as? FlutterStandardTypedData else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected raw image bytes.",
details: nil
)
)
return
}
guard let image = UIImage(data: typedData.data) else {
result(
FlutterError(
code: "transcode_failed",
message: "Unable to convert picked image to JPEG.",
details: nil
)
)
return
}
do {
guard let jpegData = try MediaSanitizer.encodeJpeg(image) else {
result(
FlutterError(
code: "transcode_failed",
message: "Unable to convert picked image to JPEG.",
details: nil
)
)
return
}
result(FlutterStandardTypedData(bytes: jpegData))
} catch {
result(
FlutterError(
code: "transcode_failed",
message: "Unable to convert picked image to JPEG.",
details: nil
)
)
}
case "transcodeVideoToMp4":
guard let sourcePath = call.arguments as? String else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected source file path as String.",
details: nil
)
)
return
}
transcodeVideoToMp4(sourcePath: sourcePath, result: result)
case "packageVoiceNoteForUpload":
guard let sourcePath = call.arguments as? String else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected source file path as String.",
details: nil
)
)
return
}
VoiceNotePackager.package(sourcePath: sourcePath, result: result)
case "generateVideoPoster":
guard let sourcePath = call.arguments as? String else {
result(
FlutterError(
code: "invalid_arguments",
message: "Expected source file path as String.",
details: nil
)
)
return
}
generateVideoPoster(sourcePath: sourcePath, result: result)
case "clipboardHasImage":
result(UIPasteboard.general.hasImages)
case "readClipboardImage":
guard let imageData = Self.clipboardImageData(from: UIPasteboard.general) else {
result(nil)
return
}
result(FlutterStandardTypedData(bytes: imageData))
default:
result(FlutterMethodNotImplemented)
}
}
static func clipboardImageData(from pasteboard: UIPasteboard) -> Data? {
if let pngData = pasteboard.data(forPasteboardType: "public.png") {
return pngData
}
if let jpegData = pasteboard.data(forPasteboardType: "public.jpeg") {
return jpegData
}
for imageType in ["public.heic", "public.heif", "org.webmproject.webp", "com.compuserve.gif"] {
if let imageData = pasteboard.data(forPasteboardType: imageType) {
return imageData
}
}
guard let image = pasteboard.image else {
return nil
}
return image.pngData()
}
private func transcodeVideoToMp4(
sourcePath: String,
result: @escaping FlutterResult
) {
let sourceURL = URL(fileURLWithPath: sourcePath)
let asset = AVURLAsset(url: sourceURL)
// Do not export the source asset directly. An iPhone video can carry GPS,
// spatial-video, and other data tracks even when its user-visible metadata
// is cleared. A fresh composition copies only one video and one audio
// track, so those private channels cannot reach the relay.
let composition = AVMutableComposition()
guard
let sourceVideo = asset.tracks(withMediaType: .video).first,
let destinationVideo = composition.addMutableTrack(
withMediaType: .video,
preferredTrackID: kCMPersistentTrackID_Invalid
)
else {
result(
FlutterError(
code: "transcode_failed",
message: "The selected file does not contain a video track.",
details: nil
)
)
return
}
do {
let sourceAudio = asset.tracks(withMediaType: .audio).first
let insertionTimes = Self.relativeTrackInsertionTimes(
videoStart: sourceVideo.timeRange.start,
audioStart: sourceAudio?.timeRange.start
)
try destinationVideo.insertTimeRange(
sourceVideo.timeRange,
of: sourceVideo,
at: insertionTimes.video
)
destinationVideo.preferredTransform = sourceVideo.preferredTransform
if let sourceAudio,
let destinationAudio = composition.addMutableTrack(
withMediaType: .audio,
preferredTrackID: kCMPersistentTrackID_Invalid
)
{
try destinationAudio.insertTimeRange(
sourceAudio.timeRange,
of: sourceAudio,
at: insertionTimes.audio ?? .zero
)
}
} catch {
result(
FlutterError(
code: "transcode_failed",
message: error.localizedDescription,
details: nil
)
)
return
}
guard
let exportSession = AVAssetExportSession(
asset: composition,
// Passthrough preserves the source's HEVC codec and container
// metadata. Buzz accepts only canonical H.264/AAC MP4s with no
// metadata channels, so re-encode instead of copying the movie.
presetName: AVAssetExportPresetMediumQuality
)
else {
result(
FlutterError(
code: "transcode_failed",
message: "Unable to create export session.",
details: nil
)
)
return
}
let outputURL = FileManager.default.temporaryDirectory
.appendingPathComponent(UUID().uuidString)
.appendingPathExtension("mp4")
exportSession.outputURL = outputURL
exportSession.outputFileType = .mp4
exportSession.shouldOptimizeForNetworkUse = true
// `forSharing()` intentionally retains playback metadata. The relay
// rejects every descriptive metadata channel to avoid leaking location or
// other private information, so write no source metadata at all.
exportSession.metadata = []
exportSession.metadataItemFilter = nil
exportSession.exportAsynchronously {
switch exportSession.status {
case .completed:
do {
// AVFoundation writes a standard sample-dependency table (`sdtp`).
// Older Buzz relays mistook that playback-only box for metadata. Keep
// its size and payload in a `free` box so chunk offsets stay valid and
// uploads work before those relays receive the validator fix.
try MP4Canonicalizer.neutralizeSampleDependencyBoxes(at: outputURL)
result(outputURL.path)
} catch {
try? FileManager.default.removeItem(at: outputURL)
result(
FlutterError(
code: "transcode_failed",
message: "Unable to canonicalize transcoded video.",
details: error.localizedDescription
)
)
}
default:
let errorMessage =
exportSession.error?.localizedDescription
?? "Video transcoding failed with status \(exportSession.status.rawValue)."
result(
FlutterError(
code: "transcode_failed",
message: errorMessage,
details: nil
)
)
// Clean up partial output on failure.
try? FileManager.default.removeItem(at: outputURL)
}
}
}
static func relativeTrackInsertionTimes(
videoStart: CMTime,
audioStart: CMTime?
) -> (video: CMTime, audio: CMTime?) {
guard let audioStart else {
return (video: .zero, audio: nil)
}
let timelineStart =
CMTimeCompare(audioStart, videoStart) < 0 ? audioStart : videoStart
return (
video: CMTimeSubtract(videoStart, timelineStart),
audio: CMTimeSubtract(audioStart, timelineStart)
)
}
private func generateVideoPoster(
sourcePath: String,
result: @escaping FlutterResult
) {
DispatchQueue.global(qos: .userInitiated).async {
let asset = AVURLAsset(url: URL(fileURLWithPath: sourcePath))
let generator = AVAssetImageGenerator(asset: asset)
generator.appliesPreferredTrackTransform = true
generator.maximumSize = CGSize(width: 720, height: 720)
generator.requestedTimeToleranceBefore = .positiveInfinity
generator.requestedTimeToleranceAfter = .positiveInfinity
do {
let durationSeconds = CMTimeGetSeconds(asset.duration)
let middleTime =
durationSeconds.isFinite && durationSeconds > 0
? min(durationSeconds / 2, 1)
: 0
let candidateTimes = [0, 0.1, middleTime]
var posterImage: CGImage?
var lastError: Error?
for seconds in candidateTimes {
do {
posterImage = try generator.copyCGImage(
at: CMTime(seconds: seconds, preferredTimescale: 600),
actualTime: nil
)
if posterImage != nil { break }
} catch {
lastError = error
}
}
guard let posterImage else {
throw lastError
?? NSError(
domain: "BuzzVideoPoster",
code: 1,
userInfo: [NSLocalizedDescriptionKey: "Unable to decode a video frame."]
)
}
guard let jpegData = try MediaSanitizer.encodeJpeg(UIImage(cgImage: posterImage)) else {
throw NSError(
domain: "BuzzVideoPoster",
code: 2,
userInfo: [NSLocalizedDescriptionKey: "Unable to encode video poster."]
)
}
DispatchQueue.main.async {
result(FlutterStandardTypedData(bytes: jpegData))
}
} catch {
DispatchQueue.main.async {
result(
FlutterError(
code: "poster_failed",
message: "Unable to create a video preview.",
details: error.localizedDescription
)
)
}
}
}
}
}
extension BuzzPushNavigationTarget {
var flutterArguments: [String: String] {
var arguments = [
"eventId": eventID,
"communityId": communityID,
"channelId": channelID,
]
arguments["threadRootId"] = threadRootID
return arguments
}
}