Introducing a new way to review your PRs.

Read the announcement

ayugram / ayugramdesktop

Desktop Telegram client with good customization and Ghost mode.

View on GitHub

13 documents · Updated Nov 16, 2026

docs / media-calls-webviews-editor

Media, Calls, Webviews, Editor, and Third-Party Media Boundaries

Overview

This area owns Telegram Desktop's high-bandwidth UI and native-integration surfaces: media playback and transcoding under Telegram/SourceFiles/media/**, one-to-one and group calls under Telegram/SourceFiles/calls/**, photo/video editors under Telegram/SourceFiles/editor/**, FFmpeg wrappers under Telegram/SourceFiles/ffmpeg/**, WebAuthn/caBLE under Telegram/SourceFiles/webauthn/**, Instant View / markdown / Ton Site webview launching under Telegram/SourceFiles/iv/**, and the checked-out third-party boundary directories Telegram/ThirdParty/**. The Telegram/lib_lottie, Telegram/lib_qr, Telegram/lib_webrtc, Telegram/lib_webview, and sampled Telegram/ThirdParty/{tgcalls,libfido2,libcbor,QR,MicroTeX,cmark-gfm} directories are empty in this checkout, so references to those libraries enter through include boundaries such as lottie/lottie_icon.h, qr/qr_generate.h, webrtc/..., webview/..., <tgcalls/...>, latex.h, and platform FIDO APIs.

Media playback has two main paths. Audio tracks are coordinated by the singleton Media::Player::Instance in Telegram/…/media_player_instance.h and Telegram/…/media_player_instance.cpp; start() initializes Media::Audio::Start, audio capture, and the singleton, while Instance tracks separate song and voice queues, sparse shared-media playlist slices, seek state, repeat/shuffle, listen tracking, and last-playback-position persistence. It also subscribes to Core::App().calls().currentCallValue() and currentGroupCallValue() and pauses/resumes voice and music when calls start or end. The low-level audio mixer and device lifecycle live in Telegram/…/media_audio.h, with FFmpeg-backed loaders in Telegram/…/media_audio_ffmpeg_loader.cpp reading from memory, byte buffers, or files through custom AVIOContext callbacks.

Video and round-media playback use the streaming subsystem. Media::Streaming::Instance in Telegram/…/media_streaming_instance.h and .cpp is a lightweight handle around a shared Media::Streaming::Document obtained from Data::Session::streaming() for documents, alternate-quality documents, and photos. It forwards play/pause/resume/stop, frame requests, waiting-spinner state, speed, quality-switch requests, and per-view priority/locking to a shared Media::Streaming::Player. Telegram/…/media_streaming_player.h defines the player as the bridge between a file-reading thread and the main thread: FileDelegate callbacks open streams, push FFmpeg packets, report buffering/full-cache state, and compute audio/video received and played positions; the main thread renders frames, drives timers, and emits Update events. Remote media loads through Telegram/…/media_streaming_loader_mtproto.cpp, which subclasses the MTProto download task, queues requested byte offsets by priority, reads already cached parts from Storage::StreamedFileDownloader, and emits speed estimates based on completed download parts.

Animated previews, GIF/WebM-style clips, and thumbnails share FFmpeg helpers. Telegram/…/ffmpeg_utility.h and .cpp own RAII wrappers for AV packets, formats, codecs, frames, scaling, resampling, error logging, custom IO, and hardware decode setup. RestrictToCustomIO() in ffmpeg_utility.cpp clears FFmpeg's protocol_whitelist before demuxing so DASH/HLS or other demuxers cannot fetch referenced network URLs or local files. Telegram/…/ffmpeg_frame_generator.cpp implements Ui::FrameGenerator for byte-backed video/WebM data, bounds decoded area, handles rotation, scales into premultiplied BGRA QImages, and loops by seeking back to the start. Telegram/…/media_clip_ffmpeg.cpp is the clip reader used for animated inline media: it skips a small number of invalid packets, seeks back at EOF to loop, enforces different frame-area caps for inspecting vs inline playback, and treats VP9/WebM alpha specially. Transcoding and send-time processing are centralized in Telegram/…/media_video_encode.cpp: it creates temporary files in a private tdtranscode directory, chooses H.264 via libopenh264 or FFmpeg's H.264 encoder and VP9 WebM via libvpx-vp9, estimates output size, applies crop/trim/quality data, transcodes audio into AAC when needed, and has a special WebM-sticker CRF ladder.

The editor layer consumes those media services rather than owning codecs. Telegram/…/photo_editor.h and .cpp create a PhotoEditor widget with PhotoEditorContent, controls, brush/tool state, color picker, and PhotoModifications done/cancel streams. Brush and text preferences are serialized into Core::App().settings().photoEditorBrush() using a versioned QDataStream format; new fields are appended so older state can still deserialize. The drawable model is Telegram/…/scene.h, a QGraphicsScene that owns numbered items, canvas, shape drafting, text editing, undo/redo save states, and blur-source callbacks. Telegram/…/video_editor.h and .cpp wrap local video content with Media::Streaming::Instance and media_streaming_loader_local, a Crop, VideoTimeline, quality slider, frame capture/bubble preview, and VideoModifications collection. Video editor frames are requested through Media::Streaming::FrameRequest; final send/export behavior depends on Media::Encode::SourceInfo and media_video_encode.cpp.

Calls are orchestrated by Calls::Instance in Telegram/…/calls_instance.h and .cpp. It starts outgoing one-to-one calls, joins or creates group/conference calls, handles MTProto call updates, owns the current Calls::Call and Calls::GroupCall, requests microphone/camera permissions, caches DH config/server config, plays call sounds through Media::Audio::Track, and shares a weakly cached tgcalls::VideoCaptureInterface across camera/screen use. One-to-one calls live in Telegram/…/calls_call.h and .cpp: they run the MTProto DH handshake (phone.requestCall, phone.acceptCall, phone.confirmCall), validate hashes/fingerprints, then create a tgcalls controller using the agreed auth key and protocol layer. Telegram/…/calls_controller_webrtc.cpp adapts Telegram call endpoint/config structs into Webrtc::CallContext::Config; signaling bytes from updatePhoneCallSignalingData are forwarded into the running controller.

Group calls are much larger and are centered on Telegram/…/calls_group_call.h and .cpp. GroupCall joins via MTProto phone.joinGroupCall/connection updates, parses join-response JSON into either RTC video endpoints or broadcast/RTMP stream details, and creates tgcalls::GroupInstanceCustomImpl descriptors with audio level callbacks, media-device IDs, camera capture, video codec preferences, optional TdE2E encrypt/decrypt, and tasks for current time, audio/video broadcast parts, and media-channel descriptions. Broadcast playback requests map back into MTPupload_GetFile with MTP_inputGroupCallStream in broadcastPartStart(). Screen sharing is a second GroupInstanceCustomImpl with a loopback audio device module and VideoContentType::Screencast. Group-call chat/message support and conference encryption are adjacent in Telegram/SourceFiles/calls/group/calls_group_messages*.{h,cpp} and calls_group_message_encryption.*.

WebAuthn/passkeys have two native branches under Telegram/SourceFiles/webauthn/**. Telegram/…/webauthn_common.h exposes caBLE and security-key entry points; comments document that Linux/macOS always use libfido2 and Windows uses webauthn.dll where present with libfido2 fallback. caBLE public API is Telegram/…/cable.h, with register/login requests carrying RP/user/client-data-hash/algorithm fields and outcomes for QR success, cancellation, security-key fallback, Bluetooth absence, or failure. Telegram/…/cable_core.h and .cpp implement CBOR, SHA/HMAC/HKDF, P-256 ECDH, AES-GCM, Noise KNpsk0/NKpsk0 handshakes, caBLE QR/advert derivation constants, and encrypted CTAP framing. Telegram/…/cable_tunnel.cpp implements the TLS WebSocket tunnel to caBLE-assigned domains with subprotocol fido.cable, validates the Sec-WebSocket-Accept header, bounds upgrade and frame payload sizes, masks client frames, and surfaces binary messages. Telegram/…/cable_box.cpp is the UI boundary: it renders a QR with qr/qr_generate.h, uses a lottie passkey icon via lottie/lottie_icon.h, offers security-key fallback, and drives sheet state for QR, connecting, continue, and error screens.

Instant View and embedded web content are mediated by Telegram/SourceFiles/iv/**. Iv::Instance in Telegram/…/iv_instance.h and .cpp opens native IV pages, rich-message markdown, local markdown files, and Ton Sites; it tracks active windows per Main::Session, in-flight full-page and rich-message requests, HTML exports, and markdown windows. Rich-message media activation in iv_instance.cpp routes photos/documents into the media viewer and external URLs through click handlers, while Ton Site URLs are delegated to a webview controller. Telegram/…/iv_controller.h owns an Iv::Controller with Webview::Window, Webview::StorageId, navigation buttons, and event types for close/open/join/report; actual webview implementation is outside the covered source in the webview/... boundary. Markdown Instant View is handled by Telegram/…/iv_markdown_controller.h, which opens top-level Ui::RpWindows with history, search, zoom, source/share options, and prepared article content. The prepare pipeline in Telegram/…/iv_markdown_prepare.cpp converts either parsed markdown or native rich pages into bounded render data; it caps table dimensions, visual nesting, and total prepared blocks. Formula rendering crosses into MicroTeX through Telegram/…/iv_markdown_microtex.cpp, which initializes bundled resources, refuses very long TeX strings, rewrites full-width equation environments, caps logical/physical image size, and paints formulas into Qt images.

Coupling

Risks

  • Telegram/…/ffmpeg_utility.cpp, Telegram/…/media_audio_ffmpeg_loader.cpp, and Telegram/…/media_clip_ffmpeg.cpp: if any demux path skips RestrictToCustomIO(), a crafted media file can make FFmpeg resolve referenced network segments or local files. The code comments explicitly call this an IP-leak/local-file-read vector.
  • Telegram/…/calls_group_call.cpp: group-call streaming depends on several asynchronous tgcalls callbacks returning to the main thread and then into MTProto requests (requestAudioBroadcastPart, requestVideoBroadcastPart, requestMediaChannelDescriptions). If task cancellation or _broadcastParts bookkeeping is wrong, tgcalls can wait on a part whose MTProto request was already cancelled or whose LoadPartTask has been erased.
  • Telegram/…/cable_tunnel.cpp: the WebSocket tunnel is implemented manually, including HTTP upgrade parsing, frame lengths, masking, ping/pong, and close handling. If length bounds or accept-header validation regress, passkey ceremonies can fail silently or accept malformed tunnel traffic.
  • Telegram/…/iv_markdown_microtex.cpp: MicroTeX parsing happens synchronously before render caps apply. The file mitigates this with kMaxFormulaTexLength; raising or removing that cap can stall the UI on hostile markdown before image-size limits are reached.