Files and media pipeline
Overview
This doc owns the file/media core under TMessagesProj/…/File*.java, Image*.java, MediaController.java, DocumentObject.java, DownloadObject.java, AnimatedFileDrawableStream.java, VideoEditedInfo.java, VideoEncodingService.java, audioinfo/**, camera/**, video/**, secretmedia/**, and the native media codecs/utilities in TMessagesProj/…/audio.c, video.c, image.cpp, gifvideo.cpp, lottie.cpp, ffmpeg/**, exoplayer/**, libwebp/**, mozjpeg/**, opus/**, and rlottie/**.
The download/upload spine is FileLoader.java. It is per-account, owns a FilePathDatabase, maps Telegram objects to stable file names/paths, and runs work on the static fileLoaderQueue. Its MEDIA_DIR_* constants route images, audio, video, documents, cache, public image/video folders, and saved files. ImageLoader.java initializes those media dirs, registers as the FileLoader.FileLoaderDelegate, mirrors file progress through NotificationCenter.fileLoadProgressChanged, and completes UI image loads when fileDidLoaded calls back.
Downloads enter through FileLoader.loadFile(...) or FileLoader.loadStreamFile(...). loadFileInternal(...) deduplicates by attach filename in loadOperationPaths, chooses a small/large FileLoaderPriorityQueue by datacenter and size, chooses the storage directory from object type and cacheType, and creates a FileLoadOperation. The operation downloads chunks via ConnectionsManager, writes temp/parts/iv/preload files on FileLoadOperation.filesQueue, decrypts Telegram secret media with AES-IGE and local encrypted cache files with AES-CTR, verifies CDN chunks against upload.getCdnFileHashes, ungzips lottie/svg responses, then renames temp files to their final path. FileLoaderPriorityQueue.java gates concurrency using MessagesController.smallQueueMaxActiveOperations and largeQueueMaxActiveOperations; high/stream priorities pause lower-priority work.
Uploads enter through FileLoader.uploadFile(...), which deduplicates encrypted and plain paths separately and allows one normal plus one small upload at a time. FileUploadOperation.java reads from the source file, resumes recent uploads from the uploadinfo shared preferences, chooses part size from Telegram's max part count and slow-network/speed-boost state, encrypts secret uploads with AES-IGE when requested, sends TL_upload_saveFilePart or TL_upload_saveBigFilePart, and returns either InputFile or InputEncryptedFile to the FileLoader delegate. Incremental producers call checkUploadNewDataAvailable(...) so estimated-size uploads can continue as bytes arrive.
Path routing is centralized in FileLoader.getPathToAttach(...) and getPathToMessage(...). Documents with localPath bypass cache lookup; encrypted documents and temporary/invalid photo locations force MEDIA_DIR_CACHE; voice, video, and regular documents route to audio/video/document dirs; photos and profile photos route to image or cache depending on size and secrecy. FilePathDatabase.java persists custom/public save paths in paths(document_id, dc_id, type, path) and dialog metadata in paths_by_dialog_id, so later path lookups can resolve files moved to public gallery or Files storage. FilesMigrationService.java moves the legacy app folder into the scoped-storage external files folder on Android R+, and FileRefController.java refreshes stale Telegram file references for send/edit/download requests by parent object key.
Image loading starts at UI code through ImageReceiver.setImage(...). ImageReceiver.java computes image, media, and thumb keys, retains/recycles drawable use counts, observes heavy-operation notifications for animations, and draws the selected media/image/thumb/static drawable with optional shaders, rounded corners, crossfade, lottie, or animated-file readiness callbacks. ImageLoader.loadImageForImageReceiver(...) first checks lottieMemCache, memCache, smallImagesMemCache, and wallpaperMemCache; when no cache hit exists it builds URLs/keys from ImageLocation.java, chooses extension/cache behavior, deduplicates CacheImage work by URL/key/tag, and either decodes a local/cache file, starts a FileLoader.loadFile, schedules an HTTP image/file task, or queues quality-thumb generation. ImageLocation.java is the routing descriptor for direct paths, secure documents, web files, Telegram photos/documents, avatar videos, sticker-set thumbs, stripped/path thumbs, and AyuGram local file hooks.
Media streaming to playback uses two bridges. ExtendedDefaultDataSource.java routes ExoPlayer file: URIs ending in .enc to EncryptedFileDataSource.java, regular file/content/asset/raw/http URIs to their ExoPlayer data sources, and tg: URIs to FileStreamLoadOperation.java. FileStreamLoadOperation reconstructs a TL_document from URI query parameters (account, rid, id, hash, dc, size, mime, reference, name), calls FileLoader.loadStreamFile(...), blocks reads on FileLoadOperationStream.newDataAvailable(), and serves bytes from the temp/final file. Animated playback uses AnimatedFileDrawableStream.java, which performs the same offset-based loadStreamFile loop for native AnimatedFileDrawable decoders and cancels both document and mp4 thumb loads when the drawable is no longer needed.
Media capture, gallery enumeration, recording, conversion, and playback metadata live in MediaController.java. It queries Android MediaStore into AlbumEntry, PhotoEntry, SearchImage, and AudioEntry; records voice messages with AudioRecord plus native Opus encoder entry points; generates Opus waveforms and persists them back through MessagesStorage; and queues video conversion with scheduleVideoConvert(...). VideoEditedInfo.java serializes trim/crop/filter/paint/entity/photo/round-video state for outgoing media. VideoEncodingService.java is only the foreground notification/progress service: it observes upload progress for the converted file path and stops on NotificationCenter.stopEncodingService.
Video conversion is implemented in MediaController.convertVideo(...) and video/MediaCodecVideoConvertor.java. The controller computes duration, rotation, result size, FPS limiting, and whether compression is needed, then writes progress through didWriteData(...) as MediaCodecVideoConvertor encodes. The converter uses Android MediaExtractor/MediaCodec, InputSurface.java, OutputSurface.java, and TextureRenderer.java to decode frames, apply GL filters/crop/paint/text/sticker/emoji overlays, encode H.264, and mux through MP4Builder.java, Mp4Movie.java, Track.java, and Sample.java. AudioRecoder.java handles audio recoding in the same package. MP4Builder writes ftyp, chunked mdat, and final movie boxes, optionally splitting mdat for secret output.
Camera capture is split between legacy Android camera control and custom GL recording. camera/CameraController.java initializes and caches camera sizes in shared preferences, opens/configures photo or round camera sessions, writes still JPEGs and image-cache entries, starts either a standard MediaRecorder path or a CameraView GL recorder, generates a video thumbnail on completion, and handles max-duration/max-size recorder callbacks. camera/CameraSession.java tracks orientation, flash/torch, zoom, recorder profiles, and photo/round/video parameter configuration. camera/CameraView.java renders preview via a TextureView/EGL thread and contains an internal encoder that records camera frames plus AudioRecord audio through MediaCodec and MP4Builder.
Audio metadata helpers under audioinfo/** parse local music files without Android MediaMetadataRetriever: AudioInfo.getAudioInfo(...) dispatches MP3 files to mp3/MP3Info.java for ID3v2/ID3v1 tags, cover art, and frame-derived duration, and M4A/MP4 files to m4a/M4AInfo.java for atom-based duration and ilst tags. DocumentObject.java supplies render helpers for theme documents and SVG/path thumbs; DownloadObject.java is a small DTO for queued downloadable TL objects.
Native boundaries are narrow but important. jni/audio.c backs MediaController.startRecord, writeFrame, stopRecord, isOpusFile, getWaveform, and getWaveform2, using opus/** and ogg/opusfile to encode voice notes and derive 100-sample packed waveforms. jni/video.c backs Utilities.convertVideoFrame using libyuv to convert ARGB frames into planar or semi-planar YUV for encoder color formats. jni/image.cpp backs Utilities bitmap pin/unpin, WebP decode through libwebp/**, JPEG/WebP-linked image helpers, blur/CDT/gradient/invert routines, and depends on mozjpeg/**. jni/gifvideo.cpp backs AnimatedFileDrawable by opening FFmpeg/libav contexts, optionally reading through the Java AnimatedFileDrawableStream, seeking, and decoding frames into Android bitmaps. jni/lottie.cpp backs RLottieDrawable creation, color replacement, optional LZ4 frame cache files, and frame rendering through rlottie/**. jni/ffmpeg/**, jni/exoplayer/**, jni/libwebp/**, jni/mozjpeg/**, jni/opus/**, and jni/rlottie/** are vendored codec/parser libraries consumed by those JNI wrappers and ExoPlayer native extensions.
Coupling
- If a new media type, extension, or cache policy is added, update both
FileLoader.getPathToAttach(...)andImageLoader.loadImageForImageReceiver(...): FileLoader.java decides final directories and ImageLoader.java independently decides cache file names, animation/lottie/svg type, and whether to force cache storage. - If
FileLoadOperationtemp/final naming changes, verifyFileLoader.loadFileInternal(...),FileLoader.getPathToAttach(...), EncryptedFileDataSource.java, and AnimatedFileDrawableStream.java; streaming and encrypted playback reopen partially downloaded files by the same names and key-file convention. - If
FilePathDatabaseschema or path-save semantics change, verifyFileLoader.loadFileInternal(...)custom public/gallery saves andImageLoadergallery insertion onfileDidLoaded(...); stale path rows can route future lookups away from the actual file. - If ExoPlayer URI formats change, update ExtendedDefaultDataSource.java and FileStreamLoadOperation.java together:
tg:streaming depends on query parameters matching the document reconstruction code, while.encfile playback depends onEncryptedFileDataSourcefinding<encrypted-file-name>.keyinFileLoader.getInternalCacheDir(). - If
VideoEditedInfoserialization fields change, verifyMediaController.convertVideo(...),MediaCodecVideoConvertor.convertVideo(...), OutputSurface.java, and TextureRenderer.java; crop/filter/paint/entity/photo flags are passed positionally through this conversion path. - If native JNI signatures or package/class names change, update both Java declarations and C/C++ exports:
MediaController↔ jni/audio.c,Utilities↔ jni/video.c/jni/image.cpp,AnimatedFileDrawable/AnimatedFileDrawableStream↔ jni/gifvideo.cpp, andRLottieDrawable↔ jni/lottie.cpp. - If camera recording output changes, verify both CameraController.java paths: standard
MediaRecorderrecording and CameraView.java's GL/MediaCodecrecorder both call back into the same thumbnail/duration finish path but produce MP4s through different encoders.
Risks
- FileLoadOperation.java finalization retries
renameToand falls back around lines that copy/delete temp files; if this logic regresses, downloads can report success while leaving temp files or losing custom public-path saves, and FilePathDatabase.java can persist a path that was never finalized. - FileLoadOperation.java CDN handling decrypts and verifies chunks before continuing; if
cdnHasheslookup or AES-CTR offset math drifts, corrupted CDN bytes are written into media files and only the SHA-256 check protects playback. - EncryptedFileInputStream.java increments
fileOffsetby the requested length rather than the actual bytes read; callers currently bound reads through EncryptedFileDataSource.java, but changing read loops without preserving that bound can desynchronize CTR/CBC decryption near EOF or short reads. FilePathDatabase.getPath(..., useQueue=true)blocks on a latch and explicitly guards against calls from its own dispatch queue; calling it from new synchronous media/database code can deadlock path resolution.- AnimatedFileDrawableStream.java blocks native decoder reads on
CountDownLatchuntilFileLoadOperationStream.newDataAvailable()fires; cancel paths must remove stream listeners and wake latches, or animated/video drawables can hang decode threads. - MediaCodecVideoConvertor.java, OutputSurface.java, and TextureRenderer.java depend on EGL, codec timeouts, and device encoder quirks; wrong result dimensions, rotation, or timeout changes can fail conversion only on specific devices and leave VideoEncodingService.java running until cancellation.