Introducing a new way to review your PRs.

Read the announcement

ayugram / ayugramdesktop

Desktop Telegram client with good customization and Ghost mode.

View on GitHub

13 documents · Updated Nov 16, 2026

docs / storage-persistence-media-transfer

Storage persistence and media transfer

Overview

This area owns Telegram Desktop's local tdata/ persistence format, in-memory media indexes, file upload/download plumbing, local media preparation, and the export takeout download path.

Local persistence is split into global, domain, and per-account boundaries:

  • Global settings are read and written by Telegram/…/localstorage.cpp. Local::start() creates tdata/, reads settings, falls back to old config and old user/MTP files when needed, and applies global theme/background/language/fallback-config state. Local::writeSettings() writes the current global settings under tdata/settings* with an encrypted payload and also saves AyuGram settings before serializing the Telegram settings blocks.
  • Domain/account discovery is handled by Telegram/…/storage_domain.cpp. The modern domain key file is tdata/key_<dataName>*: it stores passcode salt, an encrypted local storage key, and a local-key-encrypted account list plus active account index. Domain::start() first tries this modern file, falls back to legacy single-account startup through Account::legacyStart(), and rewrites accounts when the stored version is older than AppVersion.
  • Per-account storage is in Telegram/…/storage_account.cpp. Account computes an account base path tdata/<dataName hash>/, a per-account database path tdata/user_<dataName>/, and a temp path tdata/temp_<dataName>/. Its encrypted map file is an index of generated FileKey values for drafts, locations/download state, prefs, stickers, trusted peers, export settings, search suggestions, playback positions, bot storage, and other account-local state. Each value file is separately encrypted with the domain local key; the map only records which generated key belongs to which logical bucket.
  • The wire format utilities live in Telegram/…/storage_file_utilities.cpp. Files have a TDF$ header, app version, QDataStream payload, and md5 trailer. FileWriteDescriptor stages payloads in memory and hands them to an async WriteManager, which writes a modern safe ...s file using QSaveFile and removes legacy ...0/...1 backups on success. ReadFile() prefers the ...s file, otherwise tries the legacy pair by modification time and validates magic/version/md5 before exposing a QDataStream.
  • Encryption is local-key based in storage_file_utilities.cpp: CreateLocalKey() derives the modern key with salted SHA-512/PBKDF2-HMAC-SHA512, while CreateLegacyLocalKey() uses the older SHA1 PBKDF2 path. PrepareEncrypted() prefixes plaintext length, pads to AES block size, stores the first 16 bytes of SHA1 as the data key, and encrypts with MTP::aesEncryptLocal; DecryptLocal() reverses this and rejects size or hash mismatches.

The per-account map is intentionally a directory of encrypted blobs rather than one monolith. For example, Account::writeLocations() serializes file-location records, aliases, and the downloads-manager snapshot into the _locationsKey blob, then writeMap() persists the _locationsKey pointer. Account::readLocations() restores local file paths into _fileLocations and _fileLocationPairs; writeFileLocation(), readFileLocation(), and removeFileLocation() are the boundary for downloaded media path persistence. Export settings are another map-managed blob: Account::writeExportSettings()/readExportSettings() serialize Export::Settings, including media flags, output format/path, availability time, optional single peer, and date bounds.

File locking is a separate platform abstraction in Telegram/…/storage_file_lock.h, storage_file_lock_posix.cpp, and storage_file_lock_win.cpp. Both implementations reserve the first four bytes (FileLock::kSkipBytes) as the lock range. POSIX uses fcntl(F_SETLK) and may terminate the locking pid; Windows uses LockFile and Restart Manager shutdown against the file owner.

In-memory media indexes are exposed through Storage::Facade in Telegram/…/storage_facade.cpp. It wraps SharedMedia (storage_shared_media.cpp) and UserPhotos (storage_user_photos.cpp), which keep sparse message/photo id slices in memory and publish RPL updates. The shared-media lists use SparseIdsList from storage_sparse_ids_list.cpp, where known no-skip ranges merge into slices and queries can return skipped counts when the top or bottom boundary is known. These indexes are not the encrypted on-disk store; they cache server-backed slices for UI queries.

The generic download entry point is CreateFileLoader() in Telegram/…/file_download.cpp. It dispatches a DownloadLocation to:

  • mtpFileLoader in file_download_mtproto.cpp for Telegram storage files, bot/web files served through MTProto, geo-point tiles, and audio-album thumbnails.
  • webFileLoader in file_download_web.cpp for plain HTTP(S) URLs.
  • an in-memory loader for already available bytes.

FileLoader first tries local cache when LoadToCacheAsWell is enabled, using session->data().cache() and a subclass-provided cache key. Cache values may be full bytes or partial: prefixed bytes. If local cache misses and cloud loading is allowed, the subclass starts network loading. Incoming parts are written either to _data for in-memory loads or to _file for LoadToFileOnly; finalizeResult() stores local file locations through session->local().writeFileLocation(), writes small completed or partial results back into the media cache, closes downloaded files, and fires task completion.

MTProto downloads are coordinated by Telegram/…/download_manager_mtproto.cpp. DownloadManagerMtproto keeps per-DC queues, priorities, and adaptive download session counts. DownloadMtprotoTask::sendRequest() issues upload.getFile, upload.getWebFile, or CDN requests with a fixed kDownloadPartSize of 128 KiB. It handles CDN redirects by switching DC/token/key state, decrypting CDN parts with AES-CTR, verifying CDN file hashes, and reuploading/requesting hashes when the server asks. mtpFileLoader implements the task callbacks: it advances offsets, writes parts through FileLoader::writeResultPart(), marks completion on short/empty parts or requested size, and refreshes stale file references through Data::UpdatedFileReferences before retrying.

Plain web downloads use WebLoadManager in file_download_web.cpp, a shared manager with its own QNetworkAccessManager thread, at most eight concurrent requests, and a 5-redirect limit. Only http and https URLs are accepted; redirects are resolved and revalidated to avoid unsafe schemes. Non-streaming plain web loads are capped to Storage::kMaxFileInMemory, while streaming-to-file loads are capped to 4 GiB and pass incremental QNetworkReply::readAll() chunks to webFileLoader::loadProgress().

Streamed media save-as/download is bridged by Telegram/…/streamed_file_downloader.cpp. StreamedFileDownloader wraps a media streaming Reader, requests up to 32 streaming parts concurrently, writes each loaded part through FileLoader, exposes readLoadedPart() for already saved chunks, and finalizes only after every streaming part has been saved.

Local upload preparation starts with TaskQueue and FileLoadTask in Telegram/…/localimageloader.cpp, with drag/drop and list preparation helpers in storage_media_prepare.cpp. TaskQueue processes media inspection and thumbnail work on a worker thread and calls finish() back on the queue owner thread. FileLoadTask::process() determines MIME/media type from a file path or in-memory bytes, prepares images/photos, videos, voices, stickers, thumbnails, attached-sticker metadata, optional animation/video transcode jobs, and archive requests. Its result is FilePrepareResult, which carries file bytes/path, generated ids, parts/md5, thumbnail parts, MTP photo/document stubs, and send options.

Uploads are sent by Storage::Uploader in Telegram/…/file_upload.cpp. Uploader::upload() creates local upload state on PhotoData/DocumentData, queues optional video transcode or folder archive preparation, then maybeSend() schedules file parts across upload DC sessions. Photos and thumbnails use pre-sliced 32 KiB in-memory parts from FilePrepareResult; documents are streamed from memory or QFile with a size-dependent part size and use upload.saveFilePart or upload.saveBigFilePart above 30 MiB. The uploader adaptively adds/removes upload sessions based on fast/slow request timing, emits progress/failure RPL streams, and fires ready events that call ApiWrap::sendUploadedPhoto, sendUploadedDocument, secure upload completion, secondary-file completion, or video-cover upload.

Export is a separate takeout/download pipeline under Telegram/SourceFiles/export/. Export::Manager (export_manager.cpp) owns one Controller and UI panel at a time, starting whole-account, single-peer, or topic export. ControllerObject in export_controller.cpp normalizes settings, creates an output writer, drives high-level steps (initialization, dialogs list, personal info, userpics, stories, profile music, contacts, sessions, other data, dialogs, or a single topic), maps progress into ProcessingState, and stops on API or output errors.

The export MTProto work is in Telegram/…/export_api_wrap.cpp. ApiWrap::startMainSession() opens an account takeout session, setting flags according to requested data and media size limit. Subsequent requests are wrapped in InvokeWithTakeout; dialog requests may also be wrapped in message split ranges. Userpics, stories, profile music, messages, custom emoji, rich messages, and other data are converted into export/data/export_data_types.* models, then file payloads are downloaded by processFileLoad(). Export files write through Output::File (export/…/export_output_file.cpp), which prepares a collision-free relative path, opens/appends the output file, writes blocks, flushes, and increments export stats.

Export media download is intentionally independent from FileLoader. ApiWrap::loadFile() creates a FileProcess, then loadFilePart() requests upload.getFile with kFileChunkSize (8 * 128 KiB) through the takeout session and shifted export media DC. filePartDone() writes ordered chunks into Output::File, reports progress, caches previously downloaded locations in LoadedFileCache, and finishes when the known size is reached. Export does not support CDN redirects in this path. If a FILE_REFERENCE_* error occurs, filePartRefreshReference() reloads the owning story/message/custom emoji/rich message and retries with a refreshed Data::FileLocation; otherwise unavailable files are marked with a skip reason.

Coupling

  • If the on-disk file envelope changes in Telegram/…/storage_file_utilities.cpp, update all readers and writers that assume TDF$, version, payload, and md5 semantics: storage_domain.cpp, storage_account.cpp, and localstorage.cpp all call ReadFile()/FileWriteDescriptor directly and will reject files if the envelope or QDataStream version drifts.
  • If account-map key tags change in Telegram/…/storage_account.cpp, update both readMapWith() and writeMap() together. The map is the only index from logical state (drafts, locations, export settings, prefs, bot storage) to encrypted value-file keys, so adding a writer without a reader leaks data, and adding a reader without a map writer loses the state on the next rewrite.
  • If Core::FileLocation serialization changes, verify Account::writeLocations() and Account::readLocations() in Telegram/…/storage_account.cpp together with FileLoader::finalizeResult() in Telegram/…/file_download.cpp. Downloads rely on this path both to remember saved-to-disk media and to recover the downloads-manager serialized snapshot.
  • If DownloadLocation, StorageFileLocation, or cache-key generation changes, update CreateFileLoader() (file_download.cpp), mtpFileLoader::cacheKey()/fileLocationKey() (file_download_mtproto.cpp), and the MTProto request construction in download_manager_mtproto.cpp; otherwise a media object can download successfully but be cached under the wrong key or lose its persisted local file location.
  • If MTProto download part sizing changes, check Storage::kDownloadPartSize in download_manager_mtproto.h, CDN hash checking in download_manager_mtproto.cpp, partial cache resume in mtpFileLoader::startLoadingWithPartial(), and FileLoader::finalizeResult() partial-cache writes. CDN hash validation and partial-cache offsets assume one fixed part size.
  • If upload file-size thresholds or part sizes change in Telegram/…/file_upload.cpp, verify FilePrepareResult::setFileData() in localimageloader.cpp and MTProto input-file construction in Uploader::finishFront(). Small files need md5 hashes and inputFile; files above kUseBigFilesFrom use inputFileBig and no md5.
  • If local media MIME/type detection changes in Telegram/…/localimageloader.cpp or storage_media_prepare.cpp, verify Uploader::upload() and Uploader::finishFront() in file_upload.cpp. The uploader trusts FilePrepareResult::type, fileparts, thumbparts, document, and photo fields to choose progress state and final API calls.
  • If Export::Settings changes in Telegram/…/export_settings.h, update both persistence in Account::writeExportSettings()/readExportSettings() (storage_account.cpp) and normalization/step selection in export_controller.cpp/export_api_wrap.cpp. Saved export settings and takeout flags must agree on media masks, single-peer/topic fields, date limits, and size limits.
  • If export file locations or file-reference refresh rules change in export/data/export_data_types.*, also update ApiWrap::ComputeLocationKey(), processFileLoad(), filePartRefreshReference(), and the Data::RefreshFileReference() call sites in export_api_wrap.cpp. Export de-duplicates downloaded files and retries stale references by comparing parsed Data::FileLocation values.
  • If the output writer contract changes in export/…/export_output_file.cpp, verify all Output::AbstractWriter implementations and ApiWrap::writePreloadedFile()/filePartDone(). Export progress and final stats count files/bytes through Output::File, not through the generic FileLoader cache.

Risks

  • Telegram/…/storage_sparse_ids_list.cpp contains an AyuGram-specific guard replacing an Expects(noSkipRange.from <= noSkipRange.till) crash with an early return in SparseIdsList::addRangeItemsAndCountNew(). If shared-media no-skip ranges are changed to rely on invalid ranges being visible, those entries will now be silently dropped and shared media counts can diverge from server state.
  • Telegram/…/file_download_web.cpp explicitly restricts web downloads and redirects to HTTP(S) because other schemes can expose local/UNC paths. If IsSupportedWebUrl() or redirect handling is relaxed, plain web media loads can become a credential-disclosure or local-file access path.
  • Telegram/…/export_api_wrap.cpp treats upload.fileCdnRedirect as a hard export error in filePartDone(), while the normal app downloader supports CDN redirects in download_manager_mtproto.cpp. If export begins receiving CDN redirects for takeout media, affected exports will stop instead of downloading those files.
  • Telegram/…/storage_file_lock_posix.cpp and storage_file_lock_win.cpp can terminate processes holding the target file lock. If this lock is reused for user-selected files rather than app-owned coordination files, another application holding the file may be killed unexpectedly.