tgnet Networking
Overview
This doc covers the Telegram networking stack owned by TMessagesProj/src/main/java/org/telegram/tgnet/**, TMessagesProj/jni/tgnet/**, TMessagesProj/…/TgNetWrapper.cpp, TMessagesProj/…/NativeLoader.cpp, TMessagesProj/…/CMakeLists.txt, and tlrpc-patch.py. The module owns TL object serialization/deserialization, request dispatch, MTProto connection/session management, datacenter configuration persistence, DNS fallback routing, request callbacks, and the JNI bridge between Java controllers and the native network loop.
The Java entry point is TMessagesProj/…/ConnectionsManager.java. It is a per-account singleton keyed by UserConfig.MAX_ACCOUNT_COUNT, builds device/app/lang/push metadata, chooses an account-specific config directory under ApplicationLoader.getFilesDirFixed(), then calls native_init(...) with TLRPC.LAYER, BuildVars.APP_ID, proxy settings from mainconfig, push connection settings from notification preferences, current user id, network status, and performance class. Runtime state is delegated to native methods declared in the same file: pausing/resuming, network availability, current time/DC, proxy settings, DC address overrides, request cancellation, DNS config application, and request sending all cross this boundary.
TL serialization on the Java side is defined by AbstractSerializedData.java, SerializedData.java, NativeByteBuffer.java, TLObject.java, TLRPC.java, and TLClassStore.java. TLRPC.java is the generated/large schema file; each object knows its constructor, readParams, serializeToStream, and often deserializeResponse. TLObject.getObjectSize() uses a thread-local NativeByteBuffer in calculate-only mode before allocating a real buffer. SerializedData.java provides pure Java little-endian TL encoding for byte arrays, strings, bools, doubles, and primitives; NativeByteBuffer.java mirrors those operations over a direct native buffer address returned by NativeByteBuffer.native_getFreeBuffer. TLClassStore.java is only a small Java registry for unprompted update/secret-message constructors used when native cannot associate a message with a request.
The normal request flow starts in ConnectionsManager.sendRequest(...). Public overloads assign a Java request token and post sendRequestInternal(...) to Utilities.stageQueue except for sendRequestSync. Before serialization, sendRequestInternal(...) applies AyuGram behavior hooks: it can suppress typing packets, force online-status requests offline, fake read-history success and enqueue Ayu sync, or issue a follow-up read after sending a message. The TL object is serialized into a NativeByteBuffer, then native_sendRequest(...) receives the native address plus optional RequestDelegateInternal, QuickAckDelegate, and WriteToSocketDelegate. Native completion returns a response buffer pointer or error fields; Java wraps the pointer with NativeByteBuffer.wrap(...), calls the original request object's deserializeResponse(...), records networkType, dumps request/response logs, and posts the user-facing RequestDelegate or RequestDelegateTimestamp back to Utilities.stageQueue. Request cancellation and request-guid binding route through native_cancelRequest, native_cancelRequestsForGuid, and native_bindRequestToGuid.
TMessagesProj/…/TgNetWrapper.cpp is the JNI adapter. It registers native methods for NativeByteBuffer and ConnectionsManager, converts Java request buffers into TL_api_request, stores Java delegates as global refs, and calls ConnectionsManager::sendRequest(...). Its completion lambda unwraps TL_api_response to a raw buffer pointer or converts TL_error to Java error fields before invoking RequestDelegateInternal.run(long response, int errorCode, String errorText, int networkType, long timestamp, long requestMsgId). The same file installs a native Delegate implementing ConnectiosManagerDelegate; native events are routed back into static Java methods such as onUnparsedMessageReceived, onUpdate, onSessionCreated, onConnectionStateChanged, onLogout, onUpdateConfig, byte counters, DNS address requests, proxy errors, and host resolution. NativeLoader.cpp is only a stubbed breakpad initializer. CMakeLists.txt builds tgnet as a static library from TMessagesProj/…/*.cpp, links it into the shared native library via TgNetWrapper.cpp, and links BoringSSL crypto/ssl, zlib, and the larger app-native dependencies.
The native core is centered on TMessagesProj/…/ConnectionsManager.cpp and ConnectionsManager.h. Each account has a singleton with an epoll/eventfd (or pipe fallback) network thread, pending task queue, datacenter map, running/request queues, active sockets, proxy checks, and per-account config. ConnectionsManager::init(...) stores the Java-provided app metadata, starts logging with FileLog, loads tgnet.dat through Config.cpp, starts ThreadProc, and may refresh DC settings. Config.cpp reads and atomically writes tgnet.dat plus .bak; ConnectionsManager::loadConfig/saveConfig persist current DC, time difference, push session id, pending destroyed sessions, and serialized Datacenter state. Datacenter.h owns DC addresses, auth keys, salts, connection instances, handshake state, CDN markers, and session recreation. Handshake.cpp performs the MTProto auth-key handshakes, while MTProtoScheme.* defines protocol-level TL classes and TLClassStore::TLdeserialize for containers, RPC results, errors, salts, pongs, gzip, and session messages.
Native request processing uses Request.h/Request.cpp for per-request metadata and callbacks. ConnectionsManager::sendRequest(...) schedules a task on the native network thread, wraps the raw TLObject with wrapInLayer(...), then appends it to requestsQueue. wrapInLayer(...) creates invokeWithLayer(initConnection(query)) when a request needs layer initialization; it injects app version, lang/system lang, device token, certificate fingerprint, installer/package id, timezone, performance class, and proxy hints before using the current TL layer. processRequestQueue(...) moves requests into per-DC batches, enforces auth-key and authorization prerequisites, starts handshakes for missing keys, limits generic/download/upload concurrency, handles RequestFlagInvokeAfter, RequestFlagTryDifferentDc, quick-ack flags, retries, and sends NetworkMessage objects via sendMessagesToConnection(...) or sendMessagesToConnectionWithConfirmation(...).
Socket IO is split between Connection.cpp, ConnectionSocket.cpp, and ConnectionSession.*. ConnectionSocket owns nonblocking sockets, epoll events, proxy overrides, host resolution callbacks, TCP/TLS obfuscation state, timeouts, and read/write buffers. Connection adds MTProto packet framing, AES-CTR transport encryption, connection tokens, retry/suspend behavior, quick ack parsing, per-connection sessions, and timeout tuning by connection type. ConnectionsManager::onConnectionDataReceived(...) validates packet lengths and auth key ids, decrypts encrypted MTProto messages through Datacenter, deserializes protocol wrappers with native TLClassStore, dispatches server responses through processServerResponse(...), and forwards unknown generic updates to Java ConnectionsManager.onUnparsedMessageReceived(...). Java then deserializes selected update constructors through TLClassStore.java and calls MessagesController.processUpdates(...) on Utilities.stageQueue.
Path routing and DC discovery are intentionally split. Native requests can target DEFAULT_DATACENTER_ID, explicit DC ids, or typed connections (ConnectionTypeGeneric, Download, Upload, Push, Temp, Proxy, GenericMedia in Defines.h; Java exposes a subset plus ConnectionTypeDownload2 by encoding the connection number in upper bits). Datacenter selects IPv4/IPv6/download/temp address lists based on flags, while Java ConnectionsManager.getIpStrategy() inspects Android interfaces and forceTryIpV6 preferences before calling native_setIpStrategy. When native needs a hostname resolved, TgNetWrapper.cpp delegates to Java getHostByName(...); Java caches results for five minutes and uses ResolveHostByNameTask before returning through native_onHostNameResolved(...). When native needs new server addresses, onRequestNewServerIpAndPort(...) starts Firebase/DNS-over-HTTPS TXT loaders (DnsTxtLoadTask, GoogleDnsLoadTask, MozillaDnsLoadTask, etc.) and calls native_applyDnsConfig(...); native applyDnsConfig(...) decrypts simple config, replaces temp DC addresses, restarts handshakes, and triggers updateDcSettings(...).
Important interactions outside the module are concentrated in ConnectionsManager.java: AccountInstance and MessagesController receive updates, config refreshes, difference requests, logout, promo checks, and contact checks; NotificationCenter receives connection-state and proxy-error UI notifications; StatsController receives byte counters; KeepAliveJob is used for push/internal updates; UserConfig supplies account ids and activation state; SharedConfig supplies push strings and performance class; ApplicationLoader supplies network state and files directory; and AyuGram classes under com.radolyn.ayugram alter outbound read/typing/online behavior. tlrpc-patch.py post-processes TLRPC.java after schema generation by preserving original noforwards into ayuNoforwards, forcing some restriction/history flags false, and adding ayuDeleted, so schema regeneration must account for this patch.
Coupling
- If any native signature in ConnectionsManager.java changes, update both the
JNINativeMethod ConnectionsManagerMethodsentry and method implementation in TMessagesProj/…/TgNetWrapper.cpp; registration uses exact JNI descriptors, so mismatches fail at native registration or corrupt callback arguments. - If
RequestDelegateInternal.run(...)changes in RequestDelegateInternal.java, also update TgNetWrapper.cpp method lookup and call site; the bridge currently looks up(JILjava/lang/String;IJJ)Vand passes response pointer, error fields, network type, response time, and request msg id. - If request flags or connection types are added/renumbered, update both Java constants in ConnectionsManager.java and native enums in Defines.h; Java passes these integers directly into
ConnectionsManager::sendRequest, andprocessRequestQueue(...)branches on the native bit layout, including upper bits for numbered download connections. - If the TL layer or generated schema in TLRPC.java is regenerated, verify native
ApiScheme.*/MTProtoScheme.*still understand config/initConnection/protocol objects used inside native networking; Java request responses deserialize with generated Java classes, but native must parse MTProto wrappers andTL_config/DNS config itself. - If TLRPC.java generation is changed, rerun or port tlrpc-patch.py; code elsewhere can depend on AyuGram-added fields such as
ayuNoforwardsandayuDeleted, while the patch intentionally changes parsed restriction/history behavior. - If buffer ownership changes in NativeByteBuffer.java, TgNetWrapper.cpp, or native NativeByteBuffer.cpp, audit every
reuse()path: Java wraps native pointers returned from callbacks, nativeTL_api_requestreuses the Java-submitted request buffer in its destructor, and BuffersStorage.cpp recycles native buffers by capacity. - If Config.cpp or the config path passed from ConnectionsManager.java changes, verify multi-account paths still map to distinct
tgnet.datfiles; native persists auth keys and DC state per account and can trigger logout fromloadConfig()when a current DC lacks a permanent auth key. - If DNS fallback behavior changes in Java loaders or native
applyDnsConfig(...), verify the callback chain throughonRequestNewServerIpAndPort(...),native_applyDnsConfig(...),Datacenter::replaceAddresses(...), andupdateDcSettings(...); failures here silently degrade DC discovery and proxy/TLS mismatch recovery.
Risks
- TMessagesProj/…/TgNetWrapper.cpp has several exact static method lookups that must match ConnectionsManager.java; for example
onUnparsedMessageReceived, byte counters, DNS callbacks, andonProxyErrorare registered by descriptor and then invoked from the native delegate. If these descriptors or call arguments drift, native registration/callbacks can fail before networking starts. - TMessagesProj/…/ConnectionsManager.java mutates or suppresses outgoing TL requests in the AyuGram hook before native serialization. If new read-status, typing, or send-message TL classes are introduced without updating this block, ghost/read privacy behavior can leak packets or miss the fake-success/Ayu sync path.
- TMessagesProj/…/Config.cpp and ConnectionsManager.cpp persist auth keys and DC state in
tgnet.dat; if write/backup semantics or config version handling break,ConnectionsManager::loadConfig()can clear datacenters and scheduledelegate->onLogout(...), causing the Java account to be logged out. - TMessagesProj/…/TLClassStore.java only registers a small set of constructors for unparsed native updates. If Telegram introduces a new short update or encrypted-message constructor that native forwards as unparsed and the registry is not extended,
onUnparsedMessageReceived(...)logs an unknown constructor instead of passing updates toMessagesController.processUpdates(...).