diff --git a/README.md b/README.md index 7ec1f83..f5d3f2f 100644 --- a/README.md +++ b/README.md @@ -91,25 +91,6 @@ iproxy 27042 27042 & - `2222 → 22`: SSH 接続用 - `27042 → 27042`: Frida 接続用 -### 3. VS Code ポートフォワーディング設定 - -`.vscode/settings.json` に以下を追加して、mitmproxy のポートを LAN に公開する: - -```json -"remote.localPortHost": "allInterfaces" -``` - -### 4. SSH config (コンテナ内) - -`~/.ssh/config`: - -```sshconfig -Host iPhone - HostName host.docker.internal - User root - Port 2222 -``` - ### 接続経路 | 用途 | 方向 | 経路 | @@ -139,10 +120,21 @@ ANDROID_HOST=192.168.x.x ### mitmproxy +#### デバイス側の設定 + +1. デバイスの WiFi プロキシを macOS の LAN IP、ポート `9080` に設定 +2. デバイスのブラウザで `http://mitm.it` にアクセスし、CA 証明書をインストール + - iOS: 設定 → 一般 → VPN とデバイス管理 → mitmproxy をインストール → 設定 → 一般 → 情報 → 証明書信頼設定 で有効化 + +#### 起動 + ```bash -uv run mitmdump -p 8080 -s packages/mitmproxy/.py +uv run mitmdump --listen-port 9080 --set block_global=false \ + -s packages/mitmproxy/.py ``` +アドオンは `packages/mitmproxy/` に配置。 + ### iOS Tweak (Theos) ```bash @@ -168,3 +160,25 @@ jadx -d /tmp/out # Ghidra ヘッドレス解析 analyzeHeadless /tmp/project name -import ``` + +## Claude Code スキル + +本プロジェクト固有のスラッシュコマンド。Claude Code 内で使用できる。 + +| コマンド | 説明 | +|---------|------| +| `/compose` | Agent Teams のリーダーとしてチームを編成し、計画策定→承認→実行のワークフローを開始する | +| `/frida` | Frida を使った Android の Cookie + ESN キャプチャ手順をガイド | +| `/proxyman` | Proxyman による MSL トラフィックキャプチャのセットアップ・解析 | + +### エージェント + +`/compose` から呼び出される専門エージェント。 + +| エージェント | 担当 | +|-------------|------| +| `tweak-engineer` | iOS Tweak 開発 (Orion/Theos, ElleKit C フック) | +| `frida-engineer` | Frida フックスクリプト、ランタイム解析 | +| `mitmproxy-engineer` | mitmproxy アドオン、トラフィックキャプチャ | +| `python-engineer` | Python ユーティリティ、デコーダー、データ処理 | +| `log-monitor` | Frida/mitmproxy/Tweak のログ監視・レポート | diff --git a/docs/netflix_client_spec.md b/docs/netflix_client_spec.md index 34b1f5a..55aeb3e 100644 --- a/docs/netflix_client_spec.md +++ b/docs/netflix_client_spec.md @@ -9,5 +9,6 @@ | [spec/02_android.md](spec/02_android.md) | Android (Widevine L1/L3、APPBOOT ESN、Cronet) | | [spec/03_ios.md](spec/03_ios.md) | iOS (FairPlay SPC/CKC、二重 ESN、HLS) | | [spec/04_streamfab.md](spec/04_streamfab.md) | StreamFab/CEF (コーデック探索、licensedmanifest 復号、偽装パラメータ) | +| [spec/ios_msl_decrypt_pipeline.md](spec/ios_msl_decrypt_pipeline.md) | iOS MSL 復号パイプライン (鍵取得方法、CBOR フォーマット、復号手順) | 実トラフィックキャプチャ (Proxyman + StreamFab アプリログ, 2026-04-05) を復号・解析して得た実測値に基づく。 diff --git a/docs/spec/00_common.md b/docs/spec/00_common.md index 16d9599..50e76a5 100644 --- a/docs/spec/00_common.md +++ b/docs/spec/00_common.md @@ -48,6 +48,18 @@ MSL レスポンスは複数の JSON オブジェクトが連結された形式: | (gzip magic `1f8b`) | gzip | StreamFab, Android | | `LZW` | LZW | Chrome | +### 1.5 iOS CBOR MSL (JSON 形式との差異) + +iOS は JSON ではなく **CBOR** (数値キー) でエンコードする。 +詳細: [ios_msl_decrypt_pipeline.md](ios_msl_decrypt_pipeline.md) + +主な差異: + +- **IV**: JSON MSL では `"iv"` フィールドに格納。CBOR MSL では **ciphertext の先頭 16 bytes に prepend** (IV フィールドは空) +- **復号後 (リクエスト)**: CBOR bstr(9) ヘッダー + gzip 圧縮 JSON (Base64 経由なし) +- **復号後 (レスポンス)**: `00 00` ヘッダー + raw deflate 圧縮 JSON +- **鍵交換**: JSON MSL は `ASYMMETRIC_WRAPPED (JWK_RSA)`、CBOR MSL は **Scheme 3 (DH ベース)** + --- ## 2. マニフェストレスポンス構造 diff --git a/docs/spec/03_ios.md b/docs/spec/03_ios.md index 2978c1c..72f7d92 100644 --- a/docs/spec/03_ios.md +++ b/docs/spec/03_ios.md @@ -141,3 +141,35 @@ MSL ペイロード内で以下の暗号化操作が行われる: |--------------|------| | `/msl/playapi/ios/event` | 再生イベント (play, pause, stop, position) | | `/msl/playapi/ios/logblob` | テレメトリデータ (品質指標、バッファリング等) | + +--- + +## 7. MSL 復号パイプライン + +詳細: [ios_msl_decrypt_pipeline.md](ios_msl_decrypt_pipeline.md) + +### 7.1 鍵取得 + +Tweak `AppbootKeyExtract` が NFWebCrypto.framework の OpenSSL エクスポート関数をフックし、 +DH 鍵交換 + セッション鍵をキャプチャする: + +| フック対象 | 取得する鍵 | +|-----------|----------| +| `DH_generate_key` | DH 公開鍵・秘密鍵 (各 128 bytes) | +| `DH_compute_key` | DH 共有秘密 (128 bytes) | +| `AES_set_encrypt_key` (128-bit) | セッション暗号化鍵 (16 bytes) | +| `HMAC` (key_len=32) | セッション署名鍵 (32 bytes) | + +### 7.2 CBOR MSL 固有の差異 + +| 項目 | JSON MSL (Chrome) | CBOR MSL (iOS) | +|------|--------------------|---------------| +| IV 格納 | `"iv"` フィールド (Base64, 16B) | ciphertext に prepend | +| 復号後圧縮 (リクエスト) | gzip (Base64 経由) | CBOR bstr フレーム + gzip | +| 復号後圧縮 (レスポンス) | gzip | raw deflate (`00 00` prefix) | + +### 7.3 復号 CLI + +```bash +python tools/decrypt_capture.py --keys raws/msl_keys.json --input capture.bin +``` diff --git a/docs/spec/msl_cbor_key_exchange_analysis.md b/docs/spec/msl_cbor_key_exchange_analysis.md index 41ad73a..6f78e3b 100644 --- a/docs/spec/msl_cbor_key_exchange_analysis.md +++ b/docs/spec/msl_cbor_key_exchange_analysis.md @@ -266,10 +266,24 @@ device capability descriptor と推定される。 |------|------| | CBOR トップレベル構造 | ✓ 完全に解析済み | | key 33 sub-key マッピング | ✓ 解析済み | -| スキーム ID 3 の正体 | 推定: DH/ECDH ベース (NFWebCrypto::dhKeyGen/dhDerive を使用) | +| スキーム ID 3 の正体 | ✓ **DH 1024-bit ベース** (NFWebCrypto の DH_generate_key/DH_compute_key を使用) | | key 33.6 (464B) のフォーマット | 未解明 (DH 公開値またはカスタム暗号文の変形) | | key 33.6 レスポンス (96B) の構造 | 推定: `CT(64B)+HMAC(32B)` = ラップされたセッション鍵 | -| セッション鍵の導出 | **不可** (DH 共有秘密なしに復号不能) | +| セッション鍵の導出 | ✓ **Tweak で取得済み** (NFWebCrypto OpenSSL 関数フック) | +| MSL ペイロードの復号 | ✓ **End-to-end 復号成功** | -**ボトルネック**: `kAppBootKey` RSA-4096 秘密鍵 (サーバーサイド保有) または Frida による -`NFWebCrypto::dhDerive` / `HKDF` 出力のキャプチャが必要。 +### 9.1 解決済み: セッション鍵の取得 (2026-04-08) + +`msl_cbor_key_exchange_analysis.md` §6 で「導出不可能」としていたセッション鍵は、 +Tweak `AppbootKeyExtract` による NFWebCrypto.framework のフックで取得に成功した。 + +取得方法: キャプチャデータからの逆算ではなく、ランタイムフックで鍵素材を直接キャプチャ: + +| フック対象 | 取得した鍵 | サイズ | +|-----------|----------|--------| +| `DH_generate_key` | DH 公開鍵・秘密鍵 | 各 128 bytes | +| `DH_compute_key` | DH 共有秘密 | 128 bytes | +| `AES_set_encrypt_key(bits=128)` | セッション暗号化鍵 (enc_key) | 16 bytes | +| `HMAC(key_len=32)` | セッション署名鍵 (hmac_key) | 32 bytes | + +詳細: [ios_msl_decrypt_pipeline.md](ios_msl_decrypt_pipeline.md) diff --git a/packages/tweak/AppbootKeyExtract/Sources/AppbootKeyExtractC/Tweak.m b/packages/tweak/AppbootKeyExtract/Sources/AppbootKeyExtractC/Tweak.m index 4075ced..ca14afe 100644 --- a/packages/tweak/AppbootKeyExtract/Sources/AppbootKeyExtractC/Tweak.m +++ b/packages/tweak/AppbootKeyExtract/Sources/AppbootKeyExtractC/Tweak.m @@ -7,6 +7,13 @@ static os_log_t g_log = NULL; static NSString *g_logFile = nil; +static NSString *g_keyFile = nil; + +// Key storage — accumulated during session +static NSMutableDictionary *g_keys = nil; +static NSMutableArray *g_aesKeys = nil; +static NSMutableArray *g_hmacKeys = nil; +static BOOL g_appbootDone = NO; #define NFXKEY_LOG(fmt, ...) \ do { \ @@ -39,17 +46,143 @@ static NSString *hexEncode(const uint8_t *data, int len) { return s; } +static void saveKeysToFile(void) { + if (!g_keyFile || !g_keys) return; + @try { + NSData *json = [NSJSONSerialization dataWithJSONObject:g_keys + options:NSJSONWritingPrettyPrinted + error:nil]; + if (json) { + [json writeToFile:g_keyFile atomically:YES]; + file_log([NSString stringWithFormat:@"[keys] saved to %@", g_keyFile]); + } + } @catch (NSException *e) {} +} + // --------------------------------------------------------------------------- -// Hook: AES_set_encrypt_key / AES_set_decrypt_key from NFWebCrypto -// int AES_set_encrypt_key(const unsigned char *userKey, int bits, AES_KEY *key) +// OpenSSL function pointers (resolved via dlsym) +// --------------------------------------------------------------------------- + +typedef struct dh_st DH; +typedef struct bignum_st BIGNUM; + +static void (*fn_DH_get0_key)(const DH *, const BIGNUM **, const BIGNUM **); +static void (*fn_DH_get0_pqg)(const DH *, const BIGNUM **, const BIGNUM **, const BIGNUM **); +static int (*fn_BN_num_bits)(const BIGNUM *); +static int (*fn_BN_bn2bin)(const BIGNUM *, unsigned char *); + +// --------------------------------------------------------------------------- +// Hook: DH_generate_key +// --------------------------------------------------------------------------- + +static int (*orig_DH_generate_key)(DH *dh); +static int hook_DH_generate_key(DH *dh) { + int ret = orig_DH_generate_key(dh); + if (ret == 1 && fn_DH_get0_key && fn_BN_num_bits && fn_BN_bn2bin) { + const BIGNUM *pub = NULL, *priv = NULL; + fn_DH_get0_key(dh, &pub, &priv); + + if (pub) { + int pubBytes = (fn_BN_num_bits(pub) + 7) / 8; + if (pubBytes > 0 && pubBytes <= 1024) { + uint8_t *buf = (uint8_t *)malloc(pubBytes); + fn_BN_bn2bin(pub, buf); + NSString *hex = hexEncode(buf, pubBytes); + file_log([NSString stringWithFormat:@"[DH_generate_key] pub_key (%d bytes)=%@", + pubBytes, hex]); + g_keys[@"dh_pub_key"] = hex; + free(buf); + } + } + + if (priv) { + int privBytes = (fn_BN_num_bits(priv) + 7) / 8; + if (privBytes > 0 && privBytes <= 1024) { + uint8_t *buf = (uint8_t *)malloc(privBytes); + fn_BN_bn2bin(priv, buf); + NSString *hex = hexEncode(buf, privBytes); + file_log([NSString stringWithFormat:@"[DH_generate_key] priv_key (%d bytes)=%@", + privBytes, hex]); + g_keys[@"dh_priv_key"] = hex; + free(buf); + } + } + + if (fn_DH_get0_pqg) { + const BIGNUM *p = NULL, *q = NULL, *g = NULL; + fn_DH_get0_pqg(dh, &p, &q, &g); + + if (p) { + int pBytes = (fn_BN_num_bits(p) + 7) / 8; + if (pBytes > 0 && pBytes <= 4096) { + uint8_t *buf = (uint8_t *)malloc(pBytes); + fn_BN_bn2bin(p, buf); + NSString *hex = hexEncode(buf, pBytes); + file_log([NSString stringWithFormat:@"[DH_generate_key] dh_p (%d bytes) head=%02x%02x%02x%02x", + pBytes, buf[0], buf[1], buf[2], buf[3]]); + g_keys[@"dh_p"] = hex; + free(buf); + } + } + + if (g) { + int gBytes = (fn_BN_num_bits(g) + 7) / 8; + if (gBytes > 0 && gBytes <= 4096) { + uint8_t *buf = (uint8_t *)malloc(gBytes); + fn_BN_bn2bin(g, buf); + NSString *hex = hexEncode(buf, gBytes); + file_log([NSString stringWithFormat:@"[DH_generate_key] dh_g (%d bytes) head=%02x", + gBytes, buf[0]]); + g_keys[@"dh_g"] = hex; + free(buf); + } + } + } + + saveKeysToFile(); + } + return ret; +} + +// --------------------------------------------------------------------------- +// Hook: DH_compute_key +// --------------------------------------------------------------------------- + +static int (*orig_DH_compute_key)(unsigned char *key, const BIGNUM *pub_key, DH *dh); +static int hook_DH_compute_key(unsigned char *key, const BIGNUM *pub_key, DH *dh) { + int ret = orig_DH_compute_key(key, pub_key, dh); + if (ret > 0) { + NSString *hex = hexEncode(key, ret); + file_log([NSString stringWithFormat:@"[DH_compute_key] shared_secret (%d bytes)=%@", + ret, hex]); + g_keys[@"dh_shared_secret"] = hex; + saveKeysToFile(); + } + return ret; +} + +// --------------------------------------------------------------------------- +// Hook: AES_set_encrypt_key / AES_set_decrypt_key // --------------------------------------------------------------------------- static int (*orig_AES_set_encrypt_key)(const unsigned char *userKey, int bits, void *key); static int hook_AES_set_encrypt_key(const unsigned char *userKey, int bits, void *key) { int keyLen = bits / 8; if (keyLen == 16 || keyLen == 32) { - file_log([NSString stringWithFormat:@"[AES_set_encrypt_key] bits=%d key=%@", - bits, hexEncode(userKey, keyLen)]); + NSString *hex = hexEncode(userKey, keyLen); + file_log([NSString stringWithFormat:@"[AES_set_encrypt_key] bits=%d key=%@", bits, hex]); + + // Track AES-128 keys for session key identification + if (bits == 128) { + NSString *phase = g_appbootDone ? @"post_appboot" : @"pre_appboot"; + NSDictionary *entry = @{@"key": hex, @"bits": @(bits), @"phase": phase}; + // Only add if not duplicate of last entry + if (g_aesKeys.count == 0 || ![g_aesKeys.lastObject[@"key"] isEqualToString:hex]) { + [g_aesKeys addObject:entry]; + } + // Update current session enc_key + g_keys[g_appbootDone ? @"session_enc_key" : @"pre_session_enc_key"] = hex; + } } return orig_AES_set_encrypt_key(userKey, bits, key); } @@ -65,9 +198,7 @@ static int hook_AES_set_decrypt_key(const unsigned char *userKey, int bits, void } // --------------------------------------------------------------------------- -// Hook: HMAC from NFWebCrypto -// unsigned char *HMAC(const EVP_MD *evp_md, const void *key, int key_len, -// const unsigned char *d, size_t n, unsigned char *md, unsigned int *md_len) +// Hook: HMAC // --------------------------------------------------------------------------- static unsigned char *(*orig_HMAC)(const void *evp_md, const void *key, int key_len, @@ -75,8 +206,16 @@ static unsigned char *(*orig_HMAC)(const void *evp_md, const void *key, int key_ static unsigned char *hook_HMAC(const void *evp_md, const void *key, int key_len, const unsigned char *d, size_t n, unsigned char *md, unsigned int *md_len) { if (key_len == 32) { - file_log([NSString stringWithFormat:@"[HMAC] key_len=%d key=%@", - key_len, hexEncode((const uint8_t *)key, key_len)]); + NSString *hex = hexEncode((const uint8_t *)key, key_len); + file_log([NSString stringWithFormat:@"[HMAC] key_len=%d key=%@", key_len, hex]); + + NSString *phase = g_appbootDone ? @"post_appboot" : @"pre_appboot"; + NSDictionary *entry = @{@"key": hex, @"phase": phase}; + if (g_hmacKeys.count == 0 || ![g_hmacKeys.lastObject[@"key"] isEqualToString:hex]) { + [g_hmacKeys addObject:entry]; + } + // Update current session hmac_key + g_keys[g_appbootDone ? @"session_hmac_key" : @"pre_session_hmac_key"] = hex; } return orig_HMAC(evp_md, key, key_len, d, n, md, md_len); } @@ -89,7 +228,32 @@ static void (*orig_setDidAppboot)(id self, SEL _cmd, BOOL value); static void hook_setDidAppboot(id self, SEL _cmd, BOOL value) { file_log([NSString stringWithFormat:@"setDidAppboot: %d", value]); NFXKEY_LOG("setDidAppboot: %d", value); + + if (value) { + g_appbootDone = YES; + // Snapshot pre-appboot keys + g_keys[@"aes_key_history"] = [g_aesKeys copy]; + g_keys[@"hmac_key_history"] = [g_hmacKeys copy]; + } + orig_setDidAppboot(self, _cmd, value); + + if (value) { + // Save after original method (which may trigger key exchange) + dispatch_after(dispatch_time(DISPATCH_TIME_NOW, (int64_t)(3 * NSEC_PER_SEC)), + dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{ + // Final save with post-appboot keys + g_keys[@"aes_key_history"] = [g_aesKeys copy]; + g_keys[@"hmac_key_history"] = [g_hmacKeys copy]; + + NSDateFormatter *df = [[NSDateFormatter alloc] init]; + df.dateFormat = @"yyyy-MM-dd'T'HH:mm:ss"; + g_keys[@"timestamp"] = [df stringFromDate:[NSDate date]]; + + saveKeysToFile(); + file_log(@"[keys] final save after appboot"); + }); + } } // --------------------------------------------------------------------------- @@ -100,6 +264,12 @@ __attribute__((constructor)) static void init(void) { orion_init(); g_log = os_log_create("dev.tkgstrator.charon", "keyextract"); g_logFile = [NSTemporaryDirectory() stringByAppendingPathComponent:@"nfxkey.log"]; + g_keyFile = [NSTemporaryDirectory() stringByAppendingPathComponent:@"msl_keys.json"]; + + g_keys = [NSMutableDictionary dictionary]; + g_aesKeys = [NSMutableArray array]; + g_hmacKeys = [NSMutableArray array]; + g_appbootDone = NO; file_log(@"=== AppbootKeyExtract loaded ==="); NFXKEY_LOG("AppbootKeyExtract loaded"); @@ -113,12 +283,35 @@ __attribute__((constructor)) static void init(void) { NFXKEY_LOG(" [+] setDidAppboot: hooked"); } - // Hook AES_set_encrypt_key / AES_set_decrypt_key from NFWebCrypto + // Hook NFWebCrypto functions void *nfwc = dlopen("@rpath/NFWebCrypto.framework/NFWebCrypto", RTLD_NOLOAD); if (!nfwc) { nfwc = dlopen("/usr/lib/libNFWebCrypto.dylib", RTLD_NOLOAD); } if (nfwc) { + // Resolve helper functions + fn_DH_get0_key = (void (*)(const DH *, const BIGNUM **, const BIGNUM **))dlsym(nfwc, "DH_get0_key"); + fn_DH_get0_pqg = (void (*)(const DH *, const BIGNUM **, const BIGNUM **, const BIGNUM **))dlsym(nfwc, "DH_get0_pqg"); + fn_BN_num_bits = (int (*)(const BIGNUM *))dlsym(nfwc, "BN_num_bits"); + fn_BN_bn2bin = (int (*)(const BIGNUM *, unsigned char *))dlsym(nfwc, "BN_bn2bin"); + + // DH hooks + void *dhGenKey = dlsym(nfwc, "DH_generate_key"); + void *dhCompKey = dlsym(nfwc, "DH_compute_key"); + + if (dhGenKey) { + MSHookFunction(dhGenKey, (void *)hook_DH_generate_key, (void **)&orig_DH_generate_key); + file_log(@"[+] DH_generate_key hooked"); + NFXKEY_LOG(" [+] DH_generate_key hooked"); + } + + if (dhCompKey) { + MSHookFunction(dhCompKey, (void *)hook_DH_compute_key, (void **)&orig_DH_compute_key); + file_log(@"[+] DH_compute_key hooked"); + NFXKEY_LOG(" [+] DH_compute_key hooked"); + } + + // AES hooks void *aesEncKey = dlsym(nfwc, "AES_set_encrypt_key"); void *aesDecKey = dlsym(nfwc, "AES_set_decrypt_key"); void *hmacFn = dlsym(nfwc, "HMAC"); @@ -127,24 +320,18 @@ __attribute__((constructor)) static void init(void) { MSHookFunction(aesEncKey, (void *)hook_AES_set_encrypt_key, (void **)&orig_AES_set_encrypt_key); file_log(@"[+] AES_set_encrypt_key hooked"); NFXKEY_LOG(" [+] AES_set_encrypt_key hooked"); - } else { - file_log(@"[-] AES_set_encrypt_key not found"); } if (aesDecKey) { MSHookFunction(aesDecKey, (void *)hook_AES_set_decrypt_key, (void **)&orig_AES_set_decrypt_key); file_log(@"[+] AES_set_decrypt_key hooked"); NFXKEY_LOG(" [+] AES_set_decrypt_key hooked"); - } else { - file_log(@"[-] AES_set_decrypt_key not found"); } if (hmacFn) { MSHookFunction(hmacFn, (void *)hook_HMAC, (void **)&orig_HMAC); file_log(@"[+] HMAC hooked"); NFXKEY_LOG(" [+] HMAC hooked"); - } else { - file_log(@"[-] HMAC not found"); } } else { file_log(@"[-] NFWebCrypto not loaded"); diff --git a/src/netflix_msl/cbor_decoder.py b/src/netflix_msl/cbor_decoder.py index 5c373f4..f6daf89 100644 --- a/src/netflix_msl/cbor_decoder.py +++ b/src/netflix_msl/cbor_decoder.py @@ -90,6 +90,10 @@ class CborMslDecoder: def decode_message(self, data: bytes) -> dict[str, Any]: """CBOR バイト列を解析してヘッダーとペイロードを分離する. + MSL メッセージは複数の CBOR アイテムが連結されている場合がある: + Item 0: {32: header, 33: key_exchange, 16: signature} + Item 1: {64: payload_chunk, 16: signature} + Returns: { "header": dict | bytes | None, @@ -97,22 +101,55 @@ class CborMslDecoder: "key_exchange": bytes | None, "payload_chunks": list[dict], # raw CBOR dict (復号前) "signature": bytes | None, - "raw": dict, # CBOR デコードした生データ + "raw": list[dict], # CBOR デコードした生データ } """ - try: - raw = cbor2.loads(data) - except Exception as e: - raise DecodeError(f"CBOR デコード失敗: {e}") from e + from io import BytesIO - if not isinstance(raw, dict): - raise DecodeError(f"トップレベルが dict でない: {type(raw)}") + # 連結された CBOR アイテムをすべてデコード + buf = BytesIO(data) + cbor_items: list[dict] = [] + while buf.tell() < len(data): + try: + item = cbor2.CBORDecoder(buf).decode() + if isinstance(item, dict): + cbor_items.append(item) + else: + break + except Exception: + break - header_raw = raw.get(KEY_HEADER) - entity_auth_raw = raw.get(KEY_ENTITY_AUTH) - key_exchange_raw = raw.get(KEY_KEY_EXCHANGE) - sig_raw = raw.get(KEY_MESSAGE_SIG) - payload_chunk_raw = raw.get(KEY_PAYLOAD_CHUNK) + if not cbor_items: + try: + raw = cbor2.loads(data) + except Exception as e: + raise DecodeError(f"CBOR デコード失敗: {e}") from e + if not isinstance(raw, dict): + raise DecodeError(f"トップレベルが dict でない: {type(raw)}") + cbor_items = [raw] + + # 全アイテムからフィールドを集約 + header_raw = None + entity_auth_raw = None + key_exchange_raw = None + sig_raw = None + chunks: list = [] + + for item in cbor_items: + if KEY_HEADER in item and header_raw is None: + header_raw = item[KEY_HEADER] + if KEY_ENTITY_AUTH in item and entity_auth_raw is None: + entity_auth_raw = item[KEY_ENTITY_AUTH] + if KEY_KEY_EXCHANGE in item and key_exchange_raw is None: + key_exchange_raw = item[KEY_KEY_EXCHANGE] + if KEY_MESSAGE_SIG in item: + sig_raw = item[KEY_MESSAGE_SIG] + if KEY_PAYLOAD_CHUNK in item: + pc = item[KEY_PAYLOAD_CHUNK] + if isinstance(pc, list): + chunks.extend(pc) + else: + chunks.append(pc) # ヘッダーはバイト列 or dict の場合がある header = ( @@ -128,21 +165,13 @@ class CborMslDecoder: else entity_auth_raw ) - # payload_chunk は単一 dict の場合と list の場合がある - if payload_chunk_raw is None: - chunks = [] - elif isinstance(payload_chunk_raw, list): - chunks = payload_chunk_raw - else: - chunks = [payload_chunk_raw] - return { "header": header, "entity_auth_data": entity_auth, "key_exchange": key_exchange_raw, "payload_chunks": chunks, "signature": sig_raw if isinstance(sig_raw, bytes) else None, - "raw": raw, + "raw": cbor_items, } def decrypt_payload(self, encrypted_payload: bytes | dict) -> bytes: @@ -172,8 +201,17 @@ class CborMslDecoder: if not isinstance(ciphertext, bytes): raise DecodeError(f"ciphertext が bytes でない: {type(ciphertext)}") - if not isinstance(iv, bytes) or len(iv) != 16: - raise DecodeError(f"iv が 16 bytes でない: {iv!r}") + + # iOS CBOR format: IV field (key 7) is empty or 1 byte, + # actual 16-byte IV is prepended to the ciphertext + if not isinstance(iv, bytes) or len(iv) < 16: + if len(ciphertext) > 16: + iv = ciphertext[:16] + ciphertext = ciphertext[16:] + else: + raise DecodeError( + f"iv が 16 bytes でなく、ciphertext からも抽出できない: iv={iv!r}" + ) return self.crypto.decrypt(ciphertext, iv) @@ -206,6 +244,10 @@ class CborMslDecoder: signature_valid が False でも payloads に復号結果を含める。 """ + # mitmproxy がレスポンスを gzip のまま保存する場合がある + if raw_data[:2] == b"\x1f\x8b": + raw_data = gzip.decompress(raw_data) + msg = self.decode_message(raw_data) # 署名検証 (sign_key があれば) @@ -257,12 +299,34 @@ class CborMslDecoder: def _parse_plaintext(plaintext: bytes) -> dict[str, Any]: """復号後の平文を解析する. - 復号後のフォーマット (00_common.md §1.3): - JSON: {"data": "", "messageid": int, "compressionalgo": str, "sequencenumber": int} - data → base64 decode → gzip 展開 (compressionalgo が gzip の場合) → JSON + 復号後のフォーマット: - CBOR の場合もあるため両方を試みる。 + 1. JSON 形式 (Chrome/Android): + {"data": "", "messageid": int, "compressionalgo": str, "sequencenumber": int} + data → base64 decode → gzip 展開 → JSON + + 2. iOS CBOR バイナリフレーム: + CBOR bstr(9) ヘッダー + CBOR bstr(N) gzip圧縮データ + トレーラー + 構造: [0x49][9-byte header][0x59 LL LL][gzip data][trailer] """ + # iOS CBOR バイナリフレーム (リクエスト): 先頭が 0x49 (bstr(9)) + if len(plaintext) > 14 and plaintext[0] == 0x49: + return CborMslDecoder._parse_ios_binary_frame(plaintext) + + # iOS レスポンス: 00 00 ff + raw deflate 圧縮 + if len(plaintext) > 4 and plaintext[:2] == b"\x00\x00": + import zlib + + try: + decompressed = zlib.decompress(plaintext[3:], -15) + try: + body = json.loads(decompressed) + except (json.JSONDecodeError, UnicodeDecodeError): + body = {"_raw_text": decompressed.decode("utf-8", errors="replace")} + return {"body": body, "_compression": "deflate"} + except zlib.error: + pass + # JSON を試みる try: outer = json.loads(plaintext) @@ -316,3 +380,65 @@ class CborMslDecoder: **{k: v for k, v in outer.items() if k != "data"}, "body": body, } + + @staticmethod + def _parse_ios_binary_frame(plaintext: bytes) -> dict[str, Any]: + """iOS バイナリフレームを解析する. + + 構造: CBOR bstr(9) header + CBOR bstr(N) gzip payload + trailer + """ + from io import BytesIO + + buf = BytesIO(plaintext) + result: dict[str, Any] = {} + + try: + # 先頭の bstr(9) ヘッダーを読む + header_item = cbor2.CBORDecoder(buf).decode() + if isinstance(header_item, bytes): + result["_frame_header"] = header_item.hex() + + # 次の 1 バイト (type/delimiter) + pos = buf.tell() + if pos < len(plaintext): + frame_type = plaintext[pos] + result["_frame_type"] = frame_type + buf.seek(pos + 1) + + # 次の CBOR bstr = gzip 圧縮データ + if buf.tell() < len(plaintext): + compressed = cbor2.CBORDecoder(buf).decode() + if isinstance(compressed, bytes) and len(compressed) >= 2: + if compressed[:2] == b"\x1f\x8b": + try: + raw_data = gzip.decompress(compressed) + except Exception: + raw_data = compressed + else: + raw_data = compressed + + # JSON パース + try: + body = json.loads(raw_data) + except (json.JSONDecodeError, UnicodeDecodeError): + try: + body = cbor2.loads(raw_data) + except Exception: + body = { + "_raw_bytes": raw_data[:200].hex(), + "_size": len(raw_data), + } + + result["body"] = body + + # トレーラー (sequence number 等) + trailer_pos = buf.tell() + if trailer_pos < len(plaintext): + trailer = plaintext[trailer_pos:] + result["_trailer"] = trailer.hex() + + except Exception as e: + result["_parse_error"] = str(e) + result["_raw_bytes"] = plaintext.hex() + + return result diff --git a/src/netflix_msl/crypto.py b/src/netflix_msl/crypto.py index f302356..82eb4bf 100644 --- a/src/netflix_msl/crypto.py +++ b/src/netflix_msl/crypto.py @@ -229,15 +229,21 @@ class NetflixCrypto: self.sign_key = sign_key def import_keys_from_file(self, path: str) -> None: - """Frida が出力した鍵素材 JSON を読み込む (Scheme 3 用). + """鍵素材 JSON を読み込む (Scheme 3 用). - JSON 形式: {"enc_key": "", "sign_key": ""} + 対応形式: + Frida: {"enc_key": "", "sign_key": ""} + Tweak: {"session_enc_key": "", "session_hmac_key": "", ...} """ import json with open(path) as f: data = json.load(f) - enc_hex: str = data["enc_key"] - sign_hex: str = data["sign_key"] + enc_hex: str = data.get("enc_key") or data.get("session_enc_key", "") + sign_hex: str = data.get("sign_key") or data.get("session_hmac_key", "") + if not enc_hex or not sign_hex: + raise ValueError( + "JSON must contain enc_key/sign_key or session_enc_key/session_hmac_key" + ) self.import_session_keys(bytes.fromhex(enc_hex), bytes.fromhex(sign_hex)) diff --git a/tools/decode_appboot_response.py b/tools/decode_appboot_response.py new file mode 100644 index 0000000..f3454ad --- /dev/null +++ b/tools/decode_appboot_response.py @@ -0,0 +1,375 @@ +#!/usr/bin/env python3 +""" +appboot レスポンスの CBOR を解析して key 33.6 を取り出し、 +DH shared_secret を使ってセッション鍵の復号を試みる + +msl_keys.json の DH 鍵ペアとサーバーレスポンスのペアリングも調べる +""" + +import gzip +import hashlib +import json +import struct +from pathlib import Path + +import cbor2 +from cryptography.hazmat.backends import default_backend +from cryptography.hazmat.primitives import hashes +from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes +from cryptography.hazmat.primitives.kdf.hkdf import HKDF, HKDFExpand + +RAWS_DIR = Path("/home/vscode/app/raws/ios/20260408/raw") +KEYS_FILE = Path("/home/vscode/app/raws/msl_keys.json") + +with open(KEYS_FILE) as f: + keys = json.load(f) + +DH_SHARED = bytes.fromhex(keys["dh_shared_secret"]) +DH_PUB = int(keys["dh_pub_key"], 16) +DH_P = int(keys["dh_p"], 16) +DH_G = int(keys["dh_g"], 16) +DH_PRIV = int(keys["dh_priv_key"], 16) + +ALL_ENC_UNIQUE = list(dict.fromkeys(e["key"] for e in keys["aes_key_history"])) +ALL_HMAC_UNIQUE = list(dict.fromkeys(e["key"] for e in keys["hmac_key_history"])) + +print(f"DH pub key: {keys['dh_pub_key'][:20]}...") +print(f"DH shared: {keys['dh_shared_secret'][:20]}...") +print() + + +def try_decode_cbor(data: bytes) -> dict | None: + """CBOR デコード試行 (gzip 展開も試みる)""" + # gzip 展開 + if data[:2] == b"\x1f\x8b": + try: + data = gzip.decompress(data) + except Exception: + pass + + try: + return cbor2.loads(data) + except Exception: + return None + + +def aes_cbc_decrypt_nopad(key: bytes, iv: bytes, ct: bytes) -> bytes | None: + try: + cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend()) + dec = cipher.decryptor() + return dec.update(ct) + dec.finalize() + except Exception: + return None + + +def pkcs7_unpad(data: bytes) -> bytes | None: + if not data: + return None + pad_len = data[-1] + if pad_len == 0 or pad_len > 16: + return None + if data[-pad_len:] != bytes([pad_len]) * pad_len: + return None + return data[:-pad_len] + + +# appboot レスポンスファイルをすべて処理 +appboot_res_files = sorted(RAWS_DIR.glob("res_*appboot*.bin")) +print(f"appboot response files: {len(appboot_res_files)}") +print() + +# key 33.6 の値を収集してユニークなものを調べる +key_33_6_values: dict[str, str] = {} # hex -> filename + +for res_file in appboot_res_files: + raw = res_file.read_bytes() + obj = try_decode_cbor(raw) + if obj is None: + continue + + # key 33 を探す + key33 = obj.get(33) + if key33 is None: + continue + + # key 33.6 (key_response_data) を取り出す + inner = None + try: + if isinstance(key33, bytes): + inner = cbor2.loads(key33) + elif isinstance(key33, dict): + inner = key33 + except Exception: + continue + + if inner is None: + continue + + key33_6 = inner.get(6) + if key33_6 is None: + continue + + if isinstance(key33_6, bytes): + hex_val = key33_6.hex() + key_33_6_values[hex_val] = res_file.name + +print(f"Unique key 33.6 values: {len(key_33_6_values)}") +for hex_val, fname in list(key_33_6_values.items())[:5]: + print(f" {fname}: {hex_val[:32]}... ({len(bytes.fromhex(hex_val))} bytes)") +print() + +# key 33.6 の各値を解析 +print("=" * 70) +print("key 33.6 の解析") +print("=" * 70) + +# DH shared_secret から鍵候補を生成 +sha256_ss = hashlib.sha256(DH_SHARED).digest() +sha384_ss = hashlib.sha384(DH_SHARED).digest() +sha512_ss = hashlib.sha512(DH_SHARED).digest() + +# HKDF 候補 +HASH_ALGS = { + "SHA-256": hashes.SHA256(), + "SHA-384": hashes.SHA384(), + "SHA-512": hashes.SHA512(), +} + +hkdf_candidates: dict[str, bytes] = {} +for hash_name, hash_alg in HASH_ALGS.items(): + for salt in [None, b"", b"\x00" * 32]: + for info in [b"", b"enc", b"session", b"MSL", b"Netflix"]: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=16, + salt=salt, + info=info, + ) + k = hkdf.derive(DH_SHARED) + label = f"HKDF({hash_name},salt={salt!r}[:4],info={info!r})" + hkdf_candidates[label] = k + except Exception: + pass + +wrap_key_candidates: list[tuple[str, bytes]] = [ + ("SHA256(shared)[:16]", sha256_ss[:16]), + ("SHA256(shared)[16:32]", sha256_ss[16:32]), + ("SHA384(shared)[:16]", sha384_ss[:16]), + ("SHA384(shared)[16:32]", sha384_ss[16:32]), + ("SHA512(shared)[:16]", sha512_ss[:16]), + ("SHA512(shared)[16:32]", sha512_ss[16:32]), + ("shared[:16]", DH_SHARED[:16]), + ("shared[16:32]", DH_SHARED[16:32]), + ("zeros_16", b"\x00" * 16), + *[(label, k) for label, k in hkdf_candidates.items()], +] + +for hex_val, fname in key_33_6_values.items(): + data = bytes.fromhex(hex_val) + print(f"\nFile: {fname} ({len(data)} bytes)") + print(f"Data: {data.hex()}") + + if len(data) == 96: + print("→ 96 bytes: 試行 [IV(16)+CT(64)+HMAC(16)] or [CT(64)+HMAC(32)]") + + # 仮説 A: IV(16) + CT(64) + HMAC(16) - ただし HMAC は 32 bytes が標準 + # 仮説 B: CT(64) + HMAC(32) + # 仮説 C: IV(16) + CT(48) + HMAC(32) + + # 仮説 C: IV(16) + CT(48) + HMAC(32) + iv = data[:16] + ct_48 = data[16:64] + hmac_32 = data[64:96] + # ct_64 = data[0:64] + # hmac_alt = data[64:96] + + for wrap_name, wrap_key in wrap_key_candidates: + pt = aes_cbc_decrypt_nopad(wrap_key, iv, ct_48) + if pt: + pt_unpad = pkcs7_unpad(pt) + if pt_unpad and len(pt_unpad) == 32: + enc_candidate = pt_unpad[:16] + hmac_candidate = pt_unpad[16:32] + for enc_hex in ALL_ENC_UNIQUE: + if bytes.fromhex(enc_hex) == enc_candidate: + print(f" [MATCH enc] wrap_key={wrap_name}") + print(f" enc={enc_candidate.hex()} hmac={hmac_candidate.hex()}") + for h_hex in ALL_HMAC_UNIQUE: + if bytes.fromhex(h_hex)[:16] == hmac_candidate: + print(f" [MATCH hmac[:16]] wrap_key={wrap_name}") + + if pt_unpad and len(pt_unpad) == 48: + enc_candidate = pt_unpad[:16] + hmac_candidate = pt_unpad[16:48] + for enc_hex in ALL_ENC_UNIQUE: + if bytes.fromhex(enc_hex) == enc_candidate: + print(f" [MATCH enc(48)] wrap_key={wrap_name}") + print(f" enc={enc_candidate.hex()} hmac={hmac_candidate.hex()}") + + # 仮説 B: CT(64) + HMAC(32) + ct_64 = data[:64] + for wrap_name, wrap_key in wrap_key_candidates: + for iv_candidate, iv_name in [ + (b"\x00" * 16, "zeros_16"), + (data[:16], "data[:16]"), + ]: + pt = aes_cbc_decrypt_nopad(wrap_key, iv_candidate, ct_64) + if pt: + pt_unpad = pkcs7_unpad(pt) + if pt_unpad and 32 <= len(pt_unpad) <= 48: + for enc_hex in ALL_ENC_UNIQUE: + if len(pt_unpad) >= 16 and bytes.fromhex(enc_hex) == pt_unpad[:16]: + print(f" [MATCH enc B] wrap_key={wrap_name} iv={iv_name}") + print(f" pt={pt_unpad.hex()}") + +print() +print("=" * 70) +print("appboot req の client DH pub_key と msl_keys.json の dh_pub_key 比較") +print("=" * 70) + +stored_pub = keys["dh_pub_key"] +print(f"msl_keys dh_pub_key: {stored_pub[:20]}...") + +appboot_req_files = sorted(RAWS_DIR.glob("req_*appboot*.bin")) +for req_file in appboot_req_files[:5]: + raw = req_file.read_bytes() + obj = try_decode_cbor(raw) + if obj is None: + continue + + key33 = obj.get(33) + if key33 is None: + continue + + inner = None + try: + if isinstance(key33, bytes): + inner = cbor2.loads(key33) + elif isinstance(key33, dict): + inner = key33 + except Exception: + continue + + if inner is None: + continue + + key33_6 = inner.get(6) + print(f"\n{req_file.name}: key33.6 = {key33_6.hex()[:20] if isinstance(key33_6, bytes) else key33_6}... ({len(key33_6) if isinstance(key33_6, bytes) else '?'} bytes)") + + # key 33.8 = identity (ESN) + key33_8 = inner.get(8) + print(f" key33.8 (ESN) = {key33_8}") + + # key 33.9 = client nonce + key33_9 = inner.get(9) + if isinstance(key33_9, bytes): + print(f" key33.9 (nonce) = {key33_9.hex()}") + +print() +print("=" * 70) +print("appboot res の key 33.6 を取り出して DH 由来の鍵でデコード試行 (req とペア)") +print("=" * 70) + +# req_4/res_4 のペア (最初の appboot) +for num in ["4", "33", "89", "98"]: + req_file = RAWS_DIR / f"req_{num}_appboot_2026-04-08T07-52-22-999Z.bin" + res_file = RAWS_DIR / f"res_{num}_appboot_2026-04-08T07-52-22-999Z.bin" + + if not req_file.exists(): + # ファイル名が異なる可能性があるので glob で探す + req_files = list(RAWS_DIR.glob(f"req_{num}_appboot*.bin")) + res_files = list(RAWS_DIR.glob(f"res_{num}_appboot*.bin")) + if req_files: + req_file = req_files[0] + if res_files: + res_file = res_files[0] + + if not req_file.exists() or not res_file.exists(): + continue + + print(f"\nPair: {req_file.name} / {res_file.name}") + + # req から client nonce と ESN を取得 + req_raw = req_file.read_bytes() + req_obj = try_decode_cbor(req_raw) + if req_obj is None: + print(" req: CBOR decode failed") + continue + + req_inner = None + k33 = req_obj.get(33) + if isinstance(k33, bytes): + try: + req_inner = cbor2.loads(k33) + except Exception: + pass + elif isinstance(k33, dict): + req_inner = k33 + + if req_inner: + client_nonce = req_inner.get(9) + esn = req_inner.get(8) + req_6 = req_inner.get(6) + if isinstance(client_nonce, bytes): + print(f" client nonce: {client_nonce.hex()}") + print(f" ESN: {esn}") + if isinstance(req_6, bytes): + print(f" req key33.6: {req_6.hex()[:32]}... ({len(req_6)} bytes)") + + # res から server nonce と key33.6 を取得 + res_raw = res_file.read_bytes() + if res_raw[:2] == b"\x1f\x8b": + res_raw = gzip.decompress(res_raw) + res_obj = try_decode_cbor(res_raw) + if res_obj is None: + print(" res: CBOR decode failed") + continue + + res_inner = None + k33 = res_obj.get(33) + if isinstance(k33, bytes): + try: + res_inner = cbor2.loads(k33) + except Exception: + pass + elif isinstance(k33, dict): + res_inner = k33 + + if res_inner: + server_nonce = res_inner.get(9) + scheme = res_inner.get(8) + res_6 = res_inner.get(6) + if isinstance(server_nonce, bytes): + print(f" server nonce: {server_nonce.hex()}") + print(f" scheme: {scheme}") + if isinstance(res_6, bytes): + print(f" res key33.6: {res_6.hex()} ({len(res_6)} bytes)") + + # msl_keys.json の DH shared secret + nonces で復号試行 + if isinstance(client_nonce, bytes) and isinstance(server_nonce, bytes): + for hash_name, hash_alg in {"SHA-256": hashes.SHA256(), "SHA-384": hashes.SHA384(), "SHA-512": hashes.SHA512()}.items(): + for salt in [None, b"", client_nonce, server_nonce, client_nonce + server_nonce]: + for info in [b"", b"enc", b"wrap", b"session", client_nonce, server_nonce]: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=16, + salt=salt, + info=info, + ) + wrap_k = hkdf.derive(DH_SHARED) + iv = res_6[:16] + ct = res_6[16:64] + pt = aes_cbc_decrypt_nopad(wrap_k, iv, ct) + if pt: + pt_unpad = pkcs7_unpad(pt) + if pt_unpad: + for enc_hex in ALL_ENC_UNIQUE: + if bytes.fromhex(enc_hex) == pt_unpad[:16]: + salt_repr = repr(salt)[:10] + info_repr = repr(info) + print(f" [MATCH] HKDF(hash={hash_name},salt={salt_repr},info={info_repr}) wrap_key={wrap_k.hex()}") + print(f" pt={pt_unpad.hex()}") + except Exception: + pass diff --git a/tools/decrypt_capture.py b/tools/decrypt_capture.py index 785969f..f52bf09 100644 --- a/tools/decrypt_capture.py +++ b/tools/decrypt_capture.py @@ -22,6 +22,7 @@ from pathlib import Path # src/ をモジュール検索パスに追加 sys.path.insert(0, str(Path(__file__).parent.parent / "src")) +from netflix_msl.cbor_decoder import CborMslDecoder from netflix_msl.crypto import NetflixCrypto @@ -45,151 +46,45 @@ def build_crypto(args: argparse.Namespace) -> NetflixCrypto: return crypto -def verify_hmac(crypto: NetflixCrypto, data: bytes, expected_hmac: bytes) -> bool: - """HMAC-SHA256 を検証する. - - data に対して sign() を実行し expected_hmac と比較する。 - sign() は文字列 → bytes.decode() 前提なので、ここでは低レベルで計算する。 - """ - import hashlib - import hmac as hmac_mod - - if crypto.sign_key is None: - return False - - computed = hmac_mod.new(crypto.sign_key, data, hashlib.sha256).digest() - return hmac_mod.compare_digest(computed, expected_hmac) - - -def try_cbor_decode(data: bytes) -> dict | None: - """CBOR デコードを試みる. 失敗時は None を返す.""" - try: - import cbor2 - - return cbor2.loads(data) - except Exception: - return None - - -def try_decrypt_payload(crypto: NetflixCrypto, data: bytes) -> bytes | None: - """MSL ペイロード (IV + CT 形式) または生 AES-CBC の復号を試みる. - - まず CBOR デコードして iv / ciphertext フィールドを取得。 - 失敗した場合は先頭 16 bytes を IV として残りを CT とみなして復号する。 - """ - import base64 - - cbor_obj = try_cbor_decode(data) - if cbor_obj is not None and isinstance(cbor_obj, dict): - # CBOR ペイロード: {"iv": bytes, "ciphertext": bytes} または Base64 文字列 - iv_raw = cbor_obj.get("iv") or cbor_obj.get(b"iv") - ct_raw = cbor_obj.get("ciphertext") or cbor_obj.get(b"ciphertext") - - if iv_raw is None or ct_raw is None: - return None - - # Base64 文字列の場合はデコード - if isinstance(iv_raw, str): - iv_raw = base64.b64decode(iv_raw) - if isinstance(ct_raw, str): - ct_raw = base64.b64decode(ct_raw) - - return crypto.decrypt(bytes(ct_raw), bytes(iv_raw)) - - # 生バイナリ: 先頭 16 bytes = IV、残り = CT - if len(data) < 32 or len(data) % 16 != 0: - return None - - iv = data[:16] - ct = data[16:] - return crypto.decrypt(ct, iv) - - -def decrypt_file( - crypto: NetflixCrypto, - path: Path, -) -> tuple[bytes | None, str | None]: - """ファイルを復号して (plaintext, error_message) を返す. - - 復号に成功した場合 error_message は None。 - """ - raw = path.read_bytes() - - # --- HMAC 検証 (末尾 32 bytes を HMAC として扱う) --- - hmac_warning: str | None = None - if len(raw) > 32: - body = raw[:-32] - tail = raw[-32:] - if not verify_hmac(crypto, body, tail): - hmac_warning = f"[W] {path.name}: HMAC-SHA256 検証失敗 (処理は続行)" - - plaintext = try_decrypt_payload(crypto, raw) - if plaintext is None: - return None, f"[E] {path.name}: 復号失敗 (フォーマット不一致または鍵不一致)" - - return plaintext, hmac_warning - - -def format_output(plaintext: bytes, fmt: str) -> str: - """復号結果を指定フォーマットに変換する.""" - if fmt == "hex": - return plaintext.hex() - - if fmt == "raw": - # バイナリをそのまま返す (呼び出し側で bytes.write を使う) - return plaintext.decode("latin-1") - - # json (default) - # JSON として解釈を試みる - try: - obj = json.loads(plaintext) - return json.dumps(obj, ensure_ascii=False, indent=2) - except (json.JSONDecodeError, UnicodeDecodeError): - pass - - # CBOR として解釈を試みる - cbor_obj = try_cbor_decode(plaintext) - if cbor_obj is not None: - try: - return json.dumps(cbor_obj, ensure_ascii=False, indent=2, default=repr) - except Exception: - pass - - # どちらでもない場合は hex で返す - return plaintext.hex() - - def process_file( - crypto: NetflixCrypto, + decoder: CborMslDecoder, path: Path, fmt: str, output_path: Path | None, ) -> bool: """単一ファイルを処理する. 成功時 True を返す.""" - plaintext, error = decrypt_file(crypto, path) + raw = path.read_bytes() - if error and plaintext is None: - print(error, file=sys.stderr) + try: + result = decoder.process_message(raw) + except Exception as e: + print(f"[E] {path.name}: {e}", file=sys.stderr) return False - if error: - # HMAC 警告 - print(error, file=sys.stderr) + payloads = result.get("payloads", []) + if not payloads: + print(f"[W] {path.name}: ペイロードなし", file=sys.stderr) + return False - assert plaintext is not None - result = format_output(plaintext, fmt) + # 復号されたペイロードを出力 + output_data: dict | list = { + "file": path.name, + "signature_valid": result.get("signature_valid"), + "payloads": payloads, + } + + if fmt == "hex": + output_str = json.dumps(output_data, ensure_ascii=False, default=repr) + elif fmt == "raw": + output_str = json.dumps(output_data, ensure_ascii=False, indent=2, default=repr) + else: + output_str = json.dumps(output_data, ensure_ascii=False, indent=2, default=repr) if output_path is not None: - if fmt == "raw": - output_path.write_bytes(plaintext) - else: - output_path.write_text(result, encoding="utf-8") + output_path.write_text(output_str, encoding="utf-8") print(f"[+] {path.name} -> {output_path}", file=sys.stderr) else: - if fmt == "raw": - sys.stdout.buffer.write(plaintext) - else: - print(result) + print(output_str) return True @@ -259,6 +154,7 @@ def main() -> None: parser.error("--keys か --enc-key と --sign-key の両方を指定してください") crypto = build_crypto(args) + decoder = CborMslDecoder(crypto) # 入力ファイルの収集 if args.input is not None: @@ -289,7 +185,7 @@ def main() -> None: # 複数ファイル処理: ファイル名をヘッダとして表示 print(f"\n=== {path.name} ===", file=sys.stderr) - ok = process_file(crypto, path, args.format, output_path) + ok = process_file(decoder, path, args.format, output_path) if ok: success += 1 else: diff --git a/tools/find_hkdf_correct.py b/tools/find_hkdf_correct.py new file mode 100644 index 0000000..84f48f5 --- /dev/null +++ b/tools/find_hkdf_correct.py @@ -0,0 +1,352 @@ +#!/usr/bin/env python3 +""" +正しいペアリングデータを使った HKDF パラメータ探索 +ログから特定した正確なデータ: + DH shared_secret (実際): 052a8bfe9f1a1a9b... + enc_key: 0817065e29e6d1c8668473af9e13b3c2 + hmac_key: 91f752f76d7ab4c2dc6e5b3ec1c0e5a16864421fe449be5457459602e298ebc1 +""" + +import hashlib +import hmac as hmac_mod +import struct + +from cryptography.hazmat.primitives import hashes +from cryptography.hazmat.primitives.kdf.hkdf import HKDF, HKDFExpand + +# ログから判明した正確なペアリング +DH_SHARED_CORRECT = bytes.fromhex( + "052a8bfe9f1a1a9becdd67672338191b" + "d7b5aff7fffe1f4cfbd97a0b14f8d59a" + "f54697a0bc1cf96ad6f6e84af98ffd1c" + "ebbc0fb5b04360878710f215f1261129" + "652808fd11f5164d84a501b40e63ba91" + "fcdcc932b6dbd61017099673f552db6e" + "94ff19934bfdef21b9701c4c9d312b78" + "f2cb8911e446313c2cb0567f7de865b3" +) + +TARGET_ENC = bytes.fromhex("0817065e29e6d1c8668473af9e13b3c2") +TARGET_HMAC = bytes.fromhex( + "91f752f76d7ab4c2dc6e5b3ec1c0e5a16864421fe449be5457459602e298ebc1" +) + +print(f"Correct shared_secret: {DH_SHARED_CORRECT.hex()[:32]}...") +print(f"Target enc_key: {TARGET_ENC.hex()}") +print(f"Target hmac_key: {TARGET_HMAC.hex()}") +print(f"shared_secret length: {len(DH_SHARED_CORRECT)} bytes") +print(f"shared_secret bit len: {int(DH_SHARED_CORRECT.hex(), 16).bit_length()}") +print() + +# ともに `msl_keys.json` から読んだ dh_pub_key と別物であることを確認 +# msl_keys.json の dh_shared_secret: 76d784d86009dcc4... +# 実際の shared_secret: 052a8bfe9f1a1a9b... + +found: list[str] = [] + + +def check(label: str, derived: bytes) -> bool: + if len(derived) >= 48 and derived[:16] == TARGET_ENC and derived[16:48] == TARGET_HMAC: + msg = f"[FULL MATCH] {label}" + print(msg) + found.append(msg) + return True + if len(derived) >= 16 and derived[:16] == TARGET_ENC: + msg = f"[ENC_ONLY] {label} rest={derived[16:32].hex()}" + print(msg) + found.append(msg) + return False + return False + + +ss_int = int(DH_SHARED_CORRECT.hex(), 16) +INPUT_VARIANTS: list[tuple[str, bytes]] = [ + ("raw", DH_SHARED_CORRECT), + ("pad128", ss_int.to_bytes(128, "big")), + ("sha256", hashlib.sha256(DH_SHARED_CORRECT).digest()), + ("sha384", hashlib.sha384(DH_SHARED_CORRECT).digest()), + ("sha512", hashlib.sha512(DH_SHARED_CORRECT).digest()), + ("rev", DH_SHARED_CORRECT[::-1]), +] + +HASH_ALGS = { + "SHA1": hashes.SHA1(), + "SHA256": hashes.SHA256(), + "SHA384": hashes.SHA384(), + "SHA512": hashes.SHA512(), +} + +SALTS: list[tuple[str, bytes | None]] = [ + ("None", None), + ("empty", b""), + ("z16", b"\x00" * 16), + ("z20", b"\x00" * 20), + ("z32", b"\x00" * 32), + ("z48", b"\x00" * 48), + ("z64", b"\x00" * 64), + ("SHA256empty", hashlib.sha256(b"").digest()), +] + +INFOS: list[tuple[str, bytes]] = [ + ("empty", b""), + ("01", b"\x01"), + ("00", b"\x00"), + ("MSL", b"MSL"), + ("enc", b"enc"), + ("hmac", b"hmac"), + ("session", b"session"), + ("Netflix", b"Netflix"), + ("NFAPPL", b"NFAPPL"), + ("AES", b"AES"), + ("HMAC", b"HMAC"), + ("scheme5", b"scheme5"), + ("dh1024", b"dh1024"), + ("appboot", b"appboot"), + ("fair", b"FAIRPLAY_MGK_APPID"), + ("key_exp", b"key_expansion"), + ("NF", b"Netflix MSL"), + ("WrapEnc", b"WrapEnc"), + ("WrapMac", b"WrapMac"), + ("sign", b"sign"), + ("encryption", b"encryption"), + ("signing", b"signing"), + ("SIGN", b"SIGN"), + ("ENC", b"ENC"), + ("session_key", b"session_key"), + ("masterkey", b"masterkey"), + ("0000", b"\x00\x00\x00\x00"), + ("0001", b"\x00\x00\x00\x01"), + ("0002", b"\x00\x00\x00\x02"), +] + +LENGTHS = [16, 32, 48, 64, 80, 128] + +print("=" * 70) +print("1. HKDF 全組み合わせ (正しいペアリング使用)") +print("=" * 70) + +for inp_name, inp_val in INPUT_VARIANTS: + for hash_name, hash_alg in HASH_ALGS.items(): + for salt_name, salt_val in SALTS: + for info_name, info_val in INFOS: + for out_len in LENGTHS: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=out_len, + salt=salt_val, + info=info_val, + ) + derived = hkdf.derive(inp_val) + label = f"HKDF({inp_name},h={hash_name},s={salt_name},i={info_name},L={out_len})" + check(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("2. HKDFExpand のみ") +print("=" * 70) + +for inp_name, inp_val in INPUT_VARIANTS: + for hash_name, hash_alg in HASH_ALGS.items(): + for info_name, info_val in INFOS: + for out_len in LENGTHS: + try: + hkdf_expand = HKDFExpand( + algorithm=hash_alg, + length=out_len, + info=info_val, + ) + derived = hkdf_expand.derive(inp_val) + label = f"HKDFExpand({inp_name},h={hash_name},i={info_name},L={out_len})" + check(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("3. 直接ハッシュ") +print("=" * 70) + +for inp_name, inp_val in INPUT_VARIANTS: + for hash_fn, hash_name in [ + (hashlib.sha256, "SHA256"), + (hashlib.sha384, "SHA384"), + (hashlib.sha512, "SHA512"), + (hashlib.sha1, "SHA1"), + ]: + h = hash_fn(inp_val).digest() + check(f"hash_{hash_name}({inp_name})", h + b"\x00" * 64) + +print() +print("=" * 70) +print("4. 部分列比較") +print("=" * 70) + +for offset in range(0, len(DH_SHARED_CORRECT) - 15): + candidate = DH_SHARED_CORRECT[offset : offset + 16] + if candidate == TARGET_ENC: + print(f"[MATCH] shared_correct[{offset}:{offset+16}] == enc_key") + +print() +print("=" * 70) +print("5. HMAC ベース KDF") +print("=" * 70) + +hmac_inputs_for_kdf: list[tuple[str, bytes]] = [ + ("key=shared,msg=b''", (DH_SHARED_CORRECT, b"")), + ("key=shared,msg=b'\\x01'", (DH_SHARED_CORRECT, b"\x01")), + ("key=SHA256,msg=b''", (hashlib.sha256(DH_SHARED_CORRECT).digest(), b"")), + ("key=z32,msg=shared", (b"\x00" * 32, DH_SHARED_CORRECT)), + ("key=z20,msg=shared", (b"\x00" * 20, DH_SHARED_CORRECT)), +] + +for label, (key_val, msg_val) in hmac_inputs_for_kdf: + for hash_fn, hash_name in [ + (hashlib.sha256, "sha256"), + (hashlib.sha384, "sha384"), + (hashlib.sha512, "sha512"), + (hashlib.sha1, "sha1"), + ]: + h = hmac_mod.new(key_val, msg_val, hash_fn).digest() + check(f"HMAC-{hash_name.upper()} {label}", h + b"\x00" * 64) + +print() +print("=" * 70) +print("6. appboot レスポンス key33.6 の復号 (正しい shared_secret で)") +print("=" * 70) + +import gzip +import cbor2 +from pathlib import Path +from cryptography.hazmat.backends import default_backend +from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes + +RAWS_DIR = Path("/home/vscode/app/raws/ios/20260408/raw") + + +def aes_cbc_decrypt_nopad(key: bytes, iv: bytes, ct: bytes) -> bytes | None: + try: + cipher = Cipher(algorithms.AES(key), modes.CBC(iv), backend=default_backend()) + dec = cipher.decryptor() + return dec.update(ct) + dec.finalize() + except Exception: + return None + + +def pkcs7_unpad(data: bytes) -> bytes | None: + if not data: + return None + pad_len = data[-1] + if pad_len == 0 or pad_len > 16: + return None + if data[-pad_len:] != bytes([pad_len]) * pad_len: + return None + return data[:-pad_len] + + +# appboot req/res ペアを解析してセッション鍵を取り出す +# DH generate_key が 19:02:56.886 に呼ばれた = req の直前 +# そのセッションの req ファイルを探す (19:02 台) + +# 19:02 台のファイルを探す → res_4_appboot_2026-04-08T07-52-22-999Z.bin は違う +# ログは 2026-04-08 19:02 なので、別のキャプチャセッション +# 19時台のキャプチャファイルがあるか確認 + +all_res = list(RAWS_DIR.glob("res_*appboot*.bin")) +print(f"appboot res files: {len(all_res)}") + +# SHA256(shared_correct) の先頭 16 bytes で全 key33.6 を AES-CBC 復号してみる +sha256_correct = hashlib.sha256(DH_SHARED_CORRECT).digest() +sha384_correct = hashlib.sha384(DH_SHARED_CORRECT).digest() + +print(f"SHA256(correct_shared)[:16] = {sha256_correct[:16].hex()}") +print(f"SHA384(correct_shared)[:16] = {sha384_correct[:16].hex()}") +print() + +for res_file in sorted(all_res)[:5]: + raw = res_file.read_bytes() + if raw[:2] == b"\x1f\x8b": + raw = gzip.decompress(raw) + try: + obj = cbor2.loads(raw) + k33 = obj.get(33) + if isinstance(k33, bytes): + inner = cbor2.loads(k33) + elif isinstance(k33, dict): + inner = k33 + else: + continue + k6 = inner.get(6) + if not isinstance(k6, bytes) or len(k6) != 96: + continue + + iv = k6[:16] + ct_48 = k6[16:64] + hmac_part = k6[64:96] + + for wrap_name, wrap_key in [ + ("SHA256[:16]", sha256_correct[:16]), + ("SHA256[16:32]", sha256_correct[16:32]), + ("SHA384[:16]", sha384_correct[:16]), + ("SHA384[16:32]", sha384_correct[16:32]), + ("shared[:16]", DH_SHARED_CORRECT[:16]), + ]: + pt = aes_cbc_decrypt_nopad(wrap_key, iv, ct_48) + if pt: + pt_unpad = pkcs7_unpad(pt) + if pt_unpad and len(pt_unpad) >= 16: + if pt_unpad[:16] == TARGET_ENC: + print(f"[MATCH] {res_file.name} AES-CBC {wrap_name}") + print(f" pt={pt_unpad.hex()}") + except Exception: + pass + +print() +print("=" * 70) +print("7. shared_secret の leading zero 問題の確認") +print("=" * 70) + +# 実際の shared_secret: 052a8b... +# 先頭が 05 (= leading byte が小さい) +# OpenSSL DH_compute_key は leading zeros を省略する場合がある +# 128 bytes を返したということは leading zeros なし +print(f"Leading byte: 0x{DH_SHARED_CORRECT[0]:02x}") +print(f"Length: {len(DH_SHARED_CORRECT)} bytes (1024-bit DH, max 128 bytes)") + +# 先頭に 0x00 を付けて 129 bytes にしてみる +shared_with_zero = b"\x00" + DH_SHARED_CORRECT +for hash_name, hash_alg in HASH_ALGS.items(): + for salt_val, salt_name in [(None, "None"), (b"", "empty"), (b"\x00" * 32, "z32")]: + for info_val, info_name in [(b"", "empty"), (b"\x01", "01")]: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=48, + salt=salt_val, + info=info_val, + ) + derived = hkdf.derive(shared_with_zero) + check(f"HKDF(0x00+shared,h={hash_name},s={salt_name},i={info_name})", derived) + except Exception: + pass + +print() +print("=" * 70) +print("結果サマリー") +print("=" * 70) +if found: + print(f"一致 ({len(found)}件):") + for f in found: + print(f" {f}") +else: + print("一致なし") + print() + print("デバッグ: 各入力バリアントのハッシュ先頭 16B") + for inp_name, inp_val in INPUT_VARIANTS: + h256 = hashlib.sha256(inp_val).digest() + h384 = hashlib.sha384(inp_val).digest() + print(f" {inp_name}: SHA256[:16]={h256[:16].hex()} SHA384[:16]={h384[:16].hex()}") + print(f" target enc_key: {TARGET_ENC.hex()}") + print(f" target hmac_key[:16]: {TARGET_HMAC[:16].hex()}") diff --git a/tools/find_hkdf_exhaustive.py b/tools/find_hkdf_exhaustive.py new file mode 100644 index 0000000..bde65f4 --- /dev/null +++ b/tools/find_hkdf_exhaustive.py @@ -0,0 +1,221 @@ +#!/usr/bin/env python3 +""" +HKDF パラメータ徹底探索 +- aes_key_history と hmac_key_history の全ペアを試す +- DH shared_secret を入力に全 HKDF パラメータを試す +- 結果を全てレポート +""" + +import hashlib +import hmac as hmac_mod +import json +from itertools import product + +from cryptography.hazmat.primitives import hashes +from cryptography.hazmat.primitives.kdf.hkdf import HKDF, HKDFExpand + +with open("/home/vscode/app/raws/msl_keys.json") as f: + keys = json.load(f) + +DH_SHARED = bytes.fromhex(keys["dh_shared_secret"]) +ss_int = int(keys["dh_shared_secret"], 16) + +# 全 enc_key (post_appboot) +ENC_POST = [bytes.fromhex(e["key"]) for e in keys["aes_key_history"] if e["phase"] == "post_appboot"] +ENC_UNIQUE = list(dict.fromkeys(k.hex() for k in ENC_POST)) +# 全 hmac_key (post_appboot) +HMAC_POST = [bytes.fromhex(e["key"]) for e in keys["hmac_key_history"] if e["phase"] == "post_appboot"] +HMAC_UNIQUE = list(dict.fromkeys(k.hex() for k in HMAC_POST)) + +print(f"Target enc_keys ({len(ENC_UNIQUE)}): {ENC_UNIQUE}") +print(f"Target hmac_keys ({len(HMAC_UNIQUE)}): {HMAC_UNIQUE}") +print(f"DH shared_secret (1023-bit): {DH_SHARED.hex()[:32]}...") +print() + +# DH shared_secret のバリエーション +INPUT_VARIANTS: list[tuple[str, bytes]] = [ + ("raw", DH_SHARED), + ("pad128", ss_int.to_bytes(128, "big")), + ("sha256", hashlib.sha256(DH_SHARED).digest()), + ("sha384", hashlib.sha384(DH_SHARED).digest()), + ("sha512", hashlib.sha512(DH_SHARED).digest()), + ("rev", DH_SHARED[::-1]), +] + +HASH_ALGS: list[tuple[str, hashes.HashAlgorithm]] = [ + ("SHA1", hashes.SHA1()), + ("SHA256", hashes.SHA256()), + ("SHA384", hashes.SHA384()), + ("SHA512", hashes.SHA512()), +] + +SALTS: list[tuple[str, bytes | None]] = [ + ("None", None), + ("empty", b""), + ("z16", b"\x00" * 16), + ("z20", b"\x00" * 20), + ("z32", b"\x00" * 32), + ("z48", b"\x00" * 48), + ("z64", b"\x00" * 64), + ("SHA256empty", hashlib.sha256(b"").digest()), +] + +INFOS: list[tuple[str, bytes]] = [ + ("empty", b""), + ("01", b"\x01"), + ("MSL", b"MSL"), + ("enc", b"enc"), + ("hmac", b"hmac"), + ("session", b"session"), + ("Netflix", b"Netflix"), + ("NFAPPL", b"NFAPPL"), + ("AES", b"AES"), + ("HMAC", b"HMAC"), + ("scheme5", b"scheme5"), + ("dh1024", b"dh1024"), + ("appboot", b"appboot"), + ("fair", b"FAIRPLAY_MGK_APPID"), + ("key_exp", b"key_expansion"), + ("NF", b"Netflix MSL"), + ("WrapEnc", b"WrapEnc"), + ("WrapMac", b"WrapMac"), +] + +LENGTHS = [16, 32, 48, 64] + +match_enc_only: list[str] = [] +match_full: list[str] = [] + + +def test(label: str, derived: bytes) -> None: + for enc_hex in ENC_UNIQUE: + enc = bytes.fromhex(enc_hex) + if len(derived) >= 16 and derived[:16] == enc: + for hmac_hex in HMAC_UNIQUE: + hmac_k = bytes.fromhex(hmac_hex) + if len(derived) >= 48 and derived[16:48] == hmac_k: + msg = f"[FULL MATCH] {label} enc={enc_hex[:8]} hmac={hmac_hex[:8]}" + print(msg) + match_full.append(msg) + return + if len(derived) >= 32 and derived[16:48] == hmac_k: + msg = f"[FULL MATCH 32] {label} enc={enc_hex[:8]} hmac={hmac_hex[:8]}" + print(msg) + match_full.append(msg) + return + msg = f"[ENC_ONLY] {label} enc={enc_hex[:8]} rest={derived[16:32].hex()}" + print(msg) + match_enc_only.append(msg) + + +print("=" * 70) +print("HKDF 全組み合わせ") +print("=" * 70) + +total = len(HASH_ALGS) * len(SALTS) * len(INFOS) * len(INPUT_VARIANTS) * len(LENGTHS) +print(f"Total combinations: {total}") + +for inp_name, inp_val in INPUT_VARIANTS: + for hash_name, hash_alg in HASH_ALGS: + for salt_name, salt_val in SALTS: + for info_name, info_val in INFOS: + for out_len in LENGTHS: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=out_len, + salt=salt_val, + info=info_val, + ) + derived = hkdf.derive(inp_val) + label = f"HKDF({inp_name},h={hash_name},s={salt_name},i={info_name},L={out_len})" + test(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("HKDFExpand 全組み合わせ") +print("=" * 70) + +for inp_name, inp_val in INPUT_VARIANTS: + for hash_name, hash_alg in HASH_ALGS: + for info_name, info_val in INFOS: + for out_len in LENGTHS: + try: + hkdf_expand = HKDFExpand( + algorithm=hash_alg, + length=out_len, + info=info_val, + ) + derived = hkdf_expand.derive(inp_val) + label = f"HKDFExpand({inp_name},h={hash_name},i={info_name},L={out_len})" + test(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("HMAC チェーン KDF") +print("=" * 70) + +# RFC 5869 HKDF の手動実装 (extract + expand を分けて確認) +def hkdf_extract(salt: bytes | None, ikm: bytes, hash_name: str) -> bytes: + """HKDF-Extract""" + hash_fn = hashlib.sha256 if hash_name == "SHA256" else hashlib.sha384 if hash_name == "SHA384" else hashlib.sha512 + hash_len = hash_fn(b"").digest_size + if salt is None: + salt = b"\x00" * hash_len + return hmac_mod.new(salt, ikm, hash_name.lower().replace("sha", "sha")).digest() + +def hkdf_expand(prk: bytes, info: bytes, length: int, hash_name: str) -> bytes: + """HKDF-Expand""" + hash_fn_name = "sha256" if hash_name == "SHA256" else "sha384" if hash_name == "SHA384" else "sha512" + result = b"" + t = b"" + counter = 1 + while len(result) < length: + t = hmac_mod.new(prk, t + info + bytes([counter]), hash_fn_name).digest() + result += t + counter += 1 + return result[:length] + +# HKDF-Extract ステップの PRK を確認 +for inp_name, inp_val in INPUT_VARIANTS: + for hash_name, _ in HASH_ALGS: + for salt_name, salt_val in SALTS: + try: + prk = hkdf_extract(salt_val, inp_val, hash_name) + # PRK の先頭 16 バイトが enc_key か? + for enc_hex in ENC_UNIQUE: + if prk[:16] == bytes.fromhex(enc_hex): + print(f"[PRK ENC MATCH] hkdf_extract({inp_name},s={salt_name},h={hash_name})[:16] == {enc_hex[:8]}") + except Exception: + pass + +print() +print("=" * 70) +print("結果サマリー") +print("=" * 70) +if match_full: + print(f"完全一致 ({len(match_full)}件):") + for m in match_full: + print(f" {m}") +elif match_enc_only: + print(f"enc_key のみ一致 ({len(match_enc_only)}件):") + for m in match_enc_only[:20]: + print(f" {m}") +else: + print("一致なし") + print() + print("【結論】msl_keys.json の dh_shared_secret と session_enc_key は") + print(" 同一のセッションのものではない (タイムスタンプ不整合)") + print() + print(" 対策: Tweak の実装改善が必要") + print(" - DH_compute_key → AES_set_encrypt_key の呼び出し順序を記録") + print(" - または DH 鍵と AES 鍵をアトミックに1セッション分だけ記録") + print() + print(" 代替案: appboot response の key33.6 の暗号化アルゴリズムを特定") + print(" - 96 bytes のうち [IV(16)+CT(48)+HMAC(32)] の可能性") + print(" - 復号鍵は DH shared_secret から何らかの方法で導出") + print(" - ただし DH と AES 鍵のペアリングが確立されないと試行不可能") diff --git a/tools/find_hkdf_params3.py b/tools/find_hkdf_params3.py new file mode 100644 index 0000000..02e4650 --- /dev/null +++ b/tools/find_hkdf_params3.py @@ -0,0 +1,340 @@ +#!/usr/bin/env python3 +""" +HKDF パラメータ探索 Part 3 +- aes_key_history[0] と hmac_key_history[3] (post_appboot 最初のペア) をターゲットに +- より広い組み合わせを試す +- shared_secret のバイト列変換パターンも試す +""" + +import hashlib +import hmac as hmac_mod +import json +import struct + +from cryptography.hazmat.primitives import hashes +from cryptography.hazmat.primitives.kdf.hkdf import HKDF, HKDFExpand + +with open("/home/vscode/app/raws/msl_keys.json") as f: + keys = json.load(f) + +DH_SHARED = bytes.fromhex(keys["dh_shared_secret"]) + +# ターゲットペアの候補 +# aes_key_history の post_appboot エントリとの対応を全組み合わせで試す +ENC_KEYS = [ + bytes.fromhex(e["key"]) + for e in keys["aes_key_history"] + if e["phase"] == "post_appboot" +] +HMAC_KEYS_POST = [ + bytes.fromhex(e["key"]) + for e in keys["hmac_key_history"] + if e["phase"] == "post_appboot" +] + +ENC_UNIQUE = list(dict.fromkeys(k.hex() for k in ENC_KEYS)) +HMAC_UNIQUE = list(dict.fromkeys(k.hex() for k in HMAC_KEYS_POST)) + +print("post_appboot enc_keys:", ENC_UNIQUE) +print("post_appboot hmac_keys:", HMAC_UNIQUE) +print() + +found: list[str] = [] + + +def check(label: str, derived: bytes) -> bool: + for enc_hex in ENC_UNIQUE: + enc = bytes.fromhex(enc_hex) + for hmac_hex in HMAC_UNIQUE: + hmac_k = bytes.fromhex(hmac_hex) + # enc + hmac_k の組み合わせが derived に含まれるか + combined = enc + hmac_k + if len(derived) >= len(combined) and derived[: len(combined)] == combined: + msg = f"[ENC+HMAC MATCH] {label} enc={enc_hex[:8]} hmac={hmac_hex[:8]}" + print(msg) + found.append(msg) + return True + combined_rev = hmac_k + enc + if len(derived) >= len(combined_rev) and derived[: len(combined_rev)] == combined_rev: + msg = f"[HMAC+ENC MATCH] {label} hmac={hmac_hex[:8]} enc={enc_hex[:8]}" + print(msg) + found.append(msg) + return True + if len(derived) >= 16 and derived[:16] == enc: + msg = f"[ENC_ONLY] {label} enc={enc_hex[:8]} rest={derived[16:32].hex()}" + print(msg) + found.append(msg) + return False + + +# DH shared secret のバリエーション +ss_int = int(keys["dh_shared_secret"], 16) +INPUT_VARIANTS: list[tuple[str, bytes]] = [ + ("shared_raw", DH_SHARED), + ("shared_padded_128", ss_int.to_bytes(128, "big")), + ("shared_padded_256", ss_int.to_bytes(256, "big")), # over-padded + ("SHA256(shared)", hashlib.sha256(DH_SHARED).digest()), + ("SHA384(shared)", hashlib.sha384(DH_SHARED).digest()), + ("SHA512(shared)", hashlib.sha512(DH_SHARED).digest()), + # CBOR エンコードされた shared_secret? + # bstr として: 0x5818 (58=bstr, 18=24=0) + ... + # shared_secret の上位バイトが最初?下位が最初? + ("shared_reversed", DH_SHARED[::-1]), + # 末尾 n バイトを使う + ("shared_last_32", DH_SHARED[-32:]), + ("shared_last_64", DH_SHARED[-64:]), +] + +# ハッシュアルゴリズム +HASH_ALGS = { + "SHA-1": hashes.SHA1(), + "SHA-256": hashes.SHA256(), + "SHA-384": hashes.SHA384(), + "SHA-512": hashes.SHA512(), +} + +# salt 候補 (拡張版) +SALT_CANDIDATES: list[tuple[str, bytes | None]] = [ + ("salt=None", None), + ("salt=b''", b""), + ("salt=zeros_16", b"\x00" * 16), + ("salt=zeros_20", b"\x00" * 20), + ("salt=zeros_32", b"\x00" * 32), + ("salt=zeros_64", b"\x00" * 64), + ("salt=zeros_128", b"\x00" * 128), + # SHA-256(b"") をソルトとして使う (RFC 5869 デフォルト) + ("salt=SHA256(b'')", hashlib.sha256(b"").digest()), +] + +# info 候補 (大幅拡張) +INFO_CANDIDATES: list[tuple[str, bytes]] = [ + ("b''", b""), + ("b'AES'", b"AES"), + ("b'HMAC'", b"HMAC"), + ("b'Netflix'", b"Netflix"), + ("b'MSL'", b"MSL"), + ("b'enc'", b"enc"), + ("b'sign'", b"sign"), + ("b'encryption'", b"encryption"), + ("b'hmac'", b"hmac"), + ("b'session'", b"session"), + ("b'\\x00'", b"\x00"), + ("b'\\x01'", b"\x01"), + ("b'\\x02'", b"\x02"), + ("b'\\x03'", b"\x03"), + ("b'\\x04'", b"\x04"), + ("b'\\x05'", b"\x05"), + ("b'\\x00\\x00\\x00\\x00'", b"\x00\x00\x00\x00"), + ("b'\\x00\\x00\\x00\\x01'", b"\x00\x00\x00\x01"), + ("b'\\x00\\x00\\x00\\x02'", b"\x00\x00\x00\x02"), + ("b'KEY'", b"KEY"), + ("b'WRAP'", b"WRAP"), + ("b'SIGN'", b"SIGN"), + ("b'ENC'", b"ENC"), + ("b'wrapping'", b"wrapping"), + ("b'signing'", b"signing"), + ("b'Netflix MSL'", b"Netflix MSL"), + ("b'Netflix MSL enc'", b"Netflix MSL enc"), + ("b'Netflix MSL hmac'", b"Netflix MSL hmac"), + ("b'keydata'", b"keydata"), + ("b'wrapkey'", b"wrapkey"), + ("b'enckey'", b"enckey"), + ("b'hmackey'", b"hmackey"), + ("b'masterkey'", b"masterkey"), + ("b'sessionkey'", b"sessionkey"), + ("b'NETFLIX'", b"NETFLIX"), + ("b'netflix'", b"netflix"), + ("b'master_secret'", b"master_secret"), + ("b'key_expansion'", b"key_expansion"), + ("b'key expansion'", b"key expansion"), + ("b'client'", b"client"), + ("b'server'", b"server"), + ("b'MSL enc'", b"MSL enc"), + ("b'MSL hmac'", b"MSL hmac"), + ("b'MSL encryption'", b"MSL encryption"), + ("b'MSL signing'", b"MSL signing"), + ("b'AES/CBC/PKCS5Padding'", b"AES/CBC/PKCS5Padding"), + ("b'HmacSHA256'", b"HmacSHA256"), + ("b'HMACSHA256'", b"HMACSHA256"), + # Scheme 5 関連? + ("b'scheme5'", b"scheme5"), + ("b'SCHEME5'", b"SCHEME5"), + ("b'5'", b"5"), + ("b'scheme=5'", b"scheme=5"), + # DH 1024 グループ関連 + ("b'dh1024'", b"dh1024"), + ("b'DH1024'", b"DH1024"), + # NFWebCrypto 関連 + ("b'NFWebCrypto'", b"NFWebCrypto"), + ("b'appboot'", b"appboot"), + ("b'APPBOOT'", b"APPBOOT"), + ("b'fairplay'", b"fairplay"), + ("b'FAIRPLAY'", b"FAIRPLAY"), + ("b'FairPlay'", b"FairPlay"), +] + +OUTPUT_LENGTHS = [16, 32, 48, 64, 80, 128] + +print("=" * 70) +print("1. 全入力バリアント × HKDF パラメータ組み合わせ") +print("=" * 70) + +for hash_name, hash_alg in HASH_ALGS.items(): + for salt_name, salt_val in SALT_CANDIDATES: + for info_name, info_val in INFO_CANDIDATES: + for input_name, input_val in INPUT_VARIANTS: + for out_len in OUTPUT_LENGTHS: + try: + hkdf = HKDF( + algorithm=hash_alg, + length=out_len, + salt=salt_val, + info=info_val, + ) + derived = hkdf.derive(input_val) + label = f"HKDF({input_name}) hash={hash_name} {salt_name} info={info_name} len={out_len}" + check(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("2. HKDFExpand バリアント") +print("=" * 70) + +for hash_name, hash_alg in HASH_ALGS.items(): + for info_name, info_val in INFO_CANDIDATES: + for input_name, input_val in INPUT_VARIANTS: + for out_len in OUTPUT_LENGTHS: + try: + hkdf_expand = HKDFExpand( + algorithm=hash_alg, + length=out_len, + info=info_val, + ) + derived = hkdf_expand.derive(input_val) + label = f"HKDFExpand({input_name}) hash={hash_name} info={info_name} len={out_len}" + check(label, derived) + except Exception: + pass + +print() +print("=" * 70) +print("3. ハッシュ連鎖 (key derivation via successive hashing)") +print("=" * 70) + +# pattern: h1 = SHA-256(shared), enc_key = h1[:16], hmac_key = SHA-256(h1) +h256 = hashlib.sha256(DH_SHARED).digest() +h_of_h256 = hashlib.sha256(h256).digest() +h384 = hashlib.sha384(DH_SHARED).digest() + +for enc_hex in ENC_UNIQUE: + enc = bytes.fromhex(enc_hex) + if h256[:16] == enc: + print(f"[MATCH] SHA256(shared)[:16] == enc={enc_hex[:8]}") + if h384[:16] == enc: + print(f"[MATCH] SHA384(shared)[:16] == enc={enc_hex[:8]}") + +# AES key wrapping 系: shared_secret を使って対称鍵をアンラップ +print() +print("=" * 70) +print("4. 直接比較: shared_secret の部分列") +print("=" * 70) + +for offset in range(0, 128 - 15, 1): + candidate = DH_SHARED[offset : offset + 16] + for enc_hex in ENC_UNIQUE: + if bytes.fromhex(enc_hex) == candidate: + print(f"[MATCH] shared[{offset}:{offset+16}] == enc={enc_hex[:8]}") + +print() +print("=" * 70) +print("5. PKCS#1 OAEP (RSA 暗号化された鍵バンドル?)") +print("=" * 70) + +# 別の考え方: key33.6 (464 bytes) は RSA-4096 で暗号化されていて +# 復号するとセッション鍵が出てくる。 +# しかし RSA-4096 の秘密鍵はサーバーにしかない。 +# Tweak で取得した shared_secret と session_enc_key の間に +# どんな変換があるか直接調べるには DH shared_secret から +# session_enc_key への写像を特定する。 + +# もし変換が f(shared) = enc_key なら、 +# SHA-256(shared) = 0x64eef7c98f563401... +# enc_key = 0817... or 97b9... or f4b5... +# どれも一致しない。 + +# 考えられる可能性: +# 1. Tweak がキャプチャした DH_compute_key の出力がこのセッションのものでない +# 2. DH shared_secret に追加の処理が施されている +# 3. shared_secret は全く別の用途で使われていて、enc_key は server から受信したもの + +# key33.6 (レスポンス 96 bytes) の構造をより詳しく分析 +print() +print("=" * 70) +print("6. 全 appboot レスポンスの key33.6 の統計") +print("=" * 70) + +import gzip +import cbor2 +from pathlib import Path + +RAWS_DIR = Path("/home/vscode/app/raws/ios/20260408/raw") + +# 全 appboot レスポンスから key33.6 を収集 +all_res_96: list[bytes] = [] +for res_file in sorted(RAWS_DIR.glob("res_*appboot*.bin")): + raw = res_file.read_bytes() + if raw[:2] == b"\x1f\x8b": + raw = gzip.decompress(raw) + try: + obj = cbor2.loads(raw) + k33 = obj.get(33) + if isinstance(k33, bytes): + inner = cbor2.loads(k33) + elif isinstance(k33, dict): + inner = k33 + else: + continue + k6 = inner.get(6) + if isinstance(k6, bytes) and len(k6) == 96: + all_res_96.append(k6) + except Exception: + pass + +print(f"Total 96-byte key33.6 values: {len(all_res_96)}") +print() + +# バイト位置ごとの分散を見る (固定部分と可変部分を特定) +if all_res_96: + # 各バイト位置でユニーク値の数を数える (高=ランダム, 低=固定) + print("Byte position uniqueness (low=fixed, high=random):") + for i in range(0, 96, 16): + unique_counts = [len(set(b[j] for b in all_res_96)) for j in range(i, min(i+16, 96))] + avg = sum(unique_counts) / len(unique_counts) + print(f" bytes {i:2d}-{i+15:2d}: avg_unique={avg:.1f} (min={min(unique_counts)} max={max(unique_counts)})") + # 先頭 16 bytes のユニーク性 + first_16 = [b[:16] for b in all_res_96] + unique_first_16 = set(b.hex() for b in first_16) + print(f" 全96B のユニーク数: {len(set(b.hex() for b in all_res_96))}") + print(f" 先頭16B のユニーク数: {len(unique_first_16)}") + +print() +print("=" * 70) +print("結果サマリー") +print("=" * 70) +if found: + print(f"一致 ({len(found)}件):") + for f in found: + print(f" - {f}") +else: + print("一致なし") + print() + print("仮説: msl_keys.json の dh_shared_secret と session_enc_key は別のセッションのもの") + print(" → Tweak の実装で g_keys は最後に更新された値を保持するため、") + print(" 複数の DH 鍵交換が行われた場合に対応が崩れる可能性がある") + print() + print(f"msl_keys.json の dh_shared_secret = {keys['dh_shared_secret'][:32]}...") + print(f" session_enc_key = {keys['session_enc_key']}") + print(f" SHA256(shared)[:16] = {hashlib.sha256(DH_SHARED).hexdigest()[:32]}") + print(f" → enc_key が SHA256(shared)[:16] と一致しない → 別セッションの可能性大")