The Problem
I play soccer at a facility that uses a venue recording service to film matches. After every game, there's a recording accessible via a deep link URL. I wanted to archive these to YouTube — unlisted, so I could share them with teammates.
The platform architecture makes this harder than it sounds:
1. The deep link is designed for mobile — it tries to open the native app
2. On desktop, it redirects to download the mobile app.
3. On mobile, the video player only loads if you're authenticated — unauthenticated users see "Redirigiendo..." (Redirecting) and nothing else
4. The redirect fires after 200ms, before a headless browser can capture anything
What I Tried First (That Didn't Work)
The obvious approaches all failed, and understanding why took most of the investigation time.
User-agent spoofing — didn't work. The SPA doesn't detect device from the UA string. It detects OS via navigator.platform, a hardware API that reports macOS regardless of what the UA string says.
Linux UA string — same problem. navigator.platform still reported macOS.
CDP script injection to block the redirect — I tried using Page.addScriptToEvaluateOnNewDocument via Chrome DevTools Protocol to inject a script that would intercept and cancel the redirect. It doesn't fire in undetected_chromedriver.
Full CDP mobile emulation — this one partially worked. Setting Emulation.setUserAgentOverride + Emulation.setDeviceMetricsOverride + Emulation.setTouchEmulationEnabled does bypass the redirect. navigator.platform reports "Android" and the browser stays on the fallback page. But with no auth token, the page just sits at "Redirigiendo..." with no API calls and no video content loading.
Root cause confirmed: The recording only loads for authenticated users. Without valid session cookies or a bearer token, device emulation gets you past the redirect but not past the auth wall.
Getting the Auth Token via mitmproxy
The fastest path to a valid token: intercept your phone's network traffic while using the app normally. iPhone → Mac proxy setup, mitmproxy captures the Authorization: Bearer header from any API request.
Steps:
1. brew install mitmproxy → run mitmweb --listen-port 8080
2. Get your Mac's local IP: ipconfig getifaddr en0
3. iPhone → Settings → WiFi → tap network → Configure Proxy → Manual → Server: <Mac IP>, Port: 8080
4. iPhone Safari → mitm.it → install the iOS certificate profile → trust it in Settings → General → About → Certificate Trust Settings
5. Open the app → navigate to the recording
6. In mitmweb browser UI, filter by the platform's domain → find any API request → copy Authorization: Bearer <token>
Total time: about 10 minutes. Token goes into .env as REFRESH_TOKEN.
The script exchanges this refresh token against the app's API to get a short-lived access token, then uses it to call the recording endpoint directly. No browser required.
The Download Pipeline
Once authenticated, the scraper fetches the video source URL from the app's recording API. Then:
ffmpeg -i {m3u8_url} -c copy data/videos/{filename}.mp4
The -c copy flag remuxes without re-encoding — it's fast (9x realtime on my machine) and lossless.
First real run stats: a 70-minute soccer match, 3.1 GB, downloaded in 8 minutes.
Uploading to YouTube
YouTube upload uses the Data API v3 via google-api-python-client.
One-time setup: Create a project in Google Cloud Console → enable YouTube Data API v3 → create OAuth2 credentials as a Desktop app (not Web application — the port is random on each run, so you can't register a redirect URI for a web app type). Download as credentials.json. On first run, a browser window opens for OAuth consent → token.json is saved. Subsequent runs reuse it.
Key detail: set your email as a test user in the OAuth consent screen before the first run. Without that, the consent screen throws "App not verified" and the flow fails.
Upload behavior:
- Privacy: unlisted (accessible via link, not indexed publicly)
- Resumable upload (
MediaFileUpload(resumable=True), 5MB chunks) — survives transient network errors - Quota: 10,000 units/day, ~1,600 per upload → about 6 uploads per day on the free quota
- On quota exceeded (HTTP 403): backs off 60 seconds, retries up to 3 times
First upload: 12 minutes for 3.1 GB. Total end-to-end pipeline including download: ~25 minutes.
State Tracking and Idempotency
Each video is tracked through states in execution_log.json:
discovered → downloading → downloaded → uploading → uploaded
Re-running the pipeline is safe — videos already in uploaded state are skipped. If a download succeeds but the upload fails, the local MP4 is preserved and the pipeline resumes from the upload stage on the next run.
Local MP4 is only deleted after youtube_id is confirmed in state. If you upload but the API response fails to return the ID, the file stays.
What the Final Pipeline Looks Like
Recording URL
→ exchange refresh token for access token (app auth API)
→ fetch video URL from recording API
→ ffmpeg HLS → MP4 (8 min for 70min game)
→ upload to YouTube unlisted (12 min for 3.1 GB)
→ record youtube_id in state
→ delete local MP4
CLI:
python3 main.py --url [recording-deep-link] --title "Game — Apr 28"
The pipeline manages auth automatically. On each run, the scraper exchanges the refresh token for a fresh access token and writes the new refresh token back to .env — so credentials stay current without manual intervention. mitmproxy is only needed once for initial setup, or to recover if the refresh token expires after months of inactivity.