# Beatport Charts Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Aggiungi una tab "🎧 Beatport" a MusicTools che carica le Top 100 per genere dal sito Beatport, permette selezione via checkbox e scarica i brani riusando il downloader esistente (Spotify search → yt-dlp → .mp3). **Architecture:** Nuovo modulo `core/beatport.py` (parser stateless con cache in-memory), 4 nuovi metodi in `main.py` (bridge JS↔Python via pywebview), nuova sezione UI in `webui/`. Parsing dati Beatport via estrazione JSON `__NEXT_DATA__` embeddato in ogni pagina Next.js. Zero cambi lato server, zero migration. **Tech Stack:** Python 3.11+, `requests`, `re`, `json`, `dataclasses`, `pathlib` (già in requirements). `pytest` + `responses` come nuove dev-deps. Frontend: vanilla JS + HTML/CSS (nessun framework, coerente col resto dell'app). **Spec di riferimento:** `docs/superpowers/specs/2026-07-14-beatport-charts-design.md` --- ## Note operative - **Author email git:** usa `git -c user.email=info@djluza.com commit …` per ogni commit di questo piano (non modificare il config globale) - **Branch:** lavora su `feat/beatport-charts` per isolare. `git checkout -b feat/beatport-charts` prima della Task 0 - **Framework di test:** questo repo non ha ancora `tests/`. Le Task 0 e 1 lo impostano - **Non modificare** `server/`, `landing/`, `build_macos.py`, `build_windows.py` — questa feature è additive lato client - **⚠ Cloudflare:** Beatport è dietro Cloudflare Managed Challenge — `requests` puro riceve 403. La feature usa **`curl_cffi`** (TLS fingerprint impersonation di Chrome) che bypassa CF. Verificato funzionante il 2026-07-14 con `impersonate="chrome131"` → HTTP 200, ~885KB, `__NEXT_DATA__` presente. Task 1 e 6 sono aggiornate di conseguenza. --- ## Task 0: Setup branch, tests dir, dev dependencies **Files:** - Create: `tests/__init__.py` (vuoto) - Create: `tests/conftest.py` - Modify: `requirements.txt` - [ ] **Step 1: Crea branch** ```bash cd /Users/luciano/Downloads/Progetti2026/MusicDownload git checkout -b feat/beatport-charts ``` - [ ] **Step 2: Aggiungi test deps a requirements.txt** Aggiungi in fondo al file: ``` # --- dev only --- pytest>=8.0.0 responses>=0.25.0 freezegun>=1.4.0 ``` - [ ] **Step 3: Installa deps** Run: `python3 -m pip install -r requirements.txt` Expected: pytest, responses, freezegun installati senza errori - [ ] **Step 4: Crea tests/__init__.py vuoto** ```bash mkdir -p tests/fixtures touch tests/__init__.py ``` - [ ] **Step 5: Crea tests/conftest.py** ```python """Fixtures pytest condivise.""" from pathlib import Path import pytest FIXTURES_DIR = Path(__file__).parent / "fixtures" @pytest.fixture def fixtures_dir() -> Path: return FIXTURES_DIR ``` - [ ] **Step 6: Verifica pytest gira (0 test)** Run: `python3 -m pytest tests/ -v` Expected: `no tests ran in 0.XXs`, exit code 5 (nessun test) — è ok - [ ] **Step 7: Commit** ```bash git add tests/__init__.py tests/conftest.py requirements.txt git -c user.email=info@djluza.com commit -m "test: setup pytest infra per feature Beatport" ``` --- ## Task 1: HTML fixture reale + genres list script **Files:** - Create: `scripts/refresh_beatport_genres.py` - Create: `tests/fixtures/beatport_melodic_top100.html` - Modify: `requirements.txt` (aggiungi `curl_cffi>=0.9.0` come runtime dep) - [ ] **Step 0: Aggiungi `curl_cffi` a requirements.txt come runtime dep** Beatport è dietro Cloudflare Managed Challenge — `requests` puro riceve 403. Serve `curl_cffi` che impersona il TLS fingerprint di Chrome. Modifica `requirements.txt` — aggiungi prima del blocco `# --- dev only ---`: ``` curl_cffi>=0.9.0 ``` Installa: `python3 -m pip install -r requirements.txt` - [ ] **Step 1: Scarica una pagina Beatport reale come fixture (via curl_cffi)** ```bash python3 -c " from curl_cffi import requests r = requests.get( 'https://www.beatport.com/genre/melodic-house-techno/90/top-100', impersonate='chrome131', timeout=15, ) assert r.status_code == 200, f'HTTP {r.status_code}' with open('tests/fixtures/beatport_melodic_top100.html', 'w') as f: f.write(r.text) print('saved', len(r.text), 'bytes') " ``` Verifica: `ls -la tests/fixtures/beatport_melodic_top100.html` → file > 500KB (atteso ~885KB). Se è < 100KB o lo script fallisce: **STOP e chiedi supporto** (potrebbe essere cambiata la protezione CF). - [ ] **Step 2: Verifica che `__NEXT_DATA__` sia presente** Run: `grep -c '__NEXT_DATA__' tests/fixtures/beatport_melodic_top100.html` Expected: `1` (esattamente un match) Se `0`: pagina non è Next.js-served (Beatport ha cambiato tech). Interrompi e chiedi conferma design. - [ ] **Step 3: Ispeziona lo schema JSON (una tantum, non automatizzato)** ```bash python3 -c " import re, json html = open('tests/fixtures/beatport_melodic_top100.html').read() m = re.search(r'', html, re.DOTALL) data = json.loads(m.group(1)) # Naviga: props > pageProps > dehydratedState > queries > [i] > state > data > results queries = data['props']['pageProps']['dehydratedState']['queries'] for i, q in enumerate(queries): d = q.get('state', {}).get('data', {}) if isinstance(d, dict) and 'results' in d: results = d['results'] if isinstance(results, list) and len(results) >= 50: print(f'queries[{i}]: {len(results)} results') print(' sample keys:', list(results[0].keys())[:15]) " ``` Aspettato output tipo: ``` queries[N]: 100 results sample keys: ['id', 'name', 'mix_name', 'artists', 'length_ms', 'chart_position', ...] ``` Se le chiavi non sono `id`, `name`, `mix_name`, `artists`, `length_ms` (o simili): adatta i selettori nelle Task 4-5 di conseguenza — annotali qui prima di proseguire. - [ ] **Step 4: Crea script per enumerare i generi (opzionale, run manuale)** ```python # scripts/refresh_beatport_genres.py """Estrae lo slug + numeric id di tutti i generi Beatport dalla pagina indice. Uso: python3 scripts/refresh_beatport_genres.py > /tmp/genres.txt Poi copia manualmente in core/beatport.py::GENRES.""" from __future__ import annotations import re import json import sys from curl_cffi import requests def main() -> int: resp = requests.get( "https://www.beatport.com/genres", impersonate="chrome131", timeout=15, ) resp.raise_for_status() m = re.search(r'', resp.text, re.DOTALL) if not m: print("__NEXT_DATA__ non trovato", file=sys.stderr) return 1 data = json.loads(m.group(1)) queries = data["props"]["pageProps"]["dehydratedState"]["queries"] for q in queries: d = q.get("state", {}).get("data", {}) if isinstance(d, dict) and "results" in d and isinstance(d["results"], list): for g in d["results"]: if "slug" in g and "id" in g and "name" in g: print(f' "{g["slug"]}": ({g["id"]}, "{g["name"]}"),') return 0 if __name__ == "__main__": sys.exit(main()) ``` - [ ] **Step 5: Genera la mappa completa dei generi** ```bash mkdir -p scripts # crea il file dal blocco sopra python3 scripts/refresh_beatport_genres.py > /tmp/genres_output.txt head -5 /tmp/genres_output.txt wc -l /tmp/genres_output.txt ``` Expected: ~30 righe nel formato ` "slug": (id, "Name"),`. Salva questo output — servirà nella Task 2. Se lo script fallisce (bot detection): usa manualmente per ora la lista parziale nel Task 2 e apri issue separato per completarla. - [ ] **Step 6: Commit** ```bash git add scripts/refresh_beatport_genres.py tests/fixtures/beatport_melodic_top100.html git -c user.email=info@djluza.com commit -m "beatport: fixture HTML per test + script refresh generi" ``` --- ## Task 2: GENRES map + list_genres() — TDD **Files:** - Create: `core/beatport.py` - Create: `tests/test_beatport.py` - [ ] **Step 1: Scrivi test per list_genres()** Crea `tests/test_beatport.py`: ```python """Test per core.beatport.""" import pytest from core import beatport class TestListGenres: def test_returns_list_of_dicts(self): result = beatport.list_genres() assert isinstance(result, list) assert len(result) >= 10 # almeno 10 generi for g in result: assert set(g.keys()) == {"slug", "id", "name"} assert isinstance(g["slug"], str) and g["slug"] assert isinstance(g["id"], int) and g["id"] > 0 assert isinstance(g["name"], str) and g["name"] def test_sorted_alphabetically_by_name(self): result = beatport.list_genres() names = [g["name"] for g in result] assert names == sorted(names, key=str.casefold) def test_melodic_house_techno_present(self): result = beatport.list_genres() slugs = [g["slug"] for g in result] assert "melodic-house-techno" in slugs ``` - [ ] **Step 2: Verifica che i test falliscano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: FAIL con `ModuleNotFoundError: No module named 'core.beatport'` - [ ] **Step 3: Implementa core/beatport.py minimo** ```python """Fetch Top 100 Beatport per genere. Approccio: estrai il JSON `__NEXT_DATA__` dal HTML della pagina Next.js. Vedi docs/superpowers/specs/2026-07-14-beatport-charts-design.md. """ from __future__ import annotations # Mappa slug URL Beatport → (numeric_id, display_name) # Enumerata via scripts/refresh_beatport_genres.py (vedi Task 1). # Sostituisci il blocco qui sotto con l'output completo di quello script. GENRES: dict[str, tuple[int, str]] = { "afro-house": (89, "Afro House"), "deep-house": (12, "Deep House"), "drum-bass": (1, "Drum & Bass"), "dubstep": (18, "Dubstep"), "house": (5, "House"), "indie-dance": (37, "Indie Dance"), "melodic-house-techno": (90, "Melodic House & Techno"), "minimal-deep-tech": (14, "Minimal / Deep Tech"), "organic-house-downtempo": (93, "Organic House / Downtempo"), "progressive-house": (15, "Progressive House"), "tech-house": (11, "Tech House"), "techno-peak-time-driving": (6, "Techno (Peak Time / Driving)"), "techno-raw-deep-hypnotic": (92, "Techno (Raw / Deep / Hypnotic)"), "trance": (7, "Trance"), } def list_genres() -> list[dict]: """Ritorna [{slug, id, name}, ...] ordinato alfabeticamente per name.""" result = [ {"slug": slug, "id": gid, "name": name} for slug, (gid, name) in GENRES.items() ] result.sort(key=lambda g: g["name"].casefold()) return result ``` **Se hai la lista completa dallo script della Task 1:** sostituisci il blocco `GENRES = {…}` con l'output completo. Il test `test_returns_list_of_dicts` accetta ≥10 generi, quindi la lista parziale sopra è già valida. - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: 3 test PASS - [ ] **Step 5: Commit** ```bash git add core/beatport.py tests/test_beatport.py git -c user.email=info@djluza.com commit -m "beatport: GENRES + list_genres() con test" ``` --- ## Task 3: BeatportTrack dataclass — TDD **Files:** - Modify: `core/beatport.py` - Modify: `tests/test_beatport.py` - [ ] **Step 1: Aggiungi test per BeatportTrack** Aggiungi in fondo a `tests/test_beatport.py`: ```python class TestBeatportTrack: def test_display_format(self): t = beatport.BeatportTrack( position=1, title="Hot Sauce", mix="Extended", artists="Kapuchon, Miss Monique & GLZ", duration_sec=336, beatport_id=12345, ) assert t.display == "Kapuchon, Miss Monique & GLZ – Hot Sauce (Extended) (5:36)" def test_display_pads_seconds(self): t = beatport.BeatportTrack( position=1, title="X", mix="Y", artists="A", duration_sec=65, beatport_id=1, ) assert t.display.endswith("(1:05)") def test_spotify_query(self): t = beatport.BeatportTrack( position=1, title="Hot Sauce", mix="Extended", artists="Kapuchon, Miss Monique & GLZ", duration_sec=336, beatport_id=12345, ) assert t.spotify_query == "Kapuchon, Miss Monique & GLZ Hot Sauce" def test_is_frozen(self): t = beatport.BeatportTrack( position=1, title="X", mix="Y", artists="A", duration_sec=1, beatport_id=1, ) with pytest.raises(Exception): t.title = "Z" # frozen=True impedisce mutazione ``` - [ ] **Step 2: Verifica fail** Run: `python3 -m pytest tests/test_beatport.py::TestBeatportTrack -v` Expected: FAIL con `AttributeError: module 'core.beatport' has no attribute 'BeatportTrack'` - [ ] **Step 3: Aggiungi la dataclass a core/beatport.py** In cima a `core/beatport.py`, prima di `GENRES`, aggiungi: ```python from dataclasses import dataclass @dataclass(frozen=True) class BeatportTrack: position: int title: str mix: str # es. "Extended Mix", "Original Mix", "" artists: str # es. "A, B & C" già formattato duration_sec: int beatport_id: int @property def display(self) -> str: """Formato testo compatibile coi file .txt curati a mano: 'Artista – Titolo (Mix) (M:SS)'.""" m, s = divmod(self.duration_sec, 60) return f"{self.artists} – {self.title} ({self.mix}) ({m}:{s:02d})" @property def spotify_query(self) -> str: """Query pura per la search Spotify (senza mix name, che disturba il matching su titoli tipo 'Extended Mix').""" return f"{self.artists} {self.title}" ``` - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: 7 test PASS - [ ] **Step 5: Commit** ```bash git add core/beatport.py tests/test_beatport.py git -c user.email=info@djluza.com commit -m "beatport: BeatportTrack dataclass con display + spotify_query" ``` --- ## Task 4: Estrazione `__NEXT_DATA__` + eccezioni — TDD **Files:** - Modify: `core/beatport.py` - Modify: `tests/test_beatport.py` - [ ] **Step 1: Aggiungi test per parser interno** Aggiungi in fondo a `tests/test_beatport.py`: ```python class TestExtractNextData: def test_extracts_json_from_valid_html(self, fixtures_dir): html = (fixtures_dir / "beatport_melodic_top100.html").read_text() data = beatport._extract_next_data(html) assert isinstance(data, dict) assert "props" in data def test_missing_script_raises(self): with pytest.raises(beatport.BeatportParseError, match="__NEXT_DATA__ non trovato"): beatport._extract_next_data("
nulla") def test_malformed_json_raises(self): broken = '' with pytest.raises(beatport.BeatportParseError, match="JSON malformato"): beatport._extract_next_data(broken) ``` - [ ] **Step 2: Verifica fail** Run: `python3 -m pytest tests/test_beatport.py::TestExtractNextData -v` Expected: FAIL - [ ] **Step 3: Implementa eccezioni + `_extract_next_data`** Aggiungi a `core/beatport.py` (dopo gli import): ```python import json import re class BeatportError(Exception): """Base per errori Beatport.""" class BeatportUnreachableError(BeatportError): """Rete o server Beatport non raggiungibile / 5xx.""" class BeatportParseError(BeatportError): """HTML/JSON ricevuto ma non conforme allo schema atteso.""" _NEXT_DATA_RE = re.compile( r'', re.DOTALL, ) def _extract_next_data(html: str) -> dict: """Estrae il payload JSON dallo script <__NEXT_DATA__> di Next.js.""" m = _NEXT_DATA_RE.search(html) if not m: raise BeatportParseError("__NEXT_DATA__ non trovato nella pagina Beatport") try: return json.loads(m.group(1)) except json.JSONDecodeError as e: raise BeatportParseError(f"__NEXT_DATA__ JSON malformato: {e}") from e ``` - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: 10 test PASS - [ ] **Step 5: Commit** ```bash git add core/beatport.py tests/test_beatport.py git -c user.email=info@djluza.com commit -m "beatport: estrazione __NEXT_DATA__ + eccezioni tipizzate" ``` --- ## Task 5: Walk JSON → list[BeatportTrack] — TDD **Files:** - Modify: `core/beatport.py` - Modify: `tests/test_beatport.py` - [ ] **Step 1: Aggiungi test parser tracce** ⚠️ **Prima di scrivere questo test rileggi l'output della Task 1 Step 3** — le chiavi effettive del JSON Beatport potrebbero differire. Il test qui sotto assume `id`, `name`, `mix_name`, `artists` (list of `{name}`), `length_ms`. Se sono diverse, adatta sia il test che l'implementazione coerentemente. Aggiungi in fondo a `tests/test_beatport.py`: ```python class TestParseTracks: def test_extracts_100_tracks(self, fixtures_dir): html = (fixtures_dir / "beatport_melodic_top100.html").read_text() data = beatport._extract_next_data(html) tracks = beatport._parse_tracks(data) assert len(tracks) == 100 def test_positions_are_sequential_1_to_100(self, fixtures_dir): html = (fixtures_dir / "beatport_melodic_top100.html").read_text() tracks = beatport._parse_tracks(beatport._extract_next_data(html)) positions = [t.position for t in tracks] assert positions == list(range(1, 101)) def test_track_shape(self, fixtures_dir): html = (fixtures_dir / "beatport_melodic_top100.html").read_text() tracks = beatport._parse_tracks(beatport._extract_next_data(html)) first = tracks[0] assert isinstance(first, beatport.BeatportTrack) assert first.title assert first.artists assert first.duration_sec > 0 assert first.beatport_id > 0 def test_schema_missing_results_raises(self): with pytest.raises(beatport.BeatportParseError, match="results"): beatport._parse_tracks({"props": {"pageProps": {}}}) ``` - [ ] **Step 2: Verifica fail** Run: `python3 -m pytest tests/test_beatport.py::TestParseTracks -v` Expected: FAIL - [ ] **Step 3: Implementa `_parse_tracks`** Aggiungi a `core/beatport.py`: ```python def _find_tracks_results(data: dict) -> list[dict]: """Cerca dentro le queries dehydrated il primo `results` che ha almeno 50 elementi e la shape di una track (chiave `id` presente).""" try: queries = data["props"]["pageProps"]["dehydratedState"]["queries"] except (KeyError, TypeError) as e: raise BeatportParseError(f"schema JSON inatteso (queries mancanti): {e}") from e for q in queries: state_data = (q or {}).get("state", {}).get("data") if not isinstance(state_data, dict): continue results = state_data.get("results") if isinstance(results, list) and len(results) >= 50: if results and isinstance(results[0], dict) and "id" in results[0]: return results raise BeatportParseError("nessun `results` di 50+ track trovato in __NEXT_DATA__") def _format_artists(artists_field: object) -> str: """Beatport ritorna artists come lista di dict {name, ...}. Formatta come 'A, B & C' (& prima dell'ultimo).""" if not isinstance(artists_field, list) or not artists_field: return "" names = [a.get("name", "") for a in artists_field if isinstance(a, dict)] names = [n for n in names if n] if not names: return "" if len(names) == 1: return names[0] return ", ".join(names[:-1]) + " & " + names[-1] def _parse_tracks(data: dict) -> list[BeatportTrack]: """Trasforma i track dict di Beatport in BeatportTrack ordinati per posizione.""" raw = _find_tracks_results(data) out: list[BeatportTrack] = [] for i, item in enumerate(raw, 1): try: length_ms = int(item.get("length_ms") or 0) track = BeatportTrack( position=i, title=str(item.get("name") or "").strip(), mix=str(item.get("mix_name") or "").strip(), artists=_format_artists(item.get("artists")), duration_sec=length_ms // 1000, beatport_id=int(item.get("id") or 0), ) except (TypeError, ValueError) as e: raise BeatportParseError(f"track[{i}] shape inattesa: {e}") from e out.append(track) return out ``` **Se le chiavi effettive del JSON (visto in Task 1 Step 3) sono diverse:** adatta `item.get("name")`, `item.get("mix_name")`, `item.get("artists")`, `item.get("length_ms")`, `item.get("id")` di conseguenza. - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: 14 test PASS Se `test_track_shape` fallisce con `duration_sec == 0`: la chiave della durata non è `length_ms` — controlla nell'output Task 1 Step 3 e correggi. - [ ] **Step 5: Commit** ```bash git add core/beatport.py tests/test_beatport.py git -c user.email=info@djluza.com commit -m "beatport: parse JSON → list[BeatportTrack]" ``` --- ## Task 6: fetch_top100 con HTTP, retry, cache — TDD **Files:** - Modify: `core/beatport.py` - Modify: `tests/test_beatport.py` - [ ] **Step 1: Aggiungi test per fetch_top100 (mock su curl_cffi)** `responses` non funziona con `curl_cffi` — usa `unittest.mock.patch` sul modulo importato. Aggiungi in fondo a `tests/test_beatport.py`: ```python from unittest.mock import patch, MagicMock from freezegun import freeze_time def _mock_response(text: str, status_code: int = 200) -> MagicMock: resp = MagicMock() resp.text = text resp.status_code = status_code def _raise(): if status_code >= 400: raise Exception(f"HTTP {status_code}") resp.raise_for_status = _raise return resp class TestFetchTop100: @pytest.fixture def fixture_html(self, fixtures_dir): return (fixtures_dir / "beatport_melodic_top100.html").read_text() def test_success_returns_100_tracks(self, fixture_html): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response(fixture_html, 200) tracks = beatport.fetch_top100("melodic-house-techno") assert len(tracks) == 100 def test_invalid_slug_raises_value_error(self): with pytest.raises(ValueError, match="slug"): beatport.fetch_top100("not-a-real-genre") def test_5xx_retries_and_raises_unreachable(self): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response("", 503) with patch("core.beatport.time.sleep"): # skip backoff nei test with pytest.raises(beatport.BeatportUnreachableError): beatport.fetch_top100("melodic-house-techno") assert mock_get.call_count == 3 # 1 + 2 retry def test_cache_hit_within_ttl(self, fixture_html): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response(fixture_html, 200) beatport.fetch_top100("melodic-house-techno") beatport.fetch_top100("melodic-house-techno") assert mock_get.call_count == 1 # seconda call servita da cache def test_cache_expires_after_ttl(self, fixture_html): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response(fixture_html, 200) with freeze_time("2026-01-01 10:00:00") as frozen: beatport.fetch_top100("melodic-house-techno") frozen.tick(delta=beatport._CACHE_TTL_SEC + 1) beatport.fetch_top100("melodic-house-techno") assert mock_get.call_count == 2 def test_force_refresh_bypasses_cache(self, fixture_html): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response(fixture_html, 200) beatport.fetch_top100("melodic-house-techno") beatport.fetch_top100("melodic-house-techno", force_refresh=True) assert mock_get.call_count == 2 def test_uses_chrome_impersonation(self, fixture_html): beatport._cache.clear() with patch("core.beatport._cffi_requests.get") as mock_get: mock_get.return_value = _mock_response(fixture_html, 200) beatport.fetch_top100("melodic-house-techno") # verifica che venga passato impersonate="chrome..." call_kwargs = mock_get.call_args.kwargs assert "impersonate" in call_kwargs assert call_kwargs["impersonate"].startswith("chrome") ``` - [ ] **Step 2: Verifica fail** Run: `python3 -m pytest tests/test_beatport.py::TestFetchTop100 -v` Expected: FAIL - [ ] **Step 3: Implementa fetch_top100 + cache + retry (via curl_cffi)** Aggiungi a `core/beatport.py`: ```python import time # curl_cffi bypassa Cloudflare Managed Challenge tramite TLS impersonation. # Import aliased così i test possono fare patch("core.beatport._cffi_requests.get"). from curl_cffi import requests as _cffi_requests _IMPERSONATE = "chrome131" # aggiorna se CF rompe il fingerprint _REQUEST_TIMEOUT = 15 _MAX_ATTEMPTS = 3 _BACKOFF_SEC = [1, 3] # attese fra tentativo 1→2 e 2→3 _CACHE_TTL_SEC = 15 * 60 # Cache in-memory: slug → (timestamp_epoch, list[BeatportTrack]) _cache: dict[str, tuple[float, list[BeatportTrack]]] = {} def _url_for(slug: str) -> str: gid, _ = GENRES[slug] return f"https://www.beatport.com/genre/{slug}/{gid}/top-100" def _do_get(url: str) -> str: """GET con retry e backoff. Solleva BeatportUnreachableError su fallimento definitivo.""" last_exc: Exception | None = None for attempt in range(_MAX_ATTEMPTS): try: resp = _cffi_requests.get( url, impersonate=_IMPERSONATE, timeout=_REQUEST_TIMEOUT, ) if resp.status_code >= 500 or resp.status_code == 403: # 403 = Cloudflare challenge non superata → retry raise Exception(f"HTTP {resp.status_code}") resp.raise_for_status() return resp.text except Exception as e: last_exc = e if attempt < _MAX_ATTEMPTS - 1: time.sleep(_BACKOFF_SEC[attempt]) raise BeatportUnreachableError(f"Beatport irraggiungibile dopo {_MAX_ATTEMPTS} tentativi: {last_exc}") def fetch_top100(slug: str, force_refresh: bool = False) -> list[BeatportTrack]: """Fetches la Top 100 Beatport per il genere dato. Cache in-memory 15 min. `force_refresh=True` bypassa la cache. Raises: ValueError: se slug non è in GENRES. BeatportUnreachableError: rete/5xx dopo i retry. BeatportParseError: HTML/JSON non conforme allo schema. """ if slug not in GENRES: raise ValueError(f"slug genere non valido: {slug!r}") now = time.time() if not force_refresh: cached = _cache.get(slug) if cached and (now - cached[0]) < _CACHE_TTL_SEC: return cached[1] html = _do_get(_url_for(slug)) data = _extract_next_data(html) tracks = _parse_tracks(data) _cache[slug] = (now, tracks) return tracks ``` - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_beatport.py -v` Expected: 21 test PASS Nota: il test `test_5xx_retries_and_raises_unreachable` fa 3 richieste con backoff 1s+3s → dura ~4s. Ok. - [ ] **Step 5: Commit** ```bash git add core/beatport.py tests/test_beatport.py git -c user.email=info@djluza.com commit -m "beatport: fetch_top100 con HTTP + retry + cache 15min" ``` --- ## Task 7: Aggiungi search_track() a spotify_client.py — TDD **Files:** - Modify: `core/spotify_client.py` - Create: `tests/test_spotify_client.py` Il modulo `core/spotify_client.py` esiste ma **non ha** una funzione `search_track()` pubblica. La aggiungiamo. - [ ] **Step 1: Scrivi test per search_track** Crea `tests/test_spotify_client.py`: ```python """Test per core.spotify_client.search_track().""" import pytest import responses from core import spotify_client class TestSearchTrack: @responses.activate def test_returns_track_dict_when_found(self): responses.add( responses.GET, "https://api.spotify.com/v1/search", json={ "tracks": { "items": [ { "id": "abc123", "name": "Hot Sauce", "artists": [{"name": "Kapuchon"}], "external_urls": {"spotify": "https://open.spotify.com/track/abc123"}, } ] } }, status=200, ) result = spotify_client.search_track("fake-token", "Kapuchon Hot Sauce") assert result is not None assert result["id"] == "abc123" assert result["url"] == "https://open.spotify.com/track/abc123" @responses.activate def test_returns_none_when_no_results(self): responses.add( responses.GET, "https://api.spotify.com/v1/search", json={"tracks": {"items": []}}, status=200, ) result = spotify_client.search_track("fake-token", "no-match") assert result is None @responses.activate def test_sends_bearer_token(self): responses.add( responses.GET, "https://api.spotify.com/v1/search", json={"tracks": {"items": []}}, status=200, ) spotify_client.search_track("my-token", "q") assert responses.calls[0].request.headers["Authorization"] == "Bearer my-token" @responses.activate def test_query_params_correct(self): responses.add( responses.GET, "https://api.spotify.com/v1/search", json={"tracks": {"items": []}}, status=200, ) spotify_client.search_track("t", "Kapuchon Hot Sauce") params = responses.calls[0].request.params assert params["q"] == "Kapuchon Hot Sauce" assert params["type"] == "track" assert params["limit"] == "1" ``` - [ ] **Step 2: Verifica fail** Run: `python3 -m pytest tests/test_spotify_client.py -v` Expected: FAIL con `AttributeError: module 'core.spotify_client' has no attribute 'search_track'` - [ ] **Step 3: Aggiungi search_track a core/spotify_client.py** Aggiungi in fondo a `core/spotify_client.py`: ```python def search_track(token: str, query: str) -> dict | None: """Cerca un brano su Spotify e ritorna il primo match. Ritorna un dict con almeno {id, url, name, artists} oppure None se nessun match. Solleva requests.HTTPError su errori server / auth. """ resp = requests.get( "https://api.spotify.com/v1/search", headers={"Authorization": f"Bearer {token}"}, params={"q": query, "type": "track", "limit": 1}, timeout=15, ) resp.raise_for_status() items = resp.json().get("tracks", {}).get("items", []) if not items: return None t = items[0] return { "id": t.get("id"), "url": t.get("external_urls", {}).get("spotify", ""), "name": t.get("name", ""), "artists": ", ".join(a.get("name", "") for a in t.get("artists", [])), } ``` - [ ] **Step 4: Verifica test passano** Run: `python3 -m pytest tests/test_spotify_client.py -v` Expected: 4 test PASS - [ ] **Step 5: Commit** ```bash git add core/spotify_client.py tests/test_spotify_client.py git -c user.email=info@djluza.com commit -m "spotify: aggiunto search_track() per feature Beatport" ``` --- ## Task 8: Config — nuovo campo beatport_last_genre + version bump **Files:** - Modify: `core/config.py` - [ ] **Step 1: Aggiungi campo ai DEFAULTS + version bump** Modifica `core/config.py`: Cambia: ```python VERSION = "v1.7.15" ``` in: ```python VERSION = "v1.8.0" ``` Nel dict `DEFAULTS`, dopo `"theme": "dark",` aggiungi: ```python # ---- Beatport ---- "beatport_last_genre": "melodic-house-techno", # ultimo genere Top 100 caricato ``` - [ ] **Step 2: Verifica carica di default** ```bash python3 -c "from core.config import load_config; print(load_config()['beatport_last_genre'])" ``` Expected: `melodic-house-techno` (o il valore già in config.json se lo hai già) - [ ] **Step 3: Commit** ```bash git add core/config.py git -c user.email=info@djluza.com commit -m "config: v1.8.0 + beatport_last_genre nei DEFAULTS" ``` --- ## Task 9: main.py — nuovi metodi Api per bridge JS↔Python **Files:** - Modify: `main.py` - [ ] **Step 1: Ispeziona la classe Api esistente** Run: `grep -n "class Api\|def [a-z]" main.py | head -40` Individua dove si definiscono i metodi esposti a JS (probabile `class Api:` o `class WebviewApi:`), come vengono emessi i log/progress al frontend, e come funziona il pattern di stop. Annota qui: - Nome classe API: ______ - Metodo per emettere log al JS: ______ (es. `self.window.evaluate_js(...)` o simile) - Come viene passata `output_dir`: ______ - [ ] **Step 2: Aggiungi import in cima a main.py** ```python from dataclasses import asdict from pathlib import Path from core import beatport, spotify_client ``` Se `beatport` e `spotify_client` sono già importati diversamente (es. `from core.spotify_client import get_access_token`), aggiungili senza duplicare. - [ ] **Step 3: Aggiungi metodo beatport_genres** Nella classe Api, aggiungi: ```python def beatport_genres(self) -> list[dict]: """Ritorna la lista dei generi disponibili per il dropdown.""" return beatport.list_genres() ``` - [ ] **Step 4: Aggiungi metodo beatport_fetch_chart** ```python def beatport_fetch_chart(self, slug: str, force_refresh: bool = False) -> dict: """Fetches la Top 100 per il genere. Ritorna {ok, tracks} o {ok:False, error, message}.""" # Salva ultimo genere per la prossima apertura try: cfg = load_config() cfg["beatport_last_genre"] = slug save_config(cfg) except Exception: pass # non bloccare per errori di salvataggio config try: tracks = beatport.fetch_top100(slug, force_refresh=force_refresh) except ValueError as e: return {"ok": False, "error": "invalid_genre", "message": str(e)} except beatport.BeatportUnreachableError as e: return {"ok": False, "error": "unreachable", "message": str(e)} except beatport.BeatportParseError as e: return {"ok": False, "error": "parse", "message": str(e)} return {"ok": True, "tracks": [asdict(t) for t in tracks]} ``` `load_config` e `save_config` sono già importati da `core.config` — verifica. - [ ] **Step 5: Aggiungi metodo beatport_check_existing** ```python def beatport_check_existing(self, tracks: list[dict], genre_name: str) -> list[bool]: """Per ogni track ritorna True se esiste già un file .mp3 corrispondente nella cartella output di quel genere.""" cfg = load_config() out_dir = Path(cfg["output_dir"]) / "Beatport" / genre_name if not out_dir.exists(): return [False] * len(tracks) # Usa lo stesso naming del downloader esistente: "Artista - Titolo.mp3" # (verifica il pattern reale ispezionando core/downloader.py) existing = {p.stem for p in out_dir.glob("*.mp3")} result = [] for t in tracks: expected_stem = f"{t['artists']} - {t['title']}" # Match esatto o fuzzy prefisso (il downloader può aggiungere " (Extended)" ecc.) found = any(expected_stem in name or name.startswith(expected_stem) for name in existing) result.append(found) return result ``` **Nota:** l'esatto naming pattern lo produce il downloader esistente. Verifica in `core/downloader.py` come si chiamano i file di output (potrebbe usare `%(title)s` di yt-dlp). Se diverso, adatta `expected_stem`. - [ ] **Step 6: Aggiungi metodo beatport_download_selected** ```python def beatport_download_selected(self, tracks: list[dict], genre_name: str) -> None: """Scarica i brani selezionati riusando il flusso Spotify search → downloader.""" from core import downloader # locale per evitare circular imports cfg = load_config() out_dir = Path(cfg["output_dir"]) / "Beatport" / genre_name out_dir.mkdir(parents=True, exist_ok=True) client_id = cfg.get("client_id", "").strip() client_secret = cfg.get("client_secret", "").strip() if not client_id or not client_secret: self._emit_log("✗ Credenziali Spotify mancanti — vai su Impostazioni") return try: token = spotify_client.get_access_token(client_id, client_secret) except Exception as e: self._emit_log(f"✗ Errore auth Spotify: {e}") return downloader.reset_stop() total = len(tracks) for i, t in enumerate(tracks, 1): if downloader._stop_event.is_set(): self._emit_log(f"[{i}/{total}] Interrotto dall'utente") return query = f"{t['artists']} {t['title']}" self._emit_log(f"[{i}/{total}] Cerco su Spotify: {query}") try: found = spotify_client.search_track(token, query) except Exception as e: self._emit_log(f"[{i}/{total}] ✗ Errore Spotify: {e}") continue if not found: self._emit_log(f"[{i}/{total}] ✗ Non trovato su Spotify") continue self._emit_log(f"[{i}/{total}] Download da YouTube: {found['name']}") try: downloader.download_from_spotify_url( # nome esatto lo verifichi found["url"], output_dir=str(out_dir), on_progress=self._emit_progress, ) self._emit_log(f"[{i}/{total}] ✓ {t['title']}") except Exception as e: self._emit_log(f"[{i}/{total}] ✗ Download fallito: {e}") ``` **⚠️ Da adattare:** - `self._emit_log(...)` e `self._emit_progress(...)`: sostituisci coi metodi reali usati dagli altri tab per pushare log/progress al JS (visti nella Task 9 Step 1) - `downloader.download_from_spotify_url(...)`: sostituisci con il nome effettivo della funzione pubblica del downloader (ispeziona `core/downloader.py`) - [ ] **Step 7: Smoke test manuale del bridge** Avvia l'app e apri la console browser (menu View del webview): ```javascript window.pywebview.api.beatport_genres().then(console.log) ``` Expected: array di 14+ oggetti `{slug, id, name}` ```javascript window.pywebview.api.beatport_fetch_chart("melodic-house-techno").then(console.log) ``` Expected: `{ok: true, tracks: [ … 100 oggetti … ]}` Se fallisce: leggi il messaggio, sistema il metodo Python, riavvia l'app. - [ ] **Step 8: Commit** ```bash git add main.py git -c user.email=info@djluza.com commit -m "main: 4 metodi Api per feature Beatport" ``` --- ## Task 10: UI — HTML tab + section **Files:** - Modify: `webui/index.html` - [ ] **Step 1: Trova la tab bar e i pattern esistenti** Run: `grep -n 'data-tab\|class=".*tab' webui/index.html | head -30` Individua: - La `