Python scraper for public CPA license registry (CPAverify) — API-based, ~800k input records

Description: I need a Python script that enriches a list of accountant names against CPAverify (ald.nasba.org), the US national public CPA license lookup run by NASBA. Input: CSV files totaling ~800,000 rows. Relevant columns: LAST_NAME, First_NAME, BUS_ST_CODE (US state). Sample file provided on hire. After deduplication by (last name, state) the real workload is roughly 300–400k unique searches. What the script must do: Deduplicate input to unique (last name, state) pairs. For each pair, query the CPAverify search, handling pagination since results can span multiple pages. Search URL pattern, shown with example values: https://ald.nasba.org/search/cpa/results?jurisdictionId=20&lastName=WASHINGTON&page=0 — where jurisdictionId is the state's numeric ID, lastName is the surname, and page starts at 0. For every licensee returned, fetch their detail record and capture every jurisdiction they hold a license in (one person can appear in multiple states, cross-linked via an llId parameter). For each license, extract: jurisdiction, license/certification number, license type, status, issue date, expiration date, years licensed, CPE ID, and enforcement/disciplinary field. Match results back to input rows on last name plus state, keeping first-name data available for fuzzy matching on my side. Output: SQLite database plus CSV exports, two tables: licensees (one row per person per source jurisdiction, with the search key that found them) and licenses (one row per person per jurisdiction license, foreign-keyed to licensees). Technical notes (already investigated — this should save you scoping time): The site is a React SPA; raw HTML requests return an empty shell. Do not quote me a Selenium/Playwright solution as the primary approach. The frontend calls a JSON backend at https://ald-api.nasba.org/api/... (confirmed working example: GET https://ald-api.nasba.org/api/jurisdictions/all?cpaverifyStateDropdownEnabled=1 returns the full jurisdiction-ID mapping, e.g. NY = 20, NJ = 114). Your first task is to identify the search and person-detail endpoints on that same host via DevTools and use them directly. Detail pages intermittently return server errors — the script needs retry with exponential backoff, and must log permanently failed IDs to a separate file rather than crashing or silently skipping. Checkpointing is mandatory: the run takes days; the script must resume exactly where it stopped after any interruption (SQLite state, not in-memory). Polite rate limiting (configurable delay/concurrency, sane defaults, proper User-Agent). I don't want the run to hammer the site or get the IP banned mid-job. Deliverable: CSV of all data acquired.