Releases: FAAQJAVED/html-directory-scrapers
Releases · FAAQJAVED/html-directory-scrapers
V 1.2.1
Fixed
Both Engines
- Keyboard controls now work reliably — P, R, S/Q, W keys previously had
no effect unless pressed during a narrow window between page-loop iterations.
Root cause: the listener thread stored keys in_keybut the main loop
only calledget_key()once per page (every 10–30 s), so most keypresses
were missed. Fix: P and S/Q now setthreading.Eventflags (_pause_event,_stop_event) directly inside the listener thread, so they take effect
immediately — even when the main loop is blocked intime.sleep()or atqdmiteration. R clears_pause_eventimmediately. W remains stored
in_keyand is consumed by the main loop to print stats.InputControllergainsis_paused()andis_stopped()helper methods;
both scrapers updated to poll these instead of a localpausedvariable. - Dead-website retry: 0 → 1 — profile/crawl fetches now retry once on
ReadTimeoutand general connection errors.ConnectTimeout(TCP-level
failure — host is unreachable) still returns("", 0)immediately with
zero retries, so truly dead sites never stall the run.
Added
Documentation
docs/folder added to the repository root for visual assets.
Containsdocs/README.mdwith step-by-step instructions for recordingterminal-demo.gif(asciinema) and capturingexcel-preview.png.docs/.gitkeepensures Git tracks the empty folder.- README Output Preview — placeholder text replaced with an HTML comment
block that is ready to uncomment once assets are placed indocs/. - README Project Structure — updated to show the
docs/folder.