Repository navigation
Find lapsed, registerable domains in adjacent industries — no new vendor - #58
Merged
Merged
Conversation
Answers the actual ask: open the tab and see expired domains you can BUY that are relevant to the industry. The graph sources cannot do this. They only surface domains already linking to a competitor or ranking for a tracked keyword, which by construction returns more of the same vertical -- a vending operator got five vending companies. This generates names from the client's own vocabulary crossed with ADJACENT industries, then keeps the ones that had a site and are free to register today. The cost ordering is the design: generation is free, the archive check is free, and only names that once hosted a site reach the billed availability lookup. Reversing those steps would bill for every generated name, most of which nobody ever registered. DomCop was the alternative and it is 16/yr with no API on any tier; ICANN CZDS is free but ships whole multi-gigabyte TLD zone files, which a Worker cannot ingest. The Wayback availability endpoint is free, needs no key, and answers the one question that separates a dropped domain from an unregistered string. It also rate-limits: a live probe returned 429. That exposed a real cost bug here -- inconclusive archive answers are deliberately kept rather than treated as "never existed", so a throttled archive would have sent every generated name to the paid availability check. There is now a hard ceiling on billed checks per run, confirmed history is spent first, answers are cached in KV for 30 days, and a run whose archive checks all failed says so instead of quietly returning less.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Answers the actual ask: open the tab, see expired domains you can buy that are relevant to the industry.
Why the existing sources could never do this
They surface domains already linking to a competitor or ranking for a tracked keyword. That is relevance by construction — and it is also the ceiling. A vending operator gets five vending companies, because that is the entire population those sources can reach. It also mostly returns domains that are still owned, not buyable.
The free path, and why it is ordered this way
Step 2 is what makes a result an expired domain rather than an unregistered string, and putting it before step 3 is what makes this affordable. Reversed, you would bill for every generated name, most of which nobody ever registered.
On Delio's real keywords the generator produces
nutritionvending.com,foodbreakroom.com,dallasnutrition.com,snackvending.com— the adjacency that was missing.The alternatives, checked
.comalone is 100M+ domains, multi-gigabyte, daily. A Worker cannot ingest that..com.A cost bug this surfaced, and the guard for it
A live probe of archive.org returned 429 Too Many Requests. That matters because inconclusive archive answers are deliberately kept — treating them as "never existed" would discard real targets over someone else's outage. But without a ceiling, a throttled archive would have sent every generated name to the paid availability check: 60 names × 5 credits = 300, silently.
So:
MAX_AVAILABILITY_CHECKS)archiveUnavailableinstead of quietly returning lessAdjacent terms
One small model call per run via the already-configured OpenRouter key. The parser is strict and separately tested — a model that decides to explain itself must not turn a sentence into a domain name. No key configured, or the call fails: the search still runs on the client's own vocabulary, just narrower. Never throws; it is enrichment, not a precondition.
Verification
pnpm ci:checkclean;pnpm testgreen — 311 files, 3143 tests.e2e/expired-domains.spec.tsstill passes — panel mounts idle, quotes its ceiling, zero metered requests before a click.Not verified
No end-to-end run against live archive.org plus live availability — that bills, and archive.org is currently throttling this IP. The pieces are individually proven; the joined-up run is not.
🤖 Generated with Claude Code