This happened to me and I found this tool super helpful to get my site unblocked: https://dnsblacklist.org/
I purchased a valuable premium domain to host a personal art collection (of anime cels). For some bizarre reason, the site was inaccessible from my work computer and it was de-listed from Google even if I typed the url itself into search.
I hired a square space specialist to figure out why, to no avail. I then begged our company’s CISO to investigate and it turns out we had some firewall setting on UniFi that blocked the domain because it appeared on a list. Once I checked way back, it turns out that it was as an anime porn aggregator years back. I personally reached out to all the web filters out there (Google, Symantec, bing) and one by one filed tickets for them to mark it as art instead of pornography and it worked. I am now properly crawled on Google but still MIA on Bing, search console is giving me some BS error that’s incomprehensible, typical of MSFT.
Another "haunted domain" check is by trying to post about it on social media. I ran into this with my current project's domain name. After building an MVP and trying to test the social sharing functionality, I found that Facebook was blocking the domain outright. Turns out there was some spamming from it years ago. Getting it unblocked was extra fun, as the page to request manual review was itself broken! Thankfully I knew someone on the inside who alerted the relevant team, but the whole experience was quite the novel speedbump.
> Ideally, search engine algorithms would give new domain owners a fresh start.
Sadly, I think this would be instantly gamed by abusers. They would release the domain name and attempt to register as a new owner or start repeatedly doing handoffs. It's difficult to tell who the owner is changing between and whether or not the new one is a better actor than the former.
If it's instantly released, then yes. But in this thread are reports where the offensive actions happened 15 years ago. After such a long time of "good behavior" it makes no sense for me to still keep the domain blocked/downranked.
> It's difficult to tell who the owner is changing between and whether or not the new one is a better actor than the former.
This doesn't seem like that hard of a problem to solve, because these are domains with negative reputation, i.e. worse than zero.
So if a) the domain is no longer hosting any of the stuff previously complained about and b) is no longer receiving new complaints over a period of a year, it costs you nothing to reset the domain to zero. Because the bad actors don't have to behave for a year to get back to zero, they can just register a new domain.
All you're doing is giving the new owner the same fresh start that anybody can get by buying a never before registered domain for the same price as a year's renewal on the existing one.
Using a domain every second year in that environment would get it a gradually raising rank where it isn't penalized/sanitized (by accident, on principle, etc) so every restart after a $30 pause year would be much more effective than a new domain.
Google product manager interview question - Write some code with an LLM tool that leverages a LLM to determine if the new owner of a domain is doing (a) same dodgy thing as prior owner that got flagged (b) different dodgy thing as prior owner but should be flagged (c) something completely innocuous (d) needs further review.
Follow up interview question. Update the code using your LLM code gen tool of choice that, when someone submits a complaint via an online form, feeds that complaint text back into your LLM to score it again. Points deduction if the candidate ever mentions informing the complainant of anything.
Some time ago I noticed that my side project (with a domain that is not haunted) shows up fine on Google but not Bing/DuckDuckGo.
So I checked the Bing Webmaster Tools. URL Inspection says "Discovered but not crawled - The inspected URL is known to Bing but has some issues which are preventing indexation. We recommend you to follow Bing Webmaster Guidelines to increase your chances of indexation."
That's quite unhelpful. What's more, when I open the "Live URL" tab, it says, in green: "URL can be indexed by Bing."
It's a simple static Hugo site hosted on Cloudflare R2 (DNS mapped directly to bucket). https://pagespeed.web.dev gives it a score of 100 in every category.
Yup. I've regularly had problems with a static site [0]. Sometimes it's a top hit for my name on Bing, sometimes completely unlisted. Seems to flip back and forth - with that same message you get.
It's a handwritten HTML website, enhanced with JS but not reliant on it, hosted on Cloudflare. Not quite a 100 in every PageSpeed category, but just about.
OP here, and yes, I've been getting that same message for musicbox.fun. I thought it just needed some time but I requested a fresh index two weeks ago, and nothing seems to have changed. :/
If it was easy to reset reputation with search engines what's stopping people from saying "under new management" every once in a while for an existing poor reputation domain? Probably better to just cut their losses and find another domain.
I've had an opposite experience. One domain I bought was used for an entirely different purpose in the past, which got linked on a Wikipedia article in references. This gives me some good link juice and at least matches the geo area of the previous business. Since it's an extremely niche entry and low on the list of references, I decided to be slightly naughty and not touch it for a couple of years. Not sure what's the opposite of haunted in this case, but it was just as surprising.
In their defense (and I don’t defend Google often), addressing this really well means:
- knowing all the complexities of every local, state, federal, international jurisdiction that might interfere with the whitelist
- awareness of the content in question which could be millions of subpages
- a customer support team that is definitely not incentivized based on tickets triaged per day, but is somehow incentivized to spend hours on “whale” tickets.
- going through ticket history and solving the problem for everyone now that its policy to solve this
- dealing with the inevitable rush of fraud that follows every tiny change in google systems
Basic SEO stuff, you have marketplaces that check history, you have domain search engines aggregating data from multiple sources - not only ahrefs.
Checking web archive is a basic operation to test if site was hosting anything fishy - not only pirated stuff or porn - often websites has been hacked and changed into link farms or simply were bought on aftermarket simply to use it's SEO value to pass the strength to other domains.
one other thing i would suggest is to set up a catch-all email for the domain and see what gets sent to it, sometimes you can access accounts associated with the domain, socials etc
I set up a catch-all for personal use and wasn't expecting to get flooded with emails.
I was getting business emails, people trying to send money by Zelle, etc.
I was kind of hoping to get something good that I could take action on in the market, so I left it on for a little bit, but then I felt bad that people's emails were not getting answered (at least bouncing), so I turned off the catch-all. Oh well.
Interesting. Domain as a unit of trust makes sense until it doesn't. Buying a second hand domain is like a second hand car. But you may not know it is second hand!
I think the mistake here is the redirect old to new. That is always risky so only do it if deseprate. In this case I would have done the redirect from new to old. Then just use the new as a vanity url.
"Ideally, search engine algorithms would give new domain owners a fresh start."
I don't think it's possible to fix this problem without also helping bad actors. Maybe it's a problem that just isn't worth fixing. Just don't buy preexisting domains unless it's a project big enough to justify the necessary cost of due diligence.
Not quite haunted but I've had people report that my website hosted on a .quest domain is blocked on their work computer. My best guess is that their filter thinks it's gaming related (it's not) or maybe they just block all "weird" domains.
Find out the IP address of the machine hosting the domain, then do a reverse lookup on that IP address. It might show the last domain hosted on that IP address.
This happened to me and I found this tool super helpful to get my site unblocked: https://dnsblacklist.org/
I purchased a valuable premium domain to host a personal art collection (of anime cels). For some bizarre reason, the site was inaccessible from my work computer and it was de-listed from Google even if I typed the url itself into search.
I hired a square space specialist to figure out why, to no avail. I then begged our company’s CISO to investigate and it turns out we had some firewall setting on UniFi that blocked the domain because it appeared on a list. Once I checked way back, it turns out that it was as an anime porn aggregator years back. I personally reached out to all the web filters out there (Google, Symantec, bing) and one by one filed tickets for them to mark it as art instead of pornography and it worked. I am now properly crawled on Google but still MIA on Bing, search console is giving me some BS error that’s incomprehensible, typical of MSFT.
I'd be somewhat interested in seeing the cels. :)
Another "haunted domain" check is by trying to post about it on social media. I ran into this with my current project's domain name. After building an MVP and trying to test the social sharing functionality, I found that Facebook was blocking the domain outright. Turns out there was some spamming from it years ago. Getting it unblocked was extra fun, as the page to request manual review was itself broken! Thankfully I knew someone on the inside who alerted the relevant team, but the whole experience was quite the novel speedbump.
I faced the same issue with one of my project. But, as i don't know anybody at Facebook, I left the domain and buy a new one.
> Ideally, search engine algorithms would give new domain owners a fresh start.
Sadly, I think this would be instantly gamed by abusers. They would release the domain name and attempt to register as a new owner or start repeatedly doing handoffs. It's difficult to tell who the owner is changing between and whether or not the new one is a better actor than the former.
If it's instantly released, then yes. But in this thread are reports where the offensive actions happened 15 years ago. After such a long time of "good behavior" it makes no sense for me to still keep the domain blocked/downranked.
> It's difficult to tell who the owner is changing between and whether or not the new one is a better actor than the former.
This doesn't seem like that hard of a problem to solve, because these are domains with negative reputation, i.e. worse than zero.
So if a) the domain is no longer hosting any of the stuff previously complained about and b) is no longer receiving new complaints over a period of a year, it costs you nothing to reset the domain to zero. Because the bad actors don't have to behave for a year to get back to zero, they can just register a new domain.
All you're doing is giving the new owner the same fresh start that anybody can get by buying a never before registered domain for the same price as a year's renewal on the existing one.
Using a domain every second year in that environment would get it a gradually raising rank where it isn't penalized/sanitized (by accident, on principle, etc) so every restart after a $30 pause year would be much more effective than a new domain.
Google product manager interview question - Write some code with an LLM tool that leverages a LLM to determine if the new owner of a domain is doing (a) same dodgy thing as prior owner that got flagged (b) different dodgy thing as prior owner but should be flagged (c) something completely innocuous (d) needs further review.
Please don't give Google ideas for more ways they can have an algorithm arbitrarily screw you over with no recourse, they're listening.
Well, current approach guarantees you’re getting screwed over. Any improvement is beneficial unless it blocks a better approach?
You're looking at this from the perspective of a haunted domain owner. And from that perspective your idea is fine.
A good technique to evaluate ideas though is to try and view it from different perspectives.
In this case from the owner of a non-haunted domain. Can you see any potential problem with your idea when viewed from that perspective?
Now, if there are potential problems, consider the relative sizes of the two groups. Do the benefits to one outweigh harm to the other?
This technique can be used every day with pretty much any idea.
Follow up interview question. Update the code using your LLM code gen tool of choice that, when someone submits a complaint via an online form, feeds that complaint text back into your LLM to score it again. Points deduction if the candidate ever mentions informing the complainant of anything.
Sadly, the same holds true for IP addresses.
Some time ago I noticed that my side project (with a domain that is not haunted) shows up fine on Google but not Bing/DuckDuckGo.
So I checked the Bing Webmaster Tools. URL Inspection says "Discovered but not crawled - The inspected URL is known to Bing but has some issues which are preventing indexation. We recommend you to follow Bing Webmaster Guidelines to increase your chances of indexation."
That's quite unhelpful. What's more, when I open the "Live URL" tab, it says, in green: "URL can be indexed by Bing."
It's a simple static Hugo site hosted on Cloudflare R2 (DNS mapped directly to bucket). https://pagespeed.web.dev gives it a score of 100 in every category.
Anyone else had something like this happen?
Yup. I've regularly had problems with a static site [0]. Sometimes it's a top hit for my name on Bing, sometimes completely unlisted. Seems to flip back and forth - with that same message you get.
It's a handwritten HTML website, enhanced with JS but not reliant on it, hosted on Cloudflare. Not quite a 100 in every PageSpeed category, but just about.
[0] https://jamesmilne.org/
OP here, and yes, I've been getting that same message for musicbox.fun. I thought it just needed some time but I requested a fresh index two weeks ago, and nothing seems to have changed. :/
Automattic.com was bought (no idea if it was unregistered / acquired) by Matt Mullenweg when he set up the company. He also bought https://a8c.com.
Here in the UK with EE/BT that correctly redirects to automattic.com, but it might not for you depending on your ISP.
The wayback machine shows adult content links prior to the domain being put on sale, hence the blocking.
see also landslide.com - a domain that should never have been reused imo
The usual version of this is the popular SEO technique of buying an aged domain with a few backlinks and slapping a wordpress on it.
If it was easy to reset reputation with search engines what's stopping people from saying "under new management" every once in a while for an existing poor reputation domain? Probably better to just cut their losses and find another domain.
I've had an opposite experience. One domain I bought was used for an entirely different purpose in the past, which got linked on a Wikipedia article in references. This gives me some good link juice and at least matches the geo area of the previous business. Since it's an extremely niche entry and low on the list of references, I decided to be slightly naughty and not touch it for a couple of years. Not sure what's the opposite of haunted in this case, but it was just as surprising.
Not always the easiest thing to do. A haunted domain could have been haunted 15 years ago. And Google refuses to tell you why or fix their system.
Just one more place where the web gets screwed by a company too big to have to do basic customer service.
In their defense (and I don’t defend Google often), addressing this really well means:
- knowing all the complexities of every local, state, federal, international jurisdiction that might interfere with the whitelist
- awareness of the content in question which could be millions of subpages
- a customer support team that is definitely not incentivized based on tickets triaged per day, but is somehow incentivized to spend hours on “whale” tickets.
- going through ticket history and solving the problem for everyone now that its policy to solve this
- dealing with the inevitable rush of fraud that follows every tiny change in google systems
Basic SEO stuff, you have marketplaces that check history, you have domain search engines aggregating data from multiple sources - not only ahrefs.
Checking web archive is a basic operation to test if site was hosting anything fishy - not only pirated stuff or porn - often websites has been hacked and changed into link farms or simply were bought on aftermarket simply to use it's SEO value to pass the strength to other domains.
Anyways good point regarding email filters.
one other thing i would suggest is to set up a catch-all email for the domain and see what gets sent to it, sometimes you can access accounts associated with the domain, socials etc
I have an interesting 3-letter.net
I set up a catch-all for personal use and wasn't expecting to get flooded with emails.
I was getting business emails, people trying to send money by Zelle, etc.
I was kind of hoping to get something good that I could take action on in the market, so I left it on for a little bit, but then I felt bad that people's emails were not getting answered (at least bouncing), so I turned off the catch-all. Oh well.
I do that and get the occasional account signup. I also ban addresses that fet sent spam, which happens more than the account signups.
Interesting. Domain as a unit of trust makes sense until it doesn't. Buying a second hand domain is like a second hand car. But you may not know it is second hand!
I think the mistake here is the redirect old to new. That is always risky so only do it if deseprate. In this case I would have done the redirect from new to old. Then just use the new as a vanity url.
"Ideally, search engine algorithms would give new domain owners a fresh start."
I don't think it's possible to fix this problem without also helping bad actors. Maybe it's a problem that just isn't worth fixing. Just don't buy preexisting domains unless it's a project big enough to justify the necessary cost of due diligence.
"Maybe it's a problem that just isn't worth fixing."
There is a finite amount of short, memorisable names.
I wonder if there’s a market for rehabilitating domain names
*exorcizing domain names
Not quite haunted but I've had people report that my website hosted on a .quest domain is blocked on their work computer. My best guess is that their filter thinks it's gaming related (it's not) or maybe they just block all "weird" domains.
unfortunately, blocking newer TLDs altogether seems common
For running a mail server every new domain is haunted.
And cloud server IP…
I feel like this should be the registrar's responsibility. Least they could do is give a disclaimer and/or a heavy discount.
I’ll add: and if you lease a VPS, check out its address reputation and reverse DNS record.
Isn't it pretty safe to just assume that any IP addresses belonging to public clouds, especially cheap ones, have bad reputations?
How?
I'm not the person you were replying to, but in the past, I've just used an IP reputation checking website, such as:
https://www.apivoid.com/tools/ip-reputation-check/
Website unusable: Captcha forever waits using latest Firefox on latest iPhone13/iOS 18.0
Find out the IP address of the machine hosting the domain, then do a reverse lookup on that IP address. It might show the last domain hosted on that IP address.
Using dig:
$>dig yourdomain.tld
1.2.3.4
$>dig -x 1.2.3.4
evilcorp.com
that is amazing
Yet another valuable use for the WayBack Machine, glad it got a mention.
As someone who knows what active persecution on this site is I relish the opportunity to say what I really know under a pseudonym.