Been thinking about this a lot [1]. Will this fundamentally change how people find and access information? How do you create an experience so compelling that it replaces the current paradigm?
The future promised in Star Trek and even Apple's Knowledge Navigator [2] from 1987 still feels distant. In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever.
Combining two broken systems - compromised search engines and unreliable LLMs - seems unlikely to yield that vision. Legacy, ad-based search, has devolved into a wasteland of misaligned incentives, conflict of interest and prolifirated the web full of content farms optimized for ads and algos instead of humans.
Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermiediaries want them to see - which means a different business model in seach, where there are no intermediaries. I do not see a way around this. Advancing models without advancing search is like having a michelin star chef work with spoiled ingredients.
I am cautiously optimistic we will eventually get there, but boy, we will need a fundamentally different setup in terms of incentives involved in information consumption, both in tech and society.
Great find on the knowledge navigator, I had never seen it but I was a toddler when it was released haha.
It's interesting how prescient it was, but I'm more struck wondering--would anyone in 1987 have predicted it would take 40+ years to achieve this? Obviously this was speculative at the time but I know history is rife with examples of AI experts since the 60s proclaiming AGI was only a few years away
Is this time really different? There's certainly been a huge jump in capabilities in just a few years but given the long history of overoptimistic predictions I'm not confident
> In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever.
This also seems like a little ridiculous premise. Any confident statement about the real world is never fully reliable. If star trek were realistic the computer would have been wrong once in a while (preferably with dramatically disastrous consequences)—just as the humans it likely was built around are frequently wrong, even via consensus.
LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mind.
Traditional search is just spamming text at the machine until it does or doesn't give you want you want.
That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.
I agree that LLMs have opened modalities we didn't have before, namely:
- natural language input
- ability to synthesize information across multiple sources
- conversational interface for iterative interaction
That feels magical and similar to Star Trek.
However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one).
(fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any such attempt of a full system will absolutelty have search as its weakest link)
> Legacy ad-based search has devolved into a wasteland of misaligned incentives, conflict of interest and content farms optimized for ads and algos instead of humans.
> Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermediaries want them to see
These traps and patterns are not inevitable. They happen by choice. If you're actively polluting the world with AI generated drivel or SEO garbage, you're working against humanity, and you're sacrificing the gift of knowing right from wrong, abandoning life as a human to live as some insectoid automaton that's mind controlled by "business" pheromones. We are all working together every day to produce the greatest art project in the universe, the most complex society of life known to exist. Our selfish choices will tarnish the painting or create dissonance in the music accordingly.
The problem will be fixed only with culture at an individual level, especially as technology enables individuals to make more of an impact. It starts with voting against Trump next week, rejecting the biggest undue handout to a failed grifter who has no respect for law, order, or anyone other than himself.
I was thinking of the direction we are going and even wanted to write up a blog about it. IMO the best way forward would be if AI can have some logical thoughts independent of human biases but that can only happen if AI can reason unlike our current LLMs that just regurgitate historical data.
If the current iteration of search engines are producing garbage results (due to an influx of garbage + SEO gaming their ranking systems) and LLMs are producing inaccurate results without any clear method proposed to correct them, why would combining the two systems not also produce garbage?
The problem I see with search is that the input is deeply hostile to what the consumers of search want. If the LLM's are particularly tuned to try and filter out that hostility, maybe I can see this going somewhere, but I suspect that just starts another arms race that the garbage producers are likely to win.
Search engines tend to produce neutral garbage, not harmful garbage (i.e. small tidbits of data between an ocean of SEO fluff, rather than completely incorrect facts). LLMs tend to be inaccurate because in an absence of knowledge given by the user, it will sometimes make up knowledge. It's plausible to imagine that they will cover each other's weaknesses: the search engine produces an ocean of mostly-useless data, and the LLM can find the small amount of useful data and interpret that into an answer to your question.
Garbage-ness of search results is not binary, the right question is: can LLMs improve the quality of search results? But sure, it won't end the cat and mouse game.
I think that's the right broad question. Though LLMs properties mean that for some number of cases they will either make the results worse, or more confidently present wrong answers. This prompts the question: what do we mean by "quality" of results? Since the way current LLM interfaces tend to present results is quite different from traditional search.
There is no way to SEO the entire corpus of human knowledge. ChatGPT is very good for gleaning facts that are hard to surface in today's garbage search engines.
The question is what is the business model and who pays for it, that determines how much advertising you’re getting. It is not clear if OpenAI could compete in Ad-supported search. So maybe OpenAI is trying to do the basic research, outcompete the Bing research group at Microsoft and then serve as an engine for Bing. Alternatively they could be just improving the ability of LLMs to do search, targeting future uses in agentic applications.
I’d be more cynical still and ask, where is correct information found in the first place? Humans of all shape and size have biases. Most research is faulty, fabricated, or not reproducible. Missing information tells a greater story than existing one.
We don’t have a way of finding objective information, why would we be able to train a model to do so?
If I can pretty quickly tell a site is SEO spam, so should the LLM, no? Of course that would just start a new round in the SEO arms race, but could work for a while.
Genuine question: is there a present or planned value proposition for people like me who already have decent search skills? Or are these really for children/elders who (without making any normative claim about whether this is a good thing or not) can't be arsed to perform searches themselves?
Does someone else have good search skills but mingle traditional search engines with LLMs anyways? Why?
I use LLMs every day but wouldn't trust one to perform searches for me yet. I feel like you have to type more for a result that's slower and wordier, and that might stop early when it amasses what it thinks are answers from low effort SEO farms.
I find myself being unable to search for more complex subjects when I don't know the keywords, specialized terminology, or even the title of a work, yet I have a broad understanding of what I'd like to find. Traditional search engines (I'll jump between Kagi, DuckDuckGo, and Google) haven't proved as useful at pointing me in the right direction when I find that I need to spend a few sentences describing what I'm looking for.
LLMs on the other hand (free ChatGPT is the only one I've used for this, not sure which models) give me an opportunity to describe in detail what I'm looking for, and I can provide extra context if the LLM doesn't immediately give me an answer. Given LLM's propensity for hallucinations, I don't take its answers as solid truth, but I'll use the keywords, terms, and phrases in what it gives me to leverage traditional search engines to find a more authoritative source of information.
---
Separately, I'll also use LLMs to search for what I suspect is obscure-enough knowledge that it would prove difficult to wade through more popular sites in traditional search engine results pages.
> I find myself being unable to search for more complex subjects when I don't know the keywords, specialized terminology, or even the title of a work, yet I have a broad understanding of what I'd like to find.
For me this is typically a multi-step process. The results of a first search give me more ideas of terms to search for, and after some iteration I usually find the right terms. It’s a bit of an art to search for content that maybe isn’t your end goal, but will help you search for what you actually seek.
LLMs can be useful for that first step, but I always revert to Google for the final search.
I 100% get that use case; it comes up often for me.
Though I've used/trusted regular LLMs more for that. Historically it felt like search LLM is the worst of both worlds--the LLM is bound to what humans are saying on the first page of Google, rather than its perhaps-more-informative (if questionable) 'knowledge', which can then inform my own searches.
But maybe my intuition of how this works is off. I haven't really used them much except the early days of Bing Chat, or when Kagi Chat forgets my preference and toggles "Use Web" on.
I also find some use for this. Or I often ask if there's a specific term for a thing that I only know generally, which usually yields better search results, especially for obscure science and technology things. The newer GPTs are also decent at math, but I still use Wolfram Alpha for most of that stuff just because I don't have to double check it for hallucinations.
You might like what we're building in that sense :D (full disclosure, I'm the founder of Beloga). We're building a new way for search with programmable knowledge. You're essentially able to call on search from Google, Perplexity other search engines by specifying them as @ mentions together with your detailed query.
I don't overuse LLMs for now; however when I have a complex problem that would require multiple of searches and dozens of tabs opened and reading through very long docs, asking LLM allows me to iterate order of magnitude faster.
Things that were previously "log a jira and think about it when I have a full uninterrupted day" now can be approached with half an hour spare. This is game changer because "have a full day uninterrupted" almost never happens.
It's like having a very senior coworker who knows a lot of stuff and booking a 30m meeting to brainstorm with them and quickly reject useless paths vs dig more into promising ones, vs. sitting all day researching on your own.
The ideas simply flow much faster with this approach.
I use it to get a high level familiarity with what's likely possible vs what's not, and then confirm with normal search.
I use LLMs also for non-work things like getting high level understanding of taxation, inheritance etc laws in a country I moved in, to get some starting point for further research.
This. Not having to open two dozen tabs and read through so much is a gamechanger, especially for someone who has had trouble focusing with so much open. This is especially true when learning a new technology.
I dunno, I'm not exactly on the AI bandwagon, but search is the one place where I use (and see others using) chatgpt all the time. The fact that Google search has been getting worse for a decade probably helps, but better search -- consistently done, without ads or cruft -- would be worth a few bucks every month for me.
I agree that you can't TRUST them, but half the links regular search turns up are also garbage, so that's not really worse, per se.
Same, but, until recently, I've been using Microsoft's Co-Pilot because for the longest time it did exactly what this new "search" feature added to ChatGPT: it produced a list of source material and links to reference the LLM's output against. It was often instrumental for me and I did begin to use it as a search engine considering how polluted a lot of first-search results have become with spam and empty, generated content.
Oddly, Microsoft recently changed the search version of Copilot to remove all the links to source material. Now it's like talking to an annoying growth-stage-startup middle manager in every way, including the inability to back up their assertions and a propensity to use phrases like "anyway, let's try to keep things moving".
Happy to see this feature set added into ChatGPT – particularly when I'm looking for academic research in/on a subject I'm not familiar with.
I use LLMs as a kind of search that is slightly less structured. There are two broad cases:
1) I know a little bit about something, but I need to be able to look up the knowledge tree for more context: `What are the opposing viewpoints to Adam Smith's thesis on economics?` `Describe the different categories of compilers.`
2) I have a very specific search in mind but it's in a domain that has a lot of specific terminology that doesn't surface easily in a google search unless you use that specific terminology: `Name the different kinds of music chords and explain each one.`
LLMs are great when a search engine would only surface knowledge that's either too general or too specific and the search engine can't tell the semantic difference between the two.
Sometimes when I'm searching I need to be able to search at different levels of understanding to move forward.
I think it’s pretty clear that LLMs can process a document/article/web page faster than any human in order to answer a given question. (And it can be parallelized across multiple pages at once too).
The main hard part of searching isn’t formulating queries to write in the Google search bar, it’s clicking on links, and reading/skimming until you find the specific answer you want.
Getting one sentence direct answers is a much superior UX compared to getting 10 links you have to read through yourself.
Google does offer an AI summary for factual searches and I ignore it as it often hallucinates. Perplexity has the same problem. OpenAI would need to solve that for this to be truly useful
This is why my most used LLM after code suggestions is Bing. I like that it has lots of references for the things I ask it to double check and read more, but at the same time it can help me dig deeper into a subject rapidly and better formulate the exact question I'm trying to ask and give me a link to the actual data it's getting it's info from.
I find that my search skills matter less and less because search engines try to be smarter than me. Increasingly I am confronted with largely unrelated results (taking tweaked keywords or synonyms to my query as input apparently) as opposed to no results.
So my conclusion is that the search engines increasingly see the need of search skills as an anti pattern they actively want to get rid of.
On the Google search results page, activate Search tools > All results > Verbatim. You can also create your own search provider bookmark with verbatim search as the default by adding “tbs:li=1” as a query parameter to the Google search URL.
What I really hope this helps solve is covering for the huge lag in knowledge cutoff. A recent example is where it went "oh you're using Go 1.23 which doesn't exist so that's clearly the problem in your Dockerfile, let me fix that".
But I'm not keeping my hopes up, I doubt the model has been explicitly fine-tuned to double check its embedded knowledge of these types of facts, and conversely it probably hasn't even been successfully fine-tuned to only search when it truly doesn't know something (i.e. it will probably search in cases where it could've just answered without the search). At least the behavior I'm seeing now from some 15 minutes of testing indicates this, but time will tell.
I think it's more filling the niche that Google's self immolation in the name of ad revenue started. Besides kagi, there aren't really any solid search engines today (even ddg), and OpenAI has a reach way beyond kagi could dream of outside a billion dollars in marketing.
The entire tech industry for the last decade (if not more) has been aimed at people who can't be arsed to learn to use computers properly. I would be astonished if this time is somehow different.
LLMs really make it easy to quickly find documentation for me. Across a huge software project like Mediawiki with so much legacy and caveats, having an LLM parse the docs and give me specific information without me hoping that someone at Stackoverflow did it or if I'm lucky enough to stumble across what I was looking for.
Even if you are good at writing the queries, Google is so terrible that you end up getting some blogspam etc. in there (or at least I do). A model filtering that out is useful, which I find phind pretty good for. Hopefully this will be even better.
Any question that few months ago I would do to stackexchange (or expect and answer from, after a google seqrch) either coding or quantitative, I go to chat gpt now.
I consider myself quite anti LLM hype and I have to admit it has been working amazingly good for me.
For searches that remain inconclusive, I sometimes double-check with LLMs to see if I have missed anything. It rarely gives relevant new insights, but it’s good to get the confirmation I guess.
I use GPT for things that would require multiple Google searches (research). Some examples..
- I count calories... eat out always and at somewhat healthy chains (Cava, Chipolte, etc). Tell GPT (via voice while driving to & or after eating) what ive eaten half the day at those places and then later for dinner. It calculates a calorie count estimation for half the day and then later at dinner the remaining. I have checked to see if GPT is getting the right calories for things off websites and it has.
- Have hiking friends who live an hour or two hours away and we hike once a month an hour or less drive is where we meet up and hike at a new place. GPT suggests such hikes and quickly (use to take many searches on Google to do such). Our drives to these new hikes learned from GPT have always been under an hour.
So far the information with those examples has been accurate. Always enjoy hearing how others use LLMs... what research are you getting done in one or two queries which used to take MANY google searches?
GPT is proving useful for me where something is well documented, but not well explained.
Case in point: Visual Basic for Applications (the Excel macro language). This language has a broad pool of reference material and of Stack Overflow answers. It doesnt have a lot of good explicatory material because the early 2000s Internet material is aging out, being deleted as people retire or lose interest, etc.
(To be frank, Microsoft would like nothing more than to kill this off completely, but VBA exists and is insanely more powerful than the current alternatives, so it lives on.)
It seems good at finding relevant research papers. e.g.
> "Can you provide a list of the ten most important recent publications related to high-temperature helium-cooled pebble-bed reactors and the specific characteristics of their graphite pebble fuel which address past problems in fuel disintegration and dust generation?"
These were more focused and relevant results than a Google Scholar keyword-style search.
However, it did rather poorly when asked for direct links to the documentation for a set of Python libraries. Gave some junk links or just failed entirely in 3/4 of the cases.
I know what you're talking about, but also don't know how it applies in this case. Not a hater, and not asking rhetorically as a way to dunk on OpenAI. Just haven't found a use for this particular feature.
Which is also exactly something a bad-faith commenter would say, but if I lose either way, I'd rather just ask the question ¯\_(ツ)_/¯
I think this is just the first step for a full-featured agent that not only does searches for you, but also executes whatever was your goal (e.g. a restaurant reservation, etc)
To solve that problem you have to solve all the issues that make me not trust the results. As search, it's fine, since I am perusing and evaluating them. But as an agent, hallucinations and inaccurate answers have to disappear (or very close to disappear).
Makes me question why Google never bothered to create something like search sessions which could be enriched with comments/notes and would be located in a sidebar just like the chats in ChatGPT/Claude/Mistral are.
They really had the potential to do something interesting, but were just focused on their ad metrics with the "good enough" search box. What have they been doing all the time?
the FAANG giants have been government assets for ~15+ years [0]. they don't have to turn a profit every quarter, innovate, or make their search any better because they no longer play by the rules a normal business does. they are a critical "too big to fail" component of the state's global surveillance system.
I guess now Google's search stack is too complicated and not many engineers understand what to do in order to introduce a novel, big feature integrated into the full stack vertically and horizontally. And those few who capable of doing so are probably completely out of bandwidth, so some random ambitious PM cannot pull their hands into uncertain green field projects.
Let's see, if I go to " ⋮ -> History -> Grouped History" on the top right of the Chrome browser, I see a "Search History" ( chrome://history/grouped ).
For example `8 hours ago: "autohotkey hotkeys"` with 4 links to pages which I visited while searching.
But this is a Chrome feature, not a Google Search feature. https://myactivity.google.com/myactivity does (sometimes? can't see it right now) have a grouping feature of all the searches made, but this is more of a search log than a search management feature.
So chrome://history/grouped is the closest to what I mean, but I can't pin or manage these history groups, enrich them with comments or even files, like pdf's which could then get stored in Google Drive, as well as get indexed for better searches.
Search in the internet worked because people wanted to generate content to attract people to display ads or any other reason, but they wanted to attract people.
If now my content is going to be ingested and shown by a LLM or AI agent, what's the purpose to give it for free? I know it won't happen, but I would love if this type of agents have to pay to show a summarization of another website. It's only fair when done in mass like this.
Well the whole point of this product is to link back to websites. There’s no necessary link between the text and the links, which are chosen after the fact from an index. That’s different from traditional search engines, where links are directly retrieved from the index as part of ranking.
I think your comment highlights a very important shift in the market for ads and you are right that increasingly, the current atmosphere hints at there is little to no incentive to publish original creative work in the future if there is no compensation for it like Google had done.
We've like reached peak human driven novelty (or McKennaists will argue it already happened around the mid 2010s) and we'll see AI driven novelty with the difference being it will be even a smaller group of people that are paid royalty fees.
Once creative destruction reaches critical mass, we'll finally see billionaires and companies around the world succumb to demand for UBI.
If you want to see the future just look at China. Billionaires are being hunted down and threatened to give up their offshore accounts.
One thing that is quite unfortunate with the state of SEO and the web in general today is that when I asked "what are the latest versions of common programming languages and when were they released?" a large amount of the sources were "13 Tools You Should Learn Now" and the like. This might be a solvable problem within the search API they provide to the LLM, but for now I wouldn't trust current LLMs to be able to filter out these articles as less trustworthy than the official website of the programming language in question.
Given how many of those SEO spam sites are themselves generated by ChatGPT now, OpenAI can simply back-reference their own logs to find out which sites are probably SEO spam while everyone else is left guessing. That's vertical integration!
So it'll turn to yet another arms race - similar to captcha, cybersecurity and nuclear weapons. SEO will use AI to fill in fluff inside AI-generated content (which is already done).
It won't directly match ChatGPT logs and OpenAI would just be pouring precious compute to a bottomless pit trying to partial-match.
I’m sure they will be more subtle than that otherwise it will get circumvented.
I’m sure they will/are tackling this at the model level. Train them to both generate good completions while also embedding text with good performance at separating generated and human text.
Would someone even want to circumvent it though? Most sites won't care very much about encouraging scrapers to include them in LLM training data, it's not like you get paid.
> when I asked "what are the latest versions of common programming languages and when were they released?"
The issue is with the query itself. You're assuming that there's some oracle that will understand your question and surface the relevant information for you. Most likely, it will use the word themselves as part of the query, which SEO sites will exploit.
A more pragmatic search workflow would be to just search for "most common programming languages used" [0], then used the Wikipedia page to get the relevant information [1]. Much more legwork, but with sources. And still quite fast.
this is why I pay for Kagi. granted those results still come up, but you can block specific domains from ever appearing in the results and configure how listicles are displayed.
How many can you block and filter manually ? 10 ? 100 ? 10k ? Who will test sites for the blocklist ? The domain block feature is great but unless it's collaborative listing it's not gonna be super effective.
It’s super effective for me because I just block stuff as things pop up that I don’t want. I’ve also added more weight to certain domains that I want more results from. I wouldn’t want anyone touching my config, it’s mine and it works great!
We've had this problem of "good defaults" before with ad trackers blocking domains. I'm sure it'll be Sooner than later when some community lists become popular and begin being followed en mass
I meant your average user can test a handful of sites if they are seo spam or good sites but a single search return 10+ results and even more when a user searches multiple things , multiple times a day . Average user doesn't have the time to test these many websites.
Average serp page has 10 results . What if all 10 matches with your blacklist ? Not to mention you can't do anything if the engine dosen't search deeper .
SEO spam is always going to focus on the biggest market, and by doing so they can be completely transparent and obvious to whoever they're not trying to fool.
I'd assume right now the SEO target is still mainly Google rather than ChatGPT, but that's only an "I recon" not a citation.
If and when ChatGPT does become the main target for SEO spam, then Googling may start giving good results again.
As of October 31, 2024, the latest version of Java is Java 23, released on September 17, 2024. The most recent Long-Term Support (LTS) version is Java 21, released on September 19, 2023.
Yeah I did also find it to be mostly accurate. However, seeing the sources I felt like I kind of have to check all the languages just in case it picked up information from a random "X ways to do Y" article that might not have been prioritizing accuracy. And for this search query I did see several languages' actual websites, but I did another very similar query earlier where 9 out of 12 results were all numbered list articles clearly intended for SEO. 2 of them were actual official sites. And 1 was what appears to be a decent attempt at talking about programming languages (i.e. not SEO only).
Aside from the marketing-ish tone and specific deeplinks to product purchase pages, the prominent Amazon logo and product description headline implied some degree of affiliation to my eyes. It seems like the evidence is that it would be foolish not to take the money for presenting such an obvious referral of a motivated buyer.
Frankly the example they posted seems like a fairly happy one, where the user is explicitly implying that they’re seeking a specific physical product to introduce to their life. We’ve all seen where those monetization incentives lead over time though.
But you’re right—not even so much as a tiny word “Ad” like Google does…
I'd be happy to have another Google clone, that doesn't have a login and is not a chat session. Go to https://search.ai , type my search query and look through the results, with ads on the side.
You are thinking you can pay them to not use your data? Think again. They will sneakily use your data anyway. If not yours, then the data of people who do not change setting xyz. Oops, the last update must have reset that option for some users.
So the issue is if you let people opt out by paying you’re left with a low intent, likely lower net worth group of people to advertise to. As a result those eyeballs are worth less. The advertisers will turn to other platforms if you only let the worst people see their ads.
Unless enough people all pay, the whole thing stops working. But there aren’t enough people who will pay because most people don’t care.
Tldr: the ad supported business model fundamentally doesn’t work if you let all your best products (you) opt out by paying. It requires them to pay an amount far in excess of what they would be willing to pay for the system to work.
There's some truth to that, but Netflix, YouTube, etc seem to be OK with both ad-supported and paid ad-free versions, so I think the logic you described does not always dominate the considerations.
> Honestly, if I can disable ads by paying them, then I'm ok with it.
The modern maxim is: any content platform large enough to host an ad sales department will sell ads
Vanishingly few (valuable) consumers have zero tolerance for ads, so not selling ads means leaving huge sums of money on the table once you get to a certain scale. Large organizations have demonstrated that they can't resist that opportunity.
The road out is to either convince everyone to have zero tolerance for ads (good luck), to just personally opt for disperse, smaller vendors that distinguish themselves in a niche by not indulging, or to just support and use adversarial ad blockers in order to take personal control. Hoping that the next behemoth that everybody wants to use will protect you from ads is a non-starter. Sooner or later, they're going to take your money and serve you ads, just like the others.
They did not get addicted to selling ads, have billions in revenue from paying subscribers, and don't have to wean themselves off of ads (as Google and Meta would love to do).
No but once ChatGPT starts threatening Google's revenue model, maybe they will start putting effort into improving their drastically deteriorating search engine.
they need to win search share to threaten Google's revenue model: take traffic from google.com, so google will sell ads. Going to ads busyness is not necessary for this.
There are several classes of restrictions on free speech in the US. These include: obscenity, fraud, speech integral to illegal conduct, speech that incites imminent lawless action, speech that violates intellectual property law, true threats, false statements of fact, and —- most relevant here -— commercial speech such as advertising.
Advertising has far less protection than is ordinarily afforded to the kind of speech you might do as a person.
Not sure how it was able to read this user's repo name but for me, it's optimistically saying "yes they have a repo named X" of whatever I ask it and sometimes I do have that repo and sometimes I don't.
I have a private repo named "portland-things" and I asked "does this user have a repo related to portland?" and it responded with "yes it's called 'pdx'" but that's not correct at all.
I usually assume good faith, but in this particular case, I believe the chance that this repo was public before and the author just changed it to private to bait attention is far higher than that Bing/ChatGPT can actually read private repo on GitHub.
As one person pointed out in the thread, it also shows up on bing results, main repo, main.py file and releases page. But not on google.
Edit: It also shows up on duckduckgo
I'm saying that by moving towards explicit "search" and "linking to sources" they have set the stage for being able to charge to be recommended by their search features (ie, ads and pay-to-rank, same as Google search).
There aren't any ads in their demo, we haven't seen the real deal yet, but I'll be watching HN for that day.
Making billions but spending trillions for no moat (GPUs and models aren't moats) means that the only moat they have are users. Users aren't paying enough to offset costs, the only way to get value from non-subscription users for their scale is through ads.
Because it is never enough. We see this time and time again. Once they are making billions, the people in charge will demand that they start making dozens of billions, and then hundreds. The growth must never cease, because the moment you stop growing, you can't sell the dream that supports ridiculous PE ratios anymore.
Google was a very profitable business 10 years ago and the search was still decent. In the last decade they absolutely butchered their core product (and the internet along with it) in an effort to squeeze more ad dollars out, because it's not the level of profitability that they need to maintain, but the growth of that profitability.
Microsoft was a ridiculously profitable company, but that is not enough, they must show growth. So they add increasingly user hostile features to their core product because the current crop of management needs to see geometric growth during their 5 year tenure. And then in 5 years, the next crop of goobers will need to show geometric growth as well to justify their bonuses.
Think about this for a moment: the entire ecosystem is built on the (entirely preposterous) premise that there must be constant geometric growth. Nobody needs to make a decision or even accept that this is long term sustainable, every participant just wants the system to keep doing this during their particular 5-10 year tenure.
It's an interesting showcase of essentially an evolutionary algorithm/swarm optimizer falling into a local optimum while a much better global optimum is out of reach because the real world is something like a Rastrigin function with copious amounts of noise with an unknowable but fat tailed distribution.
You can ask the same for ie Apple where you pay a proper premium for products, yet their ad business keeps growing slowly into respectable proportions, and not by accident.
This might be unpopular opinion but it really isn't as big of a deal as OpenAI makes it out to be (like their previous announcements)
The truth is, I haven't used ChatGPT at all since spring of this year. Claude's Sonnet 3.5 has replaced it. I pay very little attention to what OpenAI releases and simply waits for Anthropic to implement it.
I also started using Gemini which already outperforms perplexity and this and will not switch.
I think everybody is constantly caught up with their infatuation with OpenAI and other characters that they don't realize Google, Anthropic are actually building a moat which some like Gary Marcus keeps rambling on as impossible
I'm a realist and I can see that while Google has been slower to start, it reminds me of the search engine wars of 2000s, it is dominating and winning over users.
I don't see a way for websites to monetize giving this content to openAI. This is not 2001 anymore, people should not expect to give away this information for nothing. I would be blocking openai bots ASAP
To my mind one of the great benefits of LLMS is the possibility of searching without handing over some of the most personal information that exists - your search history.
I’m happy OpenAI is advancing LLM-based search but I won’t be using it in earnest until it’s local.
I like it. It's clean, fast, and the results seem solid. Google search has become so bloated with sponsored results that I've been hoping for a tool that could provide better results than DDG or Bing.
I'm going to use this as my daily driver for a few weeks.
The contemporary web is basically an epiphenomenon of Google, and they've failed to defend it. I hope OpenAI puts a huge dent in their market share.
"""Search will be available at chatgpt.com (opens in a new window), as well as on our desktop and mobile apps. All ChatGPT Plus and Team users, as well as SearchGPT waitlist users, will have access today. Enterprise and Edu users will get access in the next few weeks. We’ll roll out to all Free users over the coming months."""
Can confirm that free users who signed up for the waitlist can use it right now (even if they didn't actually get in)
I signed up for the wait list as well and got the email. However there is no search button on the web interface (free tier). Is there anything I am missing?
It became a good name once it became a watershed viral phenomenon. Everything being equal yeah not a great name but it defined a new hype cycle so it got a pass
ChatGTP and other variants with a Levenshtein distance of 1 from ChatGPT have been typosquatted to death by subscriptionware wrappers on the App Store and the Play Store. Many of them seem to be quite successful.
ChatGPT isn't a great name, but it's easy to say, spell, and remember. And at this point, a lot of people just know them as OpenAI, which is a great name.
Perplexity sounds like a parody startup name from the Silicon Valley TV show. Way too complicated and unnatural.
You’re saying that now, but getting the initials in “ChatGPT” in the right order took a while to learn, so I wouldn’t say it’s easy to remember, and it seems easy to stumble saying it, too?
It’s all about familiarity. Once people learn it, it’s not hard.
But it didn't matter if they were in order or not, because ChatPTG or ChatTGP all go to the same place via Google, etc. It could have been called Chat + [Any 3 characters] and been fine.
Perplexity is just a nonsensical word (for those unfamiliar with the concept) that is too long and hard to spell. They'd be better off just chopping it down to Lexity, or Lex, or Plexity, or Plex, etc.
It would be a logical name if its customers were technicians familiar with LLMs, and not end businesses and consumers. Which is why Ford wasn't named Internal Combustion Engine, Apple wasn't named Graphics Processing Unit, etc.
I have been using Perplexity with the AI engine set to Claude 3.5 Sonnet for a month now, mostly for programming related questions, and it has been amazing. I mostly stopped using google.
I’m surprised at all the negativity on perplexity. I think it’s a great approach (base answers on sources) and their product seems to deliver on the premise.
That said, anecdotally, I find it’s a bit hit-miss: if it’s hit it’s a huge improvement over google (and a minor improvement over chatgpt), if it’s miss it’s still good but get the feeling you won’t get anywhere further by asking more questions.
One thing about Amazon is that I have never seen them overpay for an acquisition (as in they really penny pinch and negotiate hard). So Perplexity’s high price tag may turn Amazon off
What’s the benefit for websites to allow OpenAI/Microsoft to scrape and republish all their content without sending traffic back to them? It seems like these type of “search engines” will just get blocked.
Is this more than just ChatGPT with search API resulted concatenated the prompt?
It feels like it might be. It feels tasteful in the same way that Apple ecosystem integrations just work really nicely and intuitively. But then again, there is an art to keying and retrieving embeddings, and it might just be that.
The new web search icon appeared for me straightaway in the ChatGPT macOS desktop app, within an in-progress conversation, without even having to restart. Before I'd even seen this official launch announcement. Very smooth!
I have learned to seriously question my instincts on when something is too late as there are many niches to fill and this is likely a building block for broader functionality.
That being said, for all the talk about how bad google has become, I still prefer it to an unbroken bing.
> Anyone can compete as long as they have a sufficiently robust crawl dataset as a foundation, no?
There's some sticking power/network-effect/sticky-defaults effects, too, though.
It's _trivial_ to do a google search from anywhere on an android device with at most a tap or two. You can probably get close if a 3rd party has a well integrated native app but that'll require work on the user's behalf to make it the default (where possible).
Same goes for the default search engine for browsers/operating systems ... etc.
I will absolutely be firing off queries to google and GPTSearch in parallel and doing a quick comparison between the two. I am especially curious to see how well queries like "I need the PCI-e 4 10-gig SFP+ card that is best supported / most popular with the /r/homelab community" goes. Google struggles to do anything other than link to forums where people are already asking similar questions.
Anyone can compete as long as they have a functional URL and web page. Doesn’t make them good competition, and doesn’t mean users will use it.
The issue is that “AI search” has been a hot topic for a while now. Google (the default everywhere) just rolled out their version to billions of users. Perplexity has been iterating and acquiring customers for a while. Obviously OpenAI has great potential and brand recognition, but are enough people still interested in switching that haven’t yet?
A fossilized snapshot will only get them so far, and sites are increasingly opting to block AI-related crawlers. Apparently about a quarter of the top 1000 sites already block GPTBot: https://originality.ai/ai-bot-blocking
I guess they could be using Bing as their search backend, which would mostly get around the blocking issue (except for searching Reddit which blocks Bingbot now).
Certainly, countermeasures against crawler blocking will be a necessary component of effective search corpus aggregation in the go forward. Otherwise, search will balkanize around who will pay the most for access to public content. Common Crawl is ~10PB, this is not insurmountable.
Edit: I understand there is a freerider/economic issue here, unsure how to solve that as the balance between search engine/gen AI systems and content stores/providers becomes more adversarial.
I wonder to what degree -- for example, do they respect the Crawl-delay directive? For example, HN itself has a 30-second crawl-delay (https://news.ycombinator.com/robots.txt), meaning that crawlers are supposed to wait 30 seconds before requesting the next page. I doubt ChatGPT will delay a user's search of HN by up to 30 seconds, even though that's what robots.txt instructs them to do.
The whole issue that site owners have with these AI search engines is that there isn't a financial incentive for them to cooperate, since the summarization largely replaces the need for users to click through to the site the information came from. No click-through, no ad impressions, no possibility of the user being converted into a recurring visitor or paid subscriber, just pure freeloading by the search engine.
There should(or will probably be) be a study on how long do people take on google etc vs search powered by chatgpt to get non-trivial work done, controlling for obvious factors like age, gender, country and industry
If there is a bias towards chatgpt like tools of even ~5%, it would be worth investigating why this is. My hunch is just the conversational aspect of describing at a high level and finding answers and avoiding all the distraction of several dozen windows to do something is worth it.
I'm curious how ChatGPT Search improves recall and reduce spams. I can see how LLM helps find the most relevant content. However, it is still hard to find the right content, especially in a domain that has tons of spams. For instance, when I search for a product review on Google, I get back many review sites that look so legit that I have hard time telling which ones are spammers.
A lot of people are focusing on its reliance on existing SEO - but they say right in the post that they're integrating directly with data brokers more. They're subtly arguing for the elimination of a certain kind of middle man.
I really don't see the value of summarizing/repackaging web search hits. Given that 99% of SEO-tuned web content is just shilling for vendors who don't want to be seen, LLM search summarization will just repackage those ads into a more palatable format that is LESS useful than the original, while more successfully hiding the obvious signatures that used to be a clear warning sign that... THE.FOLLOWING.CONTENT.IS.MANIULATIVE.CRAP.
I think the value here is not in searching for SEO crap, but turning it on when you want to get references to the most current information relevant to your query.
For example, if you ask LLMs to build code using the three.js library, nearly all of them will reference version r128. Presumably because that version has the largest representation in the training data set. Now, you can turn this on and ask it to reference the latest version, and it will search the web and find r170 and the latest documentation to consider in it's response.
I was already doing this before by adding "search the web for the latest version first" in my prompts, now I can just click a button. That's useful.
I tend to agree. If I ask ChatGPT what is the best way to make pasta, it will pull from every source it’s ever been trained on. If it decides to search the web, it will mostly cater to one or two sources.
You don't think SEO-LLMs will evolve to redirect search-LLMs to 'see the world' the way the SEO-LLMs want it to? I foresee SEO-LLM-brinkmanship as the inevitable outcome. Soon THIS will be the catalyst for the real Skynet -- battling smart ad engines.
I played around with it a bit, here are some hot takes.
For context, I first tried this procession of searches on the Mac OS app.
1. "Who won the world series"
2. Who was the MVP?"
3. "Give me his bio"
My observations:
1. UX: The "search" button feels oddly placed, but I can't put my finger on it. But once I got it is a toggle, it wasn't a bit deal.
2. The first result had 3 logos, headlines and timestamps delineated, and easy to ready. The second one and third ones included a "Sources" button that opened a fly open menu. Clicking those opened a web link. The third result also included images in the fly open.
3. Citations were also inlined. The third result, for the bio, included a citation per paragraph.
4. It wasn't as fast as google. Which makes sense, given it's going through the LLM. But it will take a while to rewire my brain to expect slower responses to search.
5. Overall, I found the chat interface a very intuitive interface.
The second search I asked was "Give me a plan for a Thanksgiving meal."
I to a long response that felt like a weird mashup of LLM-generated content and search results:
1. A list of menu selections
2. Links to some recipes
3. Prepration timeline
4. Shopping list
5. Additional tips
There were 15 citations listed in the popup button, but only 3 inlined.
This was... not great. A traditional list of search results feels better here.
Overall, I like the direction. Innovation in search has been dead for close to 10 years, and this feels like I'd use it for certain inquiries.
LLMs have the chance to cannibalize the web and become the primary interface for knowledge. But if websites remain the final destination, it’s good for content creators.
The only other way to kill the web without killing LLMs in the process would be to create a way for people to upload structured public content directly into an LLM’s training. That would delay public content into release batches unless training can be sped up significantly.
I’m imagining LLMs might eventually have an upload tool, similar to Google’s site map upload, for registering content proactively instead of needing to be discovered through crawling the web or training on chat data.
The social engineering possibilities with a tool like this are endless. Google already wields enormous visibility power but ultimately just provides a list of links to other sources.
This can subtly (or not so subtly) rephrase and reshape the way we read about and think about every topic.
I think generative search itself has room for disruption and I'm not too sure if a chat interface or a perplexity style one is necessarily the right way to go about it.
I'd like to see search (or research in broader sense) a more controllable activity with the ability to specify context + sources easily in the form of apps, agents and content.
not sure what my use case for this would be if it's expecting me to type full descriptive sentences to check something quick or find a picture, and then read a whole paragraph of a useless reply. No results. No reddit posts (which aren't even what I want but understand a lot of content is buried in user-generated content) They seem to be implying this is the replacement for Google. This just isn't it.
Edit: ohh, only Pro users? Right. ok. They made it seem like this was the big search launch and to go to chatgpt.com to get into it. Moving on.
Is there any indication they're willing to improve on google in terms of e.g. excluding commercial results? If not it's not clear how this improves anything. Google has been excellent at semantic search for a long time; the issue has been the lack of controls to filter out the SEO bullshit and to remove the AI stuff from the top and the right of the results. It's been way too easy to game search with sufficient funding for well over a decade now and the AI-generated crap is a long way from production-ready (in terms of quality; obviously it generates something).
Yeah . It needs to filter seo optimized articles first and search more niche sites or it will be your average chat gpt with search project from github.
"The search model is a fine-tuned version of GPT-4o, post-trained using novel synthetic data generation techniques, including distilling outputs from OpenAI o1-preview."
I wonder if this is their own web scraper, or if they're using Bing API?
As a very experienced SEO, this is pretty exciting nonetheless, a new front in the online war opening up.
If they're using their own scraper/search algorithms, it'll be interesting to see how they weigh the winners and losers compared to how Google does it.
Is this a move to try to setup a pathway for getting some data on realtime trends? In other words something for quickly getting some model updates for hot prompts like like “what happened in the debate”.
I mean, until there is an alternative in the space that has a (good), free, anonymous ai web search then I think we can probably assume you are confusing what you want with what is "viable"
Honestly I don't live this 'streaming LLM text effect' as well as the wordiness by which ChatGPT 'chats' with me. I consider LLMs to be machines, not conversation partners, and frankly I find the notion of chatting with an artificial being a bit creepy (unless it's specifically what I want as some sort of escapism). I wish they tried to be as terse as possible (and faster too).
Asked it to generate code for a library that was released in the past year - GPT-4 couldn't do it and this one just did it flawlessly. I am super impressed.
It's able to query the relevant documentation, put it in its context and then use that to generate code. It's extremely relevant to giving existing models superior functionality.
It's crazy how much "vibes" affect perception of the product. OpenAI just always feels cold and alien to me. Compared to Anthropic and Perplexity's warmth.
That link just takes me to what looks like a normal chatgpt prompt. (I tried asking it the same things they showed in the article and I just get generic AI answers, not web search results.)
Im not logged into GPT (am subscribed for this month tho on my iPhone but that's separate) and able to do a web search and or ask GPT a question.
Actually i am logged into my iCloud on my macbook so guess that's why im seeing the search on that device of mine (not seeing on another where Im not logged into iCloud).
What the hell did you ask it / what was the source? I just did the same thing and it gave me the correct answer and used my countries best known meteroligcal site?
What I find incredible is that Google has had the knowledge and resources to do this for at least five years, yet they're still milking the "old cow". It reminds me of Intel sitting on their money while a near-bankrupt AMD sped past them.
To provide relevant responses to your questions, ChatGPT searches based on your prompts and may share disassociated search queries with third-party search providers such as Bing. For more information, see our Privacy Policy and Microsoft’s privacy policy. ChatGPT also collects general location information based on your IP address and may share it with third-party search providers to improve the accuracy of your results.
Been thinking about this a lot [1]. Will this fundamentally change how people find and access information? How do you create an experience so compelling that it replaces the current paradigm?
The future promised in Star Trek and even Apple's Knowledge Navigator [2] from 1987 still feels distant. In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever.
Combining two broken systems - compromised search engines and unreliable LLMs - seems unlikely to yield that vision. Legacy, ad-based search, has devolved into a wasteland of misaligned incentives, conflict of interest and prolifirated the web full of content farms optimized for ads and algos instead of humans.
Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermiediaries want them to see - which means a different business model in seach, where there are no intermediaries. I do not see a way around this. Advancing models without advancing search is like having a michelin star chef work with spoiled ingredients.
I am cautiously optimistic we will eventually get there, but boy, we will need a fundamentally different setup in terms of incentives involved in information consumption, both in tech and society.
[1] https://blog.kagi.com/age-pagerank-over
[2] https://www.youtube.com/watch?v=umJsITGzXd0
As Porter explained in 1980 there are three ways to compete successfully:
1. Compete on price; race to the bottom or better, do free with ads
2. Differentiation
3. Focus - targeting a specific market segment
Some things don't change. Land grabbers tend to head down route 1.
Great find on the knowledge navigator, I had never seen it but I was a toddler when it was released haha.
It's interesting how prescient it was, but I'm more struck wondering--would anyone in 1987 have predicted it would take 40+ years to achieve this? Obviously this was speculative at the time but I know history is rife with examples of AI experts since the 60s proclaiming AGI was only a few years away
Is this time really different? There's certainly been a huge jump in capabilities in just a few years but given the long history of overoptimistic predictions I'm not confident
You don’t need AGI to build that experience.
I’m the past there was a lot of overconfidence in the ability for things to scale. See Cyc (https://en.m.wikipedia.org/wiki/Cyc)
> actually surfacing the content people want to see,
Showing users what they want to see conflicts with your other goal of receiving reliable answers that don't need fact checked.
Also a lot of questions people ask don't have one right answer, or even a good answer. Reliable human knowledge is much smaller than human curiosity.
> In those visions, users simply asked questions and received reliable answers - nobody had to fact-check the answers ever.
This also seems like a little ridiculous premise. Any confident statement about the real world is never fully reliable. If star trek were realistic the computer would have been wrong once in a while (preferably with dramatically disastrous consequences)—just as the humans it likely was built around are frequently wrong, even via consensus.
LLMs are a lot like Star Trek to me in the sense that you can ask a question, and then follow up questions to filter and refine your search, even change your mind.
Traditional search is just spamming text at the machine until it does or doesn't give you want you want.
That's the magic with LLMs for me. Not that I can ask and get an answer, that's just basic web search. It's the ability to ask, refine what I'm looking for and, continue work from there.
I agree that LLMs have opened modalities we didn't have before, namely:
- natural language input
- ability to synthesize information across multiple sources
- conversational interface for iterative interaction
That feels magical and similar to Star Trek.
However they fundamentally require trustworthy search to ground their knowledge in, in order to suppress hallucination and provide accurate access to real time information. I never saw someone having to double-check computer's response in Star Trek. It is a fundamental requirement of such interface. So currently we need both model and search to be great, and finding great search is increasingly hard (I know as we are trying to build one).
(fwiw, the 'actual' Star Trek computer one day might emerge through a different tech path than LLMs + search, but that's a different topic. but for now any such attempt of a full system will absolutelty have search as its weakest link)
> Legacy ad-based search has devolved into a wasteland of misaligned incentives, conflict of interest and content farms optimized for ads and algos instead of humans.
> Path forward requires solving the core challenge: actually surfacing the content people want to see, not what intermediaries want them to see
These traps and patterns are not inevitable. They happen by choice. If you're actively polluting the world with AI generated drivel or SEO garbage, you're working against humanity, and you're sacrificing the gift of knowing right from wrong, abandoning life as a human to live as some insectoid automaton that's mind controlled by "business" pheromones. We are all working together every day to produce the greatest art project in the universe, the most complex society of life known to exist. Our selfish choices will tarnish the painting or create dissonance in the music accordingly.
The problem will be fixed only with culture at an individual level, especially as technology enables individuals to make more of an impact. It starts with voting against Trump next week, rejecting the biggest undue handout to a failed grifter who has no respect for law, order, or anyone other than himself.
I was thinking of the direction we are going and even wanted to write up a blog about it. IMO the best way forward would be if AI can have some logical thoughts independent of human biases but that can only happen if AI can reason unlike our current LLMs that just regurgitate historical data.
growing up, we had the philosophical "the speaking tree" https://www.speakingtree.in/
If trees could talk, what would they tell us. Maybe we need similarly the talkingAI
If the current iteration of search engines are producing garbage results (due to an influx of garbage + SEO gaming their ranking systems) and LLMs are producing inaccurate results without any clear method proposed to correct them, why would combining the two systems not also produce garbage?
The problem I see with search is that the input is deeply hostile to what the consumers of search want. If the LLM's are particularly tuned to try and filter out that hostility, maybe I can see this going somewhere, but I suspect that just starts another arms race that the garbage producers are likely to win.
Search engines tend to produce neutral garbage, not harmful garbage (i.e. small tidbits of data between an ocean of SEO fluff, rather than completely incorrect facts). LLMs tend to be inaccurate because in an absence of knowledge given by the user, it will sometimes make up knowledge. It's plausible to imagine that they will cover each other's weaknesses: the search engine produces an ocean of mostly-useless data, and the LLM can find the small amount of useful data and interpret that into an answer to your question.
Garbage-ness of search results is not binary, the right question is: can LLMs improve the quality of search results? But sure, it won't end the cat and mouse game.
I think that's the right broad question. Though LLMs properties mean that for some number of cases they will either make the results worse, or more confidently present wrong answers. This prompts the question: what do we mean by "quality" of results? Since the way current LLM interfaces tend to present results is quite different from traditional search.
> it won't end the cat and mouse game.
There is no way to SEO the entire corpus of human knowledge. ChatGPT is very good for gleaning facts that are hard to surface in today's garbage search engines.
The question is what is the business model and who pays for it, that determines how much advertising you’re getting. It is not clear if OpenAI could compete in Ad-supported search. So maybe OpenAI is trying to do the basic research, outcompete the Bing research group at Microsoft and then serve as an engine for Bing. Alternatively they could be just improving the ability of LLMs to do search, targeting future uses in agentic applications.
I’d be more cynical still and ask, where is correct information found in the first place? Humans of all shape and size have biases. Most research is faulty, fabricated, or not reproducible. Missing information tells a greater story than existing one.
We don’t have a way of finding objective information, why would we be able to train a model to do so?
If I can pretty quickly tell a site is SEO spam, so should the LLM, no? Of course that would just start a new round in the SEO arms race, but could work for a while.
> If I can pretty quickly tell a site is SEO spam, so should the LLM, no?
Why would you assume that?
The LLM is not a human and cannot distinguish between spam and high quality content.
Genuine question: is there a present or planned value proposition for people like me who already have decent search skills? Or are these really for children/elders who (without making any normative claim about whether this is a good thing or not) can't be arsed to perform searches themselves?
Does someone else have good search skills but mingle traditional search engines with LLMs anyways? Why?
I use LLMs every day but wouldn't trust one to perform searches for me yet. I feel like you have to type more for a result that's slower and wordier, and that might stop early when it amasses what it thinks are answers from low effort SEO farms.
I find myself being unable to search for more complex subjects when I don't know the keywords, specialized terminology, or even the title of a work, yet I have a broad understanding of what I'd like to find. Traditional search engines (I'll jump between Kagi, DuckDuckGo, and Google) haven't proved as useful at pointing me in the right direction when I find that I need to spend a few sentences describing what I'm looking for.
LLMs on the other hand (free ChatGPT is the only one I've used for this, not sure which models) give me an opportunity to describe in detail what I'm looking for, and I can provide extra context if the LLM doesn't immediately give me an answer. Given LLM's propensity for hallucinations, I don't take its answers as solid truth, but I'll use the keywords, terms, and phrases in what it gives me to leverage traditional search engines to find a more authoritative source of information.
---
Separately, I'll also use LLMs to search for what I suspect is obscure-enough knowledge that it would prove difficult to wade through more popular sites in traditional search engine results pages.
> I find myself being unable to search for more complex subjects when I don't know the keywords, specialized terminology, or even the title of a work, yet I have a broad understanding of what I'd like to find.
For me this is typically a multi-step process. The results of a first search give me more ideas of terms to search for, and after some iteration I usually find the right terms. It’s a bit of an art to search for content that maybe isn’t your end goal, but will help you search for what you actually seek.
LLMs can be useful for that first step, but I always revert to Google for the final search.
Also, Google Verbatim search is essential.
I 100% get that use case; it comes up often for me.
Though I've used/trusted regular LLMs more for that. Historically it felt like search LLM is the worst of both worlds--the LLM is bound to what humans are saying on the first page of Google, rather than its perhaps-more-informative (if questionable) 'knowledge', which can then inform my own searches.
But maybe my intuition of how this works is off. I haven't really used them much except the early days of Bing Chat, or when Kagi Chat forgets my preference and toggles "Use Web" on.
I also find some use for this. Or I often ask if there's a specific term for a thing that I only know generally, which usually yields better search results, especially for obscure science and technology things. The newer GPTs are also decent at math, but I still use Wolfram Alpha for most of that stuff just because I don't have to double check it for hallucinations.
You might like what we're building in that sense :D (full disclosure, I'm the founder of Beloga). We're building a new way for search with programmable knowledge. You're essentially able to call on search from Google, Perplexity other search engines by specifying them as @ mentions together with your detailed query.
I don't overuse LLMs for now; however when I have a complex problem that would require multiple of searches and dozens of tabs opened and reading through very long docs, asking LLM allows me to iterate order of magnitude faster.
Things that were previously "log a jira and think about it when I have a full uninterrupted day" now can be approached with half an hour spare. This is game changer because "have a full day uninterrupted" almost never happens.
It's like having a very senior coworker who knows a lot of stuff and booking a 30m meeting to brainstorm with them and quickly reject useless paths vs dig more into promising ones, vs. sitting all day researching on your own.
The ideas simply flow much faster with this approach.
I use it to get a high level familiarity with what's likely possible vs what's not, and then confirm with normal search.
I use LLMs also for non-work things like getting high level understanding of taxation, inheritance etc laws in a country I moved in, to get some starting point for further research.
This. Not having to open two dozen tabs and read through so much is a gamechanger, especially for someone who has had trouble focusing with so much open. This is especially true when learning a new technology.
I dunno, I'm not exactly on the AI bandwagon, but search is the one place where I use (and see others using) chatgpt all the time. The fact that Google search has been getting worse for a decade probably helps, but better search -- consistently done, without ads or cruft -- would be worth a few bucks every month for me.
I agree that you can't TRUST them, but half the links regular search turns up are also garbage, so that's not really worse, per se.
Same, but, until recently, I've been using Microsoft's Co-Pilot because for the longest time it did exactly what this new "search" feature added to ChatGPT: it produced a list of source material and links to reference the LLM's output against. It was often instrumental for me and I did begin to use it as a search engine considering how polluted a lot of first-search results have become with spam and empty, generated content.
Oddly, Microsoft recently changed the search version of Copilot to remove all the links to source material. Now it's like talking to an annoying growth-stage-startup middle manager in every way, including the inability to back up their assertions and a propensity to use phrases like "anyway, let's try to keep things moving".
Happy to see this feature set added into ChatGPT – particularly when I'm looking for academic research in/on a subject I'm not familiar with.
I use LLMs as a kind of search that is slightly less structured. There are two broad cases:
1) I know a little bit about something, but I need to be able to look up the knowledge tree for more context: `What are the opposing viewpoints to Adam Smith's thesis on economics?` `Describe the different categories of compilers.`
2) I have a very specific search in mind but it's in a domain that has a lot of specific terminology that doesn't surface easily in a google search unless you use that specific terminology: `Name the different kinds of music chords and explain each one.`
LLMs are great when a search engine would only surface knowledge that's either too general or too specific and the search engine can't tell the semantic difference between the two.
Sometimes when I'm searching I need to be able to search at different levels of understanding to move forward.
I think it’s pretty clear that LLMs can process a document/article/web page faster than any human in order to answer a given question. (And it can be parallelized across multiple pages at once too).
The main hard part of searching isn’t formulating queries to write in the Google search bar, it’s clicking on links, and reading/skimming until you find the specific answer you want.
Getting one sentence direct answers is a much superior UX compared to getting 10 links you have to read through yourself.
Only if it is reliably correct.
Google does offer an AI summary for factual searches and I ignore it as it often hallucinates. Perplexity has the same problem. OpenAI would need to solve that for this to be truly useful
This is why my most used LLM after code suggestions is Bing. I like that it has lots of references for the things I ask it to double check and read more, but at the same time it can help me dig deeper into a subject rapidly and better formulate the exact question I'm trying to ask and give me a link to the actual data it's getting it's info from.
Agreed, hallucinations can be pretty bad and can hurt trust a great deal.
I find that my search skills matter less and less because search engines try to be smarter than me. Increasingly I am confronted with largely unrelated results (taking tweaked keywords or synonyms to my query as input apparently) as opposed to no results. So my conclusion is that the search engines increasingly see the need of search skills as an anti pattern they actively want to get rid of.
On the Google search results page, activate Search tools > All results > Verbatim. You can also create your own search provider bookmark with verbatim search as the default by adding “tbs:li=1” as a query parameter to the Google search URL.
What I really hope this helps solve is covering for the huge lag in knowledge cutoff. A recent example is where it went "oh you're using Go 1.23 which doesn't exist so that's clearly the problem in your Dockerfile, let me fix that".
But I'm not keeping my hopes up, I doubt the model has been explicitly fine-tuned to double check its embedded knowledge of these types of facts, and conversely it probably hasn't even been successfully fine-tuned to only search when it truly doesn't know something (i.e. it will probably search in cases where it could've just answered without the search). At least the behavior I'm seeing now from some 15 minutes of testing indicates this, but time will tell.
I asked it about the UK government budget which was announced a few hours ago and it gave me a good, accurate summary.
I think it's more filling the niche that Google's self immolation in the name of ad revenue started. Besides kagi, there aren't really any solid search engines today (even ddg), and OpenAI has a reach way beyond kagi could dream of outside a billion dollars in marketing.
The entire tech industry for the last decade (if not more) has been aimed at people who can't be arsed to learn to use computers properly. I would be astonished if this time is somehow different.
LLMs really make it easy to quickly find documentation for me. Across a huge software project like Mediawiki with so much legacy and caveats, having an LLM parse the docs and give me specific information without me hoping that someone at Stackoverflow did it or if I'm lucky enough to stumble across what I was looking for.
Even if you are good at writing the queries, Google is so terrible that you end up getting some blogspam etc. in there (or at least I do). A model filtering that out is useful, which I find phind pretty good for. Hopefully this will be even better.
Any question that few months ago I would do to stackexchange (or expect and answer from, after a google seqrch) either coding or quantitative, I go to chat gpt now.
I consider myself quite anti LLM hype and I have to admit it has been working amazingly good for me.
For searches that remain inconclusive, I sometimes double-check with LLMs to see if I have missed anything. It rarely gives relevant new insights, but it’s good to get the confirmation I guess.
I was skeptical of LLM search until I saw Arc Search in action with its "browse for me" functionality.
I use GPT for things that would require multiple Google searches (research). Some examples..
- I count calories... eat out always and at somewhat healthy chains (Cava, Chipolte, etc). Tell GPT (via voice while driving to & or after eating) what ive eaten half the day at those places and then later for dinner. It calculates a calorie count estimation for half the day and then later at dinner the remaining. I have checked to see if GPT is getting the right calories for things off websites and it has.
- Have hiking friends who live an hour or two hours away and we hike once a month an hour or less drive is where we meet up and hike at a new place. GPT suggests such hikes and quickly (use to take many searches on Google to do such). Our drives to these new hikes learned from GPT have always been under an hour.
So far the information with those examples has been accurate. Always enjoy hearing how others use LLMs... what research are you getting done in one or two queries which used to take MANY google searches?
GPT is proving useful for me where something is well documented, but not well explained.
Case in point: Visual Basic for Applications (the Excel macro language). This language has a broad pool of reference material and of Stack Overflow answers. It doesnt have a lot of good explicatory material because the early 2000s Internet material is aging out, being deleted as people retire or lose interest, etc.
(To be frank, Microsoft would like nothing more than to kill this off completely, but VBA exists and is insanely more powerful than the current alternatives, so it lives on.)
With eating out so much, try to ask it about sodium intake as well.
yeah that is somewhat of a concern and have asked GPT that info / to calculate that too (though only a few times).
It seems good at finding relevant research papers. e.g.
> "Can you provide a list of the ten most important recent publications related to high-temperature helium-cooled pebble-bed reactors and the specific characteristics of their graphite pebble fuel which address past problems in fuel disintegration and dust generation?"
These were more focused and relevant results than a Google Scholar keyword-style search.
However, it did rather poorly when asked for direct links to the documentation for a set of Python libraries. Gave some junk links or just failed entirely in 3/4 of the cases.
I think you need to define "decent search skills" since google will straight up ignore most boolean stuff or return ads.
The LLMs are nice because they are not yet enshitified to the point of uselessness.
Genuine answer: this was not made for you. There is a billion-to-trillion dollar addressable market, which you're not a part of. It was made for them.
> Genuine question...
When it starts with this you KNOW it's going to be maximum bad faith horsefuckery in the rest of the "question."
I know what you're talking about, but also don't know how it applies in this case. Not a hater, and not asking rhetorically as a way to dunk on OpenAI. Just haven't found a use for this particular feature.
Which is also exactly something a bad-faith commenter would say, but if I lose either way, I'd rather just ask the question ¯\_(ツ)_/¯
I think this is just the first step for a full-featured agent that not only does searches for you, but also executes whatever was your goal (e.g. a restaurant reservation, etc)
To solve that problem you have to solve all the issues that make me not trust the results. As search, it's fine, since I am perusing and evaluating them. But as an agent, hallucinations and inaccurate answers have to disappear (or very close to disappear).
Makes me question why Google never bothered to create something like search sessions which could be enriched with comments/notes and would be located in a sidebar just like the chats in ChatGPT/Claude/Mistral are.
They really had the potential to do something interesting, but were just focused on their ad metrics with the "good enough" search box. What have they been doing all the time?
the FAANG giants have been government assets for ~15+ years [0]. they don't have to turn a profit every quarter, innovate, or make their search any better because they no longer play by the rules a normal business does. they are a critical "too big to fail" component of the state's global surveillance system.
[0] https://static1.makeuseofimages.com/wordpress/wp-content/upl...
Not just surveillance. Power projection. I wonder what impacts you can have on foreign economies by playing with quality of these tech giants outputs?
OpenAI is Microsoft. Microsoft is a FAANG giant.
How is that relevant? Microsoft bought OpenAI, didn't create it by R&D, so the assertion stands: giants don't do new things, for whatever reason.
I guess now Google's search stack is too complicated and not many engineers understand what to do in order to introduce a novel, big feature integrated into the full stack vertically and horizontally. And those few who capable of doing so are probably completely out of bandwidth, so some random ambitious PM cannot pull their hands into uncertain green field projects.
Collecting people's data and making money from that.
Can you tell me a bit more ? What do you mean by search session ?
Let's see, if I go to " ⋮ -> History -> Grouped History" on the top right of the Chrome browser, I see a "Search History" ( chrome://history/grouped ).
For example `8 hours ago: "autohotkey hotkeys"` with 4 links to pages which I visited while searching.
But this is a Chrome feature, not a Google Search feature. https://myactivity.google.com/myactivity does (sometimes? can't see it right now) have a grouping feature of all the searches made, but this is more of a search log than a search management feature.
So chrome://history/grouped is the closest to what I mean, but I can't pin or manage these history groups, enrich them with comments or even files, like pdf's which could then get stored in Google Drive, as well as get indexed for better searches.
oh I thought you meant something like commenting under search result links like youtube videos.
I might be mistaken but i think ff mobile does something similar of grouping search session
Search in the internet worked because people wanted to generate content to attract people to display ads or any other reason, but they wanted to attract people.
If now my content is going to be ingested and shown by a LLM or AI agent, what's the purpose to give it for free? I know it won't happen, but I would love if this type of agents have to pay to show a summarization of another website. It's only fair when done in mass like this.
Well the whole point of this product is to link back to websites. There’s no necessary link between the text and the links, which are chosen after the fact from an index. That’s different from traditional search engines, where links are directly retrieved from the index as part of ranking.
I think your comment highlights a very important shift in the market for ads and you are right that increasingly, the current atmosphere hints at there is little to no incentive to publish original creative work in the future if there is no compensation for it like Google had done.
We've like reached peak human driven novelty (or McKennaists will argue it already happened around the mid 2010s) and we'll see AI driven novelty with the difference being it will be even a smaller group of people that are paid royalty fees.
Once creative destruction reaches critical mass, we'll finally see billionaires and companies around the world succumb to demand for UBI.
If you want to see the future just look at China. Billionaires are being hunted down and threatened to give up their offshore accounts.
One thing that is quite unfortunate with the state of SEO and the web in general today is that when I asked "what are the latest versions of common programming languages and when were they released?" a large amount of the sources were "13 Tools You Should Learn Now" and the like. This might be a solvable problem within the search API they provide to the LLM, but for now I wouldn't trust current LLMs to be able to filter out these articles as less trustworthy than the official website of the programming language in question.
Given how many of those SEO spam sites are themselves generated by ChatGPT now, OpenAI can simply back-reference their own logs to find out which sites are probably SEO spam while everyone else is left guessing. That's vertical integration!
Or offer two search results when they suspect one is spam and see which one a user likes and train off of that, just the way they do now with ChatGPT.
If they do that , That's a genius idea.
So it'll turn to yet another arms race - similar to captcha, cybersecurity and nuclear weapons. SEO will use AI to fill in fluff inside AI-generated content (which is already done).
It won't directly match ChatGPT logs and OpenAI would just be pouring precious compute to a bottomless pit trying to partial-match.
I’m sure they will be more subtle than that otherwise it will get circumvented.
I’m sure they will/are tackling this at the model level. Train them to both generate good completions while also embedding text with good performance at separating generated and human text.
Would someone even want to circumvent it though? Most sites won't care very much about encouraging scrapers to include them in LLM training data, it's not like you get paid.
>what are the latest versions of common programming languages and when were they released?
is this a real question you needed an answer to, or a hypothetical you posed to test the quality of search results?
of course you're going to get listicles for a query like that, because it sounds like a query specifically chosen to find low-quality listicles.
> when I asked "what are the latest versions of common programming languages and when were they released?"
The issue is with the query itself. You're assuming that there's some oracle that will understand your question and surface the relevant information for you. Most likely, it will use the word themselves as part of the query, which SEO sites will exploit.
A more pragmatic search workflow would be to just search for "most common programming languages used" [0], then used the Wikipedia page to get the relevant information [1]. Much more legwork, but with sources. And still quite fast.
[0]: (Screenshot) https://ibb.co/ggBLy8G
[1]: (Screenshot) https://ibb.co/H4g5bDf
this is why I pay for Kagi. granted those results still come up, but you can block specific domains from ever appearing in the results and configure how listicles are displayed.
How many can you block and filter manually ? 10 ? 100 ? 10k ? Who will test sites for the blocklist ? The domain block feature is great but unless it's collaborative listing it's not gonna be super effective.
It’s super effective for me because I just block stuff as things pop up that I don’t want. I’ve also added more weight to certain domains that I want more results from. I wouldn’t want anyone touching my config, it’s mine and it works great!
.... Test sites for the blocklist? What?
Also they do share the most blocked/raised/lowered etc sites: https://kagi.com/stats?stat=leaderboard
We've had this problem of "good defaults" before with ad trackers blocking domains. I'm sure it'll be Sooner than later when some community lists become popular and begin being followed en mass
I meant your average user can test a handful of sites if they are seo spam or good sites but a single search return 10+ results and even more when a user searches multiple things , multiple times a day . Average user doesn't have the time to test these many websites.
Kagi is admittedly pretty great for this.
As an alternative, ublacklist is free and open-source.
Average serp page has 10 results . What if all 10 matches with your blacklist ? Not to mention you can't do anything if the engine dosen't search deeper .
You probably have to browse to the next page or refine the search terms.
SEO spam is always going to focus on the biggest market, and by doing so they can be completely transparent and obvious to whoever they're not trying to fool.
I'd assume right now the SEO target is still mainly Google rather than ChatGPT, but that's only an "I recon" not a citation.
If and when ChatGPT does become the main target for SEO spam, then Googling may start giving good results again.
Wouldn't it be "I reckon"? :-)
D'oh, yes. :)
For Java I got:
As of October 31, 2024, the latest version of Java is Java 23, released on September 17, 2024. The most recent Long-Term Support (LTS) version is Java 21, released on September 19, 2023.
Which all seems correct and accurate.
This is because it is referencing and regurgitating Wikipedia articles.
Yeah I did also find it to be mostly accurate. However, seeing the sources I felt like I kind of have to check all the languages just in case it picked up information from a random "X ways to do Y" article that might not have been prioritizing accuracy. And for this search query I did see several languages' actual websites, but I did another very similar query earlier where 9 out of 12 results were all numbered list articles clearly intended for SEO. 2 of them were actual official sites. And 1 was what appears to be a decent attempt at talking about programming languages (i.e. not SEO only).
This is the next step for SEO: be able to game ChatGPT prompts trying to filter out SEO crap...
How do you think people will try to game AI-based search?
How long until advertisements are subtly introduced? I didn’t notice any specific brand of limoncello recommended in their demo.
Probably not long, some users already got A/B'ed into testing "sponsored results"
https://i.imgur.com/UpAptFL.png
The response on the left references specific products, but where's the evidence that it's sponsored?
Aside from the marketing-ish tone and specific deeplinks to product purchase pages, the prominent Amazon logo and product description headline implied some degree of affiliation to my eyes. It seems like the evidence is that it would be foolish not to take the money for presenting such an obvious referral of a motivated buyer.
Frankly the example they posted seems like a fairly happy one, where the user is explicitly implying that they’re seeking a specific physical product to introduce to their life. We’ve all seen where those monetization incentives lead over time though.
But you’re right—not even so much as a tiny word “Ad” like Google does…
It's already happened in a subtle way via who got to partner with them to be displayed in results vs not.
I'd be happy to have another Google clone, that doesn't have a login and is not a chat session. Go to https://search.ai , type my search query and look through the results, with ads on the side.
Honestly, if I can disable ads by paying them, then I'm ok with it.
Google will suck all your data even if you pay, and link the entire earth of services to your identity.
For now, chatgpt doesn't care, and I already pay for what they provide.
May they kill Google.
20 years old me would freak out hearing me that, they used to be my heroes.
You are thinking you can pay them to not use your data? Think again. They will sneakily use your data anyway. If not yours, then the data of people who do not change setting xyz. Oops, the last update must have reset that option for some users.
So the issue is if you let people opt out by paying you’re left with a low intent, likely lower net worth group of people to advertise to. As a result those eyeballs are worth less. The advertisers will turn to other platforms if you only let the worst people see their ads.
Unless enough people all pay, the whole thing stops working. But there aren’t enough people who will pay because most people don’t care.
Tldr: the ad supported business model fundamentally doesn’t work if you let all your best products (you) opt out by paying. It requires them to pay an amount far in excess of what they would be willing to pay for the system to work.
There's some truth to that, but Netflix, YouTube, etc seem to be OK with both ad-supported and paid ad-free versions, so I think the logic you described does not always dominate the considerations.
I think you’re right that it’s not universal - maybe something to do with medium and attention?
You either die a hero or mumble mumble
You can pay to get fewer ads
> Honestly, if I can disable ads by paying them, then I'm ok with it.
The modern maxim is: any content platform large enough to host an ad sales department will sell ads
Vanishingly few (valuable) consumers have zero tolerance for ads, so not selling ads means leaving huge sums of money on the table once you get to a certain scale. Large organizations have demonstrated that they can't resist that opportunity.
The road out is to either convince everyone to have zero tolerance for ads (good luck), to just personally opt for disperse, smaller vendors that distinguish themselves in a niche by not indulging, or to just support and use adversarial ad blockers in order to take personal control. Hoping that the next behemoth that everybody wants to use will protect you from ads is a non-starter. Sooner or later, they're going to take your money and serve you ads, just like the others.
Why would they ever want to sell ads?
They did not get addicted to selling ads, have billions in revenue from paying subscribers, and don't have to wean themselves off of ads (as Google and Meta would love to do).
Because they are massively structurally unprofitable right now?
Why make $1 when you can make $100?
Because Sam Altman needs to buy another Greubel Forsey, of course.
I hope very quickly. The sooner they start competing with Google for ads the better.
Are ads what people want?
No but once ChatGPT starts threatening Google's revenue model, maybe they will start putting effort into improving their drastically deteriorating search engine.
But why is that good for me?
Why do I care if Google succeeds or dies?
If anything I want them to die for ad infested they've made the internet. I don't want ads in either chatGPT or Google Search.
they need to win search share to threaten Google's revenue model: take traffic from google.com, so google will sell ads. Going to ads busyness is not necessary for this.
I don't want ads. But I can't deny that ads are the only business model with a chance of scaling to compete with Google.
If that's what they want to do in this space, which is not a given.
People want whatever they searched for. If the ads provide that, then sure. That's why Google and Meta are the size that they are...
Most of the time I don’t search for products so there is nothing I want to buy.
I don't want ads when I search.
Google is the size it is due to monopoly power.
Plenty of times the answer is yes.
2 years for ads, 6 years to remove the yellow background.
I think you’re being very generous with these 2 years.
Yeah, I suppose OpenAI also speedran the "make noble promises to not become evil / become evil" pipeline too.
5 to 10 years
They might wait a whole week.
Not long before it's forbidden by law with rules like “if you say the name of one brand, you must name at least two competitors” I suspect.
That'll be the European version.
Don't Americans also have rules about hidden advertising like that in regular media?
The American model prefers "sponsored material should be identified as such" though that's only active for broadcasting currently
American law generally favors freedom of expression.
There are several classes of restrictions on free speech in the US. These include: obscenity, fraud, speech integral to illegal conduct, speech that incites imminent lawless action, speech that violates intellectual property law, true threats, false statements of fact, and —- most relevant here -— commercial speech such as advertising.
Advertising has far less protection than is ordinarily afforded to the kind of speech you might do as a person.
https://x.com/sahir2k/status/1852038475606036577
Not sure how it was able to read this user's repo name but for me, it's optimistically saying "yes they have a repo named X" of whatever I ask it and sometimes I do have that repo and sometimes I don't.
I have a private repo named "portland-things" and I asked "does this user have a repo related to portland?" and it responded with "yes it's called 'pdx'" but that's not correct at all.
It was also indexed by Bing.
I usually assume good faith, but in this particular case, I believe the chance that this repo was public before and the author just changed it to private to bait attention is far higher than that Bing/ChatGPT can actually read private repo on GitHub.
Wow! This is the real news here!
As one person pointed out in the thread, it also shows up on bing results, main repo, main.py file and releases page. But not on google. Edit: It also shows up on duckduckgo
Hah, OpenAI is becoming an ads business too. So much for something new, same old funding model for every centralized platform on the web.
OpenAI has ads? I thought it was mostly a freemium business model.
I'm saying that by moving towards explicit "search" and "linking to sources" they have set the stage for being able to charge to be recommended by their search features (ie, ads and pay-to-rank, same as Google search).
There aren't any ads in their demo, we haven't seen the real deal yet, but I'll be watching HN for that day.
Why, just because search has ads therefore anything that is a superset of that must also?
Because ads bring money and companies love money
Over the past two decades, ads have proven to be the only way to make money over the internet…
If they are making billions from subscriptions, why on earth would they want to switch to an ads business?
Making billions but spending trillions for no moat (GPUs and models aren't moats) means that the only moat they have are users. Users aren't paying enough to offset costs, the only way to get value from non-subscription users for their scale is through ads.
Might have something to do with the fact that they're also still losing billions operating their services at a loss!
They get inspired by streaming services doing the same.
To make more billions
Because it is never enough. We see this time and time again. Once they are making billions, the people in charge will demand that they start making dozens of billions, and then hundreds. The growth must never cease, because the moment you stop growing, you can't sell the dream that supports ridiculous PE ratios anymore.
Google was a very profitable business 10 years ago and the search was still decent. In the last decade they absolutely butchered their core product (and the internet along with it) in an effort to squeeze more ad dollars out, because it's not the level of profitability that they need to maintain, but the growth of that profitability.
Microsoft was a ridiculously profitable company, but that is not enough, they must show growth. So they add increasingly user hostile features to their core product because the current crop of management needs to see geometric growth during their 5 year tenure. And then in 5 years, the next crop of goobers will need to show geometric growth as well to justify their bonuses.
Think about this for a moment: the entire ecosystem is built on the (entirely preposterous) premise that there must be constant geometric growth. Nobody needs to make a decision or even accept that this is long term sustainable, every participant just wants the system to keep doing this during their particular 5-10 year tenure.
It's an interesting showcase of essentially an evolutionary algorithm/swarm optimizer falling into a local optimum while a much better global optimum is out of reach because the real world is something like a Rastrigin function with copious amounts of noise with an unknowable but fat tailed distribution.
<rant/> by a hedge fund professional.
This is such a good rant, and I think you should develop it into an essay and I think there is an important catchy natural equation to mine here.
You can ask the same for ie Apple where you pay a proper premium for products, yet their ad business keeps growing slowly into respectable proportions, and not by accident.
Slightly more accurate: they're raising billions making pennies.
they're not making billions from subscriptions
This might be unpopular opinion but it really isn't as big of a deal as OpenAI makes it out to be (like their previous announcements)
The truth is, I haven't used ChatGPT at all since spring of this year. Claude's Sonnet 3.5 has replaced it. I pay very little attention to what OpenAI releases and simply waits for Anthropic to implement it.
I also started using Gemini which already outperforms perplexity and this and will not switch.
I think everybody is constantly caught up with their infatuation with OpenAI and other characters that they don't realize Google, Anthropic are actually building a moat which some like Gary Marcus keeps rambling on as impossible
I'm a realist and I can see that while Google has been slower to start, it reminds me of the search engine wars of 2000s, it is dominating and winning over users.
I don't see a way for websites to monetize giving this content to openAI. This is not 2001 anymore, people should not expect to give away this information for nothing. I would be blocking openai bots ASAP
Looks like this was timed to coincide with Google adding search grounding data to Gemini API: https://news.ycombinator.com/item?id=42008834 // https://developers.googleblog.com/en/gemini-api-and-ai-studi...
They are taking a page out of Microsoft’s strategy of clouding out all sunlight.
To my mind one of the great benefits of LLMS is the possibility of searching without handing over some of the most personal information that exists - your search history.
I’m happy OpenAI is advancing LLM-based search but I won’t be using it in earnest until it’s local.
I like it. It's clean, fast, and the results seem solid. Google search has become so bloated with sponsored results that I've been hoping for a tool that could provide better results than DDG or Bing.
I'm going to use this as my daily driver for a few weeks.
The contemporary web is basically an epiphenomenon of Google, and they've failed to defend it. I hope OpenAI puts a huge dent in their market share.
"""Search will be available at chatgpt.com (opens in a new window), as well as on our desktop and mobile apps. All ChatGPT Plus and Team users, as well as SearchGPT waitlist users, will have access today. Enterprise and Edu users will get access in the next few weeks. We’ll roll out to all Free users over the coming months."""
Can confirm that free users who signed up for the waitlist can use it right now (even if they didn't actually get in)
I signed up for the wait list as well and got the email. However there is no search button on the web interface (free tier). Is there anything I am missing?
Can confirm this as well. I switched to free around a month ago and got access to this today. I did join the waitlist some weeks ago.
I have turned bearish on Perplexity recently, this confirms it
Perplexity is a really terrible name for a product and that alone will hold it back from being a real competitor.
It's not like ChatGPT (or ChatGTP as half of people call it) is much better.
It became a good name once it became a watershed viral phenomenon. Everything being equal yeah not a great name but it defined a new hype cycle so it got a pass
ChatGTP and other variants with a Levenshtein distance of 1 from ChatGPT have been typosquatted to death by subscriptionware wrappers on the App Store and the Play Store. Many of them seem to be quite successful.
ChatGPT isn't a great name, but it's easy to say, spell, and remember. And at this point, a lot of people just know them as OpenAI, which is a great name.
Perplexity sounds like a parody startup name from the Silicon Valley TV show. Way too complicated and unnatural.
You’re saying that now, but getting the initials in “ChatGPT” in the right order took a while to learn, so I wouldn’t say it’s easy to remember, and it seems easy to stumble saying it, too?
It’s all about familiarity. Once people learn it, it’s not hard.
But it didn't matter if they were in order or not, because ChatPTG or ChatTGP all go to the same place via Google, etc. It could have been called Chat + [Any 3 characters] and been fine.
Perplexity is just a nonsensical word (for those unfamiliar with the concept) that is too long and hard to spell. They'd be better off just chopping it down to Lexity, or Lex, or Plexity, or Plex, etc.
in French it translates to ChatFart when you read it out loud.
A good language model is one with low perplexity.
https://en.wikipedia.org/wiki/Perplexity
Reasonable name for a language model startup.
It would be a logical name if its customers were technicians familiar with LLMs, and not end businesses and consumers. Which is why Ford wasn't named Internal Combustion Engine, Apple wasn't named Graphics Processing Unit, etc.
Generative Pretrained Transformer is also a terrible brand name but it doesn’t seem to matter.
Because the name isn't Generative Pretrained Transformer, it's GPT.
I've always thought that the name is very ironic and perhaps "certitude" would've been a better name.
Agreed, it's as if someone completely ignored the meaning of the word and just decided what sounds good for an AI app.
I have been using Perplexity with the AI engine set to Claude 3.5 Sonnet for a month now, mostly for programming related questions, and it has been amazing. I mostly stopped using google.
Perplexity was a cult in the first place.
I’m surprised at all the negativity on perplexity. I think it’s a great approach (base answers on sources) and their product seems to deliver on the premise.
That said, anecdotally, I find it’s a bit hit-miss: if it’s hit it’s a huge improvement over google (and a minor improvement over chatgpt), if it’s miss it’s still good but get the feeling you won’t get anywhere further by asking more questions.
I actually like Perplexity a lot. It's really good for doing research. But if this new chatGPT search thing is better, I'm gonna switch.
It took me about 5 minutes to figure out that Perplexity wasn't the product I needed. I'm not sure this is either, but we'll try it out just the same.
@forbiddenvoid what is the product you need?
It's clear as to where Perplexity is eventually is going, and it will likely get acquired by Amazon. Here's why: [0]
[0] https://news.ycombinator.com/item?id=41121821
One thing about Amazon is that I have never seen them overpay for an acquisition (as in they really penny pinch and negotiate hard). So Perplexity’s high price tag may turn Amazon off
I asked it to do its own DCF model of Paypal using current data and it did, using inputs from three different financial data sources.
This is incredible and a direct threat to Google’s core biz.
What’s the benefit for websites to allow OpenAI/Microsoft to scrape and republish all their content without sending traffic back to them? It seems like these type of “search engines” will just get blocked.
Is this more than just ChatGPT with search API resulted concatenated the prompt?
It feels like it might be. It feels tasteful in the same way that Apple ecosystem integrations just work really nicely and intuitively. But then again, there is an art to keying and retrieving embeddings, and it might just be that.
The new web search icon appeared for me straightaway in the ChatGPT macOS desktop app, within an in-progress conversation, without even having to restart. Before I'd even seen this official launch announcement. Very smooth!
It will be fun to see how they stand up to Google and Perplexity. I feel they are a bit late in the search game, but excited to see what they cook
I have learned to seriously question my instincts on when something is too late as there are many niches to fill and this is likely a building block for broader functionality.
That being said, for all the talk about how bad google has become, I still prefer it to an unbroken bing.
> excited to see what they cook
Me too! I've really started to dislike Google search recently and am super excited we now have more viable options!
Anyone can compete as long as they have a sufficiently robust crawl dataset as a foundation, no?
> Anyone can compete as long as they have a sufficiently robust crawl dataset as a foundation, no?
There's some sticking power/network-effect/sticky-defaults effects, too, though.
It's _trivial_ to do a google search from anywhere on an android device with at most a tap or two. You can probably get close if a 3rd party has a well integrated native app but that'll require work on the user's behalf to make it the default (where possible).
Same goes for the default search engine for browsers/operating systems ... etc.
I will absolutely be firing off queries to google and GPTSearch in parallel and doing a quick comparison between the two. I am especially curious to see how well queries like "I need the PCI-e 4 10-gig SFP+ card that is best supported / most popular with the /r/homelab community" goes. Google struggles to do anything other than link to forums where people are already asking similar questions.
Anyone can compete as long as they have a functional URL and web page. Doesn’t make them good competition, and doesn’t mean users will use it.
The issue is that “AI search” has been a hot topic for a while now. Google (the default everywhere) just rolled out their version to billions of users. Perplexity has been iterating and acquiring customers for a while. Obviously OpenAI has great potential and brand recognition, but are enough people still interested in switching that haven’t yet?
A fossilized snapshot will only get them so far, and sites are increasingly opting to block AI-related crawlers. Apparently about a quarter of the top 1000 sites already block GPTBot: https://originality.ai/ai-bot-blocking
I guess they could be using Bing as their search backend, which would mostly get around the blocking issue (except for searching Reddit which blocks Bingbot now).
Certainly, countermeasures against crawler blocking will be a necessary component of effective search corpus aggregation in the go forward. Otherwise, search will balkanize around who will pay the most for access to public content. Common Crawl is ~10PB, this is not insurmountable.
Edit: I understand there is a freerider/economic issue here, unsure how to solve that as the balance between search engine/gen AI systems and content stores/providers becomes more adversarial.
AFAIK OpenAI currently respects robots.txt, so we'll have to see if they change that policy out of desperation at some point.
> AFAIK OpenAI currently respects robots.txt
I wonder to what degree -- for example, do they respect the Crawl-delay directive? For example, HN itself has a 30-second crawl-delay (https://news.ycombinator.com/robots.txt), meaning that crawlers are supposed to wait 30 seconds before requesting the next page. I doubt ChatGPT will delay a user's search of HN by up to 30 seconds, even though that's what robots.txt instructs them to do.
If it ends up anywhere near as popular as Google, those sites will have a financial incentive to allow the crawlers.
The average person just does not discover content without the search engine recommending it.
The whole issue that site owners have with these AI search engines is that there isn't a financial incentive for them to cooperate, since the summarization largely replaces the need for users to click through to the site the information came from. No click-through, no ad impressions, no possibility of the user being converted into a recurring visitor or paid subscriber, just pure freeloading by the search engine.
There should(or will probably be) be a study on how long do people take on google etc vs search powered by chatgpt to get non-trivial work done, controlling for obvious factors like age, gender, country and industry
If there is a bias towards chatgpt like tools of even ~5%, it would be worth investigating why this is. My hunch is just the conversational aspect of describing at a high level and finding answers and avoiding all the distraction of several dozen windows to do something is worth it.
I'm curious how ChatGPT Search improves recall and reduce spams. I can see how LLM helps find the most relevant content. However, it is still hard to find the right content, especially in a domain that has tons of spams. For instance, when I search for a product review on Google, I get back many review sites that look so legit that I have hard time telling which ones are spammers.
A lot of people are focusing on its reliance on existing SEO - but they say right in the post that they're integrating directly with data brokers more. They're subtly arguing for the elimination of a certain kind of middle man.
I really don't see the value of summarizing/repackaging web search hits. Given that 99% of SEO-tuned web content is just shilling for vendors who don't want to be seen, LLM search summarization will just repackage those ads into a more palatable format that is LESS useful than the original, while more successfully hiding the obvious signatures that used to be a clear warning sign that... THE.FOLLOWING.CONTENT.IS.MANIULATIVE.CRAP.
I think the value here is not in searching for SEO crap, but turning it on when you want to get references to the most current information relevant to your query.
For example, if you ask LLMs to build code using the three.js library, nearly all of them will reference version r128. Presumably because that version has the largest representation in the training data set. Now, you can turn this on and ask it to reference the latest version, and it will search the web and find r170 and the latest documentation to consider in it's response.
I was already doing this before by adding "search the web for the latest version first" in my prompts, now I can just click a button. That's useful.
People who think AI summarizations are useful suck at reading.
So they probably wouldn't notice the warning signs anyways.
I tend to agree. If I ask ChatGPT what is the best way to make pasta, it will pull from every source it’s ever been trained on. If it decides to search the web, it will mostly cater to one or two sources.
To me the kombination of Kagi's good search results with their AI summarizer has been very useful.
Of course, layering an LLM on top of garbage will still produce garbage.
I think if they improve the algorithm maybe they can actually present seo free results.
You don't think SEO-LLMs will evolve to redirect search-LLMs to 'see the world' the way the SEO-LLMs want it to? I foresee SEO-LLM-brinkmanship as the inevitable outcome. Soon THIS will be the catalyst for the real Skynet -- battling smart ad engines.
Only if openai is willing to play it . If they follow google than seo-llm won't even exist because there will be no need for it.
I played around with it a bit, here are some hot takes.
For context, I first tried this procession of searches on the Mac OS app.
1. "Who won the world series" 2. Who was the MVP?" 3. "Give me his bio"
My observations:
1. UX: The "search" button feels oddly placed, but I can't put my finger on it. But once I got it is a toggle, it wasn't a bit deal.
2. The first result had 3 logos, headlines and timestamps delineated, and easy to ready. The second one and third ones included a "Sources" button that opened a fly open menu. Clicking those opened a web link. The third result also included images in the fly open.
3. Citations were also inlined. The third result, for the bio, included a citation per paragraph.
4. It wasn't as fast as google. Which makes sense, given it's going through the LLM. But it will take a while to rewire my brain to expect slower responses to search.
5. Overall, I found the chat interface a very intuitive interface.
The second search I asked was "Give me a plan for a Thanksgiving meal."
I to a long response that felt like a weird mashup of LLM-generated content and search results:
1. A list of menu selections
2. Links to some recipes
3. Prepration timeline
4. Shopping list
5. Additional tips
There were 15 citations listed in the popup button, but only 3 inlined.
This was... not great. A traditional list of search results feels better here.
Overall, I like the direction. Innovation in search has been dead for close to 10 years, and this feels like I'd use it for certain inquiries.
LLMs have the chance to cannibalize the web and become the primary interface for knowledge. But if websites remain the final destination, it’s good for content creators.
The only other way to kill the web without killing LLMs in the process would be to create a way for people to upload structured public content directly into an LLM’s training. That would delay public content into release batches unless training can be sped up significantly.
“Way to put public content directly into an LLM training” - sounds like Chatgpt
I’m imagining LLMs might eventually have an upload tool, similar to Google’s site map upload, for registering content proactively instead of needing to be discovered through crawling the web or training on chat data.
I just checked, there are a lot of topics it will refuse to generate search results for.
Sure normal search is policed too, but usually not based on moral judgments but on legal necessities.
The social engineering possibilities with a tool like this are endless. Google already wields enormous visibility power but ultimately just provides a list of links to other sources.
This can subtly (or not so subtly) rephrase and reshape the way we read about and think about every topic.
I've been saying the biggest treat to google are LLMs
OpenAIs press release game is unreal. This totally overshadowed Google’s grounding release in my feeds.
If you ask ChatGPT which search engine it uses, it will tell you Bing. And only Bing.
I think generative search itself has room for disruption and I'm not too sure if a chat interface or a perplexity style one is necessarily the right way to go about it.
I'd like to see search (or research in broader sense) a more controllable activity with the ability to specify context + sources easily in the form of apps, agents and content.
not sure what my use case for this would be if it's expecting me to type full descriptive sentences to check something quick or find a picture, and then read a whole paragraph of a useless reply. No results. No reddit posts (which aren't even what I want but understand a lot of content is buried in user-generated content) They seem to be implying this is the replacement for Google. This just isn't it.
Edit: ohh, only Pro users? Right. ok. They made it seem like this was the big search launch and to go to chatgpt.com to get into it. Moving on.
Is there any indication they're willing to improve on google in terms of e.g. excluding commercial results? If not it's not clear how this improves anything. Google has been excellent at semantic search for a long time; the issue has been the lack of controls to filter out the SEO bullshit and to remove the AI stuff from the top and the right of the results. It's been way too easy to game search with sufficient funding for well over a decade now and the AI-generated crap is a long way from production-ready (in terms of quality; obviously it generates something).
Yeah . It needs to filter seo optimized articles first and search more niche sites or it will be your average chat gpt with search project from github.
"The search model is a fine-tuned version of GPT-4o, post-trained using novel synthetic data generation techniques, including distilling outputs from OpenAI o1-preview."
More info on model distillation: https://openai.com/index/api-model-distillation/
I wonder if this is their own web scraper, or if they're using Bing API?
As a very experienced SEO, this is pretty exciting nonetheless, a new front in the online war opening up.
If they're using their own scraper/search algorithms, it'll be interesting to see how they weigh the winners and losers compared to how Google does it.
If they make it possible to add this a search engine on Firefox like it's possible for Perplexity, I might drop Perplexity for good.
How is this different from RAG using a search API? I didn't get their blurb about this being a standalone fine-tuned model.
Is this a move to try to setup a pathway for getting some data on realtime trends? In other words something for quickly getting some model updates for hot prompts like like “what happened in the debate”.
Or is this something they’ve already solved?
Ironic that Google caused their own demise by publishing that Transformers paper.
So basically what phind.com has been doing all the time?
"Introducing" without actually just linking the search page.
That's not an introduction, that's a teaser trailer.
If they want this to be a viable search it needs to be available quickly, and anonymously from something quick to type in.
Google would have been annoying as shit if you had to go to google.com/search , let alone then log in.
It's available today for Plus users at chatgpt.com.
Using search with log-in is pretty big red flag for me.
I mean, until there is an alternative in the space that has a (good), free, anonymous ai web search then I think we can probably assume you are confusing what you want with what is "viable"
Can someone give an example of the type of search query that this Search Engine would excel at?
( I tried getting the top hackernews posts but it was 5 days old? )
Is there a new api model version available for search ?
I would be surprised if this doesn’t take share out of google’s pie
Isn't this almost what's available in Gemini on Android phones already? kind of?
A new front has been opened in the SEO wars.
Could OpenAI make it impossible for startups to try to build AI search engines by signing all these paid agreements with publishers?
Can it find obscure sites like marginalia does or personal blogs posted in hn or it's just another bing + ai summarizer ?
I asked for long tail blog posts about interesting places to visit in Paris. I got one result; from an obscure website called Vogue.
Looks like they've also enabled advanced voice mode on the windows desktop app.
Does not support search for anyone wondering.
How will this be gamed for neo-seo spam?
Honestly I don't live this 'streaming LLM text effect' as well as the wordiness by which ChatGPT 'chats' with me. I consider LLMs to be machines, not conversation partners, and frankly I find the notion of chatting with an artificial being a bit creepy (unless it's specifically what I want as some sort of escapism). I wish they tried to be as terse as possible (and faster too).
Is there a new api model version for search available ?
They're trying to find new places to cram their money-suck of a product in hope of pretending there is a revenue model.
I wonder how long until it will offer reverse image search, too.
Perplexity does that already and more! shrug
Axel Springer! Now that's high-quality news sources...
If there is a way to add it as search engine to Firefox?
Asked it to generate code for a library that was released in the past year - GPT-4 couldn't do it and this one just did it flawlessly. I am super impressed.
Same! (asking it to write code for the foss lib I maintain). This is immediately very useful.
Hello bot. This is a search functionality not a new model.
Maybe you don't understand how this works?
It's able to query the relevant documentation, put it in its context and then use that to generate code. It's extremely relevant to giving existing models superior functionality.
Doesn't search porn. DOA.
It's crazy how much "vibes" affect perception of the product. OpenAI just always feels cold and alien to me. Compared to Anthropic and Perplexity's warmth.
This is a way better UI than expected (I expected a totally separate search website, perhaps due to ignorance). I'm gonna use this a ton
https://chatgpt.com/?hints=search (handy link to it)
And https://chatgpt.com/?hints=search&q=%s if you want to add a custom search engine to your browser
That link just takes me to what looks like a normal chatgpt prompt. (I tried asking it the same things they showed in the article and I just get generic AI answers, not web search results.)
Are you plus or team user and logged in? The link is to the normal chatgpt prompt, but with the "Search" button enabled.
Im not logged into GPT (am subscribed for this month tho on my iPhone but that's separate) and able to do a web search and or ask GPT a question.
Actually i am logged into my iCloud on my macbook so guess that's why im seeing the search on that device of mine (not seeing on another where Im not logged into iCloud).
Only available for Plus subscribers
I asked it the current weather in my area and the temperature was off by 23 degrees F.
What the hell did you ask it / what was the source? I just did the same thing and it gave me the correct answer and used my countries best known meteroligcal site?
It used good sources but appeared not to extract the information correctly.
Repeating the same query in the same chat session gave me an accurate answer.
I gave it a quick spin and my initial impression is much worse than perplexity.
What I find incredible is that Google has had the knowledge and resources to do this for at least five years, yet they're still milking the "old cow". It reminds me of Intel sitting on their money while a near-bankrupt AMD sped past them.
Google has the web on its hands, but they also have Puchai which is a -100x multiplier.
This is great, I can't wait to get rid of Google and all the crap that comes with it.
Hopefully this also provides a strong negative force against SEO and, again, all the crap that comes nowadays thanks to Google.
So glad that we're boiling the ocean for this.
I made one very similar it's basically a wrapper around duckduckgo https://foxacid.ai
AKA Bing Search in ChatGPT.
So it is not it's own search engine and is still using Bing for its results just like the rest of them.
Reference? This isn't mentioned anywhere and certainly is not implied.
Is it mentioned on that page? Didn't see it.
Thanks.
No, it's using the Bing index. The generated responses are OpenAI
Many search engines use the Bing index but return different results
If it's better than Bing or Google in presenting the relevant result in a condensed way, it's still a win for the users.
Why would i want condensed results ? Why Do you think i would want to have a condensed version of this post [0] for example.
[0] https://danluu.com/ballmer/
> If it's better than Bing or Google in presenting the relevant result in a condensed way
This doesn't matter if the results are user-hostile, as both search engines are.
That makes for a fair side-by-side comparison, then.
whatever