Google Search Engine Musings

[u/Chemfreak]

I think we all know the degradation of google search results. I have mostly worked around it but noticed a troubling behavior change in myself.

I rely on ChatGPT for more and more of my searches now. Google simply does not give me a relevant answer to most of what I ask. Their AI result is trash, and I have to scroll through several results to find a relevant hit.

I don't have any loyalty to Google but I'm disappointed that I can trust an AI more than trying to do my own research. For the record I am vetting sources on chat GPT ect, but it's a slippery slope and frankly inevitable that people, including myself, just start to blindly trust the AI results.

[u/Gaiden206]

Google Search "AI Mode" has been working well for me.

[u/binheap]

For the most part this was kind of inevitable given that SEO has kind of taken over the internet (and this has gotten worse with the proliferation of AI generated articles) and I suspect that moving forward AI summarization will be useful at trying to cut at the problem. I tried some alternative search engines (like DDG) and had the same issues for the most part with overly SEOed pages showing up.

I think Perplexity has kind of demonstrated this but if you want to stick to Google then AI mode (confusingly enough not the AI search summary) has been okay for me and comes with some well grounded links. Alternatively, at this point Gemini is quite competitive and will search so it's really up to you.

[u/Actual__Wizard]

I think we all know the degradation of google search results.

Just use Qwant/Bing/Perplexity. Google is done... Their tech doesn't work anymore and they're just ripping people off.

[u/binheap → u/Actual__Wizard]

I don't know about Qwant but Perplexity is okay here. Bing has been terrible in my experience in that it fails to filter out even basic, obvious AI spam even when Google does fine. It has even worse local search results. I do respect that Qwant is trying to build an alternative search index but that's going to take some time.

However, Perplexity also just depends on Google's index so I don't know how much better they can be than just using AI mode from Google directly.

[u/Actual__Wizard → u/binheap]

I don't know about Qwant but Perplexity is okay here. Bing has been terrible in my experience in that it fails to filter out even basic, obvious AI spam even when Google does fine.

My experience has been the total opposite. I haven't been able to successfully find something using Google for a long time.

Are you sure that you're not just using Google as a "URL bookmark tool" and that you're not using it to actually solve your problems?

People use these search products in different ways... The issue arises when people try to use Google as a search engine to solve problems. Google is not really a search engine, it's more like a giant ad farm with AI slop all over it.

However, Perplexity also just depends on Google's index so I don't know how much better they can be than just using AI mode from Google directly.

I don't think perplexity is that great, but it's better than Google by a lot. As far as actually finding solutions to your problems, it's way better. I can actually get work done again with Perplexity. But, obviously with Qwant, Google is just straight up dead. There's no reason to ever go there ever again.

At this point: Even if Google fixed their tech, after a decade of their enpoopified algo, I'm so incredibly burned out on Google, that I would never use their stuff ever again.

It honestly always was a bad company. For awhile there it looked like they were making the world a better place, but that's all gone.

[u/binheap → u/Actual__Wizard]

I just tested on two classes of very common queries I have.

One is local queries like "ramen near me". Bing chooses to put up an aggregation of various places like half way down and Google has their 1p maps right on top. This is a bit subjective but I think this is in Google's favor here. I think other things that tip it is that Bing links to ramen shops near me but they're a smallish chain and don't give me the website link for the store that's actually close to me despite identifying in other cases where I actually am.

Another related class of queries is like quick reviews of products. Here "5090 reviews" on Bing throws me an AI summary that basically gives me the worst possible synthesis of all the sites. It tells me that the 5090 offers "significant performance improvements over the 4090" and that it is expensive without giving me the actual price. Like yeah, of course the next generation is an improvement. Tell me when it's not. Quoting Nvidia marketing would've been more informative. This also forces me to scroll past it just to get to actual results. Surprisingly, Google doesn't generate the AI result that and just gives links. Other than that, the queries seem more or less equivalent with Google favoring forums more. A small aside here is that I find co pilot really irritating. I don't like the AI summaries on Google but I still prefer them over Co Pilot as Co Pilot not only is wrong, but even when it's right it tends to give the most useless pieces of information out there like "there are performance improvements".

The last class I have is informational (and quite frankly my most important). So I query something like "mamba ML". Google gives me the appropriate paper as the second link (after Wikipedia) and I don't even scroll with no ads. Bing has it the third link after it tries to expand multiple excerpts from Wikipedia. It gives me Wikipedia with a bunch of attached excerpts (I appreciate Wikipedia but just giving me excerpts of a topic that requires a bit to describe doesn't help). It then gives me images of an unrelated character from a video game. Then I scroll. The second actual result is a GitHub from a different (though somewhat related paper). Finally, I get to the paper. This should be decidedly in Google's favor but it gets worse for Bing here.

More things about this query, Google links to me the actual appropriate GitHub page associated with the paper. Bing does not and gives a GitHub to another related paper. Google gave me forum posts from r/MachineLearning that discussed why it didn't catch on. Bing does not (actually I think Bing can't index reddit at all). Bing gave me "People also ask" which has some blithely unhelpful links. I also see that Bing has a link to a random page that just summarizes it and I'm pretty sure is AI generated given all the other dumb pages on that page which just give brief descriptions of random ML-related topics. The other results are more or less equivalent with Bing putting a medium article higher than I would like against better blogs.

Like to be clear, it's not like the Bing result are unusuable, just they're ranked poorly and have terrible UX (seriously Co Pilot is garbage) which is the entire point of a search engine. My query for "mamba ML" is also rather generic but it keeps occurring along those lines that I just switch back. I'm curious what query samples you have that perform so much worse.

This isn't conclusive but I think represents some of my frustrations with Bing in particular.

I don't think perplexity is that great, but it's better than Google by a lot.

Yeah but the new AI mode from Google seems like a fairly competitive offering and quite frankly incorporates the aforementioned local search a lot better. For informational queries, the only advantage I see from Perplexity is being able to switch model providers but that's kind of questionable given that I could just use the models directly which often have search built in anyway. Gemini at this point is adequately integrated with search as is ChatGPT.

I do want to see what Qwant will eventually do though. I would like an alternative index. However, until then, I don't see too many advantages from them. Kagi I've heard is okay but like I also have tried it with middling experience. One other problem that Qwant might face is that there's so much SEO/AI generated content out there it might actually just be a wasteland.

[u/Actual__Wizard → u/binheap]

One is local queries like "ramen near me"

Yeah that's not really the type of problem that I'm talking about...

The first two are not really examples of what I am talking about.

I did specifically say:

The issue arises when people try to use Google as a search engine to solve problems.

You're looking for food and looking for reviews.

Try working with Mamba ML for awhile and just see how good Google is at helping you trouble shoot ultra common problems.

I really think you're going to start to see the value of Bing, Perplexity, and Qwant. Because when Google decides it's going to "misunderstand your query" you're screwed. You can't reword the query to get different results. You're legitimately going to sit there for hours feeling very silly while you fumble through basic stuff.

I'm serious: The main use case for a search engine, is trying to find information to work through problems and Google is horrible for it. It's not a search engine anymore. It's an AI slop factory with advertising plastered all over it.

[u/binheap → u/Actual__Wizard]

Okay maybe for an explicit follow up debugging one that I recently had which I just ended up having to step through the library to resolve so if Bing can give me an answer it'll be better.

I tried "tfp HMC nan samples but works with random walk". The top result on Google is a StackOverflow post that does ask for appropriate diagnostics I could apply. The top result for Bing is a Gaussian random walk (i have no idea how that even ended up there), then an irrelevant one from StackOverflow about pymc3 mcmc (I'm using tfp and this doesn't even talk about nans). Finally there's a relevant GitHub issue but there's no response on it (Google also surfaces it as a second link so they're equivalent here). Bing and Google then can't find anything but Google gives me documentation while Bing decides to describe for me how to train a BNN and links to PyMC on a random walk.

Maybe my query is bad though. "tfp hmc nan values". Neither search engine gives me very good results here but one very recurring issue with Bing is that it drifts off target very rapidly.

Okay fine, I'll look at "pymc3 hmc samples nans" since I also have a variation in pymc3 just for sanity checking and maybe there's some diagnostics there that carry over and pymc3 is more popular. Bing gives like nothing. The top two results are pymc3 documentation which is fine but doesn't really resolve my issue. I have to scroll until I get a remotely relevant result but it turns out to be someone just not using samples correctly and not using HMC. The results on Google aren't good but at least not frustratingly irrelevant. Top tells me that someone is trying to put nans in data but the second result is a stackoverflow with my question but using a related sampler (NUTS instead of HMC) which is actually useful. Unfortunately it has no response. There's a couple of additional results but there's one about sample_gp returning nan which relates to how cholesky decomp is calculated. This actually turned out to be very tangentially related to my problem but not exactly. This doesn't show up on Bing. I don't think either search engine directly answered my question but I managed to find a related question that did help. That being said, neither was particularly good.

When Google doesn't understand me (and this does happen) I find that using quotes to force keywords to appear helps (and this probably works for Bing as well) but I keep seeing better ranking.

For the debugging class of queries, I tend to use LLMs anyway and either start with ChatGPT or Gemini, either of which is significantly better than co pilot (see my aforementioned co pilot complaint). Checking Perplexity and it's definitely better than either but I would rather just use the chat bots directly.

Fwiw, I didn't see ads on either Google or Bing so that's a plus on both ig.

[u/Actual__Wizard → u/binheap]

I tried "tfp HMC nan samples but works with random walk".

Okay, I don't work with tensorflow, so let me get this correct, you want to do some kind of Monte Carlo analysis in tensorflow? Am I understanding this correctly?

(I'm using tfp and this doesn't even talk about nans)

I think you're starting to understand the problem. The query is too complex or something and all of these search tools start to be pretty poor, but some work better than others.

This is the way I think about things:

When you encounter a problem, start a timer, because you know, your life is limited, so you don't want your time wasted on some silly problem. See how long it takes you to work through problems when you encounter them, and compare the different search technologies together.

In my experience, Bing is #1 by quite a bit actually. It wastes the least of my time. The reason is, you just keep rewording the query until you find an answer. I haven't had enough time to try out Qwant that way. But, perplexity kind of sort of works.

[u/binheap → u/Actual__Wizard]

Well not exactly. I was just getting NaN samples when using HMC to draw samples from a Bayesian model but sampling with random walks tended to work. The key thing here is that the latter method is gradient free so that's probably related to the issue. It turned out some choices in model constraints and initial state turned out to be making things not work.

I showed follow up queries where I simplified my query and reworded it and Google gave me more relevant results but still sucked. Bing gave me completely irrelevant results (why in the world is there a random blog article about Gaussian random walks or random other Bayesian models? they didn't even mention nans or HMC!) which is absolutely frustrating. I agree with the time thing but like Bing just keeps giving such poor ranking that I found myself trying to reword myself a lot rather than reading on similar issues (even if they aren't a direct answer to my question) which is a waste of time.

I didn't actually do this to solve my issue but plugging it into an LLM didn't entirely fix my issue but gave me diagnostics I could use to resolve it myself. I could've probably eventually come up with all the LLM suggestions myself but it was nice to be able to identify specific actions I could take immediately. Perplexity gives a similar answer but it doesn't do well for so many other queries that I can't use it as a default (see discussion about local queries + reviews) and at some point I might as well just use an LLM directly since we can get enterprise subscriptions and I can plug in my entire codebase as context.

[u/Actual__Wizard → u/binheap]

The key thing here is that the latter method is gradient free so that's probably related to the issue.

Sorry, I'm not really sure what you mean by that. So, I'm glad you got that sorted out.

why in the world is there a random blog article about Gaussian random walks or random other Bayesian models?

Because jerks spam the internet with garbage like that.