Can anyone explain why on earth VC's are making actual investment decisions based on imaginary internet points? This would be like an NFL team drafting a quarterback based on how many instagram followers they have rather than a relevant metric like pass completion, or god forbid, doing some work and actually scouting candidates. Maybe the Cleveland Browns would do that[0], but it's not a way to mount a serious Super Bowl campaign[1].
Are VC's just that lazy about making investment decisions? Is this yet another side-effect of ZIRP[2] and too much money chasing a return? Is nobody looking too hard in the hope of catching the next rocket to the moon?
From the outside, investing based on GitHub stars seems insane. Like, this can't be a serious way of investing money. If you told me you were going to invest my money based on GitHub stars, I'd laugh, and then we'd have an awkward silence while I realize there isn't a punchline coming.
Social proof has always been a factor in investments. Not the only factor, but seeing signs of popularity has always been an input to investment decisions.
The entire game of startup investing is to identify breakout companies early. Social proof (when valid, not faked) of interest is one of the strongest signals of product market fit.
If a product has a lot of attention (users, headlines, stars, downloads, DAU) that’s a signal that it could also have a lot of customers some day. This is also why all of those metrics are targets for manipulation.
> This would be like an NFL team drafting a quarterback based on how many instagram followers they have
Major sports team are about engaging fans. If a promising recruit had a huge social media presence then that could be a contributing factor toward trying to recruit that player.
This is actually easier to understand if you look at the inverse: Some times there are players with amazing stats but who have a cloud of controversy following them. Teams will skip over these problematic players despite their performance because having popular and engaging players is important for teams but having anti-popular players will drive away fans.
> Can anyone explain why on earth VC's are making actual investment decisions based on imaginary internet points?
The answer is right there in front of your face. Say it with me: VCs are morons. VCs are morons. VCs are morons. Just because someone is rich, you think that means they have any clue what they're doing?
The answer isn't that they're morons. It's that they aren't people who "invest" in "good businesses" to make money, but instead on the whole a class of individuals classed with gambling on high risk ventures that will have absolutely massive returns and they don't care if 90% of them fail and 9% flounder because the 1% that succeed bring in absolutely apeshit amounts of $$ when they are acquired by someone else.
Using things like github stars is clearly stupid, but not in the way you're suggesting. They're using the GH stars as a proxy metric for "someone else will come along and give money bags to this person later, so I should get in early so I can take that money."
They're operating on metric of success which is about influence and charisma and connectedness, not revenue or technical excellence.
Again, VCs don't care if you'll make a profitable business some day. They're just interested in if someone else will come along and pay out giant bags of cash for it later in a liquidity event. If they get even one of those successes, all the stupid GH star watching pays off.
Here's another way of framing it: any harms from the false positives around "He has a lot of GH stars" or "He went to Stanford" or "I know his father at the country club" are more than mitigated by the one exit in 1000 that makes a bunch of people filthy rich.
We shouldn't expect VCs to be something they're not. But we are missing something inbetween VCs and "self financing" and "bootstrapping"
As far as I recall it stared in 2014 or so, yes metrics could/were still gamed, but there was still a belief in VC that OSS projects could turn into Red Hats. First I heard of it was when a VC told me they were "looking for the next docker" and mentioned something about Rancher OS and how quickly it's stars/follows were growing. In VC you tend to have conviction builders, and conviction buyers. I suppose what happened was some conviction builders used growth of a project on gh as part of a leading indicator (valid or otherwise), and conviction buyers picked up on that as a method.
I don't follow American Football so I don't know how coaching contracts work for you guys, but how does someone go 1W 15L one season, survive as head coach to go 0W 16L the next season, and still start the next season after that as head coach?
Over here the fans would be singing "You're getting sacked in the morning" halfway through that first season.
I guess not having relegation makes things slightly less ruthless for you.
Yeah, exactly. The NFL is a closed system franchise. The same 32 teams play every season whether they win or lose. No team risks relegation to a lower revenue league. Every team gets a roughly equal share of the franchise revenue regardless of performance.
The prevailing narrative here is that the team was actively looking to lose to acquire draft picks. Hugh Jackson was extremely good at losing, so he stayed.
The owner of the Cleveland Browns uses the team to generate more revenue. For NFL teams, performance has little to do with their value or ability to generate additional revenue.
There is no strong financial incentive to win in the NFL, aside from the owner's ego. The Browns' owner's ego is driven by money, and the result shows on the field.
In truth, I don't follow sports much, but I'm really not sure either.
I do find the model European Football (soccer) using promotion and relegation to be much more interesting, both from the standpoint of culling out perennially hopeless teams from top-tier competition, and for having a place for people to play who aren't absolute superstars.
Owners don’t care about winning, but about profitability. And you can make a lot of money with a failing football team (selling/trading draft picks, etc) and your fans get used to losing …
Right, I forgot you guys have "The Draft", so failing is an advantage, doubly so if you can sell your draft picks, because then you can keep losing by having sold away the mechanism for getting you competitive again.
I am so glad the proposed "European super league" was killed off so hard, so that we don't get a franchise model, it produces so many adverse incentives.
More to the point, in the US losing teams get rewarded in the form of draft picks, which sometimes creates perverse incentives. This doesn't exist in European football. (Disclaimer: I know almost nothing about American sports.)
just wait until you get to the subject of tanking in the NBA
is there a tech equivalent? like you do a crappy job with your series A on purpose which helps you get a better series B. although there is the notion of a big round of layoffs to secure further investment
Browns fan in. We're owned by a criminal (truck stop-related fraud) who was convinced by a homeless person to draft Johnny Manziel. trust me, we want to put him (and Paul Dolan) into graveyard orbit. but it's not like Vercel where you can just go use AWS or Cloudflare or whatever; and it's not like switching makes you weak, you stand by your team through the hard times!
plus, what is an NFL fan going to do, stop watching football? hahahahahaha
Hey, you can say that the Dolans should/could spend more, but I don't think you really want an owner who has solidified the team in Cleveland, has the fourth-best record in baseball over the past 10 years, and has seven recent playoff appearances in the graveyard.
The Haslams? Yeah, they should really sell the team, but I figure in about 10-15 years, they'll move it out of Cleveland.
Hiring a QB based off instagram followers isn't even that unrealistic. If you can put together a team to win the Superbowl, sure do it! If not, just get together a team that people enjoy following and watching. Much of that would be putting athletes on the field that people engage with.
This has happened in multiple industries a number of times - publishers discover that people with large twitter followings sell a decent number of books, so they start selecting new authors who only have large twitter followings, and discover is was correlation and not causation.
And once it gets out that it’s a selection criteria it gets gamed to hell and back.
I appreciate this, but the stakes have to be a lot lower bringing a book to market than making a $1,000,000 to $10,000,000 seed investment. I'd sort of expect that when you're dealing with sums of money that size there would be some grown-ups in the room.
When I took over MLAgents at the end of 2021, before Unity fully shit themselves after the insane Weta acquisition, the main metric they were using to promote the project internally were github stars
Yes actually
Needless to say they didn’t like when I said this was a worthless metric and we needed to be using something like “working policies” or “time saved training”
It never made sense. Weta tools don’t work with Unity at all
There were no complementary workflows or infrastructure or anything.
It was explicitly a move to try to counter epic’s positioning and internally it was very obviously a JR versus Tim pissing contest (and JR was the only one in the contest because Tim didn’t give a fuck about Unity)
I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would.
Here are the things I look at in order:
* last commit date. Newer is better
* age. old is best if still updating. New is not great but tolerable if commits aren't rapid
* issues. Not the count, mind you, just looking at them. How are they handled, what kind of issues are lingering open.
* some of the code. No one is evaluating all of the code of libraries they use. You can certainly check some!
What does stars tell me? They are an indirect variable caused by the above things (driving real engagement and third interest) or otherwise fraud. Only way to tell is to look at the things I listed anyway.
I always treated stars like a bookmark "I'll come back to this project" and never thought of it as a quality metric. Years ago when this problem first surfaced I was surprised (but should not have been in retrospect) they had become a substitute for quality.
I hope the FTC comes down hard on this.
Edit:
* commit history: just browse the history to see what's there. What kind of changes are made and at what cadence.
> If one library has 1,000 stars and the other has 15, I'm going to default to the 1,000 stars.
There are clearly inflection points where stars become useful, with "nobody has ever used this package" and "Meta/Alphabet pays to develop/maintain this package" on the two extremes.
I'm less sure what the signal says in-between those extremes. We have 2 packages, one has 5,000 stars, the other has 10,000 stars - what does this actually tell me, apart from how many times each has gone viral on HN?
I listed many other useful heuristics. Do you not find value in them? Do you find stars more valuable than them?
Take a moment to consider stars as a useful metric may only be useful for packages created prior to ~2015 when they weren't such a strong vanity metric, and are already very well established. This is preconditioning you to think "stars can still sometimes be useful, because I took a look at Facebook's React GH and it has a quarter million stars".
Sure, it's useful for that. But you aren't going to evaluate if the "React" package is safe. You already trivially know it is.
You'll be evaluating packages like "left-pad". Or any number of packages involved in the latest round of supply chain attacks.
For that matter, VCs are the ones stars are being targeted at, and potential employers (something this article doesn't cover, but some potential hires do hope to leverage on their resume).
If you are a VC, or an employer, it is a negative metric. If you are a dev evaluating packages, it is a vacuous metric that either tells you what you already know, or would be better answered looking at literally anything else within that repo.
The article also called out how download count can be faked trivially. I admit I have relied upon this in the past by mistake. Release frequency I do use as one metric.
When I care about making decisions for a system that will ingest 50k-250k TPS or need to respond in sub-second timings (systems I have worked on multiple times), you can bet "looking at stars" is a useless metric.
For personal projects, it is equally useless.
I care about how many tutorials are online. And today, I care more about if there was enough textual artifacts for the LLMs to usefully build it into their memory and to search on. I care if their docs are good so I spend less tokens burning through their codebase for APIs. I care if they resolve issues in a timely manner. I care if they have meaningful releases and not just garbage nothings every week.
I didn't mean for this to sound like a rant. But seriously, I just can't imagine in any scenario where "I look at stars" as a useful metric. You want to add it to the list? Sure. That is fine. But it should not be a deciding factor. I have chosen libraries with less stars because it had better metrics on things I cared about, and it was the correct choice (I ended up needing to evaluate them both anyhow. But I had my preference from the start).
Choosing the wrong package will waste you so much more time. Spend the 5 minutes evaluating for stuff that is important to your project.
I use stars to try and protect myself from dependency confusion attacks.
For example, let’s say I want to run some piece of software that I’ve heard about, and let’s say I trust that the software isn’t malware because of its reputation.
Most of the time, I’d be installing the software from somewhere that’s not GitHub. A lot of package managers will let anyone upload malware with a name that’s very similar to the software I’m looking for, designed to fool people like me. I need to defend against that. If I can find a GitHub repo that has a ton of stars, I can generally assume that it’s the software I’m looking for, and not a fake imitator, and I can therefore trust the installation instructions in its readme.
Except this is also not 100% safe, because as mentioned in TFA, stars can be bought.
Sure, I suppose that is one solution, but given that buying stars has been around for at least 5 years, and I have been aware of people faking stars for longer than that, I am not sure why you would rely on stars as a primary metric.
There are many other far more useful metrics to look at first, and to focus on first, and to think about. Every time you think about stars, you'll forget the other stuff, or discount it in favor of stars.
Forget stars. They now no longer mean anything. Even if they did before, they don't anymore.
That’s not something I wanted to imply. It can also stand for "the fine article". Is there a better shorthand for "the article linked at top of the page" / "the original article"?
You call these baubles, well, it is with baubles that men are led... Do you think that you would be able to make men fight by reasoning? Never. That is only good for the scholar in his study. The soldier needs glory, distinctions, and rewards.
But to someone else, it is a meaningful metric that you bookmarked something. It doesn't matter that the star isn't you saying you liked something. It's already telling enough merely that you wanted to bookmark it.
It's only not meaningful because of how other people can game it and fabricate it, but everything you just said, if it was only people like you, that would be a very meaningful number.
It doesn't even matter why you bookmarked it, and it doesn't matter that whatever the reason was, it doesn't prove the project as a whole is overall good or useful. Maybe you bookmarked it because you hate it and you want to keep track of it for reference in your ted talk about examples of all the worst stuff you hate, but really by the numbers adding up everyone's bookmarks, the more likely is that you found something interesting. It doesn't even matter what was interesting or why. The entire project could be worthless and the thing you're bookmarking was nothing more than some markdown trick in the readme. That's fine. That counts. Or it's all terrible, not a single thing of value, and the only reason to bookmark it is because it's the only thing that turned up in a search. Even that counts, because that still shows they tried to work on something no one else even tried to work on.
It's like, it doesn't matter how little a given star means, it still does mean something, and the aggregation does actually mean something, except for the fact of fakes.
Yes...which is why I said it is an indirect variable, as caused by the other things I pointed out above. Age, quality, code, utility, whether issues are addressed, interest, etc. Or fraud. Pretty cut and dry.
FWIW, I almost never star repos. Even ones I use or like. I don't see the utility for myself.
Aim for a more concise post and don't couch your statements in doubt next time if you want a productive conversation, because I don't know what you are trying to say.
These kinds of articles make you feel like there are specific, actionable problems that just need an adjustment and then they disappear. However, the system is much worse than you'd expect. Studies like this are extremely valuable, but they don't address the systematic problems affecting all signaling channels: most signals themselves have been manufactured into a product.
Build a SaaS and you'll have "journalists" asking if they can include you in their new "Top [your category] Apps in [current year]", you just have to pay $5k for first place, $3k for second, and so on (with a promotional discount for first place, since it's your first interaction).
You'll get "promoters" offering to grow your social media following, which is one reason companies may not even realize that some of their own top accounts and GitHub stars are mostly bots.
You'll get "talent scouts" claiming they can find you experts exactly in your niche, but in practice they just scrape and spam profiles with matching keywords on platforms like LinkedIn once you show interest, while simultaneously telling candidates that they work with companies that want them.
And in hiring, you'll see candidates sitting in interview farms quite clearly in East Asia, connecting through Washington D.C. IPs, present themselves with generic European names, with synthetic camera backgrounds, who somehow ace every question, and list experience with every technology you mention in the job post in their CVs already (not hyperbole, I've seen exactly this happen).
If a metric or signal matters, there is already an ecosystem built to fake it, and faking it starts to be operational and just another part of doing business.
Short term, you pay the cost of fake signaling, which is simply deadweight loss. People spend resources to inflate signals instead of improving the actual thing.
Medium term, I suppose you could see how it increases consumption, since users would probably try something with 100k stars instead of 2, GitHub wants to seem that it's used more than it really is, repo owner is also benefiting.
Long term, the correspondence between how important a (distorted) system is perceived (Github, OSS, IT in general) vs how important it really is collapses quite abruptly and unnecessarily, and you end up with a lemon market [0] where signals stop being reliable at all.
The spoilage by money is half right, but I think the more interesting part is where the money ends up and how that influences the system.
I'm increasingly convinced the issue isn't feedback itself, but centralized, global, aggregated feedback that becomes game-able without stronger identity signals.
Right now the incentives are tied (correctly or not) to these global metrics, so you get a market for faking them, with money flowing to whoever is best at juicing that signal.
If instead the signal was based on actual usage and attributions by actual developers, the incentives shift. With localized insight (think "Yeah, I like Golang") it becomes both harder to fake and harder to get at the metric rollup.
Useful reputation on the web is actually much more localized and personal. I gladly receive updates on and would support the repos I've starred. If I could chose where to put my dollars (not an investors), it would likely include the list of repos I've personally curated.
This suggests a different direction: instead of asking "how many stars does this have?", ask "who is actually depending on this, and in what context?" or better retroactively compare your top-n repos to mine and we'll get a metric seen through our lenses. If you want to include everyone in that aggregation you'll end up where we are now, but if in stead you chose the list, well, the stars could align as a good metric once more.
The interesting part is that the web already contains most of that information, we just don't treat identity as a part of the signal (yet? universally?).
Tangential but I have more than 5k repos starred (according to my GitHub profile) organized into lists but the way I discover interesting stuff on GitHub these days is through people I follow. Follow interesting people, find interesting stuff. Sometimes it's that easy.
What's more it became obvious to me two or so years ago that GitHub is going the way of LinkedIn slowly but surely. Lots of professionals on there just because it's expected of them, some interact occasionally with the "social media" aspect of it and fewer still really thrive on that part. Time will tell how this will pan out but just look how many Developer and Linux influencers became huge on YouTube and other places this last year. Most of them barely had more than 10k subscribers 3 years ago and now people look to them for their next tech stack and hot framework/tool/library/distro and so on.
At the end it's a company choice: do you buy BS metrics or you don't.
We've recently decided to complicate life of AI bots in our repo https://archestra.ai/blog/only-responsible-ai, hoping they will just choose those AI startups who are easier to engage with.
A way for what? Evolution does not have a will, does not work for anybody in particular, and it's statistically likely that it will work against you, as almost all species that ever were went extinct and almost no individual from just 15 generations ago has left DNA to present people. [0] [1]
"We ran our own analysis sampling 150 profiles per repo across 20 projects and found repos where 36-76% of stargazers have zero followers and fork-to-star ratios 10x below organic baselines"
This does not looks like appropriate signal to use on github, i doubt that this is organic baseline.If this is used as metric than study might be flawed.
I run a tiny site that basically gave a point-at-able definition to an existing adhoc standard. As part of the effort I have a list of software and libraries following the standard on the homepage. Initially I would accept just about anything but as the list grew I started wanting to set a sort of notability baseline.
Specifically someone submitted a library that was only several days old, clearly entirely AI generated, and not particularly well built.
I noted my concerns with listing said library in my reply declining to do so, among them that it had "zero stars". The author was very aggressive and in his rant of a reply asked how many stars he needed. I declined to answer, that's not how this works. Stars are a consideration, not the be all end all.
You need real world users and more importantly real notability. Not stars. The stars are irrelevant.
This conversation happened on GitHub and since then I have had other developers wander into that conversation and demand I set a star count definition for my "vague notability requirement". I'm not going to, it's intentionally vague. When a metric becomes a target it ceases to be a good metric as they say.
I don't want the page to get overly long, and if I just listed everything with X star count I'd certainly list some sort of malware.
I am under no obligation to list your library. Stop being rude.
Nice to know the name for this — Goodhart's Law. And I think the core reason is that the cost to fake these metrics is far less than what they claim to represent. Stars, reviews, ratings, trading volumes — all cheap to manufacture, and only getting cheaper with AI.
I've been thinking about this a lot. These metrics are all just marketing signals to draw people's attention, trying to make some kind of deals. So the fix should be: make the cost of the signal match what it claims to represent. I'm obsessed with something called DUKI /djuːki/ (Decentralized Universal Kindness Income, a form of UBI) — the idea is that instead of stars or reviews, trust comes from deals pledging real money to the world for all as the deal happens. You can't fake that cheaply.
So the metric becomes the money itself — if you fake X amount, it costs you X, and the world will thank you by paying attention...
Imagine if GitHub let you back a star with real money — the more you put in, the more credible the star. And that money goes out as UBI for everyone. For attention makers, star anything you want, as much as you want. For attention takers, just follow the money to filter through all the noise that's so easy to manipulate...
Very simply, you need to see VCs as branding companies who give people with many followers brand deals.
VCs think that if you have a lot of stars, you must have hit product-market fit — or something close — because many developers have started using your open-source tool. This isn’t necessarily always the case.
Every weekend project from Andrej Karpathy gets loads of stars because he is the most famous person on GitHub.
What I’ve noticed a lot is that the repos with the most stars most of the time already came from big companies open-sourcing their tools, or people building free versions of paid software.
I think people expect the star system to be a cheap proxy for "this is a reliable piece of sorfware which has a good quality and a lot of eyes".
I think as a proxy it fails completely: astroturfing aside stars don't guarantee popularity (and I bet the correlation is very weak, a lot of very fundamental system libraries have small number of stars). Stars also don't guarantee the quality.
And given that you can read the code, stars seem to be a completely pointless proxy. I'm teaching myself to skip the stars and skim through the code and evaluate the quality of both architecture and implementation. And I found that quite a few times I prefer a less-"starry" alternative after looking directly at the repo content.
given that you can read the code, stars seem to be a completely pointless proxy
Imagine you're choosing between 3 different alternatives, and each is 100,000 LOC. Is 'reading the code' really an option? You need a proxy.
Stars isn't a good one because it's an untrusted source. Something like a referral would be much better, but in a space where your network doesn't have much knowledge a proxy like stars is the only option.
> Is 'reading the code' really an option? You need a proxy.
100k is small, but you're right, it can be millions. I usually skim through the code tho, and it's not that hard. I don't need to fully read and understand the code.
What I look at is: high-level architecture (is there any, is it modular or one big lump of code, how modular it is, what kind of modules and components it has and how they interact), code quality (structuring, naming, aesthetics), bus factor (how many people contribute and understand the code base).
Honest question: how can VCs consider the 'star' system reliable? Users who add stars often stop following the project, so poorly maintained projects can have many stars but are effectively outdated.
A better system, but certainly not the best, would be to look at how much "life" issues have, opening, closing (not automatic), and response times.
My project has 200 stars, and I struggle like crazy to update regularly without simple version bumps.
The stars have fallen to the classic problem of becoming a goal and stopping being a good metric. This can apply to your measure just as well: issues can also be gamed to be opened, closed and responded to quickly, especially now with LLMs.
Was it ever a good metric? A star from another account costs nothing and conveys nothing about the sincerity, knowledge, importance or cultural weight of the star giver. As a signal it's as weak as 'hitting that like button'.
If the number of stars are in the thousands, tens of thousands, or hundreds of thousands, that might correlate with a serious project. But that should be visible by real, costly activity such as issues, PRs, discussion and activity.
There isn't just "good metric" in vacuum - it was a good metric of exactly the popularity that you mentioned. But stars becoming an object of desire is what killed it for that purpose. Perhaps now they are a "good metric" of combined interest and investment in the project, but what they're measuring is just not useful anymore.
Yeah, I'd agree with this. I always thought of a star indicating only that a person (or account, generally) had an active interest in another project, either through being directly related or just from curiosity. Which can sort of work as a proxy for interesting, important or active, but not accurately.
A repository with zero stars has essentially no users. A repository with single-stars has a few users, but possibly most/all are personal acquiantances of the author, or members of the project.
It is the meaning of having dozens or hundreds of stars that is undermined by the practice described at the linked post.
Sometimes people open issues without proper information. It cant be replicated and nobody else is jumping in that it affects them. You may suspect its something else, maybe with their environment, but if they don't engage what else can you do? Tell them you are closing it and specify what kind of info you need if they ever get around to providing it and it can be reopened.
And sometimes the maintainer simply doesn't respond to a perfectly acceptable issue due to either the maintainer abandoning the project, not enough maintainers or simple neglect.
I wonder if there's a more graph oriented score that could work well here - something pagerank ish so that a repo scores better if it has issues reported by users who themselves have a good score. So it's at least a little resilient to crude manipulation attempts
GitHub has all kinds of private internal metrics that could update the system to show a much higher signal/quality score. A score that is impervious to manipulation. And extremely well correlated with actual quality and popularity and value, not noise.
Two projects could look exactly the same from visible metrics, and one is complete shell and the other a great project.
But they choose not to publish it.
And those same private signals more effectively spot the signal-rich stargazers than PageRank.
It would be more resilient indeed, I think. Definitely needs a way to figure out which users should have a good score, though - otherwise it's just shifting the problem somewhat. Perhaps it could be done with a reputation type of approach, where the initial reputation would be driven by a pool of "trusted" open source contributors from some major projects.
That said, I believe the core problem is that GitHub belongs to Microsoft, and so it will still go more towards operating like a social network than not - i.e. engagement matters. It will still take a good will to get rid of Social Network Disease at scale.
You are looking for different things to VCs. You are looking for markers that show software quality over the long-term. They are looking for markers that show rapidly gaining momentum over the short-term. These are often in opposition to one another.
Making the conversations about VCs expecting thousands of stars, is thinking too big. It's probably more often someone pays $20 to make one of their projects look good, for their CV, for vanity, thinking this will get them the push that they need to get clicks on reddit, or noticed over some other open source project. If there is someone offering a 10k star project an investment over 8k without looking at the project or revenue potential, I can only think they are clueless, or picking a student project to fund each summer.
The fake accounts often star my old repos to look like real users. They are usually very sketchy if you think for a minute, for example starring 5,000 projects in a month and no other GitHub activity. One time I found a GitHub Sponsor ring, which must be a money laundering / stolen credit cards thing?
Agree that sophisticated funds don't, but the ecosystem hasn't caught up. StarHub/GitStar pricing pages still sell to "seed-stage founders pre-fundraise"
Much more important is who starred it. And are they selective about giving out stars or bookmarking everything. Forks is a closer signal to usage than stargazing.
Indeed, GitHub should set up a monthly quota for available stars to give and correlate the account age with it: either make something up like a "trusted-age-factor" that multiplies any given star by that factor, or scale the available quota accordingly by that factor (and let users star repos repeatedly).
GitHub should also introduce a way to bookmark a repo, additional to the existing options of sponsor/watch/fork/star-ing it.
Many VCs are only doing one thing: how to use some magical quantitative metrics to assess whether a project is reliable without knowing the know-how. Numbers are always better than no numbers.
When a partner decides to recommend a startup to the investment committee, he needs some explicit reasons to convince the committee, not some kind of implicit vibe
Yes, I think VCs have already switched to using other metrics that are less easy to fake, such as download per month or customer interviews (or more direct, ARR, even for really early stage startups). I just want to explain the background reason of it.
This is a good idea, but from my experience most VCs (I’m not talking about the big leagues) aren’t technical, they tend to repeat buzzwords, so they don’t really understand how star systems works.
Because VCs love quantifiable metrics regardless of how reliable they actually are. They raise money from outside investors and are under pressure to deploy it. The metrics give them something concrete to justify their thesis and move on with their life.
>Honest question: how can VCs consider the 'star' system reliable?
Founders need the ability to get traction, so if a VC gets a pitch and the project's repo has 0 stars, that's a strong signal that this specific team is just not able to put themselves out there, or that what they're making doesn't resonate with anyone.
When I mentioned that a small feature I shared got 3k views when I just mentioned it on Reddit, then investors' ears perked right up and I bet you're thinking "I wonder what that is, I'd like to see that!" People like to see things that are popular.
By the way, congrats on 200 stars on your project, I think that is definitely a solid indicator of interest and quality, and I doubt investors would ignore it.
Yeah, I was pondering a few months away when I checked Pathway, an ETL solution.
I've never heard about it but I saw some news that they have created a better model than transformer. So stats:
> VC funding pipeline that treats GitHub popularity as proof of traction
Why am I not surprised big Capital corrupts everything. Also, Goodhart's law applies again: "When a measure becomes a target, it ceases to be a good measure".
HN Folks: What reliant, diverse signals do you use to quickly eval a repo's quality?
For me it is: Maintenance status, age, elegance of API and maybe commit history.
PS: From the article:
> instead tracks unique monthly contributor activity - anyone who created an issue, comment, PR, or commit. Fewer than 5% of top 10,000 projects ever exceeded 250 monthly contributors; only 2% sustained it across six months.
> [...] recommends five metrics that correlate with real adoption: package downloads, issue quality (production edge cases from real users), contributor retention (time to second PR), community discussion depth, and usage telemetry.
Overly verbose and "glitter" readme.md files is a good indicator of bad projects or at least projects which need more attention to be used. It's too often pre-rugpull or look-at-me repos where better solutions are one click away.
Finding any curse words in hidden comments in the commit history is for me a good indication of a human working on a passion project, though ymmv.
And there are always exceptions to the exception of the exceptions.
I tend to look at other people involved, like contributors but not just the volume but actual people and their other activity. If the original author is still around and active that tends to be a good sign IME
I wonder if it makes sense for GitHub to use graph-theoretic measures like PageRank instead of raw stars. In simple terms, a repo is considered important if it is starred or forked by GitHub users who maintain other important repos.
It’s more expensive to compute, but the resulting scores would be more trustworthy unless I’m missing something.
That sounds closer to achieving a good outcome. Of course I think anything that includes the set of all users as columns will be game-able. You need to either choose the set yourself from "trusted peers" or "foaf" degrees, or maybe better use retroactive signals rather than purely like-driven approaches.
In my opinion, nothing could be more wrong. GitHub's own ratings are easily manipulated and measure not necessarily the quality of the project itself, but rather its Popularity. The problem is that popularity is rarely directly proportional to the quality of the project itself.
I'm building a product and I'm seeing what important is the distribution and comunication instead of the development it self.
Unfortunately, a project's popularity is often directly proportional to the communication "built" around it and inversely proportional to its actual quality. This isn't always the case, but it often is.
Moreover, adopting effective and objective project evaluation tools is quite expensive for VCs.
I've seen the same devs refuse to use a library because the last commit was 3 months ago, despite the library being extremely popular, battle tested, and existing for 10 years.
> measure not necessarily the quality of the project itself, but rather its Popularity
Surely a project's popularity is often related to its utility. A useful and popular project seems like exactly the kind of thing a VC might be interested in.
* https://arxiv.org/abs/2412.13459 (2024/2025) - Six Million (Suspected) Fake Stars in GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware
I was literally was just looking at GitHub dataset availability and musing on this. A star from karpathy is worth a lot more than a star from open_claw_dood that just created his account 5 min ago.
In general, I’ve been dissatisfied with GitHub’s code search. It would be nice to see innovation here.
> I look at the starts when choosing dependencies, it's a first filter for sure.
Unfortunately I still look at them, too, out of habit: The project or repo's star count _was_ a first filter in the past, and we must keep in mind it no longer is.
> Good reminder that everything gets gamed given the incentives.
Also known as Goodhart's law [1]: "When a measure becomes a target, it ceases to be a good measure".
Essentially, VCs screwed this one up for the rest of us, I think?
> The project or repo's star count _was_ a first filter in the past, a
I agree that it has been a first filter, but should it ever have been? A star only says that someone had a passing interest in a project. Not significantly different from a 'like' on a social media post.
> The project or repo's star count _was_ a first filter in the past, and we must keep in mind it no longer is.
Id suggest the first question to ask is "if the project is an AI project or not?" If it is, dont pay attention to the stars - if it's not, use the stars as a first filter. That's the way I analyse projects on Github now.
Social media platforms (like Instagram) have always had this problem of "buying" followers. There was an article some time ago where hollywood types would only give roles to people with high followers so people started buying followers.
Now that money is flowing to Github stars, no wonder people are buying fake "stars"? Seems capitalism is working as expected...
Steam wishlist, itch.io number of views, YouTube views and now GitHub stars… I’m tired of all the gamification of creativity. Now if you’d upvote my comment I can get same karma please and thank you
Who knows? The way things are going VCs could start asking claude to review someone’s online presence and Claude might decide karma is the best metric for that
Personally I use stars in two ways: 1) It's interesting and I want to keep track of it for possible future use and 2) It's a fantastic idea and kudos to you even if I'll never use it.
As a side note it's kind of disheartening that everytime there is a metric related to popularity there would be some among us that will try to game it for profit, basically to manipulate our natural bias.
As a side note it's always a bit sad how the parasocial nature of the modern web make us like machine interfacing via simple widgets, becoming mechanical robot ourselves rationalising IO via simple metrics kind of forgetting that the map is never the territory.
For all the hate on star systems, whether it’s GitHub/Amazon/AppStore, I’d still take having one over having nothing at all.
They make it easier to sort through options, help with search and discovery, and at least give you a baseline signal for trust can get better over time.
So to me, some signal better than no signal at all.
I usually use stars as a bookmark list to visit later (which I rarely do). I probably would need to stop doing that and use my self-hosted "Karkeep" instance for github projects as well.
My project https://github.com/socketCluster/socketcluster has been accumulating stars slowly but steadily over about 13 years. Now it has over 6k stars but it doesn't seem to mean much nowadays as a metric. It sucks having put in the effort and seeing it get lost in a sea of scams and seeing people doubting my project's own authenticity.
It does feel like everything is a scam nowadays though. All the numbers seem fake; whether it's number of users, number of likes, number of stars, amount of money, number of re-tweets, number of shares issued, market cap... Maybe it's time we focus on qualitative metrics instead?
I came across one of these in 2018 with a "hot" open source company raising a Series B. An impressive star ramp (about 300% YoY growth) before the (high-priced/competitive) raise and three months later Github had revoked almost all the star growth from the previous year, resulting in a 20% YoY record. The company eventually got acquihired.
Yeah, I didn't think anyone would place any actual value on the stars. It almost doesn't need to be a feature, because what is it suppose to do exactly?
Github could easily crack down on this. Spend $10 at each star provider, then ban all accounts involved. A tiny bit of money could create a huge drag on the ecosystem.
I think the reason is that investors are not IT experts and don't know better metrics to evaluate.
I guess it's like fake followers on other social media platforms.
To me, it just reflects a behaviour that is typical of humans: in many situations, we make decisions in fields we don't understand, so we evaluate things poorly.
I don't know what is more, for lack of a better word, pathetic, buying stars/upvotes/platform equivalent or thinking of oneself as a serious investor and using something like that as a metric guiding your decision making process.
I'd give a lot of credit to Microsoft and the Github team if they went on a major ban/star removal wave of affected repos, akin to how Valve occasionally does a major sweep across CSGO2 banning verified cheaters.
The problem is that if this is the game now, you need to play it. I'm trying to get a new open source project off the ground and now I wonder if I need to buy fake stars.
Or buy the cheapest kind of fake stars for my competitors so they get deleted.
For Microsoft this is another kind of sunk cost, so idk how much incentive they have to fix this situation.
The issue with that is, it's a game that never ends. Now you need to inflate your npm/brew/dnf installs, then your website traffic to not make it to obvious, etc.
I am not successful at all with my current projects (admittedly am not trying to be nowadays), so feel free to dismiss this advice that predates a time before LLM driven development, but in the past, I have had decent success in forums interacting with those with a specific problem my project did address. Less in stars, more in actual exchange of helpful contributions.
Haha, have you tried that? I think in this day and age marketing is much needed activity even for open-source projects providing quality solutions to problems.
I maintain a niche-popular project that I didn't do any marketing for. My understanding is that even for popular projects, the usual dynamic is that there's just one guy doing all the work. So "getting off the ground" just means getting people to use it, and there shouldn't be any reason to artificially force that.
It depends what your objective is. Many people seem to see their open source projects as a stepping stone into some commercial activity. Putting aside whether that is a good idea or not if that is what they want to do then they will need to market in some way.
Honest question, which companies handle the process better given it is a trade-off? Yes, VAC is not as iron-clad as kernel level solutions can be, but the latter is overly invasive for many users. I'd argue neither is the objectively right or better approach here and Valves approach of longer term data collection and working on ML solutions that have the potential to catch even those cheating methods currently able to bypass kernel level anti-cheat is a good step.
On Github stars, I'd argue they are the most suitable comparison, as all the funny business regarding stars should be, if at all, detectable by Github directly and ideally, bans would have the biggest deterrent effect, if they happened in larger waves, allowing the community to see who did engage in fraudulent behaviour.
Am very much the same, took a bunch private two years ago for multitude of reasons. I can, however, see why no public repos could be a partial indicator and of concern, in conjunction with sudden star growth, simply because it is hard for a person with no prior project to suddenly and publicly strike gold. Even on Youtube it is a rare treat to stumble across a well made video by a small channel and without algos to surface repos on Github in the same way, any viral success from a previously inactive account should be treated with some suspicion. Same the other way, if you never made any PR, etc. sudden engagement is a bit odd.
Can now someone make analysis of Google Ads fake economy? I'm convinced that the data they share - specifically clicks - is false, and potentially they're paying some fraction to people to click it.
So, if star to fork ratio is the new signal, time to make an extra fake star tier, where the bot forks the repo, generates a commit with the cheapest LLM available and pushes that to gh, right?
Seen this happen first-hand with mid-to-large open source projects that sometimes "sponsor" hackathons, literally setting a task to "star the repo" to be eligible.
It’s supposed to get people to actually try your product. If they like it, they star it. Simple.
At that point, forcing the action just inflates numbers and strips them of any meaning.
Gaming stars to set it as a positive signal for the product to showcase is just SHIT.
Bots are killing opensource, but they pump product metrics so nobody cares. I maintain an open source repo and we've made a decision to limit all bot activity, even if it makes us less sexy in front of VCs.
We figured out a workaround to limit activity to prior contributors only, and add a CI job that pushes a coauthored commit after passing captcha on our website. It cut the AI slop by 90%. Full write-up https://archestra.ai/blog/only-responsible-ai
This is why people shouls use gitea. I dont understand why people keep using github at this point. Its not like theyve stolen all our data or anything:))
> Jordan Segall, Partner at Redpoint Ventures, published an analysis of 80 developer tool companies showing that the median GitHub star count at seed financing was 2,850 and at Series A was 4,980. He confirmed: "Many VCs write internal scraping programs to identify fast growing github projects for sourcing, and the most common metric they look toward is stars."
> Runa Capital publishes the ROSS (Runa Open Source Startup) Index quarterly, ranking the 20 fastest-growing open-source startups by GitHub star growth rate. Per TechCrunch, 68% of ROSS Index startups that attracted investment did so at seed stage, with $169 million raised across tracked rounds. GitHub itself, through its GitHub Fund partnership with M12 (Microsoft's VC arm), commits $10 million annually to invest in 8-10 open-source companies at pre-seed/seed stages based partly on platform traction.
This all smells like BS. If you are going to do an analysis you need to do some sound maths on amount of investment a project gets in relation to github starts.
All this says is stars are considered is some ways, which is very far from saying that you get the fake stars and then you have investment.
This smells like bait for hating on people that get investment
Who ever thought that GitHub stars were a legitimate measure of a project's popularity does not understand Goodhart's Law and such metrics were easily abused, faked, gamed and manipulated.
I asked Claude for an analysis on the maturity of various open source projects accomplishing the same thing. Its first searches were for GitHub star counts for each project. I was appalled at how dumb an approach that was and mortified at how many people must be espousing that equivocation online to make the training jump to that.
> The CMU researchers recommended GitHub adopt a weighted popularity metric based on network centrality rather than raw star counts. A change that would structurally undermine the fake star economy. GitHub has not implemented it.
> As one commenter put it: "You can fake a star count, but you can't fake a bug fix that saves someone's weekend."
I'm curious what the research says here---can you actually structurally undermine the gamification of social influence scores? And I'm pretty sure fake bugfixes are almost trivial to generate by LLMs.
that's exactly the next-round attack. StarScout's network-centrality defense works for the current generation of campaigns but won't survive LLM-generated PR/commit patterns
I’d say those CMU researchers are out of touch with the reality. GitHub can easily overhaul this with a much better system than what those researchers recommended but chooses not to.
“gstack is not a hypothetical. It’s a product with real users:
75,000+ GitHub stars in 5 weeks
14,965 unique installations (opt-in telemetry, so real number is at least 2x higher)
305,309 skill invocations recorded since January 2026
~7,000 weekly active users at peak”
GitHub stars are a meaningless metric but I don’t think a high star count necessarily indicates bought stars. I don’t think Garry is buying stars for his project.
People star things because they want to be seen as part of the in-crowd, who knows about this magical futuristic technology, not because they care to use it.
Some companies are buying stars, sure, but the methodology for identifying it in this article is bad.
I got gently admonished on here a while back for mentioning that I find those star graph things people put on their READMEs to have entirely the opposite effect than that which was intended. I see one of those and I'm considerably less likely to trust the project because a) you're chasing a stupider metric than lines of code, and b) people obviously buy stars.
The fake star problem is a symptom of a deeper issue — developers can't tell signal from noise in the agent ecosystem. The tools that actually get real adoption are the ones that solve acute production problems. Agents are hitting these in production issues of state management every day and there's almost no tooling for it. That's where genuine organic stars come from — solving a real pain, not gaming rankings
Can anyone explain why on earth VC's are making actual investment decisions based on imaginary internet points? This would be like an NFL team drafting a quarterback based on how many instagram followers they have rather than a relevant metric like pass completion, or god forbid, doing some work and actually scouting candidates. Maybe the Cleveland Browns would do that[0], but it's not a way to mount a serious Super Bowl campaign[1].
Are VC's just that lazy about making investment decisions? Is this yet another side-effect of ZIRP[2] and too much money chasing a return? Is nobody looking too hard in the hope of catching the next rocket to the moon?
From the outside, investing based on GitHub stars seems insane. Like, this can't be a serious way of investing money. If you told me you were going to invest my money based on GitHub stars, I'd laugh, and then we'd have an awkward silence while I realize there isn't a punchline coming.
[0] I'm from Cleveland. I get to pick on them.
[1] https://en.wikipedia.org/wiki/List_of_Cleveland_Browns_seaso... I think their record speaks for itself.
[2] https://en.wikipedia.org/wiki/Zero_interest-rate_policy
Social proof has always been a factor in investments. Not the only factor, but seeing signs of popularity has always been an input to investment decisions.
The entire game of startup investing is to identify breakout companies early. Social proof (when valid, not faked) of interest is one of the strongest signals of product market fit.
If a product has a lot of attention (users, headlines, stars, downloads, DAU) that’s a signal that it could also have a lot of customers some day. This is also why all of those metrics are targets for manipulation.
> This would be like an NFL team drafting a quarterback based on how many instagram followers they have
Major sports team are about engaging fans. If a promising recruit had a huge social media presence then that could be a contributing factor toward trying to recruit that player.
This is actually easier to understand if you look at the inverse: Some times there are players with amazing stats but who have a cloud of controversy following them. Teams will skip over these problematic players despite their performance because having popular and engaging players is important for teams but having anti-popular players will drive away fans.
> Can anyone explain why on earth VC's are making actual investment decisions based on imaginary internet points?
The answer is right there in front of your face. Say it with me: VCs are morons. VCs are morons. VCs are morons. Just because someone is rich, you think that means they have any clue what they're doing?
Listening to All In is a real eye opening experience. Especially when they have guests on and they're exactly like the regulars.
I have zero respect for All In. It’s a shame people pay those guys any mind.
VCs are, traditionally, people who made a lot of money in a lottery and think that makes them experts. It's virtually guaranteed they're idiots.
hey shhh the ycomb gods might hear us
The answer isn't that they're morons. It's that they aren't people who "invest" in "good businesses" to make money, but instead on the whole a class of individuals classed with gambling on high risk ventures that will have absolutely massive returns and they don't care if 90% of them fail and 9% flounder because the 1% that succeed bring in absolutely apeshit amounts of $$ when they are acquired by someone else.
Using things like github stars is clearly stupid, but not in the way you're suggesting. They're using the GH stars as a proxy metric for "someone else will come along and give money bags to this person later, so I should get in early so I can take that money."
They're operating on metric of success which is about influence and charisma and connectedness, not revenue or technical excellence.
Again, VCs don't care if you'll make a profitable business some day. They're just interested in if someone else will come along and pay out giant bags of cash for it later in a liquidity event. If they get even one of those successes, all the stupid GH star watching pays off.
Here's another way of framing it: any harms from the false positives around "He has a lot of GH stars" or "He went to Stanford" or "I know his father at the country club" are more than mitigated by the one exit in 1000 that makes a bunch of people filthy rich.
We shouldn't expect VCs to be something they're not. But we are missing something inbetween VCs and "self financing" and "bootstrapping"
As far as I recall it stared in 2014 or so, yes metrics could/were still gamed, but there was still a belief in VC that OSS projects could turn into Red Hats. First I heard of it was when a VC told me they were "looking for the next docker" and mentioned something about Rancher OS and how quickly it's stars/follows were growing. In VC you tend to have conviction builders, and conviction buyers. I suppose what happened was some conviction builders used growth of a project on gh as part of a leading indicator (valid or otherwise), and conviction buyers picked up on that as a method.
I don't follow American Football so I don't know how coaching contracts work for you guys, but how does someone go 1W 15L one season, survive as head coach to go 0W 16L the next season, and still start the next season after that as head coach?
Over here the fans would be singing "You're getting sacked in the morning" halfway through that first season.
I guess not having relegation makes things slightly less ruthless for you.
Yeah, exactly. The NFL is a closed system franchise. The same 32 teams play every season whether they win or lose. No team risks relegation to a lower revenue league. Every team gets a roughly equal share of the franchise revenue regardless of performance.
The prevailing narrative here is that the team was actively looking to lose to acquire draft picks. Hugh Jackson was extremely good at losing, so he stayed.
The owner of the Cleveland Browns uses the team to generate more revenue. For NFL teams, performance has little to do with their value or ability to generate additional revenue.
There is no strong financial incentive to win in the NFL, aside from the owner's ego. The Browns' owner's ego is driven by money, and the result shows on the field.
In truth, I don't follow sports much, but I'm really not sure either.
I do find the model European Football (soccer) using promotion and relegation to be much more interesting, both from the standpoint of culling out perennially hopeless teams from top-tier competition, and for having a place for people to play who aren't absolute superstars.
Owners don’t care about winning, but about profitability. And you can make a lot of money with a failing football team (selling/trading draft picks, etc) and your fans get used to losing …
Right, I forgot you guys have "The Draft", so failing is an advantage, doubly so if you can sell your draft picks, because then you can keep losing by having sold away the mechanism for getting you competitive again.
I am so glad the proposed "European super league" was killed off so hard, so that we don't get a franchise model, it produces so many adverse incentives.
The thing I like about EU football is that if your team sucks arse through a garden hose too long, your entire team gets demoted to a lower league.
That would put a fire under some asses!
More to the point, in the US losing teams get rewarded in the form of draft picks, which sometimes creates perverse incentives. This doesn't exist in European football. (Disclaimer: I know almost nothing about American sports.)
Draft picks + salary caps and the various workarounds involved there make it more of a financier's dream than a competitive sport.
just wait until you get to the subject of tanking in the NBA
is there a tech equivalent? like you do a crappy job with your series A on purpose which helps you get a better series B. although there is the notion of a big round of layoffs to secure further investment
Browns fan in. We're owned by a criminal (truck stop-related fraud) who was convinced by a homeless person to draft Johnny Manziel. trust me, we want to put him (and Paul Dolan) into graveyard orbit. but it's not like Vercel where you can just go use AWS or Cloudflare or whatever; and it's not like switching makes you weak, you stand by your team through the hard times!
plus, what is an NFL fan going to do, stop watching football? hahahahahaha
Hey, you can say that the Dolans should/could spend more, but I don't think you really want an owner who has solidified the team in Cleveland, has the fourth-best record in baseball over the past 10 years, and has seven recent playoff appearances in the graveyard.
The Haslams? Yeah, they should really sell the team, but I figure in about 10-15 years, they'll move it out of Cleveland.
Hiring a QB based off instagram followers isn't even that unrealistic. If you can put together a team to win the Superbowl, sure do it! If not, just get together a team that people enjoy following and watching. Much of that would be putting athletes on the field that people engage with.
I agree! Maybe the Browns have never played a Superbowl, but I would imagine they are still making a profit. Different goal.
Same reason why two companies have the same idea, one goes viral and one doesn't. Public opinion matters even if its illogical at times.
This has happened in multiple industries a number of times - publishers discover that people with large twitter followings sell a decent number of books, so they start selecting new authors who only have large twitter followings, and discover is was correlation and not causation.
And once it gets out that it’s a selection criteria it gets gamed to hell and back.
I appreciate this, but the stakes have to be a lot lower bringing a book to market than making a $1,000,000 to $10,000,000 seed investment. I'd sort of expect that when you're dealing with sums of money that size there would be some grown-ups in the room.
You just stated Goodharts law in effect.*
[*] https://en.wikipedia.org/wiki/Goodhart%27s_law
Also known as Goodharts law.
> This would be like an NFL team drafting a quarterback based on how many instagram followers they have rather than a relevant metric
sounds like how the ufc does it
> This would be like an NFL team drafting a quarterback based on how many instagram followers
I believe that is how they made the final decision on Watson over Mayfield. Oh, wait, I don't think anything can explain that decision.
Also from Cleveland.
Go Guardians! Go Cavs!
> Can anyone explain why on earth VC's are making actual investment decisions based on imaginary internet points?
I have personally seen several company CEOs (that were billionaires!) do this in different ways. Sometimes hiring people because of it.
When I took over MLAgents at the end of 2021, before Unity fully shit themselves after the insane Weta acquisition, the main metric they were using to promote the project internally were github stars
Yes actually
Needless to say they didn’t like when I said this was a worthless metric and we needed to be using something like “working policies” or “time saved training”
For those of us that don't understand what was so insane about the Weta acquisition?
It never made sense. Weta tools don’t work with Unity at all
There were no complementary workflows or infrastructure or anything.
It was explicitly a move to try to counter epic’s positioning and internally it was very obviously a JR versus Tim pissing contest (and JR was the only one in the contest because Tim didn’t give a fuck about Unity)
should have bought a few thousand stars on the internet and then taken the promotion to director
I was already a director and didn’t need the promotion
I just wanted to build a good product but unfortunately good products are not relevant
I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would.
Here are the things I look at in order:
* last commit date. Newer is better
* age. old is best if still updating. New is not great but tolerable if commits aren't rapid
* issues. Not the count, mind you, just looking at them. How are they handled, what kind of issues are lingering open.
* some of the code. No one is evaluating all of the code of libraries they use. You can certainly check some!
What does stars tell me? They are an indirect variable caused by the above things (driving real engagement and third interest) or otherwise fraud. Only way to tell is to look at the things I listed anyway.
I always treated stars like a bookmark "I'll come back to this project" and never thought of it as a quality metric. Years ago when this problem first surfaced I was surprised (but should not have been in retrospect) they had become a substitute for quality.
I hope the FTC comes down hard on this.
Edit:
* commit history: just browse the history to see what's there. What kind of changes are made and at what cadence.
> I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would
I do it all the time, whenever there are competing libraries to choose among.
It's a heuristic that saves me time.
If one library has 1,000 stars and the other has 15, I'm going to default to the 1,000 stars.
I also look at download count and release frequency. Basically I don't want to use some obscure dependency for something critical.
> If one library has 1,000 stars and the other has 15, I'm going to default to the 1,000 stars.
There are clearly inflection points where stars become useful, with "nobody has ever used this package" and "Meta/Alphabet pays to develop/maintain this package" on the two extremes.
I'm less sure what the signal says in-between those extremes. We have 2 packages, one has 5,000 stars, the other has 10,000 stars - what does this actually tell me, apart from how many times each has gone viral on HN?
> If one library has 1,000 stars and the other has 15, I'm going to default to the 1,000 stars.
Will you continue to do this after reading TFA?
> It's a heuristic that saves me time.
A bad one.
I listed many other useful heuristics. Do you not find value in them? Do you find stars more valuable than them?
Take a moment to consider stars as a useful metric may only be useful for packages created prior to ~2015 when they weren't such a strong vanity metric, and are already very well established. This is preconditioning you to think "stars can still sometimes be useful, because I took a look at Facebook's React GH and it has a quarter million stars".
Sure, it's useful for that. But you aren't going to evaluate if the "React" package is safe. You already trivially know it is.
You'll be evaluating packages like "left-pad". Or any number of packages involved in the latest round of supply chain attacks.
For that matter, VCs are the ones stars are being targeted at, and potential employers (something this article doesn't cover, but some potential hires do hope to leverage on their resume).
If you are a VC, or an employer, it is a negative metric. If you are a dev evaluating packages, it is a vacuous metric that either tells you what you already know, or would be better answered looking at literally anything else within that repo.
The article also called out how download count can be faked trivially. I admit I have relied upon this in the past by mistake. Release frequency I do use as one metric.
When I care about making decisions for a system that will ingest 50k-250k TPS or need to respond in sub-second timings (systems I have worked on multiple times), you can bet "looking at stars" is a useless metric.
For personal projects, it is equally useless.
I care about how many tutorials are online. And today, I care more about if there was enough textual artifacts for the LLMs to usefully build it into their memory and to search on. I care if their docs are good so I spend less tokens burning through their codebase for APIs. I care if they resolve issues in a timely manner. I care if they have meaningful releases and not just garbage nothings every week.
I didn't mean for this to sound like a rant. But seriously, I just can't imagine in any scenario where "I look at stars" as a useful metric. You want to add it to the list? Sure. That is fine. But it should not be a deciding factor. I have chosen libraries with less stars because it had better metrics on things I cared about, and it was the correct choice (I ended up needing to evaluate them both anyhow. But I had my preference from the start).
Choosing the wrong package will waste you so much more time. Spend the 5 minutes evaluating for stuff that is important to your project.
I use stars to try and protect myself from dependency confusion attacks.
For example, let’s say I want to run some piece of software that I’ve heard about, and let’s say I trust that the software isn’t malware because of its reputation.
Most of the time, I’d be installing the software from somewhere that’s not GitHub. A lot of package managers will let anyone upload malware with a name that’s very similar to the software I’m looking for, designed to fool people like me. I need to defend against that. If I can find a GitHub repo that has a ton of stars, I can generally assume that it’s the software I’m looking for, and not a fake imitator, and I can therefore trust the installation instructions in its readme.
Except this is also not 100% safe, because as mentioned in TFA, stars can be bought.
Sure, I suppose that is one solution, but given that buying stars has been around for at least 5 years, and I have been aware of people faking stars for longer than that, I am not sure why you would rely on stars as a primary metric.
There are many other far more useful metrics to look at first, and to focus on first, and to think about. Every time you think about stars, you'll forget the other stuff, or discount it in favor of stars.
Forget stars. They now no longer mean anything. Even if they did before, they don't anymore.
What does "TFA" mean here please?
The featured article.
The article. Pick whatever adjective you like beginning with F!
I think it's "The fucking article".
Yes and to be clear, one uses "TFA" to imply annoyance that TFA hasn't been read.
e.g. "TFA covers this already."
That’s not something I wanted to imply. It can also stand for "the fine article". Is there a better shorthand for "the article linked at top of the page" / "the original article"?
The fucking article.
[delayed]
You call these baubles, well, it is with baubles that men are led... Do you think that you would be able to make men fight by reasoning? Never. That is only good for the scholar in his study. The soldier needs glory, distinctions, and rewards.
https://en.wikiquote.org/wiki/Napoleon
> I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would.
You might not have but the makers of dependencies that you use might so still problematic.
True, but that is beyond my control. I am not evaluating every package within a dependency tree unless something happens, out of practicality.
I have limited time on this Earth and at my employer. My job is not critical to life. I am comfortable with this level of pragmatism.
I usually use stars as bookmarks to maybe come back to some repo I thought looked interesting a year later. Terrible metric to invest based on!
But to someone else, it is a meaningful metric that you bookmarked something. It doesn't matter that the star isn't you saying you liked something. It's already telling enough merely that you wanted to bookmark it.
It's only not meaningful because of how other people can game it and fabricate it, but everything you just said, if it was only people like you, that would be a very meaningful number.
It doesn't even matter why you bookmarked it, and it doesn't matter that whatever the reason was, it doesn't prove the project as a whole is overall good or useful. Maybe you bookmarked it because you hate it and you want to keep track of it for reference in your ted talk about examples of all the worst stuff you hate, but really by the numbers adding up everyone's bookmarks, the more likely is that you found something interesting. It doesn't even matter what was interesting or why. The entire project could be worthless and the thing you're bookmarking was nothing more than some markdown trick in the readme. That's fine. That counts. Or it's all terrible, not a single thing of value, and the only reason to bookmark it is because it's the only thing that turned up in a search. Even that counts, because that still shows they tried to work on something no one else even tried to work on.
It's like, it doesn't matter how little a given star means, it still does mean something, and the aggregation does actually mean something, except for the fact of fakes.
> it still does mean something
Yes...which is why I said it is an indirect variable, as caused by the other things I pointed out above. Age, quality, code, utility, whether issues are addressed, interest, etc. Or fraud. Pretty cut and dry.
FWIW, I almost never star repos. Even ones I use or like. I don't see the utility for myself.
Aim for a more concise post and don't couch your statements in doubt next time if you want a productive conversation, because I don't know what you are trying to say.
These kinds of articles make you feel like there are specific, actionable problems that just need an adjustment and then they disappear. However, the system is much worse than you'd expect. Studies like this are extremely valuable, but they don't address the systematic problems affecting all signaling channels: most signals themselves have been manufactured into a product.
Build a SaaS and you'll have "journalists" asking if they can include you in their new "Top [your category] Apps in [current year]", you just have to pay $5k for first place, $3k for second, and so on (with a promotional discount for first place, since it's your first interaction).
You'll get "promoters" offering to grow your social media following, which is one reason companies may not even realize that some of their own top accounts and GitHub stars are mostly bots.
You'll get "talent scouts" claiming they can find you experts exactly in your niche, but in practice they just scrape and spam profiles with matching keywords on platforms like LinkedIn once you show interest, while simultaneously telling candidates that they work with companies that want them.
And in hiring, you'll see candidates sitting in interview farms quite clearly in East Asia, connecting through Washington D.C. IPs, present themselves with generic European names, with synthetic camera backgrounds, who somehow ace every question, and list experience with every technology you mention in the job post in their CVs already (not hyperbole, I've seen exactly this happen).
If a metric or signal matters, there is already an ecosystem built to fake it, and faking it starts to be operational and just another part of doing business.
Well put!
Have an upvote. The first one is free.
It all boils down to making more money.
Yeah, but it's not a great way to do it.
Short term, you pay the cost of fake signaling, which is simply deadweight loss. People spend resources to inflate signals instead of improving the actual thing.
Medium term, I suppose you could see how it increases consumption, since users would probably try something with 100k stars instead of 2, GitHub wants to seem that it's used more than it really is, repo owner is also benefiting.
Long term, the correspondence between how important a (distorted) system is perceived (Github, OSS, IT in general) vs how important it really is collapses quite abruptly and unnecessarily, and you end up with a lemon market [0] where signals stop being reliable at all.
[0] https://en.wikipedia.org/wiki/The_Market_for_Lemons
The spoilage by money is half right, but I think the more interesting part is where the money ends up and how that influences the system.
I'm increasingly convinced the issue isn't feedback itself, but centralized, global, aggregated feedback that becomes game-able without stronger identity signals.
Right now the incentives are tied (correctly or not) to these global metrics, so you get a market for faking them, with money flowing to whoever is best at juicing that signal.
If instead the signal was based on actual usage and attributions by actual developers, the incentives shift. With localized insight (think "Yeah, I like Golang") it becomes both harder to fake and harder to get at the metric rollup.
Useful reputation on the web is actually much more localized and personal. I gladly receive updates on and would support the repos I've starred. If I could chose where to put my dollars (not an investors), it would likely include the list of repos I've personally curated.
This suggests a different direction: instead of asking "how many stars does this have?", ask "who is actually depending on this, and in what context?" or better retroactively compare your top-n repos to mine and we'll get a metric seen through our lenses. If you want to include everyone in that aggregation you'll end up where we are now, but if in stead you chose the list, well, the stars could align as a good metric once more.
The interesting part is that the web already contains most of that information, we just don't treat identity as a part of the signal (yet? universally?).
Tangential but I have more than 5k repos starred (according to my GitHub profile) organized into lists but the way I discover interesting stuff on GitHub these days is through people I follow. Follow interesting people, find interesting stuff. Sometimes it's that easy.
What's more it became obvious to me two or so years ago that GitHub is going the way of LinkedIn slowly but surely. Lots of professionals on there just because it's expected of them, some interact occasionally with the "social media" aspect of it and fewer still really thrive on that part. Time will tell how this will pan out but just look how many Developer and Linux influencers became huge on YouTube and other places this last year. Most of them barely had more than 10k subscribers 3 years ago and now people look to them for their next tech stack and hot framework/tool/library/distro and so on.
Of course - money is a good proxy for value in these instances. Not perfect, but good.
At the end it's a company choice: do you buy BS metrics or you don't.
We've recently decided to complicate life of AI bots in our repo https://archestra.ai/blog/only-responsible-ai, hoping they will just choose those AI startups who are easier to engage with.
Yeah imagine how nature feels with all of the fake eyes and other fake predator signals like bright colors. Evolution finds a way
A way for what? Evolution does not have a will, does not work for anybody in particular, and it's statistically likely that it will work against you, as almost all species that ever were went extinct and almost no individual from just 15 generations ago has left DNA to present people. [0] [1]
[0] http://www.stat.yale.edu/~jtc5/papers/Ancestors.pdf [1] https://pubmed.ncbi.nlm.nih.gov/11542058/
i was with them until
"We ran our own analysis sampling 150 profiles per repo across 20 projects and found repos where 36-76% of stargazers have zero followers and fork-to-star ratios 10x below organic baselines"
This does not looks like appropriate signal to use on github, i doubt that this is organic baseline.If this is used as metric than study might be flawed.
I run a tiny site that basically gave a point-at-able definition to an existing adhoc standard. As part of the effort I have a list of software and libraries following the standard on the homepage. Initially I would accept just about anything but as the list grew I started wanting to set a sort of notability baseline.
Specifically someone submitted a library that was only several days old, clearly entirely AI generated, and not particularly well built.
I noted my concerns with listing said library in my reply declining to do so, among them that it had "zero stars". The author was very aggressive and in his rant of a reply asked how many stars he needed. I declined to answer, that's not how this works. Stars are a consideration, not the be all end all.
You need real world users and more importantly real notability. Not stars. The stars are irrelevant.
This conversation happened on GitHub and since then I have had other developers wander into that conversation and demand I set a star count definition for my "vague notability requirement". I'm not going to, it's intentionally vague. When a metric becomes a target it ceases to be a good metric as they say.
I don't want the page to get overly long, and if I just listed everything with X star count I'd certainly list some sort of malware.
I am under no obligation to list your library. Stop being rude.
> When a metric becomes a target it ceases to be a good metric as they say.
https://en.wikipedia.org/wiki/Goodhart's_law
Nice to know the name for this — Goodhart's Law. And I think the core reason is that the cost to fake these metrics is far less than what they claim to represent. Stars, reviews, ratings, trading volumes — all cheap to manufacture, and only getting cheaper with AI.
I've been thinking about this a lot. These metrics are all just marketing signals to draw people's attention, trying to make some kind of deals. So the fix should be: make the cost of the signal match what it claims to represent. I'm obsessed with something called DUKI /djuːki/ (Decentralized Universal Kindness Income, a form of UBI) — the idea is that instead of stars or reviews, trust comes from deals pledging real money to the world for all as the deal happens. You can't fake that cheaply.
So the metric becomes the money itself — if you fake X amount, it costs you X, and the world will thank you by paying attention...
Imagine if GitHub let you back a star with real money — the more you put in, the more credible the star. And that money goes out as UBI for everyone. For attention makers, star anything you want, as much as you want. For attention takers, just follow the money to filter through all the noise that's so easy to manipulate...
Very simply, you need to see VCs as branding companies who give people with many followers brand deals. VCs think that if you have a lot of stars, you must have hit product-market fit — or something close — because many developers have started using your open-source tool. This isn’t necessarily always the case. Every weekend project from Andrej Karpathy gets loads of stars because he is the most famous person on GitHub. What I’ve noticed a lot is that the repos with the most stars most of the time already came from big companies open-sourcing their tools, or people building free versions of paid software.
I think people expect the star system to be a cheap proxy for "this is a reliable piece of sorfware which has a good quality and a lot of eyes".
I think as a proxy it fails completely: astroturfing aside stars don't guarantee popularity (and I bet the correlation is very weak, a lot of very fundamental system libraries have small number of stars). Stars also don't guarantee the quality.
And given that you can read the code, stars seem to be a completely pointless proxy. I'm teaching myself to skip the stars and skim through the code and evaluate the quality of both architecture and implementation. And I found that quite a few times I prefer a less-"starry" alternative after looking directly at the repo content.
given that you can read the code, stars seem to be a completely pointless proxy
Imagine you're choosing between 3 different alternatives, and each is 100,000 LOC. Is 'reading the code' really an option? You need a proxy.
Stars isn't a good one because it's an untrusted source. Something like a referral would be much better, but in a space where your network doesn't have much knowledge a proxy like stars is the only option.
> Is 'reading the code' really an option? You need a proxy.
100k is small, but you're right, it can be millions. I usually skim through the code tho, and it's not that hard. I don't need to fully read and understand the code.
What I look at is: high-level architecture (is there any, is it modular or one big lump of code, how modular it is, what kind of modules and components it has and how they interact), code quality (structuring, naming, aesthetics), bus factor (how many people contribute and understand the code base).
I don't think I have ever even considered using star count as a factor for picking from alternatives.
Looking at the commit history, closed vs open issues and pull requests provides a much more useful signal if you can't decide from the code.
Ask Claude to help. Read the dang code. You'll be more confident in your decision and better positioned to handle any issues you encounter.
The issues page used to be good for this as well. What kind of problems people are having.
(Sometimes still is, but the agents garbage does not help)
Honest question: how can VCs consider the 'star' system reliable? Users who add stars often stop following the project, so poorly maintained projects can have many stars but are effectively outdated. A better system, but certainly not the best, would be to look at how much "life" issues have, opening, closing (not automatic), and response times. My project has 200 stars, and I struggle like crazy to update regularly without simple version bumps.
The stars have fallen to the classic problem of becoming a goal and stopping being a good metric. This can apply to your measure just as well: issues can also be gamed to be opened, closed and responded to quickly, especially now with LLMs.
Was it ever a good metric? A star from another account costs nothing and conveys nothing about the sincerity, knowledge, importance or cultural weight of the star giver. As a signal it's as weak as 'hitting that like button'.
If the number of stars are in the thousands, tens of thousands, or hundreds of thousands, that might correlate with a serious project. But that should be visible by real, costly activity such as issues, PRs, discussion and activity.
There was a time when total number of hyperlinks to a site was an amazing metric measuring its quality.
Yeah, the time between Google appeared, until the time SEO became a concept people chased, a very brief moment of time.
at that time having a website took work, while having a github account can be cheaply used to sybil attack/signal marketing
There isn't just "good metric" in vacuum - it was a good metric of exactly the popularity that you mentioned. But stars becoming an object of desire is what killed it for that purpose. Perhaps now they are a "good metric" of combined interest and investment in the project, but what they're measuring is just not useful anymore.
Yeah, I'd agree with this. I always thought of a star indicating only that a person (or account, generally) had an active interest in another project, either through being directly related or just from curiosity. Which can sort of work as a proxy for interesting, important or active, but not accurately.
A repository with zero stars has essentially no users. A repository with single-stars has a few users, but possibly most/all are personal acquiantances of the author, or members of the project.
It is the meaning of having dozens or hundreds of stars that is undermined by the practice described at the linked post.
I especially love issues automatically "closed due to inactivity" just to keep the number of issues down :V
Sometimes people open issues without proper information. It cant be replicated and nobody else is jumping in that it affects them. You may suspect its something else, maybe with their environment, but if they don't engage what else can you do? Tell them you are closing it and specify what kind of info you need if they ever get around to providing it and it can be reopened.
And sometimes the maintainer simply doesn't respond to a perfectly acceptable issue due to either the maintainer abandoning the project, not enough maintainers or simple neglect.
"When a measure becomes a target, it ceases to be a good measure"
https://en.wikipedia.org/wiki/Goodhart%27s_law
I wonder if there's a more graph oriented score that could work well here - something pagerank ish so that a repo scores better if it has issues reported by users who themselves have a good score. So it's at least a little resilient to crude manipulation attempts
GitHub has all kinds of private internal metrics that could update the system to show a much higher signal/quality score. A score that is impervious to manipulation. And extremely well correlated with actual quality and popularity and value, not noise.
Two projects could look exactly the same from visible metrics, and one is complete shell and the other a great project.
But they choose not to publish it.
And those same private signals more effectively spot the signal-rich stargazers than PageRank.
It would be more resilient indeed, I think. Definitely needs a way to figure out which users should have a good score, though - otherwise it's just shifting the problem somewhat. Perhaps it could be done with a reputation type of approach, where the initial reputation would be driven by a pool of "trusted" open source contributors from some major projects.
That said, I believe the core problem is that GitHub belongs to Microsoft, and so it will still go more towards operating like a social network than not - i.e. engagement matters. It will still take a good will to get rid of Social Network Disease at scale.
Reputation doesn’t equal good taste in judging other projects.
There are much better ways of finding those who have good taste.
You are looking for different things to VCs. You are looking for markers that show software quality over the long-term. They are looking for markers that show rapidly gaining momentum over the short-term. These are often in opposition to one another.
Making the conversations about VCs expecting thousands of stars, is thinking too big. It's probably more often someone pays $20 to make one of their projects look good, for their CV, for vanity, thinking this will get them the push that they need to get clicks on reddit, or noticed over some other open source project. If there is someone offering a 10k star project an investment over 8k without looking at the project or revenue potential, I can only think they are clueless, or picking a student project to fund each summer.
The fake accounts often star my old repos to look like real users. They are usually very sketchy if you think for a minute, for example starring 5,000 projects in a month and no other GitHub activity. One time I found a GitHub Sponsor ring, which must be a money laundering / stolen credit cards thing?
Unless something has changed in the last ~3 years, I think the article vastly overstates the credibility with VC's.
Even 10 years ago most VCs we spoke to had wisened up and discarded Github stars as a vanity metric.
Agree that sophisticated funds don't, but the ecosystem hasn't caught up. StarHub/GitStar pricing pages still sell to "seed-stage founders pre-fundraise"
Much more important is who starred it. And are they selective about giving out stars or bookmarking everything. Forks is a closer signal to usage than stargazing.
Indeed, GitHub should set up a monthly quota for available stars to give and correlate the account age with it: either make something up like a "trusted-age-factor" that multiplies any given star by that factor, or scale the available quota accordingly by that factor (and let users star repos repeatedly).
GitHub should also introduce a way to bookmark a repo, additional to the existing options of sponsor/watch/fork/star-ing it.
With the advent of AI, these "life" events are probably even simpler to fake than AI though, and unlike the faking of stars not against the ToS.
Stars are a simple metric even someone like a VC investor can understand. Your "better system" sounds far too complicated and time consuming.
Many VCs are only doing one thing: how to use some magical quantitative metrics to assess whether a project is reliable without knowing the know-how. Numbers are always better than no numbers.
Honestly I don't know if that's true. Picking up on vibes might be better than something like GitHub stats.
When a partner decides to recommend a startup to the investment committee, he needs some explicit reasons to convince the committee, not some kind of implicit vibe
But given the amount of astroturfing and star-buying out there, relying on star counts may well select for deceptive founders.
Yes, I think VCs have already switched to using other metrics that are less easy to fake, such as download per month or customer interviews (or more direct, ARR, even for really early stage startups). I just want to explain the background reason of it.
because VC don't care about anything being legitimate, if it can fool VCs it can also fool market participants, then VC can profit off of it.
one VC told me, you'll get more funding and upvotes if u don't put "india" in your username.
This is a good idea, but from my experience most VCs (I’m not talking about the big leagues) aren’t technical, they tend to repeat buzzwords, so they don’t really understand how star systems works.
Because VCs love quantifiable metrics regardless of how reliable they actually are. They raise money from outside investors and are under pressure to deploy it. The metrics give them something concrete to justify their thesis and move on with their life.
>Honest question: how can VCs consider the 'star' system reliable?
Founders need the ability to get traction, so if a VC gets a pitch and the project's repo has 0 stars, that's a strong signal that this specific team is just not able to put themselves out there, or that what they're making doesn't resonate with anyone.
When I mentioned that a small feature I shared got 3k views when I just mentioned it on Reddit, then investors' ears perked right up and I bet you're thinking "I wonder what that is, I'd like to see that!" People like to see things that are popular.
By the way, congrats on 200 stars on your project, I think that is definitely a solid indicator of interest and quality, and I doubt investors would ignore it.
> how can VCs consider the 'star' system reliable
I think VCs just know that there are no reliable systems, so they go with whatever's used.
Yeah, I was pondering a few months away when I checked Pathway, an ETL solution. I've never heard about it but I saw some news that they have created a better model than transformer. So stats:
- link: https://github.com/pathwaycom/pathway
- watch: 115, fork: 1.6k, star: 63.5k
- issues: 32, PR-s: 3
And compare to other ETL tool, like Apache Airflow - used by me and many machine learning folks:
- link https://github.com/apache/airflow
- watch: 777, forks 16.9k!!!!!, Stars: (only!) 45.1k
- issues: 1200 (!!!), PR-s (501!!!).
> VC funding pipeline that treats GitHub popularity as proof of traction
Why am I not surprised big Capital corrupts everything. Also, Goodhart's law applies again: "When a measure becomes a target, it ceases to be a good measure".
HN Folks: What reliant, diverse signals do you use to quickly eval a repo's quality? For me it is: Maintenance status, age, elegance of API and maybe commit history.
PS: From the article:
> instead tracks unique monthly contributor activity - anyone who created an issue, comment, PR, or commit. Fewer than 5% of top 10,000 projects ever exceeded 250 monthly contributors; only 2% sustained it across six months.
> [...] recommends five metrics that correlate with real adoption: package downloads, issue quality (production edge cases from real users), contributor retention (time to second PR), community discussion depth, and usage telemetry.
Overly verbose and "glitter" readme.md files is a good indicator of bad projects or at least projects which need more attention to be used. It's too often pre-rugpull or look-at-me repos where better solutions are one click away.
Finding any curse words in hidden comments in the commit history is for me a good indication of a human working on a passion project, though ymmv.
And there are always exceptions to the exception of the exceptions.
I tend to look at other people involved, like contributors but not just the volume but actual people and their other activity. If the original author is still around and active that tends to be a good sign IME
I usually just read the dang code.
I wonder if it makes sense for GitHub to use graph-theoretic measures like PageRank instead of raw stars. In simple terms, a repo is considered important if it is starred or forked by GitHub users who maintain other important repos.
It’s more expensive to compute, but the resulting scores would be more trustworthy unless I’m missing something.
That sounds closer to achieving a good outcome. Of course I think anything that includes the set of all users as columns will be game-able. You need to either choose the set yourself from "trusted peers" or "foaf" degrees, or maybe better use retroactive signals rather than purely like-driven approaches.
Wait I can sell my github account for 5k ? Wow
> VCs explicitly use stars as sourcing signals
In my opinion, nothing could be more wrong. GitHub's own ratings are easily manipulated and measure not necessarily the quality of the project itself, but rather its Popularity. The problem is that popularity is rarely directly proportional to the quality of the project itself.
I'm building a product and I'm seeing what important is the distribution and comunication instead of the development it self.
Unfortunately, a project's popularity is often directly proportional to the communication "built" around it and inversely proportional to its actual quality. This isn't always the case, but it often is.
Moreover, adopting effective and objective project evaluation tools is quite expensive for VCs.
Vast majority of mid level experienced people take stars very seriously and they won't use anything under 100 stars.
I'm not supporting this view but it is what it is unfortunately.
VCs that invest based on stars do know something I guess or they are just bad investors.
IMO using projects based on start count is terrible engineering practice.
I've seen the same devs refuse to use a library because the last commit was 3 months ago, despite the library being extremely popular, battle tested, and existing for 10 years.
also and above all because it can be easily manipulated, as the research explained in the article actually demonstrates
> measure not necessarily the quality of the project itself, but rather its Popularity
Surely a project's popularity is often related to its utility. A useful and popular project seems like exactly the kind of thing a VC might be interested in.
Well, pretty sure that VCs are more interested in popularity than in quality so maybe it's not such a bad metric for them.
Yes, you're right, but popularity becomes fleeting without real quality behind the projects.
Hype helps raise funds, of course, and sells, of course.
But it doesn't necessarily lead to long-term sustainability of investments.
* https://dagster.io/blog/fake-stars (2023) - Tracking the Fake GitHub Star Black Market with Dagster, dbt and BigQuery
* https://arxiv.org/abs/2412.13459 (2024/2025) - Six Million (Suspected) Fake Stars in GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware
We're this close to rediscovering pagerank
It'd ideally be more of a peoplerank though. I think Google discovered this problem themselves when Pagerank became a well known thing.
You'd want to discard a lot of the noise in the bottom 20% of linking power. You want to focus more on the 'trust' factor.
I was literally was just looking at GitHub dataset availability and musing on this. A star from karpathy is worth a lot more than a star from open_claw_dood that just created his account 5 min ago.
In general, I’ve been dissatisfied with GitHub’s code search. It would be nice to see innovation here.
I look at the starts when choosing dependencies, it's a first filter for sure. Good reminder that everything gets gamed given the incentives.
> I look at the starts when choosing dependencies, it's a first filter for sure.
Unfortunately I still look at them, too, out of habit: The project or repo's star count _was_ a first filter in the past, and we must keep in mind it no longer is.
> Good reminder that everything gets gamed given the incentives.
Also known as Goodhart's law [1]: "When a measure becomes a target, it ceases to be a good measure".
Essentially, VCs screwed this one up for the rest of us, I think?
[1] https://en.wikipedia.org/wiki/Goodhart%27s_law
> The project or repo's star count _was_ a first filter in the past, a
I agree that it has been a first filter, but should it ever have been? A star only says that someone had a passing interest in a project. Not significantly different from a 'like' on a social media post.
> The project or repo's star count _was_ a first filter in the past, and we must keep in mind it no longer is.
Id suggest the first question to ask is "if the project is an AI project or not?" If it is, dont pay attention to the stars - if it's not, use the stars as a first filter. That's the way I analyse projects on Github now.
Average case of "once a measure becomes a target".
Social media platforms (like Instagram) have always had this problem of "buying" followers. There was an article some time ago where hollywood types would only give roles to people with high followers so people started buying followers.
Now that money is flowing to Github stars, no wonder people are buying fake "stars"? Seems capitalism is working as expected...
Steam wishlist, itch.io number of views, YouTube views and now GitHub stars… I’m tired of all the gamification of creativity. Now if you’d upvote my comment I can get same karma please and thank you
Besides the ability to downvote comments after passing a threshold of 500, what else is HN karma good for?
Who knows? The way things are going VCs could start asking claude to review someone’s online presence and Claude might decide karma is the best metric for that
Personally I use stars in two ways: 1) It's interesting and I want to keep track of it for possible future use and 2) It's a fantastic idea and kudos to you even if I'll never use it.
As a side note it's kind of disheartening that everytime there is a metric related to popularity there would be some among us that will try to game it for profit, basically to manipulate our natural bias.
As a side note it's always a bit sad how the parasocial nature of the modern web make us like machine interfacing via simple widgets, becoming mechanical robot ourselves rationalising IO via simple metrics kind of forgetting that the map is never the territory.
Next in: NPM's fake install count economy
For all the hate on star systems, whether it’s GitHub/Amazon/AppStore, I’d still take having one over having nothing at all.
They make it easier to sort through options, help with search and discovery, and at least give you a baseline signal for trust can get better over time.
So to me, some signal better than no signal at all.
I usually use stars as a bookmark list to visit later (which I rarely do). I probably would need to stop doing that and use my self-hosted "Karkeep" instance for github projects as well.
Never heard of it before.
https://github.com/karakeep-app/karakeep
Sounds useful.
I’ll star it and check it out later ;)
Tbh, for me there’s basically no difference between a repo with 2k stars and one with 20k.
Stars only matter when there are very few, like if it has almost none, that’s a red flag. Otherwise it’s just noise.
Same old story of centralised algorithms being abused.
Github stars is akin to 'link popularity' or 'pagerank' which is ripe for abuse.
One way around it is to trust well known authors/users more. But it's hard to verify who is who. And accounts get bought/closed/hacked.
Another way is to hand over the algo in a way where individuals and groups can shape it, so there's no universal answer to everyone.
My project https://github.com/socketCluster/socketcluster has been accumulating stars slowly but steadily over about 13 years. Now it has over 6k stars but it doesn't seem to mean much nowadays as a metric. It sucks having put in the effort and seeing it get lost in a sea of scams and seeing people doubting my project's own authenticity.
It does feel like everything is a scam nowadays though. All the numbers seem fake; whether it's number of users, number of likes, number of stars, amount of money, number of re-tweets, number of shares issued, market cap... Maybe it's time we focus on qualitative metrics instead?
I came across one of these in 2018 with a "hot" open source company raising a Series B. An impressive star ramp (about 300% YoY growth) before the (high-priced/competitive) raise and three months later Github had revoked almost all the star growth from the previous year, resulting in a 20% YoY record. The company eventually got acquihired.
I do not look at the stars. I look at the list of contributors, their activities and the bug reports / issues.
> I look at the list of contributors
Specifically if those avatars are cute animie girls.
> Specifically if those avatars are cute anime girls.
I know you are half joking/not joking, but this is definitely a golden signal.
Positive or negative to you? Whenever I see more than one anime-adjacent profile picture I duck out.
Positive ofc, most of them a top-tier Rust devs.
Yeah, I didn't think anyone would place any actual value on the stars. It almost doesn't need to be a feature, because what is it suppose to do exactly?
15 mins into this - Built this to identify the fraudsters https://github.com/mercurialsolo/realstars
We should do a hall of shame!
It's a pity that no one will ever see this 15-minute slop.
Github could easily crack down on this. Spend $10 at each star provider, then ban all accounts involved. A tiny bit of money could create a huge drag on the ecosystem.
I think the reason is that investors are not IT experts and don't know better metrics to evaluate.
I guess it's like fake followers on other social media platforms.
To me, it just reflects a behaviour that is typical of humans: in many situations, we make decisions in fields we don't understand, so we evaluate things poorly.
Seen this firsthand, repos with hundreds of stars and zero meaningful commits or issues. In hardware/RTL projects it's less prominent.
I don't know what is more, for lack of a better word, pathetic, buying stars/upvotes/platform equivalent or thinking of oneself as a serious investor and using something like that as a metric guiding your decision making process.
I'd give a lot of credit to Microsoft and the Github team if they went on a major ban/star removal wave of affected repos, akin to how Valve occasionally does a major sweep across CSGO2 banning verified cheaters.
The problem is that if this is the game now, you need to play it. I'm trying to get a new open source project off the ground and now I wonder if I need to buy fake stars. Or buy the cheapest kind of fake stars for my competitors so they get deleted.
For Microsoft this is another kind of sunk cost, so idk how much incentive they have to fix this situation.
The issue with that is, it's a game that never ends. Now you need to inflate your npm/brew/dnf installs, then your website traffic to not make it to obvious, etc.
I am not successful at all with my current projects (admittedly am not trying to be nowadays), so feel free to dismiss this advice that predates a time before LLM driven development, but in the past, I have had decent success in forums interacting with those with a specific problem my project did address. Less in stars, more in actual exchange of helpful contributions.
An open source project really shouldn't be something you need to "get off the ground." If it provides value then people will naturally use it.
How do people know it exists to solve their problem? Even before LLMs it was hard to get through VC funded marketing by (commercial) competitors.
My first Open Source project easily got off the ground just by being listed in SourceForge.
Haha, have you tried that? I think in this day and age marketing is much needed activity even for open-source projects providing quality solutions to problems.
I maintain a niche-popular project that I didn't do any marketing for. My understanding is that even for popular projects, the usual dynamic is that there's just one guy doing all the work. So "getting off the ground" just means getting people to use it, and there shouldn't be any reason to artificially force that.
It depends what your objective is. Many people seem to see their open source projects as a stepping stone into some commercial activity. Putting aside whether that is a good idea or not if that is what they want to do then they will need to market in some way.
Citing Valve as a model for handling cheating is not what I'd have reached for.
Honest question, which companies handle the process better given it is a trade-off? Yes, VAC is not as iron-clad as kernel level solutions can be, but the latter is overly invasive for many users. I'd argue neither is the objectively right or better approach here and Valves approach of longer term data collection and working on ML solutions that have the potential to catch even those cheating methods currently able to bypass kernel level anti-cheat is a good step.
On Github stars, I'd argue they are the most suitable comparison, as all the funny business regarding stars should be, if at all, detectable by Github directly and ideally, bans would have the biggest deterrent effect, if they happened in larger waves, allowing the community to see who did engage in fraudulent behaviour.
Maybe there is also fake upvote economy here.
Stars are like for developers? and you have a bunch of creators now entering the arena. what did you expect?
Why is zero public repos a criteria?
I paid github for years to keep my repos private...
But then I don't participate in the stars "economy" anyway, I don't star and I don't count stars, so I'm probably irrellevant for this study.
Am very much the same, took a bunch private two years ago for multitude of reasons. I can, however, see why no public repos could be a partial indicator and of concern, in conjunction with sudden star growth, simply because it is hard for a person with no prior project to suddenly and publicly strike gold. Even on Youtube it is a rare treat to stumble across a well made video by a small channel and without algos to surface repos on Github in the same way, any viral success from a previously inactive account should be treated with some suspicion. Same the other way, if you never made any PR, etc. sudden engagement is a bit odd.
I think they're using it as a signal for the accounts doing the starring, not the account being starred...
Can now someone make analysis of Google Ads fake economy? I'm convinced that the data they share - specifically clicks - is false, and potentially they're paying some fraction to people to click it.
what is this one about:
> When nobody is forking a 157,000-star repository, nobody is using it
that is completely not true, i don't fork a repo when i use it, only when i want to contribute to it (and usually cleanup my forks)
So, if star to fork ratio is the new signal, time to make an extra fake star tier, where the bot forks the repo, generates a commit with the cheapest LLM available and pushes that to gh, right?
The next step after that is going to be celebrity forks -- whether top devs and/or Milla Jovovich have forked your repo.
Why would OpenAI have bought stars for openai-fm I wonder?
Seen this happen first-hand with mid-to-large open source projects that sometimes "sponsor" hackathons, literally setting a task to "star the repo" to be eligible.
It’s supposed to get people to actually try your product. If they like it, they star it. Simple.
At that point, forcing the action just inflates numbers and strips them of any meaning.
Gaming stars to set it as a positive signal for the product to showcase is just SHIT.
Cost of signalling is way lesser than the cost of verification.
Would be nice to see the ratio of OpenClaw stars
99% stars from Claws themselves
The stick of God doesn't make sound. God's work indeed
Bots are killing opensource, but they pump product metrics so nobody cares. I maintain an open source repo and we've made a decision to limit all bot activity, even if it makes us less sexy in front of VCs.
We figured out a workaround to limit activity to prior contributors only, and add a CI job that pushes a coauthored commit after passing captcha on our website. It cut the AI slop by 90%. Full write-up https://archestra.ai/blog/only-responsible-ai
wait people trust GH start for like anything????
I spent a little bit of time at a PE firm last year. They went "all in" on ElizaOS because of the star hype. It was embarrassing.
This is why people shouls use gitea. I dont understand why people keep using github at this point. Its not like theyve stolen all our data or anything:))
now that we have AI, and github is backed by microsoft. we should ask users to justify their stars. and then they should have a classifier
The real metric is: does it solve my problem, and is the maintainer still responding to issues? Everything else is just noise.
> Jordan Segall, Partner at Redpoint Ventures, published an analysis of 80 developer tool companies showing that the median GitHub star count at seed financing was 2,850 and at Series A was 4,980. He confirmed: "Many VCs write internal scraping programs to identify fast growing github projects for sourcing, and the most common metric they look toward is stars."
> Runa Capital publishes the ROSS (Runa Open Source Startup) Index quarterly, ranking the 20 fastest-growing open-source startups by GitHub star growth rate. Per TechCrunch, 68% of ROSS Index startups that attracted investment did so at seed stage, with $169 million raised across tracked rounds. GitHub itself, through its GitHub Fund partnership with M12 (Microsoft's VC arm), commits $10 million annually to invest in 8-10 open-source companies at pre-seed/seed stages based partly on platform traction.
This all smells like BS. If you are going to do an analysis you need to do some sound maths on amount of investment a project gets in relation to github starts.
All this says is stars are considered is some ways, which is very far from saying that you get the fake stars and then you have investment.
This smells like bait for hating on people that get investment
Who ever thought that GitHub stars were a legitimate measure of a project's popularity does not understand Goodhart's Law and such metrics were easily abused, faked, gamed and manipulated.
I asked Claude for an analysis on the maturity of various open source projects accomplishing the same thing. Its first searches were for GitHub star counts for each project. I was appalled at how dumb an approach that was and mortified at how many people must be espousing that equivocation online to make the training jump to that.
Definite proof that github is social network for programmers.
> The CMU researchers recommended GitHub adopt a weighted popularity metric based on network centrality rather than raw star counts. A change that would structurally undermine the fake star economy. GitHub has not implemented it.
> As one commenter put it: "You can fake a star count, but you can't fake a bug fix that saves someone's weekend."
I'm curious what the research says here---can you actually structurally undermine the gamification of social influence scores? And I'm pretty sure fake bugfixes are almost trivial to generate by LLMs.
that's exactly the next-round attack. StarScout's network-centrality defense works for the current generation of campaigns but won't survive LLM-generated PR/commit patterns
I’d say those CMU researchers are out of touch with the reality. GitHub can easily overhaul this with a much better system than what those researchers recommended but chooses not to.
same here on HN as well
Yep, shilling and paid advertisement mascerading as posts espegially on ai
https://x.com/garrytan/status/2045404377226285538
“gstack is not a hypothetical. It’s a product with real users:
75,000+ GitHub stars in 5 weeks
14,965 unique installations (opt-in telemetry, so real number is at least 2x higher)
305,309 skill invocations recorded since January 2026
~7,000 weekly active users at peak”
GitHub stars are a meaningless metric but I don’t think a high star count necessarily indicates bought stars. I don’t think Garry is buying stars for his project.
People star things because they want to be seen as part of the in-crowd, who knows about this magical futuristic technology, not because they care to use it.
Some companies are buying stars, sure, but the methodology for identifying it in this article is bad.
I got gently admonished on here a while back for mentioning that I find those star graph things people put on their READMEs to have entirely the opposite effect than that which was intended. I see one of those and I'm considerably less likely to trust the project because a) you're chasing a stupider metric than lines of code, and b) people obviously buy stars.
The fake star problem is a symptom of a deeper issue — developers can't tell signal from noise in the agent ecosystem. The tools that actually get real adoption are the ones that solve acute production problems. Agents are hitting these in production issues of state management every day and there's almost no tooling for it. That's where genuine organic stars come from — solving a real pain, not gaming rankings