This is really cool and something I've envisioned building for a long time!
There is a bug in the entity tracking. For the entity "github", it shows a positive sentiment. HN does NOT like GitHub (for reasons good or bad). If you click on it, it shows you stories about other seemingly unrelated stories.
Thank you. I believe this is because it's not properly aggregating the story title, content, and comment hierarchy. There are going to be cases where the LLM does a poor job of understanding the conversation, but I think right now the information isn't being sent to the prompt.
Right now it seems to be only using one level of the parent comment hierarchy.
Awesome idea! The entity tracking is very exciting, most interesting part imo
I think the budget is noticeable in the sentiment analysis unfortunately, the tags and entity recognition are good but the sentiment ratings themselves seem pretty sloppy.
I think it's mostly prompting, but I will be experimenting with this more. The prompt currently is garbage IMO
You are an expert analyst of the Hacker News community. Analyze submissions for
the underlying ideas, concepts, technologies, and entities being discussed.
Write all summaries in third-person analytical prose. Do NOT start sentences
with "The user", "The commenter", "The author", or "This post". Instead, lead
with the substance: describe the idea, argument, or phenomenon directly.
Good: "Decentralized identity systems could reduce reliance on corporate
gatekeepers." Bad: "The user discusses how decentralized identity systems work."
This is virtually identical to tools the US Department of Homeland Security uses across each social media platform and major website with comments to monitor sentiment and activities.
Very interesting. LLMs open up space for transforming unstructured raw data into visualizations and dashboards. I made something just looking at “Who wants to be hired” posts.
This is really cool and something I've envisioned building for a long time!
There is a bug in the entity tracking. For the entity "github", it shows a positive sentiment. HN does NOT like GitHub (for reasons good or bad). If you click on it, it shows you stories about other seemingly unrelated stories.
https://ethos.devrupt.io/entities/github
Thank you. I believe this is because it's not properly aggregating the story title, content, and comment hierarchy. There are going to be cases where the LLM does a poor job of understanding the conversation, but I think right now the information isn't being sent to the prompt.
Right now it seems to be only using one level of the parent comment hierarchy.
(Source: https://github.com/devrupt-io/ethos/blob/67670eb2855b84d389d...)
Awesome idea! The entity tracking is very exciting, most interesting part imo
I think the budget is noticeable in the sentiment analysis unfortunately, the tags and entity recognition are good but the sentiment ratings themselves seem pretty sloppy.
I think it's mostly prompting, but I will be experimenting with this more. The prompt currently is garbage IMO
(Source: https://github.com/devrupt-io/ethos/blob/67670eb2855b84d389d...)Garbage, why? That is the insightful bit you chose to omit. How would you do it instead?
Well done.
If I could suggest, please make green colors more distinct in sentiment split wheel, they seem to be very similar now.
any blog post anywhere that explains how all of this stuff works and the architecture etc?
We wrote this https://blog.devrupt.io/posts/introducing-ethos/
This is virtually identical to tools the US Department of Homeland Security uses across each social media platform and major website with comments to monitor sentiment and activities.
Congrats, I guess.
I was also told this by someone randomly while working at a coffee shop here in DC. Something about CGA.
Jeffrey Epstein: 0.20% Positive! Lol.
Side note: this is cool, but the sentiment analysis could be a bit more sophisticated in v2.
Very interesting. LLMs open up space for transforming unstructured raw data into visualizations and dashboards. I made something just looking at “Who wants to be hired” posts.
https://hireindex.xyz/#stats
Does that use the "real" LinkedIn API or something else like Playwright?
What model does it use?
What vector database is it using?