I like to throw sports trivia at AI systems to see how well they do on something that's low-stakes and very well-documented. Here's one question Wiyomi generated for me just now about college football:
> What coaching change occurred at USC during the 2010-2011 offseason?
> A. Lane Kiffin left for the Oakland Raiders, and Steve Sarkisian took over
> B. Pete Caroll left for the Seattle Seahawks, and Lane Kiffin took over
> C. Steve Sarkisian left for Washington, and Lane Kiffin took over
> D. Lane Kiffin was fired, and Steve Sarkisian took over
Wiyomi said I was wrong, and that D is the correct answer:
> Lane Kiffin was fired as USC's head coach after the 2010 season, and Steve Sarkisian, who had been the offensive coordinator, took over as the interim head coach before being officially named head coach.
Lane Kiffin's tenure at USC lasted from 2010 to 2013. If I can't rely on these AI systems for the simplest low-stakes tasks, what good are they for?
I gave this a tough test, specifically generating a quiz about the video game football manager, and it failed quite badly.
It would either give 3 absurd/joke answers and one good correct one. Or give 3/4 correct answers and then insist that only one of them was correct and the others were wrong.
It's probably only accurate to specific topics and still requires alot of human validation and input. To me that makes it not that better than asking an AI for suggestions conversationally. Probably worse since you can make more progress and adjustments conversationally.
We built Wiyomi after struggling to stay consistent with language learning apps. Most tools felt either too gamified or too generic. So we made something simple: type a topic (or paste a book’s table of contents), and get AI-generated quizzes instantly.
You can track progress, set reminders, and actually build a habit.
We’d love your feedback, try it out at wiyomi.com!
Hi all thank you for the honest feedbacks, This MVP will solve your problem poorly but your honest feedback will help us make this MVP much much better and works!
I like to throw sports trivia at AI systems to see how well they do on something that's low-stakes and very well-documented. Here's one question Wiyomi generated for me just now about college football:
> What coaching change occurred at USC during the 2010-2011 offseason?
> A. Lane Kiffin left for the Oakland Raiders, and Steve Sarkisian took over
> B. Pete Caroll left for the Seattle Seahawks, and Lane Kiffin took over
> C. Steve Sarkisian left for Washington, and Lane Kiffin took over
> D. Lane Kiffin was fired, and Steve Sarkisian took over
I chose B because I knew that to be true: https://en.wikipedia.org/wiki/Lane_Kiffin#USC_Trojans
Wiyomi said I was wrong, and that D is the correct answer:
> Lane Kiffin was fired as USC's head coach after the 2010 season, and Steve Sarkisian, who had been the offensive coordinator, took over as the interim head coach before being officially named head coach.
Lane Kiffin's tenure at USC lasted from 2010 to 2013. If I can't rely on these AI systems for the simplest low-stakes tasks, what good are they for?
They're good for making CEOs rich(er)?
(Not trying to be sarcastic, this is genuinely the best reason I can think of)
Edit: made a change for completeness.
I gave this a tough test, specifically generating a quiz about the video game football manager, and it failed quite badly.
It would either give 3 absurd/joke answers and one good correct one. Or give 3/4 correct answers and then insist that only one of them was correct and the others were wrong.
It's probably only accurate to specific topics and still requires alot of human validation and input. To me that makes it not that better than asking an AI for suggestions conversationally. Probably worse since you can make more progress and adjustments conversationally.
No demo without signup? No video either …
Also no feedback.. :)
We built Wiyomi after struggling to stay consistent with language learning apps. Most tools felt either too gamified or too generic. So we made something simple: type a topic (or paste a book’s table of contents), and get AI-generated quizzes instantly.
You can track progress, set reminders, and actually build a habit.
We’d love your feedback, try it out at wiyomi.com!
Have you published your methodology? From what sources are the quizzes based? How do you ensure correctness? Etc
Are you aware of LLM “hallucinations”?
Pasting the table of contents for a book that the LLM has no access to will lead to a high rate of hallucinations
Hi all thank you for the honest feedbacks, This MVP will solve your problem poorly but your honest feedback will help us make this MVP much much better and works!
Wrappers be wrappin'
seems pretty useless to try to learn from a quiz written by a bullshit generator that has no idea (or care) about whether it's got anything right?