I put my whole life into a single database

(howisfelix.today)

180 points | by lukakopajtic 3 hours ago ago

63 comments

  • brodo 2 hours ago

    The takeaways at the very bottom of the page are valuable:

    > Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.

    The whole "quantified self" movement might be more about OCD and perfectionism than anything else.

    /edit: quantified, not qualified

    • noemit 2 hours ago

      A counter example:

      I've been wearing an Apple Watch for close to 10 years. I've tracked my weight as well along those years but nothing crazy like OP. The Apple watch tracked plenty.

      I had some strange symptoms and two doctors insisted I had a weak heart and potential heart failure. This was shocking! Turns out I do have a really "weak" rhythm, but heart failure is when your heart is progressively getting worse in it's pumping. I don't even remember which metric he looked at in my Apple health - but basically my heart has always been this way. A doctor looking at a single data point might think I have abnormally low blood pressure/heart rate, but if I've had this for 10 years with no change, the medical assessment is very different - it means nothing. Sometimes boring data is exactly what you need. For this reason, I will probably always wear an Apple watch (or equivalent) moving forward.

      Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.

      • nkrisc an hour ago

        But you didn’t spend hundreds of hours on it, so when it did happen to be useful it seemed like an outsized benefit.

        I would wager that for most people, most data about themselves will be useless and not worth collecting.

        Of course you can’t know what data will be useless or not, so unless the cost of collecting it is minimal or nil (wearing a smart watch, writing down your weight each day/week), it’s probably not worth it.

        Spending hundreds of hours to build a solution to capture all data about yourself to find interesting patterns has a huge assumption baked into it: that there are interesting patterns to find.

        • cj 36 minutes ago

          Like anything else, I think it comes down to having a good use case.

          I've gotten deep into weightlifting/bodybuilding over the past couple of years, and that's the kind of hobby where micro-optimizations and data tracking can have a pretty big impact on results (and sort of necessary, you can't fly blind with things like diet, especially)

          E.g. I track and weigh everything I eat, take body measuraments on a weekly basis, Dexa scans every few months, etc - for me it's worth it because I know what I want to do with the data. If I didn't have a goal, all that tracking would clearly be overkill.

      • austy69 25 minutes ago

        > Data can feel useless for 10 years until one day it becomes critical. The benefit is spiky and uneven.

        Not sure if in your case the data was critical, since the doctor likely would have just had you wear a monitor for a while after to come to the same conclusion.

    • stevekemp 2 hours ago

      I had a similar epiphany a few years back when I started wearing a step-tracker/sleep monitor.

      It was kinda interesting to see how many times I woke up, or track hours, but to be honest I realised after a few months that when my tracker said "You had good sleep", or "You had bad sleep" I was already aware - I woke up smiling, or grumpy depending on how I'd done.

      I didn't ever look at the data and think "I want to go to bed now to catch up on the four hours I missed yesterday". I continued to have mostly consistent hours, but if I was doing something interesting I'd stay awake, and if I was tired I'd go to bed earlier naturally. The graphs and data wasn't providing anything of value, or encouraging me to change my behaviour in any significant way.

      • revolvingthrow 3 minutes ago

        Eh, I found several interesting things from various tracking tools. Take a nap? Sleep is destroyed this night. Exercise in the evening? Same. Not something I’d pay attention to without noticing the chart afterwards.

        There’s also the motivation factor. I’m not sure of the total %, but I certainly did some exercising just to fill the daily goal. Nothing life-changing, but for the price of a cheapo apple watch se once every 5 years or so, more than worth it.

        It’s not unlike simplistic time tracking on my iphone. I spent a lot of time on bullshit websites. Obviously I knew it was happening, but the sheer magnitude was surprising. It’s akin to acute pain letting you know there’s a health problem vs something brewing in the background that you are vaguely aware of, but have no motivation to truly care about - one is far more noticeable than the other

      • oldandboring 2 hours ago

        Same. I had a Garmin for about 6 months and I eventually just stopped wearing it and sold it. Knowing how many steps I took today, checking it several times a day to see if I was meeting my goal, knowing how many vertical feet I skied.. none of this data ended up meaning anything to me.

      • koliber an hour ago

        Being aware, and being aware that you are aware are very similar things that are subtly different.

        I was aware that alcohol affects your next day, even a little. That's because people always say that alcohol is bad for you (surprise surprise). I heard this, so you could say that I was aware. I generally thought about this as "a hangover is bad for you." and was somewhat dismissive of the "even a single drink has a bad effect" mantra.

        I did some experimenting, and slowly realized that even a single drink can indeed have an impact on the next day. It's not a hangover, but an impact that I could feel nonetheless. I needed to do some light stats and a lot more journaling to build this awareness. I am now aware that I am aware.

        • andsoitis 29 minutes ago

          > an impact that I could feel nonetheless. I needed to do some light stats and a lot more journaling to build this awareness.

          If you could feel it why beed the stats?

    • vidarh an hour ago

      For me it's about managing ADHD-like (never diagnosed, and I don't care to assume) patterns, coupled with self-accountability. I have a self-improving dashboard (it kicks off Claude to propose additions based my positive/negative feedback on past attempts, and then builds them) that feels quite helpful. E.g. one of the things it added was fitbit integration that shoves my step count and resting heart rate in my face every day, and it's helped me drive my step count back up towards where I want it to be.

      I do think it's not worth spending a whole lot of time on, though - hence why the first thing I did was add that mechanism to have Claude build it for me, with me mostly glancing at a plan and saying yes/no. It's the perfect thing to vibe-code - if it breaks, I revert a commit and it doesn't matter because nobody depends on it but me.

      • andout_ an hour ago

        It's important to tell the readers how long you've been doing this - especially to those that also manage ADHD or ADHD-like symptoms.

        Why? Because those individuals tend to spin something up, tell everyone about it (online, and offline) and then stop doing it few days later.

        The result then ends up being a false signal for others in the same boat. People who read it, feel a spark of recognition ("someone like me actually figured this out"), and then invest real time, energy, maybe money, into replicating something the author themselves quietly abandoned two weeks later.

        Just a small heads up from someone who used to get burned in the past :)

        • vidarh an hour ago

          I have data going back many years, but this recent effort is a few months old at this point. It's however notably an iteration that has reduced the amount of time I spend on collating and reviewing data, by automating away most of my previous manual effort, including most of the coding, and so I do suspect I'll stick with this for a very long time. A significant part of the prompt to the Claude part of it is to focus a substantial portion of plans on how to automate little things that costs me time, and it's doing a decent job at that.

          I've absolutely not figured it out, but I now have an agent throwing stuff at the wall (with guidance from read access to e.g. my journal and a few other data sources) to figure it out for me, and it's gotten steadily better.

    • fxwin 13 minutes ago

      extensive tracking of self-related metrics to improve ones health is the equivalent to reading tons of self-help books to improve ones life/social skills/...

      We already mostly know what makes people happy/healthy: personal connections, physical activity, healthy diet and some sort of purpose/goal in life that goes beyond day-to-day activities. The problem is that these things generally require (hard) work and can be unpleasant sometimes, so humans do what humans do and spend unreasonable amounts of time doing the more pleasant things such as reading and gathering info rather than applying these and what they already know. (That's not to say that a project like this can't be fun or lead to insights, especially across longer time spans, but i feel like all of the questions in the first paragraph have fairly obvious answers if you know yourself at all, that don't require extensive tracking of stats to get)

    • Bullhorn9268 12 minutes ago

      I too learned this the hard way. I still haven't figured out why this is the case - like, is the inherent incidental irreducible complexity of human life too high, so it dominates majority of our actions?

    • nicbou an hour ago

      I have built a much more limited version of this that combines my journals, sketches, photos, geolocation etc. I found it useful for noticing patterns in my behaviour and in my irrationality. Journaling helped the most, because recording my feelings means acknowledging them and reviewing them.

      Above all, it's just interesting. I enjoy reading about the day-by-day progression of a crush or my brutally honest feelings about a trip that produced stunning pictures. It weaves nuance into my history.

      • andsoitis an hour ago

        Yours is qualitatively different because you’re not quantifying your life.

        A good thing.

    • flexagoon 11 minutes ago

      > The whole "quantified self" movement might be more about OCD and perfectionism than anything else.

      Quantified Self, at least in the intended form, is focused on testing specific theories, not on collecting large amounts of data and trying to find something interesting in them.

      See, for example:

      https://gwern.net/zeo/zeo#what-qs-is-not-just-data-gathering

    • coffeefirst an hour ago

      Yep. Same goes for elaborate note-taking and productivity systems. The simpler, the less bookkeeping and cognitive overhead, the better.

    • alexfoo an hour ago

      Sometimes it's the journey not the destination.

      I did something similar to pull data from my Garmin watch. This meant writing all manner of code to pull data out of FIT files (interesting and often infuriating self-describing file format), coming up with schemas to hold that data to make it queryable, adding visualisations, performing analysis, pattern matching, etc.

      The end result is nothing really useful, I had a bunch of scripts that semi-automated some jobs that would have taken 1 minute to do manually and only ever needed to be done a max of five times a day, but I learned a load of things along the way. Often these were useful lessons that can be applied to many other things when developing software.

      In a similar vein I've gone to lots of trouble to build a cooling system for my homelab rack (ESP32 to control PWM fans, Dallas 1-wire for reading temp/humidity, exposing measurements as metrics for scraping/observability, designing things to deal with the different voltages involved, etc). I could have just gone and bought an off-the-shelf solution from AC Infinity and installed it in minutes but where would the fun in that be.

    • a3w 38 minutes ago

      *quantified self

    • behehebd 2 hours ago

      I wouldnt have minded if I kept a simple daily journal with a photo a day. If I had that now with LLMs I could ask it "what year did xyz happen".

    • Noaidi 22 minutes ago

      If you are not sick, tracking your life like this is, like he said, is useless. Enjoy your life!

      On the other hand, if you are sick like me, charting your long term heath data from doctors visits and photographing skin issues can lead to great discoveries. I have been diagnosed with Erythrocytosis and a susceptibility to mycobacteria infection which caused pulmonary nodules and skin lesions. Only after showing my data collection to my doctor. Since I have mental illness they constantly over looked my physical issues so I needed hard data to convince them of my ideas.

      For those curious, I have an minor IL12B deficiency and a partial immune deficiency leading to mildly elevated levels of DexoyATP which is partially corrected with zinc supplements.

    • swarnie 2 hours ago

      This is disappointing.... Last year i noticed large chunks of my life were being monitored via many spreadsheets and i endeavoured to bring them all together in to one Oracle DB. My plan was to eventually put some ui and graphing on top very similar to OP but seeing this is making me think twice.

    • cyanydeez 2 hours ago

      The whole "this is a movement" thing might be more about mental health issues than anything else.

  • jirigalis 3 minutes ago

    Wow, this sounds amazing! I am a statistics and chart lover, I love tracking various types of data, but this? I can't even imagine how much time it must have taken to input all the data. Huge respect to you! Keep going.

  • ej31 5 minutes ago

    The value rarely shows up where you expected it to.

    I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.

    Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.

  • ismailmaj 2 hours ago

    In my experience, tracking objective things like "nutrition" and "sleep hours" is immensely useful to reflect on what went wrong, and tracking subjective things like "mood" or "stress" is useless given hedonic adaptation or heavy swings that make problems obvious, and not need tracking.

    What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.

    I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.

    • nicbou an hour ago

      I think it's good to track mood swings, because it makes you notice them. After a while it makes you call out your own BS.

      • Noaidi 3 minutes ago

        As someone with Schizoaffective Disorder Bipolar Type, if you are not diagnosed with a mood disorder, tracking "swings in you mood" when you have no clinical disorder seems like a disorder of its own.

        I have had people tell me they were "manic". Then I showed them videos I took when I was manic and they see what I mean when I tell them they are not manic.

        We have come to a place where we do not want even normal fluctuation in mood, and that is a illness of its own, but it is a cultural illness.

  • lejalv 40 minutes ago

    A simple back of the envelope calculation shows that Felix causes between 70 and 110 tonnes of CO2 emissions per year just from flying.

    Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.

    • alex_duf 26 minutes ago

      Exactly, I stopped reading when I saw the flying stats. There are people who still haven't clocked where our climate is headed.

      I get that you may have to see family abroad or maybe indulge for a holiday, but this is "I'm using an airplane to commute" kind of level.

      And here I am trying to book my train tickets to go to London instead of flying even though it costs three times as much just to avoid a few kg of CO2 (among other things), it's making me angry.

      • vladms 7 minutes ago

        Depends where you take the train to London, it is a much nicer experience anyhow than going to airports and people should consider that as well (ignoring climate stuff)!

        On the price, the very annoying thing is that fuel for planes is not taxed! Changing this would require quite some effort (falls under some specific laws, that are old and nobody wants to touch, etc.) but I think everybody should just ask "honest tax on fuels!" as this will make less people say (or thin) "but climate change is a hoax". Planes are just unfair competition to other transport due to taxes!

      • criddell 14 minutes ago

        > And here I am ...

        Reminds me of the soggy straw memes floating around now. I've been having those why bother? thoughts as well.

    • glemmaPaul 28 minutes ago

      yeah America treats flying as like taking a bus, so I am not at all surprised

      • andsoitis 27 minutes ago

        > yeah America treats flying as like taking a bus, so I am not at all surprised

        What does this have to do with Felix?

    • gardenhedge 12 minutes ago

      Isn't this a drop in the ocean? Why would any 'normal' person forgo flying? How much CO2 emissions have 'world leaders' produced going to summits, or Taylor Swift and her fans flaying to concerts or war flights?

      • fxwin 6 minutes ago

        Why do anything for the greater good at all then? (Also there's a big gap between "forgo flying" and "fly every 2 weeks for 7 years")

  • hamasho 11 minutes ago

    Why don’t you just query Palantir DB by your human ID? It shows your entire life data and much more.

    • Noaidi 2 minutes ago

      That made me laugh, but it was a sad laugh, because it is true.

  • terabyterex 11 minutes ago

    > Back in 2019, I started collecting all kinds of metrics about my life. Every single day for the last 3 years I tracked over 100 different data types - ranging from fitness & nutrition to social life, computer usage and weather.

    I know this is the type of person i would not like.

  • cafkafk 2 hours ago

    I get that everyone wants to be cynical about this, but you really can't deny that both the visualization and sheer scale of data is impressive. The way the "my life in weeks" is done is also very cool, I'll be stealing that for myself.

  • alexpadula an hour ago

    I thought you created a database from scratch, got me excited! (I’m a db guy)

  • BirAdam 2 hours ago

    This was far more interesting than I first thought it would be when clicking the link. In particular, the place/time and life events and such being presented this way told a story and was fun.

  • sbcorvus 21 minutes ago

    This looks like it requires a heavy amount of discipline to track everything consistently over time. How do you build that into your daily routine?

  • pwndByDeath 2 hours ago

    An interesting experiment, I think I'm too uncomfortable leaking data I don't yet know why someone would curate to me free of charge until I knew. If there was a FOSS suite like Home Assistant that would do a few of those things I might try it out, especially the weather (I would add air quality) correlation to mood and other subjective states.

  • minkeymaniac 32 minutes ago

    The step count in NYC stands out like One World Trade Center compared to the rest of the building when looking at the skyline ;-)

  • tap-snap-or-nap 17 minutes ago

    No need to spy on him, he publishes everything.

  • Tacite 40 minutes ago

    I'd like to make the same but with Owntracks instead of Swarm and ActivityWatch instead of RescueTime.

  • faeyanpiraat 8 minutes ago

    It's all nice and all, but I'm just sitting here thinking "How can one afford all this flying around"?

  • leke an hour ago

    I had an idea similar to this where you could add information about yourself and answer daily questions and get paid by companies who access this data. This could be an ethical way to share information benefitting all parties.

  • StefanJVA 2 hours ago

    I wonder how much time you spend daily on tracking things / data entry

    • aitchnyu an hour ago

      Waiting for a neckband/choker wearable that can take transcription from my vocal cord nerve, so I can operate a chatbot silently. Yes, scientists have made impulse-to-speech lab equipment.

    • behehebd 2 hours ago

      Just check one of the pie charts and it'll tell you!

  • lokimedes 2 hours ago

    Taking “Know thyself” to a whole new level. I’d love to have these stats on me, if it could be done by inference, rather than conscious effort.

  • nottorp an hour ago

    Hope they made backups :)

  • tymscar 2 hours ago

    So it didnt end up working too well seeing that the latest data is from 4 years ago.

  • tomovo an hour ago

    Stop bragging, mine fits into a csv file.

    • corndoge 10 minutes ago

      There are exercises and supplements you can take that improve things slightly

    • whobre an hour ago

      To be fair, CSV files can be huge nowadays.

      Don’t ask how I know…

  • TutleCpt 2 hours ago

    Yeah, we've all put our whole lives into a single database. It's called the United States Government.

  • __mharrison__ 2 hours ago

    This might sound harsh but for someone who is keen on investing time to track so many things, you should invest some time in learning how to make better visualizations. A few tweaks here and there would really improve what you have.

  • egeres an hour ago

    Related to this, I highly recommend anyone to install github.com/ActivityWatch/activitywatch, it's an amazing tool to keep track of your computer use completely locally. I think there are lots of possibilities with data analysis/AI aimed to improved one self's life