by Mike Shea on 8 January 2017
9
If we distill down the 14,241 elements of data I captured using my Lifetracker app over the course of 2016 and ask a single question: "Hey Mike, how was your 2016?", the number above is the best single-digit representation. How was my year? About a 9 on a 1 to 10 scale. Of course, that's a pretty abstract way to look at nearly fifteen thousand points of data.
Over the past year, as I did in 2014 and 2015, I tracked my life using a web application I wrote called the lifetracker. Every day, at ten in the evening, I score six daily goals I've determined are important to my life on a 1 to 10 scale. These include Create, Relax, Love, Befriend, Health, and Happiness. I also track various daily activities in key and value pairs like "Movie: Rogue One" or "Code: Python".
All of this gets stored in a database which outputs a comma-separated-value (CSV) file that I process and analyze with R and Python to see what the last year of my life looked like in data.
The following chart shows my daily score for the top six categories I choose to track for my life on a 1 to 10 scale: Create, Relax, Love, Befriend, Health, and Happiness.
That's 2,172 points of data in one chart. We see big trends and specific dips. Even though my overall median score for "Happiness" was a 9, we can see four big drops throughout the year. One of those I bet you can guess. The other three are more personal than I plan to discuss here. Bad days are REALLY bad but most days are pretty great.
On a 1 to 10 scale I consider a 7 to be the lower limit of a good day for any given goal. If we take the scores of the six goals and look at days that measured 7 or better on an event plot, we get the following chart. Each day a category scored 7 or better it's marked with a vertical tick mark. Stretches of consecutive days look like a big blue bar.
Create, Relax, Love and Befriend all scored 7 or better on more than 95% of days in 2016. I was pretty happy on 92% of the days. Health, however, is still the outlier, but not nearly as bad as it was last year (8% in 2015 compared to 75% in 2016). That's a huge leap up. There's a story there too but not one I plan to share here. A careful observer might notice that I didn't record anything on four days out of the year.
I record keys and values for many of my daily activities. Here's an event plot of those activities. Each tick mark shows which day I conducted that activity. I cut off activities I did less than 10 times over the year. This chart displays 5,529 points of data.
Because I record daily activities as keys and values, I can analyze the values to see which days I did which specific activities. For example, which video games did I play over the year? Which TV shows did I watch? Which books did I read? Here are those charts.
Biggest creative accomplishments: publishing Sly Flourish's Fantastic Locations and running the 2016 Dungeons and Dragons Dungeon Master Survey.
Favorite coding package: Python Anaconda.
Favorite Video Games of 2016: Inside and Dark Souls 3.
Favorite TV Shows of 2016: Jonathan Strange and Mr. Norrell and Stranger Things
Favorite Books of 2016: Lies of Locke Lamora, Fifth Season, Monstress, Between the World and Me
Favorite Movies of 2016: Arrival, Rogue One, Sicario, 10 Cloverfield Lane
Each day I recorded around one to three "Thinking About" tags for a total of 1,040 total "thinking about" values. These are like one-line journal entries about whatever is in my head at the end of the day. For example:
Thinking About: New D&D announcement for Volo's Guide and Giants module
After some text processing I combined lists of top words, top bi-grams (word pairs), and top tri-grams (word triplets) into a single list. These give a good gauge of the main topics I thought about in 2016. This list is rather subjective. I cut out a lot of words that don't mean a whole lot on their own and removed some topics for privacy.
Top "Thinking About" Topics: Michelle (65), Trump (58), D&D (52), work (49), game (38), Fantastic Locations (38), art (29), good (28), fun (21), data (20), great (20), D&D game (20), survey (20), bullshit (20), life (20), meeting (19), Malevolent Arcana (18), working (18), worried (18), enjoying (18), writing (17), coding (17), pain (16), adventures (15), friends (14), R (14), George (13), Starbucks (13), DM (13), Mom (13), Python (13), hard (13), adventure (13), playing (13), wrote (13), ElasticSearch (13), Fantastic Adventures (12), coffee (12), lunch (11), Strahd (11), creative (10), write (10), world (10), AI (10), election (9), Kickstarter (9), awesome (9), finished (9), boss (9), dinner (9), people (9), DM survey (9), code (9), Hearthstone (9), walk (9), vacation (9), love (9), Origins (8), Dwarven Forge (8), WoW (8), text (8), customer (8), friend (8), future (8), crazy (8), fight (8), feeling (8), games (7), family (7), visualizations (7), birthday (7), podcast (7), Oculus Rift (7), Apple (7), died (7), management (7), encounter (7), presentation (7), busy (7), data analysis (6), vampires (6), book (6), relaxing (6), graphs (6), wedding (6), Universal (6), article (6), learning (6), science (6), campaign (6), wonderful (6), Dark Souls (6), Macbook (6), encounter building (6), Arcana Rising (5), Storm Kings Thunder (5), Disney (5), iPhone (5), conversation (5), 13th Age (5), Harry Potter (5), friendships (5), Pokemon (5), combat (5), president (5), clustering (5), Sly Flourish (5), innocent (5)
Using the bi-grams found in all of my "Thinking About" tags, we can create a "word graph". This graph shows all of the interconnections between word pairs that appear twice or more so we can start to see themes. It's a pretty effective way to get the gist of the year without having to read a thousand sentences. It also beats the shit out of word clouds because it actually shows the interconnections of words, not just a gross representation of frequency.
I've enjoyed both capturing these data each day of my life and spending time on the analysis. A data-driven study like this has helped me reinforce the things that are most important to me. Tracking this stuff every day has helped me build in daily habits that support these goals. Tracking these things helped me accomplish these things.
The system I have in place to do this won't, however, work well for other people. This whole thing is built on a nest of Python, PHP, R, and a bunch of other pieces.
If someone is interested in tracking daily habits I recommend the iPhone app Streaks. It helps you track six things each day, a good low number for habit building. Streaks has a simple user interface and you can export the results into CSV so you can do your own analysis in Excel. It won't track keys and values, though, which is too bad since I've gotten so much out of that, but it's a great way to focus on six specific habits each day and track the results.
Many people look at the new year with a set of big goals they want to achieve. The more I've focused on getting things done, the more I've come to realize that big goals aren't as important as small daily habits. Tracking daily habits has given me a lot of insight into what my life looks like at both a micro and macro level. Tracking these daily goals has helped me accomplish these goals and manage my expectations. It's also a lot of fun to do.