Migrating a million words of my life

I believe every day is significant.

Recently I wrote about my renewed interest in structured journaling, centered on the functionally and elegantly phenomenal software Day One. My first goal with this project was to migrate my giant, million-word RTF into discrete Day One entries. I'm happy to report that I've achieved this.

Beginning of my daily journaling in July 2007 as I produced it for the last six years, in TextEdit. Amusing to see me here at the height of my "Truther" phase.

The beginning of my daily journaling in July 2007 as Day One entries


I considered a number of different avenues for migration before settling on a Python script combined with David Bosman's Markdown to Day One importer shell script. Code Academy, RegExOne, and RegExr were indispensable these last few weeks. I cannot emphasize how incredibly powerful and useful Regular Expressions are. A few times I felt like I was "sculpting" my text -- trimming an edge here, pulling a bit out there, making a nick somewhere else -- until I got the result I wanted and compiled it into the script. After banging my head against the table more than a few times troubleshooting my For and While loops, I finally got a script that would comb through my document and split it into the files MDtoDayOne enjoyed. If you know how to code, you'll probably laugh at this incredibly simple result.

Final version of the Python script I created to convert my Log into separate, appropriately named files. I love that it clocks in at exactly 42 lines!

As I suspected, how I named, dated, and separated my entries consistently for the last several years made this programatic splitting possible. Nevertheless there were a couple dozen anomalies across the 961 discrete entries (less than an entry per day due to gaps I'll discuss later). These I simply created manually in Day One after importing.

Next steps? Visualization

I've already been playing around with a few different methods for visualizing the quantities and qualities of my detailed journaling. Right now though this involves me finding and studying examples of related projects. I welcome any suggestions you might have! To get started, here's a basic Wordle visualization of the most frequent words in the file (minus common English words), with size proportional to frequency. Even from this simple graphic you can start to see what's been on my mind over the last several years.

More to come.

Wordle graph of word frequencies, with common English words excluded. Size proportional to frequency. Colors and orientation are aesthetic.