Throwback Thursday: Journalling to Make Progress on Really Difficult Problems

This article first appeared on Thesislink in July 2020.

I’ve been grappling with a Really Difficult Problem (RDP) on a project lately. You know the kind: necessary to deal with in order to make progress, but frustratingly complicated and hard to solve.

RDPs are a familiar part of the research journey. As researchers, we often face Really Difficult Problems because that’s part of generating new knowledge. It’s literally our job to do the tricky math; to find the hidden solutions; to articulate the complicated points; and to overcome challenges all the time. That’s part of the reason why our work is valuable.

But it also makes our jobs rather challenging. Because usually, if someone has a problem at work, they can Google the answer or ask around with colleagues. For researchers, it’s not so simple. You can’t Google a problem if you’re the only person working on it! And though colleagues or supervisors will often have helpful insights, they may not be able to help if your problem is something that you already know more about than they do.

In the case of my current Really Difficult Problem, I am the only person who can solve it. The exact details of my particular RDP are exceedingly boring, but suffice to say it’s a complex technical issue that I need to figure out in order to succeed with a project. Solving this latest RDP has been a recurring entry on every to-do list I’ve written for the past 3 months… and for every day of those 3 months (until recently), I’ve made exactly zero progress.

My first instinct with RDPs is to do everything else except solve the problem. I tend to put them in the proverbial ‘too hard basket’ and get easily distracted. Sometimes I do other work that’s easier, and gives me a sense of accomplishment. Sometimes I give up and watch TV – without really enjoying it, because I feel guilty.

But bottom line, I tend to procrastinate on solving RDPs. That’s exactly what I did in this case, for months… until the pressure mounted, and I had no choice but to find a solution.

First, I spent some time staring blankly at the RDP. Unsurprisingly, that didn’t work.

Then, I spent some more time trying the exact same ineffective solutions that I’d tried previously. That really didn’t work.

I was at an impasse. So out of desperation, I tried something left-field: I journalled about the problem.

I pulled up a blank Word document and wrote about:

• what I remembered about the RDP from when it first arose
• the techniques that I had already used in my attempt to solve it
• why those attempts didn’t work
• the roadblocks stopping me from trying other solutions

I was, in effect, narrating the problem.

The first couple of pages were a log of frustration. All I was doing was creating a picture of what didn’t work. But soon, my note-taking sparked some more ideas about what could work. I thought of other angles on the problem; other approaches to try. The journalling process helped me to think in a more structured and organised way about the RDP.

Crucially, the journalling process has also diminished the power of my RDP in my own mind. It no longer seems like an insurmountable obstacle. It’s just a problem — a hard one, for sure — but I have pages of notes guiding my path toward solving it.

I haven’t solved my RDP just yet, but I’ve made inroads. I have more potential solutions to try, and I have a lot more confidence in my ability to get past this difficult phase in my project. And now I’m kicking myself; because I made more progress in one day of journalling about the problem than I did in 3 months of ignoring it.

What are your techniques for overcoming RDPs in your research? Tell us in the comments!