Your AI Second Brain and the pitfalls of productivity tools

How AI enabled thinking might look like

In the summer of COVID - the summer of productivity apps like Roam Research - I found myself researching productivity systems. I’ve always had trouble remembering things - all the very interesting things I read, I forget. I thought with a second, silicon brain, I’ll be able to remember things. Work related context spanning many projects and many people. Tidbits of philosophy. Snippets and quotes from my favorite books.

I spent many hours on Productivity YouTube to build a productivity system. The first step in any productivity system is to capture anything interesting that comes your way and file it in an inbox. At the end of the week (usually on the weekend), you sort through your inbox to ensure everything is filed correctly.

After I had a basic productivity system going (using org-mode), I tried maintaining a capture → organize → review flow. I burned out in a month straight while being extremely unproductive. Oh sure, I’d built a great knowledge-base. But this is probably the most unproductive month I’ve ever had.

Why did I burn out though? Turns out, above and beyond the initial setup costs, maintaining a productivity system requires an immense amount of effort. You’re constantly trying to decide what is important before capturing it, which breaks your flow. Even if you have a great filing and tagging system, you still have to spend time (usually on weekends) parsing items in your inbox, reformatting the items to fit within your specific mold of an information management system. You also need to periodically (again, usually on weekends) review information from the past to keep it fresh in your brain.

To me, this is a ludicrous amount of work! It’s a constant uphill battle to stop things from decaying. The low grade stress of being on the hook for capturing, filing and reviewing information just gets to you. 

Can you do this flow? Oh, sure. There are two likely outcomes. Either, you burn out on productivity systems and revert to something simple, like Apple Notes. Or, you become consumed with the process of hoarding information. You spend hours admiring this castle of information that you've built, never to use it, to live in it.

Few people can find a happy medium. I know no one who has. Most people use rudimentary, lossy tools that get them most of the way.

Productivity workflows need to be in the service of something – to help you do your job more effectively and with less stress, to help you learn something new, to help you research and perhaps write something. Something. The moment you start feeling like you're admiring your productivity system and your pile of notes too much, that's your warning sign. Step back and reassess.

So, what is the solution? Let's discuss the present.

For most people, a simple daily note system works well enough. Create a Google Doc each month, pin the tab, and dump everything into it each day under the date’s heading. When you need to search for something, you can usually remember the month when you saw something, did something. Open the corresponding month, visually scan or use the Cmd+F function to find what you need. Additionally, use a universally accessible calendar, such as Google Calendar/Google Tasks or Apple Calendar/Apple Reminders, to record everything you do.

Does this mean that building a more elaborate productivity system is pointless? Not necessarily. However, for most people, the effort required to create and maintain a proper productivity system is not worth it. It is highly likely that you will burn out before finding a happy medium. On the other hand, using a well-built and well-maintained productivity system or second brain and extracting value from it can be a downright joyful experience! It’s extremely useful!

So where does that leave us? Most of the effort in maintaining a productivity system has to do with capturing information, parsing it later, connecting it to context in your brain, re-filing/organizing/tagging it. The searching and finding bit is usually easy if you have a well-built and well-maintained system.

We now have AI models that understand unstructured text, that understand images and all types of structured data (code, file formats, trees). With such models, do we really need to toil away in productivity hell, trying to maintain an up-to-date productivity system? I think not.

What is our brain truly doing? It interprets signals from our senses as information, determines its importance, and stores it away in our brain for quick recall (though the recall is often lossy). The more we recall and use a piece of stored information, the more unconscious the recall becomes.

Can we try to do most of this process with technology? I think so. Rewind.ai tries to tackle a part of this problem. Knowledge workers get most of their information through a screen. If a tool can record everything on my screen(s) and also let me quickly recall things that I saw/read/spoke/heard - that will go a long way to outsourcing a productivity system. Capturing everything on screen(s) has been doable for a while now. Ditto for fast search (including semantic search) on large datasets. What has changed is that modern LLMs can look at unstructured blobs of data from multiple sources and be able to derive meaningful value from it. Until now, you needed human "intelligence" to parse, restructure and extract value from such data. Now, an AI model can get you most of the way there.

What should the future look like? I like to imagine a tool which can not just capture but also understand and act on 70-80% of the information a knowledge worker comes across. As an example, imagine chatting with a friend and making an appointment to meet - the tool should see this chat as you're talking and block time and place in your calendar.

With this tool, you can recall things at the speed of thought – Cmd+Enter, type keyword, see relevant knowledge based on your recent context on-device. This tool can take pieces of diverse information, such as text, audio, video, and images, related to what you want to recall and compose it into a comprehensive whole. It allows you to progressively get more detail - like often happens with our brains - about any event, any topic, any thread of inquiry!

Such a tool, such a technology would be wonderful! It doesn’t exist yet but each day, I become more optimistic that such a future is within reach.

However, there are obvious questions that need to be addressed. Will large, faceless corporations have access to all this information? Who will build these tools? Who owns the computing infrastructure? Will this technology make us dumber and deteriorate our memories, or will it free our brains to engage in more creative and intellectual pursuits? I don't have the answers.