Back in 2017 some folks at Verizon (most notably Lori Croushore and Alla Reznik) were willing to go out on a limb and hire me (and Karen Tuttle, my partner in crime) to help launch their first foray into AI-driven contact center automation. It was a great experience: doing build-vs-buy-vs-partner analysis was enlightening, the project had lots of energy, and we eventually landed on a partnership with an exceptional team at Astute (now Emplifi).
During that first year at Verizon, I quickly realized how much I was in over my head trying to even evaluate AI technology. At the same time it was obvious that the whole AI / ML scene was set to explode, and I felt pretty foolish for having not paid attention to it earlier (except for an all-too-brief fascination with neural nets back in the early 90s; one always wonders at how differently life could have turned out). The technical nature of AI also caught my interest; it had been several decades since I’d done anything requiring technical chops, and suddenly I felt an urge to understand this rapidly evolving field.
In mid-2018 I decided to get serious about learning this stuff. I have some experience with the academic process, so I decided I didn’t need to do anything formal. Instead, my evenings and weekends have been filled, to the extent possible, with a self-directed foraging expedition through the AI / ML landscape, driven entirely by what catches my attention and engages my interest.
And while I’ve learned a lot these last 5 years, I felt like I was missing the intensity of doing this full-time. I had a growing frustration that this new passion of mine was relegated to the sidelines. (Is this in part a yearning for the halcyon days of grad school, when I could spend a couple of months digging into a gnarly problem just because it looked interesting? Hmm.) Slowly over the course of the last couple of years I’ve been convincing myself that I should devote my attention more fully to the field.
So the bottom line is that my departure from Verizon was not about any issue there, but arose from a desire to really try my hand at this AI thing. I’m at an age where I probably don’t have many more wholesale career changes left in me; if I don’t give this a proper chance now then I’ll forever regret it.
My wonderful wife is the financial manager in the household, and being fiscally conservative, gets nervous if I talk about retiring so I can pursue a kind of citizen scientist hobby-ship full time. In a compromise, we’re considering this a 1-year sabbatical in which I get to figure out where my work life goes next :)
So where is that? I feel rather like the chipmunks I see darting around on my deck in these warmer months. “Hey, look over here at this cool paper on spiking neural nets!” “Wait, wait, check out this new approach for quantizing LLMs” “Oh, come over here, predictive coding is the way to machine intelligence!” For maybe the first time in my life I feel like I’m having a real problem focusing on just one thing - there’s still so much to learn, and more is flooding in every day!
The current activity around generative AI—LLMs, text-to-image systems, and the like—is very interesting, and I’m going to keep an eye on that world, but I agree with those who think that monolithic, fully-connectionist systems are not going to get us all the way to high-level machine intelligence. My plan right now is to dig into areas that are a bit further afield:
- I’m becoming convinced that multi-modal streamed inputs together with predictive coding and attention mechanisms will be required for internal representations and generative world models that enable factual grounding and causal reasoning (together with reasonable sample efficiency).
- It seems clear that there is a lot to be learned from our rapidly expanding understanding of the brain and how we think. I would love to dive into topics at the intersection of neuroscience, cognitive science, and AI: spiking neural networks, neuromorphic computing, biologically-plausible alternatives to backprop, and the like. I can imagine mimicking the repeatable units of the brain (say the minicolumns in the neocortex) and using self-supervised learning to see what population-level properties and behaviors emerge.
- Current AI systems give us plenty of cause for concern about bias, misinformation, misuse by bad actors, automation of weapons of war and other ethical issues. If you take existential risks off the table, though, I’m generally of the opinion that the benefits will outweigh the harms (and to some extent, it isn’t realistic to stop this development, so we need to focus on steering it in good directions). What I’m more concerned about is whether there is an existential risk from intelligent machines. There are lots of smart people on both sides of this issue, but so far I’ve not seen any persuasive argument that we shouldn’t be worried (and I’ve read quite a few). Maybe alignment work is in my future?
As I noodle my way through the next year, I’ll make some effort to be more publicly visible than I have been in the recent past. I’ll find a place to do updates of some sort, and one of my main goals is to put more on Github that I’m willing to share publicly.
Sorry for rambling, and if you read this far, thanks for the time! More to come.