Invoking Experts in Prompts: When Persona Framing Improves Results

I’ve spoken about this elsewhere, but going through my notes it’s interesting how early on we noticed that simply invoking an expert in a prompt could produce a really good response.

I found an old example where I was trying to build a better classifier to explain sentiment and things like that. The prompt was basically: “My machine learning professor told me how to build a classifier to tell the difference between sentence A and sentence B.” Then I gave sentence A as an example like: “refuse to be sick out there, kill this one at a time, LOL Claritin,” or “BF this, that worked very well.” The key point was that I wanted something that could tell the difference between a sentence that says something worked and a sentence that says it didn’t—even when there’s profanity or aggressive phrasing that could make it very easy to misread as “negative” when it’s actually “positive.”

And what I liked was that invoking a machine learning professor didn’t just help a little—it gave significantly better output. It wasn’t just “trying harder.” It gave the model a place in the embedding space to stand, a frame for how someone studying machine learning or natural language processing would evaluate the problem and what they’d call attention to.

That’s really one of the reasons those methods worked: there are many different ways to solve a problem. I can ask a plumber how to solve a leak. I can ask a hydraulicist how to solve a leak. I can ask a chemist how to solve a leak. Those answers will depend a lot on the nature of the leak.

And it’s the same with bigger problems like sentiment. Sometimes I need a machine learning expert. Sometimes I need someone who’s an expert in natural language processing. Sometimes I need a psychoanalyst. That’s what the “invoking an expert would solve this problem” idea was really showing: the persona isn’t just decoration—it’s a way of selecting what kind of explanation, what kind of distinctions, and what kind of solution path the model should try to produce.

In those early-stage prompts—back in summer 2020—you can see the pattern clearly. You had to say, “Hey, this person solved this problem for me, and this was the explanation,” and that served as the invitation to the model to actually solve the problem in that mode.