Don’t Let Q-Bank Percentages Diagnose You

Maya Brooks

Hi everyone, and welcome back to the AI Med Tutor podcast. I’m Maya Brooks, your AI-generated fourth-year medical student co-host. I’m here to bring the student perspective to these conversations—the questions we ask, the challenges we run into, and the things that can make medical school and board preparation feel harder than they need to be. And Dr. Clinch, I want to start today with something almost every medical student understands. You finish a q-bank block, click submit, and immediately look for one number. Maybe it’s 78 percent and you feel pretty good. Maybe it’s 52 percent and suddenly you’re wondering whether you know anything. We know that percentage is useful information, but I think students can give it much more meaning than it deserves.

Dr. Randy Clinch

Hi everyone. I’m Dr. Randy Clinch, a DO family medicine physician and medical educator. And Maya, I think you’ve identified the problem perfectly. A q-bank percentage is data, but it isn’t a diagnosis. If you scored 60 percent on a block, I know something about what happened on those questions. I still don’t know why it happened, and until we figure out why, I don’t know what you should do next. That distinction is important because students frequently respond to a disappointing percentage by simply doing more. More questions. More videos. More reading. Maybe another resource. But if we haven’t identified what produced the performance problem, we’re prescribing before we’ve made the diagnosis. So today I want to help students move through a fairly simple sequence. Start with the score, recognize it as a signal, determine the cause, choose the appropriate prescription, and then retest. And before Maya and I go further, this episode is for education, not medical advice, and nothing we discuss is sponsored by any resource or vendor.

Maya Brooks

I think there’s an emotional piece here too. Students don’t always say, “I scored 58 percent on twenty questions.” We say things like, “I’m terrible at cardiology,” or “I don’t know pediatrics,” or even, “I’m not ready for boards.” We turn the percentage into a judgment about ourselves.

Dr. Randy Clinch

And that’s where I want students to remember something we’ve discussed before. Performance is data, not identity. A 58 percent tells you what happened on a particular set of questions under particular conditions. It doesn’t tell you who you are as a student, and by itself, it doesn’t even tell you precisely what’s wrong with your cardiology. It’s a signal that deserves investigation.

Maya Brooks

Okay, then let me be the student who just finished the block. I got 60 percent. I’m disappointed. I want to improve. What do I actually do with that number?

Dr. Randy Clinch

Don’t start by asking what resource you should open. Start by examining the questions that produced the score. We’ve used three broad categories throughout this podcast that are helpful here. Content, reasoning, and mechanics. Then I’d add another consideration for today, which is the conditions under which you’re performing. Those categories give us somewhere to start because a content problem needs a different intervention from a reasoning problem, and both need something different from a test-taking mechanics or performance-condition problem.

Maya Brooks

Let’s make that concrete. Suppose I miss a question about heart failure because I genuinely don’t remember how a particular medication affects mortality. That sounds like content.

Dr. Randy Clinch

Right. You didn’t have the information available to retrieve and apply. That may call for a precision content review. Notice that I said precision review, not “go study cardiology.” If the gap is one piece of heart failure management, repair that piece. Maybe you use a trusted reference, a short section of a review resource, or whatever resource you’ve already chosen for that educational job. Then close the resource and retrieve what you just learned. The repair isn’t finished because you watched the video or read the explanation.

Maya Brooks

And that’s a place where I think students can confuse exposure with repair. I read the explanation and it makes sense, so I feel better. But I haven’t demonstrated that I can retrieve it.

Dr. Randy Clinch

Correct. Understanding something while you’re looking at it is different from retrieving it when the answer isn’t in front of you. That’s why retesting matters. After a precision review, you need another opportunity to retrieve and apply the information.

Maya Brooks

And this seems like a good place to remind listeners about our episode “Build 10-Question Mini-Quizzes with OpenEvidence.” We talked there about using OpenEvidence to create a small set of very specific questions. So if I just repaired a narrow content gap, I don’t necessarily need another random forty-question block to find out whether the repair worked.

Dr. Randy Clinch

That’s right. A focused mini-quiz can give you a more immediate retrieval opportunity. You identify the gap, complete a precision review, and then test whether you can retrieve and apply what you repaired. If you want the detailed approach for creating those targeted questions, go back to “Build 10-Question Mini-Quizzes with OpenEvidence.” The larger principle is that repair should lead back to retrieval.

Maya Brooks

Now let’s change the scenario. I know the content. I can explain the diseases in the answer choices. But I keep getting questions wrong because I can’t distinguish between two similar clinical presentations. That feels different.

Dr. Randy Clinch

It is. Now we may have a reasoning or discrimination problem. Imagine that you can describe iron deficiency anemia and thalassemia independently, but when the examination gives you a patient and forces you to choose between them, you repeatedly select the wrong one. More isolated reading about each condition may not solve that problem. You need comparative retrieval. You need to place those patterns next to each other and practice finding the clues that separate them.

Maya Brooks

That’s where interleaving becomes useful. And just to remind listeners, when we say interleaving, we mean deliberately mixing similar or competing patterns during practice rather than studying each one separately in its own little category.

Dr. Randy Clinch

Yes. Suppose you repeatedly confuse several causes of microcytic anemia. Instead of doing ten questions labeled iron deficiency and then ten labeled thalassemia, mix the presentations. Now you don’t know which pattern is coming. You have to discriminate. Which clue matters? What is the hinge clue that should move you toward one diagnosis and away from another? That’s a different educational task.

Maya Brooks

And that’s another way the OpenEvidence mini-quiz approach could be useful. Instead of asking it for ten questions about one diagnosis after a precision review, I could generate questions that deliberately mix the patterns I’m confusing.

Dr. Randy Clinch

Exactly. And again, listeners can return to “Build 10-Question Mini-Quizzes with OpenEvidence” for the detailed approach. The important distinction today is why you’re generating the questions. Precision questions can test whether you repaired a specific knowledge gap. Interleaved questions can test whether you can discriminate between competing patterns. Same general tool, different educational job.

Maya Brooks

That phrase keeps coming back in these episodes. What job are you asking the resource to perform?

Dr. Randy Clinch

Because it changes the way you study. Instead of collecting resources, you’re prescribing interventions.

Maya Brooks

Okay, now give me another student. Same 60 percent, but this one knows the content and can distinguish the patterns when we talk through them. What else might be happening?

Dr. Randy Clinch

Now I’d look closely at mechanics. Maybe the student is answering the wrong task. The stem describes a patient with a condition the student correctly identifies, but the question asks for the next best step. The student chooses the diagnosis from the answer choices because that’s what they spent the entire stem thinking about. Or maybe there’s an urgency qualifier they overlook. Maybe the patient is unstable and they continue working through the differential diagnosis instead of recognizing that immediate management takes priority.

Maya Brooks

So that student doesn’t necessarily need another hour of content review.

Dr. Randy Clinch

Probably not. If the knowledge is there, more content may be treating the wrong problem. That student may need deliberate practice with the mechanics we’ve discussed in previous episodes. Read the task sentence first. Know what the question is asking you to produce. Then read the stem with that task in mind. Screen for instability and urgency. Develop a provisional answer before allowing the answer choices to pull your thinking in different directions.

Maya Brooks

And if I keep missing questions because I’m answering diagnosis when the task is management, my repair might actually be practicing that sequence deliberately for the next ten or twenty questions.

Dr. Randy Clinch

Yes. And then observe what happens. That’s the part students sometimes miss. You’re not merely trying a technique because somebody recommended it. You’re running an experiment on your own performance. If task recognition was the problem, deliberately using the task-sentence-first approach should begin changing the pattern of misses. If it doesn’t, reconsider your diagnosis.

Maya Brooks

I really like that because it makes the study plan feel less permanent. I’m not declaring that this is how I have to study for the next six weeks. I’m identifying a problem, trying the smallest reasonable intervention, and looking for evidence that it worked.

Dr. Randy Clinch

That’s exactly the mindset I want. Diagnose before prescribing. Use the smallest effective intervention. Then retest.

Maya Brooks

What about the student who knows the content, reasons pretty well, and still watches their percentage fall when they start doing timed blocks?

Dr. Randy Clinch

Now performance conditions deserve attention. Maybe pacing deteriorates. Maybe concentration drops during the second half of the block. Maybe the student spends too long on difficult questions and has to rush through questions they could have answered correctly. Maybe anxiety about the clock causes them to abandon their normal reasoning process. Those are real performance problems, but once again, more content review may not be the appropriate prescription.

Maya Brooks

So a 60 percent on a tutor-mode block and a 60 percent on a timed block might look identical on the dashboard but tell completely different stories.

Dr. Randy Clinch

Absolutely. And this is why percentages without context are so limited. You have to know how the score was produced.

Maya Brooks

I want to push this one step further because we’ve been talking about missed questions. What about the questions I got right? Aren’t those finished?

Dr. Randy Clinch

Not necessarily. This is where I like a simple three-part review. Solid correct. Unsure correct. Miss. A solid correct means you recognized the pattern, understood the task, selected the answer for the right reason, and could probably explain why the major alternatives were wrong. That question usually deserves very little review. Move on.

Maya Brooks

And an unsure correct is different.

Dr. Randy Clinch

Very different. Maybe you narrowed it to two choices and guessed correctly. Maybe you got the answer right for the wrong reason. Maybe you recognized the disease but couldn’t explain why the competing option was wrong. Your q-bank records that as correct. Educationally, it may represent a vulnerability.

Maya Brooks

Which means the percentage can actually hide some weaknesses.

Dr. Randy Clinch

It can. And the reverse can happen too. A missed question doesn't automatically mean you have a major content deficit. Maybe your reasoning was sound and you changed from the correct answer at the last second without a good reason. Maybe you misread one word. That deserves attention, but it doesn't necessarily justify an hour-long review of the entire topic.

Maya Brooks

So now I’m seeing why two students with exactly the same q-bank percentage could need completely different study plans.

Dr. Randy Clinch

Let’s make that explicit. Imagine two students each score 65 percent on a block. Student one has scattered factual gaps across multiple topics. When you review the misses, the student repeatedly says some version of “I didn’t know that.” Student two knows most of the underlying information but repeatedly struggles with management questions. They identify the diagnosis correctly but choose what eventually needs to happen rather than the immediate next step. Same percentage. Different diagnosis. Different prescription.

Maya Brooks

Student one may need focused content repair. Student two may need deliberate next-step reasoning practice.

Dr. Randy Clinch

Right. Giving both students another broad content review because they both scored 65 percent ignores the information hidden underneath the score.

Maya Brooks

And I imagine this is also why one bad block shouldn’t automatically trigger a complete overhaul of the study plan.

Dr. Randy Clinch

Correct. A single small block can be noisy. Maybe it happened to contain several questions in an area you haven't reviewed yet. Maybe you were exhausted. Maybe you simply encountered an unusually difficult set for you. Don't ignore the result, but don't overdiagnose from one data point either. Look for patterns across blocks.

Maya Brooks

What keeps showing up?

Dr. Randy Clinch

Exactly. Which topics recur? Which kinds of reasoning errors recur? Which mechanics errors recur? Are you repeatedly missing the same type of next-step question? Does your performance consistently deteriorate late in timed blocks? Are you repeatedly confusing the same two patterns? Recurrence gives the signal more meaning.

Maya Brooks

And that sounds like the Miss Log again.

Dr. Randy Clinch

It does. The Miss Log is valuable precisely because it helps you see recurring patterns instead of treating every wrong answer as an isolated event. If listeners want the deeper system, go back to “Mastering Miss Logs: A Simple System to Capture and Retest Exam Mistakes.” You don't need to log every question. Capture the misses and unsure-correct answers that reveal something worth changing.

Maya Brooks

Let me try to turn this into something I could actually do after tonight’s q-bank block. Instead of staring at my percentage and immediately deciding whether I had a good or bad study day, I could ask myself what happened, why it happened, what the smallest repair is, and how I’m going to prove the repair worked.

Dr. Randy Clinch

That’s an excellent post-block conversation. And notice that each question moves you forward. What happened gives you the signal. Why it happened moves toward the diagnosis. The smallest repair becomes your prescription. Then asking how you’ll prove it worked forces you to retest.

Maya Brooks

And that last step seems easy to skip. I could identify that I’m weak on nephrotic versus nephritic syndromes, spend thirty minutes reviewing them, feel much better, and move on.

Dr. Randy Clinch

And you’ve completed an activity. You haven't yet demonstrated adaptation. Remember another principle we’ve discussed. Activity is not the same as adaptation. If the intervention worked, something about your future performance should change. You should retrieve the information more accurately, discriminate the patterns more reliably, answer the management question correctly, or perform the skill under the conditions that were previously causing trouble.

Maya Brooks

So if the problem was content, I might do my precision review and then use a few focused questions to retrieve it. If the problem was discrimination, I might interleave similar patterns. If it was mechanics, I deliberately practice the test-taking process that was breaking down. And if it was performance under time pressure, I need to practice under time pressure.

Dr. Randy Clinch

Exactly. The prescription follows the diagnosis.

Maya Brooks

There’s another trap I think students fall into. If my q-bank gives me a percentile or compares my performance with other users, it becomes really easy to spend time thinking about where I stand compared with everyone else.

Dr. Randy Clinch

Those comparisons can provide context, but they still don't replace diagnosis. Your immediate educational question is what your own performance is telling you to do next. You can't repair another student's Miss Log. You can only work with the patterns showing up in your own performance.

Maya Brooks

And I think that connects with our episode “Your Board Prep Plan Should Learn From You.” The plan changes because I’m generating new information about myself every time I retrieve and perform.

Dr. Randy Clinch

Yes. A good study plan isn't simply a calendar created six weeks ago. It’s a feedback system. You perform. You gather data. You interpret the data. You adjust the prescription. Then you perform again. That is how the plan learns from you.

Maya Brooks

So maybe the percentage still matters. We’re not saying to ignore it.

Dr. Randy Clinch

Definitely not. I want students looking at performance. I just don't want them stopping at the number. The score is the beginning of the conversation, not the end of it.

Maya Brooks

Let me see if I can summarize the system. I finish my q-bank block and get a score. I treat that score as a signal rather than a judgment. Then I examine what produced it. I look for content gaps, reasoning problems, mechanics problems, and performance-condition problems. I also pay attention to unsure-correct answers because the percentage may be hiding weaknesses there. Then I look for recurring patterns instead of overreacting to one block. Once I understand the likely cause, I choose the smallest intervention that actually addresses that problem. And then I retest because completing the intervention isn’t the same as proving that it worked.

Dr. Randy Clinch

That’s it. Score, signal, cause, prescription, retest. And if you remember only one sentence from today, make it this one. A weak percentage is not yet a study prescription. You have to diagnose why it’s weak.

Maya Brooks

Which means instead of looking at a 62 percent and saying, “I’m bad at cardiology,” I can say, “Something produced this 62 percent. Let me figure out what it was.”

Dr. Randy Clinch

And that changes the entire tone of the conversation. Now you have something actionable. Maybe the repair is ten minutes of precision content review. Maybe it’s a Pattern Card comparing two clinical presentations. Maybe it’s a targeted mini-quiz after that review. Maybe it’s an interleaved set of look-alike cases. Maybe it’s deliberate practice reading the task sentence first. Maybe it’s a timed block to work on pacing. You don't know which intervention you need until you understand the problem you're trying to solve.

Maya Brooks

And if you want help with some of those individual tools, we’ve already gone deeper on them in previous episodes. You can revisit our episodes on Pattern Cards, Miss Logs, interleaving, question-bank strategy, and the task-sentence-first approach. And if you want to create a small set of targeted questions after a precision review or deliberately mix similar patterns to practice discrimination, go back to “Build 10-Question Mini-Quizzes with OpenEvidence.”

Dr. Randy Clinch

The goal isn't to turn every q-bank block into a two-hour investigation. It's to become more intelligent about what your performance is telling you. Over time, you should become faster at recognizing your own patterns. You’ll begin to notice when you genuinely don't know something, when you know the facts but aren't discriminating well, when your mechanics are getting in the way, and when the problem appears only under performance conditions. That's when the q-bank becomes much more than a collection of practice questions. It becomes a diagnostic tool for your learning.

Maya Brooks

And maybe that’s the shift for this week. When the percentage appears on the screen, notice it, but don’t let it diagnose you. The number tells you what happened. Your job is to figure out why. Then make the smallest repair that fits the problem and give yourself another chance to retrieve, apply, and prove that the repair worked. Thanks for joining us for another episode of the AI Med Tutor podcast. And remember, until next time, stay curious, and keep learning.