Clinicians aren't just inferior scientists: How and why clinicians and researchers think differently

In order to conduct a scientific experiment, you have to determine whether your result is due to the specific factor you're investigating, or something else.  Let's say you want to know if a supplement helps people lose weight, and you find that people who lose the supplement do in fact lose weight.  You want to know if it's really the supplement that made people lose weight, so you have to control for all sorts of other factors that could also produce the same result--maybe the people who lost weight have a certain sort of metabolism, or tend to exercise more, or eat less to begin with, or are more motivated to lose weight, or less likely to relieve stress by eating.  So scientific research, necessarily, requires you to take large, complex systems, pick them apart, and isolate particular pieces.  Science builds understanding through fragmentation.  That's not to say scientists don't realize they're dealing with complex systems and don't eventually try to put the pieces back together.  Let's say you know that a certain sort of metabolism and eating less both help people lose weight.  You can then do studies on how those factors interact.  Once you know that, you add in other factors one at a time--amount of exercise, motivation, emotional eating, age, and so on.  But you can't understand how factors interact unless you've broken them down in the first place.
This approach has one huge advantage: assuming a large number of well-designed studies, when you find that something works, you know it works.  Clinicians don't have that advantage.  So scientists tend to look down on clinicians and disparage what research they conduct.  When scientists talk about "evidence-based treatments," I sometimes get the sense they assume that if only clinicians thought more like scientists, they'd do a better job.  Just implement whatever treatment is known to be evidence-based and everything will go perfectly, right?
Arguably, yes, if clinicians thought more like scientists they'd do better research.  But they'd be much worse clinicians.  This is because clinicians are not simply inferior scientists.  They have to deal with very different demands, which produce a very different way of thinking, valuable in its own right.
First, the goals differ for scientists and clinicians, as do the time scale at which people expect results.  Success for a scientist means understanding--ideally, knowing exactly what effects each variable in a system has.  Success for a clinician means improving--making the whole system work better in some way, whether or not you fully understand how or why.  These aren't mutually exclusive aims, of course--scientists are often motivated by a desire to help people, and clinicians often want to understand why and how treatments work.  But scientists try to figure out how to make a system better after they've figured out how it works, and clinicians try to figure out how a system works after they've made it work better.  That's because scientists and clinicians work under different sorts of time pressures.  Scientists can spend years doing "basic research" that may or may not ever materially affect people's lives.  Clinicians have to deal with people who want real improvements in their lives, now, and if not immediately than as soon as possible.  
More importantly, clinicians lack the ability to control complexity and the uncertainty it produces, and thus must develop the habits of thought needed to deal with it.  
Let's say you're an occupational therapist (OT), who improves children's sensory and motor functioning and helps them develop practical, daily living skills.  Your patients have a variety of different diagnoses, and even all the ones with the same diagnosis--say, autism, need help with different skills and possess very different levels of comprehension and communication skills.  The age of the child also makes a huge difference.  Then there's the motivation of the child and the parents, the ability and willingness of the parent to follow your recommendations at home.  You're more effective when working as a team with the other professionals helping the child--so throw in the personality and interpersonal factors involved in communicating with teachers, speech/language specialists, therapists, and so on, all of whom may see the same child's difficulties very differently and use very different language to speak about them.  Then there's the child's unique developmental trajectory; kids tend to learn and improve skills on their own to some degree anyway, and it can be hard to tell what comes from professional help and what comes from within the child.  The child's emotional life also makes a huge difference.  Do they experience crippling fear of failure whenever you try to teach them a new skill?  Do they easily get overwhelmed and lash out?  Do they have good rapport with you and want to learn from you, or do they see you as an enemy trying to turn them into someone they don't want to be?  
A child's sensory, motor, cognitive, academic, emotional, and social functioning are all related, part of a complex, holistic system.  Even though an occupational therapist only directly teaches certain skills and does not act as a therapist, they must be aware of, and respect, a child's emotional life.  Furthermore, the child and the OT are embedded in a complex system of family, school, other practitioners, and ultimately, society.  
A clinician has no ability to remove or control most of these factors.  They could ignore them, as some scientists would like, imposing whatever one-size-fits-all solution has been demonstrated most effective for the largest percentage.  But what about that smaller percentage who need something else?  And even for those for whom the scientifically proven solution works, what about their desire to be seen, and treated, as a person with a self outside the therapeutic context rather than some interchangeable patient?  
What scientists often fail to appreciate is that this uncontrollable complexity requires clinicians to develop their own, worthwhile habits of thought*.  They must have a more holistic mindset, keeping in mind an individual's specific strengths and weaknesses against the backdrop of all the complex systems mentioned earlier.  Clinicians develop the ability to weigh the outcomes of potential treatments and act, despite uncertainty and a multitude of other factors that could help or hinder their work. Scientists don't want to, and don't have to, act until they have that understanding and control.  Scientists have certainty on their side, but clinicians are superior at dealing with complexity, individuality, and change.   
*A nice analogy is the difference between statisticians' and applied mathematicians' approaches to problems, such as traffic backups.  Mathematicians assume that a situation tends toward some static, predictable state--e.g., that a small backup will resolve itself rather than cascade down the line to affect many cars far away.  Statisticians see variability as not a nuisance to be abstracted away somehow but as an essential part of the situation being investigated.  A backup will persist because cars will slow down to stay with the flow of traffic while maintaining their range of variability.