FlorianUlrich
•
18 days ago
I started studying biochemistry in 1995 and have been actively
involved in biomedical research since 2001, when I started my PhD. I am
currently working at New York University on blood vessel formation in
the brain (using zebrafish as a model organism).
#1 - Scientists
are not Gods. Most of the studies are more like a marketplace of ideas,
they are discussed amongst scientists, and then gradually the "truth"
gets established over a time of 10 - 20 years. This is for basic
research in academia. There are some landmark studies, of course, that
are trustworthy from the get go, of course. But I don't think that's the
vast majority.
#2 - Applied research in industry cannot afford
that uncertainty and will focus on more specific topics that need to be
more rigorously testable. No surprise to me that only 11% of results
from academia were trustworthy. Academic researchers are held less
accountable than industry researchers. You get a grant, but the money is
not retracted if your results after five years are somewhat vague or
even wrong.
If a pharmacological company screws up, they lose money or in the worst case, people die.
But
how can scientists get something wrong in the first place? Isn't it all
about truth? In mathematics. maybe, But in biology, you are dealing
with living matter that can behave quite noisy.
So... you test a
hypothesis. You do an experiment and see that your test behaves
different than the control condition. Yay! You then move on and buttress
your initial observation. But in your CHOICE of the supporting
experiments you are already biased, because you are convinced that your
hypothesis MUST be correct. You saw it in the initial experiment, right?
Then, every other experiment adds a bit more bias. Sometimes your
initial observation turns out to be right. But if it was a fluke, it now
becomes just one shaky part of evidence in a great number of other
experiments. No problem at all, you think, and publish. After all,
science is not 100% certain.
In my research field, people used a
specific reagent, called "morpholino", to block the function of a gene.
People published a lot of results from these studies. Now, however,
better and easier methods are available, to block the function of a gene
much more rigorously. And just today, there was a study in a big
research journal showing that 80% (!!!) of those morpholino results can
NOT be reproduced. We all suspected that for years, but now it's
published.
Those results did not concern any human pharmacological
studies, but still... a lot of wishful thinking went into the initial
"morpholino" results.
#3 - Then you hear about "peer review". Did
you know that while the reviewers stay anonymous, the identity of the
research lab does not? So if paper A is sent to a journal by "young
group leader X", but paper B is sent in by "big shot/established
professor X", who will the reviewers be biased against? The bigger the
name, the bigger the authority of the paper.
In addition, research
fields are very specialized. If you know your peers, you can sometimes
pretty accurately guess who - amongst 1 - 3 possibilities - reviewed
your paper, even though the name is not revealed. So... as a reviewer,
you have an incentive of not being too harsh against "established
professor X".
#4 - We scientists are certainly NOT more moral and
ethical than others. If anything, political correctness has entered the
field and people are now concerned with "gender bias" in science etc. So
I am a bit worried that that reduces the number of original thinkers.
And then, the race to get funding has become more and more competitive, i don't think that always brings out the best in people.
#5
- Why do some compounds get studied and others not? In biomedical
research, you aim to place your publications in prime journals, like
"Nature" or "Science". They can chose whatever they want to publish. And
often, the bigger and more established a professor, the more successful
he can place his research in. And that determines the compounds that
are researched.
#6 - Would I approve the grant by a young
scientist if his results would destroy my legacy? don't know how i would
react 20 years in the future. I'd like to believe that I am always
impartial, but I am also only human. I'd probably believe the young
scientists that were the friendliest to me at conferences and were
always so enthusiastic about my work. I hope not, but being positive
with those that treat you well and dont say anything against you is also
a very simple human trait, i think. Some people can be more fair and
impartial than others, but in the end, it always comes down to who your
friends are.
Welcoming those that challenge us is a worthy goal to work towards, but only few people can truly be that generous.
A
couple centuries back, Michael Faraday became a trusted research
assistant to Humphry Davy. When it turned out that Faraday had actually
even better ideas than Davy, Davy started to work against Faraday and
tried to block his progress.
That happens when established
personalities get challenged. There is no reason to believe that "big
shot" scientists are ever more moral than others.
I still believe
science is a great place to exchange ideas and meet incredibly creative
and generous people, but scientists are not more moral than others. They
are like everyone else. Maybe a bit different, because this profession
selects for people with borderline Asperger's syndrome. Great social
skills are less required, while intelligence brings you far.
I
even think scientists are more open to criticism than other professions,
simply because criticizing and rethinking is their Modus operandi, but
as I said: in the end, they are humans like everyone else.
involved in biomedical research since 2001, when I started my PhD. I am
currently working at New York University on blood vessel formation in
the brain (using zebrafish as a model organism).
#1 - Scientists
are not Gods. Most of the studies are more like a marketplace of ideas,
they are discussed amongst scientists, and then gradually the "truth"
gets established over a time of 10 - 20 years. This is for basic
research in academia. There are some landmark studies, of course, that
are trustworthy from the get go, of course. But I don't think that's the
vast majority.
#2 - Applied research in industry cannot afford
that uncertainty and will focus on more specific topics that need to be
more rigorously testable. No surprise to me that only 11% of results
from academia were trustworthy. Academic researchers are held less
accountable than industry researchers. You get a grant, but the money is
not retracted if your results after five years are somewhat vague or
even wrong.
If a pharmacological company screws up, they lose money or in the worst case, people die.
But
how can scientists get something wrong in the first place? Isn't it all
about truth? In mathematics. maybe, But in biology, you are dealing
with living matter that can behave quite noisy.
So... you test a
hypothesis. You do an experiment and see that your test behaves
different than the control condition. Yay! You then move on and buttress
your initial observation. But in your CHOICE of the supporting
experiments you are already biased, because you are convinced that your
hypothesis MUST be correct. You saw it in the initial experiment, right?
Then, every other experiment adds a bit more bias. Sometimes your
initial observation turns out to be right. But if it was a fluke, it now
becomes just one shaky part of evidence in a great number of other
experiments. No problem at all, you think, and publish. After all,
science is not 100% certain.
In my research field, people used a
specific reagent, called "morpholino", to block the function of a gene.
People published a lot of results from these studies. Now, however,
better and easier methods are available, to block the function of a gene
much more rigorously. And just today, there was a study in a big
research journal showing that 80% (!!!) of those morpholino results can
NOT be reproduced. We all suspected that for years, but now it's
published.
Those results did not concern any human pharmacological
studies, but still... a lot of wishful thinking went into the initial
"morpholino" results.
#3 - Then you hear about "peer review". Did
you know that while the reviewers stay anonymous, the identity of the
research lab does not? So if paper A is sent to a journal by "young
group leader X", but paper B is sent in by "big shot/established
professor X", who will the reviewers be biased against? The bigger the
name, the bigger the authority of the paper.
In addition, research
fields are very specialized. If you know your peers, you can sometimes
pretty accurately guess who - amongst 1 - 3 possibilities - reviewed
your paper, even though the name is not revealed. So... as a reviewer,
you have an incentive of not being too harsh against "established
professor X".
#4 - We scientists are certainly NOT more moral and
ethical than others. If anything, political correctness has entered the
field and people are now concerned with "gender bias" in science etc. So
I am a bit worried that that reduces the number of original thinkers.
And then, the race to get funding has become more and more competitive, i don't think that always brings out the best in people.
#5
- Why do some compounds get studied and others not? In biomedical
research, you aim to place your publications in prime journals, like
"Nature" or "Science". They can chose whatever they want to publish. And
often, the bigger and more established a professor, the more successful
he can place his research in. And that determines the compounds that
are researched.
#6 - Would I approve the grant by a young
scientist if his results would destroy my legacy? don't know how i would
react 20 years in the future. I'd like to believe that I am always
impartial, but I am also only human. I'd probably believe the young
scientists that were the friendliest to me at conferences and were
always so enthusiastic about my work. I hope not, but being positive
with those that treat you well and dont say anything against you is also
a very simple human trait, i think. Some people can be more fair and
impartial than others, but in the end, it always comes down to who your
friends are.
Welcoming those that challenge us is a worthy goal to work towards, but only few people can truly be that generous.
A
couple centuries back, Michael Faraday became a trusted research
assistant to Humphry Davy. When it turned out that Faraday had actually
even better ideas than Davy, Davy started to work against Faraday and
tried to block his progress.
That happens when established
personalities get challenged. There is no reason to believe that "big
shot" scientists are ever more moral than others.
I still believe
science is a great place to exchange ideas and meet incredibly creative
and generous people, but scientists are not more moral than others. They
are like everyone else. Maybe a bit different, because this profession
selects for people with borderline Asperger's syndrome. Great social
skills are less required, while intelligence brings you far.
I
even think scientists are more open to criticism than other professions,
simply because criticizing and rethinking is their Modus operandi, but
as I said: in the end, they are humans like everyone else.
