How We Broke Trust In Science: Emily Kaplan

the highest predictor of whether you're going to have a C-section or not is what hospital you deliver in it's not you it's not your doctor it's the hospital Emily Kaplan has been an investigative journalist a business leader and a passionate advocate for Women's Health today she's calling for a return to the roots of scientific exploration we're one of two countries where you can directly Market pharmaceutical products to Consumers she is the co-founder and CEO of the broken science initiative we're not going to fix these big system atic problems but you can empower the individual to critically think about things and allow them to make better choices for themselves and their family so that's really the goal of the broken science initiative this is American thought leaders and I'm Yan Kell Emily Kaplan such a pleasure to have you on American thought leaders thank you for having me I'm thrilled to be here well you have an initiative called the broken science initiative and this is something I've been thinking about for a long time I mean certainly for the last four or 5 years mhm um there are real problems in what's being accomplished in science today what's being portrayed at science that isn't particularly and even this replicability crisis that John nean pointed out right in his sort of you know huge paper on the topic that a lot of science just simply doesn't pan out you couldn't you couldn't demonstrate it again which is a kind of a foundational issue right right right so so tell me about what's going on we we think of science as the empirical branch of knowledge right it should be where we look most for you know truth knowing we're never going to find Ultimate certainty or ultimate truth but it's the pursuit of that and I think when you look at things like the replication crisis these are really important symptoms of a larger break so we're focusing a lot on medicine but there are systemic problems because I think our feeling is that predictability has been replaced by consensus and so you have sort of this group think model that allows for really easily you know statistical manipulation which we can get into as well as corruption the pharmaceutical industry spent more money lobbying Congress last year than any other industry so I mean that's huge more than manufacturing more than Finance kind of blows your mind and if you look at this spending by the pharmaceutical companies on media you see a very similar Playbook we're one of two countries where you can directly Market pharmaceutical products to Consumers all of this falls under this idea of broken science in the sense that and this isn't political right and so I think people try to move this into the political sphere because it becomes people feel very defensive about it you want to trust your doctor your doctor didn't go to medical school to mistreat you but I think the power structure has been inverted um and so we're seeing all these symptoms and so I think one of the thing Greg glasman who's my partner on broken science and I have both been looking at these problems for about 20 years and so when Co happened and there was this sort of just you know real polarization and disagreement about you know what you could trust or what information was valid I actually think it was sort of a gift because I think it basically brought science to the dinner table in ways we hadn't seen before and a lot of people woke up to this notion of regardless of which side you're on that they didn't really know who to trust and my sense is we're not going to fix these big systematic problems but you can empower the individual to critically think about things and allow them to make better choices for themselves and their family so that's really the goal of the broken science initiative is to expose these problems we have a tremendous amount of scientific misconduct that's going on I mean we lost Harvard's president and Stanford's president within six months of each other to charges of scientific misconduct this is a real issue that I think Americans need to inform themselves on we're going to launch an education Society in the fall kids should understand the importance of asking good questions and challenging Authority in a polite respectful way but the only way that we make progress scientifically or as a society is trying to think of better ways to accomplish things and solve problems and I fear we've gotten away from that and I think our you know sort of the isolation and polarization that we see politically is in it's terrifying I think we lose curiosity about things that we don't we may have confidence we understand but if you're not open to listening to other people's ideas we're all in big trouble and truth is we all want better for our kids than we want for ourselves right I don't care what political party you're you're involved in for my stance health is a root of happiness and so this is a common denominator for all of us and you know the the chronic illness epidemic which you know Bobby Kennedy is taking on in a very thoughtful way and I I do not I said I wasn't going to get political but I do think he deserves a lot more attention um he has you know successfully sued many branches of the government he knows how they work he knows where the corruption lives but the chronic disease epidemic in this country I think is our number one vulnerability and there's a lot of stuff that doctors and patients can learn and do and moms and dads can help their kids in terms of lifestyle changes and getting the sugar out of the diet some sort of like rudimentary things that we don't really think of as medicine because they're preventative but they're predictable and so again like at the base of the broken science initiative is this idea of predictable outcomes as the demarcation between science and not science so I think you've created a kind of a road map here for us for what we're going to do today great okay um because we need we need to talk about predictability absolutely um but so you mentioned a few things you mentioned critical thinking uh you mentioned you know openness to new ideas um you mentioned this distinction between science by consensus versus science by predictability so explain that distinction to me it might not be obvious to everybody sure so I mean I think consensus is just we all agree right let's take a vote there is no voting in science and so that should stop us right there now if we what do you mean there's no voting in science I mean I think a lot of people would say hey look if you get the smartest people in the room and these people will vote it's this way and you then it probably is doesn't that make sense well no there's a Supreme Court case it's actually I think three cases that makes up the dber standard and it determines who can be a scientific you know expert in a court case and two of those tests are you have to have been published in peer review and you have to be accepted by the scientific Community now let's think about that for a second right if you take things like space exploration is that not science a lot of that's top secret it's not been peer reviewed are those people accepted within their scientific Community yes but what about the person who builds some you know rocket and blast it off from their backyard and they hit their Target and then they do it again they're not a scientist because nobody knows that they're doing that so we have these sort of ways of thinking and creating standards or like a litmus test for how we Define science and I don't think that definition is right we have this with peer review profoundly where I think there's a general confusion about peerreview being you know similar to good journalism peerreview is consensus right it's a group of people who have basically decided that if you get a P value that's a you know statistically significant result that that means that you have validated your hypothesis and the way the P value works is that's not right actually you're just looking at the data sets of the null hypothesis and comparing them to the data you gather through the intervention and you're saying is there a relationship between these two right did we prove or disprove the null hypothesis there's no testing the null hypothesis that's an assumption and you're not saying anything about your hypothesis with a P value you're also not saying that you can replicate the work which from the broken science initiative that's the standard you have to be able to replicate your results so that you know that you have a predictable outcome so you take that as a standard to get published you have to have a significant P value you have peer reviewers who are not paid they're told when they get a manuscript to assume that there's no scientific misconduct and that all the the sort of facts and information are right I've had peer reviewers tell me that they also are often told don't comment on the design of the study because the design the study has been concluded so this is fascinating let me let me jump in here as someone who you know was trained in experimental design many decade you know some decades ago I hav't been doing it for a while but it's actually quite difficult to come up with a very elegant design and one thing that I discovered years ago and this actually made me very concerned was that I could really stack things in the favor of getting the a result that I kind of hoped would be the result if I design things in a particular way the design is my point is the design is incredibly important it's very easy for a skilled experimental designer to to help you know the the person that's funding them for example get the idea they want whether even sometimes perhaps unconsciously right right because you know when your funding depends on something you're you're a bit conflicted right right right so part of the scientific misconduct that we're seeing so we had this big Scandal that happened at Dana Farber which is the Harvard Cancer Research Institute preeminent Hospital in the United States um where they were copying and pasting images so they were taking an image that was day one of the control group and they were pasting it into later in the intervention group as if to say the tumors didn't grow we've suppressed humors I mean that that that's obviously not a mistake right but that and that's published right and and so it gets through these Gatekeepers now is that a design failure well it's an execution failure and you know I think there's so much of this kind of image manipulation that's going on and it doesn't seem like anyone's actually looking at the images except for these sort of image slle like Elizabeth Bick who's a big hero of mine and they're all doing this on Pub pier and it's a hobby for them but my point is that the statistical significance is so easy to manipulate that you can come up with whatever outcome you want and we're not going a step further to say okay you're claiming this is a you know statistically significant result but let's look at the images what do the images say and some of the stuff you don't need sophisticated technology you can literally see that it's been copied and pasted why is this not something that's causing just complete unrest in our society you have these major major institutions committing real fraud and the outcome for the patients isn't really being considered so you're you know you're a doctor I me this is another part of this consensus and you hear something's coming from Harvard Medical School of course I'm going to trust it I'm not going to challenge that I mean there's even statistical things that are really interesting like intention to treat analysis is one that I think you know we do this thing called Journal Club where you can go on an online Zoom call and we take apart studies so we're looking at real Hallmarks studies and then finding how the information is being portrayed in the article versus what you really know and intention to treat is in very common and it basically means like you take a snapshot of day one of a trial or some part and then you have people who drop out of the study or maybe they die so you take their data and you basically supplant it for the time that they're not in the trial assuming that it was the same result consistently across that time so we just did did this we were looking at an exercise and aging study that was out of Norway and they did this so they have people who drop out of the study and they just assumed they kept exercising we don't know maybe they did maybe they didn't they weren't a part of the study anymore um these are the kinds of things where I feel like we've taken something that in its purest sense is about drilling down on uncertainty right so you want to be less wrong and then we've developed all of these sort of statistical mathematical tools that have made made this far more complex I mean I think you know there's this whole idea of like person years which is another statistical thing that will just make your brain mad if you try to figure out what it is and it's a way of estimating but it doesn't end up being very accurate and we we're not going back and rethinking how do we do these you know what statistical tests are actually meaningful and I think part of that is because they're they've become as ger gigerenzer who's a um friend of ours and has spoken at some of our events he's at Max plun and he calls it a ritual and I love that because it basically it only has the meaning that we're putting into it it's not actually a stop Gap it's not actually a safeguard against any of this stuff and I think when you're talking about peerreview you also have this notion of like we don't know who the peer reviewers are Imagine opening a newspaper and there's no Mast head you don't know who any of the editors are there's no accountability for why this work got through or not another great example of all of this was um some research that Begley and Ellis did with Amgen where they realized that cancer and hematology drugs weren't as effective as they should be I think they tested or they tried to replicate 53 trials and they went to Great Lengths to do this so they worked with the original researchers they tried to recreate the environment so that all the variables were as close and as stable to the original work and they could only replicate 11 so 11 out of 53 Hallmark cancer and hematology studies that's all they could replicate going to extreme cost and length to get this done now the other thing they did was that they promised the researchers anonymity because they needed their buyin right they needed them to help and so the researchers know that their work couldn't be replicated but none of those have been retracted so those studies still live in high impact journals as though their sound research even though the people involved know their work couldn't be replicated am Jen knows but they haven't shared it I mean that's fascinating and of course you know I referenced John Ian's work uh here I mean his sort of giant analysis right meta analysis which is just kind of shocking I can't remember the exact statistics right now but it was so it was so low yeah I mean it's called most research findings are false right and I think you know that's shocking to people but I also think if you look at people like Marsha angle who was the head of the New England Journal of Medicine editor and chief Richard Smith who was at the bmj editor there um and Richard Horton who was the head of the Lancet these are top top top medical journals they all three independently have come out and basically said like we cannot trust anything in these Publications and I mean Mara Angel wrote a book about it it's it hasn't changed anything so I mean when I say I have no hope for the sort of systematic way that we're handling these things it's because of that you have people who are calling foul on their own industry at the highest position and nothing changes why is that really you know Greg and I um he sold CrossFit in 2020 and we've spent about four years trying to really find the root cause rather than looking at these symptoms right so there's a lot of organizations who are doing great work looking at you know research that won't replicate or looking at would you call so so lay out the symptoms to me CU I think that that for a lot of people what you call a symptom they might see it as a cause even so you know there's a lot of people who are really focused on like what happened with Co and you know this shows that that science is broken or that medicine has been corrupted or captured and I think that's a symptom I think the amount of money going to the government and to the media from pharmaceutical companies is a symptom I think the fact that like I mean for me I was interested in during Co that like fouchy and Collins control all the money they should all they should not be involved in policy I mean that's such a conflict of interest interest right so if somebody disagrees with you they're potentially not going to be funded that should be church and state um that's a symptom I think peer review is a symptom I think this dobert case which I would love to see overturned is a symptom from our perspective really looking at the philosophy of science this goes back to Carl popper and his denial of induction induction in a very clear or like simple way is being able to take information from the past or that you know and apply it to a future sort of prediction and so there was concern I mean Hume actually is the first one who sort of calls into question induction and it's reasonable to do that there's a bias that's inherent if you take your past information and you apply it to Something in the future however we you're not going to ever go in and get the best outcome if you don't take into account let's say someone's medical history so you're looking at an image or an MRI you want to know well what what led this person to get the MRI that's huge important that's all inductive reasoning but what's happening is that we've LED this has sort of led to this frequentist approach where you're really like ones and zeros yes and no very binary and our goal is to you know sort of return predictive value to science because it isn't a one or a zero it's a scale and you want to know how close you are to certainty let let let me let me see if I'm getting you right so you know let's say you're someone is getting an MR MRI right and so basically the the way these MRIs are looked at don't involve the patients history the way they're being compared they do yeah no so I'm that that's an example of how we absolutely need to take the patient history into account moving forward but I think what has happened more along the lines of this denial of induction is this notion of like we have to be unbiased about how we process information so like this is where a lot of the statistical tests come from so like null hypothesis significant testing is really a product of this frequentist mindset a frequentist that's a that that's a that's a word that a lot of people won't won't make sense to them there's a frequen dis approach which is basically like you know we're going to we're going to have certainty about things we're going to be you know again it's like in computer language it would be like one or zero right and then there's a basian approach and we're more in the basan camp which is about predictive value and it's actually that if you use use baz's theorem you can test the hypothesis outcome so let me think of an example um I actually just did something on Instagram looking at mammograms so people often talk about mammogram sensitivity and specificity which is just sort of the rate of false positives or true positives or false negatives right and that's not telling you anything about you it's telling you about the test what do I as a patient want to know I want to know what is the likelihood that I have a positive mammogram that I have breast cancer so I need to have prevalence rate which is prior information right that's larger information than just what I'm looking at so sensitivity and specificity all you need to know is the outcomes of the test but if you want to know something for the patient you really need to know what the prevalence rate is and then you can come up with positive predictive value suppose you get a mammogram and receive a positive result what is the probability that you actually have breast cancer let's calculate it positive predict value is a measure of how often someone who tests positive for a disease actually has the disease positive predictive value can also be expressed as a conditional probability the probability you have breast cancer given you have a positive mamogram we can use base theorem to calculate the positive predictive value first we need to know how well mammograms correctly diagnose cancer this is known as sensitivity it is the percentage of true positives out of all the mammograms done sensitiv for a mamogram is around 84% next we need to know how common breast cancer is in the population this is the disease prevalence represented in our formula with the letter P for breast cancer 1.25% of women have it next we need to know the percentage of the time a mammogram correctly identifies an individual who does not have a disease a test with a high specificity means there are few false positives for mammograms around 91% of the time the test is negative when the person in fact does not have cancer so the specificity of mammograms is 91% let's put it all together so what does this mean if you get a mammogram and receive a positive result the probability that you actually have breast cancer is only 10.56% at first this may seem shocking but let's think about it if 1.25% of women have breast cancer and a mammogram gives a false positive 8.88% of the time for an uncommon disease most of the positive test results will be wrong now that's hugely important nobody tells patients that right but the amount of stress it causes I mean say like you could do this for anything right like you get an AIDS test and you're like am I dying I'd like to know that and it may be weeks before I can go back and get another test right so wouldn't it be nice to know the likelihood that you actually are dying mhm 5 10% I don't care about the sensitivity and specificity of the test other than how it relates to me that requires predictive power which you need to have other information and with Bas theorem truly with something like the mamogram or the AIDS test you can refine it so you could say like I have a genetic predisposition so I'm not the we don't want the general prevalence rate we actually want my cohort rate right so I'm more likely than the general population to have breast cancer so then you factor that in and you'll still be able to figure out how likely this test is to predict that I have it there's things like that that we're not doing that we could be doing the other thing for me is that I don't want to take I'm not trying to like take down medicine right I think doctor morale is at an all-time low and I and I understand why I think they got into this profession to heal and to treat and they have no time time with patients so I mean I think you know I have a friend who's a doctor who says that it used to be when the doctors walk down the hallway the administrators would like run and hide out of fear right and now it's the other way around and it's because the doctors are being yelled at for not filing the form right or not coding something right they're not accountants that's not why they got into this they got into this to hold the hand of the patient and help the person heal or prevent illness and we've taken that power away from them so I mean I like to remind doctors they're the only one with any moral Authority take the hypocritic oath and that really means something and we as patients are absolutely dependent on them because the pharmaceutical companies are beholden to their investors they have a fiduciary responsibility to deliver returns so do Hospital Systems even Academia so the only Safeguard I see in the system is the doctor who needs to be able to stand up and say you know what if I put you on this drug you're going to get these side effects and then we're going to have to put you on that drug and then you're going to have these side effects and then we're going to put you on that drug so let's just try something different let's see if we can get 50 pounds off of you and how that goes then maybe you won't need any of these that's a conversation I think we would all value and respect and it's it's not happening to near the degree that it should be happening and I think you see this in in federal funding I don't remember exactly what the statistic is but let's say it's more than 80% of medical research goes to treat and this is our taxpayer dollars goes to treatment not prevention I think like let the pharmaceutical companies pay for the treatment why doesn't the government pay for the prevention you're working for me right I don't want the treatment I'd like to prevent the disease you know what you're describing made me think of you know like holistic approaches to health or functional approaches to health B before we go there tell me a little bit about yourself cuz how how is it that you got into all this actually you have a journalism background as well yep so I got a masters at Northwestern in journalism um and I've written for newspapers magazines I worked at 2020 in Prime Time mostly covering murder um and I think I've always been just very curious about um problems corruption how things work or don't fall apart interplay between you know different groups um and thinking about how to take complex information and make it accessible to people and I think you know telling truth to power power was really important and it wasn't you know you weren't trying to be friendly with the power set when I was coming up as a journalist I had a lot of really great mentors that were really rigorous with me and then I've launched an a couple of startup companies and helped with that which I which I love and I think it's actually quite like journalism in that you have to be you know kind of Scrappy and resourceful compile lots of different data and figure out what's going to work and what's not going to work um those companies all had a heavy Tech Focus um and then Greg glasman was somebody who I had been working on a long form story about he'd sued the CDC he'd um taken on the nsca which was the Rival you know sort of personal training certification company and they had used peer review to publish a journal article in the sort of preeminent exercise physiology Journal that said that cross had caused injuries and Greg had recognized that they'd falsified all their data so he sued he won a federal judge called it the biggest case of scientific misconduct and fraud she'd ever seen in her all her years on the bench and I was in the process of writing up that story and he was canceled so he was called a racist because he put out a tweet George Floyd was murdered now this whole tweeting situation happens because the ihme was the modeling body for covid and Greg is a math guy was raised by a rocket scientist all of CrossFit his methodology that he designed and developed was based on Newtonian physics and so he recognized during Co that that like these models were wrong we didn't have a death rate we didn't have a denominator how are we making these projections so again predictive value so in the United States it was the ihme that was doing that work and he had been tweeting at them for a while being like guys your math is wrong you're leading us to you know Financial despair in this country with these policies that aren't based on solid math so when they came out they said that they were going to start modeling racism as a public health issue and I think Greg lost his mind he thought these guys have led us into quarantine right lockdown is going to disproportionately impact minority groups why in the world would we trust you all to model racism it's too important of an issue and so he wrote what is this Floyd 19 and then had this quote from this medical journal that nobody bothered to put into Google because they would have found the article but they didn't and I think because you know he's a 60-some year old white man it was easy to just call him a racist and so I was working on the story about the case that he had against the NCA and he called me and he was like I need you to help me and I was like I can't like I'm not a PR person and like you know if I do that I won't be able to ever be a journalist again the allegations against him escalated turned into toxic workplace and then sexual harassment and I knew him very well and he had more female Executives than he had male which is basically unheard of for a male run company he CrossFit as a sport was the only company or sport to ever pay women the same prize money as men and I had asked Greg years before like why did you do that and he was like what do you mean why did I do that it's like the right thing to do it's weird that that's not standard so he none of these stories matched with the man that I knew and so as the allegations escalated I felt a moral obligation to to jump in and help him and I done some negotiation training at Harvard Law School and I ran a business that was in the Middle East in the United States I was comfortable and high stakes environments and I felt like I couldn't stand on the sidelines and watch him be destroyed for stuff he hadn't done so I kind of jumped in got on the phone with the New York Times who was running a story the next day and explained like I've written for the New York Times and I am not a PR person but I knew they had the story wrong and so the reporter there to her credit worked you know she had to run the piece that she ran the next day but then we were able to work together and to Greg's credit I said to him you have I'm going to do a deep investigation into this who's behind it what's you know because this is now escalated to this point that that seems very much like a smear campaign um and he gave me access to everything I asked for so all his credit cards all his photographs all you know anything that I could use as verifiable information you can forensically rip you know date time location um and I was able to prove the allegations were all false and that launched a sort of Crisis management strategic Communications firm that I have so I do the broken sign stuff and then I help people who have been wrongly accused in the media or businesses that are trying to launch products into markets that they know are going to be tricky and it's a lot like being an investigative repor I mean I sort of go in and I try to figure out who was behind it what's true what's not it's very hard to prove a negative right it's really easy for somebody to throw an allegation at you it's very hard to prove that didn't happen and especially with things like sexual harassment these things have been weaponized to the point where um I had a client who was accused of sexual harassment had never met the woman but it was a big corporate board and somebody wanted him off the board so they spread this rumor around and the fear that it was going to be leaked to the media was enough to threaten his job we all need to take a minute and think a little bit about why somebody's saying something what their motivations are how they might just be wrong by accident or it might be something much more nefarious and not just jump on these sort of like social bandwagons because my feeling is like that does the biggest disservice to the real victims there are women who are sexually harassed so I think it's really important to be thoughtful ask good questions and be conscious of the impact that you have when you decide to you know cancel or decry somebody or in the medical sphere you decide to take a treatment do you really understand what that treatment is we just had this recently and I think we're writing it up for broken science so we have original pieces that we put out and then we also have ones that are curated news pieces and that's a part of our newsletter um but there was a you know headlines around the world saying that this new cancer therapy called car T was amazing it had stopped tumor growth in in the brain this would be remarkable we really do need some sort of cancer breakthrough but when we looked at the study all cause mortality was no different so people weren't living longer they were stopping the tumors from growing but they were dying at the same rate that's fascinating what does that mean I mean I don't have an answer but I would think maybe the treatment is killing them they're not dying from the tumor they're dying from something else that nobody in the media covered that everybody covered like cancer breakthrough look these tumors have stopped well that's great you stop the tumor but the goal is to live longer yeah I keep thinking back to your approach seems to be making sure you see the context of every scenario making sure you you factor in all these uh the elements that often are missing yeah and and being challenging both to yourself and to whatever it is that you're considering you know asking good question really good questions come from really good listening and I think we're not doing enough listening and so whether that be medical research or you know what you hear about somebody in your neighborhood there is a real detriment to not kind of having a life of learning and wanting to be curious I mean I love people who don't agree with me because I always learn from them but I think that that's less and less common and I think sometimes even for me I can be I can put people off because I challenge them and it's not because I it's a disrespectful thing it's the opposite I respect you enough to want to know how you form that idea because it's different than mine so please explain it to me doesn't mean I'm going to agree or disagree it means I want to know how you got there I just remember there's one uh study that you looked at black women at America disproportionately have can disproportionately die from child birth even compared to many other country you know industrialized countries and there's some is that is that really a thing and and why is a thing I well so again I think this is one of these things where we are looking at these sort of like statistical things and we're not peeling back to get to the root cause um and so I did a lot of reporting on maternal mortality you know in my sort of Prior life when I was a journalist still and I became really interested in C-sections because the maternal mortality thing is usually Downstream from a C-section and C-sections are really difficult to study so there's this great researcher in Boston named Neil Shaw who um has really looked at this more critically than anybody else and part because he was delivering babies in two hospitals in Boston both Harvard Medical you know teaching hospitals populations were very similar and his C-section rate was much different in one Hospital than it was in the other now he's the same doctor right there's all these variables that are constant and so he became really obsessive about trying to understand why so many more people were being given C-section right and by him and he he said to me so he was he was making the he was making the choice to do it right like what is environment what's going on in this one hospital where I'm delivering more by C-section than the other hospital and because he's one guy and the patient populations are very similar the hospitals are funded similarly he was able to really sort of unpack some of this and he said to me it's really hard to study because you never deliver a baby via C-section and think like we didn't really need to do that right the hindsight bias is so strong that you're like thank God the baby's okay right every time and so with the maternal mortality crisis we're looking you know they'll say like hemorrhaging right or there they'll name things the CDC codifies the stuff they're not calling it a C-section death and I think the race component of this comes in because it's probably a socioeconomic thing more than a race you know if you were to really dial it down if you have a C-section you're not supposed to lift anything like over 10 pounds for weeks if you have an hourly paying job that you don't get maternity leave for and you have to go back to work two weeks maybe a week after you have a baby you are at way higher risk for some sort of complication if you're a mom and you're home with other kids who you're responsible for and you've had a C-section you are at high risk so I think those populations are not cared for properly in the way that we you know sort of Don't Care for Women in in the health system very well I mean I we've talked about this before but you know I really became very interested in women's health because we know that women's Brains are different our hearts are different our lungs are different everything in your health is dictated by your endocrine system which is hormonal and women's bodies are not studied so you know in 1977 the government made it illegal for women to be in clinical trials if you were of childbearing age what's childbearing age it's like f you know the majority of your life like 15 to you know 55 or something if you want to be safe I don't know but it's like that's so we weren't studying women's bodies it was illegal and then in the I think it was 1993 they said okay women can be involved in clinical trials again but it wasn't that long ago that med school started doing female cavers and it was just this assumptive practice that our bodies are the same except that because of our hormonal cycle we complicated to study and I guess maybe people don't want complex problems sort of hard to understand but women you know the diseases we get are different the prevalence rates are different the treatments affect us different ly it's a huge amount of research that needs to be done in that realm and I think you know birth is a huge inflection point for a woman for all kinds of reasons but it's usually the first interaction that she has in a serious way with the medical system and so I think what we've done is we've over medicalized birth it used to be you'd have a midwife right unless you were at high risk and then you'd go to a hospital and you know I have interviewed a lot of nurses who say like I used to sit by the woman's bedside and tell her that like everything was normal and you know this is how it goes and count for her and do all kinds of things and now I'm sitting in with a bay of 20 monitors watching heart rates and you have a heart rate monitor in the room with you and it goes up and the the mom's you know it's monitoring the baby the Mom's heart rate is going to go through the roof she's worried about the baby there was actually a really cool Clinic that was a pilot program I think it was out of Duke where they realized even for follow follow up appointments moms will skip their appointments but they will not skip the babies so if you can book mom and baby at the same time you have the pediatrician and the ob in the same office and they they both have their appointment the mom shows up simple fix to and and that's the deal is like the maternal mortality stuff is happening because women aren't going to the hospital or they're going to a hospital that's not taking them seriously and they're not making their follow-up appointments where you might be able to monitor that there's a real problem I mean in lots of countries you give birth and somebody from the medical establishment whether it be a doula or a midwife or a doctor comes to visit you at your house you get a lot of data when you visit somebody in their house not just about the patient but what's the environment is she being cared for is she safe you know she healing properly we don't do that and I I do think that from my lens that feels again like a sort of frequentist approach because what we're doing is we're checking a box okay I mean it's like people who are um the doctors that help you get pregnant the IVF doctors and whatnot their scores of success are based on whether you get pregnant or not not if you have a healthy baby this is a misalignment right so lots of women who go through IVF try to pick the best doctor they can find will the doctor will tell you what their rate is of getting people pregnant not their Mis the miscarriage right that's really important by the way what happened to the Doctor Who was had the two hospitals that were very similar but had these different rates of csection what was what was happening there well so his premise is that it's environment that it has to do with like the mom being involved and so he actually created I haven't talked to him in years so I'm sure he's moved this along but he was developing a dashboard that would basically like allow the mom to know all of these different things and the medical team would have to go in and talk to her about certain things and I think most of his premise was this idea of like if the mom is really involved and there's open communication the C-section rate will will plummet and that in one hospital where there was all this sort of like technological ination it was leading to more C-sections and that the mom wasn't at the table that basically the doctor would come in and say like hey you know what this is you've been doing this for long enough we got to take you into the O there's a fallacy about people thinking that like women are scheduling C-sections because they want to I didn't find that in the data at all most of it is this sort of real time things aren't going quite right we know we have this other way of delivering why not just you know make that a preferred option when things don't seem to go right and I mean I think the maternal mortality is a big risk there also was a lot of data I found that said that women are far more likely to have to have a hysterectomy later in life if they've had a C-section it also prohibits your ability to have lots of kids right so it's the only time a surgeon will cut on the same scar over and over again surgeons are taught never to do that and so it it's very damaging you can find out I actually did a story I think it was for Cosmo or Boston magazine that you can find out what your hospital C-section rate is so your the highest predictor of whether you're going to have a C-section or not is what hospital you deliver in it's not you it's not your doctor it's the hospital you know it's funny because you know what that reminds me of I remember discovering years ago this was again you know probably decades ago that there was a study that was done was which type of psychotherapy works best and they were they were trying to isolate you know which which one but it turned out that actually the method of psychotherapy doesn't matter the only variable that really kind of jumps out is the identity of the psychotherapist irrespective of which method they use some people are successful and some people aren't wait tell me more about that I'm curious well I don't I don't remember a ton more right I just I just I remember that the outcome was that the the the the there's something about the person themselves which made them successful in dealing with people presumably they had some kind of very good let's call it bedside manner or they were real listeners or they somehow knew how to q and on body language I don't know it was the person that mattered so if you go to a good one they could use any method they wanted to and they would be successful and if you don't go to a good one doesn't matter what method they've got you're it's not going to work very well I think that listening is an incredibly powerful tool for healing I think that that study also spoke to you know the types of things I would have believed back then too I think that's why I remember it right but but it was it was very obviously very interesting we'd have to dig it up I don't I can't for the life of me yeah so I want to look at that I mean I feel like there's something really um that stands out in terms of this notion of research and how we think we identify these things like this protocol works better than this other one and it turns out it's no it's the practitioner right and I think that also goes to you know sort of you talk about the doctor morale problem that we're seeing which really I hope the broken science Medical Society becomes a huge Network for doctors to feel empowered again I think there's this also rejection of wisdom in our culture medicine is an art and a science it's both the more experience you have you have some intuition about something somebody comes in presenting certain symptoms and if you entered it into an AI it might not put it all together but you remember this person also mentioned last time you saw them that they had headaches right or that they had some other symptom that becomes very important again it's taking that prior knowledge and applying it and I think that's for the individual you know the patient you're treating but I also think it has to do with just experience treating patients and spending a lot of time listening to patients that we're not valuing because we're not allowing doctors to spend time with patients you know the average doctor's appointment the doctor's spending like 12 minutes with a patient it's not enough time and there is there's some interesting work that looks at Communication Breakdown between doctors and patients that I've spent a lot of time thinking about that basically indicates that you know when you're scared or you're nervous you are most likely to put forward your least significant symptoms and the doctor is only listening to your first few symptoms because they're busy and they're thinking about how to code whatever it is that you just mentioned so there is a true disconnect in a sort of communication breakdown sense where you're listing your least significant symptoms and then I'm not hearing the rest so there's little things like that that I think we're at broken science we have a class that's coming out this summer that's for patients to navigate their own Healthcare and this is one of these sort of Tricks where I'm you know as a journalist thinking like what questions can you ask that can help drive you to be a driver just like with the maternal mortality you have to have a seat at the table this is these are the most important decisions you're ever going to make in your life you can't be passive and yes You're vulnerable and yes you want to get along with people but there's a respectful way of doing this you don't have to be combative and I definitely think having an advocate or somebody go with you to the appointments if you're reluctant is is hugely helpful you know so I really appreciate what you're doing here because you know you're you're not necessarily saying that you you can solve this huge giant problem that you see you're pointing out some of the problems but you're giving people you're empowering people to help themselves in this let's call it difficult or fraught environment so how do people access you know broken science and some of these projects that you have so brokens science.org is the mothership and we just redesigned the site so the back end is actually a sort of AI operating system so it'll go in and it'll learn what your interests are and it'll start recommending things to you we also realized a lot of the material was intimidating to people so we've created summaries and versions of most of the really dense material at different grade levels so that you can really come in at you know knowing nothing or you can go and read the original work um and all of that's free and then we're starting these cohorts that are called societies so the medical society is the first one we're launching and that's going to be a networking opportunity there's a huge amount of sort of like social media type capability on the back end of the site so that people can follow each other they can share research they can invite each other to events we're doing this thing called Journal club which is the taking a part of the you know medical journal studies that will be part of the resource library for the doctors and patients I mean I'm not a doctor I'm interested in all the stuff so I don't want to preclude that but there will be individual groups that will be you know more specifically focused on profession and you know I mean I think the approach is very similar to what Greg did when he started CrossFit he knew he wasn't going to be able to save everybody but he knew anybody who would wanted to work really hard and do this you know his methodology would benefit from it and so it was a real Grassroots way of building and Harvard Business School called it the fastest growing company in world history and I think we're going to replicate that so we have our personal health Society that's launching at the end of the summer and then an education society and those cohorts will will feed into each other so I imagine our personal health Society there'll be a lot of people who do sort of self-experimentation right so I'm going to do the keto diet for six months well we're going to have doctors and the doctor cohort who can go study them and then hopefully publish that work in a journal that we'll launch in a year or so so I think there's a lot of exciting stuff our YouTube channel is got a lot I'm trying my best to do explainer videos where I really break down some of these Concepts so I have one on induction I have one on statistical significance um and I'm happy to do more of those I love it when people say like you know I've I've been doing this forever but I never I can't quite remember I don't quite understand I can't hold this and then I can try and help think of ways that we can you know animate it or whatever our Instagram is very active um and popular and that's just broken sign initiative um this new show that I'm doing that's looking at people who have changed a paradigm in art or science um is called Emily Unleashed and there's an Instagram page for that and when we launch it'll be I think on YouTube um and that really is supposed to just Inspire critical thinking so it's talking to people about how they stood up to the status quo and either it made them tons of money or it got them cancelled but they felt they couldn't not do it and I want to inspire some more of that sort of American Spirit Rebellion you know don't defer to Authority not on matters that are really important and learn to listen and be respectful to each other the ultimate certainty is the theme Here Right There is no ultimate certainty I am not 100% confident in anything that I just said to you I'm pretty confident but somebody could come in here and know more about something and I'd be open to listening to them I want to engage I want people to be engaged in that kind of debate and thought process well Emily Kaplan it's such a pleasure to have had you on thank you so much I really enjoyed this thank you all for joining Emily Kaplan and me on this episode of American thought leaders I'm your host Yan Kell [Music]