Posted at 10:59 PM | Permalink | Comments (0)

In the **last article** I posted on "The Digital Biologist", I gave a very brief and simple introduction to **Bayes' Theorem**, using cancer **biomarkers** as an example of one of the many ways in which the theorem can be applied to the evaluation of data and evidence in life science R&D. The power of the Bayesian approach was I hope, evident in the analysis of the **CA-125** biomarker for **ovarian cancer** that we considered, and I felt that it would be worthwhile in this follow-up, to round out our discussion by looking in a little more detail at the practical, actionable insights that can be gained by the application of Bayesian analysis to the design of biomarkers. It is all too often that those of us in the field of computational biology are accused of generating models, simulations and algorithms that while pretty or cool, are of little or no practical help to real world research problems. The sting of this accusation comes at least in part from the fact that all too often, this is actually true :-(

Bayesian analysis by contrast, can be a really useful and practical computational tool in life science R&D as I hope this brief discussion of its application to biomarker design will show. There are some valuable lessons for biomarker design that can be drawn from the kind of Bayesian analysis that we described in the first part of this discussion, when we considered its application to the use of CA-125 to diagnose ovarian cancer.

Let's suppose that we are a company determined to develop a more reliable biomarker than CA-125 for the early detection of ovarian cancer. One direction we might pursue is to identify a biomarker that predicts disease in actual sufferers with a higher frequency i.e. a biomarker with a better true positive hit rate. We saw in the previous article, that CA-125 only predicts the disease in about 50% of sufferers for stage I ovarian cancer and about 80% of sufferers for stage II and beyond. One of the dilemmas faced by physicians working in the oncology field, is that biomarkers like CA-125 can be poorly predictive of the disease in the early stages when the prognosis and options for treatment are better. It's disheartening for both the patient and the physician to be able to get a reliable diagnosis only when the disease has already progressed to the point at which there are fewer good options for treatment.

I have **previously used** an analogy from the behavioral sciences to describe this situation: "*broken glass and blood on the streets are the "markers" of a riot already in progress but what you really need for successful intervention are the early signs of unrest in the crowd before any real damage is done".*

So imagine that our hypothetical biomarker company has made a heavy R&D investment in identifying a biomarker with a better rate of true positives. If our new biomarker has true positive rate of 95% (a fairly significant improvement on our previous value of about 80%) and the same roughly 4% false positive test rate as previously, how much better off are we?

If we plug the numbers into our **Bayesian model**, the answer is "not much".

The chances of a patient actually having ovarian cancer given a positive test result with our new biomarker are still less than 1 in 4. *In fact, even if we were to identify a biomarker with a 99% true positive rate, we could still only declare a roughly 1 in 4 chance of disease given a positive test result.*

What if instead of pursuing a better true positive hit rate, our company had invested in reducing the false positive test rate?

Without altering the true positive rate of about 80%, reducing the biomarker's false positive rate from about 4% to 2%, increases the chance of the patient actually having the disease given a positive test result, to better than 1 in 3. If our hypothetical company can get the false positive rate down to 1%, there is actually a better than even chance of a positively-testing patient actually having the disease. Getting the false positive test rate down to 0.1% (approximately 40 times lower than the actual false positive rate for CA-125) means that the patient is very likely to have the disease given a positive test result, with a less than 1 in 10 chance of receiving a false positive diagnosis.

The Bayesian model clearly tells us in the case of ovarian cancer, that our hypothetical company is much better off investing its R&D dollars in the pursuit of lower false positive test rates rather than higher true positive test rates. Even a 99% true positive test rate barely shifts the probabilities associated with a positive test result, whereas getting the false positive test rate down to 1% improves the probability of a true diagnosis from less than 1 in 4, to better than even. Even this scenario however, is far from ideal.

If you look at the actual numbers in the model with regard to the populations of tested patients with and without the disease, there is another valuable lesson to be learned, and it is one that illuminates the reason why improving the true positive test rate while ignoring the false positive test rate is what my countrymen would refer to as "**a hiding to nothing**".

It is the overwhelmingly larger population of healthy patients versus those with the disease, that is skewing the probability numbers against us and **the lower the incidence of the disease, the worse this problem will be**.

If ovarian cancer had a higher incidence of say, 1 in 10 women instead of **1 in 72 as is actually the case**, a positive test result with CA-125 would correspond to an almost 70% probability of the patient actually having the disease. By contrast, if the ovarian cancer incidence was 1 in 1000 women, a positive test result with CA-125 would still correspond to less than 1 chance in 50 of the patient actually having the disease.

**The lower the incidence of the disease you want to diagnose, the correspondingly lower your false positive test rate needs to be.**

Imagine for example, the exigencies that a rare cancer like **adrenocortical carcinoma** which only affects 1 or 2 people in a million imposes on the development of any kind of diagnostic biomarker for that disease. In some rare diseases that have a genetic origin (such as **Type II Glycogen Storage Disease** for example), there do exist definitive genetic tests for the disease that are essentially unequivocal insofar as they have a false positive rate that is effectively zero.

The Bayesian model presented here is an extremely simple but excellent example of the way in which **models can provide intellectual frameworks with which data can be organized and reasoned about**. It is this author's opinion that the pharmaceutical and biotechnology industries could actually benefit enormously from a shift in their **current emphasis on data**, with more attention being paid to the kind of models that have the potential to explain these data, to synthesize useful knowledge from them, and to drive effective decision making based upon the underlying science.

© The Digital Biologist | All Rights Reserved

Posted at 01:13 PM in Biological Modeling, Digital Biology, Pharma and biotech, Science | Permalink | Comments (0)

As life scientists, it is seldom that we ever get to deal with anything resembling certainty. The systems that we work with are typically nonlinear and chaotic, heterogeneous, non-binary - in a word, messy. In the world of commercial life science, it is common that the real value of an R&D investment of hundred of millions or even billions of dollars, may ultimately ride on such a razor-thin edge between success and failure that it requires the calculation of something like a **t-test** to determine whether you really have a marketable product or just another placebo - or in the case of a diagnostic, a real indicator versus background noise.

The development of a new drug or diagnostic is in essence, a process of gathering evidence either for or against your working hypothesis that the use of your product will confer some net benefit over not using it. In such a case, the **null hypothesis** - that your product will confer no net benefit at all - is (and always should be) a core consideration in your approach.

With each new piece of data that we accumulate along our hopeful path to that blockbuster product, we are weighing the evidence for and against eventual success or failure. A big part of this process for a commercial life science company, is the decision based upon the current evidence, of whether or not it is worthwhile to continue the investment of time, money and resources on the product, or to pull the plug on it. All too often and for all sorts of reasons, it can be painful and difficult for a company to admit that a product is a dead-end and walk away from its investment. Killing projects in a timely fashion **is a particularly acute problem in the case of drug development**, given the exponentially increasing cost of R&D as the product progresses through the subsequent phases of development.

This process of weighing the evidence was mathematically **formalized** during the 18th century, by **Thomas Bayes**, an English priest who was fascinated with statistics and probability. As a small but amusing aside, it is ironic to reflect that there is some uncertainty about whether the only extant portrait of Thomas Bayes shown above, actually depicts the right person! But whether or not the portrait is a true likeness of the great man, the now famous **theorem** to which he gave his name, stands as a landmark in the history of probability theory. An understanding of the implications of **Bayes' Theorem** and its application to the myriad problems of truth, belief and likelihood that our uncertain world challenges us with daily, is something that every scientist (biologist or not), can put to good use in his or her own work.

So why is Bayes' Theorem so useful and what does it have to teach us as life scientists?

By way of a very simple and brief introduction to Bayes' Theorem, let's take a look at the development of **biomarkers** - an area of life science research that is directly concerned with issues of prediction and likelihood.

Let's imagine that we are looking for a reliable early indicator for a disease which affects about 1.5% of the population and that our research has uncovered a biomarker whose presence is predictive of the disease in about 80% of sufferers. Sounds pretty good right? Most people would probably look at those numbers and conclude that a positive test for the biomarker was associated with something like an 80% chance of having the disease. Not too shabby.

Not so fast.

One very important question that remains unanswered is "How many people who do not have the disease, would still get a positive test result with this biomarker?" A biomarker that produced a positive result (indicative of the presence of the disease) in 80% of all patients, with or without the disease, would obviously have no predictive power at all for signaling the presence of the disease. As with the inherent uncertainty that pervades most things in life, biomarkers are seldom if ever 100% reliable, but let's say for the purposes of our story, that the biomarker in question produces false positives in about 4% of patients without the disease (i.e. the test result indicates disease where none is actually present). Things seem to be looking up. Armed with these numbers, we might feel that this biomarker has a bright future in the clinic based upon the following reasoning - it will only fail to detect the disease in about 2 out of 10 sufferers and it will only produce a misdiagnosis of the disease in about 4 out of 100 healthy patients.

So now we're in good shape right?

Once again - not so fast.

Let's think about this biomarker's performance from the perspective of a hypothetical population of 10,000 patients. Based upon the 1.5% incidence of this disease in the population, we would expect our population to have about 150 patients with the disease and therefore, about 9,850 without it. Of those 150 patients with the disease, we would expect about 120 to test positive for the biomarker based upon an 80% positive test rate amongst people with the disease. Amongst the 9,850 patients who do not have the disease, we would expect about 394 to test positive based upon a 4% false positive test rate for the biomarker.

Now put yourself in the position of one of those patients who just got a positive test result. The first question you're going to ask is "What is the probability that I have the disease given that I tested positive for it?"

**This is really the key question. What does the test result actually mean?**

To answer that question, let's look at the overall probability of getting a positive test result under any circumstances. We expect 120 patients with the disease to test positive and 394 without the disease to test positive. So out of a total of 514 positive tests, we expect 120 patients who test positive to actually actually have the disease, corresponding to a probability of about 23%. In other words, the answer to the question of the patient who had the positive test result is that they have only about 1 chance in 4 of actually having the disease, based upon the positive test result. Put another (and perhaps more optimistic) way, despite the positive test result, there are still about 3 chances in 4 that they do *not* have the disease. Or put in yet another way - despite the positive test result, the patient is still 3 times more likely *not* to have the disease than to have it.

In the light of this new analysis of the biomarker's performance, would you still conclude that this biomarker is a useful clinical diagnostic for this disease? If you were a physician for example, would you schedule a potentially risky or expensive surgical procedure based upon the 1 in 4 chance of the disease indicated by the positive test result? Would you alternatively, recommend doing nothing at all despite the positive test result?

You might be really surprised to learn that this "hypothetical" disease biomarker example is based upon the real numbers for the **CA-125** biomarker that is actually used as a diagnostic indicator for **ovarian cancer**. A wealth of statistics have been published both for ovarian cancer incidence and for the use of CA-125 as a diagnostic marker. All that remained for me to do, was to plug these numbers into a Bayesian model.

According to the **American Cancer Society, ****the lifetime risk of a woman developing ovarian cancer** is 1 in 72 (0.0134). **In a recent study involving more than 78,000 women**, the use of CA-125 as a single indicator, yielded 3,285 false positive results (~4%) in which healthy women were diagnosed as having ovarian cancer. Of these incorrectly diagnosed women, 1080 of them actually underwent an unnecessary surgical biopsy procedure, of whom about 150 suffered severe complications as a result. As if this bad news was not already enough, the diagnostic use of CA-125 for ovarian cancer was shown to be of little use even in women who have already had ovarian cancer. **In a study that examined the benefit of using elevated CA-125 levels as an early marker for ovarian cancer relapse**, there was shown to be no survival benefit for women who were started early on chemotherapy based upon the CA-125 test results, versus those who waited until they exhibited the clinical symptoms of relapse.

It is worth noting by the way, that the diagnostic probabilities obtained from this admittedly rather crude Bayesian model presented above, do nonetheless correlate rather well with the actual statistics obtained for true and false positive tests from **studies of women who were tested with CA-125 for ovarian cancer**.

It should be clear from this example, that it is important to weigh the evidence for the efficacy of diagnostic markers carefully. Failing to do so, has the potential to add a great deal of unnecessary complication and expense to health care treatments. CA-125 by itself is a poor indicator of ovarian cancer and medical decisions based upon the sole use of such an indicator can end up subjecting patients to unnecessary pain and suffering. Consider also, the time and money that was wasted for the unnecessary medical treatement of 1080 women with false positive test results in the study cited above, let alone the costs incurred managing the severe surgical complications suffered by 150 of these women as a result of this unnecessary treatment. To be fair however, the shortcomings of CA-125 as a diagnostic marker are now well recognized and the current standard of practice recommends the use of CA-125 with other indicators such as sonograms and pelvic exams, all of whose combined results are more reliable than the diagnostic use of CA-125 alone.

This CA-125 story also highlights the urgent need for more reliable biomarkers that can be used in the early detection of diseases like ovarian cancer. The statistical probabilities that I used were actually taken from the "best case" scenario (in terms of predictive accuracy) for the use of CA-125 to diagnose ovarian cancer. I used the values observed for women with stage II or later disease in which CA-125 levels are typically more elevated but unfortunately the disease is harder to treat. Had I used instead, the values for women with earlier stage I disease where the treatment options and prognosis are better, the true positive rate for diagnosis drops from around 80% to around 50% and the probability of actually having the disease given a positive test result, drops to almost 1 in 7.

Incidentally, if you think I was exaggerating about the naivety of people's interpretation of biomarker statistics, where for example, a test that detected 80% of cases for diseased patients was equated in peoples' minds with an 80% probability of having the disease if the test is positive - well unfortunately I was not. In repeated **studies**, it has been consistently shown that even the majority of physicians, whose job it is to interpret these kinds of statistical results for their patients, struggle with their interpretation, generally ascribing more confidence to their conclusions from them, than is actually due.

The intuitive 'algorithm' that we used above to determine the probability of an event (the patient has a disease), given some prior evidence (the patient tested positive for the disease), can be captured more formally in an equation. The formal description of Bayes' Theorem, is typically presented as an equation of the form:

In the equation above, the syntax **p(B|A)** denotes the *conditional* probability of outcome **B** given outcome **A.** If we plug in the same numbers that we used in our intuitive approach in order to re-calculate the probability of having the disease given a positive test result, they look like this:

*p(disease | positive) = p(positive | disease) * p(disease) / p(positive)*

note that *p(positive)* is the total probability for all of the circumstances under which a positive test could occur - in our case, it is the sum of the probabilities for getting a positive test with and without the disease.

*p(positive) = p(positive | disease) * p(disease) + p(positive | no disease) * (no disease)*

*p (positive)* = 0.8 * 0.015 + 0.04 * 0.985 = 0.0514

therefore: *p(disease | positive)* = 0.8 * 0.015 / 0.0514 = 0.233 which corresponds to the 23% probability we arrived at using our intuitive approach.

As life scientists, the weighing of evidence is always an important component of our work. I hope that the example above makes it clear that in the case of the biomedical sciences at least, weighing the evidence naively can have the potential to be extremely costly and even life-threatening. In the life sciences, Bayes' Theorem has been successfully applied to a vast array of biological areas as diverse as **bioinformatics and computational biology**, **next-generation sequencing**, **biological network analysis**, and **disease evolution and epidemiology** to name but a very few examples.

The fundamentals of Bayes' Theorem are extremely easy to grasp, especially when dealing with the point probabilities and binary outcomes that were discussed here, but the applications of Bayes' Theorem are vast, not only in the life sciences but in any sphere of activity in which our beliefs and decisions are shaped by weighing the evidence.

© The Digital Biologist | All Rights Reserved

Posted at 01:25 AM in Biological Modeling, Digital Biology, Pharma and biotech, Science | Permalink | Comments (0)

After the feeling that the life science sector had perhaps turned an important corner in 2012, based upon what was seen as a meaningful uptick in the rate of new drug approvals, 2013 actually turned out to be something of a dud - a return to the bleak "business as usual" scenario of poor returns from hefty investments that has beset the industry for some years now. You can see the full report **here** online at the** Fierce Biotech** website, but for those of you in a hurry, the gist of the news is summarized in the FB report's introductory paragraphs.

*"The wave of new drug approvals that had been building at the FDA has broken. According to the official tally of new drug and biologics approvals at the agency, the biopharma industry registered only 27 OKs for new entities in 2013--a sharp plunge from 2012's high of 39 that once again raises big questions about the productivity and sustainability of the world's multibillion-dollar R&D business.*

*After 2012 some experts boasted that the industry had turned a corner, with the agency boasting that it was outstripping the Europeans in the speed and number of new drug approvals. But for 2013 the numbers look a lot closer to the bleak average of 24 new approvals per year seen in the first decade of the millennium than the 35 per year projected by McKinsey through 2016.*

*The agency says it was hampered by a sharp drop in the number of new drug applications, forcing a sudden plunge in the annual total--even after starting the year with a new breakthrough therapy designation (BTD) designed to speed the arrival of major therapeutic advancements."*

© The Digital Biologist | All Rights Reserved

Posted at 12:10 PM | Permalink | Comments (0)

A new **report** compiled by Deloitte and featured on the **Fierce Biotech web site**, highlights the continuing woes of the biopharma sector. The report states that the cost of bringing new drugs to market has risen by 18% since 2010, with an accompanying decline in forecast sales of 43% over the same period. In effect then, this amounts to a decline in the sector's return on investment (ROI) of approaximately 50%. This is bad news indeed for a sector already suffering from more than a decade of declining ROI - a situation we have discussed at some length here on The Digital Biologist.

© The Digital Biologist | All Rights Reserved

Posted at 07:31 AM | Permalink | Comments (0)

Here is an interesting talk I came across in my wanderings on the web, given by an investigator from a research group that is pursuing a **graph theory** approach to systems biology. This is particularly interesting from my perspective as I was once involved in this field myself, working at a startup that was seeking to commercialize a rules-based, graph theory approach to the modeling of biological signaling pathways. Enjoy!

© The Digital Biologist | All Rights Reserved

Posted at 01:48 PM | Permalink | Comments (0)

There has been a great deal of interest in the signaling of the **HER family of receptors** as a result of the central role that they appear to take in the proliferation of certain **epithelial** cancers. **Some biotechnology companies have even built innovative portfolios of biotherapeutics** around some of the q quantitative ideas that have been derived from such models.

Now a **new study** published in the **August 2013 issue of PLOS Computational Biology**, takes the modeling of the complex HER family signaling pathways, a step further. with a model based upon a comprehensive data set comprising the relative abundance and phosphorylation levels from a panel of human, mammary epithelial cells. Here is an excerpt from author's own summary of the article.

"*We constructed an integrated mathematical model of HER activation, and trafficking to quantitatively link receptor expression levels to dimerization and activation. We parameterized the model with a comprehensive set of HER phosphorylation and abundance data collected in a panel of human mammary epithelial cells expressing varying levels of EGFR/HER1, HER2 and HER3. Although parameter estimation yielded multiple solutions, predictions for dimer phosphorylation were in agreement with each other. We validated the model using experiments where pertuzumab was used to block HER2 dimerization. We used the model to predict HER dimerization and activation patterns in a panel of human mammary epithelial cells lines with known HER expression levels in response to stimulations with ligands EGF and HRG. Simulations over the range of expression levels seen in various cell lines indicate that: i) EGFR phosphorylation is driven by HER1-HER1 and HER1-HER2 dimers, and not HER1-HER3 dimers, ii) HER1-HER2 and HER2-HER3 dimers both contribute significantly to HER2 activation with the EGFR expression level determining the relative importance of these species, and iii) the HER2-HER3 dimer is largely responsible for HER3 activation. The model can be used to predict phosphorylated dimer levels for any given HER expression profile. This information in turn can be used to quantify the potencies of the various HER dimers, and can potentially inform personalized therapeutic approaches*."

© The Digital Biologist | All Rights Reserved

Posted at 01:19 PM | Permalink | Comments (0)

Biological modeling is not yet in the mainstream of biological research in the way that it is in other scientific and technical areas like physics and engineering, with exposure to the philosophy and practice of modeling amongst life scientists being generally limited to a small cadre of computational biologists. It is no surprise therefore, that the pervasive view of modeling amongst life scientists, tends to be limited to that notion that the utility of models lies purely in their predictive capabilities.

As **intellectual frameworks in which data can be organized and even transformed into useful, actionable knowledge**, models are much more than just tools for "forecasting the weather" as it were. Models are invaluable tools for reasoning about the underlying architecture of the complex systems from which biologists typically gather their data. As platforms for enhancing our cognitive grasp of these systems and for the communication of knowledge and ideas, models can also serve as vehicles for the kind meaningful collaboration that the size and complexity of living systems necessitates.

A few years back, I was working at Plectix BioSystems, a venture capital-funded startup developing a coud computing-based platform for modeling complex biological systems (whose doors were sadly shuttered by the economic crash in 2010). In the course of this work, I prepared the following video for Google's "blue sky" Sci Foo conference in which one of our founders was invited to present the work that we were doing. When I look at where we are today versus back then (in 2009), I feel that most of the road we wanted to travel, has yet to be travelled. Still, as the old chinese saying goes "may you be blessed to live in interesting times".

© The Digital Biologist | All Rights Reserved

Posted at 11:30 AM | Permalink | Comments (0)

While Digital Biology will probably never be a social network mega-phenomenon on the scale of Justin Bieber or Grumpy Cat, it is pleasing to note that during this last month, membership of the Digital Biology group on LinkedIn just passed a major milestone in getting its one thousandth member. Thanks to everybody who subscribes and especially to those who participate in the regular discussions that take place in its forum.

If have a LinkedIn account, you can join the conversation here.

© The Digital Biologist | All Rights Reserved

Posted at 01:31 PM | Permalink | Comments (0)