As a 15-year-old high school student, Jack Andraka researched and prototyped a tool he billed to be a 100% accurate pancreatic cancer-detection test.
Though his work won him grand prize at the annual Intel International Science and Engineering Fair in 2012, his test’s efficacy and accuracy has yet to be demonstrated, and many claims made by Andraka are now recognized by Andraka and others as overhyped.
On Aug. 22, 2019, The Epoch Times ran a profile of a “15-year-old science buff” who invented a 100% effective diagnostic test for pancreatic cancer. This was a curious editorial decision, as the 15-year-old in question was 22 in 2019, and the more sensational claims he had made years ago had been significantly tempered — facts Andraka himself has acknowledged but that did not make their way into the Epoch Times story.
First, here’s what the story gets right. As a 15-year-old, Jack Andraka gained fame for prototyping a potential test for pancreatic cancer that was purportedly quicker and cheaper than existing methods, for which he was awarded the grand prize at the 2012 Intel International Science and Engineering Fair. In 2014, then-Forbes healthcare reporter (and current medical reporter at STAT) Matthew Herper described the concept this way:
He created a device by dipping filter paper into a solution of carbon nanotubes coated with antibodies for a specific protein that can be elevated in patients with pancreatic cancer. He proved that the device can detect the presence of that protein, mesothelin, in cell culture solutions. He also showed that the sensor could tell the difference between the blood of mice that had human pancreatic tumors grafted onto them and those that did not.
Herper, in his capacity as the author of that year’s Forbes 30 under 30 list of promising young scientists, had reviewed a draft of a paper Andraka had written about his test along with six other expert reviewers. In a piece titled “Why Biotech Whiz Kid Jack Andraka Is Not On The Forbes 30 Under 30 List,” he relayed the concerns of five anonymous reviewers and one non-anonymous reviewer: Harvard DNA sequencing pioneer George Church. Church concluded that many of the claims “may not hold up, or at least require more work to be proven.”
Others, like the pancreatic cancer researcher who discovered mesothelin in the first place, Ira Paston, had gone further. In a 2015 Sydney Morning Herald piece, Paston stated that the method Andraka proposed “makes no scientific sense,” and that Paston didn’t “know anybody in the scientific community who believes his findings."
Andraka, for his part, has also recognized that the problem is more complex than his earlier statements might have suggested. In an email to Herper in 2014, Andraka wrote:
While promising, the project really was extremely preliminary and is by no means as sophisticated as some stories say it is … I realize that in retrospect that it was just a high school science fair project and it was a proof of concept experiment and initially I thought that it could get on the market in 1-2 years however I've learned so much over the course of this journey and realized that it will in reality take a lot longer than this.
Here we get into the weeds of the claims made (most recently) by The Epoch Times, but which form the basis of other continually recurring clickbait stories.
“168 times faster, 26,000 times less expensive, and 400 times more sensitive”
In a 2012 TEDx Talk, among other places, Andraka stated that his test was “168 times faster, 26,000 times less expensive, and 400 times more sensitive.” Any reporting citing these numbers is merely repeating, uncritically, claims which misrepresent the time and cost of existing technology. One of the few existing methods for pancreatic cancer detection is via ELISA (enzyme-linked immunosorbent assay) tests. Specifically, ELISA tests that identify the presence of a chemical known as CA 19-9 can be evidence of tumors in some cases of pancreatic cancer.
According to Church, Andraka’s time comparison overstates the amount of time such ELISA tests take. The comparison Andraka used was 14 hours. ELISA, however, is fairly quick these days. A modern CA 19-9 ELISA panel, for example, takes 1.5 hours to run. Suggesting it takes 14 hours inflates significantly the speed comparison in favor of Andraka’s test.
Church also stated that the cost comparison employed by Andraka was flawed because it compared only the manufacturing costs on his invention’s side but included profit and overhead on the ELISA test side:
Andraka is comparing the commercial cost for a test – including the manufacturer’s profit and overhead – to his own materials cost. That’s not a fair comparison. He says the only mesothelin test that he found cost $912 per kit. But other ELISA tests are for sale online for $400 per 60 tests or $600 for 96 tests – in other words, about $6.50 per test run. That still compares favorably to Andraka’s $3 per 10 tests, but remember that there would also be a commercial markup if a company decided to sell his tests.
As for sensitivity, Church argued that the results presented by Andraka varied too much to say anything about sensitivity (a measure that gets at potential false positives).
A “100 percent accurate” “cancer-detection method”
Andraka was able to spend six months in the lab of a Johns Hopkins Medical Center researcher where he tested his test using blood from mice who had pancreatic cancer compared to those who didn’t. Such tests, even if they produced promising results, would not be the same as demonstrating efficacy as a diagnostic tool for pancreatic cancer in humans.
Outside of accuracy — i.e., the ability to correctly identify humans with pancreatic cancer — the early detection of the disease was also central to the project, something Andraka would not have been able to test directly for his work at Hopkins. Ira Pastan, the National Cancer Institutes researcher who discovered mesothelin, the biomarker used in Andraka’s test, told The Sydney Morning Herald that the protein is not actually elevated in the blood in early pancreatic cancer patients and thus could not be indicative of the early stages of the disease. Other scientific studies support Pastan’s objection.
A 2013 study into the diagnostic efficacy of mesothelin on pancreatic ductal adenocarcinoma (a form of pancreatic cancer) concluded that the biomarker was “a weak diagnostic classifier on its own” but may be of use as part of a larger panel test. Mesothelin is also present in cases of other cancers as well, including “malignant mesothelioma and ... ovarian, and lung adenocarcinomas. In a 2015 statement provided to Sydney Morning Herald reporter Nick Bryant by John Hopkins University, the institution made clear the results were far too preliminary to declare such accuracy or to speculate on its future success:
“[Andraka's’] idea was considered by his mentor to be of scientific interest," the hospital told me in a written statement, "but his experiments at Johns Hopkins were highly preliminary." They represented nothing more than "the very earliest stages of determining whether such a test is feasible or likely". It also noted that his results had not been confirmed, peer-reviewed or published in a scientific journal. "Thus, it is unlikely that a test based on his idea will soon be available to people."
“[Andraka] currently also holds the international patent on the device”
Andraka does not hold an international patent on his device. Although he applied for one in 2012, that application was ultimately denied based on lacking an “inventive step.”
All told, media outlets like The Epoch Times continue to run with an out-of-date and highly clickable fairytale that has not been accurate since at least 2014. Andraka himself acknowledges the shortcomings of his earlier scientific pronouncements and is thankful for the criticism he has received, telling (for example) The Sydney Morning Herald in 2015 that “I really appreciate their concerns because their concerns made my project better. It was definitely a learning curve."