Correct and incorrect recommendations inter-mingled in one-third of the chatbot’s responses, making errors more difficult to detect. 

For many patients, the Internet serves as a powerful tool for self-education on medical topics. With ChatGPT now at patients’ fingertips, researchers from Brigham and Women’s Hospital, a founding member of the Mass General Brigham healthcare system, assessed how consistently the artificial intelligence chatbot provides recommendations for cancer treatment that align with National Comprehensive Cancer Network (NCCN) guidelines. Their findings, published in JAMA Oncology, show that in approximately one-third of cases, ChatGPT 3.5 provided an inappropriate (“non-concordant”) recommendation, highlighting the need for awareness of the technology’s limitations.

“Patients should feel empowered to educate themselves about their medical conditions, but they should always discuss with a clinician, and resources on the Internet should not be consulted in isolation,” said corresponding author Danielle Bitterman, MD, of the Department of Radiation Oncology at Brigham and Women's Hospital and the Artificial Intelligence in Medicine (AIM) Program of Mass General Brigham. “ChatGPT responses can sound a lot like a human and can be quite convincing. But, when it comes to clinical decision-making, there are so many subtleties for every patient’s unique situation. A right answer can be very nuanced, and not necessarily something ChatGPT or another large language model can provide.” 

The emergence of artificial intelligence tools in health has been groundbreaking and has the potential to positively reshape the continuum of care. Mass General Brigham, as one of the nation’s top integrated academic health systems and largest innovation enterprises, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into care delivery, workforce support, and administrative processes.

Although medical decision-making can be influenced by many factors, Bitterman and colleagues chose to evaluate the extent to which ChatGPT’s recommendations aligned with the NCCN guidelines, which are used by physicians at institutions across the country. They focused on the three most common cancers (breast, prostate, and lung cancer) and prompted ChatGPT to provide a treatment approach for each cancer based on the severity of the disease. In total, the researchers included 26 unique diagnosis descriptions and used four, slightly different prompts to ask ChatGPT to provide a treatment approach, generating a total of 104 prompts.

Nearly all responses (98 percent) included at least one treatment approach that agreed with NCCN guidelines. However, the researchers found that 34 percent of these responses also included one or more non-concordant recommendations, which were sometimes difficult to detect amidst otherwise sound guidance. A non-concordant treatment recommendation was defined as one that was only partially correct; for example, for a locally advanced breast cancer, a recommendation of surgery alone, without mention of another therapy modality. Notably, complete agreement in scoring only occurred in 62 percent of cases, underscoring both the complexity of the NCCN guidelines themselves and the extent to which ChatGPT’s output could be vague or difficult to interpret.

In 12.5 percent of cases, ChatGPT produced “hallucinations,” or a treatment recommendation absent from NCCN guidelines. These included recommendations for novel therapies or curative therapies for non-curative cancers. The authors emphasized that this form of misinformation can incorrectly set patients’ expectations about treatment and potentially impact the clinician-patient relationship.

Going forward, the researchers are exploring how well both patients and clinicians can distinguish between medical advice written by a clinician versus a large language model (LLM) like ChatGPT. They are also prompting ChatGPT with more detailed clinical cases to further evaluate its clinical knowledge.

The authors used GPT-3.5-turbo-0301, one of the largest models available at the time they conducted the study, and the model class that is currently used in the open-access version of ChatGPT (a newer version, GPT-4, is only available with the paid subscription). They also used the 2021 NCCN guidelines, because GPT-3.5-turbo-0301 was developed using data up to September 2021. While results may vary if other LLMs and/or clinical guidelines are used, the researchers emphasize that many LLMs are similar in the way they are built and the limitations they possess.

“It is an open research question as to the extent LLMs provide consistent logical responses as oftentimes ‘hallucinations’ are observed,” said first author Shan Chen, MS, of the AIM Program. “Users are likely to seek answers from the LLMs to educate themselves on health-related topics---similarly to how Google searches have been used. At the same time, we need to raise awareness that LLMs are not the equivalent of trained medical professionals.”

Based on the results of this study, ChatGPT shows limited ability to recommend guidelines-based cancer treatments. While the system is able to generate recommendations that are consistent with guidelines, it is not yet reliable enough to be used as a primary source of information for cancer treatment. Further research is needed to improve the accuracy and reliability of ChatGPT's recommendations.

Researchers at West Virginia University have discovered limitations in the use of the latest official ChatGPT plugin, called Code Interpreter, by scientists who work with biological data and use computational methods to prioritize targeted treatment for cancer and genetic disorders. However, they believe that there is potential for its use in educational settings. 

“Code Interpreter is a good thing and it’s helpful in an educational setting as it makes coding in the STEM fields more accessible to students,” said Gangqing “Michael” Hu, assistant professor in the Department of Microbiology, Immunology and Cell Biology at the WVU School of Medicine and director of the Bioinformatics Core. “However, it doesn’t have the features you need for bioinformatics. These are technical issues that can be overcome. Future developments of Code Interpreter are likely to extend its use to many fields such as bioinformatics, finance, and economics.” 

Since its release in December 2022, the popular artificial intelligence chatbot ChatGPT has gained the attention of businesses, educators, and the general public. However, it didn’t quite live up to the needs of people working in biomedical research including bioinformatics — the field where computer science meets biology — who eagerly awaited OpenAI’s Code Interpreter plugin hoping it would fill the gaps.

Hu and his team put Code Interpreter to the test on a variety of tasks to evaluate its features. Their findings, published in Annals of Biomedical Engineering, show the plugin breaks down some of the barriers, but not all of them.

For example, people without a science background will have ease of access to coding, or computer programming, with a Code Interpreter. Hu said it’s also cost-effective and sparks curiosity for students to explore data analysis and boosts their interest in learning. He points out, though, that users will need to understand how to interpret data recognize whether the results are accurate, and know how to interact with the chatbot.

Bioinformaticians rely on precise coding, computer software programs, and internet access to store, analyze, and interpret biological data such as DNA and human genome used for advancements in modern medicine.  

Despite the need for improvements specific to bioinformatics, Hu said, Code Interpreter helps users determine whether a response is accurate or if it is a fictitious answer presented with confidence, known as a hallucination. 

“People know that ChatGPT can do many impressive things, but it is not good at providing a citation or reference to support its answer. If it is asked about the source to support the claim of a response, it may start to make up references,” Hu explained. “Code Interpreter provides a solution to minimize hallucinations. For questions that can be addressed through coding, the code itself serves as the source or citation. That is a significant step forward.”

Working with Hu was Lei Wang, a postdoctoral fellow in the WVU Department of Microbiology, Immunology and Cell Biology; Xijin Ge, of South Dakota State University; and Li Liu, of Arizona State University.

The team found positive results in the Code Interpreter’s ability to convert data to charts and graphs.

Suggestions for upgrades to Code Interpreter include internet access for downloading genome data, installation of software specific to bioinformatics, expansion of storage capacity, and support for additional programming languages. In addition, researchers found a need for privacy and security applications to comply with regulations such as HIPAA. 

In testing data analysis, they discovered several limitations. The plugin supports only one computer program, Python, and few of its software packages are dedicated to bioinformatics. In addition, it doesn’t allow access to internet data and cannot work with large files.

“It allows for 100 megabytes or so, but the files we’re handling are at a gigabyte level,” Hu said. “Also, it doesn’t support parallel processing needed for large datasets which results in slow performance.”

Hu said that while he anticipates more upgrades for Code Interpreter, he plans to help students learn more about the advantages of the current plugin.

“In my class next spring, I plan to introduce this plugin to help students learn about data visualization,” Hu said. “AI is a fast-moving field. I hope by that time OpenAI may overcome some of the limitations so it can be used for a broad range of bioinformatics coding.” 

Earlier this year, Hu led another study to prepare high school and college students to harness the power of ChatGPT by learning more about coding. The process employed OPTIMAL — Optimization of Prompts Through Iterative Mentoring and Assessment — to improve communication with a chatbot.

In the long run, Hu said he will continue to monitor and test new AI programming and features.

“As new products develop, I’ll just keep going,” Hu said. “There are certainly many other innovative uses awaiting to be discovered.”

Overall, the ChatGPT plugin, Code Interpreter, has the potential to be a powerful tool for students, researchers, and medical professionals alike. It can provide a more efficient and accurate way to interpret data and can be used in a variety of fields, including education, biology, and health. While there are some potential drawbacks, such as the need for additional training and the potential for errors, the benefits of this technology far outweigh the risks. With the right implementation, ChatGPT can be a valuable resource for those in the scientific and medical communities.

Due to intensifying sea ice melting in the Arctic, sunlight is now penetrating deeper and deeper into the ocean. Since marine zooplankton respond to the available light, this is also changing their behavior – especially how the tiny organisms rise and fall within the water column. As an international team of researchers led by the Alfred Wegener Institute has now shown, in the future this could lead to more frequent food shortages for the zooplankton, and to adverse effects for larger species including seals and whales. AWI is the biggest institution for polar and ocean research and science in Germany. 

In response to anthropogenic climate change, the extent and thickness of the Arctic sea ice are declining; the mean sea ice extent is currently decreasing at a rate of 13 percent per decade. As early as 2030 – as the latest studies and supercomputer simulations indicate – the North Pole could see its first ice-free summer. As a result, the physical conditions for organisms in the Arctic Ocean are changing just as visibly. For example, due to less and thinner sea ice, sunlight can penetrate much farther below the surface. As a result, under certain conditions, the primary production – i.e., the growth – of microalgae in the water and ice can increase substantially. How these changing light conditions are affecting higher trophic levels in the food chain – like zooplankton, which feed in part on microalgae – remains poorly understood. In this regard, an international team of researchers led by Dr Hauke Flores from the Alfred Wegener Institute, Helmholtz Centre for Polar and Marine Research (AWI) has now gained valuable insights. 

According to Flores: “Every day, the largest-scale mass movement of organisms on our planet takes place in the ocean – the daily migration of the zooplankton, which include tiny copepods and krill. At night, the zooplankton rises near the water’s surface to feed. When the day comes, they migrate back to the deep, protecting them from predators. Although the individual organisms are minuscule, taken together this constitutes a tremendous daily vertical motion of biomass within the water column. But in the polar regions, the migration is different – it’s seasonal; in other words, the zooplankton follow a seasonal cycle. During the months-long brightness of the Polar Day in summer, they remain in the deep; during the months-long darkness of the Polar Night in winter, part of the zooplankton rise and remain in the near-surface water just below the ice.”

Both the daily migration at lower latitudes and the seasonal migration in the polar regions are predominantly dictated by sunlight. The tiny organisms usually prefer twilight conditions. They like to stay below a certain light intensity (critical irradiance), which is usually quite low and lies well into the twilight range. When the intensity of sunlight changes in a day or the seasons, the zooplankton go where they can find their preferred light conditions, which ultimately means they rise or sink in the water column. “Particularly when it comes to the topmost 20 meters of the water column, just below the sea ice, there was no available data on the zooplankton,” Flores explains. “But it’s precisely this hard-to-reach area that’s most interesting because it’s in and just below the ice where the microalgae that the zooplankton feed on grows.” To take readings there, the team designed and built an autonomous biophysical observatory, which they moored below the ice at the end of the MOSAiC expedition with the AWI research icebreaker Polarstern in September 2020. Here – far from any light pollution due to human activities – the system was able to continually measure the light intensity below the ice and the zooplankton’s movements.

“Based on our readings, we identified an extremely low critical irradiance for the zooplankton: 0.00024 watts per square meter,” says the AWI researcher. “We then fed this parameter into our supercomputer models for simulating the sea-ice system. This allowed us to project, for a range of climate scenarios, how the depth of this irradiance level would change by the middle of this century if the sea ice grew thinner and thinner due to climate change.” What the experts found: Due to the steadily declining ice thickness, the critical irradiance level would drop to greater depths earlier and earlier in the year and wouldn’t return to the surface layer until later and later in the year. Since the zooplankton remains in waters below this critical level, their movements would mirror this change. Accordingly, in these future scenarios, they remain at greater depths longer and longer, while their time near the surface below the ice in winter grows shorter and shorter.

“In warmer future climates, the ice will form later in the autumn, resulting in reduced ice-algae production,” Flores explains. “This, in combination with their delayed rise to the surface, could lead to more frequent food shortages for the zooplankton in winter. At the same time, if the zooplankton rises earlier in the spring, it could endanger the larvae of ecologically important zooplankton species living at deeper levels, more of which could then be eaten by the adults.”

“Altogether, our study points to a previously overlooked mechanism that could further reduce Arctic zooplankton’s chances of survival shortly,” says Flores. “If that comes to pass, it will have fatal consequences for the entire ecosystem, including seals, whales, and polar bears. But our simulations also show that the impact on vertical migration will be much less pronounced if the 1.5-degree target can be reached than if greenhouse gas emissions rise unchecked. Accordingly, every tenth of a degree of anthropogenic warming that can be avoided is critical for the Arctic ecosystem.”

The study conducted by Germany's AWI shows that climate change is having a significant effect on the seasonal vertical migration of zooplankton in the Arctic. This is a worrying trend, as the zooplankton are an important part of the food web and play a critical role in the health of the Arctic ecosystem. As temperatures continue to rise, the effects of climate change on the zooplankton will likely become even more pronounced, leading to further disruption of the Arctic food web and potentially catastrophic consequences for the Arctic environment.

According to the data presented by the Atlas VPN team, in the first half of 2023, there were 49 significant cyber incidents concerning government agencies — a rise of 11% from the same time last year. The attacks affected government bodies in at least 27 countries across the world.

The analysis is based on the information by the Center for Strategic and International Studies, which keeps track of significant cyber incidents. We focused only on cases involving government agencies, their representatives, or contractors.

This year, government agencies in the United States encountered the greatest number of attacks, with 16% specifically aimed at the country. The extended conflict between Russia and Ukraine has also resulted in several cyber incidents against state government entities in the countries.

When it comes to potential perpetrators, Russian hackers are at the forefront, believed to be responsible for approximately 29% of these attacks. Following closely are cybercriminals associated with China, accounting for 18%, while Iran ranks third with 10%.

Government agencies accumulate and store a significant amount of sensitive data, such as personal information about individual citizens. This data can be sold on the dark web or held hostage until a ransom is paid, which makes these agencies an attractive target for cybercriminals.

Apart from monetary motives, roughly a quarter (12) of all cyberattacks targeting government agencies worldwide in 2023 can be attributed to state-linked threat actors engaged in cyberespionage campaigns. Additionally, hacktivist activity played a role, constituting approximately 10% of the recorded incidents within the government sector during the first half of 2023.

Since 2005, government agencies worldwide have experienced a total of 616 significant cyberattacks. More than half of these — 56% — occurred just in the last five years, including the first half of 2023.

In conclusion, it appears that cyberattacks against governments are on the rise in 2023 and will likely continue to increase in the years to come. This is concerning as governments are struggling to keep up with the ever-evolving technology and the sophistication of these attacks. Without a concerted effort to improve security measures, the situation is likely to worsen, leaving governments vulnerable to malicious actors and potentially devastating consequences.

The full report is available here: https://atlasvpn.com/blog/cyberattacks-against-governments-are-on-the-rise-in-2023