Katharine Hayhoe: The most important climate equation | Starmus highlights

The atmospheric scientist makes a compelling case for a head-to-heart-to-hands connection as a catalyst for climate action

/by
,

How we kept the Google Play & Android app ecosystems safe in 2024

Posted by Bethel Otuteye and Khawaja Shams (Android Security and Privacy Team), and Ron Aquino (Play Trust and Safety)

Android and Google Play comprise a vibrant ecosystem with billions of users around the globe and millions of helpful apps. Keeping this ecosystem safe for users and developers remains our top priority. However, like any flourishing ecosystem, it also attracts its share of bad actors. That’s why every year, we continue to invest in more ways to protect our community and fight bad actors, so users can trust the apps they download from Google Play and developers can build thriving businesses.

Last year, those investments included AI-powered threat detection, stronger privacy policies, supercharged developer tools, new industry-wide alliances, and more. As a result, we prevented 2.36 million policy-violating apps from being published on Google Play and banned more than 158,000 bad developer accounts that attempted to publish harmful apps.

But that was just the start. For more, take a look at our recent highlights from 2024:

Google’s advanced AI: helping make Google Play a safer place

To keep out bad actors, we have always used a combination of human security experts and the latest threat-detection technology. In 2024, we used Google’s advanced AI to improve our systems’ ability to proactively identify malware, enabling us to detect and block bad apps more effectively. It also helps us streamline review processes for developers with a proven track record of policy compliance. Today, over 92% of our human reviews for harmful apps are AI-assisted, allowing us to take quicker and more accurate action to help prevent harmful apps from becoming available on Google Play.

That’s enabled us to stop more bad apps than ever from reaching users through the Play Store, protecting users from harmful or malicious apps before they can cause any damage.

Working with developers to enhance security and privacy on Google Play

To protect user privacy, we’re working with developers to reduce unnecessary access to sensitive data. In 2024, we prevented 1.3 million apps from getting excessive or unnecessary access to sensitive user data. We also required apps to be more transparent about how they handle user information by launching new developer requirements and a new “Data deletion” option for apps that support user accounts and data collection. This helps users manage their app data and understand the app’s deletion practices, making it easier for Play users to delete data collected from third-party apps.

We also worked to ensure that apps use the strongest and most up-to-date privacy and security capabilities Android has to offer. Every new version of Android introduces new security and privacy features, and we encourage developers to embrace these advancements as soon as possible. As a result of partnering closely with developers, over 91% of app installs on the Google Play Store now use the latest protections of Android 13 or newer.

Safeguarding apps from scams and fraud is an ongoing battle for developers. The Play Integrity API allows developers to check if their apps have been tampered with or are running in potentially compromised environments, helping them to prevent abuse like fraud, bots, cheating, and data theft. Play Integrity API and Play’s automatic protection helps developers ensure that users are using the official Play version of their app with the latest security updates. Apps using Play integrity features are seeing 80% lower usage from unverified and untrusted sources on average.

We’re also constantly working to improve the safety of apps on Play at scale, such as with the Google Play SDK Index. This tool offers insights and data to help developers make more informed decisions about the safety of an SDK. Last year, in addition to adding 80 SDKs to the index, we also worked closely with SDK and app developers to address potential SDK security and privacy issues, helping to build safer and more secure apps for Google Play.

Google Play’s multi-layered protections against bad apps

To create a trusted experience for everyone on Google Play, we use our SAFE principles as a guide, incorporating multi-layered protections that are always evolving to help keep Google Play safe. These protections start with the developers themselves, who play a crucial role in building secure apps. We provide developers with best-in-class tools, best practices, and on-demand training resources for building safe, high-quality apps. Every app undergoes rigorous review and testing, with only approved apps allowed to appear in the Play Store. Before a user downloads an app from Play, users can explore its user reviews, ratings, and Data safety section on Google Play to help them make an informed decision. And once installed, Google Play Protect, Android’s built-in security protection, helps to shield their Android device by continuously scanning for malicious app behavior.

Enhancing Google Play Protect to help keep users safe on Android

While the Play Store offers best-in-class security, we know it’s not the only place users download Android apps – so it’s important that we also defend Android users from more generalized mobile threats. To do this in an open ecosystem, we’ve invested in sophisticated, real-time defenses that protect against scams, malware, and abusive apps. These intelligent security measures help to keep users, user data, and devices safe, even if apps are installed from various sources with varying levels of security.


Google Play Protect automatically scans every app on Android devices with Google Play Services, no matter the download source. This built-in protection, enabled by default, provides crucial security against malware and unwanted software. Google Play Protect scans more than 200 billion apps daily and performs real-time scanning at the code-level on novel apps to combat emerging and hidden threats, like polymorphic malware. In 2024, Google Play Protect’s real-time scanning identified more than 13 million new malicious apps from outside Google Play1.

Google Play Protect is always evolving to combat new threats and protect users from harmful apps that can lead to scams and fraud. Here are some of the new improvements that are now available globally on Android devices with Google Play Services:

  • Reminder notifications in Chrome on Android to re-enable Google Play Protect: According to our research, more than 95 percent of app installations from major malware families that exploit sensitive permissions highly correlated to financial fraud came from Internet-sideloading sources like web browsers, messaging apps, or file managers. To help users stay protected when browsing the web, Chrome will now display a reminder notification to re-enable Google Play Protect if it has been turned off.
  • Additional protection against social engineering attacks: Scammers may manipulate users into disabling Play Protect during calls to download malicious Internet-sideloaded apps. To prevent this, the Play Protect app scanning toggle is now temporarily disabled during phone or video calls. This safeguard is enabled by default during traditional phone calls as well as during voice and video calls in popular third-party apps.
  • Automatically revoking app permissions for potentially dangerous apps: Since Android 11, we’ve taken a proactive approach to data privacy by automatically resetting permissions for apps that users haven’t used in a while. This ensures apps can only access the data they truly need, and users can always grant permissions back if necessary. To further enhance security, Play Protect now automatically revokes permissions for potentially harmful apps, limiting their access to sensitive data like storage, photos, and camera. Users can restore app permissions at any time, with a confirmation step for added security.

Google Play Protect’s enhanced fraud protection pilot analyzes and automatically blocks the installation of apps that may use sensitive permissions frequently abused for financial fraud when the user attempts to install the app from an Internet-sideloading source (web browsers, messaging apps, or file managers).

Building on the success of our initial pilot in partnership with the Cyber Security Agency of Singapore (CSA), additional enhanced fraud protection pilots are now active in nine regions – Brazil, Hong Kong, India, Kenya, Nigeria, Philippines, South Africa, Thailand, and Vietnam.

In 2024, Google Play Protect’s enhanced fraud protection pilots have shielded 10 million devices from over 36 million risky installation attempts, encompassing over 200,000 unique apps.

By piloting these new protections, we can proactively combat emerging threats and refine our solutions to thwart scammers and their increasingly sophisticated fraud attempts. We look forward to continuing to partner with governments, ecosystem partners, and other stakeholders to improve user protections.

App badging to help users find apps they can trust at a glance on Google Play

In 2024, we introduced a new badge for government developers to help users around the world identify official government apps. Government apps are often targets of impersonation due to the highly sensitive nature of the data users provide, giving bad actors the ability to steal identities and commit financial fraud. Badging verified government apps is an important step in helping connect people with safe, high-quality, useful, and relevant experiences. We partner closely with global governments and are already exploring ways to build on this work.

We also recently introduced a new badge to help Google Play users discover VPN apps that take extra steps to demonstrate their strong commitment to security. We allow developers who adhere to Play safety and security guidelines and have passed an additional independent Mobile Application Security Assessment (MASA) to display a dedicated badge in the Play Store to highlight their increased commitment to safety.

Collaborating to advance app security standards

In addition to our partnerships with governments, developers, and other stakeholders, we also worked with our industry peers to protect the entire app ecosystem for everyone. The App Defense Alliance, in partnership with fellow steering committee members Microsoft and Meta, recently launched the ADA Application Security Assessment (ASA) v1.0, a new standard to help developers build more secure mobile, web, and cloud applications. This standard provides clear guidance on protecting sensitive data, defending against cyberattacks, and ultimately, strengthening user trust. This marks a significant step forward in establishing industry-wide security best practices for application development.

All developers are encouraged to review and comply with the new mobile security standard. You’ll see this standard in action for all carrier apps pre-installed on future Pixel phone models.

Looking ahead


This year, we’ll continue to protect the Android and Google Play ecosystem, building on these tools and resources in response to user and developer feedback and the changing landscape. As always, we’ll keep empowering developers to build safer apps more easily, streamline their policy experience, and protect their businesses and users from bad actors.

1 Based on Google Play Protect 2024 internal data.

/by

How we estimate the risk from prompt injection attacks on AI systems

Posted by the Agentic AI Security Team at Google DeepMind

Modern AI systems, like Gemini, are more capable than ever, helping retrieve data and perform actions on behalf of users. However, data from external sources present new security challenges if untrusted sources are available to execute instructions on AI systems. Attackers can take advantage of this by hiding malicious instructions in data that are likely to be retrieved by the AI system, to manipulate its behavior. This type of attack is commonly referred to as an “indirect prompt injection,” a term first coined by Kai Greshake and the NVIDIA team.



To mitigate the risk posed by this class of attacks, we are actively deploying defenses within our AI systems along with measurement and monitoring tools. One of these tools is a robust evaluation framework we have developed to automatically red-team an AI system’s vulnerability to indirect prompt injection attacks. We will take you through our threat model, before describing three attack techniques we have implemented in our evaluation framework.



Threat model and evaluation framework



Our threat model concentrates on an attacker using indirect prompt injection to exfiltrate sensitive information, as illustrated above. The evaluation framework tests this by creating a hypothetical scenario, in which an AI agent can send and retrieve emails on behalf of the user. The agent is presented with a fictitious conversation history in which the user references private information such as their passport or social security number. Each conversation ends with a request by the user to summarize their last email, and the retrieved email in context.



The contents of this email are controlled by the attacker, who tries to manipulate the agent into sending the sensitive information in the conversation history to an attacker-controlled email address. The attack is successful if the agent executes the malicious prompt contained in the email, resulting in the unauthorized disclosure of sensitive information. The attack fails if the agent only follows user instructions and provides a simple summary of the email. 



Automated red-teaming

Crafting successful indirect prompt injections requires an iterative process of refinement based on observed responses. To automate this process, we have developed a red-team framework consisting of several optimization-based attacks that generate prompt injections (in the example above this would be different versions of the malicious email). These optimization-based attacks are designed to be as strong as possible; weak attacks do little to inform us of the susceptibility of an AI system to indirect prompt injections.



Once these prompt injections have been constructed, we measure the resulting attack success rate on a diverse set of conversation histories. Because the attacker has no prior knowledge of the conversation history, to achieve a high attack success rate the prompt injection must be capable of extracting sensitive user information contained in any potential conversation contained in the prompt, making this a harder task than eliciting generic unaligned responses from the AI system. The attacks in our framework include:



Actor Critic: This attack uses an attacker-controlled model to generate suggestions for prompt injections. These are passed to the AI system under attack, which returns a probability score of a successful attack. Based on this probability, the attack model refines the prompt injection. This process repeats until the attack model converges to a successful prompt injection. 



Beam Search: This attack starts with a naive prompt injection directly requesting that the AI system send an email to the attacker containing the sensitive user information. If the AI system recognizes the request as suspicious and does not comply, the attack adds random tokens to the end of the prompt injection and measures the new probability of the attack succeeding. If the probability increases, these random tokens are kept, otherwise they are removed, and this process repeats until the combination of the prompt injection and random appended tokens result in a successful attack.

Tree of Attacks w/ Pruning (TAP): Mehrotra et al. (2024) [3] designed an attack to generate prompts that cause an AI system to violate safety policies (such as generating hate speech). We adapt this attack, making several adjustments to target security violations. Like Actor Critic, this attack searches in the natural language space; however, we assume the attacker cannot access probability scores from the AI system under attack, only the text samples that are generated.



We are actively leveraging insights gleaned from these attacks within our automated red-team framework to protect current and future versions of AI systems we develop against indirect prompt injection, providing a measurable way to track security improvements. A single silver bullet defense is not expected to solve this problem entirely. We believe the most promising path to defend against these attacks involves a combination of robust evaluation frameworks leveraging automated red-teaming methods, alongside monitoring, heuristic defenses, and standard security engineering solutions. 




We would like to thank Vijay Bolina, Sravanti Addepalli, Lihao Liang, and Alex Kaskasoli for their prior contributions to this work.




Posted on behalf of the entire Google DeepMind Agentic AI Security team (listed in alphabetical order):

Aneesh Pappu, Andreas Terzis, Chongyang Shi, Gena Gibson, Ilia Shumailov, Itay Yona, Jamie Hayes, John “Four” Flynn, Juliette Pluto, Sharon Lin, Shuang Song

/by
,

Android enhances theft protection with Identity Check and expanded features

Posted by Jianing Sandra Guo, Product Manager, Android, Nataliya Stanetsky, Staff Program Manager, Android

Today, people around the world rely on their mobile devices to help them stay connected with friends and family, manage finances, keep track of healthcare information and more – all from their fingertips. But a stolen device in the wrong hands can expose sensitive data, leaving you vulnerable to identity theft, financial fraud and privacy breaches.

This is why we recently launched Android theft protection, a comprehensive suite of features designed to protect you and your data at every stage – before, during, and after device theft. As part of our commitment to help you stay safe on Android, we’re expanding and enhancing these features to deliver even more robust protection to more users around the world.

Identity Check rolling out to Pixel and Samsung One UI 7 devices

We’re officially launching Identity Check, first on Pixel and Samsung Galaxy devices eligible for One UI 71, to provide better protection for your critical account and device settings. When you turn on Identity Check, your device will require explicit biometric authentication to access certain sensitive resources when you’re outside of trusted locations. Identity Check also enables enhanced protection for Google Accounts on all supported devices and additional security for Samsung Accounts on One UI 7 eligible Galaxy devices, making it much more difficult for an unauthorized attacker to take over accounts signed in on the device.

As part of enabling Identity Check, you can designate one or more trusted locations. When you’re outside of these trusted places, biometric authentication will be required to access critical account and device settings, like changing your device PIN or biometrics, disabling theft protection, or accessing Passkeys.

Identity Check gives you more peace of mind that your most sensitive device assets are protected against unauthorized access, even if a thief or bad actor manages to learn your device PIN.

Identity Check is rolling out now to Pixel devices with Android 15 and will be available on One UI 7 eligible Galaxy devices in the coming weeks. It will roll out to supported Android devices from other manufacturers later this year.

Theft Detection Lock: expanding AI-powered protection to more users

One of the top theft protection features introduced last year was Theft Detection Lock, which uses an on-device AI-powered algorithm to help detect when your phone may be forcibly taken from you. If the machine learning algorithm detects a potential theft attempt on your unlocked device, it locks your screen to keep thieves out.

Theft Detection Lock is now fully rolled out to Android 10+ phones2 around the world.

Protecting your Android device from theft

We’re collaborating with the GSMA and industry experts to combat mobile device theft by sharing information, tools and prevention techniques. Stay tuned for an upcoming GSMA white paper, developed in partnership with the mobile industry, with more information on protecting yourself and your organization from device theft.

With the addition of Identity Check and the ongoing enhancements to our existing features, Android offers a robust and comprehensive set of tools to protect your devices and your data from theft. We’re dedicated to providing you with peace of mind, knowing your personal information is safe and secure.

You can turn on the new Android theft features by clicking here on a supported Android device. Learn more about our theft protection features by visiting our help center.

Notes

  1. Timing, availability and feature names may vary in One UI 7. ↩

  2. With the exclusion for Android Go smartphones ↩

/by

OSV-SCALIBR: A library for Software Composition Analysis

Posted by Erik Varga, Vulnerability Management, and Rex Pan, Open Source Security Team



In December 2022, we announced OSV-Scanner, a tool to enable developers to easily scan for vulnerabilities in their open source dependencies. Together with the open source community, we’ve continued to build this tool, adding remediation features, as well as expanding ecosystem support to 11 programming languages and 20 package manager formats. 



Today, we’re excited to release OSV-SCALIBR (Software Composition Analysis LIBRary), an extensible library for SCA and file system scanning. OSV-SCALIBR combines Google’s internal vulnerability management expertise into one scanning library with significant new capabilities such as:


  • SCA for installed packages, standalone binaries, as well as source code

  • OSes package scanning on Linux (COS, Debian, Ubuntu, RHEL, and much more), Windows, and Mac

  • Artifact and lockfile scanning in major language ecosystems (Go, Java, Javascript, Python, Ruby, and much more)

  • Vulnerability scanning tools such as weak credential detectors for Linux, Windows, and Mac

  • SBOM generation in SPDX and CycloneDX, the two most popular document formats

  • Optimization for on-host scanning of resource constrained environments where performance and low resource consumption is critical



OSV-SCALIBR is now the primary SCA engine used within Google for live hosts, code repos, and containers. It’s been used and tested extensively across many different products and internal tools to help generate SBOMs, find vulnerabilities, and help protect our users’ data at Google scale.


We offer OSV-SCALIBR primarily as an open source Go library today, and we’re working on adding its new capabilities into OSV-Scanner as the primary CLI interface.


Using OSV-SCALIBR as a library

All of OSV-SCALIBR’s capabilities are modularized into plugins for software extraction and vulnerability detection which are very simple to expand.You can use OSV-SCALIBR as a library to:

1.Generate SBOMs from the build artifacts and code repos on your live host:

import (

 “context”

 “github.com/google/osv-scalibr”

 “github.com/google/osv-scalibr/converter”

 “github.com/google/osv-scalibr/extractor/filesystem/list”

 “github.com/google/osv-scalibr/fs”

 “github.com/google/osv-scalibr/plugin”

 spdx “github.com/spdx/tools-golang/spdx/v2/v2_3”

)

func GenSBOM(ctx context.Context) *spdx.Document {

 capab := &plugin.Capabilities{OS: plugin.OSLinux}

 cfg := &scalibr.ScanConfig{

   ScanRoots: fs.RealFSScanRoots(“/”),

   FilesystemExtractors: list.FromCapabilities(capab),

   Capabilities: capab,

 }

 result := scalibr.New().Scan(ctx, cfg)

 return converter.ToSPDX23(result, converter.SPDXConfig{})

}

2. Scan a git repo for SBOMs:

Simply replace “/” with the path to your git repo. Also take a look at the various language extractors to enable for code scanning.

3. Scan a remote container for SBOMs:

Replace the scan config from the above code snippet with

import (

 …

 “github.com/google/go-containerregistry/pkg/authn”

 “github.com/google/go-containerregistry/pkg/v1/remote”

 “github.com/google/osv-scalibr/artifact/image”

 …

)

…

filesys, _ := image.NewFromRemoteName(

 “alpine:latest”,

 remote.WithAuthFromKeychain(authn.DefaultKeychain),

)

cfg := &scalibr.ScanConfig{

 ScanRoots: []*fs.ScanRoot{{FS: filesys}},

 …

}

4. Find vulnerabilities on your filesystem or a remote container:

Extract the PURLs from the SCALIBR inventory results from the previous steps:

import (

 …

 “github.com/google/osv-scalibr/converter”

 …

)

…

result := scalibr.New().Scan(ctx, cfg)

for _, i := range result.Inventories {

 fmt.Println(converter.ToPURL(i))

}

And send them to osv.dev, e.g.

$ curl -d ‘{“package”: {“purl”: “pkg:npm/dojo@1.2.3”}}’ “https://api.osv.dev/v1/query”

See the usage docs for more details.

OSV-Scanner + OSV-SCALIBR

Users looking for an out-of-the-box vulnerability scanning CLI tool should check out OSV-Scanner, which already provides comprehensive language package scanning capabilities using much of the same extraction as OSV-SCALIBR. 


Some of OSV-SCALIBR’s capabilities are not yet available in OSV-Scanner, but we’re currently working on integrating OSV-SCALIBR more deeply into OSV-Scanner. This will make more and more of OSV-SCALIBR’s capabilities available in OSV-Scanner in the next few months, including installed package extraction, weak credentials scanning, SBOM generation, and more.


Look out soon for an announcement of OSV-Scanner V2 with many of these new features available. OSV-Scanner will become the primary frontend to the OSV-SCALIBR library for users who require a CLI interface. Existing users of OSV-Scanner can continue to use the tool the same way, with backwards compatibility maintained for all existing use cases. 


For installation and usage instructions, have a look at OSV-Scanner’s documentation here.


What’s next

In addition to making all of OSV-SCALIBR’s features available in OSV-Scanner, we’re also working on additional new capabilities. Here’s some of the things you can expect:

  • Support for more OS and language ecosystems, both for regular extraction and for Guided Remediation

  • Layer attribution and base image identification for container scanning

  • Reachability analysis to reduce false positive vulnerability matches

  • More vulnerability and misconfiguration detectors for Windows

  • More weak credentials detectors

We hope that this library helps developers and organizations to secure their software and encourages the open source community to contribute back by sharing new plugins on top of OSV-SCALIBR.

If you have any questions or if you would like to contribute, don’t hesitate to reach out to us at osv-discuss@google.com or by posting an issue in our issue tracker.

/by

Google Cloud expands vulnerability detection for Artifact Registry using OSV

Posted by Greg Mucci, Product Manager, Artifact Analysis, Oliver Chang, Senior Staff Engineering, OSV, and Charl de Nysschen, Product Manager OSV


DevOps teams dedicated to securing their supply chain and predicting potential risks consistently face novel threats. Fortunately, they can now improve their image and container security by harnessing Google-grade vulnerability scanning, which offers expanded open-source coverage. A significant benefit of utilizing Google Cloud Platform is its integrated security tools, including Artifact Analysis. This scanning service leverages the same infrastructure that Google depends on to monitor vulnerabilities within its internal systems and software supply chains.


Artifact Analysis has recently expanded its scanning coverage to eight additional language packages, four operating systems, and two extensively utilized base images, making it a more robust and versatile tool than ever before.   


This enhanced coverage was achieved by integrating Artifact Analysis with the Open Source Vulnerabilities (OSV) platform and database. This integration provides industry-leading insights into open source vulnerabilities—a crucial capability as software supply chain attacks continue to grow in frequency and complexity, impacting organizations reliant on open source software.


With these recent updates, customers can now successfully scan the vast majority of the images they push to Artifact Registry. These successful scans ensure that any known vulnerabilities are detected, reported, and can be integrated into a broader vulnerability management program, allowing teams to take prompt action.



Open source vulnerabilities, with more reach 

Artifact Analysis pulls vulnerability information directly from OSV, which is the only open source, distributed vulnerability database that gets information directly from open source practitioners. OSV’s database provides a consistent, high quality, high fidelity database of vulnerabilities from authoritative sources who have adopted the OSV schema. This ensures the database has accurate information to reliably match software dependencies to known vulnerabilities—previously a difficult process reliant on inaccurate mechanisms such as CPEs (Common Platform Enumerations). 


Over the past three years, OSV has increased its total coverage to 28 language and OS ecosystems. For example, industry leaders such as GitHub, Chainguard, and Ubuntu, as well as open source ecosystems such as Rust and Python are now exporting their vulnerability discoveries in the OSV Schema. This increased coverage also includes Chainguard’s Wolfi images and Google’s Distroless images, which are popular choices for minimal container images used by many developers and organizations. Customers who rely on distroless images can count on Artifact Analysis scanning to support their minimal container image initiatives.  Each expansion in OSV’s coverage is incorporated into scanning tools that integrate with the OSV database.



Broader vulnerability detection with Artifact Analysis 

As a result of OSV’s expansion, scanners like Artifact Analysis that draw from OSV now alert users to higher quality vulnerability information across a broader set of ecosystems—meaning GCP project owners will be made aware of a more complete set of vulnerability findings and potential security risks. 


Existing Artifact Registry scanning customers don’t need to take any action to take advantage of this update. Projects that have scanning enabled will immediately benefit from this expanded coverage and vulnerability findings will continue to be available in the Artifact Registry UI, Container Analysis API, and via pub/sub (for workflows).


Existing On Demand scanning customers will also benefit from this expanded vulnerability coverage. All the same Operating Systems and Language package coverage that Registry Scanning customers enjoy are available in On Demand Scan. 



Beyond Artifact Registry 

We know that detection is just one of the first steps necessary to manage risks. We’re continually expanding Artifact Analysis capabilities and in 2025 we’ll be integrating Artifact Registry vulnerability findings with Google Cloud’s Security Command Center. Through Security Command Center customers can maintain a more comprehensive vulnerability management program, and prioritize risk across a number of different dimensions. 

/by
,

Announcing the launch of Vanir: Open-source Security Patch Validation

Today, we are announcing the availability of Vanir, a new open-source security patch validation tool. Introduced at Android Bootcamp in April, Vanir gives Android platform developers the power to quickly and efficiently scan their custom platform code for missing security patches and identify applicable available patches. Vanir significantly accelerates patch validation by automating this process, allowing OEMs to ensure devices are protected with critical security updates much faster than traditional methods. This strengthens the security of the Android ecosystem, helping to keep Android users around the world safe.

By open-sourcing Vanir, we aim to empower the broader security community to contribute to and benefit from this tool, enabling wider adoption and ultimately improving security across various ecosystems. While initially designed for Android, Vanir can be easily adapted to other ecosystems with relatively small modifications, making it a versatile tool for enhancing software security across the board. In collaboration with the Google Open Source Security Team, we have incorporated feedback from our early adopters to improve Vanir and make it more useful for security professionals. This tool is now available for you to start developing on top of, and integrating into, your systems.

The need for Vanir

The Android ecosystem relies on a multi-stage process for vulnerability mitigation. When a new vulnerability is discovered, upstream AOSP developers create and release upstream patches. The downstream device and chip manufacturers then assess the impact on their specific devices and backport the necessary fixes. This process, while effective, can present scalability challenges, especially for manufacturers managing a diverse range of devices and old models with complex update histories. Managing patch coverage across diverse and customized devices often requires considerable effort due to the manual nature of backporting.

To streamline the vital security workflow, we developed Vanir. Vanir provides a scalable and sustainable solution for security patch adoption and validation, helping to ensure Android devices receive timely protection against potential threats.

The power of Vanir

Source-code-based static analysis

Vanir’s first-of-its-kind approach to Android security patch validation uses source-code-based static analysis to directly compare the target source code against known vulnerable code patterns. Vanir does not rely on traditional metadata-based validation mechanisms, such as version numbers, repository history and build configs, which can be prone to errors. This unique approach enables Vanir to analyze entire codebases with full history, individual files, or even partial code snippets.

A main focus of Vanir is to automate the time consuming and costly process of identifying missing security patches in the open source software ecosystem. During the early development of Vanir, it became clear that manually identifying a high-volume of missing patches is not only labor intensive but also can leave user devices inadvertently exposed to known vulnerabilities for a period of time. To address this, Vanir utilizes novel automatic signature refinement techniques and multiple pattern analysis algorithms, inspired by the vulnerable code clone detection algorithms proposed by Jang et al. [1] and Kim et al. [2]. These algorithms have low false-alarm rates and can effectively handle broad classes of code changes that might appear in code patch processes. In fact, based on our 2-year operation of Vanir, only 2.72% of signatures triggered false alarms. This allows Vanir to efficiently find missing patches, even with code changes, while minimizing unnecessary alerts and manual review efforts.

Vanir’s source-code-based approach also enables rapid scaling across any ecosystem. It can generate signatures for any source files written in supported languages. Vanir’s signature generator automatically generates, tests, and refines these signatures, allowing users to quickly create signatures for new vulnerabilities in any ecosystem simply by providing source files with security patches.

Android’s successful use of Vanir highlights its efficiency compared to traditional patch verification methods. A single engineer used Vanir to generate signatures for over 150 vulnerabilities and verify missing security patches across its downstream branches – all within just five days.

Vanir for Android

Currently Vanir supports C/C++ and Java targets and covers 95% of Android kernel and userspace CVEs with public security patches. Google Android Security team consistently incorporates the latest CVEs into Vanir’s coverage to provide a complete picture of the Android ecosystem’s patch adoption risk profile.

The Vanir signatures for Android vulnerabilities are published through the Open Source Vulnerabilities (OSV) database. This allows Vanir users to seamlessly protect their codebases against latest Android vulnerabilities without any additional updates. Currently, there are over 2,000 Android vulnerabilities in OSV, and finishing scanning an entire Android source tree can take 10-20 minutes with a modern PC.

Flexible integration, adoption and expansion.

Vanir is developed not only as a standalone application but also as a Python library. Users who want to integrate automated patch verification processes with their continuous build or test chain may easily achieve it by wiring their build integration tool with Vanir scanner libraries. For instance, Vanir is integrated with a continuous testing pipeline in Google, ensuring all security patches are adopted in ever-evolving Android codebase and their first-party downstream branches.

Vanir is also fully open-sourced, and under BSD-3 license. As Vanir is not fundamentally limited to the Android ecosystem, you may easily adopt Vanir for the ecosystem that you want to protect by making relatively small modifications in Vanir. In addition, since Vanir’s underlying algorithm is not limited to security patch validation, you may modify the source and use it for different purposes such as licensed code detection or code clone detection. The Android Security team welcomes your contributions to Vanir for any direction that may expand its capability and scope. You can also contribute to Vanir by providing vulnerability data with Vanir signatures to OSV.

Vanir Results

Since early last year, we have partnered with several Android OEMs to test the tool’s effectiveness. Internally we have been able to integrate the tool into our build system continuously testing against over 1,300 vulnerabilities. Currently Vanir covers 95% of all Android, Wear, and Pixel vulnerabilities with public fixes across Android Kernel and Userspace. It has a 97% accuracy rate, which has saved our internal teams over 500 hours to date in patch fix time.

Next steps

We are happy to announce that Vanir is now available for public use. Vanir is not technically limited to Android, and we are also actively exploring problems that Vanir may help address, such as general C/C++ dependency management via integration with OSV-scanner. If you are interested in using or contributing to Vanir, please visit github.com/google/vanir. Please join our public community to submit your feedback and questions on the tool. 

We look forward to working with you on Vanir!

Notes

[1] J. Jang, A. Agrawal and D. Brumley, “ReDeBug: Finding Unpatched Code Clones in Entire OS Distributions,” 2012 IEEE Symposium on Security and Privacy, San Francisco, CA, USA, 2012, pp. 48-62, doi: 10.1109/SP.2012.13.

[2] S. Kim, S. Woo, H. Lee and H. Oh, “VUDDY: A Scalable Approach for Vulnerable Code Clone Discovery,” 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA, 2017, pp. 595-614, doi: 10.1109/SP.2017.62.

/by

Leveling Up Fuzzing: Finding more vulnerabilities with AI

Posted by Oliver Chang, Dongge Liu and Jonathan Metzman, Google Open Source Security Team

Recently, OSS-Fuzz reported 26 new vulnerabilities to open source project maintainers, including one vulnerability in the critical OpenSSL library (CVE-2024-9143) that underpins much of internet infrastructure. The reports themselves aren’t unusual—we’ve reported and helped maintainers fix over 11,000 vulnerabilities in the 8 years of the project. 


But these particular vulnerabilities represent a milestone for automated vulnerability finding: each was found with AI, using AI-generated and enhanced fuzz targets. The OpenSSL CVE is one of the first vulnerabilities in a critical piece of software that was discovered by LLMs, adding another real-world example to a recent Google discovery of an exploitable stack buffer underflow in the widely used database engine SQLite.


This blog post discusses the results and lessons over a year and a half of work to bring AI-powered fuzzing to this point, both in introducing AI into fuzz target generation and expanding this to simulate a developer’s workflow. These efforts continue our explorations of how AI can transform vulnerability discovery and strengthen the arsenal of defenders everywhere.


The story so far

In August 2023, the OSS-Fuzz team announced AI-Powered Fuzzing, describing our effort to leverage large language models (LLM) to improve fuzzing coverage to find more vulnerabilities automatically—before malicious attackers could exploit them. Our approach was to use the coding abilities of an LLM to generate more fuzz targets, which are similar to unit tests that exercise relevant functionality to search for vulnerabilities. 


The ideal solution would be to completely automate the manual process of developing a fuzz target end to end:


  1. Drafting an initial fuzz target.

  2. Fixing any compilation issues that arise. 

  3. Running the fuzz target to see how it performs, and fixing any obvious mistakes causing runtime issues.

  4. Running the corrected fuzz target for a longer period of time, and triaging any crashes to determine the root cause.

  5. Fixing vulnerabilities. 


In August 2023, we covered our efforts to use an LLM to handle the first two steps. We were able to use an iterative process to generate a fuzz target with a simple prompt including hardcoded examples and compilation errors. 


In January 2024, we open sourced the framework that we were building to enable an LLM to generate fuzz targets. By that point, LLMs were reliably generating targets that exercised more interesting code coverage across 160 projects. But there was still a long tail of projects where we couldn’t get a single working AI-generated fuzz target.


To address this, we’ve been improving the first two steps, as well as implementing steps 3 and 4.


New results: More code coverage and discovered vulnerabilities

We’re now able to automatically gain more coverage in 272 C/C++ projects on OSS-Fuzz (up from 160), adding 370k+ lines of new code coverage. The top coverage improvement in a single project was an increase from 77 lines to 5434 lines (a 7000% increase).


This led to the discovery of 26 new vulnerabilities in projects on OSS-Fuzz that already had hundreds of thousands of hours of fuzzing. The highlight is CVE-2024-9143 in the critical and well-tested OpenSSL library. We reported this vulnerability on September 16 and a fix was published on October 16. As far as we can tell, this vulnerability has likely been present for two decades and wouldn’t have been discoverable with existing fuzz targets written by humans.


Another example was a bug in the project cJSON, where even though an existing human-written harness existed to fuzz a specific function, we still discovered a new vulnerability in that same function with an AI-generated target. 


One reason that such bugs could remain undiscovered for so long is that line coverage is not a guarantee that a function is free of bugs. Code coverage as a metric isn’t able to measure all possible code paths and states—different flags and configurations may trigger different behaviors, unearthing different bugs. These examples underscore the need to continue to generate new varieties of fuzz targets even for code that is already fuzzed, as has also been shown by Project Zero in the past (1, 2).


New improvements

To achieve these results, we’ve been focusing on two major improvements:

  1. Automatically generate more relevant context in our prompts. The more complete and relevant information we can provide the LLM about a project, the less likely it would be to hallucinate the missing details in its response. This meant providing more accurate, project-specific context in prompts, such as function, type definitions, cross references, and existing unit tests for each project. To generate this information automatically, we built new infrastructure to index projects across OSS-Fuzz. 


  1. LLMs turned out to be highly effective at emulating a typical developer’s entire workflow of writing, testing, and iterating on the fuzz target, as well as triaging the crashes found. Thanks to this, it was possible to further automate more parts of the fuzzing workflow. This additional iterative feedback in turn also resulted in higher quality and greater number of correct fuzz targets. 



The workflow in action

Our LLM can now execute the first four steps of the developer’s process (with the fifth soon to come). 


1. Drafting an initial fuzz target

A developer might check the source code, existing documentation and unit tests, as well as  usages of the target function when to draft an initial fuzz target. An LLM can fulfill this role here, if we provide a prompt with this information and ask it to come up with a fuzz target. 

Prompt: 

Your goal is to write a fuzzing harness for the provided function-under-test signature using <code>LLVMFuzzerTestOneInput</code>. It is important that the provided solution compiles and actually calls the function-under-test specified by the function signature:

<function signature>

unsigned char * buffer_append_base64_decode(buffer *, const char *, size_t, base64_charset)

</function signature>

Here is the source code of the function being tested:

<code>

unsigned char* buffer_append_base64_decode(buffer *out, const char* in, size_t in_length, base64_charset charset) {

    const size_t reserve = 3*(in_length/4) + 3;

    unsigned char * const result = (unsigned char *)

      buffer_string_prepare_append(out, reserve);

    const size_t out_pos =

      li_base64_dec(result, reserve, in, in_length, charset);

    buffer_commit(out, out_pos);

    return (out_pos || !in_length) ? result : NULL;

}

</code>

Here is the source code for functions which reference the function being tested:

<code>

…  


2. Fixing any compilation issues that arise. 

Once a developer has a candidate target, they would try to compile it and look at any compilation issues that arise. Again, we can prompt an LLM with details of the compilation errors so it can provide fixes.  

Prompt: 

Below is the error to fix:

The code has the following build issues:

<error>

server_fuzzer.cc:12:46: error: only virtual member functions can be marked ‘override’

   12 |   bool OnData(const char* data, size_t size) override {

      |                                              ^~~~~~~~

server_fuzzer.cc:22:39: error: no type named ‘MultiPartFile’ in namespace ‘httplib’

   22 |                        const httplib::MultiPartFile& file) override {

      |                              ~~~~~~~~~^

server_fuzzer.cc:12:27: warning: unused parameter ‘data’ [-Wunused-parameter]

   12 |   bool OnData(const char* data, size_t size) override {

      |                           ^

server_fuzzer.cc:37:5: error: use of undeclared ‘operator()’

   37 |     operator()(request_data.data(), receiver);

      |     ^

server_fuzzer.cc:27:23: warning: private field ‘provider_’ is not used [-Wunused-private-field]

   27 |   FuzzedDataProvider& provider_;

      |                       ^

</error>

Example of compilation errors that an LLM was able to fix


3. Running the fuzz target to see how it performs, and fixing any obvious mistakes causing runtime issues.

Once all compilation errors are fixed, a developer would try running the fuzz target for a short period of time to see if there were any mistakes that led it to instantly crash, suggesting an error with the target rather than a bug discovered in the project.


The following is an example of an LLM fixing a semantic issue with the fuzzing setup: 

LLM Response:

<explanation>

The issue is that `sqlite3_blob_open` requires the table to exist before you can open a blob on it. The fix ensures that the table exists by executing a `CREATE TABLE IF NOT EXISTS` statement before calling `sqlite3_blob_open`. 

Here’s a breakdown of the changes:

1. **Create Table Query:** A string `createTableQuery` is constructed to hold the SQL statement for creating the table. It uses the fuzzed values for the table and column names.

2. **Prepare Statement:** `sqlite3_prepare_v2` is used to prepare the SQL statement from `createTableQuery`. This compiles the SQL into a byte-code program that SQLite can execute.

3. **Execute Statement:** `sqlite3_step` executes the prepared statement. Since it’s a `CREATE TABLE` statement, it doesn’t return any data, but it creates the table if it doesn’t exist.

4. **Finalize Statement:** `sqlite3_finalize` destroys the prepared statement object when it’s no longer needed.

By adding these steps before calling `sqlite3_blob_open`, we ensure that the table exists, preventing the null pointer access issue. 

</explanation>

4. Running the corrected fuzz target for a longer period of time, and triaging any crashes.

At this point, the fuzz target is ready to run for an extended period of time on a suitable fuzzing infrastructure, such as ClusterFuzz. 


Any discovered crashes would then need to be triaged, to determine the root causes and whether they represented legitimate vulnerabilities (or bugs in the fuzz target). An LLM can be prompted with the relevant context (stacktraces, fuzz target source code, relevant project source code) to perform this triage. 


Prompt: Given the following crash report, fuzz driver code and relevant project function code, analyze the cause of the crash.

First, only answer “Crash is caused by bug in fuzz driver” or “Crash is caused by bug in project”. Second, offer a succinct and to-the-point analysis.

Below is crash report:

<log>

AddressSanitizer: heap-use-after-free on address 0x502000000630 at pc 0x560036939242 bp 0x7ffd65dca850 sp 0x7ffd65dca010

READ of size 1 at 0x502000000630 thread T0

SCARINESS: 40 (1-byte-read-heap-use-after-free)

    #0 0x560036939241 in __asan_memcpy /src/llvm-project/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:63:3

    #1 0x56003697d30b in EmitLiteral<false> /src/snappy/snappy.cc:661:5

    #2 0x56003697d30b in snappy::internal::CompressFragment(char const*, unsigned long, char*, unsigned short*, int) /src/snappy/snappy.cc:954:10

In this example, the LLM correctly determines this is a bug in the fuzz target, rather than a bug in the project being fuzzed. 


LLM Response: Crash is caused by bug in fuzz driver.

The fuzz driver creates a vector of `iovec` structures, each pointing to a chunk of memory owned by the `FuzzedDataProvider`. However, the `data_chunk` vector, which holds the actual data pointed to by `iov_base`, is deallocated at the end of each fuzzer iteration when it goes out of scope. This leaves the `iov_base` pointers dangling. When `snappy::CompressFromIOVec` tries to access the memory pointed to by these dangling pointers, a heap-use-after-free occurs. 

5. Fixing vulnerabilities. 

The goal is to fully automate this entire workflow by having the LLM generate a suggested patch for the vulnerability. We don’t have anything we can share here today, but we’re collaborating with various researchers to make this a reality and look forward to sharing results soon. 


Up next

Improving automated triaging: to get to a point where we’re confident about not requiring human review. This will help automatically report new vulnerabilities to project maintainers. There are likely more than the 26 vulnerabilities we’ve already reported upstream hiding in our results.


Agent-based architecture: which means letting the LLM autonomously plan out the steps to solve a particular problem by providing it with access to tools that enable it to get more information, as well as to check and validate results. By providing LLM with interactive access to real tools such as debuggers, we’ve found that the LLM is more likely to arrive at a correct result.


Integrating our research into OSS-Fuzz as a feature: to achieve a more fully automated end-to-end solution for vulnerability discovery and patching. We hope OSS-Fuzz will be useful for other researchers to evaluate AI-powered vulnerability discovery ideas and ultimately become a tool that will enable defenders to find more vulnerabilities before they get exploited. 


For more information, check out our open source framework at oss-fuzz-gen. We’re hoping to continue to collaborate on this area with other researchers. Also, be sure to check out the OSS-Fuzz blog for more technical updates.

/by

Retrofitting spatial safety to hundreds of millions of lines of C++

Posted by Alex Rebert and Max Shavrick, Security Foundations, and Kinuko Yasuda, Core Developer

Attackers regularly exploit spatial memory safety vulnerabilities, which occur when code accesses a memory allocation outside of its intended bounds, to compromise systems and sensitive data. These vulnerabilities represent a major security risk to users. 

Based on an analysis of in-the-wild exploits tracked by Google’s Project Zero, spatial safety vulnerabilities represent 40% of in-the-wild memory safety exploits over the past decade:

Breakdown of memory safety CVEs exploited in the wild by vulnerability class.1

Google is taking a comprehensive approach to memory safety. A key element of our strategy focuses on Safe Coding and using memory-safe languages in new code. This leads to an exponential decline in memory safety vulnerabilities and quickly improves the overall security posture of a codebase, as demonstrated by our post about Android’s journey to memory safety.

However, this transition will take multiple years as we adapt our development practices and infrastructure. Ensuring the safety of our billions of users therefore requires us to go further: we’re also retrofitting secure-by-design principles to our existing C++ codebase wherever possible.

To that end, we’re working towards bringing spatial memory safety into as many of our C++ codebases as possible, including Chrome and the monolithic codebase powering our services.

We’ve begun by enabling hardened libc++, which adds bounds checking to standard C++ data structures, eliminating a significant class of spatial safety bugs. While C++ will not become fully memory-safe, these improvements reduce risk as discussed in more detail in our perspective on memory safety, leading to more reliable and secure software.

This post explains how we’re retrofitting hardened libc++ across our codebases and  showcases the positive impact it’s already having, including preventing exploits, reducing crashes, and improving code correctness.

Bounds-checked data structures: The foundation for spatial safety

One of our primary strategies for improving spatial safety in C++ is to implement bounds checking for common data structures, starting with hardening the C++ standard library (in our case, LLVM’s libc++). Hardened libc++, recently added by open source contributors, introduces a set of security checks designed to catch vulnerabilities such as out-of-bounds accesses in production.

For example, hardened libc++ ensures that every access to an element of a std::vector stays within its allocated bounds, preventing attempts to read or write beyond the valid memory region. Similarly, hardened libc++ checks that a std::optional isn’t empty before allowing access, preventing access to uninitialized memory.

This approach mirrors what’s already standard practice in many modern programming languages like Java, Python, Go, and Rust. They all incorporate bounds checking by default, recognizing its crucial role in preventing memory errors. C++ has been a notable exception, but efforts like hardened libc++ aim to close this gap in our infrastructure. It’s also worth noting that similar hardening is available in other C++ standard libraries, such as libstdc++.

Raising the security baseline across the board

Building on the successful deployment of hardened libc++ in Chrome in 2022, we’ve now made it default across our server-side production systems. This improves spatial memory safety across our services, including key performance-critical components of products like Search, Gmail, Drive, YouTube, and Maps. While a very small number of components remain opted out, we’re actively working to reduce this and raise the bar for security across the board, even in applications with lower exploitation risk.

The performance impact of these changes was surprisingly low, despite Google’s modern C++ codebase making heavy use of libc++. Hardening libc++ resulted in an average 0.30% performance impact across our services (yes, only a third of a percent).

This is due to both the compiler’s ability to eliminate redundant checks during optimization, and the efficient design of hardened libc++. While a handful of performance-critical code paths still require targeted use of explicitly unsafe accesses, these instances are carefully reviewed for safety. Techniques like profile-guided optimizations further improved performance, but even without those advanced techniques, the overhead of bounds checking remains minimal.

We actively monitor the performance impact of these checks and work to minimize any unnecessary overhead. For instance, we identified and fixed an unnecessary check, which led to a 15% reduction in overhead (reduced from 0.35% to 0.3%), and contributed the fix back to the LLVM project to share the benefits with the broader C++ community.

While hardened libc++’s overhead is minimal for individual applications in most cases, deploying it at Google’s scale required a substantial commitment of computing resources. This investment underscores our dedication to enhancing the safety and security of our products.

From tests to production

Enabling libc++ hardening wasn’t a simple flip of a switch. Rather, it required a multi-stage rollout to avoid accidentally disrupting users or creating an outage:

  1. Testing: We first enabled hardened libc++ in our tests over a year ago. This allowed us to identify and fix hundreds of previously undetected bugs in our code and tests.
  2. Baking: We let the hardened runtime “bake” in our testing and pre-production environments, giving developers time to adapt and address any new issues that surfaced. We also conducted extensive performance evaluations, ensuring minimal impact to our users’ experience.
  3. Gradual Production Rollout: We then rolled out hardened libc++ to production over several months, starting with a small set of services and gradually expanding to our entire infrastructure. We closely monitored the rollout, promptly addressing any crashes or performance regressions.

Quantifiable impact

In just a few months since enabling hardened libc++ by default, we’ve already seen benefits.

Preventing exploits: Hardened libc++ has already disrupted an internal red team exercise and would have prevented another one that happened before we enabled hardening, demonstrating its effectiveness in thwarting exploits. The safety checks have uncovered over 1,000 bugs, and would prevent 1,000 to 2,000 new bugs yearly at our current rate of C++ development.

Improved reliability and correctness: The process of identifying and fixing bugs uncovered by hardened libc++ led to a 30% reduction in our baseline segmentation fault rate across production, indicating improved code reliability and quality. Beyond crashes, the checks also caught errors that would have otherwise manifested as unpredictable behavior or data corruption.

Moving average of segfaults across our fleet over time, before and after enablement.

Easier debugging: Hardened libc++ enabled us to identify and fix multiple bugs that had been lurking in our code for more than a decade. The checks transform many difficult-to-diagnose memory corruptions into immediate and easily debuggable errors, saving developers valuable time and effort.

Bridging the gap with memory-safe languages

While libc++ hardening provides immediate benefits by adding bounds checking to standard data structures, it’s only one piece of the puzzle when it comes to spatial safety.

We’re expanding bounds checking to other libraries and working to migrate our code to Safe Buffers, requiring all accesses to be bounds checked. For spatial safety, both hardened data structures, including their iterators, and Safe Buffers are necessary.

Beyond improving the safety of our C++, we’re also focused on making it easier to interoperate with memory-safe languages. Migrating our C++ to Safe Buffers shrinks the gap between the languages, which simplifies interoperability and potentially even an eventual automated translation.

Building a safer C++ ecosystem

Hardened libc++ is a practical and effective way to enhance the safety, reliability, and debuggability of C++ code with minimal overhead. Given this, we strongly encourage organizations using C++ to enable their standard library’s hardened mode universally by default.

At Google, enabling hardened libc++ is only the first step in our journey towards a spatially safe C++ codebase. By expanding bounds checking, migrating to Safe Buffers, and actively collaborating with the broader C++ community, we aim to create a future where spatial safety is the norm.

Acknowledgements

We’d like to thank Emilia Kasper, Chandler Carruth, Duygu Isler, Matthew Riley, and Jeff Vander Stoep for their helpful feedback. We also extend our thanks to the libc++ community for developing the hardening mode that made this work possible.

  1. Based on manual analysis of CVEs from July 15, 2014 to Dec 14, 2023. Note that we could not classify 11% of CVEs.. ↩

/by
,

Safer with Google: New intelligent, real-time protections on Android to keep you safe

Posted by Lyubov Farafonova, Product Manager and Steve Kafka, Group Product Manager, Android

User safety is at the heart of everything we do at Google. Our mission to make technology helpful for everyone means building features that protect you while keeping your privacy top of mind. From Gmail’s defenses that stop more than 99.9% of spam, phishing and malware, to Google Messages’ advanced security that protects users from 2 billion suspicious messages a month and beyond, we’re constantly developing and expanding protection features that help keep you safe.

We’re introducing two new real-time protection features that enhance your safety, all while safeguarding your privacy: Scam Detection in Phone by Google to protect you from scams and fraud, and Google Play Protect live threat detection with real-time alerts to protect you from malware and dangerous apps.

These new security features are available first on Pixel, and are coming soon to more Android devices.

More intelligent AI-powered protection against scams

Scammers steal over $1 trillion dollars a year from people, and phone calls are their favorite way to do it. Even more alarming, scam calls are evolving, becoming increasingly more sophisticated, damaging and harder to identify. That’s why we’re using the best of Google AI to identify and stop scams before they can do harm with Scam Detection.

Real-time protection, built with your privacy in mind.

  • Real-time defense, right on your device: Scam Detection uses powerful on-device AI to notify you of a potential scam call happening in real-time by detecting conversation patterns commonly associated with scams. For example, if a caller claims to be from your bank and asks you to urgently transfer funds due to an alleged account breach, Scam Detection will process the call to determine whether the call is likely spam and, if so, can provide an audio and haptic alert and visual warning that the call may be a scam.
  • Private by design, you’re always in control: We’ve built Scam Detection to protect your privacy and ensure you’re always in control of your data. Scam Detection is off by default, and you can decide whether you want to activate it for future calls. At any time, you can turn it off for all calls in the Phone app Settings, or during a particular call. The AI detection model and processing are fully on-device, which means that no conversation audio or transcription is stored on the device, sent to Google servers or anywhere else, or retrievable after the call.
  • Cutting-edge AI protection, now on more Pixel phones: Gemini Nano, our advanced on-device AI model, powers Scam Detection on Pixel 9 series devices. As part of our commitment to bring powerful AI features to even more devices, this AI-powered protection is available to Pixel 6+ users thanks to other robust Google on-device machine learning models.

We’re now rolling out Scam Detection to English-speaking Phone by Google public beta users in the U.S. with a Pixel 6 or newer device.

To provide feedback on your experience, please click on Phone by Google App -> Menu -> Help & Feedback -> Send Feedback. We look forward to learning from this beta and your feedback, and we’ll share more about Scam Detection in the months ahead.

More real-time alerts to protect you from bad apps

Google Play Protect works non-stop to protect you in real-time from malware and unsafe apps. Play Protect analyzes behavioral signals related to the use of sensitive permissions and interactions with other apps and services.

With live threat detection, if a harmful app is found, you’ll now receive a real-time alert, allowing you to take immediate action to protect your device. By looking at actual activity patterns of apps, live threat detection can now find malicious apps that try extra hard to hide their behavior or lie dormant for a time before engaging in suspicious activity.

At launch, live threat detection will focus on stalkerware, code that may collect personal or sensitive data for monitoring purposes without user consent, and we will explore expanding its detection to other types of harmful apps in the future. All of this protection happens on your device in a privacy preserving way through Private Compute Core, which allows us to protect users without collecting data.

Live threat detection with real-time alerts in Google Play Protect are now available on Pixel 6+ devices and will be coming to additional phone makers in the coming months.

/by