Ghost in the Pipelines

We recently had the opportunity to work with a client that had a technology stack that differs from the usual enterprise environment that uses Active Directory, Entra ID and a mix of on-premise and cloud infrastructure. Being an organisation whose staff are mostly developers, all of their infrastructure was cloud hosted, serverless and Infrastructure-as-Code (IaC). The enterprise applications used by the organisation were also predominately Software-as-a-Service (SaaS) offerings. Additionally, the organisation had significantly less staff members than most that we normally target, reducing the attack surface of employees. As a result, the usual tricks were not going to fly. No files with domain administrator credentials sitting in the corporate network share or password spraying with Company2025!.

Despite not having extensive experience attacking the stack of technology, the high-level methodology or tactics for an attack do not change. You just need to establish what techniques (publicly reported and internal research) are available for the target technology. Instead of looking for credentials in network file shares you might be looking in Confluence. Rather than configuring scheduled tasks for persistence, you might be generating API keys for Software-as-a-Service (SaaS) applications.

Fortunately, both the Offensive and Defensive security communities are excellent at information sharing, publishing public research and threat intelligence. In most cases it's a matter of spending some time reading research related to target systems or software and supplementing the knowledge gained with trial and error. Finally, you sprinkle in reading of vendor documentation and API schemas. If you are interested in a more in-depth methodology for researching and developing techniques for unfamiliar technology, I highly recommend Brett Hawkin's Hiding in the Clouds presentation (https://www.youtube.com/watch?v=IYyfRu8Y0f4).

Introduction

The following blog outlines an attack on a fictional target (MegaCorp) which involves:

• Phishing developers for initial access
• Performing reconnaissance using APIs
• Compromising components of a CI/CD pipeline
• Post exploitation and execution on a runner
• Privilege escalation to a role with AWS Administrator access

The attack path is inspired by some of the techniques performed on the engagement described above but is also fleshed out with techniques we came across while performing research. Do not let the truth get in the way of a good yarn.

Initial Access

It was assumed that MegaCorp staff were likely to have higher than average security awareness as most staff were developers. As a result, they may have been less susceptible to common phishing attacks. It had also been noted during initial reconnaissance that MegaCorp developers were using GitHub as their source code management (SCM) solution. Reviewing their publicly available repositories it was discovered they were making use of GitHub Actions, GitHub's CI/CD platform.

As GitHub supports the OAuth 2.0 Device Authorization Flow (DAF from here on out), its services are susceptible to a lesser-known phishing technique, Device Code phishing. This technique was more suitable to target technically savvy developers with.

While Device Code phishing has been used for years by red teams, public disclosure of its use by threat actors was only recently (as far as we are aware) published by Microsoft. In their threat intelligence they describe Storm-2372's use of Device Code phishing targeting Entra ID:

Storm-2372 conducts device code phishing campaign | Microsoft Security Blog

Microsoft Threat Intelligence Center discovered an active and successful device code phishing campaign by a threat actor we track as Storm-2372. Our ongoing investigation indicates that this campaign has been active since August 2024 with the actor creating lures that resemble messaging app experiences including WhatsApp, Signal, and Microsoft Teams. Storm-2372’s targets during this time have included government, non-governmental organizations (NGOs), information technology (IT) services and technology, defense, telecommunications, health, higher education, and energy/oil and gas in Europe, North America, Africa, and the Middle East. Microsoft assesses with medium confidence that Storm-2372 aligns with Russian interests, victimology, and tradecraft.

Microsoft Security BlogMicrosoft Threat Intelligence

Device Code phishing abuses the DAF (https://auth0.com/docs/get-started/authentication-and-authorization-flow/device-authorization-flow) which is typically used by applications when it is difficult to authenticate with a username, password and second factor of authentication (2FA). Rather than prompt the user for credentials, the application will request a device code, user code and verification URL from the "/oauth/device/code" endpoint of an Identity Provider (IdP). The application will then provide the user with instructions to enter their user code to the page hosted at the verification URL. If the user already has a session for the IdP, they are not prompted for credentials and simply enter the user code and click a prompt to authorise the code. In the background the application will periodically poll the "/oauth/token" endpoint for the life time (15 minutes) of the device code to determine whether the user has authorised it. If the user has authorised the device code, the response from the "/oauth/token" endpoint will include tokens for the application to use, which have a much larger life time.

Device code phishing typically comes in two forms:

• A pretext with an already generated user code and instructions to enter the code at the verification URL

or

• A pretext to a landing page which generates the user code when accessed and provides instructions to enter the code at the verification URL

The latter option is more reliable, as it is not dependent on the user authorising the code within 15 minutes of pretext delivery. The 15 minute time starts when they access the landing page.

Device code phishing was selected as the ideal approach for targeting MegaCorp developers. As a public tool was not identified which could be used to perform GitHub device code phishing, a Python Flask application was developed. The application exposed an API which would initiate the DFA. A background thread would periodically poll the GitHub IdP for tokens. A standalone HTML file was also created to be used as the landing page, with JavaScript embedded to make a request to the Flask application.

Rather than deliver the phishing pretext over email which is heavily scrutinised (security controls and user awareness), MegaCorp developers were targeted with SMS to their mobile phones.

The landing page URL in the SMS was obfuscated with a link shortening service, which are commonly used in communications over SMS.

Upon accessing the landing page, JavaScript would poll the Flask application to have a device code generated and then update the page. The page also included instructions redirecting the user to enter the code in a web browser. If the user navigated to the legitimate GitHub IdP, they were prompted for their code.

After entering their code, GitHub provides a warning prompt disclosing the application executing the DFA and the scope requested for the token.

If the user authorised the device code within its 15-minute life time, the attack was successful, and the Flask application will have received a token from GitHub's IdP.

The valid access token provided initial access to GitHub in the context of the target user.

Reconnaissance

Reconnaissance could then be performed by using the token to make authenticated requests to GitHub's API and pulling repositories using the Git client. While ad hoc PowerShell commands were used for this example attack path, Brett Hawkin's SCMKit (https://github.com/h4wkst3r/SCMKit) for attacking Source Code Management systems would have been a good option as well.

Initially a request was made to the "https://api.github.com/user" endpoint to determine the details of the compromised user. This is the GitHub API equivalent of the "whoami" command.

The next step was to start manually mapping out the resources the compromised user had access to and gaining additional context about the organisation. Requests were made to GitHub's API to determine:

• The organisations the user was associated with (https://api.github.com/user/orgs)
• The teams the user had been assigned membership to (https://api.github.com/user/teams)
• The repositories the user had access to (https://api.github.com/user/repos)

After mapping out the resources the user had access to, each repository in the target organisation was cloned for analysis. Similar to their public repositories, MegaCorp had also configured GitHub actions for their private repositories.

The "Test" workflow for the "hello-world" repository was configured to use the GitHub hosted runners and did not make use of any secrets. The workflow had also been configured with the permissions "id-token: write" and "contents: read". The "id-token: write" delegates the runner with the privileges to retrieve an Open ID Connect (OIDC) token from GitHub's OIDC service. By reviewing the actions in the workflow it was determined that this permission was not required for it to function correctly and may be an artifact of an older version of the workflow. The "contents: read" permission provides the "GITHUB_TOKEN" secret injected into the runner, with permissions to access the repository with read permissions. This is required for the "actions/checkout" action within the workflow.

The "Dev Environment Deploy" workflow for the "log-execution" repository had the same permissions configured as the "hello-world" repository's workflow. Despite not making use of secrets, the workflow was making use of the "aws-actions/configure-aws-credentials" action to assume the "github-actions-devops-dev" role and then subsequently interact with Amazon Web Services (AWS) Elastic Container Service (ECS). If it was possible to inject code into the repository or manipulate the workflow, it would be possible to compromise the "github-actions-devops-dev" role and the resources it had access to.

Unfortunately, the compromised developer only had read and pull rights for the repository. Additionally, the workflow did not seem susceptible to injection attacks. No steps within the pipeline were executing operating systems commands.

While researching how the "aws-actions/configure-aws-credentials" action authenticates to AWS, we came across an excellent blog (https://securitylabs.datadoghq.com/articles/exploring-github-to-aws-keyless-authentication-flaws/) from DataDog. The blog described how the action authenticates to AWS by requesting a JSON Web Token (JWT) from GitHub's OIDC service using a URL and token injected into the environment variables of the GitHub runner. As the subject in the resulting JWT is set to the repository from which the workflow is running from, it is possible to authenticate to OIDC providers which have been configured to trust GitHub's OIDC service. The subject in the JWT is set in the following format:

repo:[organisation]/[repository]:[branch]

The researchers at DataDog had also determined some organisations had seriously misconfigured the trusts in their AWS OIDC provider to trust the repositories for any organisation on GitHub by not having a conditional check for the JWT subject. As a result, if the role Amazon Resource Name (ARN) was known to an attacker and the role was also configured to allow identities from GitHub's OIDC service to assume the role, the attacker could assume the role from within any GitHub runner, including their own.

At the time of writing, the AWS console required an organisation be provided when configuring trusts, preventing the previously described misconfiguration (in the web console, using the "wizard"). However, the repository and branch fields are optional. When configured without a specific repository or branch, workflows for any repository in the organisation can assume the role that the trust is configured for. This configuration does not have the same impact as the misconfiguration highlighted in the DataDog blog, but it does provide a potential avenue to compromise AWS roles from unprivileged GitHub repositories within the organisation.

Execution and Privilege Escalation

As the workflow for the "hello-world" repository had been configured with the permission "id-token: write", execution in the pipeline could be abused to retrieve a JWT from GitHub's OIDC service. To achieve execution on the GitHub runner, code was added to the "test.js" file, which was being executed by a step in the "Test" workflow. This code would post the environment variables to a C2 server. A more stealthy option may have involved including a reference to a dependency with the payload code.

To deal with the short life span of the token (valid while the workflow is executing) used to authenticate to GitHub's OIDC service and the inconsistent timing of executions on GitHub hosted runners, a Flask application with a single API was created. Upon receiving a POST request containing a runner's environment variables, the application would request a JWT from GitHub's OIDC service and if successful, authenticate to AWS's STS service to assume a role and gain access to an "AccessKeyId", "SecretAccessKey" and "SessionToken".

While the workflow within the "log-execution" repository specifically referenced the "github-actions-devops-dev" role, it was safe to assume a "github-actions-devops-prod" or "github-actions-devops-prd" might exist. This assumption was found to be correct. The role was also found to have the "AdministratorAccess" policy attached, allowing significant access to the AWS tenancy and compromise of the target systems for the engagement.

Defensive Takeaways

• Ensure workflow files for GitHub have the bare minimum permissions configured.
• Ensure that the concept of least privilege is applied to trusted identities from GitHub's OIDC service. For example, configuring granular conditional checks for the GitHub organisation and also, the repository.
• Ensure the concept of least privilege is applied with access controls for repositories. Developers with write access to a repository have the capability to compromise any credentials used by the workflow. Ensure developers do not have access (including read) to repositories unless they explicitly require access.
• For more sensitive repositories or those that use privileged identities in their workflow, consider configuring branch protections and commit signing.
• If your SCM generates appropriate logs, implement monitoring for suspicious behavior such as the cloning of a significant amount of repositories in quick succession.
• Monitor for signs of compromised AWS identities (especially non-human identities) including suspicious use of "assume-role-with-web-identity", "get-caller-identity", "list-attached-role-policies" and other STS and IAM calls used for enumerating access.
• Monitor for suspicious network connections from GitHub runners.
• Ensure end users receive regular cyber security awareness training.
• Hard code honey pot credentials into repositories or workflow files and monitor for attempts to authenticate with them.

More in-depth information on securing CI/CD environments can be found in the United States Cybersecurity and Infrastructure Security Agency's publication, "Defending Continuous Integration/Continuous Delivery (CI/CD) Environments": https://media.defense.gov/2023/Jun/28/2003249466/-1/-1/0/CSI_DEFENDING_CI_CD_ENVIRONMENTS.PDF

Palantir have also published a 3 part blog series detailing how they have approached securing their software development environment:

https://blog.palantir.com/tagged/software-supply-chain

Tactics, Techniques and Procedures

Where possible, key actions performed during the engagement have been aligned with the MITRE ATT&CK framework.