The false positive AWS found in my own engine

Three terms first

An attack path is a chain of steps from something an attacker can reach, like a public load balancer, to something worth stealing, such as a customer database or an administrator role. PerspectiveGraph builds a graph of a cloud environment and searches it for those chains.

Privilege escalation is a step where an identity gives itself more power than it was meant to have. A classic AWS example: a role allowed to attach policies to itself can attach AdministratorAccess and become administrator.

A permissions boundary is an AWS feature that sets the most a user's or role's own policies can let it do. Think of a company card with a spending limit. The card's rules say what you may buy; the limit caps it. A higher limit does not buy you anything on its own, and a purchase the limit does not cover is refused even if the rules allow it.

Where the engine went wrong

The engine generated an escalation step, CAN_ESCALATE_TO, when an identity had all the permissions required for a known escalation technique. The techniques come from published research by Rhino Security Labs - such as iam:AttachUserPolicy, iam:PutRolePolicy or iam:PassRole combined with any action that executes code. The escalation has a probability of 0.9 when the permission applies to the entire account and 0.5 when it is limited to specific resources.

To determine what an identity can do, the engine reads AWS's GetAccountAuthorizationDetails: inline policies, attached policies, permissions inherited from groups and explicit denials that apply at the account level. This response also contains the permissions boundary. The code that parsed it simply didn't have a boundary field, so the information arrived and was discarded.

This is the whole bug. A code review probably wouldn't have caught it: every line on the screen was correct and the error consisted of a nonexistent line.

How I found out: I asked AWS

A few days earlier I had created a way to test the engine against an authority other than the engine itself: iam:SimulatePrincipalPolicy, AWS's own policy simulator. This simulation answers "would this call be allowed?" without performing any action and it is free. According to AWS's documentation, it evaluates identity policies, permissions boundaries, service control policies when the account belongs to an organization, and resource policies provided to it. Conditions are evaluated with the specified context values. It does not cover resource control policies and AWS notes that results can differ from reality in some advanced configurations.

I'm only using it for one question. It can tell me if an escalation step is allowed. It can't tell me whether an attacker would get code execution on an exposed server, which is the first step of most real-world attack paths. If I asked it about whole paths, the answer might just be "I can't confirm", and considering it proof would make every score look too high. I'd be measuring the simulator's blind spot, not my engine.

One mistake I ran into while building it: never ask the simulator about iam:*. AWS interprets that as "can this identity perform all IAM actions?", which is almost always no, so a role that can actually elevate privileges is rejected. Instead the check asks for information about the specific actions required for each escalation technique, taking them from the same table the engine uses, so that the two cannot be inconsistent.

The experiment: two roles, one difference

Two roles in a real AWS account, with the same inline policy, byte for byte:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["iam:AttachUserPolicy", "iam:PutUserPolicy", "iam:CreateAccessKey"],
    "Resource": "*"
  }]
}

One of the two also has a permissions boundary:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:Get*", "ec2:Describe*"],
    "Resource": "*"
  }]
}

Look at what the boundary doesn't contain. There's no Deny, and no mention of IAM at all. This absence is a crucial point: anything the boundary does not allow, the role cannot do, whatever its own policy says. (Resource-based policies are the exception documented by AWS: a policy that directly names a user or role session isn't restricted by the boundary.)

The role without a boundary is the control, and it matters. Without it, an engine that simply stopped reporting escalations would pass the test and appear to be working.

The result

The comparison runs the engine on the account and queries AWS about the same roles, side by side:

  principal                                  engine       AWS
  --------------------------------------------------------------------------
  role/pg-boundary-lab-unbounded             ESCALATES    ESCALATES    agree
  role/pg-boundary-lab-bounded               ESCALATES    no privesc   DISAGREE

2 principal(s): 1 agree, 1 disagree, 0 unsettled

The analysis engine indicated that both roles could gain administrator privileges. AWS, however, stated that the restricted role could not perform any of the three actions its policy seemed to allow.

This is a dangerous type of false positive. It's not low-severity noise someone can filter out: it's a harmless role classified as a critical path to account administrator with a convincing chain of steps associated with it. Anyone who cared about this would have wasted time on something impossible, and after one or two cases like that people stop looking at a tool's output.

The fix: a boundary limits, never grants

What an identity can actually do is the overlap of two things: what its own policies allow and what its boundary allows. The engine now computes this overlap before looking for escalations. A Deny is not required for a boundary to revoke a permission: if the boundary doesn't mention an action, the overlap doesn't contain it.

The engine needs to account for this overlap in three questions: can this identity do X, can it do X on the entire account and is it actually an admin? The third is easy to forget, and forgetting it causes a role with Allow *:* under a read-only boundary to still appear as an administrator. It has a specific test.

Furthermore, the fix must not go too far in the opposite direction: a boundary never grants anything. A role whose boundary allows iam:* but whose policy doesn't allow anything can't do anything. Get this wrong and the fix creates new false positives.

"Has a boundary" is not "safe"

There's a tempting shortcut: if an identity has a boundary, hide the finding. This is wrong, for two reasons.

First, AdministratorAccess is sometimes used as a permissions boundary, to satisfy a rule that every role must have one, without anyone specifying which one. It doesn't limit anything. Hiding findings whenever a boundary exists would mean missing any escalation behind one of those boundaries.

Second, the costs are not symmetrical. A false alarm costs a professional a few minutes. A missed escalation costs whatever the attacker can achieve. Turning a visible false positive into an invisible false negative is not a solution.

So the rule is: apply the boundary, then ask the question again. If the escalation survives, it remains at full probability, with a note indicating that the boundary has been read and doesn't block it. "Checked and still escalates" and "never checked" are different statements, and a report should not make them appear the same. If the boundary allows the necessary actions only on specific resources, the probability drops to 0.5 like any other escalation limited to specific resources.

When the boundary cannot be read

A user or role names its boundary by its ID (an ARN), but the boundary's text isn't always available. A policy used only as a boundary might be missing from the bundle, and an exported file uploaded might not include it. Therefore, the engine knows a limit exists but can't compute it. There are three options:

• Drop the escalation. Silent and incorrect in the AdministratorAccess case above.
• Report it as certain. That is the original bug again.
• Report it and specify what is unknown.

The engine does the third. The step stays, at 0.5 instead of 0.9, and it explains why:

{
  "type": "CAN_ESCALATE_TO",
  "from": "User:3dba4e4d9c6229b3",
  "to": "IAM_Role:ec84c244999b016f",
  "exploit_probability": 0.5,
  "properties": {
    "boundary_note": "a permissions boundary is attached but its policy document was not in the input; the cap could not be computed, so this escalation is reported unverified",
    "permissions_boundary_unresolved": true,
    "primitive_count": 1,
    "primitives": "iam:AttachUserPolicy (attach AdministratorAccess to self)"
  }
}

When running on a real account, the engine avoids the situation where it can: for each boundary it can't see, it makes two read-only calls, GetPolicy and GetPolicyVersion, both covered by the SecurityAudit permissions it already requires. If these fail, it does not stop: an unreadable policy shouldn't hide all the other identities in the account.

How it stays fixed

A fixed bug unmonitored by anyone can come back silently. This bug has two control mechanisms, which fail in different ways.

The first is the experiment turned into a test that runs on every change, with no AWS account. It covers more cases than the live lab: the AdministratorAccess boundary that limits nothing, a boundary that removes admin privileges, a boundary with its own Deny, a boundary that restricts an account-level permission to named resources, and the same for users who obtain their permissions through a group.

$ go test ./internal/ingestion/iam/ -run Boundary -v
--- PASS: TestPermissionsBoundaryCapsEscalation
--- PASS: TestBoundaryDoesNotGrant
--- PASS: TestBoundaryIntersectionSemantics
    --- PASS: .../AdministratorAccess_boundary_is_a_no-op
    --- PASS: .../boundary_revokes_admin
    --- PASS: .../boundary_Deny_beats_the_identity_Allow
    --- PASS: .../boundary_narrows_an_account-wide_grant_to_scoped
--- PASS: TestUnresolvedBoundaryIsReportedUnverified
--- PASS: TestResolvedBoundaryOnASurvivingEscalationIsEvidence
--- PASS: TestUserPermissionsBoundary
ok  github.com/luiacuaniello/perspectivegraph/internal/ingestion/iam

The second test is run on a real account: make boundary-lab-aws creates the two roles, runs the engine and the simulator over them, and fails if there's a mismatch in either direction - a false positive causes it to fail, as does a miss.

  principal                                  engine       AWS
  --------------------------------------------------------------------------
  role/pg-boundary-lab-unbounded             ESCALATES    ESCALATES    agree
  role/pg-boundary-lab-bounded               no privesc   no privesc   agree

2 principal(s): 2 agree, 0 disagree, 0 unsettled

Running it is free. IAM roles and policies are free; each test is a dry run, and nothing is launched or exposed. The script deletes the lab on exit, even if it crashes. The only wait is twelve seconds for IAM to stabilize: a run immediately after creating a role can see the account as it was before the role existed.

What this does not prove

• Two roles prove this case, not the entire IAM model. The lab shows the engine now handles permissions boundaries the same way AWS does. It doesn't say anything about IAM features the engine doesn't understand.
• The engine doesn't yet evaluate service control policies. The simulator does. In an account belonging to an AWS organization, the engine can still report an escalation blocked by an organization-level policy. This will be the next aspect I'll work on in this part of the engine.
• The engine does not evaluate conditions in identity policies. An Allow permission that only applies with MFA is read as if it were always valid, a deliberate over-reporting error that the comparison with AWS helps detect. Rechecking this article, I found that the reader wasn't as one-sided as I'd claimed: a Deny that only applies under certain conditions, such as "without MFA", was read as if it were always valid, and an Allow written with NotAction ("everything except...") wasn't read at all. Both were hiding real escalations. Fixed in 1.28.3: NotAction is read, and an escalation that only a conditional Deny could block is reported at 0.5, flagged as dependent on it. A permission that depends on MFA or a source address is neither allowed nor denied until a real request occurs, so the comparison leaves such cases unresolved.
• The simulator checks permissions, not whether a technique works. It says whether each action is allowed. Rereading this article, I found that the engine credited roles with five techniques only a user can use: attaching or inlining a policy on itself, adding itself to a group, changing its own group's policies. A role is not a user and belongs to no group. AWS's check could not catch it, because the permissions really are allowed, and my comparison agreed with the engine. The lab above was affected too: two of the three permissions I gave its roles are user techniques. The control still escalates through the third, iam:CreateAccessKey, which works for a role too - it mints keys for an administrator user, and the lab account has one - so the result stands, for a narrower reason than the engine printed. Fixed in 1.28.1, in the engine and in the comparison. An outside check only covers the question you ask it.
• The simulator tests escalation steps, not complete attack paths. The first step in most paths - running code on an exposed machine - is still the model's estimate, and the engine says so.

What I take from it

A tool that reads only part of a permission model will overestimate the risk in the part it doesn't read, and it will do so confidently, because nothing in its input indicates anything is missing.

A tool can't verify itself. Everything the engine knew about that role came from the same code that had the gap, so every internal check agreed with it. The disagreement had to come from outside - and for IAM, a reliable external opinion is one free API call away.

When the desired measurement is out of reach, it's better to correctly measure a more specific aspect than to inadequately measure a broader aspect. I'd like to report how often the engine's attack paths are actually exploitable; no simulation can tell me that. However, it can tell me how often the escalation steps are real, so that's what I report, and I explain its scope.

Try it

PerspectiveGraph is open source under Apache 2.0. The same comparison can be performed on your AWS account with a single command: perspectivegraph redteam -roles -compare -region <region>. If its result isn't correct, this is the most useful information you could provide, and there is a form for it.

• privesc.go - the escalation table and how the boundary is applied
• boundary_test.go - the experiment as a test, with the control
• boundary-lab-aws.sh - the same experiment on a real account, failing on any disagreement
• AWS: permissions boundaries and the IAM policy simulator
• The public demo, read-only