Skip to content

Require evidence before reporting red-team security findings #67542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pelikhan merged 2 commits into from
Oct 11, 2026
Merged

Require evidence before reporting red-team security findings #67542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2916141
Initial plan
Copilot Oct 11, 2026
54ad2b0
Validate security scan findings before reporting
Copilot Oct 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/daily-security-red-team.lock.yml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 15 additions & 5 deletions .github/workflows/daily-security-red-team.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,8 +315,8 @@ while IFS= read -r file; do

# Pattern 1: Secret exfiltration
if grep -nE '(process\.env\.|os\.getenv|ENV\[)[^;]*\.(post|fetch|axios|request|curl|wget)' "$file" > /tmp/gh-aw/agent/pattern.txt; then
echo "⚠️ Potential secret exfiltration in $file"
FINDINGS+=("SECRET_EXFIL:$file:$(head -1 /tmp/gh-aw/agent/pattern.txt | cut -d: -f1)")
echo "⚠️ Potential secret egress candidate in $file"
FINDINGS+=("SECRET_EXFIL_CANDIDATE:$file:$(head -1 /tmp/gh-aw/agent/pattern.txt | cut -d: -f1)")
fi

# Pattern 2: Eval/exec with user input
Expand Down Expand Up @@ -430,8 +430,8 @@ while IFS= read -r file; do

# High entropy might indicate obfuscation
if [ "$ENTROPY" -gt 70 ]; then
echo "⚠️ High entropy detected in $file (entropy: $ENTROPY)"
FINDINGS+=("HIGH_ENTROPY:$file:0:entropy=$ENTROPY")
echo "⚠️ High entropy candidate detected in $file (entropy: $ENTROPY)"
FINDINGS+=("HIGH_ENTROPY_CANDIDATE:$file:0:entropy=$ENTROPY")
fi

# Check for long hex/base64 strings
Expand Down Expand Up @@ -587,6 +587,16 @@ Before proceeding to Phase 5, perform a **second-pass validation** of every find
<!-- single_pass: skip re-evaluation, proceed directly to Phase 5 -->
{{/if}}

#### Phase 4c: Evidence Validation (all scan variants)

Before Phase 5, validate every candidate in `FINDINGS[]`, including in the `single_pass` variant. The iterative second pass is additional analysis, not a substitute for these checks:

1. Verify that each cited path exists and that its line number is in range and points to the behavior being reported. Do not treat imports, comments, type declarations, or test fixtures alone as evidence of a production vulnerability.
2. Treat regex matches, suspicious keywords, entropy scores, and long encoded strings as leads only. A heuristic match does not establish malicious behavior.
3. For any `SECRET_EXFIL_CANDIDATE`, identify the sensitive value source and the outbound network sink, then trace the same value from source to sink. Cite the exact source and sink lines and explain the data flow. An environment-variable read, a network call, or high entropy on its own is insufficient. Promote the candidate to `SECRET_EXFIL` only when this path is verified; otherwise remove it from `FINDINGS[]` and record the reason in `$CACHE_DIR/dismissed-findings-${TIMESTAMP}.json`.
4. Apply the same evidence standard to other finding types: retain only findings supported by the cited code and its behavior. If a candidate cannot be verified, do not report it as a confirmed issue.
5. Ensure all remaining entries in `FINDINGS[]` are confirmed, correctly cited findings before running forensics or generating remediation tasks. If none remain, use the no-findings path in Phase 8.

#### Phase 5: Compile and Report Findings

```bash
Expand Down Expand Up @@ -658,7 +668,7 @@ if [ ${#FINDINGS[@]} -gt 0 ]; then

### 📋 Executive Summary

The daily security red team scan has detected **${#FINDINGS[@]}** potential security issues in the \`actions/setup/js\` and \`actions/setup/sh\` directories using the **$TECHNIQUE** technique.
The daily security red team scan has detected **${#FINDINGS[@]}** validated security findings in the \`actions/setup/js\` and \`actions/setup/sh\` directories using the **$TECHNIQUE** technique.

<details>
<summary><b>🔍 View Detailed Findings</b></summary>
Expand Down
Loading