Open Source Release Todo

Zagros Open-Source Release TODO

Goal: prepare Zagros for public open-source release as a self-hosted, shared security reference that teams can connect to their AI agents and internal tools.

Project scope:

• Zagros Core — ingestion, HelixDB, retrieval/RAG, CLI, MCP servers, UI.
• Zagros Skills — agent guidance for using Zagros correctly during security investigations.

P0 — Release blockers

• Resolve skill copyright/provenance before publishing.

  • Skill content is independently authored from the maintainer’s personal cybersecurity study notes and practical experience.
  • ISC2 CC influenced topic coverage, but no official ISC2 courseware or exam content is included.
  • Added explicit non-affiliation/provenance wording and third-party notices.

• Reconcile repository licensing.

  • Zagros-authored Core, documentation, and Skills use Apache-2.0.
  • Standalone skill packages include the Apache-2.0 license.
  • Third-party security datasets retain their upstream terms; see THIRD_PARTY_NOTICES.md.

• Update stale documentation.

  • VISION.md now reflects the local-first trusted-source mission, the Core + Skills model, and all seven current MCP tools.
  • Website copies of README, VISION, and SECURITY-REVIEW were synchronized.

• Refresh the security review against current code.

  • F-01, F-02, F-04, F-05, and F-06 are fixed.
  • F-03 is mitigated/documented as a HelixDB v3 limitation.
  • No unresolved High-severity finding remains from the previous review.

P1 — Trusted-source guarantees

• Store publisher/source, source version, canonical URL, retrieval URL, license, retrieved timestamp, upstream timestamp, content hash, and trust tier per record/chunk.
• Define trust tiers and claim-specific source-selection/conflict policy.
• Document freshness/staleness behavior.
• Make source updates explicit and auditable.
• Validate upstream source origins.
• Enforce response/download size limits.
• Preserve raw-source provenance where licensing permits: retrieval URLs + raw SHA-256 manifests; optional snapshots via ZAGROS_PRESERVE_RAW_SOURCES.
• Distinguish source evidence from Zagros metadata and agent interpretation.
• Require source IDs/URLs in investigation outputs.
• Document prompt-injection handling for retrieved security text.
• Document behavior when authoritative sources disagree.

P1 — Deployment and usability

• Provide one canonical 5-minute local Docker quick start.
• Provide separate cloud/self-hosted deployment guidance.
• Make Traefik optional rather than required by the default Compose setup.
• Add .env.example with safe placeholders.
• Verify Windows, Linux, and macOS/WSL installation paths.
• Document recovery/no-backup policy, upgrade, and uninstall.
• Document bounded CPU/RAM/PID requirements for the current corpus; Zagros data is rebuildable and disk use is corpus-dependent.
• Clarify which ports are local-only and which may be exposed.
• Add authentication guidance for internet-facing MCP deployments.
• Add/verify health and readiness checks for runtime services.

P1 — CI and release engineering

• Add CI for cargo fmt —all — —check.
• Add CI for cargo clippy —all-targets —all-features — -D warnings.
• Add CI for cargo test —all-targets.
• Add CI for website npm ci + npm run build.
• Add Docker image build tests for MCP, CLI, and UI.
• Add end-to-end smoke test: HelixDB -> seed -> CLI query -> MCP query.
• Add Rust/npm/container/GitHub Actions dependency scanning.
• Pin important release dependencies/actions where practical.
• Define semantic versioning and CHANGELOG policy.
• Create reproducible GitHub Releases with checksums.
• Publish container images with immutable version tags/digests.
• Automate skill package release and checksum verification.

P1 — Repository and contributor readiness

• Add CONTRIBUTING.md.
• Add CODE_OF_CONDUCT.md.
• Add CHANGELOG.md.
• Add ROADMAP.md.
• Add THIRD_PARTY_NOTICES.md.
• Add issue templates: bug, feature, data-source request.
• Add pull-request template with tests/docs/security checklist.
• Enable GitHub private vulnerability reporting. Enabled after the repository became public on 2026-09-14.
• Add branch protection and required CI checks. main now requires PR review, CODEOWNERS approval, conversation resolution, strict required checks, and blocks force-push/deletion.
• Sanitize public Git history. Reachable public history was rewritten and post-rewrite scans found no matching credential/private-key patterns or removed private-environment identifiers. A private recovery bundle is retained outside the repository until GitHub server-side cleanup is verified.
• Use GitHub noreply identity for repository commits to avoid exposing personal author email addresses.
• Remove/ignore generated artifacts that should not be versioned; Graphify caches are untracked while intentional reports remain.

P1 — Supply-chain security

• Add Rust dependency audit.
• Add npm dependency audit.
• Add Dependabot or Renovate.
• Add CodeQL.
• Add container vulnerability scanning.
• Generate SBOMs for releases.
• Use immutable Docker digests for production examples.
• Sign release artifacts where practical.
• Publish SHA256 checksums for binaries and skill packages.

P1 Review — 2026-09-12

Status legend: Done = implemented and documented; Partial = useful pieces exist but release requirement is not fully closed; Open = not implemented or not yet verified.

Trusted-source guarantees

Deployment and usability

CI and release engineering

Done: CI now gates Rust formatting, clippy, tests, website build, Compose validation, and all three Docker image builds. A deterministic HelixDB -> seed -> CLI -> MCP E2E workflow runs on pull requests, main, schedule, and manual dispatch. Security automation covers Rust/npm audits, CodeQL, container scanning, Dependabot, and live-source checks. Tagged releases build Linux/Windows binaries, the deterministic skill package, signed GHCR images, checksums, BuildKit provenance/SBOM attestations, and a downloadable SPDX SBOM. Important release actions are pinned by commit SHA. The E2E workflow pins HelixDB v3.1.1 by immutable commit and builds it from source on Ubuntu. Local Linux-container verification was attempted on 2026-09-12, but Docker Desktop terminated the build twice with unexpected EOF; no Helix compile error was observed. Hosted validation is now green on the public repository: CI run 34847953201, Security run 34847953062, OpenSSF Scorecard run 34847953135, and E2E smoke run 34847952996 all succeeded on commit 875d1b8.

Repository and contributor readiness

• Done: THIRD_PARTY_NOTICES.md, SECURITY.md, CHANGELOG.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, ROADMAP.md, issue templates, and PR template.
• Done: GitHub Private Vulnerability Reporting is enabled.
• Done: main branch protection is active with one approval, CODEOWNERS review, stale-review dismissal, conversation resolution, strict required status checks, and force-push/deletion disabled.
• Done: repository-history hygiene. Reachable history was rewritten to remove the historical .vscode/mcp.json, personal Windows path, and obsolete private deployment domain. Post-rewrite scans found no matching token/private-key patterns or those historical identifiers. A recovery bundle is retained locally outside the repository.
• Done: generated-file hygiene for P1. Build outputs and Graphify caches are ignored/untracked; intentional Graphify reports remain versioned.

Supply-chain security

Done for repository automation: Rust and npm audits, Dependabot, CodeQL, Trivy container scanning, release SBOM generation, BuildKit SBOM/provenance attestations, keyless Cosign signing, SHA256 release checksums, and immutable GHCR digest publication are implemented. Runtime images were hardened until local Trivy scans reported zero HIGH/CRITICAL findings. Production release documentation demonstrates digest-pinned deployment. Hosted CI, E2E, Security/CodeQL, dependency/container scanning, and OpenSSF Scorecard have been confirmed green on the public repository.

P3 — Public-release hardening

• Add .github/CODEOWNERS for repository and security-sensitive ownership.
• Pin every external GitHub Actions uses: reference to a full immutable commit SHA; verification found 49 external references and zero unpinned references.
• Add the official GitHub Dependency Review pull-request gate, failing on moderate-or-higher newly introduced vulnerabilities.
• Add OpenSSF Scorecard with SARIF/code-scanning upload and immutable action pins.
• Make the sanitized repository public.
• Enable Secret Scanning and Push Protection.
• Enable Dependabot security updates.
• Enable Private Vulnerability Reporting.
• Protect main with required pull-request review, CODEOWNERS review, resolved conversations, strict required checks, and force-push/deletion disabled.
• Fix CodeQL workflow permissions and verify the public Security workflow succeeds.
• Verify public hosted CI, E2E, Security, and OpenSSF Scorecard workflows all succeed on commit 875d1b8.
• Final public-history scan found zero tracked secret-like files and zero matching credential/private-key patterns in reachable main history.

P2 — Skills ecosystem

• Define a stable Zagros Skill Contract. Contract v1 is documented in docs/SKILL-CONTRACT.md and enforced by skills/validate.py.
• Keep generic security guidance harness-neutral. SKILL.md remains generic; harness/Zagros wiring is kept outside the core guidance.
• Keep Zagros-specific MCP mappings in a separate reference. Current mapping: skills/.apm/skills/cybersecurity-expert/references/zagros-mcp.md.
• Document Microsoft APM install, upgrade, uninstall, and compatibility. See skills/COMPATIBILITY.md.
• Add skill evaluation cases with expected behavior. The suite now has 12 cases including injection, source conflict, evidence/inference, stale data, and approval boundaries.
• Add examples for OpenCode, Copilot, Claude-compatible clients, and generic MCP clients. See skills/HARNESSES.md.
• Define how community-contributed skills are reviewed and trusted. See docs/SKILL-TRUST-POLICY.md.
• Version skills independently from Zagros Core when needed. Skill SemVer and Core compatibility are explicit in package metadata and skills/COMPATIBILITY.md.
• Rename skill/ to skills/ for future multi-skill growth and update release/docs/install paths.

P2 — Quality and project confidence

• Add architecture decision records for major design choices. See docs/adr/.
• Add threat model for local, LAN, and public-cloud deployments. See docs/THREAT-MODEL.md.
• Add retrieval-quality benchmarks and regression tests. Deterministic offline benchmark: 32 cases with Recall@1, Recall@5, MRR, and irrelevant-query rejection gates.
• Add corpus integrity/freshness metrics. index_status now reports per-source count thresholds, freshness age/state, and overall health.
• Publish a roadmap (ROADMAP.md).
• Label experimental vs stable interfaces/features clearly. See docs/INTERFACE-STATUS.md.
• Add screenshots/demo flow for a real security investigation. See docs/DEMO.md and the public /demo/ page.
• State non-goals clearly: Zagros is an evidence source, not compliance certification or autonomous remediation.

P2 — Graphify cleanup

• Add .graphifyignore.
• Exclude generated/unparseable inputs: target, website build/cache directories, node_modules, skill dist, Graphify outputs/caches, data, and *.astro parser-noise files.
• Investigate/remove stale references to C:/Projects/Zagros; none remain in rebuilt public Graphify outputs.
• Rebuild Graphify with --force after cleanup: 1,810 nodes / 2,312 edges / 188 communities.
• Run multigraph diagnostics: zero duplicate/collapsed/dangling/self-loop edges.
• Define public Graphify artifacts: keep GRAPH_REPORT.md, graph.json, graph.html, and manifest.json; exclude cost.json, caches, and dated backups.

Current verified baseline — 2026-09-14

• Graphify refreshed after cleanup: 1,810 nodes / 2,312 edges / 188 communities.
• Multigraph diagnostics clean: no duplicate, collapsed, dangling, or self-loop edges.
• cargo fmt --all -- --check passes.
• cargo test --all-targets passes: 9 tests passed across library/MCP/retrieval suites; E2E seed test intentionally ignored without HelixDB.
• cargo clippy --all-targets --all-features -- -D warnings passes.
• Astro website builds successfully: 37 static pages including the investigation demo and quality-metrics docs.
• Tracked-file/history scans found no obvious committed credentials/private keys or generated build caches.
• Graphify Astro/generated parser noise is excluded through .graphifyignore; only zero-node JSON warnings for tooling/eval metadata remain.

First public-release gate

Do not tag the first public release until:

1. Skill licensing/provenance is resolved.
2. Current security review has no unresolved High findings. Local fixes are prepared for the post-rewrite Trivy and RustSec failures; hosted verification is still required.
3. Fresh clone -> local Docker deployment works without a personal Traefik environment. Verified from public main on 2026-09-15.
4. CI and end-to-end tests are green.
5. Documentation accurately describes the current implementation.

Remaining release gate: publish and verify the two local security fixes, complete the remaining GitHub cleanup verification, then prepare the first public release. Tagging/releasing still requires explicit authorization.