Reports

Bytesalt generates comprehensive reports for every job. These reports are available in multiple formats, including Markdown, PDF, HTML, and JSON.

Reports provide both human-readable summaries and structured data for programmatic consumption in CI/CD pipelines.

Report Types

Bytesalt provides two levels of reports:

• Summary Report: Contains high-level metadata, the executive summary, and issue statistics.
• Full Report: Contains everything in the summary report, plus the full deduplicated list of discovered issues with detailed evidence and recommendations.

Accessing Reports

You can retrieve reports using:

• The Web Dashboard (for visual browsing)
• The CLI bytesalt report command (for downloading artifacts)
• The API (for programmatic access)

Available formats include:

• Markdown (.md) - Human-readable, great for PR comments
• PDF (.pdf) - Polished documents for stakeholders
• HTML (.html) - Shareable web-ready report output
• JSON (.json) - Structured data for automation

Policy Expressions

The --fail-if flag allows you to define custom failure conditions using CEL (Common Expression Language)  expressions. These expressions are evaluated against the job’s result data.

Default Behavior

If you don’t specify --fail-if, Bytesalt uses a sensible default:

result.issues.critical > 0 || result.issues.high > 0

This means jobs automatically fail if any critical or high severity issues are discovered.

Result Schema

The result object available in your --fail-if expressions has the following structure:

{
  verdict: string,           // Human-readable verdict: "Stable", "Notice", "Warning", "Unstable", "Blocking"
  issues: {
    total: number,           // Total number of issues found
    critical: number,        // Count of critical severity issues
    high: number,            // Count of high severity issues
    medium: number,          // Count of medium severity issues
    low: number              // Count of low severity issues
  },
  summary: string            // AI-generated executive summary
}

Policy Examples

Fail on any issues (Strict):

bytesalt start "Test app" --fail-if "result.issues.total > 0"

Fail only on critical issues:

bytesalt start "Test app" --fail-if "result.issues.critical > 0"

Fail if more than 5 medium issues:

bytesalt start "Test app" --fail-if "result.issues.medium > 5"

Fail if any high or critical issues (explicit default):

bytesalt start "Test app" --fail-if "result.issues.critical > 0 || result.issues.high > 0"

Always pass (for informational runs):

bytesalt start "Test app" --fail-if "false"

CI/CD Integration

Policy expressions make it easy to implement quality gates in your CI/CD pipelines:

GitHub Actions Example:

- name: Run Bytesalt
  run: |
    bytesalt start "Test PR changes" \
      --context pr-diff=@<(git diff origin/main) \
      --fail-if "result.issues.critical > 0 || result.issues.high > 0" \
      --output type=summary,format=markdown,dest=report.md

GitLab CI Example:

test:
  script:
    - bytesalt start "E2E tests" --fail-if "result.issues.critical > 0"
  allow_failure: false

The exit code behavior:

• Exit 0: Job succeeded and policy passed (or no policy specified)
• Exit 1: Job failed, policy failed, or execution error

Verdict Descriptions

The result.verdict field provides a human-readable assessment:

Best Practices

1. Start with the default: The built-in policy (critical > 0 || high > 0) works well for most teams.
2. Tune based on maturity: New codebases might want stricter policies (total > 0), while legacy systems may want to focus only on critical issues.
3. Document your policy: Add comments in your CI config to explain why you chose a specific threshold.
4. Use multiple gates: You can combine Bytesalt’s policy with other checks (e.g., code coverage, linting).

Downloading Reports

Use the --output flag to download specific report formats:

# Download summary as PDF
bytesalt report --output type=summary,format=pdf,dest=summary.pdf
 
# Download full report as Markdown, HTML, and JSON
bytesalt report \
  --output type=full,format=markdown,dest=report.md \
  --output type=full,format=html,dest=report.html \
  --output type=full,format=json,dest=report.json

This is useful for:

• Archiving test results
• Posting reports in Slack/Teams
• Storing artifacts in CI systems
• Generating compliance documentation