diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 0000000..5cfe442 --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,264 @@ +# Woodpecker CI 本地测试工具 + +本目录包含用于本地测试和验证 Woodpecker CI 配置的工具。 + +## 📁 文件说明 + +### 1. `validate-woodpecker.sh` - 配置验证工具 + +**功能**:全面验证 `.woodpecker.yml` 配置文件的正确性 + +**检查项目**: +- ✅ YAML 语法检查 +- ✅ 必需字段检查(steps, image, commands) +- ✅ 镜像格式验证 +- ✅ 环境变量和 secrets 检查 +- ✅ when 条件逻辑分析 +- ✅ 执行顺序模拟 + +**使用方法**: +```bash +./scripts/validate-woodpecker.sh +``` + +**输出示例**: +``` +========================================== +Woodpecker CI 配置本地验证工具 +========================================== + +✅ 文件存在: .woodpecker.yml + +1️⃣ YAML 语法检查 +---------------------------------------- +✅ YAML 语法正确 + +2️⃣ 检查必需字段 +---------------------------------------- + ✅ 步骤 'lint' 有分支条件: ['feature/**', 'dev', 'release', 'release/**'] + ✅ 步骤 'lint' 有事件条件: ['push', 'pull_request'] + ... +``` + +### 2. `test-step.sh` - 单步测试工具 + +**功能**:在本地 Docker 环境中测试单个 pipeline 步骤 + +**使用方法**: +```bash +# 查看可用步骤 +./scripts/test-step.sh + +# Dry-run 模式(仅显示配置,不执行) +./scripts/test-step.sh notify-wechat-success --dry-run + +# 实际执行步骤 +./scripts/test-step.sh lint +``` + +**特性**: +- 🔍 自动解析步骤配置 +- 🐳 使用 Docker 隔离环境 +- 🔐 模拟 Woodpecker CI 环境变量 +- 📝 显示详细执行信息 + +### 3. `test-woodpecker-local.sh` - 本地测试指南 + +**功能**:显示 Woodpecker CI 本地测试的方法和命令 + +**使用方法**: +```bash +./scripts/test-woodpecker-local.sh +``` + +## 🚀 快速开始 + +### 1. 验证配置文件 + +在提交代码前,先运行验证工具: + +```bash +./scripts/validate-woodpecker.sh +``` + +如果所有检查都通过,说明配置文件基本正确。 + +### 2. 测试单个步骤 + +如果某个步骤有问题,可以使用单步测试工具: + +```bash +# 先 dry-run 查看配置 +./scripts/test-step.sh --dry-run + +# 确认无误后执行 +./scripts/test-step.sh +``` + +### 3. 使用 Woodpecker CLI(推荐) + +安装 Woodpecker CLI: + +```bash +# macOS +brew install woodpecker-cli + +# Linux +curl -L https://github.com/woodpecker-ci/woodpecker/releases/latest/download/woodpecker-cli-linux-amd64 -o /usr/local/bin/woodpecker-cli +chmod +x /usr/local/bin/woodpecker-cli +``` + +本地运行整个 pipeline: + +```bash +woodpecker-cli exec .woodpecker.yml +``` + +### 4. 使用 Docker 模拟 + +如果没有安装 Woodpecker CLI,可以使用 Docker: + +```bash +docker run --rm \ + -v $(pwd):/woodpecker/src \ + -w /woodpecker/src \ + woodpeckerci/woodpecker-cli:latest \ + exec .woodpecker.yml +``` + +## 🔧 高级用法 + +### 测试特定分支的步骤 + +设置环境变量模拟特定分支: + +```bash +export CI_COMMIT_BRANCH="release/v1.0.0" +./scripts/test-step.sh notify-wechat-success +``` + +### 测试 secrets + +**注意**:本地测试无法访问 Woodpecker CI 中的 secrets。 + +解决方案: +1. 创建 `.env` 文件存储测试用的 secrets(**不要提交到 git**) +2. 在测试时手动设置环境变量: + +```bash +export WECHAT_WEBHOOK="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=YOUR_KEY" +./scripts/test-step.sh notify-wechat-success +``` + +### 调试环境变量 + +查看步骤会接收到哪些环境变量: + +```bash +./scripts/test-step.sh --dry-run | grep "环境变量" +``` + +## 📋 最佳实践 + +### 1. 提交前验证 + +在每次修改 `.woodpecker.yml` 后,运行: + +```bash +./scripts/validate-woodpecker.sh +``` + +### 2. 逐步测试 + +不要一次性测试整个 pipeline,而是: + +1. 先验证配置文件 +2. 再测试单个步骤 +3. 最后测试整个 pipeline + +### 3. 使用版本控制 + +将测试脚本纳入版本控制: + +```bash +git add scripts/ +git commit -m "feat: 添加 Woodpecker CI 本地测试工具" +``` + +### 4. 持续改进 + +发现新的测试需求时,更新测试脚本: + +```bash +# 编辑验证脚本 +vim scripts/validate-woodpecker.sh + +# 添加新的检查项 +``` + +## 🐛 常见问题 + +### Q1: 为什么本地测试成功,但 CI 中失败? + +**可能原因**: +1. 环境变量不同(检查 secrets) +2. 网络访问限制 +3. 文件权限问题 +4. Docker 镜像版本不一致 + +**解决方法**: +```bash +# 对比环境变量 +./scripts/test-step.sh --dry-run + +# 检查 CI 日志中的环境变量 +# 在 CI 中添加调试命令 +commands: + - env | sort + - echo "Branch: $CI_COMMIT_BRANCH" +``` + +### Q2: 如何测试需要 secrets 的步骤? + +**方法 1**:使用测试用的 secrets +```bash +export WECHAT_WEBHOOK="https://test.example.com/webhook" +./scripts/test-step.sh notify-wechat-success +``` + +**方法 2**:跳过 secrets 检查 +```bash +# 修改步骤配置,使用环境变量而不是 from_secret +``` + +### Q3: 如何测试 when 条件? + +**方法**:设置相应的环境变量 +```bash +# 测试 release 分支的步骤 +export CI_COMMIT_BRANCH="release/v1.0.0" +./scripts/test-step.sh deploy-production --dry-run + +# 测试 feature 分支的步骤 +export CI_COMMIT_BRANCH="feature/new-feature" +./scripts/test-step.sh e2e-smoke --dry-run +``` + +## 📚 相关资源 + +- [Woodpecker CI 官方文档](https://woodpecker-ci.org/docs/intro) +- [Woodpecker CLI 文档](https://woodpecker-ci.org/docs/cli) +- [Woodpecker 配置参考](https://woodpecker-ci.org/docs/usage/pipeline-syntax) + +## 🤝 贡献 + +如果你发现新的测试需求或改进点,欢迎更新这些脚本: + +1. Fork 项目 +2. 创建特性分支 +3. 提交改进 +4. 创建 Pull Request + +## 📄 许可证 + +这些测试工具遵循项目的主许可证。 diff --git a/scripts/test-step.sh b/scripts/test-step.sh new file mode 100755 index 0000000..7d87040 --- /dev/null +++ b/scripts/test-step.sh @@ -0,0 +1,273 @@ +#!/bin/bash + +set -e + +WOODPECKER_FILE=".woodpecker.yml" + +# 颜色定义 +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +print_header() { + echo "" + echo -e "${BLUE}==========================================${NC}" + echo -e "${BLUE}$1${NC}" + echo -e "${BLUE}==========================================${NC}" + echo "" +} + +print_success() { + echo -e "${GREEN}✅ $1${NC}" +} + +print_error() { + echo -e "${RED}❌ $1${NC}" +} + +print_warning() { + echo -e "${YELLOW}⚠️ $1${NC}" +} + +print_info() { + echo -e "${BLUE}ℹ️ $1${NC}" +} + +# 检查是否提供了步骤名称 +if [ -z "$1" ]; then + print_header "Woodpecker CI 单步测试工具" + echo "用法: $0 [--dry-run]" + echo "" + echo "示例:" + echo " $0 notify-wechat-success --dry-run # 仅显示命令,不执行" + echo " $0 lint # 执行 lint 步骤" + echo "" + echo "可用的步骤:" + python3 << 'PYTHON_SCRIPT' +import yaml + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) +for i, (step_name, step_config) in enumerate(steps.items(), 1): + image = step_config.get('image', 'N/A') + print(f" {i}. {step_name:<25} (镜像: {image})") +PYTHON_SCRIPT + exit 1 +fi + +STEP_NAME="$1" +DRY_RUN="${2:-}" + +print_header "测试步骤: $STEP_NAME" + +# 检查步骤是否存在 +STEP_EXISTS=$(python3 << PYTHON_SCRIPT +import yaml +import sys + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) +step_name = "$STEP_NAME" + +if step_name in steps: + print("yes") +else: + print("no") +PYTHON_SCRIPT +) + +if [ "$STEP_EXISTS" != "yes" ]; then + print_error "步骤 '$STEP_NAME' 不存在" + exit 1 +fi + +print_success "步骤 '$STEP_NAME' 存在" + +# 获取步骤配置 +python3 << PYTHON_SCRIPT +import yaml +import json +import sys + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +step_name = "$STEP_NAME" +step_config = config['steps'][step_name] + +print(f"\n📦 镜像: {step_config.get('image', 'N/A')}") + +# 显示环境变量 +env = step_config.get('environment', {}) +if env: + print("\n🔐 环境变量:") + for key, value in env.items(): + if isinstance(value, dict) and 'from_secret' in value: + print(f" - {key}: from_secret:{value['from_secret']}") + else: + print(f" - {key}: {value}") + +# 显示 when 条件 +when = step_config.get('when', {}) +if when: + print("\n📋 执行条件:") + if 'branch' in when: + branches = when['branch'] + if isinstance(branches, list): + print(f" - 分支: {', '.join(branches)}") + else: + print(f" - 分支: {branches}") + if 'event' in when: + events = when['event'] + if isinstance(events, list): + print(f" - 事件: {', '.join(events)}") + else: + print(f" - 事件: {events}") + if 'status' in when: + statuses = when['status'] + if isinstance(statuses, list): + print(f" - 状态: {', '.join(statuses)}") + else: + print(f" - 状态: {statuses}") + +# 显示命令 +commands = step_config.get('commands', []) +if commands: + print("\n📝 命令:") + for i, cmd in enumerate(commands, 1): + # 显示前100个字符 + if len(cmd) > 100: + print(f" {i}. {cmd[:100]}...") + else: + print(f" {i}. {cmd}") + +# 输出 JSON 配置(用于后续处理) +print("\n" + json.dumps(step_config)) +PYTHON_SCRIPT + +# 如果是 dry-run 模式,只显示信息,不执行 +if [ "$DRY_RUN" = "--dry-run" ]; then + print_info "Dry-run 模式,不执行命令" + exit 0 +fi + +# 询问是否继续执行 +echo "" +read -p "是否继续执行此步骤?(y/N) " -n 1 -r +echo "" +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + print_info "已取消执行" + exit 0 +fi + +# 执行步骤 +print_header "执行步骤" + +# 获取镜像 +IMAGE=$(python3 << PYTHON_SCRIPT +import yaml + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +step_name = "$STEP_NAME" +print(config['steps'][step_name].get('image', '')) +PYTHON_SCRIPT +) + +if [ -z "$IMAGE" ]; then + print_error "无法获取镜像信息" + exit 1 +fi + +print_info "使用镜像: $IMAGE" + +# 拉取镜像 +print_info "拉取镜像..." +docker pull "$IMAGE" || { + print_warning "镜像拉取失败,尝试使用本地镜像" +} + +# 获取命令 +COMMANDS=$(python3 << 'PYTHON_SCRIPT' +import yaml +import json + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +step_name = "$STEP_NAME" +commands = config['steps'][step_name].get('commands', []) + +# 将命令转换为 JSON 字符串 +print(json.dumps(commands)) +PYTHON_SCRIPT +) + +# 执行命令 +print_info "执行命令..." + +# 创建临时脚本 +TEMP_SCRIPT=$(mktemp) +trap "rm -f $TEMP_SCRIPT" EXIT + +# 写入命令 +python3 << PYTHON_SCRIPT > "$TEMP_SCRIPT" +import yaml +import json + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +step_name = "$STEP_NAME" +commands = config['steps'][step_name].get('commands', []) + +for cmd in commands: + print(cmd) +PYTHON_SCRIPT + +chmod +x "$TEMP_SCRIPT" + +# 设置环境变量(模拟 Woodpecker CI) +export CI="true" +export CI_COMMIT_BRANCH="${CI_COMMIT_BRANCH:-release/v1.0.0}" +export CI_COMMIT_SHA="${CI_COMMIT_SHA:-$(git rev-parse HEAD 2>/dev/null || echo 'abc123def456')}" +export CI_COMMIT_MESSAGE="${CI_COMMIT_MESSAGE:-$(git log -1 --pretty=%B 2>/dev/null || echo 'Test commit')}" +export CI_COMMIT_AUTHOR="${CI_COMMIT_AUTHOR:-$(git log -1 --pretty=%an 2>/dev/null || echo 'Test Author')}" +export CI_PIPELINE_NUMBER="${CI_PIPELINE_NUMBER:-999}" +export CI_REPO_ID="${CI_REPO_ID:-1}" + +print_info "环境变量:" +echo " CI_COMMIT_BRANCH: $CI_COMMIT_BRANCH" +echo " CI_COMMIT_SHA: $CI_COMMIT_SHA" +echo " CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE" +echo " CI_COMMIT_AUTHOR: $CI_COMMIT_AUTHOR" +echo " CI_PIPELINE_NUMBER: $CI_PIPELINE_NUMBER" +echo "" + +# 使用 Docker 执行 +docker run --rm \ + -v "$(pwd):/woodpecker/src" \ + -w /woodpecker/src \ + -e CI \ + -e CI_COMMIT_BRANCH \ + -e CI_COMMIT_SHA \ + -e CI_COMMIT_MESSAGE \ + -e CI_COMMIT_AUTHOR \ + -e CI_PIPELINE_NUMBER \ + -e CI_REPO_ID \ + "$IMAGE" \ + sh -c "$(cat "$TEMP_SCRIPT")" + +if [ $? -eq 0 ]; then + print_success "步骤执行成功" +else + print_error "步骤执行失败" + exit 1 +fi diff --git a/scripts/validate-woodpecker.sh b/scripts/validate-woodpecker.sh new file mode 100755 index 0000000..a53b0cf --- /dev/null +++ b/scripts/validate-woodpecker.sh @@ -0,0 +1,253 @@ +#!/bin/bash + +set -e + +WOODPECKER_FILE=".woodpecker.yml" + +echo "==========================================" +echo "Woodpecker CI 配置本地验证工具" +echo "==========================================" +echo "" + +# 检查文件是否存在 +if [ ! -f "$WOODPECKER_FILE" ]; then + echo "❌ 错误: $WOODPECKER_FILE 文件不存在" + exit 1 +fi + +echo "✅ 文件存在: $WOODPECKER_FILE" +echo "" + +# 1. YAML 语法检查 +echo "1️⃣ YAML 语法检查" +echo "----------------------------------------" +if command -v python3 &> /dev/null; then + if python3 -c "import yaml; yaml.safe_load(open('$WOODPECKER_FILE'))" 2>&1; then + echo "✅ YAML 语法正确" + else + echo "❌ YAML 语法错误" + exit 1 + fi +else + echo "⚠️ Python3 未安装,跳过 YAML 语法检查" +fi +echo "" + +# 2. 检查必需的字段 +echo "2️⃣ 检查必需字段" +echo "----------------------------------------" +python3 << 'PYTHON_SCRIPT' +import yaml +import sys + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +errors = [] +warnings = [] + +# 检查 steps +if 'steps' not in config: + errors.append("缺少 'steps' 字段") +else: + steps = config['steps'] + if not steps: + errors.append("'steps' 不能为空") + else: + for step_name, step_config in steps.items(): + # 检查 image + if 'image' not in step_config: + errors.append(f"步骤 '{step_name}' 缺少 'image' 字段") + + # 检查 commands + if 'commands' not in step_config: + warnings.append(f"步骤 '{step_name}' 没有 'commands' 字段") + + # 检查 when 条件 + if 'when' in step_config: + when = step_config['when'] + if 'branch' in when: + print(f" ✅ 步骤 '{step_name}' 有分支条件: {when['branch']}") + if 'event' in when: + print(f" ✅ 步骤 '{step_name}' 有事件条件: {when['event']}") + +# 检查 workspace +if 'workspace' in config: + print(f" ✅ workspace 配置: {config['workspace']}") + +# 检查 clone +if 'clone' in config: + print(f" ✅ clone 配置: {config['clone']}") + +if errors: + print("\n❌ 错误:") + for error in errors: + print(f" - {error}") + sys.exit(1) + +if warnings: + print("\n⚠️ 警告:") + for warning in warnings: + print(f" - {warning}") + +print("\n✅ 所有必需字段检查通过") +PYTHON_SCRIPT + +if [ $? -ne 0 ]; then + exit 1 +fi +echo "" + +# 3. 检查镜像是否存在 +echo "3️⃣ 检查镜像格式" +echo "----------------------------------------" +python3 << 'PYTHON_SCRIPT' +import yaml +import re + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) +image_pattern = re.compile(r'^[a-z0-9\-_./]+(?::[a-z0-9\-_.]+)?$') + +for step_name, step_config in steps.items(): + image = step_config.get('image', '') + if image: + # 检查镜像格式 + if image.startswith('*'): + # YAML anchor,跳过 + print(f" ℹ️ 步骤 '{step_name}' 使用 YAML anchor: {image}") + elif image_pattern.match(image): + print(f" ✅ 步骤 '{step_name}' 镜像格式正确: {image}") + else: + print(f" ⚠️ 步骤 '{step_name}' 镜像格式可能有问题: {image}") + +print("\n✅ 镜像格式检查完成") +PYTHON_SCRIPT +echo "" + +# 4. 检查环境变量和 secrets +echo "4️⃣ 检查环境变量和 secrets" +echo "----------------------------------------" +python3 << 'PYTHON_SCRIPT' +import yaml + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) +secrets_used = set() + +for step_name, step_config in steps.items(): + env = step_config.get('environment', {}) + if isinstance(env, dict): + for key, value in env.items(): + if isinstance(value, dict) and 'from_secret' in value: + secret_name = value['from_secret'] + secrets_used.add(secret_name) + print(f" 🔐 步骤 '{step_name}' 使用 secret: {secret_name}") + +if secrets_used: + print(f"\n📋 需要配置的 secrets:") + for secret in sorted(secrets_used): + print(f" - {secret}") + print("\n⚠️ 请确保在 Woodpecker CI 中配置了这些 secrets") +else: + print(" ℹ️ 没有使用 secrets") + +print("\n✅ 环境变量检查完成") +PYTHON_SCRIPT +echo "" + +# 5. 检查 when 条件的逻辑 +echo "5️⃣ 检查 when 条件逻辑" +echo "----------------------------------------" +python3 << 'PYTHON_SCRIPT' +import yaml + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) +steps_with_conditions = [] +steps_without_conditions = [] + +for step_name, step_config in steps.items(): + if 'when' in step_config: + steps_with_conditions.append(step_name) + else: + steps_without_conditions.append(step_name) + +if steps_with_conditions: + print(f" ✅ 有条件执行的步骤 ({len(steps_with_conditions)}):") + for step in steps_with_conditions: + print(f" - {step}") + +if steps_without_conditions: + print(f"\n ⚠️ 无条件执行的步骤 ({len(steps_without_conditions)}):") + for step in steps_without_conditions: + print(f" - {step}") + print("\n 💡 建议: 大部分步骤应该有 when 条件,避免不必要的执行") + +print("\n✅ when 条件检查完成") +PYTHON_SCRIPT +echo "" + +# 6. 模拟执行顺序 +echo "6️⃣ 模拟执行顺序" +echo "----------------------------------------" +python3 << 'PYTHON_SCRIPT' +import yaml + +with open('.woodpecker.yml', 'r') as f: + config = yaml.safe_load(f) + +steps = config.get('steps', {}) + +print(" 📋 步骤执行顺序:") +for i, (step_name, step_config) in enumerate(steps.items(), 1): + image = step_config.get('image', 'N/A') + when = step_config.get('when', {}) + + conditions = [] + if 'branch' in when: + branches = when['branch'] + if isinstance(branches, list): + conditions.append(f"branch: {', '.join(branches)}") + else: + conditions.append(f"branch: {branches}") + + if 'event' in when: + events = when['event'] + if isinstance(events, list): + conditions.append(f"event: {', '.join(events)}") + else: + conditions.append(f"event: {events}") + + if 'status' in when: + statuses = when['status'] + if isinstance(statuses, list): + conditions.append(f"status: {', '.join(statuses)}") + else: + conditions.append(f"status: {statuses}") + + condition_str = f" [{', '.join(conditions)}]" if conditions else "" + print(f" {i}. {step_name}{condition_str}") + print(f" 镜像: {image}") + +print("\n✅ 执行顺序分析完成") +PYTHON_SCRIPT +echo "" + +echo "==========================================" +echo "✅ 所有检查完成!" +echo "==========================================" +echo "" +echo "💡 提示:" +echo " - 如果所有检查都通过,配置文件基本正确" +echo " - 建议使用 Woodpecker CLI 进行本地测试:" +echo " woodpecker-cli exec .woodpecker.yml" +echo " - 或者使用 Docker:" +echo " docker run --rm -v \$(pwd):/woodpecker/src -w /woodpecker/src woodpeckerci/woodpecker-cli:latest exec .woodpecker.yml" +echo ""