feat: 添加 Woodpecker CI 本地测试工具
ci/woodpecker/push/woodpecker Pipeline failed

- validate-woodpecker.sh: 全面验证配置文件
- test-step.sh: 单步测试工具,支持 Docker 隔离环境
- README.md: 详细使用文档和最佳实践

这些工具可以在本地验证和测试 CI/CD 配置,避免通过持续提交来测试
This commit is contained in:
张翔
2026-03-28 22:24:46 +08:00
parent d1e23cf519
commit 6797a1ee2d
3 changed files with 790 additions and 0 deletions
+264
View File
@@ -0,0 +1,264 @@
# Woodpecker CI 本地测试工具
本目录包含用于本地测试和验证 Woodpecker CI 配置的工具。
## 📁 文件说明
### 1. `validate-woodpecker.sh` - 配置验证工具
**功能**:全面验证 `.woodpecker.yml` 配置文件的正确性
**检查项目**
- ✅ YAML 语法检查
- ✅ 必需字段检查(steps, image, commands
- ✅ 镜像格式验证
- ✅ 环境变量和 secrets 检查
- ✅ when 条件逻辑分析
- ✅ 执行顺序模拟
**使用方法**
```bash
./scripts/validate-woodpecker.sh
```
**输出示例**
```
==========================================
Woodpecker CI 配置本地验证工具
==========================================
✅ 文件存在: .woodpecker.yml
1️⃣ YAML 语法检查
----------------------------------------
✅ YAML 语法正确
2️⃣ 检查必需字段
----------------------------------------
✅ 步骤 'lint' 有分支条件: ['feature/**', 'dev', 'release', 'release/**']
✅ 步骤 'lint' 有事件条件: ['push', 'pull_request']
...
```
### 2. `test-step.sh` - 单步测试工具
**功能**:在本地 Docker 环境中测试单个 pipeline 步骤
**使用方法**
```bash
# 查看可用步骤
./scripts/test-step.sh
# Dry-run 模式(仅显示配置,不执行)
./scripts/test-step.sh notify-wechat-success --dry-run
# 实际执行步骤
./scripts/test-step.sh lint
```
**特性**
- 🔍 自动解析步骤配置
- 🐳 使用 Docker 隔离环境
- 🔐 模拟 Woodpecker CI 环境变量
- 📝 显示详细执行信息
### 3. `test-woodpecker-local.sh` - 本地测试指南
**功能**:显示 Woodpecker CI 本地测试的方法和命令
**使用方法**
```bash
./scripts/test-woodpecker-local.sh
```
## 🚀 快速开始
### 1. 验证配置文件
在提交代码前,先运行验证工具:
```bash
./scripts/validate-woodpecker.sh
```
如果所有检查都通过,说明配置文件基本正确。
### 2. 测试单个步骤
如果某个步骤有问题,可以使用单步测试工具:
```bash
# 先 dry-run 查看配置
./scripts/test-step.sh <step_name> --dry-run
# 确认无误后执行
./scripts/test-step.sh <step_name>
```
### 3. 使用 Woodpecker CLI(推荐)
安装 Woodpecker CLI
```bash
# macOS
brew install woodpecker-cli
# Linux
curl -L https://github.com/woodpecker-ci/woodpecker/releases/latest/download/woodpecker-cli-linux-amd64 -o /usr/local/bin/woodpecker-cli
chmod +x /usr/local/bin/woodpecker-cli
```
本地运行整个 pipeline
```bash
woodpecker-cli exec .woodpecker.yml
```
### 4. 使用 Docker 模拟
如果没有安装 Woodpecker CLI,可以使用 Docker
```bash
docker run --rm \
-v $(pwd):/woodpecker/src \
-w /woodpecker/src \
woodpeckerci/woodpecker-cli:latest \
exec .woodpecker.yml
```
## 🔧 高级用法
### 测试特定分支的步骤
设置环境变量模拟特定分支:
```bash
export CI_COMMIT_BRANCH="release/v1.0.0"
./scripts/test-step.sh notify-wechat-success
```
### 测试 secrets
**注意**:本地测试无法访问 Woodpecker CI 中的 secrets。
解决方案:
1. 创建 `.env` 文件存储测试用的 secrets(**不要提交到 git**
2. 在测试时手动设置环境变量:
```bash
export WECHAT_WEBHOOK="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=YOUR_KEY"
./scripts/test-step.sh notify-wechat-success
```
### 调试环境变量
查看步骤会接收到哪些环境变量:
```bash
./scripts/test-step.sh <step_name> --dry-run | grep "环境变量"
```
## 📋 最佳实践
### 1. 提交前验证
在每次修改 `.woodpecker.yml` 后,运行:
```bash
./scripts/validate-woodpecker.sh
```
### 2. 逐步测试
不要一次性测试整个 pipeline,而是:
1. 先验证配置文件
2. 再测试单个步骤
3. 最后测试整个 pipeline
### 3. 使用版本控制
将测试脚本纳入版本控制:
```bash
git add scripts/
git commit -m "feat: 添加 Woodpecker CI 本地测试工具"
```
### 4. 持续改进
发现新的测试需求时,更新测试脚本:
```bash
# 编辑验证脚本
vim scripts/validate-woodpecker.sh
# 添加新的检查项
```
## 🐛 常见问题
### Q1: 为什么本地测试成功,但 CI 中失败?
**可能原因**
1. 环境变量不同(检查 secrets)
2. 网络访问限制
3. 文件权限问题
4. Docker 镜像版本不一致
**解决方法**
```bash
# 对比环境变量
./scripts/test-step.sh <step_name> --dry-run
# 检查 CI 日志中的环境变量
# 在 CI 中添加调试命令
commands:
- env | sort
- echo "Branch: $CI_COMMIT_BRANCH"
```
### Q2: 如何测试需要 secrets 的步骤?
**方法 1**:使用测试用的 secrets
```bash
export WECHAT_WEBHOOK="https://test.example.com/webhook"
./scripts/test-step.sh notify-wechat-success
```
**方法 2**:跳过 secrets 检查
```bash
# 修改步骤配置,使用环境变量而不是 from_secret
```
### Q3: 如何测试 when 条件?
**方法**:设置相应的环境变量
```bash
# 测试 release 分支的步骤
export CI_COMMIT_BRANCH="release/v1.0.0"
./scripts/test-step.sh deploy-production --dry-run
# 测试 feature 分支的步骤
export CI_COMMIT_BRANCH="feature/new-feature"
./scripts/test-step.sh e2e-smoke --dry-run
```
## 📚 相关资源
- [Woodpecker CI 官方文档](https://woodpecker-ci.org/docs/intro)
- [Woodpecker CLI 文档](https://woodpecker-ci.org/docs/cli)
- [Woodpecker 配置参考](https://woodpecker-ci.org/docs/usage/pipeline-syntax)
## 🤝 贡献
如果你发现新的测试需求或改进点,欢迎更新这些脚本:
1. Fork 项目
2. 创建特性分支
3. 提交改进
4. 创建 Pull Request
## 📄 许可证
这些测试工具遵循项目的主许可证。
+273
View File
@@ -0,0 +1,273 @@
#!/bin/bash
set -e
WOODPECKER_FILE=".woodpecker.yml"
# 颜色定义
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
print_header() {
echo ""
echo -e "${BLUE}==========================================${NC}"
echo -e "${BLUE}$1${NC}"
echo -e "${BLUE}==========================================${NC}"
echo ""
}
print_success() {
echo -e "${GREEN}$1${NC}"
}
print_error() {
echo -e "${RED}$1${NC}"
}
print_warning() {
echo -e "${YELLOW}⚠️ $1${NC}"
}
print_info() {
echo -e "${BLUE}$1${NC}"
}
# 检查是否提供了步骤名称
if [ -z "$1" ]; then
print_header "Woodpecker CI 单步测试工具"
echo "用法: $0 <step_name> [--dry-run]"
echo ""
echo "示例:"
echo " $0 notify-wechat-success --dry-run # 仅显示命令,不执行"
echo " $0 lint # 执行 lint 步骤"
echo ""
echo "可用的步骤:"
python3 << 'PYTHON_SCRIPT'
import yaml
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
for i, (step_name, step_config) in enumerate(steps.items(), 1):
image = step_config.get('image', 'N/A')
print(f" {i}. {step_name:<25} (镜像: {image})")
PYTHON_SCRIPT
exit 1
fi
STEP_NAME="$1"
DRY_RUN="${2:-}"
print_header "测试步骤: $STEP_NAME"
# 检查步骤是否存在
STEP_EXISTS=$(python3 << PYTHON_SCRIPT
import yaml
import sys
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
step_name = "$STEP_NAME"
if step_name in steps:
print("yes")
else:
print("no")
PYTHON_SCRIPT
)
if [ "$STEP_EXISTS" != "yes" ]; then
print_error "步骤 '$STEP_NAME' 不存在"
exit 1
fi
print_success "步骤 '$STEP_NAME' 存在"
# 获取步骤配置
python3 << PYTHON_SCRIPT
import yaml
import json
import sys
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
step_name = "$STEP_NAME"
step_config = config['steps'][step_name]
print(f"\n📦 镜像: {step_config.get('image', 'N/A')}")
# 显示环境变量
env = step_config.get('environment', {})
if env:
print("\n🔐 环境变量:")
for key, value in env.items():
if isinstance(value, dict) and 'from_secret' in value:
print(f" - {key}: from_secret:{value['from_secret']}")
else:
print(f" - {key}: {value}")
# 显示 when 条件
when = step_config.get('when', {})
if when:
print("\n📋 执行条件:")
if 'branch' in when:
branches = when['branch']
if isinstance(branches, list):
print(f" - 分支: {', '.join(branches)}")
else:
print(f" - 分支: {branches}")
if 'event' in when:
events = when['event']
if isinstance(events, list):
print(f" - 事件: {', '.join(events)}")
else:
print(f" - 事件: {events}")
if 'status' in when:
statuses = when['status']
if isinstance(statuses, list):
print(f" - 状态: {', '.join(statuses)}")
else:
print(f" - 状态: {statuses}")
# 显示命令
commands = step_config.get('commands', [])
if commands:
print("\n📝 命令:")
for i, cmd in enumerate(commands, 1):
# 显示前100个字符
if len(cmd) > 100:
print(f" {i}. {cmd[:100]}...")
else:
print(f" {i}. {cmd}")
# 输出 JSON 配置(用于后续处理)
print("\n" + json.dumps(step_config))
PYTHON_SCRIPT
# 如果是 dry-run 模式,只显示信息,不执行
if [ "$DRY_RUN" = "--dry-run" ]; then
print_info "Dry-run 模式,不执行命令"
exit 0
fi
# 询问是否继续执行
echo ""
read -p "是否继续执行此步骤?(y/N) " -n 1 -r
echo ""
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
print_info "已取消执行"
exit 0
fi
# 执行步骤
print_header "执行步骤"
# 获取镜像
IMAGE=$(python3 << PYTHON_SCRIPT
import yaml
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
step_name = "$STEP_NAME"
print(config['steps'][step_name].get('image', ''))
PYTHON_SCRIPT
)
if [ -z "$IMAGE" ]; then
print_error "无法获取镜像信息"
exit 1
fi
print_info "使用镜像: $IMAGE"
# 拉取镜像
print_info "拉取镜像..."
docker pull "$IMAGE" || {
print_warning "镜像拉取失败,尝试使用本地镜像"
}
# 获取命令
COMMANDS=$(python3 << 'PYTHON_SCRIPT'
import yaml
import json
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
step_name = "$STEP_NAME"
commands = config['steps'][step_name].get('commands', [])
# 将命令转换为 JSON 字符串
print(json.dumps(commands))
PYTHON_SCRIPT
)
# 执行命令
print_info "执行命令..."
# 创建临时脚本
TEMP_SCRIPT=$(mktemp)
trap "rm -f $TEMP_SCRIPT" EXIT
# 写入命令
python3 << PYTHON_SCRIPT > "$TEMP_SCRIPT"
import yaml
import json
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
step_name = "$STEP_NAME"
commands = config['steps'][step_name].get('commands', [])
for cmd in commands:
print(cmd)
PYTHON_SCRIPT
chmod +x "$TEMP_SCRIPT"
# 设置环境变量(模拟 Woodpecker CI
export CI="true"
export CI_COMMIT_BRANCH="${CI_COMMIT_BRANCH:-release/v1.0.0}"
export CI_COMMIT_SHA="${CI_COMMIT_SHA:-$(git rev-parse HEAD 2>/dev/null || echo 'abc123def456')}"
export CI_COMMIT_MESSAGE="${CI_COMMIT_MESSAGE:-$(git log -1 --pretty=%B 2>/dev/null || echo 'Test commit')}"
export CI_COMMIT_AUTHOR="${CI_COMMIT_AUTHOR:-$(git log -1 --pretty=%an 2>/dev/null || echo 'Test Author')}"
export CI_PIPELINE_NUMBER="${CI_PIPELINE_NUMBER:-999}"
export CI_REPO_ID="${CI_REPO_ID:-1}"
print_info "环境变量:"
echo " CI_COMMIT_BRANCH: $CI_COMMIT_BRANCH"
echo " CI_COMMIT_SHA: $CI_COMMIT_SHA"
echo " CI_COMMIT_MESSAGE: $CI_COMMIT_MESSAGE"
echo " CI_COMMIT_AUTHOR: $CI_COMMIT_AUTHOR"
echo " CI_PIPELINE_NUMBER: $CI_PIPELINE_NUMBER"
echo ""
# 使用 Docker 执行
docker run --rm \
-v "$(pwd):/woodpecker/src" \
-w /woodpecker/src \
-e CI \
-e CI_COMMIT_BRANCH \
-e CI_COMMIT_SHA \
-e CI_COMMIT_MESSAGE \
-e CI_COMMIT_AUTHOR \
-e CI_PIPELINE_NUMBER \
-e CI_REPO_ID \
"$IMAGE" \
sh -c "$(cat "$TEMP_SCRIPT")"
if [ $? -eq 0 ]; then
print_success "步骤执行成功"
else
print_error "步骤执行失败"
exit 1
fi
+253
View File
@@ -0,0 +1,253 @@
#!/bin/bash
set -e
WOODPECKER_FILE=".woodpecker.yml"
echo "=========================================="
echo "Woodpecker CI 配置本地验证工具"
echo "=========================================="
echo ""
# 检查文件是否存在
if [ ! -f "$WOODPECKER_FILE" ]; then
echo "❌ 错误: $WOODPECKER_FILE 文件不存在"
exit 1
fi
echo "✅ 文件存在: $WOODPECKER_FILE"
echo ""
# 1. YAML 语法检查
echo "1️⃣ YAML 语法检查"
echo "----------------------------------------"
if command -v python3 &> /dev/null; then
if python3 -c "import yaml; yaml.safe_load(open('$WOODPECKER_FILE'))" 2>&1; then
echo "✅ YAML 语法正确"
else
echo "❌ YAML 语法错误"
exit 1
fi
else
echo "⚠️ Python3 未安装,跳过 YAML 语法检查"
fi
echo ""
# 2. 检查必需的字段
echo "2️⃣ 检查必需字段"
echo "----------------------------------------"
python3 << 'PYTHON_SCRIPT'
import yaml
import sys
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
errors = []
warnings = []
# 检查 steps
if 'steps' not in config:
errors.append("缺少 'steps' 字段")
else:
steps = config['steps']
if not steps:
errors.append("'steps' 不能为空")
else:
for step_name, step_config in steps.items():
# 检查 image
if 'image' not in step_config:
errors.append(f"步骤 '{step_name}' 缺少 'image' 字段")
# 检查 commands
if 'commands' not in step_config:
warnings.append(f"步骤 '{step_name}' 没有 'commands' 字段")
# 检查 when 条件
if 'when' in step_config:
when = step_config['when']
if 'branch' in when:
print(f" ✅ 步骤 '{step_name}' 有分支条件: {when['branch']}")
if 'event' in when:
print(f" ✅ 步骤 '{step_name}' 有事件条件: {when['event']}")
# 检查 workspace
if 'workspace' in config:
print(f" ✅ workspace 配置: {config['workspace']}")
# 检查 clone
if 'clone' in config:
print(f" ✅ clone 配置: {config['clone']}")
if errors:
print("\n❌ 错误:")
for error in errors:
print(f" - {error}")
sys.exit(1)
if warnings:
print("\n⚠️ 警告:")
for warning in warnings:
print(f" - {warning}")
print("\n✅ 所有必需字段检查通过")
PYTHON_SCRIPT
if [ $? -ne 0 ]; then
exit 1
fi
echo ""
# 3. 检查镜像是否存在
echo "3️⃣ 检查镜像格式"
echo "----------------------------------------"
python3 << 'PYTHON_SCRIPT'
import yaml
import re
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
image_pattern = re.compile(r'^[a-z0-9\-_./]+(?::[a-z0-9\-_.]+)?$')
for step_name, step_config in steps.items():
image = step_config.get('image', '')
if image:
# 检查镜像格式
if image.startswith('*'):
# YAML anchor,跳过
print(f" ️ 步骤 '{step_name}' 使用 YAML anchor: {image}")
elif image_pattern.match(image):
print(f" ✅ 步骤 '{step_name}' 镜像格式正确: {image}")
else:
print(f" ⚠️ 步骤 '{step_name}' 镜像格式可能有问题: {image}")
print("\n✅ 镜像格式检查完成")
PYTHON_SCRIPT
echo ""
# 4. 检查环境变量和 secrets
echo "4️⃣ 检查环境变量和 secrets"
echo "----------------------------------------"
python3 << 'PYTHON_SCRIPT'
import yaml
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
secrets_used = set()
for step_name, step_config in steps.items():
env = step_config.get('environment', {})
if isinstance(env, dict):
for key, value in env.items():
if isinstance(value, dict) and 'from_secret' in value:
secret_name = value['from_secret']
secrets_used.add(secret_name)
print(f" 🔐 步骤 '{step_name}' 使用 secret: {secret_name}")
if secrets_used:
print(f"\n📋 需要配置的 secrets:")
for secret in sorted(secrets_used):
print(f" - {secret}")
print("\n⚠️ 请确保在 Woodpecker CI 中配置了这些 secrets")
else:
print(" ️ 没有使用 secrets")
print("\n✅ 环境变量检查完成")
PYTHON_SCRIPT
echo ""
# 5. 检查 when 条件的逻辑
echo "5️⃣ 检查 when 条件逻辑"
echo "----------------------------------------"
python3 << 'PYTHON_SCRIPT'
import yaml
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
steps_with_conditions = []
steps_without_conditions = []
for step_name, step_config in steps.items():
if 'when' in step_config:
steps_with_conditions.append(step_name)
else:
steps_without_conditions.append(step_name)
if steps_with_conditions:
print(f" ✅ 有条件执行的步骤 ({len(steps_with_conditions)}):")
for step in steps_with_conditions:
print(f" - {step}")
if steps_without_conditions:
print(f"\n ⚠️ 无条件执行的步骤 ({len(steps_without_conditions)}):")
for step in steps_without_conditions:
print(f" - {step}")
print("\n 💡 建议: 大部分步骤应该有 when 条件,避免不必要的执行")
print("\n✅ when 条件检查完成")
PYTHON_SCRIPT
echo ""
# 6. 模拟执行顺序
echo "6️⃣ 模拟执行顺序"
echo "----------------------------------------"
python3 << 'PYTHON_SCRIPT'
import yaml
with open('.woodpecker.yml', 'r') as f:
config = yaml.safe_load(f)
steps = config.get('steps', {})
print(" 📋 步骤执行顺序:")
for i, (step_name, step_config) in enumerate(steps.items(), 1):
image = step_config.get('image', 'N/A')
when = step_config.get('when', {})
conditions = []
if 'branch' in when:
branches = when['branch']
if isinstance(branches, list):
conditions.append(f"branch: {', '.join(branches)}")
else:
conditions.append(f"branch: {branches}")
if 'event' in when:
events = when['event']
if isinstance(events, list):
conditions.append(f"event: {', '.join(events)}")
else:
conditions.append(f"event: {events}")
if 'status' in when:
statuses = when['status']
if isinstance(statuses, list):
conditions.append(f"status: {', '.join(statuses)}")
else:
conditions.append(f"status: {statuses}")
condition_str = f" [{', '.join(conditions)}]" if conditions else ""
print(f" {i}. {step_name}{condition_str}")
print(f" 镜像: {image}")
print("\n✅ 执行顺序分析完成")
PYTHON_SCRIPT
echo ""
echo "=========================================="
echo "✅ 所有检查完成!"
echo "=========================================="
echo ""
echo "💡 提示:"
echo " - 如果所有检查都通过,配置文件基本正确"
echo " - 建议使用 Woodpecker CLI 进行本地测试:"
echo " woodpecker-cli exec .woodpecker.yml"
echo " - 或者使用 Docker:"
echo " docker run --rm -v \$(pwd):/woodpecker/src -w /woodpecker/src woodpeckerci/woodpecker-cli:latest exec .woodpecker.yml"
echo ""