diff --git a/scripts/README.md b/scripts/README.md index 5cfe442..7cab4b6 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -1,264 +1,87 @@ -# Woodpecker CI 本地测试工具 +# Scripts 目录 -本目录包含用于本地测试和验证 Woodpecker CI 配置的工具。 +本目录包含项目的所有脚本文件,按功能分类整理。 -## 📁 文件说明 +## 目录结构 -### 1. `validate-woodpecker.sh` - 配置验证工具 - -**功能**:全面验证 `.woodpecker.yml` 配置文件的正确性 - -**检查项目**: -- ✅ YAML 语法检查 -- ✅ 必需字段检查(steps, image, commands) -- ✅ 镜像格式验证 -- ✅ 环境变量和 secrets 检查 -- ✅ when 条件逻辑分析 -- ✅ 执行顺序模拟 - -**使用方法**: -```bash -./scripts/validate-woodpecker.sh +``` +scripts/ +├── deployment/ # 部署相关脚本 +├── monitoring/ # 监控相关脚本 +├── diagnosis/ # 诊断相关脚本 +├── security/ # 安全相关脚本 +├── maintenance/ # 维护相关脚本 +└── tools/ # 工具脚本 ``` -**输出示例**: -``` -========================================== -Woodpecker CI 配置本地验证工具 -========================================== +## 脚本分类 -✅ 文件存在: .woodpecker.yml +### 部署脚本 (deployment/) -1️⃣ YAML 语法检查 ----------------------------------------- -✅ YAML 语法正确 +- `deploy.sh` - 项目部署脚本 +- `setup-ssl.sh` - SSL 证书配置脚本 -2️⃣ 检查必需字段 ----------------------------------------- - ✅ 步骤 'lint' 有分支条件: ['feature/**', 'dev', 'release', 'release/**'] - ✅ 步骤 'lint' 有事件条件: ['push', 'pull_request'] - ... -``` +### 监控脚本 (monitoring/) -### 2. `test-step.sh` - 单步测试工具 +- `monitor-pipeline.sh` - CI/CD 流水线监控 +- `monitor-pipeline-continuous.sh` - 持续监控脚本 +- `monitor-pipeline-32.sh` - 流水线监控(32位系统) +- `ralph-auto-monitor.sh` - Ralph 自动监控 +- `ralph-loop.sh` - Ralph 循环监控 +- `ralph-loop.py` - Ralph 循环监控(Python 版本) -**功能**:在本地 Docker 环境中测试单个 pipeline 步骤 +### 诊断脚本 (diagnosis/) -**使用方法**: -```bash -# 查看可用步骤 -./scripts/test-step.sh +- `diagnose-webhook-detail.sh` - Webhook 详细诊断 +- `diagnose-woodpecker.py` - Woodpecker CI 诊断 +- `diagnose-auto-trigger.py` - 自动触发诊断 +- `diagnose-cicd-issues.sh` - CI/CD 问题诊断 -# Dry-run 模式(仅显示配置,不执行) -./scripts/test-step.sh notify-wechat-success --dry-run +### 工具脚本 (tools/) -# 实际执行步骤 -./scripts/test-step.sh lint -``` +- `test-wechat-notify-*.sh` - 微信通知测试脚本(多个版本) +- `test-webhook-headers.sh` - Webhook 头部测试 +- `test-woodpecker-config.py` - Woodpecker 配置测试 +- `test-branch-matching.py` - 分支匹配测试 +- `test-scenarios.py` - 场景测试 +- `update-jenkins-nginx.sh` - Jenkins Nginx 更新 +- `fix-jenkins-nginx.sh` - Jenkins Nginx 修复 +- `capture-webhook.sh` - Webhook 捕获 +- `analyze-best-practices.py` - 最佳实践分析 +- `check-job-triggers.groovy` - Jenkins 任务触发检查 +- `check-woodpecker-logs.sh` - Woodpecker 日志检查 -**特性**: -- 🔍 自动解析步骤配置 -- 🐳 使用 Docker 隔离环境 -- 🔐 模拟 Woodpecker CI 环境变量 -- 📝 显示详细执行信息 +## 使用说明 -### 3. `test-woodpecker-local.sh` - 本地测试指南 +### 运行脚本 -**功能**:显示 Woodpecker CI 本地测试的方法和命令 - -**使用方法**: -```bash -./scripts/test-woodpecker-local.sh -``` - -## 🚀 快速开始 - -### 1. 验证配置文件 - -在提交代码前,先运行验证工具: +大多数脚本可以直接运行: ```bash -./scripts/validate-woodpecker.sh +# 部署脚本 +bash scripts/deployment/deploy.sh + +# 监控脚本 +bash scripts/monitoring/monitor-pipeline.sh + +# 诊断脚本 +python scripts/diagnosis/diagnose-woodpecker.py ``` -如果所有检查都通过,说明配置文件基本正确。 +### 注意事项 -### 2. 测试单个步骤 +1. **权限问题**:某些脚本可能需要 root 权限或特定用户权限 +2. **环境变量**:部分脚本依赖环境变量,请确保正确配置 +3. **依赖工具**:某些脚本依赖特定工具(如 jq、curl、python 等),请确保已安装 -如果某个步骤有问题,可以使用单步测试工具: +## 维护说明 -```bash -# 先 dry-run 查看配置 -./scripts/test-step.sh --dry-run +- **添加新脚本**:请根据脚本功能放入对应的子目录 +- **更新脚本**:请在脚本头部添加更新说明和版本信息 +- **删除脚本**:请确保脚本不再使用后再删除 -# 确认无误后执行 -./scripts/test-step.sh -``` +## 相关文档 -### 3. 使用 Woodpecker CLI(推荐) - -安装 Woodpecker CLI: - -```bash -# macOS -brew install woodpecker-cli - -# Linux -curl -L https://github.com/woodpecker-ci/woodpecker/releases/latest/download/woodpecker-cli-linux-amd64 -o /usr/local/bin/woodpecker-cli -chmod +x /usr/local/bin/woodpecker-cli -``` - -本地运行整个 pipeline: - -```bash -woodpecker-cli exec .woodpecker.yml -``` - -### 4. 使用 Docker 模拟 - -如果没有安装 Woodpecker CLI,可以使用 Docker: - -```bash -docker run --rm \ - -v $(pwd):/woodpecker/src \ - -w /woodpecker/src \ - woodpeckerci/woodpecker-cli:latest \ - exec .woodpecker.yml -``` - -## 🔧 高级用法 - -### 测试特定分支的步骤 - -设置环境变量模拟特定分支: - -```bash -export CI_COMMIT_BRANCH="release/v1.0.0" -./scripts/test-step.sh notify-wechat-success -``` - -### 测试 secrets - -**注意**:本地测试无法访问 Woodpecker CI 中的 secrets。 - -解决方案: -1. 创建 `.env` 文件存储测试用的 secrets(**不要提交到 git**) -2. 在测试时手动设置环境变量: - -```bash -export WECHAT_WEBHOOK="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=YOUR_KEY" -./scripts/test-step.sh notify-wechat-success -``` - -### 调试环境变量 - -查看步骤会接收到哪些环境变量: - -```bash -./scripts/test-step.sh --dry-run | grep "环境变量" -``` - -## 📋 最佳实践 - -### 1. 提交前验证 - -在每次修改 `.woodpecker.yml` 后,运行: - -```bash -./scripts/validate-woodpecker.sh -``` - -### 2. 逐步测试 - -不要一次性测试整个 pipeline,而是: - -1. 先验证配置文件 -2. 再测试单个步骤 -3. 最后测试整个 pipeline - -### 3. 使用版本控制 - -将测试脚本纳入版本控制: - -```bash -git add scripts/ -git commit -m "feat: 添加 Woodpecker CI 本地测试工具" -``` - -### 4. 持续改进 - -发现新的测试需求时,更新测试脚本: - -```bash -# 编辑验证脚本 -vim scripts/validate-woodpecker.sh - -# 添加新的检查项 -``` - -## 🐛 常见问题 - -### Q1: 为什么本地测试成功,但 CI 中失败? - -**可能原因**: -1. 环境变量不同(检查 secrets) -2. 网络访问限制 -3. 文件权限问题 -4. Docker 镜像版本不一致 - -**解决方法**: -```bash -# 对比环境变量 -./scripts/test-step.sh --dry-run - -# 检查 CI 日志中的环境变量 -# 在 CI 中添加调试命令 -commands: - - env | sort - - echo "Branch: $CI_COMMIT_BRANCH" -``` - -### Q2: 如何测试需要 secrets 的步骤? - -**方法 1**:使用测试用的 secrets -```bash -export WECHAT_WEBHOOK="https://test.example.com/webhook" -./scripts/test-step.sh notify-wechat-success -``` - -**方法 2**:跳过 secrets 检查 -```bash -# 修改步骤配置,使用环境变量而不是 from_secret -``` - -### Q3: 如何测试 when 条件? - -**方法**:设置相应的环境变量 -```bash -# 测试 release 分支的步骤 -export CI_COMMIT_BRANCH="release/v1.0.0" -./scripts/test-step.sh deploy-production --dry-run - -# 测试 feature 分支的步骤 -export CI_COMMIT_BRANCH="feature/new-feature" -./scripts/test-step.sh e2e-smoke --dry-run -``` - -## 📚 相关资源 - -- [Woodpecker CI 官方文档](https://woodpecker-ci.org/docs/intro) -- [Woodpecker CLI 文档](https://woodpecker-ci.org/docs/cli) -- [Woodpecker 配置参考](https://woodpecker-ci.org/docs/usage/pipeline-syntax) - -## 🤝 贡献 - -如果你发现新的测试需求或改进点,欢迎更新这些脚本: - -1. Fork 项目 -2. 创建特性分支 -3. 提交改进 -4. 创建 Pull Request - -## 📄 许可证 - -这些测试工具遵循项目的主许可证。 +- [部署文档](../docs/deployment/) +- [监控文档](../docs/guides/monitoring.md) +- [CI/CD 文档](../docs/guides/ci-cd.md) diff --git a/scripts/deployment/deploy.sh b/scripts/deployment/deploy.sh new file mode 100755 index 0000000..b178e34 --- /dev/null +++ b/scripts/deployment/deploy.sh @@ -0,0 +1,202 @@ +#!/bin/bash + +set -e + +SERVER_IP="139.155.109.62" +SERVER_USER="root" +DEPLOY_ROOT="/home/novalon/docker-app" +PROJECT_NAME="novalon-website" +PROJECT_DIR="$DEPLOY_ROOT/$PROJECT_NAME" +CONTAINER_NAME="novalon-website" +NGINX_CONTAINER_NAME="novalon-nginx" +VERSION="1.0.0" + +while getopts "i:u:p:c:v:h" opt; do + case $opt in + i) SERVER_IP="$OPTARG" ;; + u) SERVER_USER="$OPTARG" ;; + p) PROJECT_NAME="$OPTARG" ;; + c) CONTAINER_NAME="$OPTARG" ;; + v) VERSION="$OPTARG" ;; + h) + echo "用法: $0 [选项]" + echo "选项:" + echo " -i IP地址 服务器IP地址 (默认: 139.155.109.62)" + echo " -u 用户名 SSH用户名 (默认: root)" + echo " -p 项目名 项目名称 (默认: novalon-website)" + echo " -c 容器名 容器名称 (默认: novalon-website)" + echo " -v 版本号 版本号 (默认: 1.0.0)" + echo " -h 显示帮助信息" + exit 0 + ;; + \?) + echo "无效选项: -$OPTARG" >&2 + exit 1 + ;; + esac +done + +PROJECT_DIR="$DEPLOY_ROOT/$PROJECT_NAME" + +LOG_DIR="./logs" +LOG_FILE="$LOG_DIR/deploy_$(date +%Y%m%d_%H%M%S).log" + +mkdir -p "$LOG_DIR" +exec > >(tee -a "$LOG_FILE") 2>&1 + +echo "🚀 开始部署Novalon网站到服务器 $SERVER_IP" +echo "📁 部署根目录: $DEPLOY_ROOT" +echo "📁 项目目录: $PROJECT_DIR" +echo "🐳 容器名称: $CONTAINER_NAME" +echo "📦 版本号: $VERSION" +echo "📋 部署日志: $LOG_FILE" +echo "" + +echo "📋 步骤0: 部署前检查..." + +for file in docker-compose.yml Dockerfile nginx.conf .env.example setup-ssl.sh; do + if [ ! -f "$file" ]; then + echo "❌ 缺少必要文件: $file" + exit 1 + fi +done + +if ! command -v docker-compose &> /dev/null; then + echo "❌ 本地docker-compose不可用" + exit 1 +fi + +echo "✅ 部署前检查通过" + +echo "" +echo "📋 步骤1: 验证SSH连接..." +if ! ssh -o ConnectTimeout=5 "$SERVER_USER@$SERVER_IP" exit; then + echo "❌ 无法连接到服务器 $SERVER_IP" + exit 1 +fi + +if ! ssh "$SERVER_USER@$SERVER_IP" "docker --version"; then + echo "❌ 服务器上Docker不可用" + exit 1 +fi + +echo "✅ SSH连接验证成功" + +echo "" +echo "📋 步骤2: 上传部署文件..." +ssh "$SERVER_USER@$SERVER_IP" "mkdir -p '$PROJECT_DIR'" +scp -r docker-compose.yml Dockerfile nginx.conf .env.example setup-ssl.sh "$SERVER_USER@$SERVER_IP:$PROJECT_DIR/" +echo "✅ 部署文件已上传" + +echo "" +echo "📋 步骤3: 在服务器上执行部署..." +ssh "$SERVER_USER@$SERVER_IP" << ENDSSH +cd '$PROJECT_DIR' + +echo "🔒 配置SSL证书..." +chmod +x setup-ssl.sh +./setup-ssl.sh + +echo "📋 检查环境变量文件..." +if [ ! -f .env ]; then + echo "📝 创建.env文件..." + cp .env.example .env + echo "⚠️ 请编辑.env文件,填入正确的环境变量" + echo "⚠️ 必须配置: DATABASE_URL, NEXTAUTH_SECRET, NEXTAUTH_URL, RESEND_API_KEY, OPS_ALERT_EMAIL" + exit 1 +fi + +echo "🐳 启动Docker容器..." +docker-compose down +docker-compose pull +docker-compose up -d + +echo "📋 等待服务启动..." +timeout=60 +elapsed=0 +check_interval=3 + +while [ $elapsed -lt $timeout ]; do + if docker inspect --format='{{.State.Status}}' "$CONTAINER_NAME" 2>/dev/null | grep -q "running"; then + if curl -f -s -o /dev/null "http://localhost:3000" --max-time 5 2>/dev/null; then + echo "✅ 服务已启动并响应正常" + break + else + echo "⏳ 容器运行中,等待服务响应..." + fi + else + echo "⏳ 等待容器启动..." + fi + sleep $check_interval + elapsed=$((elapsed + check_interval)) +done + +if [ $elapsed -ge $timeout ]; then + echo "❌ 服务启动超时" + echo "📋 当前容器状态:" + docker ps -a | grep "$CONTAINER_NAME" + echo "📋 容器日志:" + docker logs "$CONTAINER_NAME" --tail 20 + exit 1 +fi + +echo "📋 检查容器状态..." +docker ps | grep "$CONTAINER_NAME" + +echo "📋 检查容器日志..." +if docker logs "$CONTAINER_NAME" --tail 50 2>/dev/null; then + echo "✅ 容器日志检查完成" +else + echo "⚠️ 容器日志为空或无法访问" +fi + +echo "📋 配置SSL证书自动续期..." +if ! crontab -l | grep -q "certbot renew"; then + if ! (crontab -l 2>/dev/null; echo "0 0,12 * * * certbot renew --quiet --post-hook 'docker restart $NGINX_CONTAINER_NAME'") | crontab -; then + echo "❌ SSL证书自动续期任务配置失败" + exit 1 + fi + echo "✅ SSL证书自动续期任务已配置" +else + echo "✅ SSL证书自动续期任务已存在" +fi + +echo "✅ 部署完成!" +ENDSSH + +echo "" +echo "📋 步骤4: 部署后验证..." + +if ! ssh "$SERVER_USER@$SERVER_IP" "docker ps | grep -q '$CONTAINER_NAME'"; then + echo "❌ 容器未运行" + exit 1 +fi + +if ! curl -f -s -o /dev/null "http://$SERVER_IP" --max-time 10; then + echo "⚠️ HTTP服务响应异常" +else + echo "✅ HTTP服务正常" +fi + +if ! curl -f -s -o /dev/null "https://$SERVER_IP" --max-time 10; then + echo "⚠️ HTTPS服务响应异常" +else + echo "✅ HTTPS服务正常" +fi + +echo "✅ 部署后验证通过" + +echo "" +echo "🎉 部署脚本执行完成!" +echo "📋 访问地址:" +echo " HTTP: http://$SERVER_IP" +echo " HTTPS: https://$SERVER_IP" +echo " 域名: https://novalon.cn" +echo "" +echo "📋 后续步骤:" +echo " 1. 验证网站可访问性" +echo " 2. 检查容器运行状态: docker ps" +echo " 3. 查看容器日志: docker logs $CONTAINER_NAME" +echo " 4. 验证HTTPS配置" +echo " 5. 测试网站主要功能" +echo " 6. 检查SSL证书自动续期: crontab -l" \ No newline at end of file diff --git a/scripts/deployment/setup-ssl.sh b/scripts/deployment/setup-ssl.sh new file mode 100755 index 0000000..3258557 --- /dev/null +++ b/scripts/deployment/setup-ssl.sh @@ -0,0 +1,38 @@ +#!/bin/bash + +SSL_DIR="./ssl" +CERTBOT_DIR="/var/www/certbot" +DOMAIN="novalon.cn" + +mkdir -p "$SSL_DIR" +mkdir -p "$CERTBOT_DIR" + +echo "🔒 开始配置SSL证书..." + +if [ ! -f "$SSL_DIR/fullchain.pem" ] || [ ! -f "$SSL_DIR/privkey.pem" ]; then + echo "📝 SSL证书不存在,需要手动配置Let's Encrypt证书" + echo "📋 请按照以下步骤操作:" + echo "1. 在服务器上安装certbot:" + echo " sudo apt-get update" + echo " sudo apt-get install certbot" + echo "" + echo "2. 获取SSL证书:" + echo " sudo certbot certonly --webroot -w $CERTBOT_DIR -d $DOMAIN -d www.$DOMAIN" + echo "" + echo "3. 复制证书文件到SSL目录:" + echo " sudo cp /etc/letsencrypt/live/$DOMAIN/fullchain.pem $SSL_DIR/" + echo " sudo cp /etc/letsencrypt/live/$DOMAIN/privkey.pem $SSL_DIR/" + echo "" + echo "4. 设置证书文件权限:" + echo " sudo chmod 644 $SSL_DIR/fullchain.pem" + echo " sudo chmod 600 $SSL_DIR/privkey.pem" + echo "" + echo "5. 配置自动续期:" + echo " 添加cron任务: 0 0,12 * * * certbot renew --quiet" +else + echo "✅ SSL证书已存在" + echo "📋 证书信息:" + ls -lh "$SSL_DIR" +fi + +echo "🎉 SSL证书配置完成!" \ No newline at end of file diff --git a/scripts/diagnosis/diagnose-auto-trigger.py b/scripts/diagnosis/diagnose-auto-trigger.py new file mode 100644 index 0000000..33c9e19 --- /dev/null +++ b/scripts/diagnosis/diagnose-auto-trigger.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI 自动触发诊断工具 +排查 CI 无法自动触发的可能原因 +""" + +import yaml +from pathlib import Path + + +def diagnose_auto_trigger(config_path): + """诊断自动触发问题""" + + with open(config_path, 'r', encoding='utf-8') as f: + config = yaml.safe_load(f) + + print("="*70) + print("Woodpecker CI 自动触发诊断") + print("="*70) + + print("\n🔍 可能导致 CI 无法自动触发的原因:") + print("-"*70) + + reasons = [ + { + "原因": "1. Webhook 未配置或配置错误", + "检查": "Git 仓库设置 → Webhooks → 确认有指向 Woodpecker CI 的 Webhook", + "解决": "添加 Webhook,URL 格式: http://woodpecker-server/hook" + }, + { + "原因": "2. Woodpecker CI 仓库未激活", + "检查": "Woodpecker CI Web 界面 → 确认仓库已激活", + "解决": "在 Woodpecker CI 中激活仓库" + }, + { + "原因": "3. 分支保护或限制", + "检查": "Woodpecker CI 仓库设置 → 查看 'Trusted' 和 'Protected' 设置", + "解决": "取消分支保护或添加受信任的分支" + }, + { + "原因": "4. 配置文件语法错误", + "检查": "使用 yamllint 或在线 YAML 验证器检查配置文件", + "解决": "修复 YAML 语法错误" + }, + { + "原因": "5. when 条件过于严格", + "检查": "检查配置文件中的 when 条件", + "解决": "确保 when 条件包含正确的分支和事件" + }, + { + "原因": "6. Woodpecker CI 全局配置限制", + "检查": "检查 Woodpecker CI 的全局配置文件", + "解决": "修改全局配置,允许自动触发" + }, + { + "原因": "7. Git 仓库权限问题", + "检查": "确认 Woodpecker CI 有访问仓库的权限", + "解决": "重新授权 Woodpecker CI 访问仓库" + }, + { + "原因": "8. 提交信息包含跳过关键词", + "检查": "检查提交信息是否包含 [skip ci], [ci skip] 等", + "解决": "避免在提交信息中使用跳过关键词" + } + ] + + for i, reason in enumerate(reasons, 1): + print(f"\n{reason['原因']}") + print(f" 检查: {reason['检查']}") + print(f" 解决: {reason['解决']}") + + # 检查配置文件中的 when 条件 + print("\n\n📋 当前配置的 when 条件:") + print("-"*70) + + for step_name, step_config in config.get('steps', {}).items(): + if not isinstance(step_config, dict): + continue + + when = step_config.get('when', {}) + if not when: + continue + + if isinstance(when, dict): + events = when.get('event', []) + branches = when.get('branch', []) + if events or branches: + print(f"\n步骤: {step_name}") + if events: + print(f" 事件: {events}") + if branches: + print(f" 分支: {branches}") + elif isinstance(when, list): + print(f"\n步骤: {step_name}") + for condition in when: + if isinstance(condition, dict): + events = condition.get('event', []) + branches = condition.get('branch', []) + if events: + print(f" 事件: {events}") + if branches: + print(f" 分支: {branches}") + + print("\n\n💡 快速排查步骤:") + print("="*70) + print("1. 访问 Git 仓库设置 → Webhooks") + print(" - 确认有 Woodpecker CI 的 Webhook") + print(" - 查看 'Recent Deliveries' 是否有发送记录") + print("\n2. 访问 Woodpecker CI Web 界面") + print(" - 确认仓库已激活") + print(" - 检查仓库设置中的 'Trusted' 选项") + print("\n3. 查看提交记录") + print(" - 确认提交信息不包含 [skip ci] 等关键词") + print("\n4. 手动触发测试") + print(" - 在 Woodpecker CI 中手动触发 Pipeline") + print(" - 观察是否能够正常执行") + + print("\n" + "="*70) + + +if __name__ == "__main__": + diagnose_auto_trigger(".woodpecker.yml") diff --git a/scripts/diagnosis/diagnose-cicd-issues.sh b/scripts/diagnosis/diagnose-cicd-issues.sh new file mode 100755 index 0000000..5ac3a67 --- /dev/null +++ b/scripts/diagnosis/diagnose-cicd-issues.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +echo "==========================================" +echo "CI/CD 问题诊断脚本" +echo "==========================================" +echo "" + +echo "📋 问题1: Git LFS 配置检查" +echo "----------------------------------------" +if command -v git-lfs &> /dev/null; then + echo "✅ Git LFS 已安装" + git lfs version +else + echo "❌ Git LFS 未安装" +fi + +if [ -f ".gitattributes" ]; then + echo "✅ .gitattributes 文件存在" + cat .gitattributes +else + echo "❌ .gitattributes 文件不存在(项目未使用LFS)" +fi + +echo "" +echo "📋 问题2: 环境变量检查" +echo "----------------------------------------" +echo "当前环境变量:" +echo " CI_COMMIT_BRANCH: ${CI_COMMIT_BRANCH:-未设置}" +echo " CI_COMMIT_SHA: ${CI_COMMIT_SHA:-未设置}" +echo " CI_COMMIT_MESSAGE: ${CI_COMMIT_MESSAGE:-未设置}" +echo " CI_COMMIT_AUTHOR: ${CI_COMMIT_AUTHOR:-未设置}" +echo " CI_PIPELINE_NUMBER: ${CI_PIPELINE_NUMBER:-未设置}" +echo " CI_REPO_ID: ${CI_REPO_ID:-未设置}" + +echo "" +echo "📋 问题3: Woodpecker CI 配置验证" +echo "----------------------------------------" +if command -v python3 &> /dev/null; then + echo "运行 Python 诊断脚本..." + python3 diagnose-woodpecker.py 2>/dev/null || echo "诊断脚本执行失败" +else + echo "⚠️ Python3 未安装,跳过配置验证" +fi + +echo "" +echo "==========================================" +echo "诊断完成" +echo "==========================================" diff --git a/scripts/diagnosis/diagnose-webhook-detail.sh b/scripts/diagnosis/diagnose-webhook-detail.sh new file mode 100644 index 0000000..683aabb --- /dev/null +++ b/scripts/diagnosis/diagnose-webhook-detail.sh @@ -0,0 +1,37 @@ +#!/bin/bash + +echo "=== Woodpecker CI Webhook 诊断 ===" +echo "" + +echo "1. 检查 Forgejo Webhook 配置..." +echo " Webhook URL: https://ci.f.novalon.cn/api/hook?access_token=..." +echo " Content Type: application/json" +echo " Trigger: push" +echo "" + +echo "2. 检查 Woodpecker CI 期望的 Header..." +echo " X-Gitea-Event: push" +echo " X-Gitea-Delivery: " +echo " X-Gitea-Signature: " +echo "" + +echo "3. 检查 Nginx 配置..." +docker exec novalon-nginx cat /etc/nginx/conf.d/ci.f.novalon.cn.conf | grep -A 15 "location /api/" +echo "" + +echo "4. 测试 Webhook 接收..." +echo " 发送测试 webhook..." +curl -X POST \ + -H "Content-Type: application/json" \ + -H "X-Gitea-Event: push" \ + -H "X-Gitea-Delivery: test-123" \ + -d '{"ref":"refs/heads/test"}' \ + "https://ci.f.novalon.cn/api/hook?access_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJmb3JnZS1pZCI6IjEiLCJyZXBvLWZvcmdlLXJlbW90ZS1pZCI6IjEiLCJ0eXBlIjoiaG9vayJ9.gu3mi1VAQfGB3d9HcuwWmMAcf-0BmmvQyGjqdiC20dA" \ + -v 2>&1 | grep -E "(< HTTP|X-Gitea|hook)" +echo "" + +echo "5. 检查 Woodpecker CI 日志..." +docker logs woodpecker-server --since 10s 2>&1 | grep -E "(hook|event|push)" +echo "" + +echo "=== 诊断完成 ===" diff --git a/scripts/diagnosis/diagnose-woodpecker.py b/scripts/diagnosis/diagnose-woodpecker.py new file mode 100644 index 0000000..272b245 --- /dev/null +++ b/scripts/diagnosis/diagnose-woodpecker.py @@ -0,0 +1,137 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI 配置诊断工具 +检查配置文件中可能导致 CI 未触发的问题 +""" + +import yaml +from pathlib import Path + + +def diagnose_woodpecker_config(config_path): + """诊断 Woodpecker CI 配置""" + + with open(config_path, 'r', encoding='utf-8') as f: + config = yaml.safe_load(f) + + print("="*70) + print("Woodpecker CI 配置诊断报告") + print("="*70) + + issues = [] + warnings = [] + + # 检查是否有 steps + if 'steps' not in config: + issues.append("❌ 缺少 'steps' 配置") + else: + print(f"\n✅ 找到 {len(config['steps'])} 个步骤") + + # 检查每个步骤的 when 条件 + print("\n📋 步骤触发条件检查:") + print("-" * 70) + + for step_name, step_config in config.get('steps', {}).items(): + if not isinstance(step_config, dict): + continue + + when = step_config.get('when', {}) + + if not when: + warnings.append(f"⚠️ 步骤 '{step_name}' 没有 when 条件,将始终执行") + continue + + # 检查 when 条件的格式 + if isinstance(when, list): + print(f"\n步骤: {step_name}") + print(f" when 条件格式: 列表(多个条件)") + for i, condition in enumerate(when): + if isinstance(condition, dict): + events = condition.get('event', []) + branches = condition.get('branch', []) + print(f" 条件 {i+1}:") + print(f" 事件: {events}") + print(f" 分支: {branches}") + elif isinstance(when, dict): + print(f"\n步骤: {step_name}") + print(f" when 条件格式: 字典(单个条件)") + events = when.get('event', []) + branches = when.get('branch', []) + print(f" 事件: {events}") + print(f" 分支: {branches}") + + # 检查可能导致问题的配置 + print("\n\n🔍 潜在问题分析:") + print("-" * 70) + + # 检查是否有 skip_clone + if config.get('skip_clone'): + warnings.append("⚠️ skip_clone 设置为 true,可能影响代码获取") + + # 检查 clone 配置 + clone_config = config.get('clone', {}) + if clone_config: + print(f"\nClone 配置: {clone_config}") + + # 检查 services + services = config.get('services', {}) + if services: + print(f"\n服务配置: {list(services.keys())}") + + # 检查 workspace + workspace = config.get('workspace', {}) + if workspace: + print(f"\n工作区配置: {workspace}") + + # 输出问题 + print("\n\n📊 诊断结果:") + print("="*70) + + if issues: + print("\n❌ 发现的问题:") + for issue in issues: + print(f" {issue}") + + if warnings: + print("\n⚠️ 警告:") + for warning in warnings: + print(f" {warning}") + + if not issues and not warnings: + print("\n✅ 配置文件语法正确,未发现明显问题") + + # 输出可能的原因 + print("\n\n🔍 CI 未触发的可能原因:") + print("-" * 70) + possible_reasons = [ + "1. Woodpecker CI 的 Webhook 未正确配置", + "2. Git 仓库设置中禁用了该分支的 CI 触发", + "3. Woodpecker CI 服务器未运行或配置错误", + "4. 配置文件中的分支匹配规则与 Woodpecker CI 版本不兼容", + "5. 需要在 Woodpecker CI 界面手动激活该仓库", + "6. Woodpecker CI 的全局配置限制了某些分支", + "7. 推送的提交信息触发了 CI 跳过(如包含 [skip ci])", + ] + + for reason in possible_reasons: + print(f" {reason}") + + print("\n\n💡 建议的排查步骤:") + print("-" * 70) + suggestions = [ + "1. 检查 Woodpecker CI Web 界面,确认仓库已激活", + "2. 检查 Git 仓库的 Webhook 设置", + "3. 查看 Woodpecker CI 的日志", + "4. 尝试手动触发 CI(如果支持)", + "5. 检查 Woodpecker CI 的全局配置", + "6. 创建一个简单的测试分支验证配置", + ] + + for suggestion in suggestions: + print(f" {suggestion}") + + print("\n" + "="*70) + + +if __name__ == "__main__": + diagnose_woodpecker_config(".woodpecker.yml") diff --git a/scripts/monitoring/monitor-pipeline-32.sh b/scripts/monitoring/monitor-pipeline-32.sh new file mode 100755 index 0000000..eeede7b --- /dev/null +++ b/scripts/monitoring/monitor-pipeline-32.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline/32" +COMMIT_SHA="bf35020" + +echo "==========================================" +echo "Pipeline #32 监控" +echo "==========================================" +echo "" +echo "Pipeline URL: $PIPELINE_URL" +echo "Commit SHA: $COMMIT_SHA" +echo "" +echo "请在浏览器中打开以下链接查看Pipeline状态:" +echo "$PIPELINE_URL" +echo "" +echo "关键检查点:" +echo " 1. ✅ Clone步骤(Git LFS已禁用)" +echo " 2. ⏳ Lint检查" +echo " 3. ⏳ Type检查" +echo " 4. ⏳ 单元测试(覆盖率阈值已调整)" +echo " 5. ⏳ 构建步骤" +echo " 6. ⏳ 企业微信通知" +echo "" +echo "等待Pipeline执行完成..." diff --git a/scripts/monitoring/monitor-pipeline-continuous.sh b/scripts/monitoring/monitor-pipeline-continuous.sh new file mode 100755 index 0000000..e90c575 --- /dev/null +++ b/scripts/monitoring/monitor-pipeline-continuous.sh @@ -0,0 +1,73 @@ +#!/bin/bash + +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline/33" +COMMIT_SHA="232f481" +MAX_CHECKS=20 +CHECK_INTERVAL=30 + +echo "==========================================" +echo "Ralph Loop 持续监控模式" +echo "==========================================" +echo "" +echo "Pipeline URL: $PIPELINE_URL" +echo "Commit SHA: $COMMIT_SHA" +echo "最大检查次数: $MAX_CHECKS" +echo "检查间隔: ${CHECK_INTERVAL}秒" +echo "" +echo "开始监控..." +echo "" + +for i in $(seq 1 $MAX_CHECKS); do + echo "==========================================" + echo "检查 #$i / $MAX_CHECKS" + echo "时间: $(date '+%Y-%m-%d %H:%M:%S')" + echo "==========================================" + echo "" + + echo "请检查Pipeline状态:" + echo " $PIPELINE_URL" + echo "" + + echo "输入状态 (pass/fail/running/quit):" + read -t $CHECK_INTERVAL status || status="running" + + case $status in + pass) + echo "" + echo "✅ Pipeline已通过!" + echo "Ralph Loop完成。" + exit 0 + ;; + fail) + echo "" + echo "❌ Pipeline失败!" + echo "请输入失败的步骤名称:" + read step_name + echo "失败步骤: $step_name" + echo "" + echo "Ralph Loop将自动修复..." + exit 1 + ;; + running) + echo "" + echo "⏳ Pipeline仍在运行,等待${CHECK_INTERVAL}秒后继续检查..." + sleep $CHECK_INTERVAL + ;; + quit) + echo "" + echo "⚠️ 用户退出监控" + exit 2 + ;; + *) + echo "" + echo "⚠️ 无效状态: $status" + echo "继续监控..." + sleep $CHECK_INTERVAL + ;; + esac +done + +echo "" +echo "⚠️ 达到最大检查次数 ($MAX_CHECKS)" +echo "Pipeline仍在运行,请手动检查" +exit 3 diff --git a/scripts/monitoring/monitor-pipeline.sh b/scripts/monitoring/monitor-pipeline.sh new file mode 100755 index 0000000..1f3648d --- /dev/null +++ b/scripts/monitoring/monitor-pipeline.sh @@ -0,0 +1,84 @@ +#!/bin/bash + +echo "==========================================" +echo "CI/CD Pipeline 实时监控" +echo "==========================================" +echo "" + +COMMIT_SHA="34ce9fb" +BRANCH="release/v1.0.0" +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline" + +echo "📋 提交信息:" +echo " SHA: $COMMIT_SHA" +echo " 分支: $BRANCH" +echo " 提交信息: fix: 修复CI/CD流程问题并建立监控机制" +echo "" + +echo "🔗 CI/CD 监控链接:" +echo " $PIPELINE_URL" +echo "" + +echo "📊 预期执行步骤(release/v1.0.0 分支):" +echo " 1. ✅ lint - 代码检查" +echo " 2. ✅ type-check - 类型检查" +echo " 3. ⚠️ security-scan - 安全扫描(允许失败)" +echo " 4. ✅ unit-tests - 单元测试" +echo " 5. ✅ e2e-standard - E2E标准测试" +echo " 6. ✅ e2e-deep - E2E深度测试" +echo " 7. ✅ e2e-performance - 性能测试" +echo " 8. ✅ e2e-accessibility - 无障碍测试" +echo " 9. ✅ e2e-visual - 视觉测试" +echo " 10. ✅ build-image - 构建Docker镜像" +echo " 11. ✅ deploy-production - 部署到生产环境" +echo " 12. ✅ archive-to-main - 归档到main分支" +echo " 13. ✅ notify-wechat-success - 企业微信通知(成功)" +echo " 或 notify-wechat-failure - 企业微信通知(失败)" +echo "" + +echo "🔍 关键验证点:" +echo "" +echo " ✅ Git LFS 禁用验证:" +echo " - Clone步骤不应出现 'git lfs fetch'" +echo " - Clone步骤不应出现 'git lfs checkout'" +echo "" + +echo " ✅ 企业微信通知验证:" +echo " - 环境变量应正确展开" +echo " - 消息内容应包含实际的分支、提交、作者信息" +echo " - 不应出现变量名(如 \${BRANCH})" +echo "" + +echo " ✅ 部署验证:" +echo " - 健康检查应通过" +echo " - 不应触发回滚机制" +echo "" + +echo "==========================================" +echo "监控指南" +echo "==========================================" +echo "" +echo "1. 访问 CI/CD 界面:" +echo " $PIPELINE_URL" +echo "" +echo "2. 查看最新构建(Pipeline #30 或更新)" +echo "" +echo "3. 重点关注:" +echo " - Clone 步骤日志(验证LFS是否禁用)" +echo " - 企业微信通知步骤日志(验证变量展开)" +echo " - 部署步骤日志(验证健康检查)" +echo "" +echo "4. 验证企业微信通知:" +echo " - 检查企业微信群聊是否收到通知" +echo " - 验证通知内容是否正确显示变量值" +echo "" +echo "5. 如有问题,运行诊断脚本:" +echo " ./diagnose-cicd-issues.sh" +echo "" + +echo "==========================================" +echo "等待 CI/CD 执行..." +echo "==========================================" +echo "" +echo "💡 提示: CI/CD 通常需要 10-20 分钟完成所有步骤" +echo "" diff --git a/scripts/monitoring/ralph-auto-monitor.sh b/scripts/monitoring/ralph-auto-monitor.sh new file mode 100755 index 0000000..a72928e --- /dev/null +++ b/scripts/monitoring/ralph-auto-monitor.sh @@ -0,0 +1,83 @@ +#!/bin/bash + +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline/33" +COMMIT_SHA="232f481" +MAX_ITERATIONS=10 + +echo "==========================================" +echo "Ralph Loop 自动监控模式" +echo "==========================================" +echo "" +echo "Pipeline URL: $PIPELINE_URL" +echo "Commit SHA: $COMMIT_SHA" +echo "最大迭代次数: $MAX_ITERATIONS" +echo "" +echo "监控策略:" +echo " - 每60秒检查一次Pipeline状态" +echo " - 自动识别失败步骤" +echo " - 立即实施修复" +echo "" + +for i in $(seq 1 $MAX_ITERATIONS); do + echo "==========================================" + echo "迭代 #$i / $MAX_ITERATIONS" + echo "时间: $(date '+%Y-%m-%d %H:%M:%S')" + echo "==========================================" + echo "" + + echo "📋 当前Pipeline状态检查" + echo "请访问: $PIPELINE_URL" + echo "" + + echo "请输入以下信息:" + echo " - 'pass': Pipeline已通过" + echo " - 'fail ': 指定失败的步骤" + echo " - 'running': 仍在运行" + echo " - 'auto': 自动检测(需要手动查看后输入)" + echo "" + + read -p "状态: " input + + if [[ $input == "pass" ]]; then + echo "" + echo "✅ Pipeline已通过!" + echo "Ralph Loop完成。" + exit 0 + elif [[ $input == fail* ]]; then + STEP_NAME=$(echo "$input" | awk '{print $2}') + echo "" + echo "❌ 失败步骤: $STEP_NAME" + echo "" + echo "🔧 Ralph Loop将自动修复..." + echo "$STEP_NAME" + exit 1 + elif [[ $input == "running" ]]; then + echo "" + echo "⏳ Pipeline仍在运行,等待60秒..." + sleep 60 + elif [[ $input == "auto" ]]; then + echo "" + echo "🤖 自动检测模式" + echo "请手动查看Pipeline页面后,输入状态或失败步骤名称" + read -p "输入: " manual_input + if [[ $manual_input == "pass" ]]; then + echo "" + echo "✅ Pipeline已通过!" + exit 0 + elif [[ $manual_input != "" ]]; then + echo "" + echo "❌ 失败步骤: $manual_input" + echo "$manual_input" + exit 1 + fi + else + echo "" + echo "⚠️ 无效输入,继续监控..." + sleep 60 + fi +done + +echo "" +echo "⚠️ 达到最大迭代次数 ($MAX_ITERATIONS)" +echo "请手动检查Pipeline状态" +exit 2 diff --git a/scripts/monitoring/ralph-loop.py b/scripts/monitoring/ralph-loop.py new file mode 100755 index 0000000..59d4265 --- /dev/null +++ b/scripts/monitoring/ralph-loop.py @@ -0,0 +1,183 @@ +#!/usr/bin/env python3 + +import subprocess +import time +import json +from pathlib import Path + +class RalphLoop: + def __init__(self, max_iterations=10): + self.max_iterations = max_iterations + self.current_iteration = 0 + self.pipeline_url = "https://ci.f.novalon.cn/repos/1/pipeline/31" + self.commit_sha = "1e10118" + self.branch = "release/v1.0.0" + + def log(self, message, level="INFO"): + timestamp = time.strftime("%Y-%m-%d %H:%M:%S") + print(f"[{timestamp}] [{level}] {message}") + + def run_command(self, cmd, check=True): + self.log(f"执行命令: {cmd}") + result = subprocess.run(cmd, shell=True, capture_output=True, text=True) + if check and result.returncode != 0: + self.log(f"命令失败: {result.stderr}", "ERROR") + return None + return result.stdout.strip() + + def check_pipeline_status(self): + self.log("="*70) + self.log(f"迭代 #{self.current_iteration} / {self.max_iterations}") + self.log("="*70) + + self.log(f"Pipeline URL: {self.pipeline_url}") + self.log(f"Commit SHA: {self.commit_sha}") + self.log(f"Branch: {self.branch}") + + self.log("\n📋 请手动检查Pipeline状态:") + self.log(f" {self.pipeline_url}") + + return input("\nPipeline状态 (pass/fail/running): ").strip().lower() + + def identify_failure(self): + self.log("\n🔍 识别失败步骤...") + self.log("请查看Pipeline页面,输入失败的步骤名称") + self.log("常见步骤:") + self.log(" - lint") + self.log(" - type-check") + self.log(" - security-scan") + self.log(" - unit-tests") + self.log(" - e2e-standard") + self.log(" - e2e-deep") + self.log(" - build-image") + self.log(" - deploy-production") + self.log(" - notify-wechat-success") + self.log(" - notify-wechat-failure") + + return input("\n失败步骤名称: ").strip() + + def analyze_failure(self, step_name): + self.log(f"\n🔬 分析失败原因: {step_name}") + + failure_patterns = { + "lint": { + "possible_causes": [ + "ESLint配置问题", + "代码格式不符合规范", + "未使用的变量或导入" + ], + "fix_commands": [ + "npm run lint -- --fix", + "npm run lint 2>&1 | head -50" + ] + }, + "type-check": { + "possible_causes": [ + "TypeScript类型错误", + "类型定义缺失", + "类型不匹配" + ], + "fix_commands": [ + "npm run type-check 2>&1 | head -50" + ] + }, + "unit-tests": { + "possible_causes": [ + "测试用例失败", + "测试覆盖率不足", + "测试环境配置问题" + ], + "fix_commands": [ + "npm run test:coverage:check 2>&1 | tail -100" + ] + }, + "notify-wechat-success": { + "possible_causes": [ + "脚本权限问题", + "环境变量未传递", + "Webhook URL错误" + ], + "fix_commands": [ + "chmod +x scripts/notify-wechat.sh", + "cat scripts/notify-wechat.sh" + ] + } + } + + if step_name in failure_patterns: + pattern = failure_patterns[step_name] + self.log("\n可能原因:") + for i, cause in enumerate(pattern["possible_causes"], 1): + self.log(f" {i}. {cause}") + + self.log("\n诊断命令:") + for cmd in pattern["fix_commands"]: + self.log(f" $ {cmd}") + else: + self.log("⚠️ 未知步骤,请手动分析") + + return input("\n输入修复描述(或'skip'跳过): ").strip() + + def implement_fix(self, step_name, fix_description): + if fix_description.lower() == 'skip': + self.log("跳过修复") + return False + + self.log(f"\n🔧 实施修复: {step_name}") + self.log(f"修复描述: {fix_description}") + + self.log("\n请执行以下操作:") + self.log(" 1. 修复代码或配置") + self.log(" 2. 测试修复效果") + self.log(" 3. 提交更改") + + input("\n修复完成后按Enter继续...") + + # 提交修复 + self.log("\n提交修复...") + commit_msg = f"fix: 修复{step_name}步骤失败\n\n{fix_description}" + self.run_command(f'git add -A') + self.run_command(f'git commit -m "{commit_msg}"') + self.run_command(f'git push origin {self.branch}') + + self.log("✅ 修复已提交并推送") + return True + + def run(self): + self.log("🚀 Ralph Loop 启动") + self.log("目标: 修复Pipeline直到通过") + self.log(f"最大迭代次数: {self.max_iterations}") + + while self.current_iteration < self.max_iterations: + self.current_iteration += 1 + + status = self.check_pipeline_status() + + if status == "pass": + self.log("\n✅ Pipeline已通过!") + self.log("Ralph Loop完成。") + return True + elif status == "running": + self.log("\n⏳ Pipeline正在运行,等待...") + time.sleep(30) + continue + elif status == "fail": + step_name = self.identify_failure() + fix_description = self.analyze_failure(step_name) + + if self.implement_fix(step_name, fix_description): + self.log("\n⏳ 等待Pipeline重新执行...") + time.sleep(10) + else: + self.log("\n⚠️ 未实施修复,继续下一次迭代") + else: + self.log(f"\n❌ 无效状态: {status}") + + self.log("\n⚠️ 达到最大迭代次数") + self.log("Pipeline仍未通过,请手动检查") + return False + +if __name__ == "__main__": + ralph = RalphLoop(max_iterations=10) + success = ralph.run() + exit(0 if success else 1) diff --git a/scripts/monitoring/ralph-loop.sh b/scripts/monitoring/ralph-loop.sh new file mode 100755 index 0000000..68bbe4b --- /dev/null +++ b/scripts/monitoring/ralph-loop.sh @@ -0,0 +1,111 @@ +#!/bin/bash + +set -e + +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline/31" +COMMIT_SHA="1e10118" +MAX_ITERATIONS=10 + +echo "==========================================" +echo "Ralph Loop: CI/CD Pipeline 自动修复" +echo "==========================================" +echo "" +echo "Pipeline URL: $PIPELINE_URL" +echo "Commit SHA: $COMMIT_SHA" +echo "Max Iterations: $MAX_ITERATIONS" +echo "" + +for i in $(seq 1 $MAX_ITERATIONS); do + echo "==========================================" + echo "迭代 #$i / $MAX_ITERATIONS" + echo "==========================================" + echo "" + + echo "📋 步骤1: 检查Pipeline状态" + echo "访问: $PIPELINE_URL" + echo "" + + echo "🔍 步骤2: 分析失败原因" + echo "请手动检查Pipeline页面,识别失败的步骤" + echo "" + + echo "💡 步骤3: 等待用户输入" + echo "请输入以下选项之一:" + echo " - 'pass': Pipeline已通过,结束循环" + echo " - 'fail ': 指定失败的步骤名称" + echo " - 'retry': 重新检查状态" + echo " - 'quit': 退出循环" + echo "" + + read -p "输入选项: " choice + + case $choice in + pass) + echo "" + echo "✅ Pipeline已通过!" + echo "Ralph Loop完成。" + exit 0 + ;; + fail*) + STEP_NAME=$(echo "$choice" | awk '{print $2}') + echo "" + echo "❌ 失败步骤: $STEP_NAME" + echo "" + echo "🔧 步骤4: 分析失败原因" + + case $STEP_NAME in + lint) + echo "Lint检查失败" + echo "可能原因:" + echo " - ESLint配置问题" + echo " - 代码格式问题" + echo "修复方案:" + echo " npm run lint -- --fix" + ;; + type-check) + echo "类型检查失败" + echo "可能原因:" + echo " - TypeScript类型错误" + echo "修复方案:" + echo " npm run type-check" + ;; + unit-tests) + echo "单元测试失败" + echo "可能原因:" + echo " - 测试用例失败" + echo " - 覆盖率不足" + echo "修复方案:" + echo " npm run test:coverage:check" + ;; + *) + echo "未知步骤: $STEP_NAME" + echo "请手动分析失败原因" + ;; + esac + + echo "" + echo "请修复问题后,提交并推送代码" + read -p "修复完成后输入 'continue' 继续: " confirm + ;; + retry) + echo "" + echo "🔄 重新检查状态..." + continue + ;; + quit) + echo "" + echo "⚠️ 用户退出循环" + exit 1 + ;; + *) + echo "" + echo "❌ 无效选项: $choice" + echo "请重新输入" + ;; + esac +done + +echo "" +echo "⚠️ 达到最大迭代次数 ($MAX_ITERATIONS)" +echo "Pipeline仍未通过,请手动检查" +exit 1 diff --git a/scripts/tools/analyze-best-practices.py b/scripts/tools/analyze-best-practices.py new file mode 100644 index 0000000..a43b107 --- /dev/null +++ b/scripts/tools/analyze-best-practices.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI 最佳实践对比分析 +对比当前配置与 Woodpecker CI 官方最佳实践 +""" + +import yaml +from pathlib import Path + + +class BestPracticeAnalyzer: + """最佳实践分析器""" + + def __init__(self, config_path: str): + self.config_path = Path(config_path) + with open(self.config_path, 'r', encoding='utf-8') as f: + self.config = yaml.safe_load(f) + + self.best_practices = { + "分支策略": { + "✅ 使用通配符": "支持 feature/**, release/** 等通配符", + "✅ 分层触发": "不同分支触发不同深度的测试", + "✅ 保护主分支": "main 分支只接收自动归档", + "⚠️ 缺少分支保护": "建议在 Git 仓库设置分支保护规则" + }, + "测试策略": { + "✅ 分层测试": "feature(dev/smoke) < dev(standard) < release(full)", + "✅ 快速反馈": "feature 分支使用 smoke test 快速验证", + "✅ 质量门禁": "测试失败阻止合并/部署", + "✅ 覆盖率检查": "单元测试包含覆盖率检查" + }, + "部署安全": { + "✅ 健康检查": "部署后执行健康检查", + "✅ 自动回滚": "健康检查失败自动回滚", + "✅ 备份机制": "部署前备份当前版本", + "✅ Secret 管理": "使用 Secret 管理敏感信息", + "✅ SSH 密钥": "使用 SSH 密钥进行 Git 操作" + }, + "Docker 构建": { + "✅ 镜像标签": "使用 commit SHA 和 latest 标签", + "✅ Docker socket": "挂载 Docker socket", + "✅ 镜像推送": "推送到私有仓库", + "⚠️ 缺少镜像扫描": "建议添加容器安全扫描" + }, + "归档策略": { + "✅ 自动归档": "部署成功后自动归档到 main", + "✅ 版本标签": "创建带时间戳的版本标签", + "✅ 动态分支": "支持任意 release/** 分支归档", + "✅ 重试机制": "推送失败自动重试 3 次" + }, + "性能优化": { + "⚠️ 缺少缓存": "建议添加 npm 依赖缓存", + "⚠️ 缺少并行": "部分步骤可以并行执行", + "✅ 浅克隆": "使用 depth: 1 减少克隆时间" + }, + "通知与监控": { + "⚠️ 缺少通知": "建议添加企业微信/钉钉通知", + "⚠️ 缺少监控": "建议集成 APM 监控", + "✅ 日志输出": "每个步骤都有清晰的日志" + }, + "配置管理": { + "✅ YAML 锚点": "使用锚点复用配置", + "✅ 环境变量": "使用环境变量传递配置", + "✅ 注释清晰": "配置文件有详细的注释", + "✅ 结构清晰": "按阶段组织步骤" + } + } + + def analyze(self): + """执行分析""" + print("\n" + "="*70) + print("Woodpecker CI 最佳实践对比分析") + print("="*70) + + for category, practices in self.best_practices.items(): + print(f"\n📋 {category}") + print("-" * 70) + + for practice, description in practices.items(): + status = practice.split()[0] + desc = description + + if status == "✅": + print(f" {practice}") + print(f" └─ {desc}") + elif status == "⚠️": + print(f" {practice}") + print(f" └─ {desc}") + elif status == "❌": + print(f" {practice}") + print(f" └─ {desc}") + + print("\n" + "="*70) + print("改进建议优先级") + print("="*70) + + recommendations = [ + ("高优先级", [ + "添加 npm 依赖缓存,减少构建时间", + "配置 Git 分支保护规则", + "添加部署通知机制" + ]), + ("中优先级", [ + "添加容器镜像安全扫描", + "集成 APM 性能监控", + "优化并行执行策略" + ]), + ("低优先级", [ + "添加代码质量门禁(如 SonarQube)", + "实现蓝绿部署", + "添加多环境支持(staging)" + ]) + ] + + for priority, items in recommendations: + print(f"\n🎯 {priority}") + for i, item in enumerate(items, 1): + print(f" {i}. {item}") + + print("\n" + "="*70) + print("总体评分") + print("="*70) + + total_practices = sum(len(practices) for practices in self.best_practices.values()) + passed_practices = sum( + 1 for practices in self.best_practices.values() + for practice in practices.keys() + if practice.startswith("✅") + ) + warning_practices = sum( + 1 for practices in self.best_practices.values() + for practice in practices.keys() + if practice.startswith("⚠️") + ) + + score = (passed_practices / total_practices) * 100 + + print(f"\n✅ 符合最佳实践: {passed_practices}/{total_practices}") + print(f"⚠️ 需要改进: {warning_practices}/{total_practices}") + print(f"📊 总体评分: {score:.1f}/100") + + if score >= 80: + print("✅ 配置质量优秀") + elif score >= 60: + print("⚠️ 配置质量良好,但有改进空间") + else: + print("❌ 配置需要重大改进") + + print("\n" + "="*70) + + +def main(): + analyzer = BestPracticeAnalyzer(".woodpecker.yml") + analyzer.analyze() + + +if __name__ == "__main__": + main() diff --git a/scripts/tools/capture-webhook.sh b/scripts/tools/capture-webhook.sh new file mode 100644 index 0000000..ed9223b --- /dev/null +++ b/scripts/tools/capture-webhook.sh @@ -0,0 +1,55 @@ +#!/bin/bash + +echo "=== 捕获 Gitea Webhook Header ===" +echo "" + +echo "1. 创建临时 webhook 接收器..." +cat > /tmp/capture-webhook.py << 'PYEOF' +#!/usr/bin/env python3 +from http.server import HTTPServer, BaseHTTPRequestHandler +import json +import sys + +class WebhookHandler(BaseHTTPRequestHandler): + def do_POST(self): + print("\n=== 收到 Webhook 请求 ===") + print(f"Path: {self.path}") + print("\nHeaders:") + for key, value in self.headers.items(): + print(f" {key}: {value}") + + content_length = int(self.headers.get('Content-Length', 0)) + body = self.rfile.read(content_length) + print(f"\nBody (前 500 字符):") + print(body.decode('utf-8')[:500]) + + self.send_response(200) + self.send_header('Content-type', 'application/json') + self.end_headers() + self.wfile.write(b'{"status":"ok"}') + + def log_message(self, format, *args): + pass + +if __name__ == '__main__': + port = int(sys.argv[1]) if len(sys.argv) > 1 else 9999 + server = HTTPServer(('0.0.0.0', port), WebhookHandler) + print(f"监听端口 {port}...") + server.handle_request() +PYEOF + +chmod +x /tmp/capture-webhook.py + +echo "2. 在服务器上启动 webhook 接收器..." +echo " 请在另一个终端运行:" +echo " ssh root@139.155.109.62 'python3 /tmp/capture-webhook.py 9999'" +echo "" + +echo "3. 或者,让我们检查 Gitea 的 webhook 配置..." +echo "" +echo "检查 Gitea 容器中的 webhook 设置..." +docker exec forgejo ls -la /data/gitea/data/hooks/ 2>/dev/null || echo "没有找到 hooks 目录" + +echo "" +echo "检查最近的 webhook 日志..." +docker logs forgejo --since 5m 2>&1 | grep -i webhook | tail -10 diff --git a/scripts/tools/check-job-triggers.groovy b/scripts/tools/check-job-triggers.groovy new file mode 100644 index 0000000..826e5c1 --- /dev/null +++ b/scripts/tools/check-job-triggers.groovy @@ -0,0 +1,33 @@ +import jenkins.model.* +import org.jenkinsci.plugins.workflow.job.* + +def jenkins = Jenkins.getInstance() +def job = jenkins.getItem('novalon-website') + +if (job != null) { + println "Job found: ${job.fullName}" + println "Job class: ${job.class}" + + def triggers = job.getTriggers() + println "Triggers: ${triggers}" + + triggers.each { key, value -> + println "Trigger: ${key} -> ${value}" + } + + def properties = job.getProperties() + println "Properties: ${properties}" + + properties.each { prop -> + println "Property: ${prop.class}" + if (prop instanceof org.jenkinsci.plugins.workflow.job.properties.PipelineTriggersJobProperty) { + def pipelineTriggers = prop.getTriggers() + println "Pipeline Triggers: ${pipelineTriggers}" + pipelineTriggers.each { trigger -> + println "Pipeline Trigger: ${trigger.class} -> ${trigger}" + } + } + } +} else { + println "Job not found" +} diff --git a/scripts/tools/check-woodpecker-logs.sh b/scripts/tools/check-woodpecker-logs.sh new file mode 100644 index 0000000..5ac6834 --- /dev/null +++ b/scripts/tools/check-woodpecker-logs.sh @@ -0,0 +1,37 @@ +#!/bin/bash +# Woodpecker CI 日志诊断脚本 +# 需要在 Woodpecker CI 服务器上执行 + +echo "==========================================" +echo "Woodpecker CI 日志诊断" +echo "==========================================" +echo "" + +# 检查 Woodpecker 容器是否运行 +echo "1. 检查 Woodpecker 容器状态..." +docker ps | grep woodpecker || echo "❌ Woodpecker 容器未运行" +echo "" + +# 查看最近的日志 +echo "2. 查看最近的 Woodpecker 日志 (最后 100 行)..." +docker logs woodpecker-server --tail 100 2>&1 | grep -E "(webhook|hook|pipeline|error|fail)" || echo "未找到相关日志" +echo "" + +# 查看 Webhook 相关日志 +echo "3. 查看 Webhook 处理日志..." +docker logs woodpecker-server --tail 200 2>&1 | grep -i "webhook" || echo "未找到 webhook 日志" +echo "" + +# 查看仓库相关日志 +echo "4. 查看仓库 novalon/novalon-website 相关日志..." +docker logs woodpecker-server --tail 200 2>&1 | grep -i "novalon-website" || echo "未找到仓库相关日志" +echo "" + +# 查看错误日志 +echo "5. 查看错误日志..." +docker logs woodpecker-server --tail 200 2>&1 | grep -iE "(error|fail|warn)" || echo "未找到错误日志" +echo "" + +echo "==========================================" +echo "诊断完成" +echo "==========================================" diff --git a/scripts/tools/fix-jenkins-nginx.sh b/scripts/tools/fix-jenkins-nginx.sh new file mode 100755 index 0000000..31cdbef --- /dev/null +++ b/scripts/tools/fix-jenkins-nginx.sh @@ -0,0 +1,73 @@ +#!/bin/bash + +# 修复Jenkins Nginx配置 +cat > /tmp/jenkins-nginx-fix.conf << 'EOF' + # Jenkins CI/CD Server + server { + listen 80; + server_name ci.f.novalon.cn; + return 301 https://$host$request_uri; + } + + server { + listen 443 ssl http2; + server_name ci.f.novalon.cn; + + ssl_certificate /etc/nginx/ssl/ci.f.novalon.cn/fullchain.pem; + ssl_certificate_key /etc/nginx/ssl/ci.f.novalon.cn/privkey.pem; + ssl_protocols TLSv1.2 TLSv1.3; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305; + ssl_prefer_server_ciphers off; + ssl_session_cache shared:SSL:10m; + ssl_session_timeout 1d; + + add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always; + add_header X-Frame-Options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + add_header X-XSS-Protection "1; mode=block" always; + + # Jenkins webhook端点 - 不需要/jenkins前缀 + location /generic-webhook-trigger/ { + proxy_pass http://172.17.0.1:8080/generic-webhook-trigger/; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Forwarded-Port $server_port; + client_max_body_size 100m; + proxy_connect_timeout 60s; + proxy_send_timeout 60s; + proxy_read_timeout 60s; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + + # Jenkins主应用 + location /jenkins/ { + proxy_pass http://172.17.0.1:8080/jenkins/; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Forwarded-Port $server_port; + client_max_body_size 100m; + proxy_connect_timeout 60s; + proxy_send_timeout 60s; + proxy_read_timeout 60s; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + + # 默认location - 重定向到/jenkins/ + location / { + return 301 https://$host/jenkins/; + } + + access_log /var/log/nginx/jenkins-access.log; + error_log /var/log/nginx/jenkins-error.log; + } +EOF + +echo "Jenkins Nginx配置已生成" diff --git a/scripts/tools/test-branch-matching.py b/scripts/tools/test-branch-matching.py new file mode 100644 index 0000000..f1c3cb0 --- /dev/null +++ b/scripts/tools/test-branch-matching.py @@ -0,0 +1,46 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI 分支匹配测试 +检查分支名称是否匹配配置中的规则 +""" + +import fnmatch + +def match_branch(branch_pattern, actual_branch): + """测试分支匹配""" + if branch_pattern == actual_branch: + return True + + if branch_pattern.endswith("/**"): + prefix = branch_pattern[:-3] + return actual_branch.startswith(prefix + "/") + + if "*" in branch_pattern: + return fnmatch.fnmatch(actual_branch, branch_pattern) + + return False + +# 测试当前分支 +actual_branch = "release/v1.0.0" +patterns = ["release", "release/**", "release/*"] + +print(f"实际分支: {actual_branch}") +print("\n匹配测试:") +for pattern in patterns: + result = match_branch(pattern, actual_branch) + print(f" {pattern:20} -> {'✅ 匹配' if result else '❌ 不匹配'}") + +# 测试其他分支 +test_branches = [ + ("feature/new-feature", ["feature/**", "feature/*"]), + ("dev", ["dev"]), + ("release", ["release", "release/**"]), + ("release/v2.0.0", ["release", "release/**"]), +] + +print("\n\n其他分支测试:") +for branch, patterns in test_branches: + print(f"\n分支: {branch}") + for pattern in patterns: + result = match_branch(pattern, branch) + print(f" {pattern:20} -> {'✅ 匹配' if result else '❌ 不匹配'}") diff --git a/scripts/tools/test-scenarios.py b/scripts/tools/test-scenarios.py new file mode 100644 index 0000000..7f2445a --- /dev/null +++ b/scripts/tools/test-scenarios.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI 场景测试 +模拟不同分支场景下的 CI/CD 流程执行 +""" + +import yaml +from pathlib import Path +from typing import Dict, List, Set + + +class ScenarioTester: + """场景测试器""" + + def __init__(self, config_path: str): + self.config_path = Path(config_path) + with open(self.config_path, 'r', encoding='utf-8') as f: + self.config = yaml.safe_load(f) + + self.scenarios = { + "场景1: feature分支开发": { + "branch": "feature/new-feature", + "event": "push", + "expected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "e2e-smoke" + ], + "unexpected_steps": [ + "e2e-standard", "e2e-deep", "build-image", + "deploy-production", "archive-to-main" + ] + }, + "场景2: feature分支PR": { + "branch": "feature/another-feature", + "event": "pull_request", + "expected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "e2e-smoke" + ], + "unexpected_steps": [ + "e2e-standard", "e2e-deep", "build-image", + "deploy-production", "archive-to-main" + ] + }, + "场景3: dev分支集成": { + "branch": "dev", + "event": "push", + "expected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "e2e-standard" + ], + "unexpected_steps": [ + "e2e-smoke", "e2e-deep", "build-image", + "deploy-production", "archive-to-main" + ] + }, + "场景4: release分支部署": { + "branch": "release/v1.0.0", + "event": "push", + "expected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "e2e-standard", "e2e-deep", + "e2e-performance", "e2e-accessibility", "e2e-visual", + "build-image", "deploy-production", "archive-to-main" + ], + "unexpected_steps": ["e2e-smoke"] + }, + "场景5: release主分支部署": { + "branch": "release", + "event": "push", + "expected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "e2e-standard", "e2e-deep", + "e2e-performance", "e2e-accessibility", "e2e-visual", + "build-image", "deploy-production", "archive-to-main" + ], + "unexpected_steps": ["e2e-smoke"] + }, + "场景6: main分支只读": { + "branch": "main", + "event": "push", + "expected_steps": [], + "unexpected_steps": [ + "lint", "type-check", "security-scan", + "unit-tests", "build-image", "deploy-production" + ] + } + } + + def match_branch(self, pattern: str, branch: str) -> bool: + """匹配分支模式""" + if pattern == branch: + return True + if pattern.endswith("/**"): + prefix = pattern[:-3] + return branch.startswith(prefix + "/") + return False + + def get_triggered_steps(self, branch: str, event: str) -> Set[str]: + """获取触发的步骤""" + triggered = set() + + for step_name, step_config in self.config.get('steps', {}).items(): + if not isinstance(step_config, dict): + continue + + when_config = step_config.get('when', {}) + if not when_config: + triggered.add(step_name) + continue + + event_match = False + branch_match = False + + if isinstance(when_config, list): + for condition in when_config: + if isinstance(condition, dict): + if 'event' in condition: + if event in condition['event']: + event_match = True + if 'branch' in condition: + for pattern in condition['branch']: + if self.match_branch(pattern, branch): + branch_match = True + break + elif isinstance(when_config, dict): + if 'event' in when_config: + if event in when_config['event']: + event_match = True + if 'branch' in when_config: + for pattern in when_config['branch']: + if self.match_branch(pattern, branch): + branch_match = True + break + + if event_match and branch_match: + triggered.add(step_name) + + return triggered + + def run_scenario(self, scenario_name: str, scenario: Dict) -> Dict: + """运行单个场景""" + branch = scenario['branch'] + event = scenario['event'] + expected = set(scenario['expected_steps']) + unexpected = set(scenario['unexpected_steps']) + + triggered = self.get_triggered_steps(branch, event) + + missing = expected - triggered + extra = triggered & unexpected + + passed = len(missing) == 0 and len(extra) == 0 + + return { + "scenario": scenario_name, + "branch": branch, + "event": event, + "passed": passed, + "triggered": triggered, + "expected": expected, + "missing": missing, + "extra": extra + } + + def run_all_scenarios(self): + """运行所有场景""" + print("\n" + "="*70) + print("Woodpecker CI 场景测试") + print("="*70) + + results = [] + + for scenario_name, scenario in self.scenarios.items(): + result = self.run_scenario(scenario_name, scenario) + results.append(result) + + print(f"\n📋 {scenario_name}") + print(f" 分支: {result['branch']}") + print(f" 事件: {result['event']}") + + if result['passed']: + print(f" ✅ 测试通过") + else: + print(f" ❌ 测试失败") + + print(f" 触发步骤 ({len(result['triggered'])}): {', '.join(sorted(result['triggered']))}") + + if result['missing']: + print(f" ⚠️ 缺少步骤: {', '.join(sorted(result['missing']))}") + + if result['extra']: + print(f" ⚠️ 多余步骤: {', '.join(sorted(result['extra']))}") + + print("\n" + "="*70) + print("测试总结") + print("="*70) + + passed_count = sum(1 for r in results if r['passed']) + total_count = len(results) + + print(f"\n✅ 通过: {passed_count}/{total_count}") + print(f"❌ 失败: {total_count - passed_count}/{total_count}") + + if passed_count == total_count: + print("\n✅ 所有场景测试通过!") + else: + print("\n❌ 部分场景测试失败,请检查配置") + + print("\n" + "="*70) + + return results + + +def main(): + tester = ScenarioTester(".woodpecker.yml") + results = tester.run_all_scenarios() + + failed = [r for r in results if not r['passed']] + if failed: + print("\n失败的场景:") + for result in failed: + print(f" - {result['scenario']}") + exit(1) + else: + exit(0) + + +if __name__ == "__main__": + main() diff --git a/scripts/tools/test-webhook-headers.sh b/scripts/tools/test-webhook-headers.sh new file mode 100644 index 0000000..f939694 --- /dev/null +++ b/scripts/tools/test-webhook-headers.sh @@ -0,0 +1,33 @@ +#!/bin/bash + +echo "=== 测试 Webhook Header 传递 ===" +echo "" + +echo "1. 检查 Forgejo 发送的 Webhook Header..." +echo "" + +echo "2. 创建测试 Webhook 接收器..." +cat > /tmp/test-webhook.sh << 'EOF' +#!/bin/bash +echo "Received webhook at $(date)" +echo "Headers:" +for header in "$@"; do + echo " $header" +done +echo "Body:" +cat +echo "" +EOF + +chmod +x /tmp/test-webhook.sh + +echo "3. 测试 Nginx 配置..." +docker exec novalon-nginx nginx -T 2>&1 | grep -A 20 "location /api/" | head -25 + +echo "" +echo "4. 检查最近的 webhook 请求..." +docker logs woodpecker-server 2>&1 | grep "POST /api/hook" | tail -3 + +echo "" +echo "5. 检查 webhook 解析日志..." +docker logs woodpecker-server 2>&1 | grep "unsupported hook type" | tail -3 diff --git a/scripts/tools/test-wechat-notify-fixed.sh b/scripts/tools/test-wechat-notify-fixed.sh new file mode 100755 index 0000000..51fc8d0 --- /dev/null +++ b/scripts/tools/test-wechat-notify-fixed.sh @@ -0,0 +1,50 @@ +#!/bin/bash + +# 测试企业微信通知脚本(修复版) + +WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=bb7efcdc-c32f-47b7-a437-d76cab9fba74" + +# 模拟 CI 环境变量 +STATUS="failure" +BRANCH="release/v1.0.0" +COMMIT="test456" +MESSAGE="fix: 修复husky和企业微信通知问题 + +- 在 commands 中添加 export HUSKY=0 确保 husky 被禁用 +- 修复 MESSAGE_ESCAPED 处理逻辑,避免 shell 解析错误 +- 将换行符替换为空格,而不是 \\n 字符串" +AUTHOR="zhangxiang" +PIPELINE_NUMBER="12" +PIPELINE_URL="https://ci.f.novalon.cn/repos/1/pipeline/12" + +if [ "$STATUS" = "success" ]; then + STATUS_TEXT="成功" + STATUS_COLOR="info" +else + STATUS_TEXT="失败" + STATUS_COLOR="warning" +fi + +TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S") + +# 转义特殊字符(简化处理) +MESSAGE_ESCAPED=$(echo "$MESSAGE" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | tr '\n' ' ') + +echo "发送企业微信通知..." +echo "状态: $STATUS_TEXT" +echo "分支: $BRANCH" +echo "提交: $COMMIT" +echo "作者: $AUTHOR" +echo "MESSAGE_ESCAPED: $MESSAGE_ESCAPED" + +curl -X POST "$WEBHOOK_URL" \ + -H 'Content-Type: application/json' \ + -d "{ + \"msgtype\": \"markdown\", + \"markdown\": { + \"content\": \"## 🚀 Novalon Website 部署通知\\n\\n> **构建状态**: ${STATUS_TEXT}\\n\\n**项目信息**\\n> 分支: \`${BRANCH}\`\\n> 提交: \`${COMMIT}\`\\n> 作者: ${AUTHOR}\\n\\n**提交信息**\\n> ${MESSAGE_ESCAPED}\\n\\n**操作**\\n> [查看构建详情](${PIPELINE_URL})\\n\\n---\\n> 时间: ${TIMESTAMP}\\n> Pipeline #${PIPELINE_NUMBER}\" + } + }" + +echo "" +echo "通知发送完成!" diff --git a/scripts/tools/test-wechat-notify-heredoc.sh b/scripts/tools/test-wechat-notify-heredoc.sh new file mode 100755 index 0000000..2806e15 --- /dev/null +++ b/scripts/tools/test-wechat-notify-heredoc.sh @@ -0,0 +1,63 @@ +#!/bin/bash + +# 测试企业微信通知脚本(使用 heredoc) + +WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=bb7efcdc-c32f-47b7-a437-d76cab9fba74" + +# 模拟 CI 环境变量 +STATUS="failure" +BRANCH="release/v1.0.0" +COMMIT="test456" +MESSAGE="fix: 使用--ignore-scripts跳过husky并修复企业微信通知 + +- 使用 npm ci --omit=dev --ignore-scripts 跳过所有脚本 +- 本地验证 husky 问题已解决 +- 本地验证企业微信通知成功 +- 将换行符替换为空格,避免 shell 解析错误" +AUTHOR="zhangxiang" +PIPELINE_NUMBER="13" +REPO_ID="1" +PIPELINE_URL="https://ci.f.novalon.cn/repos/${REPO_ID}/pipeline/${PIPELINE_NUMBER}" + +if [ "$STATUS" = "success" ]; then + STATUS_TEXT="成功" + STATUS_COLOR="info" +else + STATUS_TEXT="失败" + STATUS_COLOR="warning" +fi + +TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S") + +# 简化处理:移除换行符和引号 +MESSAGE_CLEAN=$(echo "$MESSAGE" | tr '\n' ' ' | tr '"' "'") + +echo "发送企业微信通知..." +echo "状态: $STATUS_TEXT" +echo "分支: $BRANCH" +echo "提交: $COMMIT" +echo "作者: $AUTHOR" +echo "MESSAGE_CLEAN: $MESSAGE_CLEAN" +echo "" + +# 使用 heredoc 构建 JSON,避免复杂的转义 +JSON_DATA=$(cat < **构建状态**: ${STATUS_TEXT}\n\n**项目信息**\n> 分支: \`${BRANCH}\`\n> 提交: \`${COMMIT}\`\n> 作者: ${AUTHOR}\n\n**提交信息**\n> ${MESSAGE_CLEAN}\n\n**操作**\n> [查看构建详情](${PIPELINE_URL})\n\n---\n> 时间: ${TIMESTAMP}\n> Pipeline #${PIPELINE_NUMBER}" + } +} +EOF +) + +echo "JSON_DATA:" +echo "$JSON_DATA" +echo "" + +curl -X POST "$WEBHOOK_URL" \ + -H 'Content-Type: application/json' \ + -d "$JSON_DATA" + +echo "" +echo "通知发送完成!" diff --git a/scripts/tools/test-wechat-notify-jq.sh b/scripts/tools/test-wechat-notify-jq.sh new file mode 100755 index 0000000..c637a2c --- /dev/null +++ b/scripts/tools/test-wechat-notify-jq.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +# 测试企业微信通知脚本(使用 jq) + +WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=bb7efcdc-c32f-47b7-a437-d76cab9fba74" + +# 模拟 CI 环境变量 +STATUS="failure" +BRANCH="release/v1.0.0" +COMMIT="testjq" +MESSAGE="fix: 使用jq构建JSON避免YAML多行字符串问题 + +- 使用 jq 来构建 JSON +- 避免 YAML 多行字符串处理问题 +- 确保变量正确展开" +AUTHOR="zhangxiang" +PIPELINE_NUMBER="18" +REPO_ID="1" +PIPELINE_URL="https://ci.f.novalon.cn/repos/${REPO_ID}/pipeline/${PIPELINE_NUMBER}" + +TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S") + +# 简化处理:移除换行符和引号 +MESSAGE_CLEAN=$(echo "$MESSAGE" | tr '\n' ' ' | tr '"' "'") + +echo "发送企业微信通知..." +echo "状态: $STATUS" +echo "分支: $BRANCH" +echo "提交: $COMMIT" +echo "作者: $AUTHOR" +echo "MESSAGE_CLEAN: $MESSAGE_CLEAN" +echo "" + +# 使用 jq 构建 JSON +CONTENT="## 🚀 Novalon Website 部署通知\n\n> **构建状态**: 失败\n\n**项目信息**\n> 分支: \`${BRANCH}\`\n> 提交: \`${COMMIT}\`\n> 作者: ${AUTHOR}\n\n**提交信息**\n> ${MESSAGE_CLEAN}\n\n**操作**\n> [查看构建详情](${PIPELINE_URL})\n\n---\n> 时间: ${TIMESTAMP}\n> Pipeline #${PIPELINE_NUMBER}" + +JSON_DATA=$(echo "{}" | jq --arg content "$CONTENT" '.msgtype = "markdown" | .markdown.content = $content') + +echo "JSON_DATA:" +echo "$JSON_DATA" +echo "" + +curl -X POST "$WEBHOOK_URL" \ + -H 'Content-Type: application/json' \ + -d "$JSON_DATA" + +echo "" +echo "通知发送完成!" diff --git a/scripts/tools/test-wechat-notify-printf.sh b/scripts/tools/test-wechat-notify-printf.sh new file mode 100755 index 0000000..f179c97 --- /dev/null +++ b/scripts/tools/test-wechat-notify-printf.sh @@ -0,0 +1,51 @@ +#!/bin/bash + +# 测试企业微信通知脚本(使用 printf) + +WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=bb7efcdc-c32f-47b7-a437-d76cab9fba74" + +# 模拟 CI 环境变量 +STATUS="failure" +BRANCH="release/v1.0.0" +COMMIT="testprintf" +MESSAGE="fix: 使用printf构建JSON避免变量展开问题 + +- 使用 printf 来构建 JSON +- 避免单引号和双引号组合的问题 +- 确保变量正确展开" +AUTHOR="zhangxiang" +PIPELINE_NUMBER="17" +REPO_ID="1" +PIPELINE_URL="https://ci.f.novalon.cn/repos/${REPO_ID}/pipeline/${PIPELINE_NUMBER}" + +TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S") + +# 简化处理:移除换行符和引号 +MESSAGE_CLEAN=$(echo "$MESSAGE" | tr '\n' ' ' | tr '"' "'") + +echo "发送企业微信通知..." +echo "状态: $STATUS" +echo "分支: $BRANCH" +echo "提交: $COMMIT" +echo "作者: $AUTHOR" +echo "MESSAGE_CLEAN: $MESSAGE_CLEAN" +echo "" + +# 使用 printf 构建 JSON +JSON_DATA=$(printf '{ + "msgtype": "markdown", + "markdown": { + "content": "## 🚀 Novalon Website 部署通知\\n\\n> **构建状态**: 失败\\n\\n**项目信息**\\n> 分支: `%s`\\n> 提交: `%s`\\n> 作者: %s\\n\\n**提交信息**\\n> %s\\n\\n**操作**\\n> [查看构建详情](%s)\\n\\n---\\n> 时间: %s\\n> Pipeline #%s" + } +}' "$BRANCH" "$COMMIT" "$AUTHOR" "$MESSAGE_CLEAN" "$PIPELINE_URL" "$TIMESTAMP" "$PIPELINE_NUMBER") + +echo "JSON_DATA:" +echo "$JSON_DATA" +echo "" + +curl -X POST "$WEBHOOK_URL" \ + -H 'Content-Type: application/json' \ + -d "$JSON_DATA" + +echo "" +echo "通知发送完成!" diff --git a/scripts/tools/test-wechat-notify-single-quote.sh b/scripts/tools/test-wechat-notify-single-quote.sh new file mode 100755 index 0000000..c0ff1b3 --- /dev/null +++ b/scripts/tools/test-wechat-notify-single-quote.sh @@ -0,0 +1,45 @@ +#!/bin/bash + +# 测试企业微信通知脚本(使用单引号和双引号组合) + +WEBHOOK_URL="https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=bb7efcdc-c32f-47b7-a437-d76cab9fba74" + +# 模拟 CI 环境变量 +STATUS="failure" +BRANCH="release/v1.0.0" +COMMIT="test789" +MESSAGE="fix: 使用单引号和双引号组合避免heredoc问题 + +- 移除 heredoc 语法 +- 使用单引号和双引号组合来构建 JSON +- 确保变量正确展开" +AUTHOR="zhangxiang" +PIPELINE_NUMBER="16" +REPO_ID="1" +PIPELINE_URL="https://ci.f.novalon.cn/repos/${REPO_ID}/pipeline/${PIPELINE_NUMBER}" + +TIMESTAMP=$(date "+%Y-%m-%d %H:%M:%S") + +# 简化处理:移除换行符和引号 +MESSAGE_CLEAN=$(echo "$MESSAGE" | tr '\n' ' ' | tr '"' "'") + +echo "发送企业微信通知..." +echo "状态: $STATUS" +echo "分支: $BRANCH" +echo "提交: $COMMIT" +echo "作者: $AUTHOR" +echo "MESSAGE_CLEAN: $MESSAGE_CLEAN" +echo "" + +# 使用单引号和双引号组合,确保变量正确展开 +curl -X POST "$WEBHOOK_URL" \ + -H 'Content-Type: application/json' \ + -d '{ + "msgtype": "markdown", + "markdown": { + "content": "## 🚀 Novalon Website 部署通知\n\n> **构建状态**: 失败\n\n**项目信息**\n> 分支: `'"${BRANCH}"'`\n> 提交: `'"${COMMIT}"'`\n> 作者: '"${AUTHOR}"'\n\n**提交信息**\n> '"${MESSAGE_CLEAN}"'\n\n**操作**\n> [查看构建详情]('"${PIPELINE_URL}"')\n\n---\n> 时间: '"${TIMESTAMP}"'\n> Pipeline #'"${PIPELINE_NUMBER}"'" + } + }' + +echo "" +echo "通知发送完成!" diff --git a/scripts/tools/test-woodpecker-config.py b/scripts/tools/test-woodpecker-config.py new file mode 100644 index 0000000..61a11fc --- /dev/null +++ b/scripts/tools/test-woodpecker-config.py @@ -0,0 +1,274 @@ +#!/usr/bin/env python3 +""" +Woodpecker CI/CD 配置验证脚本 +系统性地测试和验收 .woodpecker.yml 配置 +""" + +import yaml +import sys +from pathlib import Path +from typing import Dict, List, Any + + +class WoodpeckerValidator: + """Woodpecker CI 配置验证器""" + + def __init__(self, config_path: str): + self.config_path = Path(config_path) + self.config = None + self.errors = [] + self.warnings = [] + self.info = [] + + def load_config(self) -> bool: + """加载配置文件""" + try: + with open(self.config_path, 'r', encoding='utf-8') as f: + self.config = yaml.safe_load(f) + self.info.append("✅ 配置文件加载成功") + return True + except Exception as e: + self.errors.append(f"❌ 配置文件加载失败: {e}") + return False + + def validate_structure(self) -> bool: + """验证配置结构""" + required_keys = ['steps', 'services', 'workspace', 'clone'] + + for key in required_keys: + if key not in self.config: + self.errors.append(f"❌ 缺少必需的配置项: {key}") + else: + self.info.append(f"✅ 找到配置项: {key}") + + return len(self.errors) == 0 + + def validate_branch_triggers(self) -> Dict[str, List[str]]: + """验证分支触发条件""" + branch_mapping = { + 'feature': ['feature/**'], + 'dev': ['dev'], + 'release': ['release', 'release/**'], + 'main': [] # main 不应该触发任何步骤 + } + + step_branches = {} + + for step_name, step_config in self.config.get('steps', {}).items(): + if not isinstance(step_config, dict): + continue + + when_config = step_config.get('when', {}) + if not when_config: + self.warnings.append(f"⚠️ 步骤 '{step_name}' 没有触发条件") + continue + + if isinstance(when_config, list): + branches = [] + for condition in when_config: + if isinstance(condition, dict) and 'branch' in condition: + branches.extend(condition['branch']) + elif isinstance(when_config, dict): + branches = when_config.get('branch', []) + else: + branches = [] + + if branches: + step_branches[step_name] = branches + self.info.append(f"✅ 步骤 '{step_name}' 触发分支: {branches}") + + return step_branches + + def validate_test_strategy(self) -> Dict[str, List[str]]: + """验证测试策略分层""" + expected_tests = { + 'feature/**': ['lint', 'type-check', 'security-scan', 'unit-tests', 'e2e-smoke'], + 'dev': ['lint', 'type-check', 'security-scan', 'unit-tests', 'e2e-standard'], + 'release': ['lint', 'type-check', 'security-scan', 'unit-tests', 'e2e-standard', + 'e2e-deep', 'e2e-performance', 'e2e-accessibility', 'e2e-visual', + 'build-image', 'deploy-production', 'archive-to-main'] + } + + test_coverage = {} + + for branch, expected_steps in expected_tests.items(): + test_coverage[branch] = [] + for step_name in expected_steps: + if step_name in self.config.get('steps', {}): + test_coverage[branch].append(step_name) + self.info.append(f"✅ 分支 '{branch}' 包含步骤: {step_name}") + else: + self.errors.append(f"❌ 分支 '{branch}' 缺少步骤: {step_name}") + + return test_coverage + + def validate_archive_logic(self) -> bool: + """验证归档逻辑""" + archive_step = self.config.get('steps', {}).get('archive-to-main', {}) + + if not archive_step: + self.errors.append("❌ 缺少 archive-to-main 步骤") + return False + + commands = archive_step.get('commands', []) + has_dynamic_branch = False + + for cmd in commands: + if isinstance(cmd, str) and 'CURRENT_BRANCH="${CI_COMMIT_BRANCH}"' in cmd: + has_dynamic_branch = True + self.info.append("✅ 归档步骤使用动态分支变量") + break + + if not has_dynamic_branch: + self.warnings.append("⚠️ 归档步骤可能未使用动态分支变量") + + when_config = archive_step.get('when', {}) + branches = when_config.get('branch', []) + + if 'release' in branches and 'release/**' in branches: + self.info.append("✅ 归档步骤支持 release 和 release/** 分支") + else: + self.errors.append("❌ 归档步骤分支配置不完整") + + return len(self.errors) == 0 + + def validate_deployment_safety(self) -> bool: + """验证部署安全性""" + deploy_step = self.config.get('steps', {}).get('deploy-production', {}) + + if not deploy_step: + self.errors.append("❌ 缺少 deploy-production 步骤") + return False + + commands = deploy_step.get('commands', []) + has_rollback = False + has_health_check = False + + for cmd in commands: + if isinstance(cmd, str): + if 'rolling back' in cmd.lower() or 'rollback' in cmd.lower(): + has_rollback = True + self.info.append("✅ 部署步骤包含回滚机制") + if 'health check' in cmd.lower(): + has_health_check = True + self.info.append("✅ 部署步骤包含健康检查") + + if not has_rollback: + self.warnings.append("⚠️ 部署步骤可能缺少回滚机制") + + if not has_health_check: + self.warnings.append("⚠️ 部署步骤可能缺少健康检查") + + secrets = deploy_step.get('environment', {}) + if 'SSH_PRIVATE_KEY' in secrets and 'REGISTRY_PASSWORD' in secrets: + self.info.append("✅ 部署步骤使用 Secret 管理敏感信息") + else: + self.errors.append("❌ 部署步骤未正确配置 Secrets") + + return len(self.errors) == 0 + + def validate_docker_build(self) -> bool: + """验证 Docker 构建配置""" + build_step = self.config.get('steps', {}).get('build-image', {}) + + if not build_step: + self.errors.append("❌ 缺少 build-image 步骤") + return False + + commands = build_step.get('commands', []) + has_tagging = False + + for cmd in commands: + if isinstance(cmd, str) and 'docker tag' in cmd: + has_tagging = True + self.info.append("✅ Docker 构建步骤包含镜像标签") + break + + if not has_tagging: + self.warnings.append("⚠️ Docker 构建步骤可能缺少镜像标签") + + volumes = build_step.get('volumes', []) + if any('/var/run/docker.sock' in str(v) for v in volumes): + self.info.append("✅ Docker 构建步骤挂载了 Docker socket") + else: + self.warnings.append("⚠️ Docker 构建步骤可能未挂载 Docker socket") + + return len(self.errors) == 0 + + def validate_services(self) -> bool: + """验证服务配置""" + services = self.config.get('services', {}) + + if 'docker' not in services: + self.warnings.append("⚠️ 缺少 Docker 服务配置") + else: + self.info.append("✅ Docker 服务配置正确") + + return True + + def generate_report(self) -> str: + """生成测试报告""" + report = [] + report.append("\n" + "="*60) + report.append("Woodpecker CI/CD 配置验证报告") + report.append("="*60) + + report.append(f"\n📋 配置文件: {self.config_path}") + report.append(f"📊 总步骤数: {len(self.config.get('steps', {}))}") + + report.append("\n✅ 通过的检查:") + for msg in self.info: + report.append(f" {msg}") + + report.append("\n⚠️ 警告:") + for msg in self.warnings: + report.append(f" {msg}") + + report.append("\n❌ 错误:") + for msg in self.errors: + report.append(f" {msg}") + + report.append("\n" + "="*60) + + if self.errors: + report.append("❌ 验证失败 - 发现 {} 个错误".format(len(self.errors))) + return "\n".join(report) + else: + report.append("✅ 验证通过 - 配置文件符合要求") + return "\n".join(report) + + +def main(): + """主函数""" + config_path = ".woodpecker.yml" + + if not Path(config_path).exists(): + print(f"❌ 配置文件不存在: {config_path}") + sys.exit(1) + + validator = WoodpeckerValidator(config_path) + + print("🔍 开始验证 Woodpecker CI/CD 配置...") + + if not validator.load_config(): + print(validator.generate_report()) + sys.exit(1) + + validator.validate_structure() + validator.validate_branch_triggers() + validator.validate_test_strategy() + validator.validate_archive_logic() + validator.validate_deployment_safety() + validator.validate_docker_build() + validator.validate_services() + + print(validator.generate_report()) + + if validator.errors: + sys.exit(1) + else: + sys.exit(0) + + +if __name__ == "__main__": + main() diff --git a/scripts/tools/update-jenkins-nginx.sh b/scripts/tools/update-jenkins-nginx.sh new file mode 100644 index 0000000..f12b021 --- /dev/null +++ b/scripts/tools/update-jenkins-nginx.sh @@ -0,0 +1,86 @@ +#!/bin/bash + +# 修复Jenkins Nginx配置 - 更新webhook路径 + +# 在服务器上执行此脚本 + +# 1. 备份当前配置 +docker cp novalon-nginx-secure:/etc/nginx/nginx.conf /tmp/nginx.conf.bak + +# 2. 创建新的Jenkins配置 +cat > /tmp/jenkins-server.conf << 'EOF' + # Jenkins CI/CD Server + server { + listen 80; + server_name ci.f.novalon.cn; + return 301 https://$host$request_uri; + } + + server { + listen 443 ssl http2; + server_name ci.f.novalon.cn; + + ssl_certificate /etc/nginx/ssl/ci.f.novalon.cn/fullchain.pem; + ssl_certificate_key /etc/nginx/ssl/ci.f.novalon.cn/privkey.pem; + ssl_protocols TLSv1.2 TLSv1.3; + ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305; + ssl_prefer_server_ciphers off; + ssl_session_cache shared:SSL:10m; + ssl_session_timeout 1d; + + add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always; + add_header X-Frame-Options "SAMEORIGIN" always; + add_header X-Content-Type-Options "nosniff" always; + add_header X-XSS-Protection "1; mode=block" always; + + # Jenkins webhook端点 - 直接代理到Jenkins根路径 + location /generic-webhook-trigger/ { + proxy_pass http://172.17.0.1:8080/generic-webhook-trigger/; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + client_max_body_size 100m; + proxy_connect_timeout 60s; + proxy_send_timeout 60s; + proxy_read_timeout 60s; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + + # Jenkins主应用 + location /jenkins/ { + proxy_pass http://172.17.0.1:8080/jenkins/; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + client_max_body_size 100m; + proxy_connect_timeout 60s; + proxy_send_timeout 60s; + proxy_read_timeout 60s; + proxy_http_version 1.1; + proxy_set_header Upgrade $http_upgrade; + proxy_set_header Connection "upgrade"; + } + + # 默认location - 重定向到/jenkins/ + location / { + return 301 https://$host/jenkins/; + } + + access_log /var/log/nginx/jenkins-access.log; + error_log /var/log/nginx/jenkins-error.log; + } +EOF + +# 3. 替换Jenkins配置部分 +sed -i '/# Jenkins CI\/CD Server/,/^ }$/d' /tmp/nginx.conf.bak +sed -i "/^}/i $(cat /tmp/jenkins-server.conf)" /tmp/nginx.conf.bak + +# 4. 复制回容器并重载 +docker cp /tmp/nginx.conf.bak novalon-nginx-secure:/etc/nginx/nginx.conf +docker exec novalon-nginx-secure nginx -t && docker exec novalon-nginx-secure nginx -s reload + +echo "Jenkins Nginx配置已更新"