张翔
446 lines
8.6 KiB
Markdown
446 lines
8.6 KiB
Markdown
# Allure 测试报告使用指南
|
||
|
||
## 概述
|
||
|
||
Allure Framework 是一个灵活的、轻量级的、多语言测试报告工具，它不仅以简洁的Web报告形式展示测试结果，还提供了完整的测试执行历史记录。
|
||
|
||
## 安装状态
|
||
|
||
✅ **已安装组件**:
|
||
- `allure-playwright`: ^3.5.0 (Playwright集成)
|
||
- `allure-commandline`: ^2.37.0 (命令行工具)
|
||
|
||
✅ **已配置**:
|
||
- Playwright配置文件中已集成Allure reporter
|
||
- 分层测试配置支持Allure报告生成
|
||
|
||
## 快速开始
|
||
|
||
### 1. 运行测试并生成报告
|
||
|
||
```bash
|
||
# 运行分层测试（会自动生成allure-results）
|
||
npm run test:tier:fast
|
||
npm run test:tier:standard
|
||
npm run test:tier:deep
|
||
|
||
# 或者运行所有分层测试
|
||
npm run test:tier:all
|
||
```
|
||
|
||
### 2. 生成HTML报告
|
||
|
||
```bash
|
||
# 生成Allure报告
|
||
npm run test:allure
|
||
|
||
# 或者直接打开报告
|
||
npm run test:allure:open
|
||
|
||
# 或者实时serve报告
|
||
npm run test:allure:serve
|
||
```
|
||
|
||
## 报告功能
|
||
|
||
### 📊 测试概览
|
||
|
||
Allure报告提供以下信息：
|
||
|
||
1. **测试统计**
|
||
- 总测试数
|
||
- 通过/失败/跳过数量
|
||
- 成功率
|
||
- 执行时间
|
||
|
||
2. **测试分类**
|
||
- 按套件分组
|
||
- 按标签分组（@smoke, @regression等）
|
||
- 按严重程度分组
|
||
|
||
3. **历史趋势**
|
||
- 测试结果历史对比
|
||
- 失败率趋势
|
||
- 执行时间趋势
|
||
|
||
### 📈 详细信息
|
||
|
||
每个测试用例包含：
|
||
|
||
- **测试步骤**: 详细的执行步骤
|
||
- **附件**: 截图、视频、日志
|
||
- **参数**: 测试参数和配置
|
||
- **时间线**: 执行时间分布
|
||
- **环境信息**: 浏览器、操作系统等
|
||
|
||
### 🎯 测试分层支持
|
||
|
||
分层测试配置已集成Allure报告：
|
||
|
||
| 测试层级 | 配置文件 | 报告输出 |
|
||
|---------|---------|---------|
|
||
| Fast | playwright.config.tiered.ts | allure-results/ |
|
||
| Standard | playwright.config.tiered.ts | allure-results/ |
|
||
| Deep | playwright.config.tiered.ts | allure-results/ |
|
||
|
||
## 使用场景
|
||
|
||
### 场景1: 本地开发调试
|
||
|
||
```bash
|
||
# 1. 运行特定测试
|
||
cd e2e
|
||
npx playwright test --grep "contact-form"
|
||
|
||
# 2. 实时查看报告
|
||
npm run test:allure:serve
|
||
```
|
||
|
||
### 场景2: CI/CD集成
|
||
|
||
```yaml
|
||
# .woodpecker.yml示例
|
||
pipeline:
|
||
test:
|
||
image: node:18
|
||
commands:
|
||
- npm run test:tier:fast
|
||
- npm run test:tier:standard
|
||
- npm run test:allure
|
||
when:
|
||
event: [push, pull_request]
|
||
|
||
publish-report:
|
||
image: node:18
|
||
commands:
|
||
- allure generate allure-results -o allure-report
|
||
when:
|
||
status: [success, failure]
|
||
```
|
||
|
||
### 场景3: 测试失败分析
|
||
|
||
```bash
|
||
# 1. 运行失败的测试
|
||
cd e2e
|
||
npx playwright test --last-failed
|
||
|
||
# 2. 查看详细报告
|
||
npm run test:allure:open
|
||
|
||
# 3. 在报告中查看：
|
||
# - 失败的断言
|
||
# - 错误堆栈
|
||
# - 截图和视频
|
||
# - 执行日志
|
||
```
|
||
|
||
## 报告定制
|
||
|
||
### 添加测试标签
|
||
|
||
在测试文件中添加标签：
|
||
|
||
```typescript
|
||
import { test, expect } from '@playwright/test';
|
||
|
||
test('用户登录测试 @smoke @critical', async ({ page }) => {
|
||
// 测试代码
|
||
});
|
||
```
|
||
|
||
### 添加测试步骤
|
||
|
||
```typescript
|
||
import { test } from '@playwright/test';
|
||
|
||
test('复杂测试流程', async ({ page }) => {
|
||
await test.step('打开登录页面', async () => {
|
||
await page.goto('/login');
|
||
});
|
||
|
||
await test.step('输入用户名', async () => {
|
||
await page.fill('#username', 'test@example.com');
|
||
});
|
||
|
||
await test.step('输入密码', async () => {
|
||
await page.fill('#password', 'password123');
|
||
});
|
||
|
||
await test.step('提交表单', async () => {
|
||
await page.click('#submit');
|
||
});
|
||
});
|
||
```
|
||
|
||
### 添加附件
|
||
|
||
```typescript
|
||
import { test } from '@playwright/test';
|
||
|
||
test('带附件的测试', async ({ page }, testInfo) => {
|
||
await page.goto('/');
|
||
|
||
// 添加截图
|
||
const screenshot = await page.screenshot();
|
||
await testInfo.attach('首页截图', {
|
||
body: screenshot,
|
||
contentType: 'image/png'
|
||
});
|
||
|
||
// 添加文本日志
|
||
await testInfo.attach('测试日志', {
|
||
body: '这是测试日志内容',
|
||
contentType: 'text/plain'
|
||
});
|
||
});
|
||
```
|
||
|
||
## 报告分析技巧
|
||
|
||
### 1. 识别不稳定测试
|
||
|
||
在报告的"Categories"部分查看：
|
||
- 标记为"Product defects"的测试：真正的bug
|
||
- 标记为"Test defects"的测试：测试代码问题
|
||
|
||
### 2. 性能分析
|
||
|
||
在"Timeline"标签页：
|
||
- 查看测试执行时间分布
|
||
- 识别慢速测试
|
||
- 优化测试性能
|
||
|
||
### 3. 失败模式分析
|
||
|
||
在"Graphs"标签页：
|
||
- 查看失败率趋势
|
||
- 识别常见失败原因
|
||
- 追踪测试稳定性
|
||
|
||
## 最佳实践
|
||
|
||
### ✅ 推荐做法
|
||
|
||
1. **使用有意义的测试名称**
|
||
```typescript
|
||
test('用户应该能够成功登录 @smoke', async ({ page }) => {
|
||
// ...
|
||
});
|
||
```
|
||
|
||
2. **添加详细的测试步骤**
|
||
- 每个关键操作都是一个step
|
||
- 步骤名称清晰描述操作
|
||
|
||
3. **为失败测试添加附件**
|
||
- 失败时自动截图
|
||
- 保存页面HTML
|
||
- 记录控制台日志
|
||
|
||
4. **使用标签分类测试**
|
||
- @smoke: 冒烟测试
|
||
- @regression: 回归测试
|
||
- @critical: 关键测试
|
||
- @flaky: 不稳定测试
|
||
|
||
### ❌ 避免的做法
|
||
|
||
1. **不要在报告中包含敏感信息**
|
||
- 密码
|
||
- API密钥
|
||
- 个人数据
|
||
|
||
2. **不要过度使用附件**
|
||
- 只在必要时添加
|
||
- 避免报告过大
|
||
|
||
3. **不要忽略失败测试**
|
||
- 及时修复或标记
|
||
- 分析根本原因
|
||
|
||
## CI/CD集成示例
|
||
|
||
### GitHub Actions
|
||
|
||
```yaml
|
||
name: Test with Allure Report
|
||
|
||
on: [push, pull_request]
|
||
|
||
jobs:
|
||
test:
|
||
runs-on: ubuntu-latest
|
||
steps:
|
||
- uses: actions/checkout@v3
|
||
|
||
- name: Setup Node.js
|
||
uses: actions/setup-node@v3
|
||
with:
|
||
node-version: '18'
|
||
|
||
- name: Install dependencies
|
||
run: |
|
||
npm ci
|
||
cd e2e && npm ci
|
||
|
||
- name: Run tests
|
||
run: |
|
||
npm run test:tier:fast
|
||
npm run test:tier:standard
|
||
|
||
- name: Generate Allure Report
|
||
if: always()
|
||
run: npm run test:allure
|
||
|
||
- name: Upload Allure Report
|
||
if: always()
|
||
uses: actions/upload-artifact@v3
|
||
with:
|
||
name: allure-report
|
||
path: e2e/allure-report
|
||
```
|
||
|
||
### Woodpecker CI
|
||
|
||
```yaml
|
||
pipeline:
|
||
install:
|
||
image: node:18
|
||
commands:
|
||
- npm ci
|
||
- cd e2e && npm ci
|
||
|
||
test:
|
||
image: node:18
|
||
commands:
|
||
- npm run test:tier:fast
|
||
- npm run test:tier:standard
|
||
when:
|
||
event: [push, pull_request]
|
||
|
||
report:
|
||
image: node:18
|
||
commands:
|
||
- cd e2e && npm run test:allure
|
||
when:
|
||
status: [success, failure]
|
||
|
||
publish:
|
||
image: node:18
|
||
commands:
|
||
- cd e2e && npm run test:allure:open
|
||
when:
|
||
status: [success, failure]
|
||
```
|
||
|
||
## 故障排查
|
||
|
||
### 问题1: 报告无法生成
|
||
|
||
**症状**: 运行`npm run test:allure`时报错
|
||
|
||
**解决方案**:
|
||
```bash
|
||
# 检查allure-commandline是否安装
|
||
cd e2e
|
||
npm ls allure-commandline
|
||
|
||
# 如果未安装，重新安装
|
||
npm install --save-dev allure-commandline
|
||
```
|
||
|
||
### 问题2: 报告内容为空
|
||
|
||
**症状**: 报告生成成功，但没有测试数据
|
||
|
||
**解决方案**:
|
||
```bash
|
||
# 检查allure-results目录
|
||
ls -la e2e/allure-results
|
||
|
||
# 确保测试已运行
|
||
npm run test:tier:fast
|
||
```
|
||
|
||
### 问题3: 截图未显示
|
||
|
||
**症状**: 报告中看不到截图
|
||
|
||
**解决方案**:
|
||
```typescript
|
||
// 确保Playwright配置中启用了截图
|
||
use: {
|
||
screenshot: 'only-on-failure', // 或 'on'
|
||
video: 'retain-on-failure',
|
||
trace: 'retain-on-failure',
|
||
}
|
||
```
|
||
|
||
## 进阶功能
|
||
|
||
### 1. 自定义报告配置
|
||
|
||
创建`allure.config.js`:
|
||
|
||
```javascript
|
||
module.exports = {
|
||
resultsDir: 'allure-results',
|
||
reportDir: 'allure-report',
|
||
cleanResultsDir: true,
|
||
cleanReportDir: true,
|
||
};
|
||
```
|
||
|
||
### 2. 集成到测试框架
|
||
|
||
```typescript
|
||
// playwright.config.ts
|
||
import { defineConfig } from '@playwright/test';
|
||
|
||
export default defineConfig({
|
||
reporter: [
|
||
['allure-playwright', {
|
||
outputFolder: 'allure-results',
|
||
detail: true,
|
||
suiteTitle: false,
|
||
}],
|
||
],
|
||
});
|
||
```
|
||
|
||
### 3. 报告历史记录
|
||
|
||
使用Allure TestOps或Allure Report Server保存历史报告：
|
||
|
||
```bash
|
||
# 安装Allure Report Server
|
||
npm install -g allure-report-server
|
||
|
||
# 启动服务器
|
||
allure-report-server --port 8080
|
||
```
|
||
|
||
## 参考资源
|
||
|
||
- [Allure官方文档](https://docs.qameta.io/allure/)
|
||
- [Allure Playwright集成](https://github.com/allure-framework/allure-js)
|
||
- [Playwright测试最佳实践](https://playwright.dev/docs/best-practices)
|
||
- [测试分层指南](./test-tiering-best-practices.md)
|
||
|
||
## 总结
|
||
|
||
Allure测试报告已完全集成到项目中，提供了：
|
||
|
||
✅ **自动化报告生成**
|
||
✅ **分层测试支持**
|
||
✅ **丰富的测试信息**
|
||
✅ **历史趋势分析**
|
||
✅ **CI/CD集成**
|
||
|
||
通过合理使用Allure报告，可以：
|
||
- 快速定位测试失败原因
|
||
- 追踪测试稳定性
|
||
- 分析测试性能
|
||
- 提升测试质量
|