testomat.io управление авто тестаминашел случайно один чеклист по проекту автоматизации тестирования, который мы когда-то реализовали, я думаю будет правильно его опубликовать в базе знаний. Мы использовали этот чеклист для обсуждения предстоящей автоматизации с клиентом и эти пункты помогли нам определить правильную и согласованную стратегию автоматизации тестирования
. Я недеюсь список будет каким-то образом полезным для вас.
Goals (make better quality faster)
- test automation as definition of done
- test automation as integral part of each software testing and development process
- ideally fulfilled by each scrum team
- each team member can run it and use it for particular purposes
- effectively cover end2end scenario
- as longest chain as possible
- different testing levels (component, integration, UI) on testing purposes
- block system to have ability to use some parts for different testing purposes (configuration testing, order sending, order validation, etc )
- 1-2 hours test execution no matter what amount of test automation scripts
- execution time is important, tester should have ability to run automation 5-7 times a day
- test automation thru effective and quick interfaces
- scope coverage ? %
- define tests to be automated
- define complexity and what % of tests should be automated
- less time to develop test - ? hours
- need to be defined
- less time to maintain test - ? hours
- need to be defined
Attributes of successful test automation
- extensible and scalable architecture
- easy to add-modify-remove system\component with minor impact on the overall solution
- easy to reuse components for similar functionality
- easy to support different environments
- easy to support configurations
- easy to manage different types of test data
- threads safe architecture for parallel execution
- ability to cover extra test automation (desktop, mobile, serverside, etc)
- cost-effective without permissions
- no cost to buy
- no approval to start test automation
- no maintenance cost
- easy to deprecate without limitations
- easy to develop and reuse
- right levels of abstraction and DSL (domain-specific language)
- easy to develop separate parts by different developers in parallel
- easy to reuse any part of test/logic/component/system
- easy to run locally
- decentralized version control without blocking
- automatically generated documentation from code
- easy to maintain
- defect clustering and trends
- smart defects and issues recognition
- historical test execution information
- automatic development team notification
- decentralized systems support
- decentralized automation solution support by different developers at the same time
- integration with bug tracking tools
- quick execution
- anyone can run automation without any help and installation
- anyone can see results without any help and installation
- results should be available quickly, ideally in real-time
- ability to run particular sets, tests by patterns, test parts
- ability to run by context (severity, urgency, flaky tests, product types, configuration, etc)
- ability to subscribe to test execution results
- ability to run by schedule, trigger or manually
- integration capabilities
- with test management tools
- with bug tracking systems
- with other test automation solution
- with development solutions
- with deployment tools
Team
- involved
- component/system teams, to reuse their solutions
- test automation in a scrum team, to develop/maintain tests
- test automation core development teams, to develop common architecture and infrastructure
- responsibilities
- core development team
- overall test automation process setup and maintenance
- overall architecture setup, optimization, and support
- infrastructure setup, optimization, and support
- new technical interfaces and integration development
- necessary tools development
- strategic changes and ongoing optimization
- automation in the scrum team
- scope management
- develop and maintain tests
- defects and issues management
- environment health tracking and monitoring
- test engineer
- scope management
- test data management
- scenario description
- test suites generation
- issues escalations
- product owners
- scope and priority management
- strategic changes and ongoing optimization
- management team
- budget management
- communications
- issues follow up and escalations
- core development team
Process (scrum, kanban)
- test automation requirements management
- defined by scrum team with product owners with the core dev team
- KPI list production
- test automation strategy approval and changes
- scope and scope changes
- backlog management
- ongoing scope and feature changes tracking thru the backlog
- process, communications and negotiations on changes, approvals
- test descriptions
- owned by the scrum team
- descriptions to be done as close as possible to test automation formats
- all feature changes should be outlined in the description after changes
- appropriate communication on changes
- process automation
- test case development
- definition of done for test automation development of one test
- common design patterns for development on defined architecture
- parallel development
- code and peer review
- development and deployment automation
- test data management
- unified standards for everyone
- support by scrum team on a constant basis with appropriate responsibilities
- data re-usage
- tests maintenance
- to be done on a daily basis
- fix and deployment procedures
- change management
- defects tracking and monitoring
Documentation
- scope
- every team should have a testing scope with prioritization and details
- the scope should be available for all teams
- scenarios
- unified standards
- described with re-usability in mind
- distinguish system, action, checkpoints, test data, preconditions, postconditions
- test data
- unified standards
- re-usage for software testing and test automation
- configuration descriptions
- stubs
- usage agreements
- the description of internal implementation
- the description of changes for testing purposes
- auto-generated test automation docs
- based on code
Technical architecture
- implementation
- SOA oriented and micro-services to easy reuse any system\component
- decentralized to easily support different system
- modularity and abstraction to expression scenarios in DSL
- high level of re-usage
- low-cost changes and maintenance
- system coverage
- quick system interfaces usage where possible
- interface creation if such an interface is not provided
- test data management
- agreed and centralized among teams
- easy to reuse (XML soap messages, CSV, excel data sheets, etc)
- test parametrization on test data
- modularity
- overall level
- by country
- by system
- by context
- by steps
- by checkpoints
- infrastructure support
- multi-environment support
- configuration support
- parallel execution
- logging and reporting
- unified standards
- accessible for everyone in scrum team thru UI
Test environment
- centralized and coordinated by test automation in the scrum team
- test data management
- should be always synchronized with the environment
- stubs
- easy to stub necessary integrations
- configuration to change for testing purposes
- stubs dashboards to track changes
- automatic notifications
- health tracking
- smoke test
- automatic notifications
- recovery mechanisms
Maintenance
- done by automation in a scrum team
- real-time logging and reporting capabilities
- historical data on tests execution
- automatic Jira tickets management
- automatic issues reporting and triggers to check it automatically
- test automation bots in chats
- defects clustering notification per system/component
- integration to test management and reporting
- triggers system
- environment health check system
Integration
- to (test) management tools
- to component test automation
- to development tools and processes
- to issue tracking
- to reporting systems
- to change management processes
Infrastructure
- centralized
- local servers cloud to easy extend automation on load
- permission system
- tools support and integration
- rest support for external usage
- reporting
- history data
- clustering
- easy navigation
- health tracking system
Extra tools for testing and teams
- stubs management, auto-notifications, dashboard
- run a particular test or step
- smart defects clustering
- configuration testing tool
- performance testing capabilities
- test automation bots
- environment health tracking
- integrations real-time monitoring
- coverage tracking and change management
- test data creation tool
- trigger system with notifications
- defect creation tool
Open questions
- agreed software testing strategy
- no test automation integration strategy into software testing process
- no defined test automation goals
- no teams common vision
Ставь лайк
если эта заметка полезна для тебя! И опубликую похожую заметку из какого-то из твоих проектов в базу знаний, будет интересно почитать всему сообществу.