Чеклист старта проекта по автоматизации тестирования или как не забыть важные пункты перед началом проекта

нашел случайно один чеклист по проекту автоматизации тестирования, который мы когда-то реализовали, я думаю будет правильно его опубликовать в базе знаний. Мы использовали этот чеклист для обсуждения предстоящей автоматизации с клиентом и эти пункты помогли нам определить правильную и согласованную стратегию автоматизации тестирования :rocket:. Я недеюсь список будет каким-то образом полезным для вас.

Goals (make better quality faster)

  • test automation as definition of done
    • test automation as integral part of each software testing and development process
    • ideally fulfilled by each scrum team
    • each team member can run it and use it for particular purposes
  • effectively cover end2end scenario
    • as longest chain as possible
    • different testing levels (component, integration, UI) on testing purposes
    • block system to have ability to use some parts for different testing purposes (configuration testing, order sending, order validation, etc )
  • 1-2 hours test execution no matter what amount of test automation scripts
    • execution time is important, tester should have ability to run automation 5-7 times a day
    • test automation thru effective and quick interfaces
  • scope coverage ? %
    • define tests to be automated
    • define complexity and what % of tests should be automated
  • less time to develop test - ? hours
    • need to be defined
  • less time to maintain test - ? hours
    • need to be defined

Attributes of successful test automation

  • extensible and scalable architecture
    • easy to add-modify-remove system\component with minor impact on the overall solution
    • easy to reuse components for similar functionality
    • easy to support different environments
    • easy to support configurations
    • easy to manage different types of test data
    • threads safe architecture for parallel execution
    • ability to cover extra test automation (desktop, mobile, serverside, etc)
  • cost-effective without permissions
    • no cost to buy
    • no approval to start test automation
    • no maintenance cost
    • easy to deprecate without limitations
  • easy to develop and reuse
    • right levels of abstraction and DSL (domain-specific language)
    • easy to develop separate parts by different developers in parallel
    • easy to reuse any part of test/logic/component/system
    • easy to run locally
    • decentralized version control without blocking
    • automatically generated documentation from code
  • easy to maintain
    • defect clustering and trends
    • smart defects and issues recognition
    • historical test execution information
    • automatic development team notification
    • decentralized systems support
    • decentralized automation solution support by different developers at the same time
    • integration with bug tracking tools
  • quick execution
    • anyone can run automation without any help and installation
    • anyone can see results without any help and installation
    • results should be available quickly, ideally in real-time
    • ability to run particular sets, tests by patterns, test parts
    • ability to run by context (severity, urgency, flaky tests, product types, configuration, etc)
    • ability to subscribe to test execution results
    • ability to run by schedule, trigger or manually
  • integration capabilities
    • with test management tools
    • with bug tracking systems
    • with other test automation solution
    • with development solutions
    • with deployment tools

Team

  • involved
    • component/system teams, to reuse their solutions
    • test automation in a scrum team, to develop/maintain tests
    • test automation core development teams, to develop common architecture and infrastructure
  • responsibilities
    • core development team
      • overall test automation process setup and maintenance
      • overall architecture setup, optimization, and support
      • infrastructure setup, optimization, and support
      • new technical interfaces and integration development
      • necessary tools development
      • strategic changes and ongoing optimization
    • automation in the scrum team
      • scope management
      • develop and maintain tests
      • defects and issues management
      • environment health tracking and monitoring
    • test engineer
      • scope management
      • test data management
      • scenario description
      • test suites generation
      • issues escalations
    • product owners
      • scope and priority management
      • strategic changes and ongoing optimization
    • management team
      • budget management
      • communications
      • issues follow up and escalations

Process (scrum, kanban)

  • test automation requirements management
    • defined by scrum team with product owners with the core dev team
    • KPI list production
    • test automation strategy approval and changes
  • scope and scope changes
    • backlog management
    • ongoing scope and feature changes tracking thru the backlog
    • process, communications and negotiations on changes, approvals
  • test descriptions
    • owned by the scrum team
    • descriptions to be done as close as possible to test automation formats
    • all feature changes should be outlined in the description after changes
    • appropriate communication on changes
    • process automation
  • test case development
    • definition of done for test automation development of one test
    • common design patterns for development on defined architecture
    • parallel development
    • code and peer review
    • development and deployment automation
  • test data management
    • unified standards for everyone
    • support by scrum team on a constant basis with appropriate responsibilities
    • data re-usage
  • tests maintenance
    • to be done on a daily basis
    • fix and deployment procedures
    • change management
    • defects tracking and monitoring

Documentation

  • scope
    • every team should have a testing scope with prioritization and details
    • the scope should be available for all teams
  • scenarios
    • unified standards
    • described with re-usability in mind
    • distinguish system, action, checkpoints, test data, preconditions, postconditions
  • test data
    • unified standards
    • re-usage for software testing and test automation
    • configuration descriptions
  • stubs
    • usage agreements
    • the description of internal implementation
    • the description of changes for testing purposes
  • auto-generated test automation docs
    • based on code

Technical architecture

  • implementation
    • SOA oriented and micro-services to easy reuse any system\component
    • decentralized to easily support different system
    • modularity and abstraction to expression scenarios in DSL
    • high level of re-usage
    • low-cost changes and maintenance
  • system coverage
    • quick system interfaces usage where possible
    • interface creation if such an interface is not provided
  • test data management
    • agreed and centralized among teams
    • easy to reuse (XML soap messages, CSV, excel data sheets, etc)
    • test parametrization on test data
  • modularity
    • overall level
    • by country
    • by system
    • by context
    • by steps
    • by checkpoints
  • infrastructure support
    • multi-environment support
    • configuration support
    • parallel execution
  • logging and reporting
    • unified standards
    • accessible for everyone in scrum team thru UI

Test environment

  • centralized and coordinated by test automation in the scrum team
  • test data management
    • should be always synchronized with the environment
  • stubs
    • easy to stub necessary integrations
    • configuration to change for testing purposes
    • stubs dashboards to track changes
    • automatic notifications
  • health tracking
    • smoke test
    • automatic notifications
    • recovery mechanisms

Maintenance

  • done by automation in a scrum team
  • real-time logging and reporting capabilities
  • historical data on tests execution
  • automatic Jira tickets management
  • automatic issues reporting and triggers to check it automatically
  • test automation bots in chats
  • defects clustering notification per system/component
  • integration to test management and reporting
  • triggers system
  • environment health check system

Integration

  • to (test) management tools
  • to component test automation
  • to development tools and processes
  • to issue tracking
  • to reporting systems
  • to change management processes

Infrastructure

  • centralized
  • local servers cloud to easy extend automation on load
  • permission system
  • tools support and integration
  • rest support for external usage
  • reporting
    • history data
    • clustering
    • easy navigation
  • health tracking system

Extra tools for testing and teams

  • stubs management, auto-notifications, dashboard
  • run a particular test or step
  • smart defects clustering
  • configuration testing tool
  • performance testing capabilities
  • test automation bots
  • environment health tracking
  • integrations real-time monitoring
  • coverage tracking and change management
  • test data creation tool
  • trigger system with notifications
  • defect creation tool

Open questions

  • agreed software testing strategy
  • no test automation integration strategy into software testing process
  • no defined test automation goals
  • no teams common vision

Ставь лайк :+1:если эта заметка полезна для тебя! И опубликую похожую заметку из какого-то из твоих проектов в базу знаний, будет интересно почитать всему сообществу.

5 лайков

А ты делаешь что-то подобное перед началом проекта по автоматизации тестирования?

  • Да
  • Нет

0 участников

Суть проекта - построить автоматизацию?
Мне кажется что для проекта который постепенно наращивает костяк автоматизации - это подход может быть слишком сложным)

Это консалтинговый проект, где автоматизация уже была создана, но создана неправильно. Нужно было сделать аудит, анализ, предложить варианты, определить стратегию.

Мне кажется, если на практике начать задавать такие вопросы и в таком кол-ве, то нанимающая сторона подумает что я издеваюсь над ними) Часто бывает так, что люди не могут ответить в каком виде репорты им нужны, а не то что список на 50 пунктов) Но, наверное, это относится в основном к СНГ компаниям. Мне хочется верить что данный чек-лист выполняется где то в компаниях посерьезнее.