t.me/atinfo_chat Telegram группа по автоматизации тестирования

Чеклист старта проекта по автоматизации тестирования или как не забыть важные пункты перед началом проекта

Теги: #<Tag:0x00007fb302f69a70>

(Mykhailo Poliarush) #1

нашел случайно один чеклист по проекту автоматизации тестирования, который мы когда-то реализовали, я думаю будет правильно его опубликовать в базе знаний. Мы использовали этот чеклист для обсуждения предстоящей автоматизации с клиентом и эти пункты помогли нам определить правильную и согласованную стратегию автоматизации тестирования :rocket:. Я недеюсь список будет каким-то образом полезным для вас.

Goals (make better quality faster)

  • test automation as definition of done
    • test automation as integral part of each software testing and development process
    • ideally fulfilled by each scrum team
    • each team member can run it and use it for particular purposes
  • effectively cover end2end scenario
    • as longest chain as possible
    • different testing levels (component, integration, UI) on testing purposes
    • block system to have ability to use some parts for different testing purposes (configuration testing, order sending, order validation, etc )
  • 1-2 hours test execution no matter what amount of test automation scripts
    • execution time is important, tester should have ability to run automation 5-7 times a day
    • test automation thru effective and quick interfaces
  • scope coverage ? %
    • define tests to be automated
    • define complexity and what % of tests should be automated
  • less time to develop test - ? hours
    • need to be defined
  • less time to maintain test - ? hours
    • need to be defined

Attributes of successful test automation

  • extensible and scalable architecture
    • easy to add-modify-remove system\component with minor impact on the overall solution
    • easy to reuse components for similar functionality
    • easy to support different environments
    • easy to support configurations
    • easy to manage different types of test data
    • threads safe architecture for parallel execution
    • ability to cover extra test automation (desktop, mobile, serverside, etc)
  • cost-effective without permissions
    • no cost to buy
    • no approval to start test automation
    • no maintenance cost
    • easy to deprecate without limitations
  • easy to develop and reuse
    • right levels of abstraction and DSL (domain-specific language)
    • easy to develop separate parts by different developers in parallel
    • easy to reuse any part of test/logic/component/system
    • easy to run locally
    • decentralized version control without blocking
    • automatically generated documentation from code
  • easy to maintain
    • defect clustering and trends
    • smart defects and issues recognition
    • historical test execution information
    • automatic development team notification
    • decentralized systems support
    • decentralized automation solution support by different developers at the same time
    • integration with bug tracking tools
  • quick execution
    • anyone can run automation without any help and installation
    • anyone can see results without any help and installation
    • results should be available quickly, ideally in real-time
    • ability to run particular sets, tests by patterns, test parts
    • ability to run by context (severity, urgency, flaky tests, product types, configuration, etc)
    • ability to subscribe to test execution results
    • ability to run by schedule, trigger or manually
  • integration capabilities
    • with test management tools
    • with bug tracking systems
    • with other test automation solution
    • with development solutions
    • with deployment tools


  • involved
    • component/system teams, to reuse their solutions
    • test automation in a scrum team, to develop/maintain tests
    • test automation core development teams, to develop common architecture and infrastructure
  • responsibilities
    • core development team
      • overall test automation process setup and maintenance
      • overall architecture setup, optimization, and support
      • infrastructure setup, optimization, and support
      • new technical interfaces and integration development
      • necessary tools development
      • strategic changes and ongoing optimization
    • automation in the scrum team
      • scope management
      • develop and maintain tests
      • defects and issues management
      • environment health tracking and monitoring
    • test engineer
      • scope management
      • test data management
      • scenario description
      • test suites generation
      • issues escalations
    • product owners
      • scope and priority management
      • strategic changes and ongoing optimization
    • management team
      • budget management
      • communications
      • issues follow up and escalations

Process (scrum, kanban)

  • test automation requirements management
    • defined by scrum team with product owners with the core dev team
    • KPI list production
    • test automation strategy approval and changes
  • scope and scope changes
    • backlog management
    • ongoing scope and feature changes tracking thru the backlog
    • process, communications and negotiations on changes, approvals
  • test descriptions
    • owned by the scrum team
    • descriptions to be done as close as possible to test automation formats
    • all feature changes should be outlined in the description after changes
    • appropriate communication on changes
    • process automation
  • test case development
    • definition of done for test automation development of one test
    • common design patterns for development on defined architecture
    • parallel development
    • code and peer review
    • development and deployment automation
  • test data management
    • unified standards for everyone
    • support by scrum team on a constant basis with appropriate responsibilities
    • data re-usage
  • tests maintenance
    • to be done on a daily basis
    • fix and deployment procedures
    • change management
    • defects tracking and monitoring


  • scope
    • every team should have a testing scope with prioritization and details
    • the scope should be available for all teams
  • scenarios
    • unified standards
    • described with re-usability in mind
    • distinguish system, action, checkpoints, test data, preconditions, postconditions
  • test data
    • unified standards
    • re-usage for software testing and test automation
    • configuration descriptions
  • stubs
    • usage agreements
    • the description of internal implementation
    • the description of changes for testing purposes
  • auto-generated test automation docs
    • based on code

Technical architecture

  • implementation
    • SOA oriented and micro-services to easy reuse any system\component
    • decentralized to easily support different system
    • modularity and abstraction to expression scenarios in DSL
    • high level of re-usage
    • low-cost changes and maintenance
  • system coverage
    • quick system interfaces usage where possible
    • interface creation if such an interface is not provided
  • test data management
    • agreed and centralized among teams
    • easy to reuse (XML soap messages, CSV, excel data sheets, etc)
    • test parametrization on test data
  • modularity
    • overall level
    • by country
    • by system
    • by context
    • by steps
    • by checkpoints
  • infrastructure support
    • multi-environment support
    • configuration support
    • parallel execution
  • logging and reporting
    • unified standards
    • accessible for everyone in scrum team thru UI

Test environment

  • centralized and coordinated by test automation in the scrum team
  • test data management
    • should be always synchronized with the environment
  • stubs
    • easy to stub necessary integrations
    • configuration to change for testing purposes
    • stubs dashboards to track changes
    • automatic notifications
  • health tracking
    • smoke test
    • automatic notifications
    • recovery mechanisms


  • done by automation in a scrum team
  • real-time logging and reporting capabilities
  • historical data on tests execution
  • automatic Jira tickets management
  • automatic issues reporting and triggers to check it automatically
  • test automation bots in chats
  • defects clustering notification per system/component
  • integration to test management and reporting
  • triggers system
  • environment health check system


  • to (test) management tools
  • to component test automation
  • to development tools and processes
  • to issue tracking
  • to reporting systems
  • to change management processes


  • centralized
  • local servers cloud to easy extend automation on load
  • permission system
  • tools support and integration
  • rest support for external usage
  • reporting
    • history data
    • clustering
    • easy navigation
  • health tracking system

Extra tools for testing and teams

  • stubs management, auto-notifications, dashboard
  • run a particular test or step
  • smart defects clustering
  • configuration testing tool
  • performance testing capabilities
  • test automation bots
  • environment health tracking
  • integrations real-time monitoring
  • coverage tracking and change management
  • test data creation tool
  • trigger system with notifications
  • defect creation tool

Open questions

  • agreed software testing strategy
  • no test automation integration strategy into software testing process
  • no defined test automation goals
  • no teams common vision

Ставь лайк :+1:если эта заметка полезна для тебя! И опубликую похожую заметку из какого-то из твоих проектов в базу знаний, будет интересно почитать всему сообществу.

(Mykhailo Poliarush) #3

А ты делаешь что-то подобное перед началом проекта по автоматизации тестирования?

  • Да
  • Нет

0 участников


Суть проекта - построить автоматизацию?
Мне кажется что для проекта который постепенно наращивает костяк автоматизации - это подход может быть слишком сложным)

(Mykhailo Poliarush) #5

Это консалтинговый проект, где автоматизация уже была создана, но создана неправильно. Нужно было сделать аудит, анализ, предложить варианты, определить стратегию.

(Maxim Andryushchenkov) #6

Мне кажется, если на практике начать задавать такие вопросы и в таком кол-ве, то нанимающая сторона подумает что я издеваюсь над ними) Часто бывает так, что люди не могут ответить в каком виде репорты им нужны, а не то что список на 50 пунктов) Но, наверное, это относится в основном к СНГ компаниям. Мне хочется верить что данный чек-лист выполняется где то в компаниях посерьезнее.