Как настроить Hudson для запуска распределенных тестов с помощью RemoteWebDriver?

belikov_as88 · 22.Ноябрь.2013 14:55:43

Всем привет!

Решил попробовать настроить прогон тестов на CI Hudson. Репозиторий в Git. Сейчас тесты запускаются с помощью selenium grid. Запускается hud на локальной машине, запускаются 8 нодов на виртуальных машинах (на них же запущено приложение). Хаб видит ноды, собственно все, тесты пошли. Драйвер инициализируется следующим образом:

driver = new RemoteWebDriver(new URL("http://192.168.123.111:4444/wd/hub"),capability)

Теперь собственно вопросы:

Запуск тестов на CI - те же яйца, только вид сбоку? Т.е на хадсоне должен быть запущен hub и подключенные к нему ноды? И все это в одном Job-е?
Как проинициализировать драйвер в этом случае? (RemoteWebDriver) Указывается ссылка к хабу на хадсоне?

Вот лог, который я сейчас получаю:

Started by user anonymous
Checkout:workspace / /var/lib/hudson/jobs/master/workspace - hudson.remoting.LocalChannel@3e63488a
Using strategy: Default
Last Built Revision: Revision 19b76789047f185bf78b35e498587e6645c40fd1 (origin/hudson)
Checkout:workspace / /var/lib/hudson/jobs/master/workspace - hudson.remoting.LocalChannel@3e63488a
Fetching changes from the remote Git repository
Fetching upstream changes from git@git.assembla.com:БЛАБЛАБЛА.git
Commencing build of Revision 19b76789047f185bf78b35e498587e6645c40fd1 (origin/hudson)
Checking out Revision 19b76789047f185bf78b35e498587e6645c40fd1 (origin/hudson)
Xvfb starting$ /usr/bin//Xvfb :0 -screen 0 1024x768x16 -fbdir /var/lib/hudson/2013-11-22_19-43-09122850671685693594xvfb -ac
SELinux: Disabled on system, not enabling in X server
(EE) AIGLX error: dlopen of /usr/lib/dri/swrast_dri.so failed (/usr/lib/dri/swrast_dri.so: cannot open shared object file: No such file or directory)
(EE) GLX: could not load software renderer
[dix] Could not init font path element /usr/share/fonts/X11/cyrillic, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi/:unscaled, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/Type1, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/100dpi, removing from list!
[dix] Could not init font path element /usr/share/fonts/X11/75dpi, removing from list!
[dix] Could not init font path element /var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType, removing from list!
[INFO] Using Maven 3 installation: mvn
[INFO] Checking Maven 3 installation environment
[workspace] $ /opt/apache-maven-3.0.4/bin/mvn --help
[INFO] Checking Maven 3 installation version
[INFO] Detected Maven 3 installation version: 3.0.4
[workspace] $ /opt/apache-maven-3.0.4/bin/mvn test -V -B -DDISPLAY=:0 -Dmaven.ext.class.path=/var/lib/hudson/maven/slavebundle/resources:/var/lib/hudson/maven/slavebundle/lib/maven3-eventspy-3.0.jar:/var/lib/hudson/war/webapp/WEB-INF/lib/hudson-remoting-3.0.1.jar -Dhudson.eventspy.port=55133 -f pom.xml
[DEBUG] Waiting for connection on port: 55133
Apache Maven 3.0.4 (r1232337; 2012-01-17 12:44:56+0400)
Maven home: /opt/apache-maven-3.0.4
Java version: 1.7.0_04, vendor: Oracle Corporation
Java home: /opt/jdk1.7.0_04/jre
Default locale: ru_RU, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-5-amd64", arch: "amd64", family: "unix"
[DEBUG] Connected to remote
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building microfinance-selenium-tests 1.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- gmaven-plugin:1.4:compile (compile-groovy) @ microfinance-selenium-tests ---
[INFO] Compiled 5 Groovy classes
[INFO] 
[INFO] --- gmaven-plugin:1.4:testCompile (compile-groovy) @ microfinance-selenium-tests ---
[INFO] Compiled 5 Groovy classes
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ microfinance-selenium-tests ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource
[INFO] 
[INFO] --- maven-antrun-plugin:1.3:run (createThriftDir) @ microfinance-selenium-tests ---
[INFO] Executing tasks
   [delete] Deleting directory /var/lib/hudson/jobs/master/workspace/target/surefire-reports/screenshots
    [mkdir] Created dir: /var/lib/hudson/jobs/master/workspace/target/surefire-reports/screenshots
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ microfinance-selenium-tests ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ microfinance-selenium-tests ---
[debug] execute contextualize
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /var/lib/hudson/jobs/master/workspace/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ microfinance-selenium-tests ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.13:test (default-test) @ microfinance-selenium-tests ---
[INFO] Surefire report directory: /var/lib/hudson/jobs/master/workspace/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
e[31mdictionary.DictionaryTests:addEmptySource                                                  FAILEDe
Tests run: 16, Failures: 2, Errors: 0, Skipped: 14, Time elapsed: 44.901 sec <<< FAILURE!
addEmptySource(dictionary.DictionaryTests)  Time elapsed: 22.406 sec  <<< FAILURE!
org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: 'dfb1306b85be4934d23c123122e06e602a15e446', revision: 'unknown', time: '2013-01-17 15:05:54'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '2.6.32-5-amd64', java.version: '1.7.0_04'
Driver info: driver.version: RemoteWebDriver
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:151)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:125)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.openqa.selenium.remote.HttpCommandExecutor.fallBackExecute(HttpCommandExecutor.java:331)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:310)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:506)
        at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:216)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:111)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:129)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:194)
        at utils.Browser.<init>(Browser.groovy:26)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
        at utils.Dispatcher.<init>(Dispatcher.groovy:59)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:182)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:186)
        at utils.Dispatcher.getSeleniumInstance(Dispatcher.groovy:84)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1671)
        at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3408)
        at org.codehaus.groovy.runtime.callsite.ClassMetaClassGetPropertySite.getProperty(ClassMetaClassGetPropertySite.java:48)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:227)
        at pages.LoginPage.open(LoginPage.groovy:9)
        at pages.LoginPage$open.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
        at dictionary.DictionaryTests.addEmptySource(DictionaryTests.groovy:75)

logOut(dictionary.DictionaryTests)  Time elapsed: 43.73 sec  <<< FAILURE!
org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: 'dfb1306b85be4934d23c123122e06e602a15e446', revision: 'unknown', time: '2013-01-17 15:05:54'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '2.6.32-5-amd64', java.version: '1.7.0_04'
Driver info: driver.version: RemoteWebDriver
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:151)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:125)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.openqa.selenium.remote.HttpCommandExecutor.fallBackExecute(HttpCommandExecutor.java:331)
        at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:310)
        at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:506)
        at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:216)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:111)
        at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:129)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:194)
        at utils.Browser.<init>(Browser.groovy:26)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:102)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:190)
        at utils.Dispatcher.<init>(Dispatcher.groovy:59)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
        at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:77)
        at org.codehaus.groovy.reflection.MixinInMetaClass.getMixinInstance(MixinInMetaClass.java:70)
        at org.codehaus.groovy.runtime.metaclass.MixinInstanceMetaMethod.invoke(MixinInstanceMetaMethod.java:53)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1671)
        at groovy.lang.ExpandoMetaClass.getProperty(ExpandoMetaClass.java:1140)
        at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3408)
        at groovy.lang.ExpandoMetaClass.getProperty(ExpandoMetaClass.java:1152)
        at org.codehaus.groovy.runtime.callsite.PogoMetaClassGetPropertySite.getProperty(PogoMetaClassGetPropertySite.java:48)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:231)
        at dictionary.DictionaryTests.logOut(DictionaryTests.groovy:213)


Results :

Failed tests: 
  DictionaryTests.addEmptySource:75 » UnreachableBrowser Could not start a new s...
  DictionaryTests.logOut:213 » UnreachableBrowser Could not start a new session....

Tests run: 16, Failures: 2, Errors: 0, Skipped: 14

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 49.725s
[INFO] Finished at: Fri Nov 22 19:44:04 MSK 2013
[INFO] Final Memory: 17M/172M
[INFO] ------------------------------------------------------------------------
[INFO] o.h.m.e.h.MavenExecutionResultHandler - Build failed with exception(s)
[INFO] o.h.m.e.h.MavenExecutionResultHandler - [1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project microfinance-selenium-tests: There are test failures.

Please refer to /var/lib/hudson/jobs/master/workspace/target/surefire-reports for the individual test results.
[DEBUG] Closing connection to remote
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project microfinance-selenium-tests: There are test failures.
[ERROR] 
[ERROR] Please refer to /var/lib/hudson/jobs/master/workspace/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[DEBUG] Waiting for process to finish
[DEBUG] Result: 1
Xvfb stopping
Finished: FAILURE

Не может стартануть новую сессию. Почему? Что ему для этого необходимо?

joemast · 25.Ноябрь.2013 06:43:48

ИМХО, роль Hudson (ныне Jenkins) должна быть простой.

Он просто должен запускать Maven goal, к примеру mvn test
Maven surefire-плагин в свою очередь запускает TestNG (JUnit)
который запускает собственно тест
который уже коннектится к Grid на ту ноду, где он сейчас установлен

belikov_as88 · 25.Ноябрь.2013 07:29:21

Т.е я правильно понимаю, что на хадсоне должен быть запущен хаб и нод? Выполнение теста будет происходить на ноде? Установил вчера Selenium plugin на хадсоне, ковырял, ковырял, пока безрезультатно. И не пойму как все таки должен быть проинициализирован web driver? RemoteWebDriver? FirefoxDriver?

joemast · 25.Ноябрь.2013 08:00:22

Вы про этот плагин говорите?

Как я понял, он просто позволяет развернуть инфраструктуру Selenium Grid на кластере Jenkins. Если вам надо это, то это один вопрос.

А если вы хотите просто запускать тесты в режиме CI, то можно это делать и на существующей инфраструктуре Selenium Grid, я так понял она у вас уже развернута.

Сейчас у вас тесты Maven-ом запускаются или как?

belikov_as88 · 25.Ноябрь.2013 08:06:29

да, именно этот плагин я установил. Да, тесты запускаются maven-ом, инфраструктура для grid готова, и вполне используется

joemast · 25.Ноябрь.2013 08:31:03

Ну тогда не используй пока этот плагин.

Можно делать просто:

вытягиваешь код из репозитория
запускаешь сборку через maven как обычно. Тесты будут запускаться как сейчас, коннектиться к текущему Grid

Всё это делается средствами Jenkins. У него есть специализированные шаги. Если для мавена нет спец-плагина (уже не помню детали), можно просто по ssh его запускать, например