Jenkins Jobs of Ivy/IvyDE

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée
Hi,

I worked on the Jenkins jobs to make them work.

There was some issues with job dependencies. Following the instructions there [1] worked.
I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.
I made the Ivy-checks build work, some https changed in favor of http.
The test matrix is now separated into two builds now, since no Ant installation is common for both Windows and Ubuntu. They are both testing JDK7 and JDK8.
The IvyDE job is no longer downloading Ivy’s jar via a hard coded http url. It is relying now on a plugin which copies artifacts around.

There are still some error though, most probably not related to the Jenkins configuration:
- the Windows build of Ivy is broken, the retrieve doesn’t work [2].
- the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is reproductible locally.

I’ll look at the IvyDE updatesite job when the build of IvyDE will be fixed.

Nicolas

[1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>
[2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
[3] https://builds.apache.org/job/IvyDE/315/console <https://builds.apache.org/job/IvyDE/315/console>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Gintautas Grigelionis
[3]: the path looks wrong, it starts with "\:\:\"
[2]: please check that unpacking of drops puts files into the correct
places (the contents should go into "features" and "plugins" directories of
PDE).

Gintas

2017-06-25 16:57 GMT+02:00 Nicolas Lalevée <[hidden email]>:

> Hi,
>
> I worked on the Jenkins jobs to make them work.
>
> There was some issues with job dependencies. Following the instructions
> there [1] worked.
> I have made the Ivy Job also build the snapshot-bin, like the nightly.
> Maybe now the nightly is redundant.
> I made the Ivy-checks build work, some https changed in favor of http.
> The test matrix is now separated into two builds now, since no Ant
> installation is common for both Windows and Ubuntu. They are both testing
> JDK7 and JDK8.
> The IvyDE job is no longer downloading Ivy’s jar via a hard coded http
> url. It is relying now on a plugin which copies artifacts around.
>
> There are still some error though, most probably not related to the
> Jenkins configuration:
> - the Windows build of Ivy is broken, the retrieve doesn’t work [2].
> - the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is
> reproductible locally.
>
> I’ll look at the IvyDE updatesite job when the build of IvyDE will be
> fixed.
>
> Nicolas
>
> [1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <
> https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>
> [2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,
> jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/
> job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
> [3] https://builds.apache.org/job/IvyDE/315/console <
> https://builds.apache.org/job/IvyDE/315/console>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée

> Le 25 juin 2017 à 17:50, Gintautas Grigelionis <[hidden email]> a écrit :
>
> [3]: the path looks wrong, it starts with "\:\:\ »

I have seen that too.

> [2]: please check that unpacking of drops puts files into the correct
> places (the contents should go into "features" and "plugins" directories of
> PDE).

I have done that already. I even checked that the placed in the plugins directory have the proper OSGi metadata. It is reproductible locally if you wish to debug and suggest a fix.

Nicolas

>
> Gintas
>
> 2017-06-25 16:57 GMT+02:00 Nicolas Lalevée <[hidden email] <mailto:[hidden email]>>:
>
>> Hi,
>>
>> I worked on the Jenkins jobs to make them work.
>>
>> There was some issues with job dependencies. Following the instructions
>> there [1] worked.
>> I have made the Ivy Job also build the snapshot-bin, like the nightly.
>> Maybe now the nightly is redundant.
>> I made the Ivy-checks build work, some https changed in favor of http.
>> The test matrix is now separated into two builds now, since no Ant
>> installation is common for both Windows and Ubuntu. They are both testing
>> JDK7 and JDK8.
>> The IvyDE job is no longer downloading Ivy’s jar via a hard coded http
>> url. It is relying now on a plugin which copies artifacts around.
>>
>> There are still some error though, most probably not related to the
>> Jenkins configuration:
>> - the Windows build of Ivy is broken, the retrieve doesn’t work [2].
>> - the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is
>> reproductible locally.
>>
>> I’ll look at the IvyDE updatesite job when the build of IvyDE will be
>> fixed.
>>
>> Nicolas
>>
>> [1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <
>> https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>>
>> [2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows <https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows>,
>> jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/ <https://builds.apache.org/>
>> job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
>> [3] https://builds.apache.org/job/IvyDE/315/console <https://builds.apache.org/job/IvyDE/315/console> <
>> https://builds.apache.org/job/IvyDE/315/console <https://builds.apache.org/job/IvyDE/315/console>>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée
In reply to this post by Nicolas Lalevée

> Le 25 juin 2017 à 16:57, Nicolas Lalevée <[hidden email]> a écrit :
>
> Hi,
>
> I worked on the Jenkins jobs to make them work.
>
> There was some issues with job dependencies. Following the instructions there [1] worked.

Then didn’t worked…
Setting the Authorization to « Run as User who Triggered Build » seems to only works when triggered manually, not by the cron. So now I have set « Run as Specific User », me, « hibou ». It now works with the cron. Most probably for me manually too. But maybe not for anybody else…

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Gintautas Grigelionis
In reply to this post by Nicolas Lalevée
2017-06-25 18:23 GMT+02:00 Nicolas Lalevée <[hidden email]>:

> > [2]: please check that unpacking of drops puts files into the correct
> > places (the contents should go into "features" and "plugins" directories
> of
> > PDE).
>
> I have done that already. I even checked that the placed in the plugins
> directory have the proper OSGi metadata. It is reproductible locally if you
> wish to debug and suggest a fix.
>
> Nicolas
>

I am sorry, my memory of what I was doing back in December is incorrect.
I guess I somehow updated
configuration/org.eclipse.equinox.simpleconfigurator/bundles.info or did
something else.
Luckily, there's a simpler way, the dropins should go into their own
special directory, which needs the following change

--- a/build.xml
+++ b/build.xml
@@ -555,11 +555,11 @@ You have to specify the Ivy to install with one of
the following property:
         <delete dir="${basedir}/dependencies/eclipse" failonerror="false"
/>
         <delete dir="${basedir}/dependencies/${eclipse.download.sdk.name}"
failonerror="false" />
         <unzip src="${basedir}/dependencies/${eclipse.download.sdk.name}.zip"
dest="${basedir}/dependencies/" />
-        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name}.zip"
dest="${basedir}/dependencies/" />
-        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name}.zip"
dest="${basedir}/dependencies/" />
-        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name}.zip"
dest="${basedir}/dependencies/" />
-        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name}.zip"
dest="${basedir}/dependencies/" />
-        <unzip src="${basedir}/dependencies/${eclipse.download.zest.name}.zip"
dest="${basedir}/dependencies/" />
+        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name}.zip"
dest="${basedir}/dependencies/eclipse/dropins" />
+        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name}.zip"
dest="${basedir}/dependencies/eclipse/dropins" />
+        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name}.zip"
dest="${basedir}/dependencies/eclipse/dropins" />
+        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name}.zip"
dest="${basedir}/dependencies/eclipse/dropins" />
+        <unzip src="${basedir}/dependencies/${eclipse.download.zest.name}.zip"
dest="${basedir}/dependencies/eclipse/dropins" />
         <move file="${basedir}/dependencies/eclipse"
tofile="${basedir}/dependencies/${eclipse.download.sdk.name}" />
     </target>

I don't understand why "jenkins-prepare" (neé "hudson-prepare") would skip
"download-ivy" (because "jenkins-install-ivy" sets ivy.jar.url to "" rather
than ${hudson.ivy.jar.url}) and fail?

Gintas

> 2017-06-25 16:57 GMT+02:00 Nicolas Lalevée <[hidden email]
> <mailto:[hidden email]>>:
> >
> >> Hi,
> >>
> >> I worked on the Jenkins jobs to make them work.
> >>
> >> There was some issues with job dependencies. Following the instructions
> >> there [1] worked.
> >> I have made the Ivy Job also build the snapshot-bin, like the nightly.
> >> Maybe now the nightly is redundant.
> >> I made the Ivy-checks build work, some https changed in favor of http.
> >> The test matrix is now separated into two builds now, since no Ant
> >> installation is common for both Windows and Ubuntu. They are both
> testing
> >> JDK7 and JDK8.
> >> The IvyDE job is no longer downloading Ivy’s jar via a hard coded http
> >> url. It is relying now on a plugin which copies artifacts around.
> >>
> >> There are still some error though, most probably not related to the
> >> Jenkins configuration:
> >> - the Windows build of Ivy is broken, the retrieve doesn’t work [2].
> >> - the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is
> >> reproductible locally.
> >>
> >> I’ll look at the IvyDE updatesite job when the build of IvyDE will be
> >> fixed.
> >>
> >> Nicolas
> >>
> >> [1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization
> <
> >> https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <
> https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>>
> >> [2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows <
> https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows>,
> >> jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/ <
> https://builds.apache.org/>
> >> job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
> >> [3] https://builds.apache.org/job/IvyDE/315/console <
> https://builds.apache.org/job/IvyDE/315/console> <
> >> https://builds.apache.org/job/IvyDE/315/console <
> https://builds.apache.org/job/IvyDE/315/console>>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée

> I am sorry, my memory of what I was doing back in December is incorrect.
> I guess I somehow updated
> configuration/org.eclipse.equinox.simpleconfigurator/bundles.info or did
> something else.
> Luckily, there's a simpler way, the dropins should go into their own
> special directory, which needs the following change
>
> --- a/build.xml
> +++ b/build.xml
> @@ -555,11 +555,11 @@ You have to specify the Ivy to install with one of
> the following property:
>         <delete dir="${basedir}/dependencies/eclipse" failonerror="false"
> />
>         <delete dir="${basedir}/dependencies/${eclipse.download.sdk.name}"
> failonerror="false" />
>         <unzip src="${basedir}/dependencies/${eclipse.download.sdk.name}.zip"
> dest="${basedir}/dependencies/" />
> -        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name}.zip"
> dest="${basedir}/dependencies/" />
> -        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name}.zip"
> dest="${basedir}/dependencies/" />
> -        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name}.zip"
> dest="${basedir}/dependencies/" />
> -        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name}.zip"
> dest="${basedir}/dependencies/" />
> -        <unzip src="${basedir}/dependencies/${eclipse.download.zest.name}.zip"
> dest="${basedir}/dependencies/" />
> +        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name}.zip"
> dest="${basedir}/dependencies/eclipse/dropins" />
> +        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name}.zip"
> dest="${basedir}/dependencies/eclipse/dropins" />
> +        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name}.zip"
> dest="${basedir}/dependencies/eclipse/dropins" />
> +        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name}.zip"
> dest="${basedir}/dependencies/eclipse/dropins" />
> +        <unzip src="${basedir}/dependencies/${eclipse.download.zest.name}.zip"
> dest="${basedir}/dependencies/eclipse/dropins" />
>         <move file="${basedir}/dependencies/eclipse"
> tofile="${basedir}/dependencies/${eclipse.download.sdk.name}" />
>     </target>

I don’t understand what you are suggesting. The issue at hand is resolving the Ivy jar. The suggested patch is about installing eclipse plugins in dropins. But the build is still not finding the ivy plugin.

> I don't understand why "jenkins-prepare" (neé "hudson-prepare") would skip
> "download-ivy" (because "jenkins-install-ivy" sets ivy.jar.url to "" rather
> than ${hudson.ivy.jar.url}) and fail?

Again, sorry, but I don’t understand what you are talking about. Are you seing the same error as the one Jenkins is showing ? The error is occurring with the eclipse build started, way later.

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Jaikiran Pai
In reply to this post by Nicolas Lalevée
The retrieve failure looks like a genuine bug which might be related to one of the changes I had done in one of the PRs a while back. I’ll take a look at this today.

-Jaikiran
On 25-Jun-2017, at 8:27 PM, Nicolas Lalevée <[hidden email]> wrote:

Hi,

I worked on the Jenkins jobs to make them work.

There was some issues with job dependencies. Following the instructions there [1] worked.
I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.
I made the Ivy-checks build work, some https changed in favor of http.
The test matrix is now separated into two builds now, since no Ant installation is common for both Windows and Ubuntu. They are both testing JDK7 and JDK8.
The IvyDE job is no longer downloading Ivy’s jar via a hard coded http url. It is relying now on a plugin which copies artifacts around.

There are still some error though, most probably not related to the Jenkins configuration:
- the Windows build of Ivy is broken, the retrieve doesn’t work [2].
- the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is reproductible locally.

I’ll look at the IvyDE updatesite job when the build of IvyDE will be fixed.

Nicolas

[1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>
[2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
[3] https://builds.apache.org/job/IvyDE/315/console <https://builds.apache.org/job/IvyDE/315/console>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Gintautas Grigelionis
In reply to this post by Nicolas Lalevée
If I understand the process correctly, Jenkins executes "ant
prepare-jenkins" first, which does not work locally any more, because the
property used to figure out where to get Ivy jar from is set to an empty
string. I wondered what was the reason for that change. Actually, when I
look at it again, the change is even more drastic. Previously,
"hudson-install-ivy" would depend on both "hudson-install-ivy-jar" (setting
"ivy.jar.url") and "hudson-install-ivy-release" (setting "ivy.version" if
"hudson.ivy.jar.url" was not available). In addition, "baseLocation" was
set. The new target "jenkins-install-ivy" does not make sense to me.

In the next step, Jenkins executes "ant dist checkstyle rat" which fails
because of "missing" Eclipse plugins. That is apparently caused by dropins
being unpacked in the wrong place in the first build stage
("prepare-jenkins"). In my first suggestion, I was obviously wrong about
what that place should be. I do recall using getting Eclipse to work
locally with an older IvyDE build.xml, but I did not document whether that
involved any extra steps. However, Eclipse is able to bootstrap itself
locally when dropins are unpacked in "dropins" directory, see Eclipse
documentation [1]. That's what the patch is for.

Gintas

[1] https://wiki.eclipse.org/Equinox/p2/Getting_Started#Dropins


2017-06-25 22:27 GMT+02:00 Nicolas Lalevée <[hidden email]>:

>
> > I am sorry, my memory of what I was doing back in December is incorrect.
> > I guess I somehow updated
> > configuration/org.eclipse.equinox.simpleconfigurator/bundles.info or did
> > something else.
> > Luckily, there's a simpler way, the dropins should go into their own
> > special directory, which needs the following change
> >
> > --- a/build.xml
> > +++ b/build.xml
> > @@ -555,11 +555,11 @@ You have to specify the Ivy to install with one of
> > the following property:
> >         <delete dir="${basedir}/dependencies/eclipse"
> failonerror="false"
> > />
> >         <delete dir="${basedir}/dependencies/${eclipse.download.sdk.name
> }"
> > failonerror="false" />
> >         <unzip src="${basedir}/dependencies/${eclipse.download.sdk.name
> }.zip"
> > dest="${basedir}/dependencies/" />
> > -        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name
> }.zip"
> > dest="${basedir}/dependencies/" />
> > -        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name
> }.zip"
> > dest="${basedir}/dependencies/" />
> > -        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name
> }.zip"
> > dest="${basedir}/dependencies/" />
> > -        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name
> }.zip"
> > dest="${basedir}/dependencies/" />
> > -        <unzip src="${basedir}/dependencies/${
> eclipse.download.zest.name}.zip"
> > dest="${basedir}/dependencies/" />
> > +        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name
> }.zip"
> > dest="${basedir}/dependencies/eclipse/dropins" />
> > +        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name
> }.zip"
> > dest="${basedir}/dependencies/eclipse/dropins" />
> > +        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name
> }.zip"
> > dest="${basedir}/dependencies/eclipse/dropins" />
> > +        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name
> }.zip"
> > dest="${basedir}/dependencies/eclipse/dropins" />
> > +        <unzip src="${basedir}/dependencies/${
> eclipse.download.zest.name}.zip"
> > dest="${basedir}/dependencies/eclipse/dropins" />
> >         <move file="${basedir}/dependencies/eclipse"
> > tofile="${basedir}/dependencies/${eclipse.download.sdk.name}" />
> >     </target>
>
> I don’t understand what you are suggesting. The issue at hand is resolving
> the Ivy jar. The suggested patch is about installing eclipse plugins in
> dropins. But the build is still not finding the ivy plugin.
>
> > I don't understand why "jenkins-prepare" (neé "hudson-prepare") would
> skip
> > "download-ivy" (because "jenkins-install-ivy" sets ivy.jar.url to ""
> rather
> > than ${hudson.ivy.jar.url}) and fail?
>
> Again, sorry, but I don’t understand what you are talking about. Are you
> seing the same error as the one Jenkins is showing ? The error is occurring
> with the eclipse build started, way later.
>
> Nicolas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée

> Le 26 juin 2017 à 07:46, Gintautas Grigelionis <[hidden email]> a écrit :
>
> If I understand the process correctly, Jenkins executes "ant
> prepare-jenkins" first, which does not work locally any more, because the
> property used to figure out where to get Ivy jar from is set to an empty
> string. I wondered what was the reason for that change. Actually, when I
> look at it again, the change is even more drastic. Previously,
> "hudson-install-ivy" would depend on both "hudson-install-ivy-jar" (setting
> "ivy.jar.url") and "hudson-install-ivy-release" (setting "ivy.version" if
> "hudson.ivy.jar.url" was not available). In addition, "baseLocation" was
> set. The new target "jenkins-install-ivy" does not make sense to me.

This is because of the change in the configuration of Jenkins Job, as I briefly explained in the first mail. Now the Ivy artifact is not downloaded via an url, there is a Jenkins plugin, a build step, which handle the copy of the artifacts. Relying on this internal copy of artifact seems more stable that relying on a http/https url. The jenkins-install-ivy target is then calling install-ivy with a path rather than an url.

Now the jenkins-* targets are really dedicated to being ran on jenkins and are not suited for a local environment. That is why I renamed the targets to download and setup an eclipse, removing the jenkins- prefix: these can be used in a dev environment.
If you do that: ant prepare-eclipse
And do the install: ant install-ivy -Divy.jar.url=https://builds.apache.org/view/A/view/Ant/job/Ivy/lastSuccessfulBuild/artifact/build/artifact/jars/ivy.jar
Then you will be able to reproduce the issue. At least I can.

> In the next step, Jenkins executes "ant dist checkstyle rat" which fails
> because of "missing" Eclipse plugins. That is apparently caused by dropins
> being unpacked in the wrong place in the first build stage
> ("prepare-jenkins"). In my first suggestion, I was obviously wrong about
> what that place should be. I do recall using getting Eclipse to work
> locally with an older IvyDE build.xml, but I did not document whether that
> involved any extra steps. However, Eclipse is able to bootstrap itself
> locally when dropins are unpacked in "dropins" directory, see Eclipse
> documentation [1]. That's what the patch is for.

I don’t have much experience with using dropins, so if you think this will help make the build more stable, you’re welcomed to open a PR.

Nicolas

>
> Gintas
>
> [1] https://wiki.eclipse.org/Equinox/p2/Getting_Started#Dropins
>
>
> 2017-06-25 22:27 GMT+02:00 Nicolas Lalevée <[hidden email]>:
>
>>
>>> I am sorry, my memory of what I was doing back in December is incorrect.
>>> I guess I somehow updated
>>> configuration/org.eclipse.equinox.simpleconfigurator/bundles.info or did
>>> something else.
>>> Luckily, there's a simpler way, the dropins should go into their own
>>> special directory, which needs the following change
>>>
>>> --- a/build.xml
>>> +++ b/build.xml
>>> @@ -555,11 +555,11 @@ You have to specify the Ivy to install with one of
>>> the following property:
>>>        <delete dir="${basedir}/dependencies/eclipse"
>> failonerror="false"
>>> />
>>>        <delete dir="${basedir}/dependencies/${eclipse.download.sdk.name
>> }"
>>> failonerror="false" />
>>>        <unzip src="${basedir}/dependencies/${eclipse.download.sdk.name
>> }.zip"
>>> dest="${basedir}/dependencies/" />
>>> -        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name
>> }.zip"
>>> dest="${basedir}/dependencies/" />
>>> -        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name
>> }.zip"
>>> dest="${basedir}/dependencies/" />
>>> -        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name
>> }.zip"
>>> dest="${basedir}/dependencies/" />
>>> -        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name
>> }.zip"
>>> dest="${basedir}/dependencies/" />
>>> -        <unzip src="${basedir}/dependencies/${
>> eclipse.download.zest.name}.zip"
>>> dest="${basedir}/dependencies/" />
>>> +        <unzip src="${basedir}/dependencies/${eclipse.download.wtp.name
>> }.zip"
>>> dest="${basedir}/dependencies/eclipse/dropins" />
>>> +        <unzip src="${basedir}/dependencies/${eclipse.download.emf.name
>> }.zip"
>>> dest="${basedir}/dependencies/eclipse/dropins" />
>>> +        <unzip src="${basedir}/dependencies/${eclipse.download.xsd.name
>> }.zip"
>>> dest="${basedir}/dependencies/eclipse/dropins" />
>>> +        <unzip src="${basedir}/dependencies/${eclipse.download.gef.name
>> }.zip"
>>> dest="${basedir}/dependencies/eclipse/dropins" />
>>> +        <unzip src="${basedir}/dependencies/${
>> eclipse.download.zest.name}.zip"
>>> dest="${basedir}/dependencies/eclipse/dropins" />
>>>        <move file="${basedir}/dependencies/eclipse"
>>> tofile="${basedir}/dependencies/${eclipse.download.sdk.name}" />
>>>    </target>
>>
>> I don’t understand what you are suggesting. The issue at hand is resolving
>> the Ivy jar. The suggested patch is about installing eclipse plugins in
>> dropins. But the build is still not finding the ivy plugin.
>>
>>> I don't understand why "jenkins-prepare" (neé "hudson-prepare") would
>> skip
>>> "download-ivy" (because "jenkins-install-ivy" sets ivy.jar.url to ""
>> rather
>>> than ${hudson.ivy.jar.url}) and fail?
>>
>> Again, sorry, but I don’t understand what you are talking about. Are you
>> seing the same error as the one Jenkins is showing ? The error is occurring
>> with the eclipse build started, way later.
>>
>> Nicolas
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Gintautas Grigelionis
2017-06-26 13:54 GMT+02:00 Nicolas Lalevée <[hidden email]>:

>
> > Le 26 juin 2017 à 07:46, Gintautas Grigelionis <[hidden email]>
> a écrit :
> >
> > If I understand the process correctly, Jenkins executes "ant
> > prepare-jenkins" first, which does not work locally any more, because the
> > property used to figure out where to get Ivy jar from is set to an empty
> > string. I wondered what was the reason for that change. Actually, when I
> > look at it again, the change is even more drastic. Previously,
> > "hudson-install-ivy" would depend on both "hudson-install-ivy-jar"
> (setting
> > "ivy.jar.url") and "hudson-install-ivy-release" (setting "ivy.version" if
> > "hudson.ivy.jar.url" was not available). In addition, "baseLocation" was
> > set. The new target "jenkins-install-ivy" does not make sense to me.
>
> This is because of the change in the configuration of Jenkins Job, as I
> briefly explained in the first mail. Now the Ivy artifact is not downloaded
> via an url, there is a Jenkins plugin, a build step, which handle the copy
> of the artifacts. Relying on this internal copy of artifact seems more
> stable that relying on a http/https url. The jenkins-install-ivy target is
> then calling install-ivy with a path rather than an url.
>

Sorry, I misunderstood the brief description. In my world,
"prepare-eclipse" should also download Ivy and add it to PDE.


> Now the jenkins-* targets are really dedicated to being ran on jenkins and
> are not suited for a local environment. That is why I renamed the targets
> to download and setup an eclipse, removing the jenkins- prefix: these can
> be used in a dev environment.
> If you do that: ant prepare-eclipse
> And do the install: ant install-ivy -Divy.jar.url=https://builds.
> apache.org/view/A/view/Ant/job/Ivy/lastSuccessfulBuild/
> artifact/build/artifact/jars/ivy.jar
> Then you will be able to reproduce the issue. At least I can.
>

Thanks, now I get the idea.

> In the next step, Jenkins executes "ant dist checkstyle rat" which fails
> > because of "missing" Eclipse plugins. That is apparently caused by
> dropins
> > being unpacked in the wrong place in the first build stage
> > ("prepare-jenkins"). In my first suggestion, I was obviously wrong about
> > what that place should be. I do recall using getting Eclipse to work
> > locally with an older IvyDE build.xml, but I did not document whether
> that
> > involved any extra steps. However, Eclipse is able to bootstrap itself
> > locally when dropins are unpacked in "dropins" directory, see Eclipse
> > documentation [1]. That's what the patch is for.
>
> I don’t have much experience with using dropins, so if you think this will
> help make the build more stable, you’re welcomed to open a PR.
>

Dropins (or "drops" as they're called in the distributions) is what IvyDE
builds were based on all the time. I am still confounded why it worked
before and stopped working now. Yet, reading the documentation I became
convinced that the dropins must be unpacked in another location. So I just
wanted to apologize for an erroneous suggestion and share some information
in order for it to be independently confirmed. I may open a PR, but since
you're working on this, I assumed that you could move faster than me.
There's another issue to be resolved for local builds to work hassle-free
-- currently, they need an environment variable BUILD_NUMBER set up by
Jenkins.

Gintas


> > [1] https://wiki.eclipse.org/Equinox/p2/Getting_Started#Dropins
> >
> >
> > 2017-06-25 22:27 GMT+02:00 Nicolas Lalevée <[hidden email]>:
> >
> >>
> >>> I am sorry, my memory of what I was doing back in December is
> incorrect.
> >>> I guess I somehow updated
> >>> configuration/org.eclipse.equinox.simpleconfigurator/bundles.info or
> did
> >>> something else.
> >>> Luckily, there's a simpler way, the dropins should go into their own
> >>> special directory, which needs the following change
> >>>
> >>> --- a/build.xml
> >>> +++ b/build.xml
> >>> @@ -555,11 +555,11 @@ You have to specify the Ivy to install with one
> of
> >>> the following property:
> >>>        <delete dir="${basedir}/dependencies/eclipse"
> >> failonerror="false"
> >>> />
> >>>        <delete dir="${basedir}/dependencies/${
> eclipse.download.sdk.name
> >> }"
> >>> failonerror="false" />
> >>>        <unzip src="${basedir}/dependencies/${eclipse.download.sdk.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> -        <unzip src="${basedir}/dependencies/${
> eclipse.download.wtp.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> -        <unzip src="${basedir}/dependencies/${
> eclipse.download.emf.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> -        <unzip src="${basedir}/dependencies/${
> eclipse.download.xsd.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> -        <unzip src="${basedir}/dependencies/${
> eclipse.download.gef.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> -        <unzip src="${basedir}/dependencies/${
> >> eclipse.download.zest.name}.zip"
> >>> dest="${basedir}/dependencies/" />
> >>> +        <unzip src="${basedir}/dependencies/${
> eclipse.download.wtp.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/eclipse/dropins" />
> >>> +        <unzip src="${basedir}/dependencies/${
> eclipse.download.emf.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/eclipse/dropins" />
> >>> +        <unzip src="${basedir}/dependencies/${
> eclipse.download.xsd.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/eclipse/dropins" />
> >>> +        <unzip src="${basedir}/dependencies/${
> eclipse.download.gef.name
> >> }.zip"
> >>> dest="${basedir}/dependencies/eclipse/dropins" />
> >>> +        <unzip src="${basedir}/dependencies/${
> >> eclipse.download.zest.name}.zip"
> >>> dest="${basedir}/dependencies/eclipse/dropins" />
> >>>        <move file="${basedir}/dependencies/eclipse"
> >>> tofile="${basedir}/dependencies/${eclipse.download.sdk.name}" />
> >>>    </target>
> >>
> >> I don’t understand what you are suggesting. The issue at hand is
> resolving
> >> the Ivy jar. The suggested patch is about installing eclipse plugins in
> >> dropins. But the build is still not finding the ivy plugin.
> >>
> >>> I don't understand why "jenkins-prepare" (neé "hudson-prepare") would
> >> skip
> >>> "download-ivy" (because "jenkins-install-ivy" sets ivy.jar.url to ""
> >> rather
> >>> than ${hudson.ivy.jar.url}) and fail?
> >>
> >> Again, sorry, but I don’t understand what you are talking about. Are you
> >> seing the same error as the one Jenkins is showing ? The error is
> occurring
> >> with the eclipse build started, way later.
> >>
> >> Nicolas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée

> Dropins (or "drops" as they're called in the distributions) is what IvyDE
> builds were based on all the time. I am still confounded why it worked
> before and stopped working now. Yet, reading the documentation I became
> convinced that the dropins must be unpacked in another location. So I just
> wanted to apologize for an erroneous suggestion and share some information
> in order for it to be independently confirmed. I may open a PR, but since
> you're working on this, I assumed that you could move faster than me.

Actually I am not looking into it at the present time. I’ll get into it at some point, if it is not fixed before. So if you think you can solve this, I’ll welcome your help.

On the side note, I am now an Intellij user, I only start Eclipse to build IvyDE. And I haven’t done Eclipse plugin development in years, so I am not fast. :)

> There's another issue to be resolved for local builds to work hassle-free
> -- currently, they need an environment variable BUILD_NUMBER set up by
> Jenkins.

Is it required ? This env variable is only needed when calling the jenkins target. In a dev environment we shouldn't call them.

I think in this mailing list we told to run some jenkins target to make easier the setup of the dev environment. But it as actually to call the targets which download and setup Eclipse. That’s why I renamed them, so we should continue to make the proper distinction to the tweaks for Jenkins, and the dev targets.

Nicolas

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Jaikiran Pai
In reply to this post by Jaikiran Pai
I pushed a fix for this a few days back and the job is now functioning correctly.

-Jaikiran
On 26-Jun-2017, at 8:15 AM, Jaikiran Pai <[hidden email]> wrote:

The retrieve failure looks like a genuine bug which might be related to one of the changes I had done in one of the PRs a while back. I’ll take a look at this today.

-Jaikiran
On 25-Jun-2017, at 8:27 PM, Nicolas Lalevée <[hidden email]> wrote:

Hi,

I worked on the Jenkins jobs to make them work.

There was some issues with job dependencies. Following the instructions there [1] worked.
I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.
I made the Ivy-checks build work, some https changed in favor of http.
The test matrix is now separated into two builds now, since no Ant installation is common for both Windows and Ubuntu. They are both testing JDK7 and JDK8.
The IvyDE job is no longer downloading Ivy’s jar via a hard coded http url. It is relying now on a plugin which copies artifacts around.

There are still some error though, most probably not related to the Jenkins configuration:
- the Windows build of Ivy is broken, the retrieve doesn’t work [2].
- the IvyDE is failing to resolve the OSGi metadata of Ivy [3]. It is reproductible locally.

I’ll look at the IvyDE updatesite job when the build of IvyDE will be fixed.

Nicolas

[1] https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization <https://cwiki.apache.org/confluence/display/INFRA/Job+Authorization>
[2] https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console <https://builds.apache.org/job/Ivy-tests-Windows/OS=Windows,jdk=JDK%201.7%20(latest)/2/console>
[3] https://builds.apache.org/job/IvyDE/315/console <https://builds.apache.org/job/IvyDE/315/console>




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Jaikiran Pai
In reply to this post by Nicolas Lalevée

On 25-Jun-2017, at 8:27 PM, Nicolas Lalevée <[hidden email]> wrote:

> I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.


I think yes, the nightly job is now probably redundant - which is fine, we can always direct users to this new job for the latest snapshots. The only way this job differs from the nightly one now is that the nightly one currently publishes the latest (site) docs too. I guess that’s not a big thing since we will soon be publishing the latest site docs live on the Ant site (under the “master” section).

-Jaikiran
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Nicolas Lalevée

> Le 29 juin 2017 à 07:48, Jaikiran Pai <[hidden email]> a écrit :
>
>
> On 25-Jun-2017, at 8:27 PM, Nicolas Lalevée <[hidden email]> wrote:
>
>> I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.
>
>
> I think yes, the nightly job is now probably redundant - which is fine, we can always direct users to this new job for the latest snapshots. The only way this job differs from the nightly one now is that the nightly one currently publishes the latest (site) docs too. I guess that’s not a big thing since we will soon be publishing the latest site docs live on the Ant site (under the “master” section).

I have done that. For some reason it took a very long time to get actually published, but it is there now:
http://ant.apache.org/ivy/history/master/index.html <http://ant.apache.org/ivy/history/master/index.html>

I have also update the doc about the site:
http://svn.apache.org/repos/asf/ant/site/README.txt <http://svn.apache.org/repos/asf/ant/site/README.txt>

Nicolas

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Jenkins Jobs of Ivy/IvyDE

Jaikiran Pai
In reply to this post by Jaikiran Pai
I have now disabled the nightly job[1] and added a note to the job's description stating that the latest binaries can be obtained from the other Ivy job[2].


[1] https://builds.apache.org/job/Ivy-NightlyDistribution/
[2] https://builds.apache.org/job/Ivy/lastSuccessfulBuild/

-Jaikiran
On 29-Jun-2017, at 11:18 AM, Jaikiran Pai <[hidden email]> wrote:


On 25-Jun-2017, at 8:27 PM, Nicolas Lalevée <[hidden email]> wrote:

> I have made the Ivy Job also build the snapshot-bin, like the nightly. Maybe now the nightly is redundant.


I think yes, the nightly job is now probably redundant - which is fine, we can always direct users to this new job for the latest snapshots. The only way this job differs from the nightly one now is that the nightly one currently publishes the latest (site) docs too. I guess that’s not a big thing since we will soon be publishing the latest site docs live on the Ant site (under the “master” section).

-Jaikiran


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...