Ivy - Goals for the upcoming release?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Ivy - Goals for the upcoming release?

Jaikiran Pai
The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.

I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).

I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.

I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.


[1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20IVY%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20affectedVersion%20in%20(2.1.0%2C%202.2.0%2C%202.3.0%2C%202.4.0%2C%202.4.0-RC1)%20AND%20updated%20%3E%3D%202014-01-01%20ORDER%20BY%20updated%20DESC

-Jaikiran
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Jaikiran Pai
One thing I forgot to note is that we need to do a similar thing for IvyDE. I haven’t looked at the JIRA issues for that project [1] and am not familiar with Eclipse in general. So someone more familiar with it would have to look into those. Having said that, Gintas has helped with getting the IvyDE project itself come to life recently, so I guess we can target to do something similar in terms of release goals, for that project too.

-Jaikiran


[1] https://issues.apache.org/jira/browse/IVYDE
On 21-May-2017, at 7:58 PM, J Pai <[hidden email]> wrote:

The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.

I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).

I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.

I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.


[1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20IVY%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20affectedVersion%20in%20(2.1.0%2C%202.2.0%2C%202.3.0%2C%202.4.0%2C%202.4.0-RC1)%20AND%20updated%20%3E%3D%202014-01-01%20ORDER%20BY%20updated%20DESC

-Jaikiran


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Gintautas Grigelionis
Now that Ant is upgraded, I suggest converting tests to JUnit 4 (which
requires at least Ant 1.9.5, if I'm not mistaken). I can submit a patch
pretty soon.

Introducing generics throughout (hey, that's a Java 5 feature!) would make
the code easier to read (I've seen incorrect comments of what goes into a
Map somewhere in the code). Unfortunately, in Ivy's case that requires even
some changes in the API (can't have arrays of generics) and a few other
design problems.

As for IvyDE, I was looking into getting rid of any deprecations and fixing
the build.xml so that the dependencies can be retrieved by Ivy from Eclipse
update sites. However, the honour of getting the current builds to work
goes to Nicolas.

Gintas

2017-05-21 16:41 GMT+02:00 J Pai <[hidden email]>:

> One thing I forgot to note is that we need to do a similar thing for
> IvyDE. I haven’t looked at the JIRA issues for that project [1] and am not
> familiar with Eclipse in general. So someone more familiar with it would
> have to look into those. Having said that, Gintas has helped with getting
> the IvyDE project itself come to life recently, so I guess we can target to
> do something similar in terms of release goals, for that project too.
>
> -Jaikiran
>
>
> [1] https://issues.apache.org/jira/browse/IVYDE
> On 21-May-2017, at 7:58 PM, J Pai <[hidden email]> wrote:
>
> The past few days I’ve sent some PRs for bug fixes and some enhancements
> in preparation for the proposed release of Ivy. Thanks Nicolas for
> reviewing them and merging those that were good enough.
>
> I’ve been using this JIRA filter [1] to narrow down on the issues to look
> into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0,
> 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it
> should cover most of the issues that we probably should look into (doesn’t
> necessarily mean fix all of them, but just to check if any of them are big
> enough to focus on).
>
> I’ve also sent one PR for upgrading our internal library dependencies and
> plan to send a couple more for similar upgrades. My intention is to use the
> latest released versions of these dependencies instead of sticking with
> dependencies that are years old. My intention _isn’t_ to upgrade to a
> version that isn’t API backward compatible, so these upgrades are mostly
> bug fixes and should be “drop-in upgrades”.
>
> I would be really glad to hear any thoughts about these changes or any
> other/different plans, that can get us to a releasable state in the near
> future, especially from members/users who have been involved with Ant/Ivy
> project in the past or present. Ultimately, I think if we can agree on a
> goal for the upcoming release, it will help release something that will set
> the right expectation with the end users when it comes to using it. My
> opinion is that we consider this release to push out some bug fixes and
> internal upgrades and _not_ introduce any major features unless those are
> reasonably quick to implement. Once this release is done and (hopefully
> some of the) community gets back behind the Ivy project, we can always
> introduce major features in subsequent releases.
>
>
> [1] <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%">https://issues.apache.org/jira/issues/?jql=project%20%3D%
> 20IVY%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%
> 2C%20Reopened)%20AND%20affectedVersion%20in%20(2.1.0%2C%202.
> 2.0%2C%202.3.0%2C%202.4.0%2C%202.4.0-RC1)%20AND%20updated%
> 20%3E%3D%202014-01-01%20ORDER%20BY%20updated%20DESC
>
> -Jaikiran
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Nicolas Lalevée
In reply to this post by Jaikiran Pai

> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>
> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>
> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>
> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>
> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.

Sounds like a good plan.

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Nicolas Lalevée
In reply to this post by Gintautas Grigelionis

> Le 21 mai 2017 à 18:34, Gintautas Grigelionis <[hidden email]> a écrit :
>
> Now that Ant is upgraded, I suggest converting tests to JUnit 4 (which
> requires at least Ant 1.9.5, if I'm not mistaken). I can submit a patch
> pretty soon.
>
> Introducing generics throughout (hey, that's a Java 5 feature!) would make
> the code easier to read (I've seen incorrect comments of what goes into a
> Map somewhere in the code). Unfortunately, in Ivy's case that requires even
> some changes in the API (can't have arrays of generics) and a few other
> design problems.

I don’t want to break your enthusiasm, but we should try to keep the API as stable as possible. I know the API is very wide, a lot is exposed, but we should be really careful at not breaking downside projects.

Also, PR are starting to pill up thanks to the good work of Jaikiran, so if there is big code change like adding a lot of generic in the code, it would be nice to wait a little bit so that merges will be easier.

> As for IvyDE, I was looking into getting rid of any deprecations and fixing
> the build.xml so that the dependencies can be retrieved by Ivy from Eclipse
> update sites. However, the honour of getting the current builds to work
> goes to Nicolas.

Any contribution to upgrade the dependencies of IvyDE is welcomed. For instance, I don’t even know if a PDE build can still be done with Eclipse 4.

Also, if some dependencies are too hard to get, like the GEF and Zest, we could get rid of them by stopping to support the « resolve visualizer ». And it could be decided that it is not abandoned but just temporarily disabled until someone find a way to correctly build it.

Nicolas

>
> Gintas
>
> 2017-05-21 16:41 GMT+02:00 J Pai <[hidden email]>:
>
>> One thing I forgot to note is that we need to do a similar thing for
>> IvyDE. I haven’t looked at the JIRA issues for that project [1] and am not
>> familiar with Eclipse in general. So someone more familiar with it would
>> have to look into those. Having said that, Gintas has helped with getting
>> the IvyDE project itself come to life recently, so I guess we can target to
>> do something similar in terms of release goals, for that project too.
>>
>> -Jaikiran
>>
>>
>> [1] https://issues.apache.org/jira/browse/IVYDE
>> On 21-May-2017, at 7:58 PM, J Pai <[hidden email]> wrote:
>>
>> The past few days I’ve sent some PRs for bug fixes and some enhancements
>> in preparation for the proposed release of Ivy. Thanks Nicolas for
>> reviewing them and merging those that were good enough.
>>
>> I’ve been using this JIRA filter [1] to narrow down on the issues to look
>> into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0,
>> 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it
>> should cover most of the issues that we probably should look into (doesn’t
>> necessarily mean fix all of them, but just to check if any of them are big
>> enough to focus on).
>>
>> I’ve also sent one PR for upgrading our internal library dependencies and
>> plan to send a couple more for similar upgrades. My intention is to use the
>> latest released versions of these dependencies instead of sticking with
>> dependencies that are years old. My intention _isn’t_ to upgrade to a
>> version that isn’t API backward compatible, so these upgrades are mostly
>> bug fixes and should be “drop-in upgrades”.
>>
>> I would be really glad to hear any thoughts about these changes or any
>> other/different plans, that can get us to a releasable state in the near
>> future, especially from members/users who have been involved with Ant/Ivy
>> project in the past or present. Ultimately, I think if we can agree on a
>> goal for the upcoming release, it will help release something that will set
>> the right expectation with the end users when it comes to using it. My
>> opinion is that we consider this release to push out some bug fixes and
>> internal upgrades and _not_ introduce any major features unless those are
>> reasonably quick to implement. Once this release is done and (hopefully
>> some of the) community gets back behind the Ivy project, we can always
>> introduce major features in subsequent releases.
>>
>>
>> [1] <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%">https://issues.apache.org/jira/issues/?jql=project%20%3D%
>> 20IVY%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%
>> 2C%20Reopened)%20AND%20affectedVersion%20in%20(2.1.0%2C%202.
>> 2.0%2C%202.3.0%2C%202.4.0%2C%202.4.0-RC1)%20AND%20updated%
>> 20%3E%3D%202014-01-01%20ORDER%20BY%20updated%20DESC
>>
>> -Jaikiran
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Nicolas Lalevée
In reply to this post by Nicolas Lalevée
One thing though.

This revival of the community has been triggered by the comments on this issue:
https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
It would be a shame if it is not fixed within the next release.

The issue and the fix are not easy to understand. Any review will be welcomed.

Nicolas

> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>
>
>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>
>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>
>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>
>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>
>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>
> Sounds like a good plan.
>
> Nicolas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Gintautas Grigelionis
In reply to this post by Nicolas Lalevée
2017-05-21 19:00 GMT+02:00 Nicolas Lalevée <[hidden email]>:

>
> I don’t want to break your enthusiasm, but we should try to keep the API
> as stable as possible. I know the API is very wide, a lot is exposed, but
> we should be really careful at not breaking downside projects
>

I think generics in IvyDE is not a big deal. I dived into Ivy because of
some inconsistencies (Map vs Properties), and I'm not very enthusiastic
about it... no ETA at this point. Maybe a branch after 2.5.0...


> > As for IvyDE, I was looking into getting rid of any deprecations and
> fixing
> > the build.xml so that the dependencies can be retrieved by Ivy from
> Eclipse
> > update sites. However, the honour of getting the current builds to work
> > goes to Nicolas.
>
> Any contribution to upgrade the dependencies of IvyDE is welcomed. For
> instance, I don’t even know if a PDE build can still be done with Eclipse 4.
>
> Also, if some dependencies are too hard to get, like the GEF and Zest, we
> could get rid of them by stopping to support the « resolve visualizer ».
> And it could be decided that it is not abandoned but just temporarily
> disabled until someone find a way to correctly build it.
>
> PDE is available, the drops are not, but the dependencies are not a
problem by themselves. I almost made Equinox to do the job, but the
incantations look pretty terrifying on the command line already and then
they must be wrapped in Ant exec... While researching it I saw discussions
about using Maven with Eclipse components, but updatesite resolver looks
like the cleanest approach.

Gintas
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Jaikiran Pai
In reply to this post by Nicolas Lalevée
I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).

Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:

<ivy-module version="2.0">
    <info organisation="org"
          module="hello-world"
          revision="1.2.3"
         
    />
    <configurations>
        <conf name="conf1"/>
        <conf name="conf2"/>
    </configurations>
    <dependencies>
        <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
        <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
    </dependencies>
</ivy-module>

Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:

<ivy-module version="1.0">
  <info organisation="org"
         module="module2"
         revision="1.0"
  />
    <publications>
        <artifact/>
    </publications>
  <dependencies>
    <dependency org="org" name="module1" rev="2.0"/>
  </dependencies>
</ivy-module>

So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).

Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.

Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.

So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.

Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.

[1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
[2] https://www.mail-archive.com/dev@.../msg45402.html

-Jaikiran

On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:

One thing though.

This revival of the community has been triggered by the comments on this issue:
https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
It would be a shame if it is not fixed within the next release.

The issue and the fix are not easy to understand. Any review will be welcomed.

Nicolas

> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>
>
>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>
>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>
>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>
>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>
>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>
> Sounds like a good plan.
>
> Nicolas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Jaikiran Pai
One other thing about this issue - this is reproducible (as shown in the test case) with static revisions too and isn’t specific to any dynamic revision resolution.

-Jaikiran
On 22-May-2017, at 11:02 AM, J Pai <[hidden email]> wrote:

I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).

Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:

<ivy-module version="2.0">
   <info organisation="org"
         module="hello-world"
         revision="1.2.3"

   />
   <configurations>
       <conf name="conf1"/>
       <conf name="conf2"/>
   </configurations>
   <dependencies>
       <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
       <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
   </dependencies>
</ivy-module>

Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:

<ivy-module version="1.0">
 <info organisation="org"
        module="module2"
        revision="1.0"
 />
   <publications>
       <artifact/>
   </publications>
 <dependencies>
   <dependency org="org" name="module1" rev="2.0"/>
 </dependencies>
</ivy-module>

So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).

Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.

Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.

So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.

Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.

[1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
[2] https://www.mail-archive.com/dev@.../msg45402.html

-Jaikiran

On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:

One thing though.

This revival of the community has been triggered by the comments on this issue:
https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
It would be a shame if it is not fixed within the next release.

The issue and the fix are not easy to understand. Any review will be welcomed.

Nicolas

> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>
>
>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>
>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>
>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>
>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>
>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>
> Sounds like a good plan.
>
> Nicolas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Goals for the upcoming release?

Nicolas Lalevée
Thank you very much Jaikiran for your detailed explanation.

I have to admit that I don’t know the answer to which is the source of truth about the resolution report. I have also wondered why the resolve report should have to be always wrote and then read from file. I’ll try to get an answer myself too while reading the code.

Nicolas

> Le 22 mai 2017 à 07:37, J Pai <[hidden email]> a écrit :
>
> One other thing about this issue - this is reproducible (as shown in the test case) with static revisions too and isn’t specific to any dynamic revision resolution.
>
> -Jaikiran
> On 22-May-2017, at 11:02 AM, J Pai <[hidden email]> wrote:
>
> I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).
>
> Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:
>
> <ivy-module version="2.0">
>   <info organisation="org"
>         module="hello-world"
>         revision="1.2.3"
>
>   />
>   <configurations>
>       <conf name="conf1"/>
>       <conf name="conf2"/>
>   </configurations>
>   <dependencies>
>       <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
>       <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
>   </dependencies>
> </ivy-module>
>
> Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:
>
> <ivy-module version="1.0">
> <info organisation="org"
>        module="module2"
>        revision="1.0"
> />
>   <publications>
>       <artifact/>
>   </publications>
> <dependencies>
>   <dependency org="org" name="module1" rev="2.0"/>
> </dependencies>
> </ivy-module>
>
> So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).
>
> Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.
>
> Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.
>
> So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.
>
> Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.
>
> [1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
> [2] https://www.mail-archive.com/dev@.../msg45402.html
>
> -Jaikiran
>
> On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:
>
> One thing though.
>
> This revival of the community has been triggered by the comments on this issue:
> https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
> It would be a shame if it is not fixed within the next release.
>
> The issue and the fix are not easy to understand. Any review will be welcomed.
>
> Nicolas
>
>> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>>
>>
>>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>>
>>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>>
>>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>>
>>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>>
>>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>>
>> Sounds like a good plan.
>>
>> Nicolas
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

IVY-1485, resolve report and resolved versions

Nicolas Lalevée
After some digging, I found out that the resolved revisions property file is very old (git is awesome, even better within sourcetree). It is so old that at that time the xml resolve report didn’t exist. And this property file seems to have been created just in order to make the deliver task work [1]. And a comment still state this too [2]. And the purpose of the lines from 248 to 326 are about to compute these versions and write them down, it can be extracted in a function without side effect.

So most probably this property file is redundant with the xml resolve report. But the thing is the resolve may not be produced, it is an option [3], defaulting to true [4].

Two options:
- continue to support the property file and correctly produce it to fix IVY-1485 [5].
- drop this property file, make the xml report always produced, and make the deliver read the report rather than the property file.

Makes sense ?

Nicolas

[1] https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325 <https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325>
[2] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247>
[3] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338>
[4] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99>
[5] https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>

> Le 25 mai 2017 à 12:11, Nicolas Lalevée <[hidden email]> a écrit :
>
> Thank you very much Jaikiran for your detailed explanation.
>
> I have to admit that I don’t know the answer to which is the source of truth about the resolution report. I have also wondered why the resolve report should have to be always wrote and then read from file. I’ll try to get an answer myself too while reading the code.
>
> Nicolas
>
>> Le 22 mai 2017 à 07:37, J Pai <[hidden email]> a écrit :
>>
>> One other thing about this issue - this is reproducible (as shown in the test case) with static revisions too and isn’t specific to any dynamic revision resolution.
>>
>> -Jaikiran
>> On 22-May-2017, at 11:02 AM, J Pai <[hidden email]> wrote:
>>
>> I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).
>>
>> Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:
>>
>> <ivy-module version="2.0">
>>  <info organisation="org"
>>        module="hello-world"
>>        revision="1.2.3"
>>
>>  />
>>  <configurations>
>>      <conf name="conf1"/>
>>      <conf name="conf2"/>
>>  </configurations>
>>  <dependencies>
>>      <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
>>      <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
>>  </dependencies>
>> </ivy-module>
>>
>> Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:
>>
>> <ivy-module version="1.0">
>> <info organisation="org"
>>       module="module2"
>>       revision="1.0"
>> />
>>  <publications>
>>      <artifact/>
>>  </publications>
>> <dependencies>
>>  <dependency org="org" name="module1" rev="2.0"/>
>> </dependencies>
>> </ivy-module>
>>
>> So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).
>>
>> Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.
>>
>> Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.
>>
>> So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.
>>
>> Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.
>>
>> [1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
>> [2] https://www.mail-archive.com/dev@.../msg45402.html
>>
>> -Jaikiran
>>
>> On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:
>>
>> One thing though.
>>
>> This revival of the community has been triggered by the comments on this issue:
>> https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
>> It would be a shame if it is not fixed within the next release.
>>
>> The issue and the fix are not easy to understand. Any review will be welcomed.
>>
>> Nicolas
>>
>>> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>>>
>>>
>>>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>>>
>>>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>>>
>>>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>>>
>>>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>>>
>>>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>>>
>>> Sounds like a good plan.
>>>
>>> Nicolas
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IVY-1485, resolve report and resolved versions

Jaikiran Pai
Thank you very much Nicolas for digging more into this and getting hold of these details. What you explain, backed by the details, makes clear sense.

I will attempt to go with this option that you suggest:

> - drop this property file, make the xml report always produced, and make the deliver read the report rather than the property file.

From looking at the code and documentation, the option that decides whether to create the XML report seems to be an internal only option (in Ivy 2.x world) i.e. not exposed or documented as something that the user can fiddle with. That’s a good thing since removing it or fiddling with its semantics, won’t affect the users.

P.S: I don’t mean to step on your toes here if you were planning to work on this. If not, I’ll gladly try and come up with the suggested fix.

-Jaikiran


On 26-May-2017, at 2:20 PM, Nicolas Lalevée <[hidden email]> wrote:

After some digging, I found out that the resolved revisions property file is very old (git is awesome, even better within sourcetree). It is so old that at that time the xml resolve report didn’t exist. And this property file seems to have been created just in order to make the deliver task work [1]. And a comment still state this too [2]. And the purpose of the lines from 248 to 326 are about to compute these versions and write them down, it can be extracted in a function without side effect.

So most probably this property file is redundant with the xml resolve report. But the thing is the resolve may not be produced, it is an option [3], defaulting to true [4].

Two options:
- continue to support the property file and correctly produce it to fix IVY-1485 [5].
- drop this property file, make the xml report always produced, and make the deliver read the report rather than the property file.

Makes sense ?

Nicolas

[1] https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325 <https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325>
[2] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247>
[3] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338>
[4] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99>
[5] https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>

> Le 25 mai 2017 à 12:11, Nicolas Lalevée <[hidden email]> a écrit :
>
> Thank you very much Jaikiran for your detailed explanation.
>
> I have to admit that I don’t know the answer to which is the source of truth about the resolution report. I have also wondered why the resolve report should have to be always wrote and then read from file. I’ll try to get an answer myself too while reading the code.
>
> Nicolas
>
>> Le 22 mai 2017 à 07:37, J Pai <[hidden email]> a écrit :
>>
>> One other thing about this issue - this is reproducible (as shown in the test case) with static revisions too and isn’t specific to any dynamic revision resolution.
>>
>> -Jaikiran
>> On 22-May-2017, at 11:02 AM, J Pai <[hidden email]> wrote:
>>
>> I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).
>>
>> Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:
>>
>> <ivy-module version="2.0">
>> <info organisation="org"
>>       module="hello-world"
>>       revision="1.2.3"
>>
>> />
>> <configurations>
>>     <conf name="conf1"/>
>>     <conf name="conf2"/>
>> </configurations>
>> <dependencies>
>>     <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
>>     <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
>> </dependencies>
>> </ivy-module>
>>
>> Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:
>>
>> <ivy-module version="1.0">
>> <info organisation="org"
>>      module="module2"
>>      revision="1.0"
>> />
>> <publications>
>>     <artifact/>
>> </publications>
>> <dependencies>
>> <dependency org="org" name="module1" rev="2.0"/>
>> </dependencies>
>> </ivy-module>
>>
>> So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).
>>
>> Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.
>>
>> Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.
>>
>> So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.
>>
>> Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.
>>
>> [1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
>> [2] https://www.mail-archive.com/dev@.../msg45402.html
>>
>> -Jaikiran
>>
>> On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:
>>
>> One thing though.
>>
>> This revival of the community has been triggered by the comments on this issue:
>> https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
>> It would be a shame if it is not fixed within the next release.
>>
>> The issue and the fix are not easy to understand. Any review will be welcomed.
>>
>> Nicolas
>>
>>> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>>>
>>>
>>>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>>>
>>>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>>>
>>>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>>>
>>>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>>>
>>>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>>>
>>> Sounds like a good plan.
>>>
>>> Nicolas
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: IVY-1485, resolve report and resolved versions

Nicolas Lalevée

> Le 26 mai 2017 à 13:23, J Pai <[hidden email]> a écrit :
>
> Thank you very much Nicolas for digging more into this and getting hold of these details. What you explain, backed by the details, makes clear sense.
>
> I will attempt to go with this option that you suggest:
>
>> - drop this property file, make the xml report always produced, and make the deliver read the report rather than the property file.
>
> From looking at the code and documentation, the option that decides whether to create the XML report seems to be an internal only option (in Ivy 2.x world) i.e. not exposed or documented as something that the user can fiddle with. That’s a good thing since removing it or fiddling with its semantics, won’t affect the users.

ha, ok. good then.

> P.S: I don’t mean to step on your toes here if you were planning to work on this. If not, I’ll gladly try and come up with the suggested fix.

Please, be my guest.

Nicolas

>
> -Jaikiran
>
>
> On 26-May-2017, at 2:20 PM, Nicolas Lalevée <[hidden email]> wrote:
>
> After some digging, I found out that the resolved revisions property file is very old (git is awesome, even better within sourcetree). It is so old that at that time the xml resolve report didn’t exist. And this property file seems to have been created just in order to make the deliver task work [1]. And a comment still state this too [2]. And the purpose of the lines from 248 to 326 are about to compute these versions and write them down, it can be extracted in a function without side effect.
>
> So most probably this property file is redundant with the xml resolve report. But the thing is the resolve may not be produced, it is an option [3], defaulting to true [4].
>
> Two options:
> - continue to support the property file and correctly produce it to fix IVY-1485 [5].
> - drop this property file, make the xml report always produced, and make the deliver read the report rather than the property file.
>
> Makes sense ?
>
> Nicolas
>
> [1] https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325 <https://github.com/apache/ant-ivy/commit/3081a1b16a2fcb13b7a13bf761d6a625598d0325>
> [2] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L247>
> [3] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveEngine.java#L338>
> [4] https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99 <https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/core/resolve/ResolveOptions.java#L99>
> [5] https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
>> Le 25 mai 2017 à 12:11, Nicolas Lalevée <[hidden email]> a écrit :
>>
>> Thank you very much Jaikiran for your detailed explanation.
>>
>> I have to admit that I don’t know the answer to which is the source of truth about the resolution report. I have also wondered why the resolve report should have to be always wrote and then read from file. I’ll try to get an answer myself too while reading the code.
>>
>> Nicolas
>>
>>> Le 22 mai 2017 à 07:37, J Pai <[hidden email]> a écrit :
>>>
>>> One other thing about this issue - this is reproducible (as shown in the test case) with static revisions too and isn’t specific to any dynamic revision resolution.
>>>
>>> -Jaikiran
>>> On 22-May-2017, at 11:02 AM, J Pai <[hidden email]> wrote:
>>>
>>> I have had a look at that issue, this last week and I have been able to reproduce it in a simple test case[1]. I kind of understand what the issue is in there, but given my lack of knowledge of the Ivy codebase, I haven’t been able to figure how to fix this correctly. In fact, this issue is what prompted me to ask this question [2] in the dev list a day or so back, since basically comes down to those files. Here’s my understanding of the problem (backed by that test case[1] which reproduces it).
>>>
>>> Imagine you have a module org:hello-world and imagine it has 2 configurations conf1 and conf2. Now consider the case where this hello-world module depends on org:module1:1.0 in conf1 (a direct dependency) and on org:module2:1.0 in conf2 (a direct dependency). That translates to a module descriptor like:
>>>
>>> <ivy-module version="2.0">
>>> <info organisation="org"
>>>      module="hello-world"
>>>      revision="1.2.3"
>>>
>>> />
>>> <configurations>
>>>    <conf name="conf1"/>
>>>    <conf name="conf2"/>
>>> </configurations>
>>> <dependencies>
>>>    <dependency org="org" name="module1" rev="1.0" conf="conf1->default"/>
>>>    <dependency org="org" name="module2" rev="1.0" conf="conf2->default"/>
>>> </dependencies>
>>> </ivy-module>
>>>
>>> Now take this one step further and consider that org:module2:1.0 (that hello-world depends on, in conf2) has a dependency of its own on org:module1:2.0. So module2’s module descriptor looks like:
>>>
>>> <ivy-module version="1.0">
>>> <info organisation="org"
>>>     module="module2"
>>>     revision="1.0"
>>> />
>>> <publications>
>>>    <artifact/>
>>> </publications>
>>> <dependencies>
>>> <dependency org="org" name="module1" rev="2.0"/>
>>> </dependencies>
>>> </ivy-module>
>>>
>>> So ultimately, when you resolve the hello-world module, you expect it to have org:module1:1.0 as an dependency in conf1 and org:module1:2.0 as an dependency in conf2 (transitively via org:module2:1.0).
>>>
>>> Does the resolve work as expected for this use case? Yes it does and the resolution pulls in the right set of dependencies in the right configurations. Internally it creates resolution report (as an xml) plus a resolution properties file for this resolution. No (obvious/apparent) issues at this point.
>>>
>>> Now, let’s say a “deliver” is triggered against this resolution, for conf1. What I would expect is, that it would deliver a file for hello-world which then has a dependency on org:module1:1.0 in conf1 (because that’s what it was correctly resolved to previously). However, the delivered file instead has a dependency on org:module1:2.0 in conf1 and I believe that’s the issue being reported in that JIRA.
>>>
>>> So is this a bug in the deliver task itself? I don’t think so. So far what I have narrowed it down to is that the resolve task that happened previously creates more than one representation of that resolution (one a resolution report xml and one a resolution properties file). The resolution report XML has all the necessary and correct information about which dependency (either direct or transitive) belongs to which configuration. However, the resolution properties file contains the “wrong” dependency for module1 - it stores the dependency as org:module1:2.0. I am not even sure if the properties file is capable enough of supporting/understanding dependencies per configuration. The deliver task then uses this properties file (instead of the resolution report XML) to decide what dependencies to write out. I’m guessing some other (post-resolve) tasks too use this properties file for their decision making, so this really boils down to a potential bug in what gets written out to this resolution properties file, during resolve.
>>>
>>> Unfortunately, my reading of the code so far hasn’t given me answers on what role this file plays and why can’t it be just skipped and the resolution report XML instead be considered the single source of truth. Hence that question [2] in the dev list. I can’t really think of a solution/fix for this issue, without reading more of the current Ivy code to understand what role these files play.
>>>
>>> [1] https://github.com/jaikiran/ant-ivy/commit/529a3fa05ed5dd115bdf8046d0cf0ffe034905e0#diff-6ed8218b6bb9460014cf3558ff227172R571
>>> [2] https://www.mail-archive.com/dev@.../msg45402.html
>>>
>>> -Jaikiran
>>>
>>> On 21-May-2017, at 10:49 PM, Nicolas Lalevée <[hidden email]> wrote:
>>>
>>> One thing though.
>>>
>>> This revival of the community has been triggered by the comments on this issue:
>>> https://issues.apache.org/jira/browse/IVY-1485 <https://issues.apache.org/jira/browse/IVY-1485>
>>> It would be a shame if it is not fixed within the next release.
>>>
>>> The issue and the fix are not easy to understand. Any review will be welcomed.
>>>
>>> Nicolas
>>>
>>>> Le 21 mai 2017 à 18:46, Nicolas Lalevée <[hidden email]> a écrit :
>>>>
>>>>
>>>>> Le 21 mai 2017 à 16:28, J Pai <[hidden email]> a écrit :
>>>>>
>>>>> The past few days I’ve sent some PRs for bug fixes and some enhancements in preparation for the proposed release of Ivy. Thanks Nicolas for reviewing them and merging those that were good enough.
>>>>>
>>>>> I’ve been using this JIRA filter [1] to narrow down on the issues to look into. That filter essentially is of “open issues that affect 2.1.0, 2.2.0, 2.3.0 or 2.4.0 and have been updated/created since Jan 1st 2014”. So it should cover most of the issues that we probably should look into (doesn’t necessarily mean fix all of them, but just to check if any of them are big enough to focus on).
>>>>>
>>>>> I’ve also sent one PR for upgrading our internal library dependencies and plan to send a couple more for similar upgrades. My intention is to use the latest released versions of these dependencies instead of sticking with dependencies that are years old. My intention _isn’t_ to upgrade to a version that isn’t API backward compatible, so these upgrades are mostly bug fixes and should be “drop-in upgrades”.
>>>>>
>>>>> I would be really glad to hear any thoughts about these changes or any other/different plans, that can get us to a releasable state in the near future, especially from members/users who have been involved with Ant/Ivy project in the past or present. Ultimately, I think if we can agree on a goal for the upcoming release, it will help release something that will set the right expectation with the end users when it comes to using it. My opinion is that we consider this release to push out some bug fixes and internal upgrades and _not_ introduce any major features unless those are reasonably quick to implement. Once this release is done and (hopefully some of the) community gets back behind the Ivy project, we can always introduce major features in subsequent releases.
>>>>
>>>> Sounds like a good plan.
>>>>
>>>> Nicolas
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...