Ivy - Move to symlink creation standard Java API?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Ivy - Move to symlink creation standard Java API?

Jaikiran Pai
I was looking into a JIRA related to symlinking in Ivy and realized that
(for reasons noted in the docs and the implementation) ourimplementation
of symlinking relies on launching a process from within the JVM to
invoke a shell command to create the symlinks. "retrieve" task is the
only one that deals with symlink creation, from what I can see.
Furthermore, knowing that this operation is expensive, we even had to
introduce an additional optionfor that task to allow "mass
symlinking"[1] so that we launch a single process to create N symlinks
instead of N processes.

In short, the current implementation of symlinking in Ivy is expensive
for reasons that have been known. Now that we have moved to Java 7,
which has a standard APIfor symlink creation[2], I think we should just
change our internal implementation to use this new standard API.Of
course, we continue to fail to create symlinks on systems that don't
support it, just like we do now, except that we let the Java API
implementation handle those details.

Furthermore, once we move to thisstandard API, I don't think we need the
"symlinkmass" option onthe retrieve task anymore since the whole purpose
of it was to avoid launching N processes. So I think we can deprecate
that option in the upcoming release.

Any thoughts?

[1] https://ant.apache.org/ivy/history/latest-milestone/use/retrieve.html

[2]
https://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#createSymbolicLink(java.nio.file.Path,%20java.nio.file.Path,%20java.nio.file.attribute.FileAttribute...)

-Jaikiran


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Move to symlink creation standard Java API?

Nicolas Lalevée

> Le 26 juil. 2017 à 15:25, Jaikiran Pai <[hidden email]> a écrit :
>
> I was looking into a JIRA related to symlinking in Ivy and realized that (for reasons noted in the docs and the implementation) ourimplementation of symlinking relies on launching a process from within the JVM to invoke a shell command to create the symlinks. "retrieve" task is the only one that deals with symlink creation, from what I can see. Furthermore, knowing that this operation is expensive, we even had to introduce an additional optionfor that task to allow "mass symlinking"[1] so that we launch a single process to create N symlinks instead of N processes.
>
> In short, the current implementation of symlinking in Ivy is expensive for reasons that have been known. Now that we have moved to Java 7, which has a standard APIfor symlink creation[2], I think we should just change our internal implementation to use this new standard API.Of course, we continue to fail to create symlinks on systems that don't support it, just like we do now, except that we let the Java API implementation handle those details.
>
> Furthermore, once we move to thisstandard API, I don't think we need the "symlinkmass" option onthe retrieve task anymore since the whole purpose of it was to avoid launching N processes. So I think we can deprecate that option in the upcoming release.
>
> Any thoughts?

Sounds good to me.

Nicolas


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

AW: Ivy - Move to symlink creation standard Java API?

Jan Matèrne (jhm)
If the new Java version supports the symlink creation and it's an implementation detail
of Ivy (so not part of an API other could use) I am in favour of getting rid of the old
implementation.
Getting rid of that means being more standard compliant and having less to maintain. ;)

Jan



> -----Ursprüngliche Nachricht-----
> Von: Nicolas Lalevée [mailto:[hidden email]]
> Gesendet: Mittwoch, 26. Juli 2017 18:19
> An: Ant Developers List
> Betreff: Re: Ivy - Move to symlink creation standard Java API?
>
>
> > Le 26 juil. 2017 à 15:25, Jaikiran Pai <[hidden email]> a
> écrit :
> >
> > I was looking into a JIRA related to symlinking in Ivy and realized
> that (for reasons noted in the docs and the implementation)
> ourimplementation of symlinking relies on launching a process from
> within the JVM to invoke a shell command to create the symlinks.
> "retrieve" task is the only one that deals with symlink creation, from
> what I can see. Furthermore, knowing that this operation is expensive,
> we even had to introduce an additional optionfor that task to allow
> "mass symlinking"[1] so that we launch a single process to create N
> symlinks instead of N processes.
> >
> > In short, the current implementation of symlinking in Ivy is
> expensive for reasons that have been known. Now that we have moved to
> Java 7, which has a standard APIfor symlink creation[2], I think we
> should just change our internal implementation to use this new standard
> API.Of course, we continue to fail to create symlinks on systems that
> don't support it, just like we do now, except that we let the Java API
> implementation handle those details.
> >
> > Furthermore, once we move to thisstandard API, I don't think we need
> the "symlinkmass" option onthe retrieve task anymore since the whole
> purpose of it was to avoid launching N processes. So I think we can
> deprecate that option in the upcoming release.
> >
> > Any thoughts?
>
> Sounds good to me.
>
> Nicolas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email] For additional
> commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Move to symlink creation standard Java API?

Gintautas Grigelionis
There might be some issues with Cygwin, see Eclipse/JGit discussion [1]

Gintas

[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=354367

2017-07-27 10:44 GMT+02:00 Jan Matèrne (jhm) <[hidden email]>:

> If the new Java version supports the symlink creation and it's an
> implementation detail
> of Ivy (so not part of an API other could use) I am in favour of getting
> rid of the old
> implementation.
> Getting rid of that means being more standard compliant and having less to
> maintain. ;)
>
> Jan
>
>
>
> > -----Ursprüngliche Nachricht-----
> > Von: Nicolas Lalevée [mailto:[hidden email]]
> > Gesendet: Mittwoch, 26. Juli 2017 18:19
> > An: Ant Developers List
> > Betreff: Re: Ivy - Move to symlink creation standard Java API?
> >
> >
> > > Le 26 juil. 2017 à 15:25, Jaikiran Pai <[hidden email]> a
> > écrit :
> > >
> > > I was looking into a JIRA related to symlinking in Ivy and realized
> > that (for reasons noted in the docs and the implementation)
> > ourimplementation of symlinking relies on launching a process from
> > within the JVM to invoke a shell command to create the symlinks.
> > "retrieve" task is the only one that deals with symlink creation, from
> > what I can see. Furthermore, knowing that this operation is expensive,
> > we even had to introduce an additional optionfor that task to allow
> > "mass symlinking"[1] so that we launch a single process to create N
> > symlinks instead of N processes.
> > >
> > > In short, the current implementation of symlinking in Ivy is
> > expensive for reasons that have been known. Now that we have moved to
> > Java 7, which has a standard APIfor symlink creation[2], I think we
> > should just change our internal implementation to use this new standard
> > API.Of course, we continue to fail to create symlinks on systems that
> > don't support it, just like we do now, except that we let the Java API
> > implementation handle those details.
> > >
> > > Furthermore, once we move to thisstandard API, I don't think we need
> > the "symlinkmass" option onthe retrieve task anymore since the whole
> > purpose of it was to avoid launching N processes. So I think we can
> > deprecate that option in the upcoming release.
> > >
> > > Any thoughts?
> >
> > Sounds good to me.
> >
> > Nicolas
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email] For additional
> > commands, e-mail: [hidden email]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Move to symlink creation standard Java API?

Gintautas Grigelionis
I did a few simple tests on Windows 7 + Cygwin and NIO.2 seems to work just
fine (if privilege to create symlinks is granted).

+1

Gintas

2017-07-27 11:40 GMT+02:00 Gintautas Grigelionis <[hidden email]>:

> There might be some issues with Cygwin, see Eclipse/JGit discussion [1]
>
> Gintas
>
> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=354367
>
> 2017-07-27 10:44 GMT+02:00 Jan Matèrne (jhm) <[hidden email]>:
>
>> If the new Java version supports the symlink creation and it's an
>> implementation detail
>> of Ivy (so not part of an API other could use) I am in favour of getting
>> rid of the old
>> implementation.
>> Getting rid of that means being more standard compliant and having less
>> to maintain. ;)
>>
>> Jan
>>
>>
>>
>> > -----Ursprüngliche Nachricht-----
>> > Von: Nicolas Lalevée [mailto:[hidden email]]
>> > Gesendet: Mittwoch, 26. Juli 2017 18:19
>> > An: Ant Developers List
>> > Betreff: Re: Ivy - Move to symlink creation standard Java API?
>> >
>> >
>> > > Le 26 juil. 2017 à 15:25, Jaikiran Pai <[hidden email]> a
>> > écrit :
>> > >
>> > > I was looking into a JIRA related to symlinking in Ivy and realized
>> > that (for reasons noted in the docs and the implementation)
>> > ourimplementation of symlinking relies on launching a process from
>> > within the JVM to invoke a shell command to create the symlinks.
>> > "retrieve" task is the only one that deals with symlink creation, from
>> > what I can see. Furthermore, knowing that this operation is expensive,
>> > we even had to introduce an additional optionfor that task to allow
>> > "mass symlinking"[1] so that we launch a single process to create N
>> > symlinks instead of N processes.
>> > >
>> > > In short, the current implementation of symlinking in Ivy is
>> > expensive for reasons that have been known. Now that we have moved to
>> > Java 7, which has a standard APIfor symlink creation[2], I think we
>> > should just change our internal implementation to use this new standard
>> > API.Of course, we continue to fail to create symlinks on systems that
>> > don't support it, just like we do now, except that we let the Java API
>> > implementation handle those details.
>> > >
>> > > Furthermore, once we move to thisstandard API, I don't think we need
>> > the "symlinkmass" option onthe retrieve task anymore since the whole
>> > purpose of it was to avoid launching N processes. So I think we can
>> > deprecate that option in the upcoming release.
>> > >
>> > > Any thoughts?
>> >
>> > Sounds good to me.
>> >
>> > Nicolas
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: [hidden email] For additional
>> > commands, e-mail: [hidden email]
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Ivy - Move to symlink creation standard Java API?

Jaikiran Pai
Thank you for testing that kind of setup.

I have now pushed a commit which does the necessary changes to use Java
standard API and also deprecates that use of symlinkmass option (which
is no longer relevant) of the retrieve task. The testcases have been
updated to make sure the tests are run on all systems (earlier they used
to be run only on specific systems) and the logic in the tests has been
updated to do a more robust check on ability to create symbolic links.

I have also updated the documentation and release notes to note and
explain the deprecation of the symlinkmass option.

-Jaikiran


On 27/07/17 5:07 PM, Gintautas Grigelionis wrote:

> I did a few simple tests on Windows 7 + Cygwin and NIO.2 seems to work just
> fine (if privilege to create symlinks is granted).
>
> +1
>
> Gintas
>
> 2017-07-27 11:40 GMT+02:00 Gintautas Grigelionis <[hidden email]>:
>
>> There might be some issues with Cygwin, see Eclipse/JGit discussion [1]
>>
>> Gintas
>>
>> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=354367
>>
>> 2017-07-27 10:44 GMT+02:00 Jan Matèrne (jhm) <[hidden email]>:
>>
>>> If the new Java version supports the symlink creation and it's an
>>> implementation detail
>>> of Ivy (so not part of an API other could use) I am in favour of getting
>>> rid of the old
>>> implementation.
>>> Getting rid of that means being more standard compliant and having less
>>> to maintain. ;)
>>>
>>> Jan
>>>
>>>
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Nicolas Lalevée [mailto:[hidden email]]
>>>> Gesendet: Mittwoch, 26. Juli 2017 18:19
>>>> An: Ant Developers List
>>>> Betreff: Re: Ivy - Move to symlink creation standard Java API?
>>>>
>>>>
>>>>> Le 26 juil. 2017 à 15:25, Jaikiran Pai <[hidden email]> a
>>>> écrit :
>>>>> I was looking into a JIRA related to symlinking in Ivy and realized
>>>> that (for reasons noted in the docs and the implementation)
>>>> ourimplementation of symlinking relies on launching a process from
>>>> within the JVM to invoke a shell command to create the symlinks.
>>>> "retrieve" task is the only one that deals with symlink creation, from
>>>> what I can see. Furthermore, knowing that this operation is expensive,
>>>> we even had to introduce an additional optionfor that task to allow
>>>> "mass symlinking"[1] so that we launch a single process to create N
>>>> symlinks instead of N processes.
>>>>> In short, the current implementation of symlinking in Ivy is
>>>> expensive for reasons that have been known. Now that we have moved to
>>>> Java 7, which has a standard APIfor symlink creation[2], I think we
>>>> should just change our internal implementation to use this new standard
>>>> API.Of course, we continue to fail to create symlinks on systems that
>>>> don't support it, just like we do now, except that we let the Java API
>>>> implementation handle those details.
>>>>> Furthermore, once we move to thisstandard API, I don't think we need
>>>> the "symlinkmass" option onthe retrieve task anymore since the whole
>>>> purpose of it was to avoid launching N processes. So I think we can
>>>> deprecate that option in the upcoming release.
>>>>> Any thoughts?
>>>> Sounds good to me.
>>>>
>>>> Nicolas
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email] For additional
>>>> commands, e-mail: [hidden email]
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: AW: Ivy - Move to symlink creation standard Java API?

Jaikiran Pai
In reply to this post by Jan Matèrne (jhm)

On 27/07/17 2:14 PM, Jan Matèrne (jhm) wrote:
> Getting rid of that means being more standard compliant and having less to maintain. ;)
Agreed. Plus the fact that our previous implementation of this feature
relied on the presence of the "ln" command on the systems, which
effectively ruled out this feature on native Windows completely. With
the usage of the standard API, that's no longer the case.

-Jaikiran

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Loading...