Fallback to copy symlinks on Windows

[larskanis]

Symlinks are not permitted by default for a Windows user. To use them, a switch called "Development Mode" in the system settings has to be enabled.

What was the end-user or developer problem that led to this PR?

Ordinary users as well as administrators are unable per default to install gems using symlinks.
One such problematical gem is haml-rails-3.0.0.
It uses symlinks for files and directories.
The resulting error message is not very helpful:

$ gem inst haml-rails
Fetching haml-rails-3.0.0.gem
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the directory. (Gem::FilePermissionError)
        C:/ruby/lib/ruby/4.0.0/rubygems/installer.rb:308:in 'Gem::Installer#install'
        C:/ruby/lib/ruby/4.0.0/rubygems/resolver/specification.rb:105:in 'Gem::Resolver::Specification#install'
        C:/ruby/lib/ruby/4.0.0/rubygems/request_set.rb:192:in 'block in Gem::RequestSet#install'
        C:/ruby/lib/ruby/4.0.0/rubygems/request_set.rb:183:in 'Array#each'
        C:/ruby/lib/ruby/4.0.0/rubygems/request_set.rb:183:in 'Gem::RequestSet#install'
        C:/ruby/lib/ruby/4.0.0/rubygems/commands/install_command.rb:207:in 'Gem::Commands::InstallCommand#install_gem'
        C:/ruby/lib/ruby/4.0.0/rubygems/commands/install_command.rb:223:in 'block in Gem::Commands::InstallCommand#install_gems'
        C:/ruby/lib/ruby/4.0.0/rubygems/commands/install_command.rb:216:in 'Array#each'
        C:/ruby/lib/ruby/4.0.0/rubygems/commands/install_command.rb:216:in 'Gem::Commands::InstallCommand#install_gems'
        C:/ruby/lib/ruby/4.0.0/rubygems/commands/install_command.rb:162:in 'Gem::Commands::InstallCommand#execute'
        C:/ruby/lib/ruby/4.0.0/rubygems/command.rb:326:in 'Gem::Command#invoke_with_build_args'
        C:/ruby/lib/ruby/4.0.0/rubygems/command_manager.rb:252:in 'Gem::CommandManager#invoke_command'
        C:/ruby/lib/ruby/4.0.0/rubygems/command_manager.rb:193:in 'Gem::CommandManager#process_args'
        C:/ruby/lib/ruby/4.0.0/rubygems/command_manager.rb:151:in 'Gem::CommandManager#run'
        C:/ruby/lib/ruby/4.0.0/rubygems/gem_runner.rb:56:in 'Gem::GemRunner#run'
        C:/ruby/bin/gem.cmd:20:in '<main>'

What is your fix for the problem, implemented in this PR?

Instead of working around the situation in the affected gem or to skip symlinks completely, I think the better solution would be to make copies of the files in question. This would allow Windows users to install and use the gem smoothly.

I didn't adjust the rubygems tests to support this non-developer-mode use case, because I want to ask if this approach is acceptable, first.

The switch for the "Developer Mode" is available in the Windows registry under
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AppModelUnlock
entry
AllowDevelopmentWithoutDevLicense

Maybe we could add a Github Action run with AllowDevelopmentWithoutDevLicense=0, and adjust the tests to succeed in both cases.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags

[larskanis]

Is copying the files as a fallback to symlink an option for the rubygems maintainers? If so I would like to add proper tests.

[larskanis]

This PR is ready for review now. It adds a test run with non-admin user and no symlink permission to simulate a more realistic Windows environment. Tests run green in both conditions.

[larskanis]

This silence is new to me in rubygems+bundler. This used to be a very vital repository! I really miss @deivid-rodriguez, @simi, @segiddins and @indirect . Windows topics like this or #9276 are not so famous, but still important for a part of the community.

Can I help with taking a more official role for Windows specific issues? Since I maintain the RubyInstaller for Windows, I know most of the time about the issues on that platform. But it is annoying to invest time for creating or reviewing patches and then wait for months to get some answer. Having no idea why a PR isn't merged is frustrating.

[hsbt]

@larskanis You may not know this, but of all the maintainers I've ever had, I was one of the few who owned and maintained Windows, and still do today.

Right now I'm extremely busy and have higher priorities, but I'm not ignoring you. I'm very disappointment by your comments. Among the Ruby core maintainers, only I and nobu maintain Windows. If you care about Windows, you should respect to such maintainers.

[simi]

@larskanis You may not know this, but of all the maintainers I've ever had, I was one of the few who owned and maintained Windows, and still do today.

ℹ️ I have Windows machine and I was working on Windows issues all the time.

[larskanis]

Thank you @kou for all the comments! I updated the PR accordingly.

The difference between admin user with symlink and non-admin user without symlink is visible in the test output:

2692 tests, 14215 assertions, 0 failures, 0 errors, 37 pendings, 0 omissions, 0 notifications
versus non-admin:
2692 tests, 14192 assertions, 0 failures, 0 errors, 45 pendings, 2 omissions, 0 notifications

[simi]

Is symlink something valid gems should support? What about putting a warnings into gem policy for now to at least raise the awareness of the troubles it can cause?

[larskanis]

The example I mentioned above is a very valid use case of symlinks: https://github.com/haml/haml-rails/tree/9f4703ddff0644ba52529c5cf41c1624829b16a7/lib/generators/haml/scaffold/templates
It avoids duplicated code in a simple, comprehensible way.

[simi]

I understand, but since symlink is not platform agnostic concept, maybe it will be better to just warn on the usage and send PRs to popular gems to not use it. I think a little code to update the lookup can do the same as symlink in this haml-rails case.

[kou]

Can we focus on only the "symlink on Windows" in this issue?

[kou]

I understand, but since symlink is not platform agnostic concept, maybe it will be better to just warn on the usage and send PRs to popular gems to not use it. I think a little code to update the lookup can do the same as symlink in this haml-rails case.

If we want to recommend to not use symlink, how about resolving symlinks when gem authors run gem build?

[simi]

I understand, but since symlink is not platform agnostic concept, maybe it will be better to just warn on the usage and send PRs to popular gems to not use it. I think a little code to update the lookup can do the same as symlink in this haml-rails case.

If we want to recommend to not use symlink, how about resolving symlinks when gem authors run gem build?

I see various troubles, like if symlink is needed and expected in runtime, resolving symlinks will not make the gem 1:1 as expected. Calls like File.symlink? will work differently. IMHO the best to warn on gem build about known issues with symlinks and make the decision on author if worth the issues to keep the symlink.

---

Can we focus on only the "symlink on Windows" in this issue?

Happy to if everyone will respect each other and keep it friendly space.

[kou]

I see various troubles, like if symlink is needed and expected in runtime, resolving symlinks will not make the gem 1:1 as expected. Calls like File.symlink? will work differently.

Is this a real world issue or possible issue? Do you know a gem that depends on symlink?

[hsbt]

@simi This is your final warning: stop talking about anything but the code. I know you're intentionally blocking our development, and I won't tolerate that behavior.

[simi]

Is this a real world issue or possible issue? Do you know a gem that depends on symlink?

@kou no, haven't seen one. Just trying to think about the unexpected impact of "resolving the symlink".

This is your final warning:

Please if I'm breaking any rules, point me to them. I'm discussing the issue and reacting to other comments in the PR. @hsbt you have started to talk about anything but the code in here sharing false info on the previous maintainers effort on maintaining Windows here.

I know you're intentionally blocking our development, and I won't tolerate that behavior.

Now you're accusing me of intentionally blocking the development, I'm actually trying to help since you have mentioned few times there's not enough time of current maintainers. I have shared some valid concerns regarding resolving the issues to consider before going with "resolve symlink", I have contacted the mentioned gem author to see if the symlink causing real issue to someone can't be replaced at haml/haml-rails#197 (comment) and I have reacted to other comments. What exactly is blocking the development?

---

Clearly if there's better place to discuss those, please point me to.

[larskanis]

IMHO it is the best solution to create a copy when extracting the gem and when symlink creation fails. This is for the following reasons:

• Symlinks are common on Unix systems and they are not so uncommon on Windows these days from a developer point of view. It has changed in the last 10 years. We have good cross platform support for symlinks in Ruby for years. Rubygems supports symlinks for years. Git on Windows can handle symlinks since 2020 or so. So there is no need to avoid them at all.
• Warning about possible issues or duplication of symlinked files at gem build is exaggerated, since they work actually on all major operating systems. There's no need to avoid them. This is different to Improve handling of paths with invalid Windows characters #9276, where it is simply impossible to use file names with certain characters on Windows.
• Symlinks are not enabled by default. They need to be enabled by a switch in Windows settings called "Developer Mode". They aren't enabled for an ordinary user, so that we need a fallback to allow an non-admin ordinary user to install ruby applications.
• At the same time the "Developer Mode" enables insecure app installation. It can also be prohibited on companies Windows. So we shouldn't make it a precondition to install certain gems.
• Gems shouldn't modify it's own files after gem install. In a multi user setup, this isn't permitted anyways.
• Usage of File.symlink? or so on copied vs. symlinked files is only a theoretical issue. But in case it happens, then it should be discussed with the gem author like other platform specific issues.

[simi]

Symlinks ... and they are not so uncommon on Windows these days vs Symlinks are not enabled by default. this contradicts IMHO. Anyway I think it will be the best to combine all ideas together.

Should we soft-require Windows 10+ Developer Mode to be enabled for full compatibility with all RubyGems features? We can add some light warning for symlinks during build, just sharing it can potentially cause issues to Windows users. We can add something to gem env, bundle doctor (or bundle env), maybe warn during gem install if symlink failed (add in this PR?)...

I'm happy to contribute the other suggested improvements to raise the visibility of this potential setup issue for debug commands.

[kou]

(I don't have a strong opinion which approach is better.)

Symlinks ... and they are not so uncommon on Windows these days vs Symlinks are not enabled by default. this contradicts IMHO.

I agree with it.

duplication of symlinked files at gem build is exaggerated

If we do it, we can simplify our code.

We just need to change

rubygems/lib/rubygems/package.rb

Lines 263 to 265 in 884d80f

	if stat.symlink?
	tar.add_symlink file, File.readlink(file), stat.mode
	end

to use tar.add_file_simple.

We don't need this PR change. (We don't need to use mysterious icacls /grant options in our CI: system("icacls . /grant #{testuser}:(OI)(CI)(IO)F"))

(This doesn't recommend to use this approach. This just provides an implementation idea of this approach.)