Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Breaking change after 7.2.0 ? #1040

Open
sailro opened this issue May 7, 2024 · 9 comments
Open

Breaking change after 7.2.0 ? #1040

sailro opened this issue May 7, 2024 · 9 comments
Labels
unscheduled

Comments

@sailro
Copy link

sailro commented May 7, 2024 •
edited
Loading

Hi,

We are using license finder to audit all our projects. We capture stdout to find new licences to check.

For maven-based projects, we use license_finder --no-prepare --enabled-package-managers=maven --maven-options=package --quiet --maven-include-groups=true

Since 7.2.0 released today, the output is polluted by messages coming from our test suite, so breaking everything.

Did something changed regarding this ? Can we pass extra options to license finder to either ignore tests (like -Dmaven.test.skip=true -DskipTests=true we commonly use when invoking maven directly) or simply ignore test output ?

Thank you !
@cf-gitbot
Copy link
Collaborator

We have created an issue in Pivotal Tracker to manage this. Unfortunately, the Pivotal Tracker project is private so you may be unable to view the contents of the story.

The labels on this github issue will be updated when the story is started.

@sailro
Copy link
Author

sailro commented May 7, 2024 •
edited
Loading

okay so using :

license_finder --no-prepare --enabled-package-managers=maven --maven-options="package -Dmaven.test.skip=true -DskipTests=true" --quiet --maven-include-groups=true seems to fix the issue.

So not sure if it's really a bug in license finder but the behavior definitively changed between 7.1.0 and 7.2.0

@xtreme-shane-lattanzio
Copy link
Contributor

Hey @sailro! There were quite a bit of maven changes for this release added by @rhuitl. I added more to the changelog to try to highlight a bit more. I am not sure what exactly caused the logs to change but im glad the workaround resolves it!

@rhuitl
Copy link
Contributor

rhuitl commented May 7, 2024 •
edited
Loading

Hmm, there was a patch version update for Maven, and this: 0da6d30#diff-b958f840c0529a7d4cc364d0e80267591bd50ed398b563d8d530513a5c9a3869R61. Can you see from your logs which Maven call produced the output? Can you make a minimal reproduction case maybe?

The actual call to Maven to extract the licenses did not change.

@ttomaszewski
Copy link

ttomaszewski commented May 8, 2024 •
edited
Loading

After upgrading to 7.2.0 we experienced the following Zip::Error with Maven.
Reverted back to 7.1.0, which is working fine.

LicenseFinder::Maven: is active for '/home/circleci/repo/service'
/home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /home/circleci/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.107.Final/netty-resolver-dns-native-macos-4.1.107.Final-jakarta.jar not found (Zip::Error)
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:46:in `candidates_from_zip'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:19:in `find'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package_utils/license_files.rb:11:in `find'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/packages/maven_package.rb:32:in `license_files'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:130:in `licensing'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:124:in `activations'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/package.rb:120:in `licenses'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:60:in `with_approval'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:32:in `map'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/decision_applier.rb:8:in `initialize'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/core.rb:79:in `new'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/core.rb:79:in `decision_applier'
        from /home/circleci/.rvm/rubies/ruby-3.3.1/lib/ruby/3.3.0/forwardable.rb:234:in `any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:17:in `block in any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:15:in `map'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/license_aggregator.rb:15:in `any_packages?'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/lib/license_finder/cli/main.rb:121:in `action_items'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
        from /home/circleci/.rvm/gems/ruby-3.3.1/gems/license_finder-7.2.0/bin/license_finder:6:in `<top (required)>'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/license_finder:25:in `load'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/license_finder:25:in `<main>'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/ruby_executable_hooks:22:in `eval'
        from /home/circleci/.rvm/gems/ruby-3.3.1/bin/ruby_executable_hooks:22:in `<main>'

@xtreme-shane-lattanzio
Copy link
Contributor

@rhuitl @ttomaszewski I pushed a quick fix for this ba920d1

Im not sure what the jakarta jar is so this at least maintains functionality like the previous release if needed

@akilleen
Copy link

Hello,

To add on to this, I am getting a similar error, just with a different zip file...

LicenseFinder::Maven: is active for '/home/runner/work/repo/repo'
/var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /home/runner/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.90.Final/netty-resolver-dns-native-macos-4.1.90.Final.jar not found (Zip::Error)
	from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
	from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:46:in `candidates_from_zip'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:19:in `find'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:11:in `find'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/packages/maven_package.rb:32:in `license_files'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:130:in `licensing'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:124:in `activations'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:120:in `licenses'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:60:in `with_approval'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `map'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:8:in `initialize'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `new'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `decision_applier'
	from /usr/lib/ruby/3.0.0/forwardable.rb:232:in `acknowledged'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:51:in `block in aggregate_packages'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `each'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `flat_map'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `aggregate_packages'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:11:in `dependencies'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/cli/main.rb:161:in `report'
	from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
	from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
	from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
	from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
	from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder:6:in `<top (required)>'
	from /usr/local/bin/license_finder:25:in `load'
	from /usr/local/bin/license_finder:25:in `<main>'

I also noticed that in my maven cache that file is either missing or just named differently.

% ls -l /Users/killead/.m2/repository/io/netty/netty-resolver-dns-native-macos/4.1.90.Final                
total 104
-rw-r--r--@ 1 killead  staff    267 Jun 11 13:40 _remote.repositories
-rw-r--r--@ 1 killead  staff  18724 Mar 14  2023 netty-resolver-dns-native-macos-4.1.90.Final-osx-x86_64.jar
-rw-r--r--@ 1 killead  staff     40 Jun 11 13:40 netty-resolver-dns-native-macos-4.1.90.Final-osx-x86_64.jar.sha1
-rw-r--r--@ 1 killead  staff  17999 Mar 14  2023 netty-resolver-dns-native-macos-4.1.90.Final.pom
-rw-r--r--@ 1 killead  staff     40 Jun 11 13:40 netty-resolver-dns-native-macos-4.1.90.Final.pom.sha1

For now I have downgraded back down to 7.1.0 and it works.

@knoxg
Copy link

knoxg commented Jul 11, 2024

^ I just encounted the same zip file error, although on a different file:

/var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:106:in `initialize': File /root/.m2/repository/com/google/javascript/closure-compiler-rhino/v20140407/closure-compiler-rhino-v20140407.jar not found (Zip::Error)
        from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `new'
        from /var/lib/gems/3.0.0/gems/rubyzip-2.3.2/lib/zip/file.rb:121:in `open'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:47:in `candidates_from_zip'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:41:in `candidate_files_and_dirs'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:29:in `paths_of_candidate_files'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:19:in `find'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb:11:in `find'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/packages/maven_package.rb:32:in `license_files'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:130:in `licensing'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:124:in `activations'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package.rb:120:in `licenses'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:60:in `with_approval'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:34:in `block in apply_decisions'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `map'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:32:in `apply_decisions'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/decision_applier.rb:8:in `initialize'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `new'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/core.rb:79:in `decision_applier'
        from /usr/lib/ruby/3.0.0/forwardable.rb:232:in `acknowledged'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:51:in `block in aggregate_packages'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `each'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `flat_map'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:49:in `aggregate_packages'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/license_aggregator.rb:11:in `dependencies'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/cli/main.rb:161:in `report'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/command.rb:28:in `run'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/invocation.rb:127:in `invoke_command'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor.rb:527:in `dispatch'
        from /var/lib/gems/3.0.0/gems/thor-1.3.1/lib/thor/base.rb:584:in `start'
        from /var/lib/gems/3.0.0/gems/license_finder-7.2.1/bin/license_finder:6:in `<top (required)>'
        from /usr/local/bin/license_finder:25:in `load'
        from /usr/local/bin/license_finder:25:in `<main>'

I was able to get past this by patching license_files.rb so that it doesn't attempt to open non-existing files.
Can ignore the commented-out puts line, it was just there to add some logging to see what was happening:

--- /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb.orig        2024-07-11 02:02:00.826350646 +0000
+++ /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/license_files.rb     2024-07-11 02:01:40.415265274 +0000
@@ -43,6 +43,8 @@
     end

     def candidates_from_zip
+      # puts install_path.to_s
+      return [] if !File.exist?(install_path.to_s)
       Zip::File.open(install_path.to_s) do |zip_file|
         zip_file.glob(CANDIDATE_PATH_WILDCARD, File::FNM_EXTGLOB)
       end

@knoxg
Copy link

knoxg commented Jul 11, 2024

Following up from that last comment, I also needed to apply the following patch in order to generate a CSV report:

--- /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/notice_files.rb.orig 2024-07-11 06:04:55.697786572 +0000
+++ /var/lib/gems/3.0.0/gems/license_finder-7.2.1/lib/license_finder/package_utils/notice_files.rb      2024-07-11 06:37:45.816310897 +0000
@@ -43,6 +43,7 @@
     end

     def candidates_from_zip
+      return [] if !File.exist?(install_path.to_s)
       Zip::File.open(install_path.to_s) do |zip_file|
         zip_file.glob("*/#{CANDIDATE_PATH_WILDCARD_STRICT}", File::FNM_EXTGLOB)
       end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
unscheduled
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@knoxg @rhuitl @sailro @akilleen @xtreme-shane-lattanzio @cf-gitbot @ttomaszewski