Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes for OpenSearch #50

Open
wants to merge 23 commits into
base: 7.10
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
110 changes: 27 additions & 83 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# A plugin bundle for Elastisearch
# A plugin bundle for Opensearch

image:https://api.travis-ci.org/jprante/elasticsearch-plugin-bundle.svg[title="Build status", link="https://travis-ci.org/jprante/elasticsearch-plugin-bundle/"]
image:https://img.shields.io/sonar/http/nemo.sonarqube.com/org.xbib.elasticsearch.plugin%3Aelasticsearch-plugin-bundle/coverage.svg?style=flat-square[title="Coverage", link="https://sonarqube.com/dashboard/index?id=org.xbib.elasticsearch.plugin%3Aelasticsearch-plugin-bundle"]
Expand All @@ -9,101 +9,45 @@ image:https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif[title="PayPa

This plugin is the combination of the following plugins:

- elasticsearch-analysis-autophrase
- elasticsearch-analysis-baseform
- elasticsearch-analysis-concat
- elasticsearch-analysis-decompound
- elasticsearch-analysis-german
- elasticsearch-analysis-hyphen
- elasticsearch-analysis-icu
- elasticsearch-analysis-naturalsort
- elasticsearch-analysis-reference
- elasticsearch-analysis-sortform
- elasticsearch-analysis-standardnumber
- elasticsearch-analysis-symbolname
- elasticsearch-analysis-worddelimiter
- elasticsearch-analysis-year
- elasticsearch-mapper-crypt
- elasticsearch-mapper-langdetect
- opensearch-analysis-autophrase
- opensearch-analysis-baseform
- opensearch-analysis-concat
- opensearch-analysis-decompound
- opensearch-analysis-german
- opensearch-analysis-hyphen
- opensearch-analysis-icu
- opensearch-analysis-naturalsort
- opensearch-analysis-reference
- opensearch-analysis-sortform
- opensearch-analysis-standardnumber
- opensearch-analysis-symbolname
- opensearch-analysis-worddelimiter
- opensearch-analysis-year
- opensearch-mapper-crypt
- opensearch-mapper-langdetect

The plugin code in each plugin is equivalent to the code in this combined bundle plugin.

.Compatibility matrix
[frame="all"]
|===
| Plugin version | Elasticsearch version | Release date
| 6.3.2.2 | 6.3.2 | Oct 2, 2018
| 5.4.1.0 | 5.4.0 | Jun 1, 2017
| 5.4.0.1 | 5.4.0 | May 12, 2017
| 5.4.0.0 | 5.4.0 | May 4, 2017
| 5.3.1.0 | 5.3.1 | Apr 25, 2017
| 5.3.0.0 | 5.3.0 | Apr 4, 2017
| 5.2.2.0 | 5.2.2 | Mar 2, 2017
| 5.2.1.0 | 5.2.1 | Feb 27, 2017
| 5.1.1.2 | 5.1.1 | Feb 27, 2017
| 5.1.1.0 | 5.1.1 | Dec 31, 2016
| 2.3.4.0 | 2.3.4 | Jul 30, 2016
| 2.3.3.0 | 2.3.3 | May 23, 2016
| 2.3.2.0 | 2.3.2 | May 11, 2016
| 2.2.0.6 | 2.2.0 | Mar 25, 2016
| 2.2.0.3 | 2.2.0 | Mar 6, 2016
| 2.2.0.2 | 2.2.0 | Mar 3, 2016
| 2.2.0.1 | 2.2.0 | Feb 22, 2016
| 2.2.0.0 | 2.2.0 | Feb 8, 2016
| 2.1.1.2 | 2.1.1 | Dec 30, 2015
| 2.1.1.0 | 2.1.1 | Dec 21, 2015
| 2.1.0.0 | 2.1.0 | Nov 27, 2015
| 2.0.0.0 | 2.0.0 | Oct 28, 2015
| 1.6.0.0 | 1.6.0 | Jun 30, 2015
| 1.5.2.1 | 1.5.2 | Jun 30, 2015
| 1.5.2.0 | 1.5.2 | Apr 27, 2015
| 1.5.1.0 | 1.5.1 | Apr 23, 2015
| 1.5.0.0 | 1.5.0 | Mar 31, 2015
| 1.4.4.0 | 1.4.4 | Apr 26, 2015
| 1.4.0.6 | 1.4.0 | Feb 23, 2015
| 1.4.0.5 | 1.4.0 | Jan 28, 2015
| 1.4.0.4 | 1.4.0 | Jan 19, 2015
| 1.4.0.3 | 1.4.0 | Dec 16, 2014
| 1.4.0.1 | 1.4.0 | Nov 10, 2014
| Plugin version | Opensearch version | Release date
| 2.1.0.0 | 2.1.0 | Jul 12, 2022
| 2.0.1.0 | 2.0.1 | Jul 12, 2022
| 1.3.3.0 | 1.3.3 | Jul 12, 2022
| 1.0.0.0 | 1.0.0 | Jul 12, 2022
|===


## Installation

### Elasticsearch 5.x
### Opensearch 1.x, 2.x

[source]
----
./bin/elasticsearch-plugin install http://search.maven.org/remotecontent?filepath=org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/5.4.0.0/elasticsearch-plugin-bundle-5.4.0-plugin.zip
opensearch-plugin install -b https://github.com/grmblfrz/elasticsearch-plugin-bundle/releases/download/opensearch-2.1.0/opensearch-plugin-bundle-2.1.0.0.zip
----

or

[source]
----
./bin/elasticsearch-plugin install http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/5.3.0.0/elasticsearch-plugin-bundle-5.3.0.0-plugin.zip
----
Do not forget to restart the node after installing.

### Elasticsearch 2.x

[source]
----
./bin/plugin install 'http://search.maven.org/remotecontent?filepath=org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/2.3.3.0/elasticsearch-plugin-bundle-2.3.3.0-plugin.zip'
----
or
[source]
----
./bin/plugin install 'http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/2.3.4.0/elasticsearch-plugin-bundle-2.3.4.0-plugin.zip'
----
Do not forget to restart the node after installing.

### Elasticsearch 1.x

[source]
----
./bin/plugin -install bundle -url http://xbib.org/repository/org/xbib/elasticsearch/plugin/elasticsearch-plugin-bundle/1.6.0.0/elasticsearch-plugin-bundle-1.6.0.0-plugin.zip
----
Do not forget to restart the node after installing.

# Documentation
Expand All @@ -130,7 +74,7 @@ More to come.

## German normalizer

The `german_normalizer` is equivalent to Elasticsearch `german_normalization`. It performs umlaut treatment
The `german_normalizer` is equivalent to OpenSearch `german_normalization`. It performs umlaut treatment
with vocal expansion which is typical for german language.

[source]
Expand Down Expand Up @@ -617,7 +561,7 @@ GET _analyze

# Decompound

This is an implementation of a word decompounder plugin for link:http://github.com/elasticsearch/elasticsearch[Elasticsearch].
This is an implementation of a word decompounder plugin for link:http://github.com/opensearch-project/OpenSearch[OpenSearch].

Compounding several words into one word is a property not all languages share.
Compounding is used in German, Scandinavian Languages, Finnish and Korean.
Expand Down Expand Up @@ -673,7 +617,7 @@ In the mapping, use a token filter of type "decompound"::
"Die Jahresfeier der Rechtsanwaltskanzleien auf dem Donaudampfschiff hat viel Ökosteuer gekostet" will be tokenized into
"Die", "Die", "Jahresfeier", "Jahr", "feier", "der", "der", "Rechtsanwaltskanzleien", "Recht", "anwalt", "kanzlei", "auf", "auf", "dem", "dem", "Donaudampfschiff", "Donau", "dampf", "schiff", "hat", "hat", "viel", "viel", "Ökosteuer", "Ökosteuer", "gekostet", "gekosten"

It is recommended to add the `Unique token filter <http://www.elasticsearch.org/guide/reference/index-modules/analysis/unique-tokenfilter.html>`_ to skip tokens that occur more than once.
It is recommended to add the `Unique token filter https://www.elastic.co/guide/en/elasticsearch/reference/7.10/analysis-unique-tokenfilter.html to skip tokens that occur more than once.

Also the Lucene german normalization token filter is provided::

Expand Down
64 changes: 36 additions & 28 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,30 @@ import java.time.ZonedDateTime
buildscript {
repositories {
mavenCentral()
jcenter()
gradlePluginPortal()
}
dependencies {
classpath "org.elasticsearch.gradle:build-tools:${project.property('elasticsearch.version')}"
classpath "org.opensearch.gradle:build-tools:${project.property('opensearch.version')}"
}
}

plugins {
id "org.sonarqube" version "2.8"
id "io.codearte.nexus-staging" version "0.21.1"
id "org.xbib.gradle.plugin.jflex" version "1.4.0"
id "org.xbib.gradle.plugin.asciidoctor" version "1.5.6.0.1"
id "org.xbib.gradle.plugin.jflex" version "3.0.2"
id "org.xbib.gradle.plugin.asciidoctor" version "3.0.2"
}

apply plugin: 'java-library'
apply plugin: 'maven'
apply plugin: 'elasticsearch.esplugin'
apply plugin: 'maven-publish'
apply plugin: 'opensearch.opensearchplugin'
apply plugin: 'org.xbib.gradle.plugin.jflex'
apply plugin: 'elasticsearch.rest-resources'
apply plugin: 'opensearch.rest-resources'

ext {
pluginName = 'bundle'
pluginClassname = 'org.xbib.elasticsearch.plugin.bundle.BundlePlugin'
pluginDescription = 'A bundle of plugins for Elasticsearch'
pluginClassname = 'org.xbib.opensearch.plugin.bundle.BundlePlugin'
pluginDescription = 'A bundle of plugins for Opensearch'
user = 'jprante'
name = 'elasticsearch-plugin-bundle'
name = 'opensearch-plugin-bundle'
scmUrl = 'https://github.com/' + user + '/' + name
scmConnection = 'scm:git:git://github.com/' + user + '/' + name + '.git'
scmDeveloperConnection = 'scm:git:git://github.com/' + user + '/' + name + '.git'
Expand All @@ -45,30 +43,29 @@ sourceSets {
}
}

esplugin {
description 'Plugin bundle for Elasticsearch'
name 'elasticsearch-plugin-bundle'
classname 'org.xbib.elasticsearch.plugin.bundle.BundlePlugin'
opensearchplugin {
description 'Plugin bundle for Opensearch'
name 'opensearch-plugin-bundle'
classname 'org.xbib.opensearch.plugin.bundle.BundlePlugin'
}

dependencies {
// we must use "compile" and not "implementation" otherwise ES build does not detect dependencies
compile "com.ibm.icu:icu4j:${project.property('icu4j.version')}"
compile "org.xbib:standardnumber:${project.property('standardnumber.version')}"
testImplementation "org.codelibs.elasticsearch.module:analysis-common:${project.property('elasticsearch.version')}"
implementation "com.ibm.icu:icu4j:${project.property('icu4j.version')}"
implementation "org.xbib:standardnumber:${project.property('standardnumber.version')}"
testImplementation "org.opensearch.plugin:analysis-common:${project.property('opensearch.version')}"
}

compileJava {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
options.compilerArgs = [
"-Xlint:-try"
]
}

compileTestJava {
sourceCompatibility = JavaVersion.VERSION_11
targetCompatibility = JavaVersion.VERSION_11
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
options.compilerArgs = [
"-Xlint:-cast,-deprecation,-rawtypes,-try,-unchecked",
"-proc:none"
Expand All @@ -78,6 +75,7 @@ compileTestJava {
javadoc {
enabled = true
options.addStringOption('Xdoclint:none', '-quiet')
dependsOn('generateJflex')
}

jar {
Expand All @@ -90,9 +88,9 @@ jar {
'Build-Java-Version': System.getProperty("java.version"),
'X-Compile-Source-JDK': compileJava.targetCompatibility,
'X-Compile-Target-JDK': compileJava.sourceCompatibility,
'X-Compile-Elasticsearch-Version': project.property('elasticsearch.version'),
'X-Compile-Opensearch-Version': project.property('opensearch.version'),
'X-Compile-Lucene-Version': project.property('lucene.version'),
'X-Compile-Elasticsearch-Snapshot': false
'X-Compile-Opensearch-Snapshot': false
)
}
}
Expand Down Expand Up @@ -195,7 +193,6 @@ task xbibUpload(type: Upload) {
nexusStaging {
packageGroup = "org.xbib"
}
*/

sonarqube {
properties {
Expand All @@ -207,6 +204,7 @@ sonarqube {
property "sonar.junit.reportsPath", "build/test-results/test/"
}
}
*/

asciidoctor {
attributes toc: 'left',
Expand All @@ -220,7 +218,7 @@ asciidoctor {
'source-highlighter': 'coderay'
}

checkstyleMain.source = 'src/main/java'
//checkstyleMain.source = 'src/main/java'
// we use @Ignore
forbiddenApisTest.enabled = false
forbiddenPatterns.exclude('**/*.txt', '**/*.brk', '**/*.fst', '**/*.tree', '**/*.nrm')
Expand All @@ -230,3 +228,13 @@ thirdPartyAudit.enabled = false
loggerUsageCheck.enabled = false
testingConventions.enabled = false
validateNebulaPom.enabled = false

tasks.named('sourcesJar').configure {
dependsOn('generateJflex')
}
tasks.named('filepermissions').configure {
dependsOn('generateJflex')
}
tasks.named('forbiddenPatterns').configure {
dependsOn('generateJflex')
}
22 changes: 11 additions & 11 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
group = org.xbib.elasticsearch
name = elasticsearch-plugin-bundle
version = 7.10.2.0
group = org.xbib.opensearch
name = opensearch-plugin-bundle
version = 2.13.0.0

elasticsearch.version = 7.10.2
lucene.version = 8.7.0
opensearch.version = 2.13.0
lucene.version = 9.10.0

icu4j.version = 67.1
standardnumber.version = 1.0.1
jackson.version = 2.9.10
icu4j.version = 70.1
standardnumber.version = 1.2.0
jackson.version = 2.17.0
spatial4j.version = 0.7
jts.version = 1.15.1
jna.version = 4.5.1
jna.version = 5.13.0
checkstyle.version = 8.13
wagon.version = 3.0.0

log4j.version = 2.13.1
junit.version = 4.13
log4j.version = 2.21.0
junit.version = 4.13.2
Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
4 changes: 3 additions & 1 deletion gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-6.6.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-8.5-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Loading