Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add data indexing API #1255

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ module.exports = {
jest: true
},
rules: {
indent: ['error', 2],
indent: 'off',
'no-underscore-dangle': [0],
'no-plusplus': [0],
'no-return-await': [0],
Expand Down
128 changes: 128 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,11 @@ All operation use es7 async/await to implement. All api is async function.
- [.extendBucketWorm(name, wormId, days[, options])](#extendBucketWormname-wormId-days-options)
- [.getBucketWorm(name[, options])](#getBucketWormname-options)
- [.initiateBucketWorm(name, days[, options])](#initiateBucketWormname-days-options)
- Data Indexing
- [.openMetaQuery(bucketName[, options])](#openMetaQueryBucketName-options)
- [.getMetaQueryStatus(bucketName[, options])](#getMetaQueryStatusBucketName-options)
- [.doMetaQuery(bucketName, queryParam[, options])](#doMetaQueryBucketName-queryParam-options)
- [.closeMetaQuery(bucketName[, options])](#closeMetaQueryBucketName-options)

- [Object Operations](#object-operations)
- [.list(query[, options])](#listquery-options)
Expand Down Expand Up @@ -1702,6 +1707,129 @@ Success will return:

---

### .openMetaQuery(bucketName[, options])

Enables the metadata management feature for a bucket.

parameters:

- bucketName {String} bucket name
- [options] {Object} optional args
shungang marked this conversation as resolved.
Show resolved Hide resolved

Success will return:

- status {Number} response status
- res {Object} response info

---

### .getMetaQueryStatus(bucketName[, options])

Queries the information about the metadata index library of a bucket.

parameters:

- bucketName {String} bucket name
- [options] {Object} optional args
shungang marked this conversation as resolved.
Show resolved Hide resolved

Success will return:

- status {Number} response status
- res {Object} response info
- phase {String} the scan type
- state {String} the status of the metadata index library
- createTime {String} the time when the metadata index library was created
- updateTime {String} the time when the metadata index library was updated

---

### .doMetaQuery(bucketName, queryParam[, options])

Queries objects that meet specified conditions and lists the information about objects based on specified fields and sorting methods.

parameters:

- bucketName {String} the bucket name
- queryParam {Object} query parameters
- [nextToken] {String} The token that is used for the next query when the total number of objects exceeds the value of MaxResults. The object information is returned in alphabetical order starting from the value of NextToken. When this operation is called for the first time, this field does not need to be set.
- [maxResults] {Number} The maximum number of objects to return. Valid values: 0 to 100.
- query {Object} The query condition.
- [field] {String} the fields. For more information about supported fields and supported operators, see <a href="https://www.alibabacloud.com/help/en/oss/developer-reference/appendix-supported-fields-and-operators#concept-2084036" target="_blank">Appendix: Supported fields and operators</a>.
- [value] {String} The value of the field.
- operation {String} The operators. Valid values: eq (equal to), gt (greater than), gte (greater than or equal to), lt (less than), lte (less than or equal to), match (fuzzy query), prefix (prefix query), and (AND), or (OR), and not (NOT).
- [subQueries] {Array} The subquery conditions. Options that are included in this element are the same as those of simple query. You must set subquery conditions only when Operation is set to AND, OR, or NOT.
- [sort] {String} The field based on which the results are sorted. For more information about the fields that can be sorted, see <a href="https://www.alibabacloud.com/help/en/oss/developer-reference/appendix-supported-fields-and-operators#concept-2084036" target="_blank">Appendix: Supported fields and operators</a>.
- [order] {String} `asc` | `desc` The order in which you want to sort the queried data. Default value: desc. Valid values: asc and desc.
- [aggregations] {Object} The information about aggregate operations.
- field {String} The name of the field. For more information about supported fields and supported operators, see <a href="https://www.alibabacloud.com/help/en/oss/developer-reference/appendix-supported-fields-and-operators#concept-2084036" target="_blank">Appendix: Supported fields and operators</a>.
- operation {String} The operator for aggregate operations. Valid values: min, max, average, sum, count, distinct and group.
- [options] {Object} optional parameters
- [timeout] {Number} the operation timeout (ms)

Success will return:

- status {Number} response status
- res {Object} response info, including
- status {Number} response status
- headers {Object} response headers
- size {Number} response size
- rt {Number} request total use time (ms)
- nextToken {String} the token that is used for the next query when the total number of objects exceeds the value of MaxResults
shungang marked this conversation as resolved.
Show resolved Hide resolved
- files {Array} the information about objects
- aggregations {Array} the information about aggregate operations

---
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1732-1750行是不是多余了


example:

```js
const result = await store.put('test.txt', 'hello world');
console.log(result);
```

---

###.get(name[, options])

Get the object content.

parameters:

- name {String} object name store on OSS

---

example:

```js
const queryParam = {
maxResults: 2,
query: { operation: 'and', subQueries: [{ field: 'Size', value: '1048575', operation: 'lt' }] },
sort: 'Size',
order: 'asc'
};
const result = await store.doMetaQuery(bucket, queryParam);
console.log(result);
```

---

### .closeMetaQuery(bucketName[, options])

Disables the metadata management feature for a bucket.

parameters:

- bucketName {String} bucket name
- [options] {Object} optional args
shungang marked this conversation as resolved.
Show resolved Hide resolved

Success will return:

- status {Number} response status
- res {Object} response info

---

## Object Operations

All operations function return Promise, except `signatureUrl`.
Expand Down
2 changes: 1 addition & 1 deletion karma.conf.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ module.exports = function (config) {
concurrency: 1,
client: {
mocha: {
timeout: 6000
timeout: 16000
}
}
});
Expand Down
3 changes: 3 additions & 0 deletions lib/browser/client.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,9 @@ merge(proto, require('../common/bucket/getBucketWebsite'));
merge(proto, require('../common/bucket/putBucketWebsite'));
merge(proto, require('../common/bucket/deleteBucketWebsite'));

// bucket data index
merge(proto, require('../common/bucket/dataIndex'));

// lifecycle
merge(proto, require('../common/bucket/getBucketLifecycle'));
merge(proto, require('../common/bucket/putBucketLifecycle'));
Expand Down
68 changes: 68 additions & 0 deletions lib/common/bucket/dataIndex.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
export declare function openMetaQuery(
this: any,
bucketName: string,
options?: {}
): Promise<{
res: any;
status: any;
}>;
export declare function getMetaQueryStatus(
this: any,
bucketName: string,
options?: {}
): Promise<{
res: any;
status: any;
phase: any;
state: any;
createTime: any;
updateTime: any;
}>;
interface IQuery {
field?: string;
value?: string;
operation: string;
subQueries?: IQuery[];
}
declare enum EOperation {
min = 'min',
max = 'max',
average = 'average',
sum = 'sum',
count = 'count',
distinct = 'distinct',
group = 'group'
}
interface IAggregation {
field: string;
operation: EOperation;
}
interface IMetaQuery {
nextToken?: string;
maxResults?: number;
query: IQuery;
sort?: string;
order?: 'asc' | 'desc';
aggregations?: IAggregation[];
}
export declare function doMetaQuery(
this: any,
bucketName: string,
queryParam: IMetaQuery,
options?: {}
): Promise<{
res: any;
status: any;
nextToken: any;
files: any[];
aggregations: any[];
}>;
export declare function closeMetaQuery(
this: any,
bucketName: string,
options?: {}
): Promise<{
res: any;
status: any;
}>;
export {};
Loading