Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The Browse data menu does not show database's schema and tables when use aws glue hive metastore #134

Open
ahululu opened this issue Jul 17, 2024 · 0 comments

Comments

@ahululu
Copy link

ahululu commented Jul 17, 2024

Metastore: AWS Glue

trino helm settings:

image:
  tag: "450"
server:
  workers: 1
  exchangeManager:
    baseDir: "s3://xxxx"
  autoscaling:
    enabled: true
    maxReplicas: 20
    targetCPUUtilizationPercentage: 70
  config:
    query:
      maxMemory: "4GB"
service:
  type: ClusterIP
  port: 8080
coordinator:
  jvm:
    maxHeapSize: "4G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "2GB"
worker:
  jvm:
    maxHeapSize: "12G"
  additionalJVMConfig:
    - "-XX:+UnlockDiagnosticVMOptions"
  config:
    query:
      maxMemoryPerNode: "8GB"
  resources:
    requests:
      cpu: 4000m
      memory: 12Gi
additionalConfigProperties:
  - "retry-policy=TASK"
  - "exchange.compression-enabled=true"
  - "query.low-memory-killer.delay=0s"
  - "query.remote-task.max-error-duration=1m"
  - "query.hash-partition-count=50"
additionalExchangeManagerProperties:
  - "exchange.s3.region=xxxxx"
  - "exchange.s3.iam-role=xxxxxx"
additionalCatalogs:
  hive: |-
    connector.name=hive
    hive.metastore=glue
    hive.metastore.glue.region=xxxxx
    hive.metastore.glue.endpoint-url=https://glue.xxxxx.amazonaws.com
    hive.metastore.glue.max-connections=30
    hive.metastore.glue.max-error-retries=3
    hive.metastore.glue.catalogid=1111111111
    hive.metastore.glue.iam-role=xxxxx
serviceAccount:
  create: true
  name: trino-sa
accessControl:
  type: configmap
  refreshPeriod: 60s
  configFile: "rules.json"
  rules:
    rules.json: |-
      {
        "catalogs": [
          {
            "user": "xxxxx",
            "catalog": ".*",
            "allow": "all"
          }
        ],
        "schemas": [
          {
            "user": "xxxxx",
            "schema": ".*",
            "owner": true
          }

      ],
        "tables": [
          {
            "user": "xxxx",
            "privileges": ["SELECT", "OWNERSHIP"]
          }
        ]
      }
ingress:
  enabled: true
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/certificate-arn: xxx
    alb.ingress.kubernetes.io/group.name: xxx
    alb.ingress.kubernetes.io/healthcheck-path: /
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/subnets: sxxx
    alb.ingress.kubernetes.io/success-codes: xx
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
  hosts:
  - paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: trino
            port:
              number: 8080
    host: xx.xx.xx

Metabase database settings and operation:

1. Database type: Starburst
2. Host: xxxx
3. Port: 443
4. Catalog: hive
5. Schema: 
6. Username: xxxxx
7. Use a secure connection (SSL): enable
8. Rerun queries for simple explorations: enable

click button:  'Sync database schema now' and 'Re-scan field values now' 
Then go back to the metabase home page, Browse data select the corresponding database, there is no schema display, but you can use 'Sql query' to query the data

@ahululu ahululu changed the title The Browse data menu does not show database's schema and tables when use aws glue The Browse data menu does not show database's schema and tables when use aws glue hive metastore Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant